Generating Fast and Slow for Entree Level Medical Knowledge

Patient Satisfaction Clinical Trial

Official title:

Generating Fast and Slow for Entree Level Medical Knowledge

Clinical Trial Details — Status: Not yet recruiting

Administrative data

• NCT number: NCT06247475
• Other study ID #: 202308059RINB
• Status: Not yet recruiting
• Start date: February 2024
• Est. completion date: June 2024

Study information

• Verified date: January 2024
• Source: National Taiwan University Hospital
• Contact: n/a
• Is FDA regulated: No
• Study type: Observational

Clinical Trial Summary

The generative artificial intelligence tool, ChatGPT, has garnered widespread interest since its launch. This innovative multimedia platform has the potential to enhance medical communication and health education, thereby improving medical accessibility and reducing the burden on healthcare professionals. Some studies have indicated that ChatGPT achieves higher levels of satisfaction in counseling compared to human healthcare professionals. Additionally, research has shown that ChatGPT's performance in answering objective structured clinical examination questions is comparable to that of typical medical students. However, in both scenarios, it still requires editing by professionals before being used. Moreover, a recent meta-analysis evaluating ChatGPT's ability in various types of medical exams revealed inconsistent results. Before its application and actual integration into clinical practice, the investigators need to comprehend the advantages, disadvantages, and relevant limitations of ChatGPT in the field of medical communication. This study aims to simulate virtual consultations between ChatGPT, acting as a health professional, and study participants, serving as patients. It will evaluate the participants' satisfaction with the virtual consultation questions, categorized by different levels of cognition through Bloom's Taxonomy. This study plans to recruit medical professionals, healthcare-related professionals, medical students from National Taiwan University Hospital, and the general public. Two researchers will select 20 questions from the USMLE step 3 practice tests and the second stage of the Taiwan Medical Licensing Examination. These questions will be categorized into different levels of cognition based on knowledge, comprehension, application, analysis, synthesis, or evaluation. ChatGPT 3.5 and 4.0 will answer these 20 questions without specifying the question's level of cognition. Each participant will review the answers to these 20 questions and assign a satisfaction score based on the appropriateness of each answer.

Recruitment information / eligibility

• Status: Not yet recruiting
• Enrollment: 120
• Est. completion date: June 2024
• Est. primary completion date: June 2024
• Accepts healthy volunteers: Accepts Healthy Volunteers
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

1. National Taiwan University Hospital

• Medical professionals
• Healthcare-related professionals
• Medical students

2. Taiwan Emergency Medical Technician Association.

• Volunteers

Exclusion Criteria:

• N/A.

Related Conditions & MeSH terms

• Patient Satisfaction

Intervention

Other:
• Answers generated by ChatGPT
Participants will assign a satisfaction score based on the appropriateness of each answer.

Locations

Country	Name	City	State
Taiwan	National Taiwan University	Taipei

Sponsors (2)

Lead Sponsor	Collaborator
National Taiwan University Hospital	Taiwan Emergency Medical Technician Association

Country where clinical trial is conducted

Taiwan, 

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Correlation between satisfaction and Bloom's taxonomy	Divided by the four participant groups	baseline (at completion of study questionnaire)
Secondary	Correctness of answers	Answers from ChatGPT 3.5/4.0 and the answers provided by Medical Licencing Exam institutions	baseline (at completion of study questionnaire)