Clinical Trials Logo

Clinical Trial Details — Status: Recruiting

Administrative data

NCT number NCT06208423
Other study ID # 71319b
Secondary ID
Status Recruiting
Phase N/A
First received
Last updated
Start date December 28, 2023
Est. completion date March 30, 2024

Study information

Verified date February 2024
Source Stanford University
Contact Robert J Gallo, MD
Phone 6507234000
Email rjgallo@stanford.edu
Is FDA regulated No
Health authority
Study type Interventional

Clinical Trial Summary

This study will evaluate the effect of providing access to GPT-4, a large language model, compared to traditional management decision support tools on performance on case-based management reasoning tasks.


Description:

Artificial intelligence (AI) technologies, specifically advanced large language models like OpenAI's ChatGPT, have the potential to improve medical decision-making. Although ChatGPT-4 was not developed for its use in medical-specific applications, it has demonstrated promise in various healthcare contexts, including medical note-writing, addressing patient inquiries, and facilitating medical consultation. However, little is known about how ChatGPT augments the clinical reasoning abilities of clinicians. Clinical reasoning is a complex process involving pattern recognition, knowledge application, and probabilistic reasoning. Integrating AI tools like ChatGPT-4 into physician workflows could potentially help reduce clinician workload and decrease the likelihood of mismanagement. However, ChatGPT-4 was not developed for clinical reasoning nor has it been validated for this purpose. Further, it may be subject to disinformation, including convincing confabulations that may mislead clinicians. If clinicians misuse this tool, it may not improve reasoning and could even cause harm. Therefore, it is important to study how clinicians use large language models to augment clinical reasoning prior to routine incorporation into patient care. In this study, participants will be randomized to answer clinical management cases with or without access to ChatGPT-4. Each case has multiple components, and the participants will be asked to discuss their reasoning for each component. Answers will be graded by independent reviewers blinded to treatment assignment. A grading rubric was developed for each case by a panel of 4-7 expert discussants. Discussants independently developed a rubric for each case, and then any discrepancies were resolved through multiple rounds of discussions.


Recruitment information / eligibility

Status Recruiting
Enrollment 50
Est. completion date March 30, 2024
Est. primary completion date March 30, 2024
Accepts healthy volunteers Accepts Healthy Volunteers
Gender All
Age group N/A and older
Eligibility Inclusion Criteria: - Participants must be licensed physicians and have completed at least post-graduate year 2 (PGY2) of medical training. - Training in Internal medicine, family medicine, or emergency medicine. Exclusion Criteria: - Not currently practicing clinically.

Study Design


Related Conditions & MeSH terms


Intervention

Other:
GPT-4
OpenAI's GPT-4 large language model with chat interface.

Locations

Country Name City State
United States Stanford University Palo Alto California

Sponsors (3)

Lead Sponsor Collaborator
Stanford University Beth Israel Deaconess Medical Center, University of Minnesota

Country where clinical trial is conducted

United States, 

Outcome

Type Measure Description Time frame Safety issue
Primary Management Reasoning Percent correct (range: 0 to 100) for each case. Within one-hour study
Secondary Time Spent on Management Time (in minutes) participants spend per case between the two study arms. Within one-hour study
See also
  Status Clinical Trial Phase
Not yet recruiting NCT05054049 - Point of Care Testing for Advanced Practitioners (Paramedics) N/A
Completed NCT03875846 - Intraoperative Simultaneous Pressure Guided Revascularization Study
Completed NCT04602676 - The Acceptability and Impact of Diarrheal Etiology Prediction (DEP) Algorithm N/A
Recruiting NCT05182008 - A Patient Decision Aid for Method of Early Abortion: A Randomized Control Trial N/A
Completed NCT04139642 - Standing Balance as the Fifth Vital Sign in Clinical Setting
Completed NCT03271658 - Mixed Methods Study Web-based Life Support Decision Aid N/A
Withdrawn NCT03283592 - Review and Assessment of Network Meta-analyses for Knowledge Uptake Study N/A
Recruiting NCT04218188 - Evaluation of the Cell Detachment Ratio on pH-responsive Chitosan as a Prognostic Factor in Lung Cancer
Completed NCT00298727 - The Effect and Mediators of Two Knowledge Translation Strategies Phase 3
Unknown status NCT01255488 - Research Abstract Usefulness in Clinical Decision-making N/A
Completed NCT06235060 - The Effect of Two Dıfferent Technıques Used in Psychomotor Skılls Traınıng on Nursıng Students N/A
Completed NCT05657002 - A Study to the Impact of Accuracy Problem Lists in Electronic Health Records on Correctness and Speed of Clinical Decision-making Performed by Dutch Healthcare Providers N/A
Completed NCT03449862 - Key Information Can Influence Clinician Ordering of Brain CTs N/A