Physician Reasoning on Management Cases With Large Language Models

Status Recruiting

Clinical Trial Summary

This study will evaluate the effect of providing access to GPT-4, a large language model, compared to traditional management decision support tools on performance on case-based management reasoning tasks.

Clinical Trial Description

Artificial intelligence (AI) technologies, specifically advanced large language models like OpenAI's ChatGPT, have the potential to improve medical decision-making. Although ChatGPT-4 was not developed for its use in medical-specific applications, it has demonstrated promise in various healthcare contexts, including medical note-writing, addressing patient inquiries, and facilitating medical consultation. However, little is known about how ChatGPT augments the clinical reasoning abilities of clinicians. Clinical reasoning is a complex process involving pattern recognition, knowledge application, and probabilistic reasoning. Integrating AI tools like ChatGPT-4 into physician workflows could potentially help reduce clinician workload and decrease the likelihood of mismanagement. However, ChatGPT-4 was not developed for clinical reasoning nor has it been validated for this purpose. Further, it may be subject to disinformation, including convincing confabulations that may mislead clinicians. If clinicians misuse this tool, it may not improve reasoning and could even cause harm. Therefore, it is important to study how clinicians use large language models to augment clinical reasoning prior to routine incorporation into patient care. In this study, participants will be randomized to answer clinical management cases with or without access to ChatGPT-4. Each case has multiple components, and the participants will be asked to discuss their reasoning for each component. Answers will be graded by independent reviewers blinded to treatment assignment. A grading rubric was developed for each case by a panel of 4-7 expert discussants. Discussants independently developed a rubric for each case, and then any discrepancies were resolved through multiple rounds of discussions. ;

Study Design

Related Conditions & MeSH terms

Clinical Decision-making

Clinical Trial Details

NCT number	NCT06208423
Study type	Interventional
Source	Stanford University
Contact	Robert J Gallo, MD
Phone	6507234000
Email	rjgallo@stanford.edu
Status	Recruiting
Phase	N/A
Start date	December 28, 2023
Completion date	March 30, 2024

View Details

See also
Status	Clinical Trial	Phase
Not yet recruiting	`NCT05054049` - Point of Care Testing for Advanced Practitioners (Paramedics)	N/A
Completed	`NCT03875846` - Intraoperative Simultaneous Pressure Guided Revascularization Study
Completed	`NCT04602676` - The Acceptability and Impact of Diarrheal Etiology Prediction (DEP) Algorithm	N/A
Recruiting	`NCT05182008` - A Patient Decision Aid for Method of Early Abortion: A Randomized Control Trial	N/A
Completed	`NCT04139642` - Standing Balance as the Fifth Vital Sign in Clinical Setting
Completed	`NCT03271658` - Mixed Methods Study Web-based Life Support Decision Aid	N/A
Withdrawn	`NCT03283592` - Review and Assessment of Network Meta-analyses for Knowledge Uptake Study	N/A
Recruiting	`NCT04218188` - Evaluation of the Cell Detachment Ratio on pH-responsive Chitosan as a Prognostic Factor in Lung Cancer
Completed	`NCT00298727` - The Effect and Mediators of Two Knowledge Translation Strategies	Phase 3
Unknown status	`NCT01255488` - Research Abstract Usefulness in Clinical Decision-making	N/A
Completed	`NCT06235060` - The Effect of Two Dıfferent Technıques Used in Psychomotor Skılls Traınıng on Nursıng Students	N/A
Completed	`NCT05657002` - A Study to the Impact of Accuracy Problem Lists in Electronic Health Records on Correctness and Speed of Clinical Decision-making Performed by Dutch Healthcare Providers	N/A
Completed	`NCT03449862` - Key Information Can Influence Clinician Ordering of Brain CTs	N/A

Clinical trials are conducted in a series of steps, called phases - each phase is designed to answer a separate research question.

Phase 1: Researchers test a new drug or treatment in a small group of people for the first time to evaluate its safety, determine a safe dosage range, and identify side effects.
Phase 2: The drug or treatment is given to a larger group of people to see if it is effective and to further evaluate its safety.
Phase 3: The drug or treatment is given to large groups of people to confirm its effectiveness, monitor side effects, compare it to commonly used treatments, and collect information that will allow the drug or treatment to be used safely.
Phase 4: Studies are done after the drug or treatment has been marketed to gather information on the drug's effect in various populations and any side effects associated with long-term use.