Clinical Trials Logo

Clinical Trial Details — Status: Completed

Administrative data

NCT number NCT06334796
Other study ID # 10476
Secondary ID
Status Completed
Phase Early Phase 1
First received
Last updated
Start date October 1, 2023
Est. completion date January 1, 2024

Study information

Verified date March 2024
Source Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia
Contact n/a
Is FDA regulated No
Health authority
Study type Interventional

Clinical Trial Summary

This study examines the use of an AI-powered virtual assistant for quickly identifying and handling neurological emergencies, particularly in places with limited medical resources. The research aimed to check if this AI tool is safe and accurate enough to move on to more advanced testing stages. In a first-of-its-kind trial, the virtual assistant was tested with patients having urgent neurological issues. Neurologists first reviewed the AI's recommendations using clinical records and then assessed its performance directly with patients. The findings were as follows: neurologists agreed with the AI's decisions nearly all the time, and the AI outperformed earlier versions of Chat GPT in every tested aspect. Patients and doctors found the AI to be highly effective, rating it as excellent or very good in most cases. This suggests the AI could significantly enhance how quickly and accurately neurological emergencies are dealt with, although further trials are needed before it can be widely used.


Description:

Background and Objectives: Neurological emergencies pose significant challenges in medical care, especially in resource-limited countries. Artificial Intelligence (AI), particularly health chatbots, offers a promising solution. However, rigorous validation is required to ensure safety and accuracy. The objective of our work is to evaluate the diagnostic accuracy and resolution effectiveness of an AI-powered virtual assistant designed for the triage of emergency neurological pathologies, to ensure the minimum standard of safety that allows for the progression to successive validation tests. Methods: This Phase 1 trial evaluates the performance of an AI-powered virtual assistant for emergency neurological triage. Ten patients over 18 years old with urgent neurological pathologies were selected. In the first stage, nine neurologists assessed the safety of the virtual assistant using their clinical records. In the second part, the assistant's accuracy when used by patients was evaluated. Finally, its performance was compared with Chat GPT 3.5 and 4.


Recruitment information / eligibility

Status Completed
Enrollment 10
Est. completion date January 1, 2024
Est. primary completion date January 1, 2024
Accepts healthy volunteers No
Gender All
Age group 18 Years and older
Eligibility Inclusion Criteria: - Patients over 18 years old consulting in the ER due to a neurological emergency Exclusion Criteria: - Pregnancy

Study Design


Intervention

Diagnostic Test:
Virtual Assistant
Stage 1 focused on safety, using only medical information from clinical records for the virtual assistant. In Stage 2, which evaluated accuracy, participants interacted with the virtual assistant post-medical stabilization. Additionally, participants also provided initial symptom details for Chat-GPT input. Nine neurologists specializing in emergency participated in the study. In Stage 1, they assessed the virtual assistant's performance using clinical history information. In Stage 2, they analyzed the results from participant interactions with the assistant and performed a comparative evaluation of Chat-GPT. The virtual assistant functioned as a chatbot on WhatsApp and Telegram, using Spanish and incorporating advanced algorithms, decision trees, and large language models for interaction. For comparison, we utilized Chat-GPT versions 3.5 and 4, employing two prompt types in natural Spanish: one incorporating clinical record data and the other based on participant narratives.

Locations

Country Name City State
Argentina Fleni Buenos Aires

Sponsors (2)

Lead Sponsor Collaborator
Fundación para la Lucha contra las Enfermedades Neurológicas de la Infancia Entelai

Country where clinical trial is conducted

Argentina, 

References & Publications (3)

Au Yeung J, Wang YY, Kraljevic Z, Teo JTH. Artificial intelligence (AI) for neurologists: do digital neurones dream of electric sheep? Pract Neurol. 2023 Nov 23;23(6):476-488. doi: 10.1136/pn-2023-003757. — View Citation

Haug CJ, Drazen JM. Artificial Intelligence and Machine Learning in Clinical Medicine, 2023. N Engl J Med. 2023 Mar 30;388(13):1201-1208. doi: 10.1056/NEJMra2302038. No abstract available. — View Citation

Patel UK, Anwar A, Saleem S, Malik P, Rasul B, Patel K, Yao R, Seshadri A, Yousufuddin M, Arumaithurai K. Artificial intelligence as an emerging technology in the current care of neurological disorders. J Neurol. 2021 May;268(5):1623-1642. doi: 10.1007/s00415-019-09518-3. Epub 2019 Aug 26. — View Citation

Outcome

Type Measure Description Time frame Safety issue
Primary Diagnostic performance Refers to the accuracy and effectiveness of medical tests or diagnostic tools in correctly identifying a disease or condition in patients.
Syndromic diagnosis agreement: evaluating neurologists considered a syndromic diagnosis accurate when AI tools could identify a condition based on a set of commonly coexisting signs and symptoms, rather than identifying a specific disease. This method is applied when the precise disease causing the symptoms is not immediately identifiable, allowing healthcare providers to effectively monitor and treat the patient's presenting symptoms.
Differential diagnosis agreement: a differential diagnosis was considered accurate when the differentials provided by each AI tool matched those presented by the participants.
The gold standard for diagnosis was considered to be the one given in the emergency department, unchanged over a one-month period.
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
Secondary Appropriate medical conduct or recommendation Case resolution was evaluated based on appropriate medical conduct or recommendation, categorizing 1) urgency as immediate, 2) short-term (within 48 hours), 3) or non-urgent.
The recommendations provided by each AI tool were assessed based on information gathered from clinical histories and input from participants.
The gold standard of appropriate medical conduct or recommendation was considered to be that given in the emergency department, with no changes over a period of one month.
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
Secondary Assessment of Usability and Satisfaction Usability was measured by the time and number of questions needed for final diagnosis and resolution, both by neurologists and participants. For Chat GPT, we evaluated the time taken to draft the consultation reason.
A satisfaction scale from 1 to 5 was implemented, with 1 indicating a negative experience ("poor", potentially risky for the patient) and 5 highly positive ("excellent", potentially surpassing non-specialized human triage). A simple yes/no survey was also applied to participants, asking about the comprehensibility of the assistant's questions, the adequacy of referral according to urgency, and whether they considered the assistant could replace non-specialized triage or reduce emergency arrival time.
The first interaction between participants and the virtual assistant occurred within less than a year after the event. Outcome measures were evaluated immediately after the interaction between patients and the virtual assistant.
See also
  Status Clinical Trial Phase
Recruiting NCT04043052 - Mobile Technologies and Post-stroke Depression N/A
Recruiting NCT03869138 - Alternative Therapies for Improving Physical Function in Individuals With Stroke N/A
Completed NCT04034069 - Effects of Priming Intermittent Theta Burst Stimulation on Upper Limb Motor Recovery After Stroke: A Randomized Controlled Trial N/A
Completed NCT04101695 - Hemodynamic Response of Anodal Transcranial Direct Current Stimulation Over the Cerebellar Hemisphere in Healthy Subjects N/A
Terminated NCT03052712 - Validation and Standardization of a Battery Evaluation of the Socio-emotional Functions in Various Neurological Pathologies N/A
Completed NCT00391378 - Cerebral Lesions and Outcome After Cardiac Surgery (CLOCS) N/A
Recruiting NCT06204744 - Home-based Arm and Hand Exercise Program for Stroke: A Multisite Trial N/A
Active, not recruiting NCT06043167 - Clinimetric Application of FOUR Scale as in Treatment and Rehabilitation of Patients With Acute Cerebral Injury
Active, not recruiting NCT04535479 - Dry Needling for Spasticity in Stroke N/A
Completed NCT03985761 - Utilizing Gaming Mechanics to Optimize Telerehabilitation Adherence in Persons With Stroke N/A
Recruiting NCT00859885 - International PFO Consortium N/A
Recruiting NCT06034119 - Effects of Voluntary Adjustments During Walking in Participants Post-stroke N/A
Completed NCT03622411 - Tablet-based Aphasia Therapy in the Chronic Phase N/A
Completed NCT01662960 - Visual Feedback Therapy for Treating Individuals With Hemiparesis Following Stroke N/A
Recruiting NCT05854485 - Robot-Aided Assessment and Rehabilitation of Upper Extremity Function After Stroke N/A
Active, not recruiting NCT05520528 - Impact of Group Participation on Adults With Aphasia N/A
Completed NCT03366129 - Blood-Brain Barrier Disruption in People With White Matter Hyperintensities Who Have Had a Stroke
Completed NCT03281590 - Stroke and Cerebrovascular Diseases Registry
Completed NCT05805748 - Serious Game Therapy in Neglect Patients N/A
Recruiting NCT05993221 - Deconstructing Post Stroke Hemiparesis