Clinical Trials Logo

Clinical Trial Details — Status: Active, not recruiting

Administrative data

NCT number NCT06018545
Other study ID # 310995 - A
Secondary ID
Status Active, not recruiting
Phase
First received
Last updated
Start date June 1, 2023
Est. completion date June 1, 2025

Study information

Verified date June 2024
Source Oxford University Hospitals NHS Trust
Contact n/a
Is FDA regulated No
Health authority
Study type Observational

Clinical Trial Summary

This study has been added as a sub study to the Simulation Training for Emergency Department Imaging 2 study (ClinicalTrials.gov ID NCT05427838). The purpose of the study is to assess the impact of an Artificial Intelligence (AI) tool called qER 2.0 EU on the performance of readers, including general radiologists, emergency medicine clinicians, and radiographers, in interpreting non-contrast CT head scans. The study aims to evaluate the changes in accuracy, review time, and diagnostic confidence when using the AI tool. It also seeks to provide evidence on the diagnostic performance of the AI tool and its potential to improve efficiency and patient care in the context of the National Health Service (NHS). The study will use a dataset of 150 CT head scans, including both control cases and abnormal cases with specific abnormalities. The results of this study will inform larger follow-up studies in real-life Emergency Department (ED) settings.


Recruitment information / eligibility

Status Active, not recruiting
Enrollment 33
Est. completion date June 1, 2025
Est. primary completion date September 1, 2023
Accepts healthy volunteers Accepts Healthy Volunteers
Gender All
Age group N/A and older
Eligibility Inclusion Criteria: - Radiologists/Radiographers/ED clinicians who review CT head scans as part of their clinical practice Exclusion Criteria: - Neuroradiologists. - Non-radiologist groups: Clinicians with previous formal postgraduate CT reporting training - Emergency Medicine group: Clinicians with previous career in radiology/neurosurgery to registrar level

Study Design


Intervention

Other:
Ground truthing
Two Consultant neuroradiologists will independently review the images to establish the 'ground truth' findings on the CT scans which will be used as the reference standard. In the case of disagreement, a third senior neuroradiologist's opinion will be sought for arbitration.
Reading
All 30 readers will review all 150 cases, in each of two study phases. The readers will provide their opinion on the presence or absence of some acute abnormalities, including intracranial haemorrhage, infarct, midline shift and fracture. They will provide a confidence in their diagnosis (10-point visual analogue scale), and a single click point to mark the location of each abnormality that they consider as being present. The time taken for each scan will be automatically recorded.

Locations

Country Name City State
United Kingdom NHS Greater Glasgow and Clyde Glasgow
United Kingdom Guy's & St Thomas NHS Foundation Trust London
United Kingdom Northumbria Healthcare NHS Foundation Trust Newcastle Upon Tyne
United Kingdom Oxford University Hospitals NHS Foundation Trust Oxford Oxfordshire

Sponsors (1)

Lead Sponsor Collaborator
Oxford University Hospitals NHS Trust

Country where clinical trial is conducted

United Kingdom, 

References & Publications (21)

Andralojc LE, Kim DH, Edwards AJ. Diagnostic accuracy of a decision-support software for the detection of intracranial large-vessel occlusion in CT angiography. Clin Radiol. 2023 Apr;78(4):e313-e318. doi: 10.1016/j.crad.2022.10.017. Epub 2023 Jan 11. — View Citation

Arbabshirani MR, Fornwalt BK, Mongelluzzo GJ, Suever JD, Geise BD, Patel AA, Moore GJ. Advanced machine learning in action: identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration. NPJ Digit Med. 2018 Apr 4;1:9. doi: 10.1038/s41746-017-0015-z. eCollection 2018. — View Citation

Chan J, Fan KS, Mak TLA, Loh SY, Ng SWY, Adapala R. Pre-Operative Imaging can Reduce Negative Appendectomy Rate in Acute Appendicitis. Ulster Med J. 2020 Jan;89(1):25-28. Epub 2020 Feb 18. — View Citation

Chilamkurthy S, Ghosh R, Tanamala S, Biviji M, Campeau NG, Venugopal VK, Mahajan V, Rao P, Warier P. Deep learning algorithms for detection of critical findings in head CT scans: a retrospective study. Lancet. 2018 Dec 1;392(10162):2388-2396. doi: 10.1016/S0140-6736(18)31645-3. Epub 2018 Oct 11. — View Citation

Davis MA, Rao B, Cedeno PA, Saha A, Zohrabian VM. Machine Learning and Improved Quality Metrics in Acute Intracranial Hemorrhage by Noncontrast Computed Tomography. Curr Probl Diagn Radiol. 2022 Jul-Aug;51(4):556-561. doi: 10.1067/j.cpradiol.2020.10.007. Epub 2020 Nov 15. — View Citation

Dyer T, Chawda S, Alkilani R, Morgan TN, Hughes M, Rasalingham S. Validation of an artificial intelligence solution for acute triage and rule-out normal of non-contrast CT head scans. Neuroradiology. 2022 Apr;64(4):735-743. doi: 10.1007/s00234-021-02826-4. Epub 2021 Oct 8. — View Citation

Finck T, Moosbauer J, Probst M, Schlaeger S, Schuberth M, Schinz D, Yigitsoy M, Byas S, Zimmer C, Pfister F, Wiestler B. Faster and Better: How Anomaly Detection Can Accelerate and Improve Reporting of Head Computed Tomography. Diagnostics (Basel). 2022 Feb 10;12(2):452. doi: 10.3390/diagnostics12020452. — View Citation

Greenhalgh R, Howlett DC, Drinkwater KJ. Royal College of Radiologists national audit evaluating the provision of imaging in the severely injured patient and compliance with national guidelines. Clin Radiol. 2020 Mar;75(3):224-231. doi: 10.1016/j.crad.2019.10.025. Epub 2019 Dec 19. — View Citation

Guo Y, He Y, Lyu J, Zhou Z, Yang D, Ma L, Tan HT, Chen C, Zhang W, Hu J, Han D, Ding G, Liu S, Qiao H, Xu F, Lou X, Dai Q. Deep learning with weak annotation from diagnosis reports for detection of multiple head disorders: a prospective, multicentre study. Lancet Digit Health. 2022 Aug;4(8):e584-e593. doi: 10.1016/S2589-7500(22)00090-5. Epub 2022 Jun 17. Erratum In: Lancet Digit Health. 2022 Aug;4(8):e572. — View Citation

Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med. 2005 May 30;24(10):1579-607. doi: 10.1002/sim.2024. — View Citation

Huang SC, Pareek A, Jensen M, Lungren MP, Yeung S, Chaudhari AS. Self-supervised learning for medical image classification: a systematic review and implementation guidelines. NPJ Digit Med. 2023 Apr 26;6(1):74. doi: 10.1038/s41746-023-00811-0. — View Citation

Juszczyk K, Ireland K, Thomas B, Kroon HM, Hollington P. Reduction in hospital admissions with an early computed tomography scan: results of an outpatient management protocol for uncomplicated acute diverticulitis. ANZ J Surg. 2019 Sep;89(9):1085-1090. doi: 10.1111/ans.15285. Epub 2019 Jun 17. — View Citation

Lee JY, Kim JS, Kim TY, Kim YS. Detection and classification of intracranial haemorrhage on CT images using a novel deep-learning algorithm. Sci Rep. 2020 Nov 25;10(1):20546. doi: 10.1038/s41598-020-77441-z. — View Citation

Lin E, Yuh EL. Computational Approaches for Acute Traumatic Brain Injury Image Recognition. Front Neurol. 2022 Mar 9;13:791816. doi: 10.3389/fneur.2022.791816. eCollection 2022. — View Citation

Mallon DH, Taylor EJR, Vittay OI, Sheeka A, Doig D, Lobotesis K. Comparison of automated ASPECTS, large vessel occlusion detection and CTP analysis provided by Brainomix and RapidAI in patients with suspected ischaemic stroke. J Stroke Cerebrovasc Dis. 2022 Oct;31(10):106702. doi: 10.1016/j.jstrokecerebrovasdis.2022.106702. Epub 2022 Aug 19. — View Citation

Obuchowski NA. Sample size tables for receiver operating characteristic studies. AJR Am J Roentgenol. 2000 Sep;175(3):603-8. doi: 10.2214/ajr.175.3.1750603. — View Citation

Sheth SA, Giancardo L, Colasurdo M, Srinivasan VM, Niktabe A, Kan P. Machine learning and acute stroke imaging. J Neurointerv Surg. 2023 Feb;15(2):195-199. doi: 10.1136/neurintsurg-2021-018142. Epub 2022 May 25. — View Citation

Wardlaw JM, Mair G, von Kummer R, Williams MC, Li W, Storkey AJ, Trucco E, Liebeskind DS, Farrall A, Bath PM, White P. Accuracy of Automated Computer-Aided Diagnosis for Stroke Imaging: A Critical Evaluation of Current Evidence. Stroke. 2022 Jul;53(7):2393-2403. doi: 10.1161/STROKEAHA.121.036204. Epub 2022 Apr 20. — View Citation

Warman R, Warman A, Warman P, Degnan A, Blickman J, Chowdhary V, Dash D, Sangal R, Vadhan J, Bueso T, Windisch T, Neves G. Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage. Cureus. 2022 Oct 13;14(10):e30264. doi: 10.7759/cureus.30264. eCollection 2022 Oct. — View Citation

Yeo M, Tahayori B, Kok HK, Maingard J, Kutaiba N, Russell J, Thijs V, Jhamb A, Chandra RV, Brooks M, Barras CD, Asadi H. Review of deep learning algorithms for the automatic detection of intracranial hemorrhages on computed tomography head imaging. J Neurointerv Surg. 2021 Apr;13(4):369-378. doi: 10.1136/neurintsurg-2020-017099. Epub 2021 Jan 21. — View Citation

Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLoS Med. 2018 Nov 6;15(11):e1002683. doi: 10.1371/journal.pmed.1002683. eCollection 2018 Nov. — View Citation

* Note: There are 21 references in allClick here to view all references

Outcome

Type Measure Description Time frame Safety issue
Primary Reader performance: Sensitivity, specificity, comparative between with and without AI assistance. Reader performance will be evaluated as sensitivity, specificity, with and without AI assistance. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary Reader performance: Positive and negative predictive value, comparative between with and without AI assistance. Reader performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV), with and without AI assistance. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary Reader performance: Area Under Receiver Operating Characteristic Curve (AUROC), comparative between with and without AI assistance. Reader performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC), with and without AI assistance. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary Reader speed: Mean time taken to review a scan, with versus without AI assistance. Reader speed will be evaluated as the man time taken to review a scan, using time unite of seconds. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary Reader confidence: Self-reported diagnostic confidence on a 10 point visual analogue scale, with vs without AI assistance. On the reading platform (RAIQC), one of the questions asks the level of confidence that the participant has in their diagnostic opinion. The question offers a scale of 1 to 10, where 1 is not confident, and 10 is highly confident. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary qER (AI algorithm) performance: Sensitivity and specificity qER performance will be evaluated as sensitivity, specificity. During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary qER (AI algorithm) performance: Positive and negative predictive value. qER performance will be evaluated as Positive Predictive Value (PPV) and negative predictive value (NPV). During 6 weeks, which is the period for reading or reviewing the cases/scans.
Primary qER (AI algorithm) performance: Area Under Receiver Operating Characteristic Curve (AUROC). qER performance will be evaluated as Area Under Receiver Operating Characteristic Curve (AUROC) During 6 weeks, which is the period for reading or reviewing the cases/scans.
See also
  Status Clinical Trial Phase
Recruiting NCT06113848 - Adjunctive Use of Intra-Arterial TNK and Albumin Following Thrombectomy Phase 3
Completed NCT04069546 - The Efficacy of Remote Ischemic Conditioning on Stroke-induced Immunodeficiency N/A
Active, not recruiting NCT05700097 - Dengzhanxixin Injection for Acute Ischemic Stroke Receiving Reperfusion Therapy Phase 2
Recruiting NCT06058130 - Combination of Antiplatelet and Anticoagulation for AIS Patients Witn Concomitant NVAF and Extracranial/Intracranial Artery Stenosis N/A
Recruiting NCT04415164 - Evaluation of Xueshuantong in Patients With AcutE IschemiC STroke Phase 4
Recruiting NCT05363397 - Safety and Tolerability of Adjunctive TBO-309 in Reperfusion for Stroke Phase 2
Completed NCT05429658 - Single Arm Trial to Evaluate the Safety and Effectiveness of the Route 92 Medical Reperfusion System N/A
Recruiting NCT05390580 - Neuromodulation Using Vagus Nerve Stimulation Following Ischemic Stroke as Therapeutic Adjunct N/A
Enrolling by invitation NCT05515393 - A Study of XY03-EA Tablets in the Treatment of Acute Ischemic Stroke Phase 2
Active, not recruiting NCT05070260 - ACTISAVE: ACuTe Ischemic Stroke Study Evaluating Glenzocimab Used as Add-on Therapy Versus placEbo Phase 2/Phase 3
Terminated NCT05547412 - Validation of Velocity Curvature Index as a Diagnostic Biomarker Tool for Assessment of Large Vessel Stroke
Completed NCT03366818 - New Stent Retriever, VERSI System for AIS N/A
Not yet recruiting NCT06040476 - Human Umbilical Cord Blood Infusion in Patients With Acute Ischemic Stroke (AIS) Phase 2
Not yet recruiting NCT06437431 - Glenzocimab in Anterior Stroke With Large Ischemic Core Eligible for Endovascular Therapy Phase 2/Phase 3
Not yet recruiting NCT05293080 - Early Treatment of Atrial Fibrillation for Stroke Prevention Trial in Acute STROKE Phase 3
Completed NCT02223273 - Brazilian Intervention to Increase Evidence Usage in Practice - Stroke (BRIDGE-Stroke) N/A
Completed NCT02586233 - Study to Assess the Safety, Pharmacokinetics, and Pharmacodynamics of DS-1040b in Subjects With Acute Ischemic Stroke Phase 1/Phase 2
Not yet recruiting NCT01594190 - Physical Activity Immediately After Acute Cerebral Ischemia N/A
Terminated NCT01694381 - Research Into the Effect of a Clot-dissolving Agent and Its Inhibitor Early Phase 1
Completed NCT01120301 - Efficacy and Safety Trial of Transcranial Laser Therapy Within 24 Hours From Stroke Onset (NEST-3) Phase 3