Ultrasound Operator Influence on Diagnostics With AI for Thyroid Nodules - Clinial Trial

Thyroid Nodule Clinical Trial

Official title:

The Ultrasound Operator's Influence on the Diagnostic Accuracy of Artificial Intelligence for Thyroid Nodules Assessment - a Clinical Trial

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT06306599
• Other study ID #: Thyroid AI US operator exp
• Status: Completed
• Phase: N/A
• Start date: September 1, 2023
• Est. completion date: November 4, 2023

Study information

• Verified date: March 2024
• Source: Rigshospitalet, Denmark
• Contact: n/a
• Is FDA regulated: No
• Study type: Interventional

Clinical Trial Summary

This is a prospective clinical study aiming to test how the experience of the ultrasound operator influences the performance of AI-based (artificial intelligence-based) diagnostics when analysing thyroid nodules on ultrasound scans. The investigators set up an experiment with five stations, each with a patient with a thyroid nodule and an ultrasound machine with the deep learning based system S-Detect for Thyroid installed. 20 study participants where recruited: 8 medical students of novice ultrasound skill, 3 junior ENT (ear-nose-throat) registrars of intermediate ultrasound skill, and 9 senior ENT registrars experienced in ultrasound. The participants scanned all the patients and recorded their analyses of the nodules using the EUTIRADS (European thyroid imagining reporting and data system) system in three different ways: a analysis of their own, S-Detect's analysis, and an analysis combining the two previous.

The hypothesis was that the AI system would perform equally well when between the participant groups. In addition, it was expected that the experienced participants would perform better than the students without AI help, and that the doctors would gain little from AI input, but that the students would have their performance improved by AI input.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 20
• Est. completion date: November 4, 2023
• Est. primary completion date: November 4, 2023
• Accepts healthy volunteers: Accepts Healthy Volunteers
• Gender: All
• Age group: N/A and older

Eligibility

Medical students

Inclusion Criteria:

• Last year student

Exclusion Criteria:

• Experience with ultrasound beyond that which is taught at the University of Copenhagen

Junior ENT registrar doctors

Inclusion Criteria:

• Doctor enrolled in introductory training as ENT physician.

Senior ENT registrar doctors

Inclusion Criteria:

• Doctor enrolled in ENT training.

Related Conditions & MeSH terms

• Thyroid Diseases
• Thyroid Nodule

Intervention

Diagnostic Test:
• S-Detect for Thyroid
Deep learning based program on Samsung ultrasound machines designed to do real-time semi-automated analysis of thyroid nodules. The ultrasound operator freezes a transverse image of the patient's thyroid nodule and activates S-Detect. The operator selects the nodule on the screen, and the program automatically draws a region of interest. Then S-Detect gives a dichotomous diagnosis of either "Possibly benign" and "Possibly malignant". In addition, it measures the nodule and characterises it with a lexicon based on EUTIRADS.

Locations

Country	Name	City	State
Denmark	Rigshospitalet	Copenhagen

Sponsors (1)

Lead Sponsor	Collaborator
Rigshospitalet, Denmark

Country where clinical trial is conducted

Denmark, 

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Accuracy of S-Detect diagnosis	Number of correct thyroid nodule malignancy diagnoses out of total malignancy diagnoses by the AI-based ultrasound diagnostic system "S-Detect" on the five patients' thyroid nodules. Gold standard is cytology and histology of the nodules.	1 day (day of experiment)
Secondary	Accuracy of biopsy recommendation	Number of correct biopsy recommendations on the five patients' thyroid nodules. Recommendation is derived from EUTIRADS analyses done by participants with and without AI assistance. Gold standard is biopsy recommendation derived from expert consensus EUTIRADS assessment of the nodules.	1 day (day of experiment)
Secondary	Nodule classification agreement	Agreement of nodule classification with gold standard. Nodule classifications are derived from EUTIRADS done by participants, S-Detect, and participant with S-Detect input on the five patients' thyroid nodules. Gold standard is expert consensus EUTIRADS assessment of the nodules.	1 day (day of experiment)
Secondary	Accuracy of EUTIRADS scores	Number of correct EUTIRADS scores given by analyses from participants, S-Detect, and participants with S-Detect input on the five patients' thyroid nodules. Gold standard is EUTIRADS score from expert consensus EUTIRADS assessment of the nodules.	1 day (day of experiment)
Secondary	Nodule measurement	Measurement of the five patients' thyroid nodules done by participants and S-Detect. Gold standard are measurements from expert consensus assessment analysis of the nodules.	1 day (day of experiment)
Secondary	OSAUS score	Mean OSAUS (objective structured assessment of ultrasound skills) scores of participants as assessed from their ultrasound scans of the five patients. Assessment are independently by two head-and-neck ultrasound experts.	1 day (day of experiment)