Evaluation of Contextflow DETECT Lung CT Nodule Detection Software in Chest CT Scans

Lung Cancer Clinical Trial

Official title:

Evaluation of Computer-Aided Lung Nodule Detection Software in Chest CT Scans With an Assessment of Its Impact on Readers Decision-Making Process

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT05481762
• Other study ID #: 0020220401
• Status: Completed
• Start date: April 1, 2022
• Est. completion date: October 30, 2022

Study information

• Verified date: February 2023
• Source: contextflow GmbH
• Contact: n/a
• Is FDA regulated: No
• Study type: Observational

Clinical Trial Summary

contextflow DETECT Lung CT is a Artificial Intelligence (AI)-based computed-aided detection (CADe) system, intended to support radiologists in the detection of lung nodules in chest computed tomography (CT) scans. System is intended to be used as a second-reader, therefore results provided by the software are meant to complement the radiologist's findings and decisions.

Proposed study will be multi-reader, multi case (MRMC) retrospective reader study. The goal of the study is to evaluate the influence of CADe on the effectiveness of lung nodule detection. During the study, 10 radiologists will analyze 350 chest CT scans of adult patients, with and without the assistance of CADe. The study will be conducted remotely. CT scans will be uploaded to a web-based image submission and annotation platform, in which every participant of the study will be provided with individual account and assigned task list.

The primary objective of the study determine if the diagnostic accuracy of radiologists with CADe assistance is superior to the diagnostic accuracy of radiologists without CADe assistance in localizing the pulmonary nodules with enhanced area under the free-response operating characteristic curve (AUC of FROC).

The study will target approximately 350 asymptomatic adult patients, whose CT scans were acquired during routine CT examination. The patient population will include patients with and without lung nodules.

Description:

1. Background and rationale

According to the American Cancer Society, lung cancer was the second commonly diagnosed cancer with an estimated 2.2 million new cases in 2020. Currently, the most effective way of reducing cancer mortality is early diagnosis of lung cancer. However, in routine clinical practice, many early-stage lung cancer were delayed in diagnosis, due to the fact that most patients are often asymptomatic. That's the main reason why fast and the least invasive lung disease screening is so important for the early detection of pathological changes in asymptomatic patients from high-risk cohorts. Computed tomography (CT) is the most recent dominated imagining tool used to capture lung images, and therefore also for nodule detection and characterization. Consequently, computer-aided detection and software systems based on artificial intelligence have become one of the most prominent and valuable tools for detection of pulmonary nodules.

The aim of the proposed reader study is to evaluate one of such tools, contextflow DETECT Lung CT, and to assess to what extent supporting radiologists with a computer-aided detection system improves the detection of nodules and supports radiologists in the decision-making process.

contextflow DETECT Lung CT is an AI-based web application for 3D medical imaging data. It's intended to be used for the detection and visualization of pulmonary nodules in chest CT scans of clinically asymptomatic adult patients.

The system is intended to alert radiologists to specific locations in which nodules can be detected that may otherwise be overlooked. The system provides measurements of the nodules, such as average diameter, long-axis diameter, short-axis diameter as well as the total volume.

contextflow DETECT Lung CT is intended to be used as a second reader, in addition to original reading performed by radiologists. Results provided by contextflow DETECT Lung CT are intended to complement the information that radiologists take into account during their decision-making process.

2. Study design

This will be a retrospective, multi-reader multi-case (MRMC) study. The performance of radiologists alone will be compared with the performance of radiologists supported by the detection software (as a second reader). The study will include 10 readers, and a final dataset of 350 scans collected from at least 3 radiology centers and revised by truthers. During the study three following reads are planned: Read 1 (Truthing), Read 2 (Baseline, without CADe assistance), Read 3 (Intervention, with CADe assistance).

The study will consist of 350 CT images collected from asymptomatic adult patients. At least 50% of the cases shall be US data, other cases will be collected from European radiology centers. At least 65% of the cases will include at least one lung nodule. The dataset will also include cases without nodules.

3. Selection of readers

Readers qualification criteria:

• US-board of Radiology or equivalent specialty certification

• At least 5 years of chest CT interpretation experience

• Fellowship-trained in thoracic/chest imaging,

• Successful training on the use of study software.

4. Selection of patients and imaging data

Patients, whose chest CT images were included into the dataset, must meet all the following criteria:

• Adult patients (aged 22 or older) with the number of nodules ranged from 0 to 10;

• For patients with at least one lung nodule: nodules detected and confirmed in the ground-truthing procedure;

• Asymptomatic patients, who undergo a routine chest CT scan.

Imaging data must meet all the following inclusion criteria to be in this study:

• Images are CT images of the chest showing full left and right lung;

• Images do not show any movement artifacts of the patient;

• Images of the chest must be reconstructed in axial plane;

• The number of nodules per image was 10 or fewer;

• All nodules measuring between 4 - 30 mm in maximal long-axis diameter at CT scan.

5. Reader study procedures

Ground truth process workflow: 3 US board-certified thoracic radiologists with at least 5 years experience after corresponding fellowship will be asked to read CT scans and identify actionable nodules in images from ground truthing dataset (370 CT scans randomly chosen from balanced candidate database) by marking them by manually drawing bounding boxes around them.

Reading process workflow: For each CT scan, the reading process will take place in two steps: (1) reader's stand-alone reading, without contextflow DETECT Lung CT assistance and (2) second reader's reading with contextflow DETECT Lung CT as a second-reader. In each step, readers will analyze 350 CT scans dataset, with limited clinical information.

6. Blinding and randomization

The identification of cases selected for the study, as well as individual analyzes between readers will be blinded to readers. Readers will not have access to the results from truthing procedure or to the information how many actionable nodules have been localized in individual CT images.

Image submission and annotation platform will generate the random task list for each reader. The reader order will be different for all of the readers.

7. Study objectives and endpoints

The primary and secondary objectives of the reader study is to demonstrate enhanced diagnostic accuracy of radiologists with CADe in comparison with the diagnostic accuracy of radiologists without CADe in

Primary

• Localizing the lesions with the enhanced area under the free-response receiver operating characteristic curve (AUC of FROC)

Secondary

• Detecting disease using (AUC of ROC)

• Enhancing disease identification with AUC of LROC

The following endpoints will also be evaluated:

• The difference in sensitivities of lesion detection with and without CADe given specified average false positive numbers of lesions per image;

• The difference in sensitivities of disease diagnostic with and without CADe given specified specificity;

• Positive predictive value for disease diagnosis

• Negative predictive value for disease diagnosis

8. Data management

All data that is part of the study, both shared by our partners or downloaded from public sources, was anonymized following HIPAA and/or GDPR regulations and does not include any protected health information.

9. End of the study

The reader study will be considered completed when a sufficient number of images have been analyzed to achieve a satisfactory statistical test performance in the means of statistical power. To achieve that, at least 300 scans should be analyzed.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 337
• Est. completion date: October 30, 2022
• Est. primary completion date: October 30, 2022
• Accepts healthy volunteers: Accepts Healthy Volunteers
• Gender: All
• Age group: 22 Years and older

Eligibility

Inclusion Criteria:

• adult asymptomatic patients, who undergo a routine chest CT scan.

Exclusion Criteria:

• symptomatic patients.

Related Conditions & MeSH terms

• Lung Cancer

Intervention

Device:
• Aided read with contextflow DETECT Lung CT
Radiologists read chest CT scabs with and without the aid of contextflow DETECT Lung CT as a second reader

Locations

Country	Name	City	State
Austria	contextflow GmbH	Vienna

Sponsors (1)

Lead Sponsor	Collaborator
contextflow GmbH

Country where clinical trial is conducted

Austria, 

References & Publications (3)

Hansell DM, Bankier AA, MacMahon H, McLoud TC, Muller NL, Remy J. Fleischner Society: glossary of terms for thoracic imaging. Radiology. 2008 Mar;246(3):697-722. doi: 10.1148/radiol.2462070712. Epub 2008 Jan 14. — View Citation

Qian F, Yang W, Chen Q, Zhang X, Han B. Screening for early stage lung cancer and its correlation with lung nodule detection. J Thorac Dis. 2018 Apr;10(Suppl 7):S846-S859. doi: 10.21037/jtd.2017.12.123. — View Citation

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021 May;71(3):209-249. doi: 10.3322/caac.21660. Epub 2021 Feb 4. — View Citation

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Diagnostic accuracy	The primary objective of the reader study is to determine if the diagnostic accuracy of radiologists with CADe assistance is superior to the diagnostic accuracy of radiologists without CADe assistance in localizing the pulmonary nodules with enhanced area under the free-response operating characteristic curve (AUC of FROC).; The true positive rate (or sensitivity) is calculated as the identified positive lesion among the true positive divided by the total number of true positive lesions among all images. The number of false positive findings is collected per image.	20 hours
Secondary	Disease diagnosis capabilities	To evaluate the capabilities in disease diagnosis by radiologists with and without CADe assistance using the area under the receiver operating characteristics curve (AUC of ROC).; The ROC curve is plotted for the true positive and false positive rates based on various confidence thresholds in the reader's diagnostic decision.	20 hours
Secondary	Disease identification capabilities	To evaluate the capabilities in disease identification of radiologists with accurate localization with and without CADe assistance using the area under the localization receiver operating characteristics curve (AUC of LROC).; The LROC curve is plotted for the true positive and false positive rates based on various confidence thresholds in readers.	20 hours

External Links

•   American Cancer Society, Cancer Statistics Center