Clinical Trial Details
— Status: Recruiting
Administrative data
NCT number |
NCT05427838 |
Other study ID # |
310995 |
Secondary ID |
|
Status |
Recruiting |
Phase |
|
First received |
|
Last updated |
|
Start date |
April 7, 2022 |
Est. completion date |
June 30, 2025 |
Study information
Verified date |
June 2024 |
Source |
Oxford University Hospitals NHS Trust |
Contact |
Alex Novak, Medicine |
Phone |
01865222003 |
Email |
alex.novak[@]ouh.nhs.uk |
Is FDA regulated |
No |
Health authority |
|
Study type |
Observational
|
Clinical Trial Summary
Background and study aims:
Computerised Tomography (CT) head scans are frequently requested by Emergency Department (ED)
clinicians as one of the investigations for their patients. This often causes a delay when
waiting for specialist radiologists to report the findings of the scan. The purpose of this
study is to see if online training can improve the ability of ED clinicians to interpret the
scans themselves, to a level sufficient to make clinical decisions based on their findings
and to explore what aspects of this process they find most challenging.
Participants:
Emergency Department clinicians who are working in the Emergency Departments of participating
sites between April to September 2022 (inclusive), who request CT Head scans as part of their
routine clinical practice.
What does the study involve?:
180 ED clinicians will be recruited across 6 hospital sites in the United Kingdom. All will
undertake a baseline online assessment to measure their accuracy in interpreting CT head
scans.
One group will then undertake an online training module, with a subsequent assessment
immediately afterwards, then over the following 3 months will record interpretations for 30
CT head scans.
Head images encountered in participants' routine clinical practice, and their findings, will
be compared with the radiology reports for each scan. Participants will then undertake
further online assessments 3 and 6 months after the start of the study. Their overall results
will be compared with a control group, who will undergo the same process, but undertake the
online training after they have tried to interpret 30 scans in their clinical practice.
Participants will continue to base their clinical decisions on radiologist reports, not their
own interpretations, so patient care will not be affected by this study.
Description:
STUDY DESIGN
This is a prospective interventional cohort study to assess the improvement in the image
interpretation accuracy and confidence of reporters using online simulation training for CT
head scan interpretation.
Using the RAIQC online platform (www.raiqc.com), the trial participants will be required to
complete a baseline assessment where users will review and interpret a pre-existing set of CT
scans, which have been curated and reported by senior radiologists. They will be asked to
record their interpretation, which will be subsequently compared against that of the
radiologists. The participants will be blinded to both original reports for the images, and
their overall performance results. After completing the baseline assessment, participants
will be given access to an online training package. After completing the training, the
participants will complete a further assessment. The improvement in diagnostic accuracy pre-
and post-training will be measured.
Once the participants have been trained, they will each be asked to evaluate a minimum total
of 30 CT scans during clinical shifts spread over a 3-month period. For each CT scan included
they will be asked to record their scan interpretation, the time the scan was performed, the
time they reviewed the scan and the time the radiology report was issued. The radiology
report will be considered the gold-standard diagnosis. Online assessments will be repeated at
the end of the 3 month period and at 6months post training, to monitor changes in performance
over time.
Subsequent comparison and analysis will aim to measure the real-world accuracy of ED staff at
interpreting CT head scans. In addition, the investigators will also be able to measure
potential time saving by ED staff interpreting the scans compared with the radiology report
being issued.
Population:
- 180 Emergency Department clinicians (30 per site) of varying grades who routinely interpret
images as part of their usual role, including: i. Junior Doctors (F1-ST2) ii. Registrars
(ST3-ST6) iii. Consultants iv. Radiographers v. Nursing and Allied Health Professionals
(NMAHPs) - Advanced Nurse Practitioners, Physician Associates
Methods:
Participants will be randomised to control (see section below - no online training, 30
participants, 5 from each site) versus intervention (online training - 150 participants)
groups. Prior to the start of the study, each participant ID will be randomised to either
grou via simple randomisation using an online number generator - the outcome of this
allocation will be recorded on a linkage document held by the central study team in Oxford.
Intervention Group:
- All intervention group participants will first be required to complete a baseline online
assessment during their non-clinical time, which will consist of CT head images with a
clinical vignette of varying difficulty and pathology where participants will review a
set of scans and provide a diagnosis and confidence score before submitting the case.
Time taken to complete each individual case will be recorded by the RAIQC platform.
Participants will be blinded to the results.
- Participants will be given access to an existing bespoke online training module on the
RAIQC platform (www.raiqc.com) with a post-training assessment consisting of a different
set of cases.
- Participants will then be required to evaluate a minimum of 30 CT Head scans each -
encountered during their clinical shifts over a 3-month period. They will be asked to
record results in real time on a standardised eCRF, including:
- date/time at which CT scan was performed
- date/time of participant review
- participant interpretation and confidence score
- date/time of radiology report
- radiology report findings (reference-standard)
NB: Participants will be instructed to base their subsequent clinical decision making based
on the formal radiology report as per standard clinical practice, not their own
interpretation.
- Participants will then each repeat a 50-case assessment at the end of the 3-month period
and again at 6 months post-training (to assess skill retention)
- Qualitative surveys will be completed by participants prior to each assessment module to
obtain feedback and comment regarding their experiences of the project and monitor any
changes over the course of the study. Participants will be issued with a certificate to
recognise that they have undergone imaging training for Continuous Professional
Development purposes
Controls:
- To provide a control for comparison as to the relative benefits of the online training
phase and the clinical interpretation phase in terms of improving reporting performance,
30 participants (5 participants from each site) will be randomised to undertake the
clinical interpretation phase prior to the training module
- Baseline accuracy statistics from each participant will be compared to subsequent
results from each stage of the study
Analysis:
- Training:
i. Changes in reporting accuracy (sensitivity/specificity), confidence and speed will be
calculated in a pooled analysis for all readers at all sites, as well as for the following
subgroups:
- Clinical role
- Level of seniority
- Pathological finding
- Difficulty of image
ii. These will be repeated for each of the four sequential assessments
- Clinical Interpretation:
i. Accuracy (sensitivity/specificity), confidence and speed will be calculated for all pooled
data, and the following subgroups:
- Clinical role
- Level of seniority
- Pathological finding
ii. Total and mean potential time saving will be calculated by comparing time of participant
interpretation versus time of radiology report, and will be used to calculate potential cost
benefit of clinician reporting versus radiology reporting
iii. Results from the qualitative surveys will be collated and used to explore the
experiences and factors affecting the uptake of new image reporting roles in the ED clinical
population
PARTICIPANT IDENTIFICATION
Informed Consent:
Participants will be provided with an electronic copy of the PIS via email. They will express
an interest in participating in the study to the site Principal Investigator, who will
discuss the study further with them and answer any questions. Consent will be sought from
participants to participate in the study and for their anonymised results to be used for
analysis and publication purposes. If they agree to the study, they will then be asked to
complete the consent form online and return it to the site Principal Investigator.
Discontinuation/Withdrawal of Participants from Study:
Each participant has the right to withdraw from the study at any time. In addition, the
Investigator may discontinue a participant from the study at any time if the Investigator
considers it necessary for any reason including:
- Ineligibility (either arising during the study or retrospectively having been overlooked
at screening)
- Significant protocol deviation
- Significant non-compliance with training regimen or study requirements
- Withdrawal of Consent
- Loss to follow up All data obtained to that point will be retained for analysis
Withdrawn participants will not be replaced. The reason for withdrawal will be recorded
in the CRF.
STATISTICS AND ANALYSIS
Sample Size:
During the pilot study, the diagnostic accuracy of participants in detecting acute
intracranial abnormality improved by a mean of 8% (SD15%). The investigators calculated a
sample size of 30 participants using a power of 80% and a level of significance of 5% (two
sided). This was a realistically and practically achievable target for each of the six sites,
and 180 participants was, therefore, chosen as the overall target for the current study.
Statistical Analyses:
- Training Assessments:
1) Changes in reporting accuracy/sensitivity/specificity/inter-reader
variability/confidence/speed will be calculated in a pooled analysis for all readers at
all sites, as well as for the following subgroups:
- Clinical role
- Level of seniority
- Pathological finding
- Difficulty of image
2) These will be repeated for each of the four sequential assessments
- Clinical Interpretation Phase:
1. Accuracy/sensitivity/specificity/inter-reader variability/confidence will
calculated for all pooled data, and the following subgroups
Clinical role Level of seniority Pathological finding
2. Total and mean potential time saving will by calculated by comparing time of
participant interpretation versus time of radiology report, and will be used to
calculate potential cost benefit of clinician reporting versus radiology reporting
3. Results from the qualitative surveys will be collated and used explore the
experiences and factors affecting the uptake of new image reporting roles in the ED
clinical population