Clinical Trial Details
— Status: Completed
Administrative data
| NCT number |
NCT05489341 |
| Other study ID # |
CMO2016-3045-Project-20011 |
| Secondary ID |
|
| Status |
Completed |
| Phase |
|
| First received |
|
| Last updated |
|
| Start date |
February 1, 2022 |
| Est. completion date |
November 1, 2023 |
Study information
| Verified date |
July 2022 |
| Source |
Radboud University Medical Center |
| Contact |
n/a |
| Is FDA regulated |
No |
| Health authority |
|
| Study type |
Observational
|
Clinical Trial Summary
The PI-CAI challenge aims to validate the diagnostic performance of artificial intelligence
(AI) and radiologists at clinically significant prostate cancer (csPCa) detection/diagnosis
in MRI, with respect to histopathology and follow-up (≥ 3 years) as reference. The study
hypothesizes that state-of-the-art AI algorithms, trained using thousands of patient exams,
are non-inferior to radiologists reading bpMRI. As secondary end-points, it investigates the
optimal AI model for csPCa detection/diagnosis, and the effects of dynamic contrast-enhanced
imaging and reader experience on diagnostic accuracy and inter-reader variability.
Description:
Prostate cancer (PCa) is one of the most prevalent cancers in men worldwide. One million men
receive a diagnosis and 300,000 die from clinically significant PCa (csPCa) (defined as
ISUP≥2), each year, worldwide. Multiparametric magnetic resonance imaging (mpMRI) is playing
an increasingly important role in the early diagnosis of prostate cancer, and has been
recommended by the European Association of Urology (EAU), prior to biopsies. However, current
guidelines for reading prostate mpMRI (i.e. PI-RADS v2.1) follow a semi-quantitative
assessment that mandates substantial expertise for proper usage. This can lead to low
inter-reader agreement (<50%), sub-optimal interpretation and overdiagnosis.
Modern artificial intelligence (AI) algorithms have paved the way for powerful computer-aided
detection and diagnosis (CAD) systems that rival human performance in medical image analysis.
Clinical trials are the gold standard for assessing new medications and interventions in a
controlled and comparative manner, and the equivalent for developing AI algorithms are
international competitions or "grand challenges", where increasingly large datasets are
released to public to solve clinically relevant tasks with AI. Grand challenges can address
the lack of trust, scientific evidence and adequate validation among AI solutions, by
providing the means to compare algorithms against each other using common datasets and a
unified experimental setup.
PI-CAI (Prostate Imaging: Cancer AI) is an all-new grand challenge, with over 10,000
carefully-curated prostate MRI exams to validate modern AI algorithms and estimate
radiologists' performance at csPCa detection and diagnosis. Key aspects of the study design
have been established in conjunction with an international, multi-disciplinary scientific
advisory board (16 experts in prostate AI, radiology and urology) -to unify and standardize
present-day guidelines, and to ensure meaningful validation of prostate-AI towards clinical
translation.
The 2022 edition of PI-CAI will focus on validating AI at automated 3D detection and
diagnosis of csPCa in bpMRI. PI-CAI primarily consists of two sub-studies:
- AI Study (Grand Challenge): An annotated multi-center, multi-vendor dataset of 1500
bpMRI exams (including their clinical and acquisition variables) is made publicly
available for all participating teams and the research community at large. Teams can use
this dataset to develop AI models, and submit their trained algorithms (in Docker
containers) for evaluation. At the end of this open development phase, all algorithms
are ranked, based on their performance on a hidden testing cohort of 1000 unseen scans.
In the closed testing phase, organizers retrain the top-ranking 5 AI algorithms using a
larger dataset of 7500-9500 bpMRI scans (including additional training scans from a
private dataset). Finally, their performance is re-evaluated on the hidden testing
cohort (with rigorous statistical analyses), to determine the top 3 AI algorithms for
automated 3D detection and diagnosis of csPCa in bpMRI (i.e. the winners of the grand
challenge).
- Reader Study: 50+ international prostate radiologists perform a reader study using a
subset of 400 scans from the hidden testing cohort. For each case, radiologists complete
their assessments in two rounds. At first, using clinical and acquisition variables plus
bpMRI sequences only, enabling head-to-head comparisons against AI trained on the same.
And then, using clinical and acquisition variables plus full mpMRI sequences, enabling
comparisons between AI and current clinical practice (PI-RADS v2.1). Overall, the goal
of this study is to estimate the performance of the average radiologist at detection and
diagnosis of csPCa in MRI.
In the end, PI-CAI aims to benchmark state-of-the-art AI algorithms developed in the grand
challenge, against prostate radiologists participating in the reader study -to evaluate the
clinical viability of modern prostate-AI solutions at csPCa detection and diagnosis in MRI.
