E-CLAIR: Efficiency and Cost-effectiveness of Artificial Intelligence Based Diabetic Retinopathy Screening in Flanders

Diabetic Retinopathy Clinical Trial

— E-CLAIR  

Official title:

E-CLAIR: Efficiency and Cost-effectiveness of Artificial Intelligence Based Diabetic Retinopathy Screening in Flanders

Clinical Trial Details — Status: Recruiting

Administrative data

• NCT number: NCT05391659
• Other study ID #: S64955
• Status: Recruiting
• Phase: N/A
• Start date: June 17, 2021
• Est. completion date: December 1, 2022

Study information

• Verified date: May 2022
• Source: Universitaire Ziekenhuizen Leuven
• Contact: Liesje Prové
• Phone: 003216342874
• Email: lies.prove[@]uzleuven.be
• Is FDA regulated: No
• Study type: Interventional

Clinical Trial Summary

To evaluate the efficiency and cost-effectiveness of an artificial intelligence based diabetic retinopathy screening program in Flanders

Description:

The increase of diabetes patients is a 21st century global health challenge with a predicted 642 million people suffering from the disease by 2040. Diabetes mellitus is characterized by high blood sugar levels over a prolonged period of time. These uncontrolled blood sugar levels can damage the inner lining of blood vessels which on the long term causes microvascular complications that affect small blood vessels. Retinopathy is the most prevalent microvascular complication of diabetes and is caused by small blood vessel damage, and neural damage at the back layer of the eye, the retina.

Diabetic retinopathy (DR) is the leading cause of blindness and visual disability in the working population. According to a study of the Eye Diseases Prevalence Research group, 40% of adult diabetes patients in the United States have some degree of DR and 8% have vision-threatening forms of DR. In addition, the DR Barometer study indicated that many patients with diabetes do not have a regular appointment with ophthalmology for an eye examination. Risk of vision loss can be significantly decreased with annual retinal screening and detection of cases that need to be referred for follow-up and treatment. The best example showing the value of eye screening is from the United Kingdom (UK). As a result of an implementation of a nationwide screening program, DR is no longer the leading cause of irreversible blindness in the UK.

In Flanders, and in Belgium as a whole, no such well-organized, nationwide DR screening program is in place and the approach is more fragmented. Flemish guidelines for diabetes care recommend an annual visit to the ophthalmologist for all the diabetic patients who receive insulin therapy in order to check if they have DR. About 30% of the diabetics will be diagnosed for DR and 70% are disease free or in a very early stage that doesn't need further treatment. However, manual detection of DR performed by an occupied, scarce ophthalmologist is labor-intensive and expensive, causing long waiting times for the patient and possibly resulting in a lack of care when needed.

Given the extent of the diabetes population in Flanders it is self-evident that there are difficulties to screen all patients in a timely manner by ophthalmologists. Indeed, a large amount of diabetes type 2 patients do not follow the annual referral by their general practitioner (GP) and are therefore screened at a too late stage, resulting in high, avoidable costs for the patient and society. Even more, the screening of the diabetic patients by an ophthalmologist put a resource burden on our healthcare system. Task differentiation, where trained graders or GP's instead of ophthalmologists grade for referable DR, can offer a solution for the too long waiting times and the high cost. Nevertheless, manual grading of DR still is labor-intensive and costly. Even more, despite the implementation of nationwide screening programs for DR and their accompanying grading protocols, there is still substantial room for improvement in the accuracy of manual DR grading.

Recently, deep learning (DL), a form of artificial intelligence (AI), has been introduced for automated analysis of images. In a landmark paper, Gulshan and co-workers published on a deep learning algorithm with high sensitivity and specificity for detecting referable DR. This study paved the way for further developments in the field of deep learning for automated DR detection, resulting in DL models that achieve specialist-level accuracy in diagnosing DR severity. IDx, for example, obtained the first-ever FDA authorization for an AI diagnostic system in any field of medicine for DR detection.

Implementation of software for automated analysis is seen as a cost-effective solution to support decision-making in an eye screening program. In the study by Tufail et al. three different AI grading tools were retrospectively compared for their performance and cost-effectiveness in the DR screening program in the UK. In a follow-up study by Heydon et al. the most promising AI grading tool was prospectively evaluated for use in the UK screening program, demonstrating high sensitivity with a specificity that could halve the workload of the manual graders.

Despite recent research there is still an existing gap for AI to be implemented effectively and efficiently in DR screening programs. For example, the high false-positive rate of AI based results hamper the clinical workflow. Also important to note is that DL models cannot replace the breadth and contextual knowledge of human specialists. It is the case that even the most accurate models will still need to be implemented into an existing clinical workflow before they can improve patient care at all. Besides, the real-world uptake of AI applications is slow and this is partly due to a lack of convincing evidence of the economical impact.

Taken all together, renewal within diabetes care in Flanders, and more in particular further development of a more efficient DR screening pathway, is necessary to ensure that the accessibility and quality of diabetic eye care can be guaranteed at manageable costs. Flanders can undoubtedly benefit from a more efficient and cost-effective AI-assisted DR screening workflow that is at least as accurate as a human specialist. Note that the translation of study results abroad to the Flanders situation is limited. After all, one cannot simply assume that cost-effectiveness ratios from foreign economic evaluations also apply in the Flanders context. Meaning that policymakers cannot base their decisions on the possible introduction of preventive screening interventions in Flanders directly on foreign studies. These findings demonstrate the clear need to set up a specific research project in Flanders to evaluate the efficiency and cost-effectiveness of a tailor-made DR screening program in Flanders.

Recruitment information / eligibility

• Status: Recruiting
• Enrollment: 1200
• Est. completion date: December 1, 2022
• Est. primary completion date: November 1, 2022
• Accepts healthy volunteers: No
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

• Diagnosis of diabetes mellitus

• Age > 18 years old

• Patient is capable of giving informed consent

• Fluent in written and oral Dutch, or interpreter present

Exclusion Criteria:

• - History of treatment for diabetic retinopathy or diabetic macular edema (laser or intravitreal injections)

• Participant is contraindicated for imaging by fundus imaging systems used in the study

Related Conditions & MeSH terms

• Diabetic Retinopathy
• Retinal Diseases

Intervention

Device:
• deep learning
a form of artificial intelligence (AI), has been introduced for automated analysis of images

Diagnostic Test:
• remote grading of fundus images
referrable cases identified by DR AI tool will be remotely graded by a human
• gold standard
examination by ophthalmologist

Locations

Country	Name	City	State
Belgium	ZNA	Antwerp
Belgium	UZA	Antwerpen
Belgium	AZ sint Jan	Brugge
Belgium	AZ Turnhout	Turnhout

Sponsors (1)

Lead Sponsor	Collaborator
Universitaire Ziekenhuizen Leuven

Country where clinical trial is conducted

Belgium, 

References & Publications (16)

Abràmoff MD, Folk JC, Han DP, Walker JD, Williams DF, Russell SR, Massin P, Cochener B, Gain P, Tang L, Lamard M, Moga DC, Quellec G, Niemeijer M. Automated analysis of retinal images for detection of referable diabetic retinopathy. JAMA Ophthalmol. 2013 Mar;131(3):351-7. doi: 10.1001/jamaophthalmol.2013.1743. — View Citation

Farley TF, Mandava N, Prall FR, Carsky C. Accuracy of primary care clinicians in screening for diabetic retinopathy using single-image retinal photography. Ann Fam Med. 2008 Sep-Oct;6(5):428-34. doi: 10.1370/afm.857. — View Citation

Harding SP, Broadbent DM, Neoh C, White MC, Vora J. Sensitivity and specificity of photography and direct ophthalmoscopy in screening for sight threatening eye disease: the Liverpool Diabetic Eye Study. BMJ. 1995 Oct 28;311(7013):1131-5. — View Citation

Heydon P, Egan C, Bolter L, Chambers R, Anderson J, Aldington S, Stratton IM, Scanlon PH, Webster L, Mann S, du Chemin A, Owen CG, Tufail A, Rudnicka AR. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol. 2021 May;105(5):723-728. doi: 10.1136/bjophthalmol-2020-316594. Epub 2020 Jun 30. — View Citation

Kapetanakis VV, Rudnicka AR, Liew G, Owen CG, Lee A, Louw V, Bolter L, Anderson J, Egan C, Salas-Vega S, Rudisill C, Taylor P, Tufail A. A study of whether automated Diabetic Retinopathy Image Assessment could replace manual grading steps in the English National Screening Programme. J Med Screen. 2015 Sep;22(3):112-8. doi: 10.1177/0969141315571953. Epub 2015 Mar 5. — View Citation

Kempen JH, O'Colmain BJ, Leske MC, Haffner SM, Klein R, Moss SE, Taylor HR, Hamman RF; Eye Diseases Prevalence Research Group. The prevalence of diabetic retinopathy among adults in the United States. Arch Ophthalmol. 2004 Apr;122(4):552-63. — View Citation

Lee R, Wong TY, Sabanayagam C. Epidemiology of diabetic retinopathy, diabetic macular edema and related vision loss. Eye Vis (Lond). 2015 Sep 30;2:17. doi: 10.1186/s40662-015-0026-2. eCollection 2015. Review. — View Citation

Liew G, Michaelides M, Bunce C. A comparison of the causes of blindness certifications in England and Wales in working age adults (16-64 years), 1999-2000 with 2009-2010. BMJ Open. 2014 Feb 12;4(2):e004015. doi: 10.1136/bmjopen-2013-004015. — View Citation

Lin DY, Blumenkranz MS, Brothers RJ, Grosvenor DM. The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography. Am J Ophthalmol. 2002 Aug;134(2):204-13. — View Citation

Ogurtsova K, da Rocha Fernandes JD, Huang Y, Linnenkamp U, Guariguata L, Cho NH, Cavan D, Shaw JE, Makaroff LE. IDF Diabetes Atlas: Global estimates for the prevalence of diabetes for 2015 and 2040. Diabetes Res Clin Pract. 2017 Jun;128:40-50. doi: 10.1016/j.diabres.2017.03.024. Epub 2017 Mar 31. — View Citation

Sayres R, Taly A, Rahimy E, Blumer K, Coz D, Hammel N, Krause J, Narayanaswamy A, Rastegar Z, Wu D, Xu S, Barb S, Joseph A, Shumski M, Smith J, Sood AB, Corrado GS, Peng L, Webster DR. Using a Deep Learning Algorithm and Integrated Gradients Explanation to Assist Grading for Diabetic Retinopathy. Ophthalmology. 2019 Apr;126(4):552-564. doi: 10.1016/j.ophtha.2018.11.016. Epub 2018 Dec 13. — View Citation

Sussman EJ, Tsiaras WG, Soper KA. Diagnosis of diabetic eye disease. JAMA. 1982 Jun 18;247(23):3231-4. — View Citation

Tufail A, Kapetanakis VV, Salas-Vega S, Egan C, Rudisill C, Owen CG, Lee A, Louw V, Anderson J, Liew G, Bolter L, Bailey C, Sadda S, Taylor P, Rudnicka AR. An observational study to assess if automated diabetic retinopathy image assessment software can replace one or more steps of manual imaging grading and to determine their cost-effectiveness. Health Technol Assess. 2016 Dec;20(92):1-72. — View Citation

Tufail A, Rudisill C, Egan C, Kapetanakis VV, Salas-Vega S, Owen CG, Lee A, Louw V, Anderson J, Liew G, Bolter L, Srinivas S, Nittala M, Sadda S, Taylor P, Rudnicka AR. Automated Diabetic Retinopathy Image Assessment Software: Diagnostic Accuracy and Cost-Effectiveness Compared with Human Graders. Ophthalmology. 2017 Mar;124(3):343-351. doi: 10.1016/j.ophtha.2016.11.014. Epub 2016 Dec 23. — View Citation

Wolff J, Pauling J, Keck A, Baumbach J. The Economic Impact of Artificial Intelligence in Health Care: Systematic Review. J Med Internet Res. 2020 Feb 20;22(2):e16866. doi: 10.2196/16866. — View Citation

Xie Y, Nguyen QD, Hamzah H, Lim G, Bellemo V, Gunasekeran DV, Yip MYT, Qi Lee X, Hsu W, Li Lee M, Tan CS, Tym Wong H, Lamoureux EL, Tan GSW, Wong TY, Finkelstein EA, Ting DSW. Artificial intelligence for teleophthalmology-based diabetic retinopathy screening in a national programme: an economic analysis modelling study. Lancet Digit Health. 2020 May;2(5):e240-e249. doi: 10.1016/S2589-7500(20)30060-1. Epub 2020 Apr 23. — View Citation

* Note: There are 16 references in all — Click here to view all references

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	sensitivity	To evaluate the efficiency of the use of AI in screening for DRP: sensitivity	6 months
Primary	specificity	To evaluate the efficiency of the use of AI in screening for DRP: specificity	6 months
Primary	AUC	To evaluate the efficiency of the use of AI in screening for DRP: AUC	6 months
Secondary	precision	performance of three DR screening workflows: precision	6 months
Secondary	decision tree model	cost-effectiveness of three DR screening workflows: decision tree model	6 months
Secondary	recall	performance of three DR screening workflows : recall	6 months
Secondary	F1 score	performance of three DR screening workflows: F1 score	6 months
Secondary	false positives and false negatives	performance of three DR screening workflows: false positives and false negatives	6 months