Using AI as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia

Skin Diseases Clinical Trial

Official title:

Using Artificial Intelligence as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT04562168
• Other study ID #: P20/159-P
• Status: Completed
• Phase: N/A
• Start date: January 15, 2021
• Est. completion date: December 31, 2021

Study information

• Verified date: March 2022
• Source: Jordi Gol i Gurina Foundation
• Contact: n/a
• Is FDA regulated: No
• Study type: Interventional

Clinical Trial Summary

Background: Dermatological conditions are a relevant health problem. Machine learning models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, specially for skin cancer detection and classification.

Objective: The objective of this study is to perform a prospective validation of an image analysis ML model, which is capable of screening 44 different skin disease types, comparing its diagnostic capacity with that of General Practitioners (GPs) and dermatologists.

Methods: In this prospective study 100 consecutive patients who visit a participant GP with a skin problem in central Catalonia will be recruited, data collection is planned to last 7 months. Skin diseases anonymized pictures will be taken and introduced in the ML model interface, which will return top 5 accuracy diagnosis. The same image will be also sent as a teledermatology consultation, following the current workflow. GP, ML model and dermatologist/s assessments will be compared to calculate the precision, sensitivity, specificity and accuracy of the ML model.

Description:

A secure anonymous stand alone web interface that is compatible to any mobile device will be integrated with the Autoderm API. The study conducted in this project will consist in a prospective study aimed to evaluate the ML model performance, comparing its diagnostic capacity with GPs and dermatologists.

To conduct the study the following procedure will be executed until the required number of samples is reached:

1. A suitable patient with skin concern is asked to participate and sign the patient's study agreement.

2. GP will diagnose the skin condition.

3. GP (or nurse) will take one good quality image of the skin condition.

4. GP will send the photograph as a teledermatology consultation following the current workflow.

5. The image is entered in the Autoderm ML interface.

6. Dermatologist will diagnose the skin condition.

The study will be conducted in primary care centers managed by the Catalan Health Institute. Participant PCP will be located in rural and metropolitan areas in Central Catalonia, which includes the regions of Anoia, Bages, Moianès, Berguedà and Osona. The reference population included in the study will be about 512,050. The recruitment of prospective subjects will consist on a consecutive basis.

General practitioners will collect data from consecutive patients who meet the inclusion criteria after obtaining written informed consent. Collected data will be reported exclusively in case report form (attached at Annex V and VI).

The GP will diagnose the skin condition and will fill the "Face-to-face assessment by GP". For each patient, the GP using a smartphone camera will take a close up good quality image of the skin problem. The image will be anonymous and it will be not possible to identify patients. The GP will use the Autoderm ML interface to upload the anonymized image and will fill the "Assessment provided by the ML model" questionnaire with the top 3 diagnoses generated by the ML model.

In order to get a second opinion, the GP will incorporate the anonymized image and an accurate description of the skin lesion into the patient's medical history following the current teledermatology flow. The GP will fill "Assessment by teledermatology" questionnaire after receiving the information, being response time about 2-7 days.

In case of dermatology referral, the GP will fill "Assessment by in person dermatologist", by accessing electronic health records as they become available, being the average waiting time for referral from 30 to 90 days.

Questionnaire case number will be the same for all questionnaires and it will not be possible to identify the patient, as case number will be predefined before the initiation of the data collection phase.

To compare the performance of the ML model with that of the GPs and dermatologists, it will be required a sample size of 100 images of skin diseases from patients who meet the inclusion criteria. The proposed sample size is based on sample size calculation used in similar research.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 100
• Est. completion date: December 31, 2021
• Est. primary completion date: December 31, 2021
• Accepts healthy volunteers: No
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

• Patients who have cutaneous disease reason-for-visit.

• Patients who provide written informed consent.

• Patients who are 18 years of age or older.

Exclusion Criteria:

• Patients with advanced dementia.

• Patients with a cutaneous lesion which can't be photographed with a smartphone and images with poor quality.

• Patients who have conditions associated with risk of poor protocol compliance.

Related Conditions & MeSH terms

• Skin Diseases

Intervention

Diagnostic Test:
• Autoderm® dermatology search engine (ML model) testing
GP using a smartphone camera will take an image of the skin problem and will use the Autoderm ML interface to upload the anonymized image. The obtained predicted diagnosis will be recorded in case report form.

Locations

Country	Name	City	State
Spain	CAP Navàs	Navàs	Barcelona

Sponsors (3)

Lead Sponsor	Collaborator
Jordi Gol i Gurina Foundation	iDoc24, Institut Català de la Salut

Country where clinical trial is conducted

Spain, 

References & Publications (16)

Activitat assistencial de la xarxa sanitària de Catalunya, any 2012: registre del conjunt mínim bàsic de dades (CMBD). Barcelona: Departament de Salut. 2013.

Börve A, Dahlén Gyllencreutz J, Terstappen K, Johansson Backman E, Aldenbratt A, Danielsson M, Gillstedt M, Sandberg C, Paoli J. Smartphone teledermoscopy referrals: a novel process for improved triage of skin cancer patients. Acta Derm Venereol. 2015 Feb — View Citation

Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C, Schilling B, Haferkamp S, Schadendorf D, Holland-Letz T, Utikal JS, von Kalle C; Collaborators. Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma imag — View Citation

Dokotor.se [Internet]. Survey Telemedicine statistics Dokotor.se, the % of queries that are dermatology related 2019 [cited 2019]

Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017 Feb 2;542(7639):115-118. doi: 10.1038/nature21056. Epub 2017 Jan 25. Erratum in: Nature. 2017 Jun 2 — View Citation

Evaluation of the diagnostic accuracy of an online artificial intelligence app for skin disease diagnosis. Alexander Larson, Degree Project in Medicine, Sahlgrenska University Hospital Department of Dermatology and Venereology, Gothenburg, Sweden 2018.

Ferrer RT, Bezares AP, Mañes AL, Mas AV, Gutiérrez IT, Lladó CN, Estaràs GM. [Diagnostic reliability of an asynchronous teledermatology consultation]. Aten Primaria. 2009 Oct;41(10):552-7. doi: 10.1016/j.aprim.2008.11.012. Epub 2009 Jun 5. Spanish. — View Citation

Gomolin A, Netchiporouk E, Gniadecki R, Litvinov IV. Artificial Intelligence Applications in Dermatology: Where Do We Stand? Front Med (Lausanne). 2020 Mar 31;7:100. doi: 10.3389/fmed.2020.00100. eCollection 2020. Review. — View Citation

Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A, Kalloo A, Hassen ABH, Thomas L, Enk A, Uhlmann L; Reader study level-I and level-II Groups, Alt C, Arenbergerova M, Bakos R, Baltzer A, Bertlich I, Blum A, Bokor-Billmann T, Bowling J, Brag — View Citation

Kamulegeya LH, Okello M, Bwanika JM, Musinguzi D, Lubega W, Rusoke D, et al. Using artificial intelligence on dermatology conditions in Uganda: A case for diversity in training data sets for machine learning. bioRxiv [Internet]. 2019 Jan 1;826057. Availab

Lim HW, Collins SAB, Resneck JS Jr, Bolognia JL, Hodge JA, Rohrer TA, Van Beek MJ, Margolis DJ, Sober AJ, Weinstock MA, Nerenz DR, Smith Begolka W, Moyano JV. The burden of skin disease in the United States. J Am Acad Dermatol. 2017 May;76(5):958-972.e2. — View Citation

Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P, Kanada K, de Oliveira Marinho G, Gallegos J, Gabriele S, Gupta V, Singh N, Natarajan V, Hofmann-Wellenhof R, Corrado GS, Peng LH, Webster DR, Ai D, Huang SJ, Liu Y, Dunn RC, Coz D. A deep learning system for dif — View Citation

López Seguí F, Franch Parella J, Gironès García X, Mendioroz Peña J, García Cuyàs F, Adroher Mas C, García-Altés A, Vidal-Alaball J. A Cost-Minimization Analysis of a Medical Record-based, Store and Forward and Provider-to-provider Telemedicine Compared t — View Citation

Lowell BA, Froelich CW, Federman DG, Kirsner RS. Dermatology in primary care: Prevalence and patient disposition. J Am Acad Dermatol. 2001 Aug;45(2):250-5. — View Citation

Porta N, San Juan J, Grasa MP, Simal E, Ara M, Querol MA. [Diagnostic agreement between primary care physicians and dermatologists in the health area of a referral hospital]. Actas Dermosifiliogr. 2008 Apr;99(3):207-12. Spanish. — View Citation

Schofield JK, Fleming D, Grindlay D, Williams H. Skin conditions are the commonest new reason people present to general practitioners in England and Wales. Br J Dermatol. 2011 Nov;165(5):1044-50. doi: 10.1111/j.1365-2133.2011.10464.x. Epub 2011 Sep 22. — View Citation

* Note: There are 16 references in all — Click here to view all references

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Sensitivity of the ML model	True positive rate of the ML model	1 year
Primary	Specificity of the ML model	True negative rate of the ML model	1 year
Primary	Accuracy of the ML model	Ratio of number of correct predictions to the total number of input samples	1 year
Primary	Area under the receiver operating characteristic curve of the ML model	Diagnostic ability of the ML model	1 year
Secondary	Rate of the eligible participants who agree to participate in the study	Frequency of patients who agree to participate in the clinical trial and are eligible.	1 year