Autonomous Telephone Follow-up After Cataract Surgery

Cataract Clinical Trial

Official title:

A Clinical Investigation of an Autonomous Phone Conversational Agent for Cataract Surgery Follow-up

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT05213390
• Other study ID #: 21WE6780
• Status: Completed
• Phase: N/A
• Start date: September 17, 2021
• Est. completion date: March 24, 2022

Study information

• Verified date: September 2022
• Source: University of Plymouth
• Contact: n/a
• Is FDA regulated: No
• Study type: Interventional

Clinical Trial Summary

This project will apply AI technology to meet the gap between increasing demand and limited capacity of high- volume healthcare services. The project will develop evidence that will support the safe deployment of Ufonia's automated telemedicine platform to deliver calls to cataract surgery patients at two large NHS hospital trusts.

The proposed study will implement DORA in addition to the current standard of care for a cohort of patients at Imperial College Healthcare Trust and Oxford University Hospitals NHS Foundation Trust. The study will evaluate the agreement of DORA's decision with an expert clinician. In addition it will test the acceptability of the solution for patients and clinicians; the sensitivity and specificity of the system in deciding if a patient requires additional review; and the health economic benefits of the solution to patients (reduced time and travel) and the local healthcare system. If successful, a proposal will be developed to roll the solution out to all patients at each site in anticipation of an application to a late phase award for wider NHS deployment.

Description:

Background

Due to an ageing population and increased expectation, the demand for many services is exceeding the capacity of the clinical workforce. As a result, staff are facing a crisis of burnout from being pressured to deliver high- volume workloads, driving increasing costs for providers. Artificial intelligence, in the form of conversational agents, presents a possible opportunity to enable efficiencies in the delivery of care.

Aims and Objectives

This study aims to evaluate the effectiveness, usability and acceptability of DORA - an AI-enabled autonomous telemedicine call - for detection of post-operative cataract surgery patients who require further assessment. The study's objectives are: to establish efficacy of DORA's decision making in comparison to an expert human clinician; baseline sensitivity and specificity for detection of true complications; evaluation of patient acceptability; evidence for cost-effectiveness; and to capture data that may support further studies.

Project plan and methods used

Based on implementation science, the interdisciplinary study will be a mixed-methods phase one pilot establishing inter-observer reliability; as well as usability and acceptability.

Timelines for delivery

The study will last eighteen months: seven months of evaluation and intervention refinement, nine months of implementation and follow-up, and two months of post-evaluation analysis and write-up.

Anticipated Impact and Dissemination

The project's key contributions will be evidence on artificial intelligence voice conversational agent effectiveness, and associated usability and acceptability. Results will be disseminated in peer-reviewed journals and at international medical sciences and engineering conferences.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 225
• Est. completion date: March 24, 2022
• Est. primary completion date: January 31, 2022
• Accepts healthy volunteers: No
• Gender: All
• Age group: N/A and older

Eligibility

Inclusion Criteria:

• Willing and able to provide informed consent;

• Aged 18 years or older;

• On the waiting list for routine cataract surgery. Cataract surgery as part of a combined procedure with other ocular surgery will not be included;

• No history or presence of significant ocular comorbidities that would be expected to alter the risks of cataract surgery or normal post-operative follow-up schedule. Note that significant ocular comorbidities do not include stable, chronic, or inactive ocular conditions such as amblyopia, drop-controlled stable glaucoma or ocular hypertension, previous squint surgery, inactive macular pathology, previous refractive surgery, or previous vitreoretinal surgery with stable retina.

Exclusion Criteria:

• Individuals with any condition that could preclude the ability to comply with the study or follow-up procedures;

• Presence of ocular or systemic uncontrolled disease (unless deemed not clinically significant by the Investigator and Sponsor);

• Involved in current research related to this technology or been involved in related research to this technology prior to recruitment;

• Cognitive difficulties, hearing impairment or non-English speakers;

• History of current or severe, unstable or uncontrolled systemic disease (unless deemed not clinically significant by the Investigator and Sponsor).

Related Conditions & MeSH terms

• Capsule Opacification
• Cataract

Intervention

Other:
• Dora
DORA uses a variety of AI technologies to deliver the patient follow-up call, including: speech transcription, natural language understanding, a machine-learning conversation model to enable contextual conversations, and speech generation. Together, these technologies cover the input, processing and analysis, and output needed to maintain a natural conversation. DORA is configured to deliver calls through a telephone connection as a real-time, stand-alone system: the operator inputs individual patient details to initiate the call and completes a summary in the electronic health record (EHR) afterwards.

Locations

Country	Name	City	State
United Kingdom	Imperial College Healthcare NHS Trust	London
United Kingdom	Oxford University Hospitals NHS Foundation Trust	Oxford

Sponsors (3)

Lead Sponsor	Collaborator
University of Plymouth	Imperial College London, University of Oxford

Country where clinical trial is conducted

United Kingdom, 

References & Publications (2)

de Pennington N, Mole G, Lim E, Milne-Ives M, Normando E, Xue K, Meinert E. Safety and Acceptability of a Natural Language Artificial Intelligence Assistant to Deliver Clinical Follow-up to Cataract Surgery Patients: Proposal. JMIR Res Protoc. 2021 Jul 28;10(7):e27227. doi: 10.2196/27227. — View Citation

Milne-Ives M, de Cock C, Lim E, Shehadeh MH, de Pennington N, Mole G, Normando E, Meinert E. The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review. J Med Internet Res. 2020 Oct 22;22(10):e20346. doi: 10.2196/20346. — View Citation

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Agreement	Inter-rater reliability: the degree of agreement between DORA and the clinician on their assessments of the individual symptoms and the management plan; Whether or not the clinician had to interrupt the call to ask clarifying questions	6 months
Secondary	Clinical complications identified or missed by DORA system	Complications identified from patients' electronic health records up to 90 days following cataract surgery; Congruence between complications identified and management planned in DORA call and face-to-face follow up (Imperial); Comparison to data from patients attending eye casualty (Oxford)	Up to 90 days post surgery
Secondary	Proportion of calls completed without intervention	Proportion of autonomous calls that were completed without needing any intervention from the supervising clinician; Clinician-reported reasons for asking clarifying questions	6 months
Secondary	System usability	Measured using the System Usability Scale (minimum of 0, maximum of 100, higher scores indicate better usability)	6 months
Secondary	Usability of telehealth system implementation	Measured using the Telehealth Usability Questionnaire (minimum score of 1, maximum score of 5, averaged across 19 items; higher scores indicate better usability)	6 months
Secondary	Qualitative patient perspectives of usability	Qualitative feedback from semi-structured interviews	6 months
Secondary	Acceptability of AI follow-up phone call	Qualitative feedback from semi-structured interviews	6 months
Secondary	Satisfaction with AI follow-up phone call	Qualitative feedback from semi-structured interviews	6 months
Secondary	Appropriateness of AI for follow-up assessment	Qualitative feedback from semi-structured interviews	6 months
Secondary	Cost impact	Comparison of the costs of implementing DORA and the costs of the usual standard of care	6 months