Clinical Trial Details
— Status: Completed
Administrative data
| NCT number |
NCT04577079 |
| Other study ID # |
KUH507P004 |
| Secondary ID |
|
| Status |
Completed |
| Phase |
|
| First received |
|
| Last updated |
|
| Start date |
September 1, 2020 |
| Est. completion date |
November 18, 2022 |
Study information
| Verified date |
November 2022 |
| Source |
Kuopio University Hospital |
| Contact |
n/a |
| Is FDA regulated |
No |
| Health authority |
|
| Study type |
Observational
|
Clinical Trial Summary
Digital health technologies (DHT) are increasingly developed to support healthcare systems
around the world. However, they are frequently lacking evidence-based medicine and medical
validation. There is considerable need in the western countries to allocate healthcare
resources accurately and give the population detailed and reliable health information
enabling to take greater responsibility for their health. Intelligent patient flow management
system (IPFM, product name Klinik Frontline) is developed to meet these needs. In practice,
IPFM is used for decision support in the triaging and diagnostic processes as well as
automatizing the management of inflow of the patients. The core of the IPFM is a clinical
artificial intelligence (AI), which utilizes a comprehensive medical database of clinical
correlations generated by medical doctors.
The study population of this research consists of patients from the Emergency Department of
Kuopio University Hospital (KUH). Data will be gathered during 2 weeks of piloting, after
which the results will be analysed. Anticipated number of patients to the study is minimum of
246 patients, with objective to be several hundreds. When attending to the hospital, patients
will report their demographics, background information and symptoms using structured IPFM
online form. Patients entering the unit in an ambulance or with need of immediate care of
healthcare professionals due to severe and acute conditions are referred similar to normal
process to ensure the patient safety. Results obtained from IPFM are blinded from the
healthcare professional and IPFM does not affect professional's clinical decision making in
any way. The data obtained from IPFM online form and clinical data from the emergency
department and KUH will be analysed after the data collection.
The main aim of the research is to validate the use of IPFM by evaluating the association of
IPFM output with 1) urgency and severity of the conditions (using Emergency Severity Index
[ESI], an international triaging protocol for emergency units, and an assessment by triage
nurse); and 2) actual diagnoses diagnosed by medical doctors. The main hypotheses of the
research are that 1) IPFM is safe and sensitive in evaluating the urgency of the conditions
of arriving patients at the emergency department and that 2) IPFM has sufficient correlation
of differential diagnosis with actual diagnosis made by medical doctor.
Description:
1. Background
Continuous development of digital technology provides new opportunities also in
healthcare area. As healthcare costs are constantly growing both worldwide and in
Finland (1), innovations are needed to enhance allocation of limited healthcare
resources and to reduce economic burden. Digital health technologies (DHT) have the
potential to reduce these costs. They are increasingly developed to support healthcare
systems around the world. DHT are, however, often lacking evidence-based medicine (2)
and both the importance and the lack of their validation has been acknowledged widely
(3, 4).
Work in triage and urgent care centers is highly demanding, with considerable time
pressure. When operating with high patient volumes, acute conditions and severe stress,
human errors are likely to result at least at some extent. Physicians have been found to
have a ~5 % diagnostic error rate (5) with half of these potentially harmful (6). DHT
have been suggested to have a significant role helping both patient management and
triage (7), and thus, reducing also potential human errors. For instance, crude online
clinical decision support tool has been found to reliably demonstrate emergency triage
severity in experimental setting (7).
The validation of DHT and decision support tools is highly crucial. Tools with false
negative rates could have serious consequences when ignoring severe conditions such as
cardiac ischaemia (4). DHT should be thorough and optimal in the medical field and their
internal and external validity should have been evaluated. Recently, guidelines for
evaluating DHT have been published (4). According to guidelines, the first step in the
assessment is to show the results in an early observational study in clinical setting.
In addition to safety, also efficacy and effectiveness of DHT should be evaluated.
To meet these needs, Intelligent Patient Flow Management system (IPFM) has been
developed to help the user (i.e. patient) and the healthcare professionals to evaluate
user's condition and its severity. IPFM enables semi-automated triaging of patients and
enhances the management of patient inflow and demand. In addition, IPFM provides written
and precise information for the healthcare professionals communicated by the patients
themselves of their conditions and symptoms. The core of IPFM is a broad medical
database generated by medical doctors, combined with artificial intelligence (AI) and
feedback loops for accuracy improvement. In other words, IPFM is an AI-induced triage
and decision support tool developed and monitored by medical doctors to help healthcare
professionals to evaluate the urgency of the patients. It evaluates user's condition and
infers the urgency using user demographics, background information and symptoms.
The IPFM is developed by a healthcare technology company Klinik Healthcare Solutions Oy
and it has already been implemented in over 400 primary care and dental facilities in
the Nordics and was recently launched in the United Kingdom and Portugal. It has been
previously shown to reduce the costs in primary care by 14 % in overall total service
costs of the patients by streamlining the patient flow (8). IPFM is currently classified
and registered as a CE class 1 medical device, which requires continuous scientific
validation research and systematic development and quality management processes. The
system is used according to its intended use in this study and therefore the study does
not require permission or registration for FIMEA's clinical device study register.
Klinik's IPFM (product names Klinik Access, Klinik Frontline) provides a full-stack
solution, with interface for patient input, backend for database and algorithm
calculations and interface for handling of case outputs for triaging and combined
statistics for healthcare staff. IPFM uses medically golden standard Bayesian
methodology for inferring the effect of clinical features on the probabilities of the
conditions. The severity and the urgency of the condition is inferred using specific
severity symptoms and by setting a threshold for the probabilities of relevant
conditions. All inflow of data is dynamically gathered and analysed ensemble and
visualised, which can be used by (management) staff to monitor and optimally meet demand
ad hoc.
2. Aims and objectives
The main aim of this study is to validate the use of IPFM in a hospital setting by
evaluating the association of IPFM output with 1) clinical urgency and severity of the
conditions (using Emergency Severity Index [ESI], an international triaging protocol for
emergency units, and an assessment by triage nurse); and 2) actual diagnoses made by the
hospital doctors. The objective is also to assess the correlation of IPFM output with
redirection or referral to various specialties (e.g. internist or surgeon).
Main hypotheses are:
- IPFM is safe (< 1 % emergency cases missed) and accurate (> 80 % agreement with
triaging nurse) in evaluating the urgency of the conditions of arriving patients at
the emergency department
- IPFM has sufficient correlation (> 60 % overlap) of differential diagnosis with
actual diagnosis diagnosed by medical doctor
Validating IPFM, an AI-induced triage support and decision support tool, in real-life
environment and in hospital setting is essential for its use in hospitals and clinical
work in the future to support evidence-based medicine. An accurate and efficient tool
will decrease delays of treatment of urgent conditions and has the potential to decrease
human errors that are inevitable in environment with high patient volumes, acute
conditions and severe stress. When in full implementation, it can also help to optimize
emergency department resources and to target them more accurately. In addition, this
study provides important information on patient history and symptoms provided by
patients themselves when entering emergency department compared to clinical data.
3. Materials and methods
The study population consists of patients entering the emergency department of Kuopio
University Hospital (KUH) during a 2-week research period. The calculated needed sample
size is a minimum of 246 patients (confidence level 95%, margin of error 5%, study
population proportion 80%). The data consists of registry data from the electronic
health records, other quantitative data (such as extracts from KUH emergency department
reporting software) and qualitative data acquired through observation. Both qualitative
and statistical methods will be applied to the data to infer outcomes.
Case-specific information in the IPFM (demographics, background information, symptoms)
provided by the patient will be analyzed with clinical data (Emergency Severity Class
index (ESI), triage category, diagnoses) collected from the electronic patient records
of the hospital (emergency and other possible departments) and IPFM professional
dashboard, and the safety, specificity and sensitivity of IPFM output are analyzed.
The urgency of the condition is evaluated by IPFM and the analyses with clinical data
and validation will be performed after the data collection. They urgency is evaluated by
IPFM as: 1. suitable for self-care if no prescription medication or other treatment is
suggested (i.e. doesn't require doctor's evaluation, however might benefit from nurse
check-up and advice); 2. non-urgent if ailment is prolonged or chronic, but benefits
from doctor's evaluation and would more efficiently be managed through non-urgent
appointment (> 3 days); 3. urgent if ailment is (semi-)acute and benefits from doctor's
evaluation within 1-3 days; and 4. emergency if ailment requires evaluation and
treatment within the same day (or immediately) to avoid risk of serious health hazard.
This output is correlated with the ESI classification used by KUH emergency and other
data collected from electronic health records regarding urgency of the condition.
When attending to the hospital, patients will report their demographics, background
information, symptoms, other case specific data and satisfaction using structured IPFM
online form. Patients entering the unit in an ambulance or with need of immediate care
due to severe and acute conditions are managed through traditional process: they do not
fill the form and, hence, are out of scope of the pre-evaluation process and this study.
All information gathered is transferred to the IPFM dashboard, where the patient
reported clinical information, AI-induced suggestions on the urgency of the care needed
and potential diagnosis (a differential of 3-10 probable diagnosis) is demonstrated.
However, in this study design, healthcare professionals cannot see any clinical
information in the IPFM dashboard, i.e. the output of IPFM is blinded.
Patient data will be saved to the registry from IPFM: 1) age; 2) gender; 3) duration of
the symptoms; 4) body location of the symptoms; 5) symptoms chosen from the symptom
cloud(s) by the patient (symptom cloud is dynamic, consist of 430 potential symptoms,
symptom selection varies depending on the previous actions of the patient); 6) severity
symptoms chosen from the severity symptom cloud (severity symptom cloud is dynamic,
consist of 100 potential severity symptoms, selection varies depending on the previous
actions of the patient); 7) free text fields regarding the selected symptoms; 8) answers
(yes/no) to the questions: a) have you used any medication or other treatment for your
ailment, b) has someone (e.g. a physician or a physiotherapist) already examined or
treated your ailment; 9) identification number of the form; 10) unique case id number;
11) name; 12) social security number; and 13) telephone number.
After the primary IPFM data collection stage at the beginning of each ER visit, a
secondary stage of data retrieval is initiated after the study period. During this only
critical information required to assess the validity of the IPFM system outcomes will be
fetched from the electronic health care system: 1) ESI classification; 2)
diagnosis/diagnoses of the patient; 3) speciality & care path chosen in the urgency
care; 4) traditional non-ESI urgency assessment comparable to our urgency scale (if
stored). The final data points will be equivalent to these, but might due to changes in
ER practices and/or data systems vary. The data points are chosen to optimally allow
comparative analysis of results. This information is only used after data collection
when evaluating the validity of IPFM.
Despite excluding patients with need of acute immediate care of healthcare professionals
(arriving with an ambulance), also children/adolescents and patients with otherwise
restricted capabilities or developmental disorders are excluded from the study. Adult
patients with mild mental disorders can be included, excluding the patients with
guardian. Additionally, pregnant or breastfeeding patients are excluded from the study.
4. Ethical aspects of the research
Participation to the study is voluntary for the patients. After receiving oral and
written information on location at KUH emergency department, patients will sign an
informed consent form. As described in the chapter 5, the study protocol will not harm
any study subjects and will not interfere negatively in their treatment. The patients
participate in the study only once, on location when arriving to the emergency
department, and thus no further activity nor financial burden is generated.
An independent research data registry will be generated for this study. When creating
the registry, social security number or telephone number is used to merge the data from
electronic health records and IPFM. After the data has been merged and the combined
registry is created, randomly generated case ids are used as research subject
identifiers and information that allows for identification of single patients is not
available. The merging of the data is done by the KYS staff prior to access into
research registry being granted for the research group. The combined research registry
will be stored in a digital format, on a computer owned only by the research group
members. There will be a password protection for a user to enter a) the computer and b)
the registry file itself. The file will be stored on a local folder not accessed online
(i.e. not a cloud storage folder). The registry file will be stored for five (5) years
after the research has deemed finished and will be deleted after the period according to
the required handling standards. The administrator of the case ids is the person
responsible of the project and Klinik Healthcare Solutions Oy is responsible for storing
the registry file. The study will be registered into international "ClinicalTrials.gov"
research register. The results of this project will be reported in a group level and,
thus, no individual information can be recognized.
The application for the ethical committee will be submitted in May 2020.
5. Risk assessment
The healthcare professionals are evaluating the patients by their own clinical judgement
and they do not utilize output of IPFM in their decision making. IPFM data is only
assessed after the data collection.
There is a potential risk of acquiring volunteers below the intended volume for the
study. This will be addressed by extending the study period or overlap of daily working
hours.
There is a hypothetical risk of causing undesired delay in the registration process of
the patients that would cause critical cases to be missed due to increased waiting times
for the registration. Our previous user studies have shown that filling out the IPFM
form takes circa 3-5 minutes, which would not be a life threatening delay for walk-in
patients. This risk will be monitored and if noticed, managed in real time.
6. Time schedule and funding plans
This study is initially planned to be executed between 1.7.2020 and 30.6.2023. Duration
of the data collection is approximately 2 weeks and is designed to complete during fall
2020.
The project is anticipated to have a small scale economic effect for the parties. The
patients in this study will not experience any economic harm. The service provider
Klinik Healthcare Solutions Oy will cover for the costs of it's researchers and
assisting staff in the form of a moderate hourly pay. The service provider will also
provide the digital equipment used in the research in collecting the information from
the patients. In the approval of the equipment it will work closely with the ICT
department of Kuopio University Hospital District. The hospital or the emergency
department will not have any additional costs regarding the research project. There are
no additional economical compensations to the researchers or to the study population.
7. Research group, resources and conflicts of interest
The research group consists of MD, PhD Juhani Määttä (Medical Research Center Oulu,
University of Oulu), MD, PhD candidate Petteri Hirvonen (University of Helsinki), MD
Rony Lindell (University of Helsinki) and PhD Tero Martikainen, chief doctor of Kuopio
University Hospital A&E department.
Juhani Määttä, Petteri Hirvonen and Rony Lindell have financial and business interests
in Klinik Healthcare Solutions Oy.
8. Expected results and scientific impact
The study is expected to validate high accuracy (i.e. safe and reliable usability) of IPFM
tool in KUH hospital context. The results are generalizable to other hospital emergency
departments, however with caution by doing contextual analysis and process comparisons prior
to implementation considerations. Through assessing IPFM output with data from patient's
electronic health records, the study is expected to show: 1) strong correlation of the
urgency category; and 2) good correlation between calculated differential diagnosis and the
diagnoses of ED or ward doctors. The purpose is to publish the results in internationally
recognized medical journals with high quality.
According to our review of the literature on this subject, an AI-supported automation of the
inbound patient traffic in an emergency department setting has not been studied previously,
especially not in this scale and patient variation. An increasing public and professional
interest is arising regarding AI-based solutions to support professionals in their work and
decision making. To meet these needs, this study provides and scientifically validates a
solution which works semi-independently, taking up initial tasks normally reserved for
healthcare professionals. The research group anticipates this being of great interest to the
scientific community - and will add to the discussion of the possibilities of the DHT to
provide marked cost-effectiveness while not compromising on patient safety and effectiveness
of the use of professional resources.
