Evaluation of the Artificial Intelligence-based Prescription Support Software iAST® for the Choice of Empirical and Semi-targeted Antibiotic Treatment

Bacterial Infections Clinical Trial

— EVIAST  

Official title:

Evaluation of the Artificial Intelligence-based Prescription Support Software iAST® for the Choice of Empirical and Semi-targeted Antibiotic Treatment

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT06174519
• Other study ID #: EVIAST
• Status: Completed
• Start date: August 1, 2023
• Est. completion date: December 1, 2023

Study information

• Verified date: December 2023
• Source: Pragmatech AI Solutions
• Contact: n/a
• Is FDA regulated: No
• Study type: Observational

Clinical Trial Summary

Inadequate treatment of infections frequently leads to complications that cause new visits to the doctor, lengthen hospital stays and can lead to sepsis, even causing the death of affected patients. Several scientific studies have documented that up to 20%-30% of antibiotic prescriptions are incorrect and do not cover the microorganism causing the infection. iAST® is a simple antibiotic prescribing aid tool that applies complex algorithms based on the latest artificial intelligence technologies to accurately predict the best specific antibiotic for a patient, before knowing the definitive microbiological results (bacterial identification and antibiogram). The objective of the present trial is to demonstrate the non-inferiority of iAST® with respect to physicians for the appropriate choice of empiric and semi-directed therapy of common infectious diseases, including sepsis, urinary tract infections and ventilator-associated pneumonias or tracheobronchitis. The adequacy of the medical prescription and the iAST® prediction will be compared taking the antibiogram report as a reference. The study design is retrospective, so that no intervention will be done on the patients. The investigators will conduct a retrospective search for infection cases and note the antibiotic treatment prescribed by the doctors. In parallel, they will enter basic patient data such as age, sex, service where they were treated, type of infection and microorganism (in the case of semi-directed treatment evaluation) into the iAST® software and will write down the first three treatment options recommended by the tool. The treatments of both arms (medical treatment and iAST® prediction) will be compared with the microbiological results and the success rate of each of them will be calculated.

Description:

Background:

Infections are one of the main causes of consultation in primary care and emergency services. In addition, a high percentage of the patients admitted to hospitals suffer from an infection during their stays. According to data from the European Center for Disease Prevention and Control (ECDC), approximately 11% of patients admitted to European hospitals suffer from a healthcare-associated infection. Moreover, according to this organization, 35% of patients admitted to European hospitals are under antibiotic therapy, with this percentage varying between 21.4% and 54.7% depending on the hospital and the country.

Inadequate treatment of infections often leads to complications associated with an extension in hospitalization periods or sepsis development, which finally could cause the death of the affected patients. Moreover, ineffective treatments due to an inappropriate antibiotic selection have an enormous cost and impact to health care systems. Conversely, there is extensive scientific evidence that early initiation of adequate antibiotic treatment greatly reduces the morbidity and mortality of infections and significantly reduces patient hospital stays. Previous studies have reported that 20-30% of antibiotic prescriptions are inadequate, leading to health complications, especially health care associated infections. Moreover, an adequate selection of antibiotic treatment avoids the spread of resistant bacteria strains, which has become an increasing problem in recent years.

As previously noted, infectious mortality increases enormously over time if an adequate antibiotic treatment is not initiated. Thus, obtaining microbiological results and identify the bacteria strain causing the infection is crucial to provide effective treatments to the subjects. The microbiological profile description is normally performed by microbiology laboratories, which support the doctors in the antibiotic treatment selection. Nevertheless, although reliable results are generated, there are usually obtained 48 hours after the initial patient evaluation.

Rationale:

Cumulative antibiogram data from multiple microorganisms and patients have great epidemiological and clinical value, since they allow monitoring and detecting variations in antimicrobial susceptibility trends. Besides, this data can also help to select the best empiric therapies from the different infectious syndromes. Clinical microbiologists traditionally carry out cumulative antibiogram reports of with a certain frequency (most times annually). These reports are made by selecting the available data for each antibiotic and each microorganism, counting the susceptible and resistant bacteria and calculating the percentage of sensitivity for each of them. However, these cumulative antibiograms are rarely consulted in real life by prescribing doctors and could be biased. For instance, sensitivity varies depending on the characteristics of the patients, the ward where they are treated, previous bacterial cultures, etc...

Due to the available technology in current microbiology, there is often a 24-hour lag between the identification of a microorganism from a clinical sample and the result of its antibiotic susceptibility profile. As soon as the bacterial identification is available, a "semi-targeted" treatment oriented to the specific pathogen can be established, which, with the help of the accumulated antibiograms reports, allows to initiate a prompt accurate treatment until the definitive antibiogram is known, significantly reducing the degree of empiricism with which treats infectious diseases.

Cumulative antibiograms data can be analyzed to extract behavioral patterns using machine learning techniques. Machine learning is a part of artificial intelligence focused on developing models based on data, applying techniques that allow computers to automatically learn the knowledge implicit in the data, detect patterns, transform data into predictive models, that help in decision making. In the last few years, some researchers have published works in which they evaluated the use of machine learning techniques for empiric susceptibility/resistance prediction. However, these works were evaluated in a research environment and were reduced to specific cases of a few infections and etiological agents.

Medical device overview:

In the last three years, Pragmatech AI Solutions has focused its research on the use of artificial intelligence and machine learning techniques to analyze cumulative antibiograms and to predict what are the best empiric and semi-targeted therapies for specific patients. This has materialized in a product that is in the pre-marketing phase called iAST®. The iAST® software is classified as a Class IIa medical device according to the European Union regulation 2017/745.

Study objective:

The iAST® tool have mathematically demonstrated that it can increase the probability of bacterial coverage in the early antibiotic management of infections in patients treated in hospitals.

In this sense, the primary objective of the present retrospective and observational study is the assessment of the accuracy of the iAST® software for the early adequate choice of the empiric and semi-targeted therapy of common infectious diseases in a real clinical setting.

Clinical Investigation Design:

The clinical investigation has been designed as a single center, retrospective and observational study. Antibiograms and hospital records will be retrospectively reviewed to identify patients who meet the inclusion criteria to analyze the primary and secondary endpoints of the study. Antibiograms that were used to set-up the artificial intelligence model will not be considered to assess the accuracy of the iAST® software. Only cases from February 2023 will be included. For each case, investigators will register the empirical and semi-targeted therapy that the physician prescribed (if applicable). The comparison of the success rate of both the iAST® software and the physicians with respect to the antibiogram will be carried out in two ways:

1. For the eligible subjects, investigators will use the iAST® tool to predict which antibiotics would have been recommended as the top three choices both for empiric and semi-targeted therapy and will record this data with the percentage of coverage predicted (the first three antibiotics in the iAST® ranking that have been tested in the antibiogram of the center where the study is carried out will be chosen). Investigators will check the final microbiological reports and will log if the recovered bacteria were susceptible to the drug prescribed by doctors and simulated by the iAST® tool according to the final antibiogram results. The rate of success of the physician prescription will be statistically compared with rate of success of the three top antibiotics recommended by iAST® in an independent way. In those cases in which clinicians did not prescribed a semi-targeted therapy, the drug recommended by iAST® as semi-targeted therapy will be compared with the clinician's empirical treatment.

2. Investigators will also register the probability rate of success that iAST® predicted for the drug prescribed by the physician. The mean of probabilities predicted by iAST® for the antibiotic that the doctor prescribed will be compared between two groups: the cases in which the bacterium was susceptible and the cases in which it was resistant, according to the antibiogram results. It will be analyzed if this difference is statistically significant.

The appropriateness of antibiotic prescription of physicians and the iAST® prediction will be compared taking the antibiogram report as a reference. A subgroup analyses will be developed for each type of infection. According to this, the accuracy of iAST® for the hypothetical empiric and semi-targeted treatment prediction in comparison of doctors prescriptions will be evaluated through:

• Data from patients who attended to the Emergency Department with a diagnosis of UTI and/or urinary sepsis.

• Data from patients who suffered bacteremia or sepsis, when the causal agent had been definitively identified or when the characteristics of the Gram stain were available.

• Data from patients who had suffered tracheobronchitis or pneumonia associated with mechanical ventilation in whom the causal agent had been definitively identified or when the characteristics of its Gram stain were available.

• Data from patients who had suffered any other infection in which the causal agent had been identified but for which there still was no result of an antibiogram.

Investigators will register whether the antibiotics that the doctors prescribed or iAST® predicted in each case belonged to the access, watch or reserve groups of the World Health Organization AWARE classification. In this way, the rate of use/recommendation of each of these antibiotics in each arm (physician prescription vs iAST® prediction) will be calculated.

All data from the study will be registered by the investigators in an electronic Case Report Form (eCRF).

Definitions

• Empiric therapy: Those therapies which are prescribed when an infection is suspected but the microorganism that produces it and its antibiotic susceptibility is yet unknown.

• Semi-targeted therapy: Those therapy which is prescribed when an infection is suspected, the etiology is known but not yet its antibiotic susceptibility.

• Adequate antibiotic according to microbiological results: An antibiotic treatment will be considered correct if the bacteria identified in the clinical sample of the patient to whom it is directed is susceptible according to the result of the antibiogram or an extrapolation thereof.

• Inadequate antibiotic according to microbiological results: On the contrary, a treatment will be considered incorrect if the bacteria against which it is directed is resistant according to the results of the antibiogram or its intrinsic resistance.

Limitations:

The limitations of the study are those stemming from its design, given that it is a retrospective, non-interventional study. However, the study has been designed in order to reduce possible biases. Some of the measures aimed to bias minimization are:

• the same cases will be used to compare the physician prescription with the iAST® recommendation, so there will be no data imbalance.

• investigators are highly qualified in infectious diseases, so they will perfectly interpret the data from antibiograms and the clinical records of the cases included in the study, thus reducing interpretation bias.

• patient data will be anonymized The data to be collected is usually available in the patients' medical records and medical charts and in the Microbiology Department's data base. However, it cannot be guaranteed that the medical records or medical charts are complete, which may hinder to collect all the variables of interest specified in the study protocol. In case of missing data, missing values will not be considered to calculate percentages for the analysis.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 325
• Est. completion date: December 1, 2023
• Est. primary completion date: December 1, 2023
• Accepts healthy volunteers: No
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

1. Data for analysis should proceed from subjects over 18 years old that were admitted into HM Hospitals from 01Feb2023.

2. Subjects who:

• have attended the Emergency Department of the hospital with suspected urinary tract infection (UTI) or;

• have presented an episode of bacteremia/sepsis at/during hospital admission or;

• have been admitted to the hospital ICU and presented a tracheobronchitis or pneumonia associated with mechanical ventilation or;

• have presented another type of infection, were treated and which have a bacterium identified with an antibiogram result.

Exclusion Criteria:

1. Patients with concomitant infections.

2. Data from subjects suffered from infections with no bacterial etiology: fungal or viral infections.

3. Data from subjects with infections without microbiological documentation (including antibiogram results).

4. Data from subjects prescribed with more than one antibiotic.

Related Conditions & MeSH terms

• Bacterial Infections
• Infections

Intervention

Device:
• Medical device simulation
For the subjects included, investigators used the iAST® tool to predict which antibiotics would have been recommended as the top three choices both for empiric and semi-targeted therapy and recorded this data with the percentage of coverage predicted (the first three antibiotics in the iAST® ranking that have been tested in the antibiogram of the center where the study was carried were chosen). Investigators checked the final microbiological reports and logged if the recovered bacteria were susceptible to the drug prescribed by doctors and simulated by the iAST® tool according to the final antibiogram results.

Locations

Country	Name	City	State
Spain	Grupo HM Hospitales	Madrid

Sponsors (3)

Lead Sponsor	Collaborator
Pragmatech AI Solutions	Grupo Hospital de Madrid, NAMSA

Country where clinical trial is conducted

Spain, 

References & Publications (19)

Anahtar MN, Yang JH, Kanjilal S. Applications of Machine Learning to the Problem of Antimicrobial Resistance: an Emerging Model for Translational Research. J Clin Microbiol. 2021 Jun 18;59(7):e0126020. doi: 10.1128/JCM.01260-20. Epub 2021 Jun 18. — View Citation

Andersson DI, Hughes D. Antibiotic resistance and its cost: is it possible to reverse resistance? Nat Rev Microbiol. 2010 Apr;8(4):260-71. doi: 10.1038/nrmicro2319. Epub 2010 Mar 8. — View Citation

D'Onofrio V, Salimans L, Bedenic B, Cartuyvels R, Barisic I, Gyssens IC. The Clinical Impact of Rapid Molecular Microbiological Diagnostics for Pathogen and Resistance Gene Identification in Patients With Sepsis: A Systematic Review. Open Forum Infect Dis. 2020 Aug 13;7(10):ofaa352. doi: 10.1093/ofid/ofaa352. eCollection 2020 Oct. — View Citation

Fernandez J, Vazquez F. The Importance of Cumulative Antibiograms in Diagnostic Stewardship. Clin Infect Dis. 2019 Aug 30;69(6):1086-1087. doi: 10.1093/cid/ciz082. No abstract available. — View Citation

Fleming-Dutra KE, Hersh AL, Shapiro DJ, Bartoces M, Enns EA, File TM Jr, Finkelstein JA, Gerber JS, Hyun DY, Linder JA, Lynfield R, Margolis DJ, May LS, Merenstein D, Metlay JP, Newland JG, Piccirillo JF, Roberts RM, Sanchez GV, Suda KJ, Thomas A, Woo TM, Zetts RM, Hicks LA. Prevalence of Inappropriate Antibiotic Prescriptions Among US Ambulatory Care Visits, 2010-2011. JAMA. 2016 May 3;315(17):1864-73. doi: 10.1001/jama.2016.4151. — View Citation

Gandra S, Barter DM, Laxminarayan R. Economic burden of antibiotic resistance: how much do we really know? Clin Microbiol Infect. 2014 Oct;20(10):973-80. doi: 10.1111/1469-0691.12798. Epub 2014 Nov 7. — View Citation

Jorgensen JH, Ferraro MJ. Antimicrobial susceptibility testing: a review of general principles and contemporary practices. Clin Infect Dis. 2009 Dec 1;49(11):1749-55. doi: 10.1086/647952. — View Citation

Kollef MH, Sherman G, Ward S, Fraser VJ. Inadequate antimicrobial treatment of infections: a risk factor for hospital mortality among critically ill patients. Chest. 1999 Feb;115(2):462-74. doi: 10.1378/chest.115.2.462. — View Citation

Kumar A, Roberts D, Wood KE, Light B, Parrillo JE, Sharma S, Suppes R, Feinstein D, Zanotti S, Taiberg L, Gurka D, Kumar A, Cheang M. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock. Crit Care Med. 2006 Jun;34(6):1589-96. doi: 10.1097/01.CCM.0000217961.75225.E9. — View Citation

Larrosa MN, Canut-Blasco A, Benito N, Canton R, Cercenado E, Docobo-Perez F, Fernandez-Cuenca F, Fernandez-Dominguez J, Guinea J, Lopez-Navas A, Moreno MA, Morosini MI, Navarro F, Martinez-Martinez L, Oliver A. Spanish Antibiogram Committee (COESANT) recommendations for cumulative antibiogram reports. Enferm Infecc Microbiol Clin (Engl Ed). 2023 Aug-Sep;41(7):430-435. doi: 10.1016/j.eimce.2022.09.002. Epub 2022 Sep 26. — View Citation

Leekha S, Terrell CL, Edson RS. General principles of antimicrobial therapy. Mayo Clin Proc. 2011 Feb;86(2):156-67. doi: 10.4065/mcp.2010.0639. — View Citation

Lewin-Epstein O, Baruch S, Hadany L, Stein GY, Obolski U. Predicting Antibiotic Resistance in Hospitalized Patients by Applying Machine Learning to Electronic Medical Records. Clin Infect Dis. 2021 Jun 1;72(11):e848-e855. doi: 10.1093/cid/ciaa1576. — View Citation

Livermore DM. Minimising antibiotic resistance. Lancet Infect Dis. 2005 Jul;5(7):450-9. doi: 10.1016/S1473-3099(05)70166-3. — View Citation

Mancini A, Vito L, Marcelli E, Piangerelli M, De Leone R, Pucciarelli S, Merelli E. Machine learning models predicting multidrug resistant urinary tract infections using "DsaaS". BMC Bioinformatics. 2020 Aug 21;21(Suppl 10):347. doi: 10.1186/s12859-020-03566-7. — View Citation

Moehring RW, Hazen KC, Hawkins MR, Drew RH, Sexton DJ, Anderson DJ. Challenges in Preparation of Cumulative Antibiogram Reports for Community Hospitals. J Clin Microbiol. 2015 Sep;53(9):2977-82. doi: 10.1128/JCM.01077-15. Epub 2015 Jul 15. — View Citation

Rowe M. An Introduction to Machine Learning for Clinicians. Acad Med. 2019 Oct;94(10):1433-1436. doi: 10.1097/ACM.0000000000002792. — View Citation

Tumbarello M, Sanguinetti M, Montuori E, Trecarichi EM, Posteraro B, Fiori B, Citton R, D'Inzeo T, Fadda G, Cauda R, Spanu T. Predictors of mortality in patients with bloodstream infections caused by extended-spectrum-beta-lactamase-producing Enterobacteriaceae: importance of inadequate initial antimicrobial treatment. Antimicrob Agents Chemother. 2007 Jun;51(6):1987-94. doi: 10.1128/AAC.01509-06. Epub 2007 Mar 26. Erratum In: Antimicrob Agents Chemother. 2007 Sep;51(9):3469. — View Citation

van den Bosch CM, Hulscher ME, Akkermans RP, Wille J, Geerlings SE, Prins JM. Appropriate antibiotic use reduces length of hospital stay. J Antimicrob Chemother. 2017 Mar 1;72(3):923-932. doi: 10.1093/jac/dkw469. — View Citation

Zilberberg MD, Nathanson BH, Sulham K, Fan W, Shorr AF. 30-day readmission, antibiotics costs and costs of delay to adequate treatment of Enterobacteriaceae UTI, pneumonia, and sepsis: a retrospective cohort study. Antimicrob Resist Infect Control. 2017 Dec 6;6:124. doi: 10.1186/s13756-017-0286-9. eCollection 2017. — View Citation

* Note: There are 19 references in all — Click here to view all references

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	To demonstrate the non-inferiority of iAST® compared to physicians for the prescription of the empiric and semitargeted antibiotic therapy in patients with common infectious diseases.	The appropriateness of antibiotic prescription and the iAST® prediction will be compared with the results from the antibiogram report as standard.Two-sided 95% confidence intervals (CIs) for the difference between treatments will be calculated using the unstratified method of Miettinen and Nurminen. The demonstration of non-inferiority of iAST® to doctor prescription for both primary and secondary efficacy endpoints will be established if the lower limit of the two-sided 95% CI for the treatment difference exceeded 5%. Additionally, a p-value will be computed for the corresponding one-sided non-inferiority hypothesis test.	4 months
Secondary	To assess the accuracy in the antibiotic prescription from the physicians and the software iAST® predictions (for empiric and semitargeted therapy) compared to the antibiogram report, respectively.	The success rate of each arm will be measured, with respect to the antibiogram.	4 months
Secondary	To evaluate the software iAST® accuracy in the antibiotic prediction of the 4 study population subgroups compared to the antibiogram report as standard.	Subgroups are: UTI, bacteremia/sepsis, pneumonia/tracheobronchitis and other infections	4 months
Secondary	To compare the rate of used/recommended antibiotics from the Access, Watch and Reserve antibiotics list (from the WHO Aware classification), between the prescriptions from physicians and the iAST® software predictions.	The use of each antibiotic in the Aware classification group and the impact of the software in terms of antibiotic stewardship will be measured.	4 months
Secondary	To collect information related to user experience by completing a usability questionnaire by physicians when working with the software iAST®.	Data about the usability of the software will be collected in order to have feedback about it.	4 months