A1Check: the External Validation of a Machine Learning Model Predicting Colorectal Anastomotic Leakage

Anastomotic Leak Clinical Trial

— A1Check  

Official title:

The External Validation of a Machine Learning Model Predicting Anastomotic Leakage Intraoperatively in Patients Undergoing a Colorectal Resection - A1Check Study: Protocol for a Multicenter Observational Study

Clinical Trial Details — Status: Recruiting

Administrative data

• NCT number: NCT05810207
• Other study ID #: 2021.0626
• Status: Recruiting
• Start date: February 1, 2022
• Est. completion date: December 31, 2024

Study information

• Verified date: April 2023
• Source: Amsterdam UMC, location VUmc
• Contact: Erik W. Ingwersen, MD
• Phone: 0031623028928
• Email: e.ingwersen[@]amsterdamumc.nl
• Is FDA regulated: No
• Study type: Observational [Patient Registry]

Clinical Trial Summary

Anastomotic leakage is a severe complication that can arise following a colorectal resection. It impairs both the short- and long-term outcomes, and negatively influences cancer recurrence rates. Its detrimental effects resound in healthcare costs of a patient after anastomotic leakage, €71,978, versus patients with an uncomplicated course, €17,647. Despite multiple innovations within the field of colorectal surgery, the incidence of colorectal anastomotic leakage did not reduce in the past decade. Mitigation strategies such as prehabilitation, intraoperative optimization, selective bowel decontamination, and reconstruction techniques are promising but do not completely eliminate the risk of leakage. The only true prevention of colorectal anastomotic leakage is the omission of an anastomosis and implies an ostomy, which in itself has a negative impact on the quality of life. A stoma is associated with stoma-related morbidity and should, therefore, be avoided in patients who do not need it. Predicting anastomotic leakage intra-operatively, just before the construction of the anastomosis, may offer a solution. A stoma will then only be constructed in those at high risk of anastomotic leakage. Currently, there are prediction models for anastomotic leakage based on conventional multivariate logistic regression analysis, however, these are not useful for clinical practice due to suboptimal results. Machine learning algorithms, on the other hand, take well into account the multifactorial nature of complications and might thus be able to predict anastomotic leakage more accurately. The machine learning model we created proved to be well capable of making accurate predictions. This model was developed based on a database containing both pre- and intra-operative data from 2,483 patients. Before these models can be used in daily practice, external validation is essential. Our models should be tested on unseen data from patients treated in centers that were not previously involved in the database that was used to train the model in order to achieve high reproducibility. Our hypothesis is that with our model, we can accurately predict anastomotic leakage intra-operatively during colorectal surgery.

Description:

Study procedure The aim of the study is to externally validate a machine learning model predicting colorectal anastomotic leakage. The prediction model that will be externally validated, is developed on a prospective database. This database contained data of 2,483 colorectal cancer patients who underwent a surgical procedure between January 2016 and April 2021 in 14 hospitals, both rural and academic in four different countries (the Netherlands, Italy, Belgium, Australia). Some 189 patients (7.6%) developed colorectal anastomotic leakage. The models predicted risk of colorectal anastomotic leakage intraoperatively, just prior to the construction of the anastomosis, using a total of 31 variables. These variables contain both preoperatively available data and the variables regarding the intraoperative condition of the patient. The models were internally validated using 10-fold cross validation and subsequently tested on 20% of unseen data of the database. The area under the curve - receiver operating characteristics (AUROC) of the best performing machine learning model on the test set was 0.84, with a sensitivity of 0.86, specificity of 0.78, a positive predictive value of 0.24 and a negative predictive value of 0.99.

During this prospective simulation study there are no direct benefits or risks for participating patients. This prospective simulation study will be non-interventional, the prediction models do not alter the original daily practice and in this phase, it is not intended to be used as a diagnostic device. Intraoperatively, just prior to the construction of the anastomosis, the prediction model will predict, using patient, tumor, and intraoperatively variables (listed in the Data Dictionary paragraph), the probability of anastomotic leakage. SAS Viya is used for development of the machine learning model. During the prospective simulation study, the scores of these predictions are only available to the principle and research investigators, and thus unknown to the participating hospitals or operating surgeons in order to prevent any influence on current daily practice in this stage of the research. Thirty days postoperatively, data of the patients regarding the occurrence of anastomotic leakage will be collected. AUROC, sensitivity, specificity, and accuracy then will be calculated based on the number of patients assessed as true positive, true negative, false positive or false negative. After a minimum of 100 events and 100 non-events, the external validation is completed and the final AUROC, sensitivity and specificity scores will be presented.

Quality assurance plan, data checks, source data verification Data will be handled confidentially and anonymously. Data will be pseudo-anonymized for the principal investigator and the research investigators. Pseudo-anonymized data are entered in a Castor database. A data dictionary is attached to the original dataset with metadata to describe the data. All participating hospitals have a Data Sharing Agreement to safely share data of included patients with the principal investigator and the research investigators. A data management plan will be created according to our institute's polices with the assistance of a data management expert, along with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines.The characteristics of the collected and generated data is clinical data extracted from the electronic health records. This contains continuous, nominal, and dichotomous variables. Data will not be reused or coupled to existing data. Informed consent of patients is necessary to predict the outcome using the developed model. Privacy policies and laws are applicable to this project. The project will also comply with all data protection principles as is defined in the General Data Protection Regulation. The anonymized dataset can be accessed via a Castor database. Long term data will be saved in the Amsterdam University Medical Center repository with help of the research data management (RDM) department. The data will be saved for five years after the project has ended.

Data dictionary

The following variables will be collected:

i. Patient and tumor characteristics Age; sex; body mass index; American Society of Anesthesiologists (ASA) classification; intoxications (smoking and/or alcohol consumption); medical history of diabetes; steroid use (not nasal); hemoglobin; benign or malignant disease. If there is malignant disease: TNM-stage, tumor distance from anal verge, neoadjuvant treatment.

ii. Perioperative characteristics Surgical procedure, surgical approach; conversion; occurrence of intraoperative event (hypoxic events, hypercarbia, bradycardia, hypotension, embolism, reanimation, more extensive resection than planned, serosa lesions, bladder and ureteral injuries, intraoperative bleeding, splenectomy) iii. Characteristics just prior to the creation of the anastomosis Patient temperature; time of antibiotic administration; administration of vasopressors; blood loss; O2 saturation; mean arterial pressure; fluid administration; urine production; presence of fecal contamination; subjective assessment of local perfusion; epidural analgesia; dosing movements; time from incision until the creation of the anastomosis, intention to create stoma.

iv. Postoperative characteristics Colorectal anastomotic leakage within 30 days and length of hospital stay.

Standard Operating procedures Patients eligible for inclusion are detected in the first multidisciplinary team meeting. If eligible, the surgeon will inform and discuss this study with the patient in the preoperative consultation for surgery. If the patient consents to participation, written informed consent is required. The patient may withdraw this consent at any time.

Sample size calculation In the participating hospitals, around 100 to 400 colorectal resections are performed annually, with an approximate incidence of anastomotic leakage of 5 to 15%. Multiple studies demonstrated a minimum of 100 events and 100 nonevents as an appropriate sample size for external validation. With an expected total of 1,200 patients included annually and a leakage percentage around 10%, including 100 events takes approximately one to two years.

Handling missing data The machine learning model will make a prediction in patients with more than 80% of the required data available. Missing data are imputed using predictive mean matching with ten iterations.

Statistical analysis plan The external validation will be performed on at least 100 events (anastomotic leakage) and 100 non-events (no anastomotic leakage). The machine learning model with the best predictive performance in terms of AUROC will be used as the implementation model. Colorectal anastomotic leakage rate will be compared in a multivariate logistic regression model. All analyses will be carried out under the supervision of a clinical epidemiologist.

Recruitment information / eligibility

• Status: Recruiting
• Enrollment: 1000
• Est. completion date: December 31, 2024
• Est. primary completion date: July 1, 2024
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

• patients undergoing colorectal resection with the construction of an anastomosis

• patients with the age of 18 years or older

• patients able to give informed consent

Exclusion Criteria:

• when more than 25% of the target variables are missing on which the machine learning model bases the prediction

• non-elective surgeries

Related Conditions & MeSH terms

• Anastomotic Complication
• Anastomotic Leak
• Anastomotic Leak Rectum

Intervention

Procedure:
• Colorectal resection
Patients undergoing a colorectal resection with the construction of a primary anastomosis

Locations

Country	Name	City	State
Netherlands	ZGT	Almelo	Overijssel
Netherlands	Meander MC	Amersfoort	Utrecht
Netherlands	Gelre Ziekenhuis	Apeldoorn	Gelderland
Netherlands	Deventer ziekenhuis	Deventer	Overijssel
Netherlands	Slingeland Ziekenhuis	Doetinchem	Gelderland
Netherlands	Medisch Spectrum Twente	Enschede	Overijssel
Netherlands	Tjongerschans ziekenhuis	Heerenveen	Friesland
Netherlands	Zuyderland MC	Heerlen	Limburg

Sponsors (2)

Lead Sponsor	Collaborator
Freek Daams	SAS Institute

Country where clinical trial is conducted

Netherlands, 

References & Publications (15)

Abis GSA, Stockmann HBAC, Bonjer HJ, van Veenendaal N, van Doorn-Schepens MLM, Budding AE, Wilschut JA, van Egmond M, Oosterling SJ; SELECT trial study group. Randomized clinical trial of selective decontamination of the digestive tract in elective colorectal cancer surgery (SELECT trial). Br J Surg. 2019 Mar;106(4):355-363. doi: 10.1002/bjs.11117. Epub 2019 Feb 25. — View Citation

Branagan G, Finnis D; Wessex Colorectal Cancer Audit Working Group. Prognosis after anastomotic leakage in colorectal surgery. Dis Colon Rectum. 2005 May;48(5):1021-6. doi: 10.1007/s10350-004-0869-4. — View Citation

Bruns ERJ, van Rooijen SJ, Argillander TE, van der Zaag ES, van Grevenstein WMU, van Duijvendijk P, Buskens CJ, Bemelman WA, van Munster BC, Slooter GD, van den Heuvel B. Improving Outcomes in Oncological Colorectal Surgery by Prehabilitation. Am J Phys Med Rehabil. 2019 Mar;98(3):231-238. doi: 10.1097/PHM.0000000000001025. — View Citation

Collins GS, Ogundimu EO, Altman DG. Sample size considerations for the external validation of a multivariable prognostic model: a resampling study. Stat Med. 2016 Jan 30;35(2):214-26. doi: 10.1002/sim.6787. Epub 2015 Nov 9. — View Citation

Fujita S, Teramoto T, Watanabe M, Kodaira S, Kitajima M. Anastomotic leakage after colorectal cancer surgery: a risk factor for recurrence and poor prognosis. Jpn J Clin Oncol. 1993 Oct;23(5):299-302. — View Citation

Huisman DE, Reudink M, van Rooijen SJ, Bootsma BT, van de Brug T, Stens J, Bleeker W, Stassen LPS, Jongen A, Feo CV, Targa S, Komen N, Kroon HM, Sammour T, Lagae EAGL, Talsma AK, Wegdam JA, de Vries Reilingh TS, van Wely B, van Hoogstraten MJ, Sonneveld DJA, Veltkamp SC, Verdaasdonk EGG, Roumen RMH, Slooter GD, Daams F. LekCheck: A Prospective Study to Identify Perioperative Modifiable Risk Factors for Anastomotic Leakage in Colorectal Surgery. Ann Surg. 2022 Jan 1;275(1):e189-e197. doi: 10.1097/SLA.0000000000003853. — View Citation

Ingwersen EW, van der Beek PJK, Dekker JWT, van Dieren S, Daams F. One Decade of Declining Use of Defunctioning Stomas After Rectal Cancer Surgery in the Netherlands: Are We on the Right Track? Dis Colon Rectum. 2023 Jan 6. doi: 10.1097/DCR.0000000000002625. Online ahead of print. — View Citation

Koedam TWA, Bootsma BT, Deijen CL, van de Brug T, Kazemier G, Cuesta MA, Furst A, Lacy AM, Haglind E, Tuynman JB, Daams F, Bonjer HJ; COLOR COLOR II study group. Oncological Outcomes After Anastomotic Leakage After Surgery for Colon or Rectal Cancer: Increased Risk of Local Recurrence. Ann Surg. 2022 Feb 1;275(2):e420-e427. doi: 10.1097/SLA.0000000000003889. — View Citation

La Regina D, Di Giuseppe M, Lucchelli M, Saporito A, Boni L, Efthymiou C, Cafarotti S, Marengo M, Mongelli F. Financial Impact of Anastomotic Leakage in Colorectal Surgery. J Gastrointest Surg. 2019 Mar;23(3):580-586. doi: 10.1007/s11605-018-3954-z. Epub 2018 Sep 13. — View Citation

Mirnezami A, Mirnezami R, Chandrakumaran K, Sasapu K, Sagar P, Finan P. Increased local recurrence and reduced survival from colorectal cancer following anastomotic leak: systematic review and meta-analysis. Ann Surg. 2011 May;253(5):890-9. doi: 10.1097/SLA.0b013e3182128929. — View Citation

Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, Vickers AJ, Ransohoff DF, Collins GS. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015 Jan 6;162(1):W1-73. doi: 10.7326/M14-0698. — View Citation

Reisinger KW, Poeze M, Hulsewe KW, van Acker BA, van Bijnen AA, Hoofwijk AG, Stoot JH, Derikx JP. Accurate prediction of anastomotic leakage after colorectal surgery using plasma markers for intestinal damage and inflammation. J Am Coll Surg. 2014 Oct;219(4):744-51. doi: 10.1016/j.jamcollsurg.2014.06.011. Epub 2014 Jun 25. — View Citation

Stam WT, Ingwersen EW, Ali M, Spijkerman JT, Kazemier G, Bruns ERJ, Daams F. Machine learning models in clinical practice for the prediction of postoperative complications after major abdominal surgery. Surg Today. 2023 Feb 25. doi: 10.1007/s00595-023-02662-4. Online ahead of print. — View Citation

van Rooijen SJ, Huisman D, Stuijvenberg M, Stens J, Roumen RMH, Daams F, Slooter GD. Intraoperative modifiable risk factors of colorectal anastomotic leakage: Why surgeons and anesthesiologists should act together. Int J Surg. 2016 Dec;36(Pt A):183-200. doi: 10.1016/j.ijsu.2016.09.098. Epub 2016 Oct 15. — View Citation

Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005 May;58(5):475-83. doi: 10.1016/j.jclinepi.2004.06.017. — View Citation

* Note: There are 15 references in all — Click here to view all references

Outcome

Type	Measure	Description	Time frame	Safety issue
Other	The predictive performance of the prediction model	the predicted probability of anastomotic leakage for every participant will be evaluated with the actual outcome after 30 days postoperatively. The discrimination of the external validation cohort is reported with the area under the curve - receiver operating characteristic, the sensitivity, specificity, and accuracy. Calibration of the prediction model will be visualized in a calibration plot.	30 days postoperatively.
Primary	Colorectal anastomotic leakage	colorectal anastomotic leakage is defined according to Reisinger: "clinically relevant anastomotic leakage is defined as extra luminal presence of contrast fluid on contrast-enhanced computed tomography scans and/or leakage when relaparotomy was performed, requiring reintervention or treatment."	the occurrence of the primary outcome is assessed after 30 days postoperatively.
Secondary	Length of Hospital Stay	To evaluate the impact of anastomotic leakage on length of stay, the duration of a patient's hospital stay after undergoing a colorectal resection is investigated.	The length of hospital stay is assessed 90 days postoperatively

External Links

•   Homepage of all studies our research group is involved in, concerning the prevention and treatment of anastomotic leakage