Inter-rater Reliability of Preoperative Mortality Risk Calculators

Reliability Clinical Trial

Official title:

Inter-rater Reliability of Preoperative Mortality Risk Calculators Used for High-risk Noncardiac Surgical Patients

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT06410183
• Other study ID #: 2022-2161
• Status: Completed
• Start date: February 1, 2019
• Est. completion date: August 1, 2021

Study information

• Verified date: June 2024
• Source: Rijnstate Hospital
• Contact: n/a
• Is FDA regulated: No
• Study type: Observational

Clinical Trial Summary

The goal of this study is to investigate the inter-rater variability of mortality risk calculations for high-risk non-cardiac surgical patients. For this purpose patient information from health care files were used. Five anesthesiologists calculated the mortality risks with three calculators for 34 high-risk non-cardiac surgical patients. The patients were discussed in a preoperative multidisciplinary discussion in a large teaching hospital in The Netherlands.

Description:

Identifying high-risk patients for perioperative treatment and decision-making remains a challenge due to difficulty in reliably estimating morbidity and mortality risks. Assessing surgical risk helps allocating resources, obtaining informed consent, and making shared decisions with a multidisciplinary team (MDT). Multimorbidity is increasing globally in an ageing population with a growing burden of chronic diseases. It has been shown that high-risk non-cardiac surgical patients suffer disproportionally from perioperative complications. Preoperative mortality risk calculators are available and may help to identify high-risk non-cardiac surgical patients already before surgery, triggering efforts to lower the burden of possible complications, e.g., by extended monitoring or specified treatments. A systematic review suggested a significant risk of bias in developing current preoperative risk calculators due to lack of external validation, highlighting the need for enhanced performance and reliability to ensure their effectiveness in clinical practice. Low reliability and performance may, despite their general availability, be why that preoperative risk calculators are not yet consistently used in clinical practice. For daily clinical use, good predictive performance, low inter rater variability and user friendliness are essential. Discrepancies in predictor measurements can cause miscalibration, changes in discriminatory ability, and overall accuracy, leading to clinically relevant variability in risk calculator results. Previous studies have shown that physicians must trust a mortality risk calculator before utilization. High-risk patients suffer especially from complications, and it has been shown that the complications often result in death perioperatively. Therefore, adequate preoperative calculation of mortality risks and early recognition of high-risk non-cardiac surgical patients could benefit from reliable preoperative risk calculation. These high-risk patients can then be discussed in a preoperative multidisciplinary discussion to lower complications and perioperative deaths.

The current study evaluated the inter-rater reliability of calculating preoperative mortality risk scores for high-risk non-cardiac surgical patients in clinical practice among five anesthesiologists. The hypothesis was that the available risk calculators would show moderate to good reliability. For this purpose, five anesthesiologists used the following three risk calculators: the preoperative score to predict postoperative morbidity [POSPOM, the American College of Surgeons surgical risk calculator, and the surgical outcome risk tool [SORT].

Recruitment information / eligibility

• Status: Completed
• Enrollment: 34
• Est. completion date: August 1, 2021
• Est. primary completion date: August 1, 2021
• Gender: All
• Age group: 18 Years and older

Eligibility

Inclusion Criteria:

• Patients discussed in a preoperative multidisciplinary team (MDT) meeting.

Exclusion Criteria:

• No complete patient file to calculate the mortality risks

Related Conditions & MeSH terms

• Reliability

Intervention

Other:
• no intervention
no intervention

Locations

Country	Name	City	State
Netherlands	Rijnstate Hospital	Arnhem	Gelderland

Sponsors (3)

Lead Sponsor	Collaborator
Rijnstate Hospital	Amsterdam University Medical Centers (UMC), Location Academic Medical Center (AMC), Utrecht University

Country where clinical trial is conducted

Netherlands, 

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	inter-rater reliability of three risk calculators: POSPOM, SRC and SORT	Five anesthesiologists calculated mortality risks retrospectively for 34 patients. The intra-class correlation coefficient (ICC) was calculated per risk calculator	After calculation of the mortality risks (1 month)
Secondary	mortality risk calculations or 34 patients	the Preoperative scale of postoperative outcome: (POSPOM), the surgical risk calculator: (SRC) and the surgical outcome risk tool: (SORT) mortality risk calculators were used for (retrospective) 30-day mortality risk calculation of 34 patients	1 month