Clinical Trial Details
— Status: Active, not recruiting
Administrative data
| NCT number |
NCT05641025 |
| Other study ID # |
3879_1 |
| Secondary ID |
|
| Status |
Active, not recruiting |
| Phase |
|
| First received |
|
| Last updated |
|
| Start date |
November 16, 2022 |
| Est. completion date |
January 30, 2023 |
Study information
| Verified date |
December 2022 |
| Source |
Hospital Italiano de Buenos Aires |
| Contact |
n/a |
| Is FDA regulated |
No |
| Health authority |
|
| Study type |
Observational
|
Clinical Trial Summary
The aim of this study is to identify predictive factors of infections caused by
multidrug-resistant organisms in patients with cirrhosis and to develop and validate
(internally and externally) a predictive model that might be useful to use in clinical
settings to stratify the risk and lead clinical decision-making strategies.
Description:
Background Bacterial infections are currently recognized as a surrogate for the final stage
of chronic liver disease. The relevance of bacterial infections as a prognostic factor has
been clearly stated in a meta-analysis that found that bacterial infections increase
mortality four-fold in this population, considering 30% of patients die within one month, and
another 30% die one year after these infections are diagnosed.
Patients with cirrhosis are particularly prone to be colonized or infected by
multidrug-resistant organisms because they are frequently admitted to hospitals, they require
admissions to critical care units and receive organ support, they are exposed to invasive
procedures and they frequently receive therapeutic and prophylactic antibiotics.
Prior studies in patients with cirrhosis mainly described risk factors of infections by
multidrug-resistant organisms infections, without providing a risk assessment. Therefore, it
would be desirable to count on tools to identify which patients are at higher risk than
others of presenting a multidrug-resistant organism infection, such as a predictive model
that might help practitioners to select the empirical antibiotic treatment.
Objectives To identify easily accessible predictors of multidrug resistance in adult patients
with cirrhosis at the time of bacterial infection diagnosis or suspicion.
To develop and validate (goodness of fit, discrimination, calibration, and diagnostic
performance) predictive models of multidrug resistance in adult patients with cirrhosis at
the time of bacterial infection
Setting and Study design
Cross-sectional study using data from two prospective cohort studies:
- The international prospective cohort study (Piano el at, DOI:
10.1053/j.gastro.2018.12.00) that included 1,302 patients from 46 centers in Europe,
Asia, and America, from October 2015 to September 2016. This dataset will be used for
model development and internal validation and will be referred to as the development
dataset from now on.
- The prospective cohort study from Argentina and Uruguay (Marciano et al, NCT03919032),
that included 472 patients from 22 centers in Argentina and Uruguay, from September 2018
to December 2020. This dataset will be used for external validation and will be referred
to as the validation dataset from now on.
Population As mentioned, the objective of this research is to develop and validate a
predictive model of multidrug resistance. However, multidrug resistance is observable only in
episodes of culture-positive bacterial infections, and it is known that approximately 50% of
all bacterial infections taken together are culture-negative. Since the predictive score is
expected to be used bedside at the time of the suspicion or confirmation of the bacterial
infection, but prior to the results of the cultures, all patients (culture-positive and
culture-negative) from the aforementioned studies will be used to develop and internally and
externally validate the model.
Outcome variable The outcome will be infection by multidrug-resistant organisms defined in
both studies according to the current recommendation: an infection caused by an organism with
acquired resistance to at least one antibiotic of three different families. All episodes of
infection with a culture yielding a multidrug-resistant organism will be classified as 1. All
culture-negative episodes of infection together with all culture-positive infections without
multidrug-resistant organisms will be classified as 0 in the principal analysis. In the
secondary analysis for validation, all culture-negative episodes of infections will be
excluded.
Sampling and sample size calculation In both studies, sampling was consecutive. Regarding
sample size calculation for the development of the model, the rule of thumb will be used.
Even though there are more modern proposals to approach the sample size to generate
predictive models, the lack of prior literature in this field prevents the research group
from having the necessary information that this new method requires. Therefore, as the
dataset that will be used to develop the predictive model contains a total of 253 events
(infection by multidrug-resistant organisms infection) a model starting with 13 to 25
parameters could be explored, considering the rule of thumb proposes to include 10 to 20
events per parameter (EPP).
The prevalence of multidrug-resistant organisms is approximately 20% (253/1302) in the
development dataset. A parsimonious predictive model with around 15 potential parameters will
be used. The sample size was calculated to keep a MAPE (mean absolute prediction error) of
0.05, a shrinkage factor<10%, and a low optimism of 0.05. The expected Cox-Snell R2 of 0.1
was used. The following are the sample size estimations using Riley's approach in 4 steps to
estimate sample size: 1. Total sample size 246 (3.28 EPP) to estimate the independent
parameter with adequate precision; 2. Total sample size 860 (14.3 EPP) to keep predictive
precision (12 because it is the higher number of parameters allowed,
https://mvansmeden.shinyapps.io/BeyondEPV/ ); 3. Total sample size 1274 (16.99 EPP) to
minimize overfitting; and 4. Total sample size 443 (5.91 EPP) to minimize optimism. The
higher total sample size of 1274 was chosen to fulfill all 4 criteria. The sample size was
estimated using Ensor and Riley's pmsampsize for STATA.
Regarding sample size for external validation, the literature is more conflicting on how many
events and no events are necessary. Given that it is not easy to count with a database to
perform external validation, and that some experts suggest that at least 100 events and no
events are necessary to do so, the validation dataset was considered adequate since it
contains 103 events and a significantly greater number of no-events.
Descriptive and exploratory statistics In both studies, the prevalence of multidrug-resistant
organisms infections will be presented with 95% confidence intervals (CI) and will be
estimated over the entire population (culture-positive plus culture-negative infections) and
also only over the subgroup of patients with culture-positive infections.
The distribution of potential predictors and the outcome will be presented for both cohorts
using mean and standard deviations (DS) or median and 25th-75th percentiles for numerical
variables, and absolute numbers and percentages for categorical variables.
The prevalence of infections by multidrug-resistant organisms varies in different countries,
regions, and even within the same country and medical centers. The frequency of
multidrug-resistant organisms will be presented for each country and descriptive statistics
will be provided. This is a proxy of multidrug-resistant organisms' frequency in different
settings, and hence potentially may be related to the probability of getting a
multidrug-resistant organisms infection. Since the aim was to develop and validate a
predictive model that can be used in any setting, countries will not be explored as
predictors. This potential limitation in the predictive model enhances the possibility of
developing a useful model that is not restricted to the countries included in this study.
Development of the predictive model. All patients with complete information (complete case
analysis) regarding the study outcome and potential predictors included in the development
dataset will be included in the development and internal validation of the predictive model
(Piano et al, DOI: 10.1053/j.gastro.2018.12.00, N= 1,302). The following candidate predictors
of MDRO infections will be pre-selected by the research group according to prior publications
and knowledge in the field and expert opinion: age, sex, norfloxacin prophylaxis,
rifaximin_treatment, beta_blockers, prior invasive_procedures, MELD-Na, albumin, intensive
care unit admission, acute-on-chronic liver failure, ACLF, ascites, encephalopathy, systemic
inflammatory response syndrome, vasopressors, pneumonia, urinary tract infection, spontaneous
bacterial peritonitis.
The association between each potential predictor and the outcome of interest will b evaluated
with logistic regression models. Odds ratios (OR) with 95% CI will be presented.
Pre-specified relevant interactions between pairs of candidate predictors which are expected
to occur according to prior publications or knowledge in the field will be explored and
included in the model. In case there is no linear association between continuous predictors
and the log odds of multidrug-resistant organisms infection, multivariable fractional
polynomials (MFP). For this purpose, the STATA mfp command will be used, which performs a
multivariable logistic regression with backward stepwise selection for final predictor
selection, applying Akaike's Information Criteria (AIC) for the removal of variables,
together with the generation of MFPs. The AIC will be approached in STATA using backward
elimination with a p-value of 0.157 as a proxy since STATA would not perform it
automatically. The following predictors which are considered to have great clinical relevance
will be forced to be in the final model: therapeutic antibiotic use over the last three
months, and type of infection. Homoscedasticity will be explored graphically with a Scatter
plot of standardized Pearson residuals over the predicted probability of infection by
multidrug-resistant organisms. The collinearity of the predictors will be explored with dot
plots and correlation coefficients. Extreme values will be explored using Cook's distance.
The full linear predictor of the model will be presented. By full linear predictor, it is
meant what is highlighted in the following equation: Logit (pi)= A + B1X1 + B2X2
+......+BnXn.. Presenting the full linear predictor is a key element for external validation
of the developed model. All coefficients will be presented with their standard errors, ORs
(95 CI%), and p values. The individual linear predictor will be summarized with mean and SD
or median and percentiles 25%-75%, and ranges. Uniform shrinkage will be applied to the full
linear predictor to correct for optimism, and presented as well, like this: A1 + [ S* (B1X1 +
B2X2 +......+BnXn.), where A1 is the intercept re-estimated to ensure calibration-in-
the-large, and S is the shrinkage factor. The uniform shrinkage factor is the
optimism-adjusted calibration slope calculated during bootstrapping internal validation
(explained below).
The goodness of fit of the developed model will be explored with the following estimands:
Brier´s score, which ranges from 0 to 0.25 (the lowest the better the model fits).
Nagelkerke´s R2 (the higher the better the model fits). Akaike's information criteria (the
lowest the better)
Apparent validation Apparent validation will be explored in the original dataset, without
resampling, and will be evaluated with calibration and discrimination measures. Calibration
will be evaluated with observed/expected ratio, calibration plots, calibration in the large
(CITL), and calibration slope. Discrimination will be evaluated by estimating the area under
the ROC curve (also named C-index or C-statistic when applying logistic regression).
Additionally, the diagnostic performance will be evaluated, reporting the sensitivity,
specificity, positive predictive, and negative predictive values of different cutoff points
or risks of multidrug-resistant bacterial infection.
Internal Validation
For internal validation, bootstrapping will be applied using 300 samples of the same size
drawn with replacement from the development dataset. This strategy mimics sampling from the
target population by taking the original dataset and randomly selecting units of analysis
(i.e patients) with replacement until a dataset of the same size is obtained. With every
bootstrap sample, a model is generated using the same methods as described above, and its
performance is evaluated, using the original dataset. The following steps are applied for the
purpose of internal validation using bootstrap :
Produce a bootstrap sample Develop a model in the bootstrap sample (using the same methods as
originally done) Calculate apparent performance (in the bootstrap sample) and test
performance (in the original data) Calculate optimism for each estimator (apparent - test)
Repeat steps prior steps 300 times to obtain the distribution of optimism estimates Calculate
average optimism, and adjust original model estimates: AUC, calibration slope and CITL.
The reliability of predictors selection using the same bootstrapping procedure will be
presented. The reliability of prediction selection is the percentage of bootstrapping samples
in which a particular predictor remains in the final predictive model according to the same
automatic procedure used to construct the model, divided by the total number of bootstrapping
samples. Predictors reliability over 50% are considered stable and this is strong evidence
that should be included in the final model.
External validation of the predictive model All patients from the external validation dataset
(Marciano et al, NCT03919032, N=472) with complete data on the predictors and the outcome
will be used for external validation, with the intention to explore the transportability of
the predictive model. The full shrunken linear predictor (i.e beta coefficients and
intercept) of the developed model will be used. Calibration and discrimination will be
estimated in the external validation of the predictive model. Calibration will be evaluated
with observed/expected ratio, calibration plots, calibration in the large (CITL), and
calibration slope. Discrimination will be evaluated by estimating the area under the ROC
curve.
Subgroup analysis Validation evaluation will be repeated as a secondary analysis considering
only patients with culture-positive infections.
Management of missing data The proportion of missing data is expected to be low considering
the information available from both cohorts' publications or communications in scientific
meetings. Since the predominant missing mechanism may be Missing Completely at Random,
complete cases as a random sample of all the observations will be considered. Under this
assumption, all data analysis will be performed as a complete case analysis. The number of
missing values will be reported.
