Outcome
Type |
Measure |
Description |
Time frame |
Safety issue |
Primary |
Adherence to standardized handoff intervention (Fidelity) measured by number of handoff steps followed. |
Fidelity is defined as adherence to the standardized handoff intervention. Fidelity is a count of handoff steps followed, with scores ranging from 0 to 10. Fidelity will be measured by trained site-based observers. Field notes by trained observers will inform the interpretation of fidelity. Units of analysis: patient level (handoff level), ICU level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5. |
|
Primary |
New-onset organ failure |
Per-protocol handoffs enable clinicians to follow expected care practices and to anticipate and avoid postoperative deterioration. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year5 |
|
Secondary |
Perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory (Acceptability) |
Acceptability is defined as "the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory." (Proctor et al, Adm Policy Ment Health (2011) 38:65-76). Acceptability will be measured using the Acceptability of Intervention Measure (AIM) developed by Weiner et al (Implement Sci. 2017;12(1):108; see Appendix). Interview and focus group transcripts and field notes will be analyzed to develop a more nuanced understanding of acceptability. Unit of analysis: individual (employed staff) |
Baseline (Year 1); within 2months of implementation; within 2 months of beginning sustainment |
|
Secondary |
Perceived fit, relevance, or compatibility of the innovation in a practice setting or to address a particular issue (Appropriateness), measured by the Intervention Appropriateness Measure |
Appropriateness is defined as "the perceived fit, relevance, or compatibility of the innovation or evidence based practice for a given practice setting, provider, or consumer; and/or perceived fit of the innovation to address a particular issue or problem." (Proctor et al, Adm Policy Ment Health (2011) 38:65-76). Appropriateness will be measured using the Intervention Appropriateness Measure (IAM) developed by Weiner et al (Implement Sci. 2017;12(1):108; see Appendix). Interview and focus group transcripts and field notes will be analyzed to develop a more nuanced understanding of appropriateness. Unit of analysis: individual (employed staff) |
Baseline (Year 1); within 2months of implementation; within 2 months of beginning sustainment |
|
Secondary |
Feasibility of treatment or innovation to be carried out in a practice setting, measured using the Feasibility of Intervention Measure. |
Feasibility is defined as "the extent to which a new treatment, or an innovation, can be successfully used or carried out within a given agency or setting." (Proctor et al, Adm Policy Ment Health (2011) 38:65-76). Feasibility will be measured using the Feasibility of Intervention Measure (FIM) developed by Weiner et al (Implement Sci. 2017;12(1):108; see Appendix). Interview and focus group transcripts and field notes will be analyzed to develop a more nuanced understanding of feasibility. Unit of analysis: individual (employed staff) |
Baseline (Year 1); within 2months of implementation; within 2 months of beginning sustainment |
|
Secondary |
Fidelity over time (Sustainment) |
Sustainment is defined as fidelity over time. We define the sustainment phase as starting two years after implementation. Units of analysis: patient level (handoff level), ICU level. |
This outcome will be measured at monthly intervals starting 2 years after implementation, up to 5 years. |
|
Secondary |
Monetary and non-monetary costs measured using the line item classification system |
We will conduct a retrospective cost analysis as described by Hoeft et al (Implementation Science 2019;14(1):37). We will use the line item classification system, in which accounting principles are used to calculate costs, and will attempt to include both monetary costs and non-monetary costs (e.g., opportunity costs). Activities undertaken only for research (e.g., interviews and focus groups) will not be included in this analysis. Field notes will be the primary data source for this outcome. Unit of analysis: ICU level. |
This outcome will be measured within 2 months of implementation; within2 months of beginning sustainment |
|
Secondary |
Teamwork (effectiveness) measured using the Team Emergency Assessment Measure. |
Teamwork will be measured using a validated instrument, the Team Emergency Assessment Measure (TEAM) (Cooper et al, Resuscitation 2010;81(4):446-452). Unit of analysis: handoff team. |
This outcome will be measured at quarterly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5 |
|
Secondary |
Clinician satisfaction measured at the individual level. |
Clinician satisfaction will be measured using a non-validated instrument adapted from the parent HATRICC study (see Appendix). Unit of analysis: individual. |
This outcome will be measured annually from Years 1 to 5. |
|
Secondary |
Workload measured using the NASA Task Load Index. |
Workload is a measure of effectiveness and will be measured using the validated NASA Task Load Index. |
This outcome will be measured at quarterly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5. |
|
Secondary |
Completeness of information exchange (Information Omissions) measured using a structured observation form. |
Information omissions is a measure of the completeness of information exchange. It will be measured at the same time as fidelity using a structured observation form (see Appendix). Unit of analysis: patient level (handoff level). |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year5 |
|
Secondary |
Weighted average of the reliability-adjusted observed-to-expected ratios for component indicators of adverse events |
This is a composite measure defined by AHRQ as "the weighted average of the reliability-adjusted observed-to-expected ratios (indirect standardization of the smoothed rates) for the following component indicators: PSI 03 Pressure Ulcer Rate; PSI 06 Iatrogenic Pneumothorax Rate; PSI 07 Central Venous Catheter-Related Blood Stream Infection Rate; PSI 08 Postoperative Hip Fracture Rate; PSI 09 Perioperative Hemorrhage or Hematoma Rate; PSI 10 Postoperative Physiologic and Metabolic Derangement Rate; PSI 11 Postoperative Respiratory Failure Rate; PSI 12 Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate; PSI 13 Postoperative Sepsis Rate; PSI 14 Postoperative Wound Dehiscence Rate; PSI 15 Accidental Puncture or Laceration Rate." We note that some of the components of this measure overlap with our primary effectiveness outcome. Unit of analysis: ICU-level. |
This outcome will be measured at quarterly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5. |
|
Secondary |
AHRQ PSI 9: Perioperative hemorrhage or hematoma rate(effectiveness) |
This measure is one of the five components of the co-primary outcome of new-onset organ failure. It is defined by AHRQ as "perioperative hemorrhage or hematoma cases with control of perioperative hemorrhage, drainage of hematoma, or a miscellaneous hemorrhage- or hematoma-related procedure following surgery per 1,000 surgical discharges." AHRQ collects this information for patients aged 18 and older, but we will collect this outcome for pediatric patients in the two pediatric ICUs. Unit of analysis: ICU-level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5 |
|
Secondary |
AHRQ PSI 10: Postoperative physiologic or metabolic derangement rate (effectiveness) |
This measure is one of the five components of the co-primary outcome of new-onset organ failure. It is defined by AHRQ as "postoperative acute renal failure (secondary diagnosis) with dialysis per 1,000 elective surgical discharges." AHRQ collects this information for patients aged 18 and older, but we will collect this outcome for pediatric patients in the two pediatric ICUs. Unit of analysis: ICU-level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5. |
|
Secondary |
AHRQ PSI 11: Postoperative respiratory failure rate (effectiveness) |
This measure is one of the five components of the co-primary outcome of new-onset organ failure. It is defined by AHRQ as "postoperative respiratory failure (secondary diagnosis), prolonged mechanical ventilation, or reintubation cases per 1,000 elective surgical discharges." AHRQ collects this information for patients aged 18 and older, but we will collect this outcome for pediatric patients in the two pediatric ICUs. Unit of analysis: ICU-level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5 |
|
Secondary |
AHRQ PSI 12: Perioperative pulmonary embolism or DVT rate (effectiveness) |
This measure is one of the five components of the co-primary outcome of new-onset organ failure. It is defined by AHRQ as "perioperative pulmonary embolism or deep vein thrombosis (secondary diagnosis) per 1,000 surgical discharges." AHRQ collects this information for patients aged 18 and older, but we will collect this outcome for pediatric patients in the two pediatric ICUs. Unit of analysis: ICU-level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5 |
|
Secondary |
AHRQ PSI 13: Post operative sepsis rate (effectiveness) |
This measure is one of the five components of the co-primary outcome of new-onset organ failure. It is defined by AHRQ as "postoperative sepsis cases (secondary diagnosis) per 1,000 elective surgical discharges." AHRQ collects this information for patients aged 18 and older, but we will collect this outcome for pediatric patients in the two pediatric ICUs. Unit of analysis: ICU-level. |
This outcome will be measured at monthly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5 |
|
Secondary |
Professionalism of handoff team during handoff. |
Professionalism will be characterized qualitatively using field notes from trained observers. Unit of analysis: handoff team. |
At quarterly intervals starting Year 1, Quarter 2 and continuing until the end of Year 5. |
|