Competency-Based Education Clinical Trial
Official title:
Effect of a Competence Based Medical Education Programme on Training Quality in Intensive Care Medicine. COBALIDATION TRIAL.
This study evaluates whether the implementation of a competency-based medical education
program called CoBaTrICE (Competency Based Training program in Intensive Care in Europe)
provides higher levels of competency in comparison with the current official time-based
program in Intensive Care Medicine (ICM) in Spain.
The hypothesis will be confirmed or rejected through a multicenter cluster randomized trial
of 14 ICU Departments from 14 academic referral hospitals located in Spain. A total of 38
trainees on the 3rd year of the specialization period will be followed during the three years
of their specific training period in Intensive Care Medicine. CoBaTrICE (seven hospitals)
will be compared with the current official model of training in ICM in Spain (seven
hospitals), which is based on exposure to experiences through clinical rotations. The
implementation of CoBaTrICE will include the three following essential elements: 1) Training
the trainers; 2) Workplace-based assessments; 3)The use of an electronic portfolio.
The level of competency achieved by each participant will be determined by a simulation-based
Objective Structured Clinical Exam (OSCE) performed at the end of the third year of traning
(baseline) and at the end of the 5th year of training period.
Competence Based Medical Education (CBME) stands for a shift in emphasis away from time-based
programs, based solely on exposure to experiences such as clinical rotations, in favour of an
emphasis on needs-based graduate outcomes authenticity, and learner-centeredness (1). The new
paradigm defies the assumption that competence is achieved based on time spent on rotations
and instead requires residents to demonstrate competence, what involves performing a
significant number of formative assessments (2,3). The CBME model for Intensive Care Medicine
(ICM) is called CoBaTrICE (Competency-Based Training in Intensive Care Medicine in Europe)
(4), an international partnership of professional organisations and critical care clinicians
whose ultimate aims are: to assure a high quality level education in ICM, to harmonize
training in ICM without interfering with national specific regulations and to allow for free
movement of ICM professionals across Europe (5). CoBaTrICE program used an international
grounded approach to defining specialist competencies, and incorporated the views of 1,398
patients and relatives from 70 ICUs in eight EU countries (6). The final program includes 102
competence statements grouped into 12 domains and a definition of the level of expertise
required of a safe practitioner at the end of their specialist training. CoBaTrICE has been
formally adopted by 15 of 32 countries in Europe with a further 12 planning to do so, Spain
is included in this latter group (7,8).
Importance:
CBME is gaining acceptance as a solution to address the challenges associated with the
current time-based models of physician training. However, whether CBME programs provide
better outcomes than the traditional ones is actually unknown.
Hypothesis:
The implementation of the CoBaTrICE will provide higher levels of competency in comparison
with the current official time-based programme in ICM in Spain.
Objectives:
1. To determine the effect of CoBaTrICE implementation on:
1. The level of competence achieved by the trainees.
2. The percentage of critical performance elements.
2. To identify gaps in performance that could be addressed in future educational
interventions.
3. To investigate the reliability and feasibility of conducting simulation based assessment
at multiple sites.
4. To determine compliance with CoBaTrICE and barriers to an effective implementation.
Design:
A multicenter cluster randomized trial of 14 ICU Departments from 14 academic referral
hospitals located in Spain. The participating ICUs are general medical and surgical ICUs
accredited to train 2-3 new residents in ICM per year.
Participants:
A total of 38 trainees on the 3rd year of the specialization period (R3) will be followed
during the three years of their specific training period in ICM (Stage 2). Enrollment will
start on February 1, 2019, and the follow-up will end on May 31, 2021.
Intervention: see the specific item in the protocol section.
Main outcomes and measures (see also the specific item in the protocol section):
To determine the level of competency achieved for each participant, a simulation based OSCE
(18-26) will be performed at the end of the 3rd year(baseline) and at the end of the 5th year
of training in ICM. The OSCE will be performed simultaneously at four simulation centers
(Hospital la Fe, Valencia; Francisco de Vitoria, Madrid; IAVANTE, Granada; and Hospital
Clinic, Barcelona). Each participant will perform in five 15-min standardized patient or
high-fidelity simulated clinical crisis scenarios.
Rating instruments and guide to rating:
Via a Delphi technique, an independent panel of 10 intensivists subject matter experts
(simulation instructors and European Diplome in Intensive Care [EDIC] examiners) will perform
the following tasks: 1) to select the competences to be assessed; 2) to design the scenarios;
and 3) to define the items of the checklist for each scenario: a) the critical essential
performance elements (CEPE), and b) the critical non-essential performance elements (CNEPE)
that must be observed and scored in a yes/no format. CEPEs are defined as essential steps or
actions in the management which if missed could have an immediate significant impact on
morbidity and mortality. CNEPE are also important for the adequate management of the scenario
but they do not have an immediate influence on the outcome. The performances will be video
recorded (26). All the video recordings will be later rated by two blinded raters, members of
the experts panel, using specific checklists which include 25 items with a detailed
description of the CEPEs and CNEPEs, the competencies technical (diagnosis and treatment) and
non-technical (communication, team leadership, resource management) associated with each
item, as well as specific information about what is expected to be done by the trainee. After
each video-assessment, the performance of the trainee will be classified in five levels of
competency: Level I: The participant completed less than 60% of the CEPEs. The participant
needs guidance and direct supervision to perform the activity in all cases. Level II: The
participant completed ≥ 60% of the CEPEs but less than 80% of CEPEs. The participant needs
guidance and supervision to perform the activity in most of situations. Level III: The
participant completed ≥ 80% of the CEPEs but less than 100% of CEPEs. The participant needs
some guidance and supervision to perform the activity in complex situations. Level IV: The
participant completed 100% of the CEPEs but less than 80% of CNEPEs. The participant can
perform the activity under indirect supervision. Level V: The participant completed 100% of
the CEPEs and ≥ 80% of the CEPEs.The participant is independent to perform the activity.
Finally, raters also will qualify the performance as: poor, pass, good, outstanding.
Measures will include: 1) the percentage of CEPEs observed; 2) the percentage of CNEPEs
observed; 3) the overall competency level achieved on a descriptive scale of 1 to 5 (novice
to independent practitioner); 4) the qualitative rating given by the raters based on whether
the performance is at the level expected according to the predefined level for the year of
training.
Standardization of Scenario Delivery (18-22). In order to facilitate reproducible scenario
delivery, rules, detailed scripts and a guidebook for each scenario will be created. The
scenario script will describe the elements of the simulated clinical environment (e.g., the
equipment and medications available), evolution of the patient's condition throughout the
crises and the responses to interventions, standardized answers to anticipated participant
questions, and criteria that define successful completion of CPEs. Participants will be
briefed on relevant mannequin characteristics, basic rules for participating in simulation
scenarios, and location and uses of medications, clinical equipment, and other resources.
After finishing the OSCE, resident feedback will be asked through a specific satisfaction
survey.
Statistical Analysis:
Sample size: A power analysis has been performed in order to determine the minimum sample
size required to detect with a power of .95 (α = .05, 1 - β = .95) a medium effect size (f =
.30), requiring a minimum sample size of 90 observations (30 residents).
A mixed factorial ANOVA design will be applied on the time factor (R3-R4-R5) with the group
factor (intervention vs. control) for the assessment of professional competences considered,
applying post-hoc Bonferroni tests to analyze the principal and interaction effects between
the two factors.
In order to estimate the effectiveness of CoBaTrICE, the differences between intervention and
control group regarding the primary and secondary end points will be analyzed by a
multivariate logistic regression analysis, the adjusted odds ratio will be estimated.
Simulation experience defined as participation in at least two crisis resource management
courses will be included in the analysis.
Development phases of the study:
The first phase of the project will begin by setting up an independent panel of 10
intensivists experts/executors that will perform the tasks mentioned above. After randomizing
the hospitals to the experimental group, the tutors of the this group will be trained through
a twelve-hour course integrating: 1) a detailed explanation of the principles, structure, and
competencies contemplated in the CoBaTrICE program; 2) the basic principles of the formative
assessment, techniques applied, and effective feedback; and 4) the use of the electronic
portfolio to record the volume and results of the formative assessments and the progress of
the trainee.
At the time of recruitment, residents and tutors of the participant ICUs will complete online
several anonymous surveys to explore educational environment, engagement and satisfaction,
and professional burnout.
The second phase of the study will consist of the CoBaTrICE program implementation and the
comparison of the participating residents' levels of performance through the OSCE at specific
points in time, which will be the end of R3 (baseline OSCE) and R5. It will determine: a)
whether there are significant differences between the level of competencies shown by the
residents depending on the type of training; b) whether there are significant differences in
the pace of acquiring the competencies in the two types of programs; and c) the most common
errors made by the residents through the different training levels and the possible
differences between both programs. The tutors involved in the CoBaTrICE group will receive
support to overcome barriers and problems found in the implementation of the program and/or
in the use of the tools it incorporates. The tutors will be sent an "ad hoc" questionnaire
designed to collect their suggestions. A pilot OSCE with local non-participants residents in
the trial will be carried out at the simulation Center In Hospital La Fe, Valencia, Spain, in
order to check the feasibility and reliability of the scenarios and the video rating
process.The third phase of the study will be devoted to analyzing and publishing the results
of the study. Interim analysis will be performed in order to determine the adequate
implementation of the program and asses heterogeneity and/or possible bias selection.
;
Status | Clinical Trial | Phase | |
---|---|---|---|
Recruiting |
NCT05269576 -
Clinical Simulation as a Learning Tool in Medical Students
|
N/A | |
Not yet recruiting |
NCT06452901 -
PROgressive struCturEd Simulation-based Surgical Training Program (PROCESS) - Open Vascular Surgery
|
N/A | |
Completed |
NCT02247115 -
Rapid Assessment of Trainee Endoscopy Skills (RATES) Study
|
N/A | |
Completed |
NCT04822714 -
Effectiveness of Mobile-phone Career Competencies Intervention for Malaysian Public Sector Managers
|
N/A |