Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations

Emergencies Clinical Trial

— OPPE16  

Official title:

Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations

Clinical Trial Details — Status: Completed

Administrative data

• NCT number: NCT03229226
• Other study ID #: 15100805-IRB01-AM01
• Status: Completed
• Phase: N/A
• First received: July 17, 2017
• Last updated: July 26, 2017
• Start date: October 1, 2015
• Est. completion date: May 1, 2016

Study information

• Verified date: July 2017
• Source: Rush University Medical Center
• Contact: n/a
• Is FDA regulated: No
• Study type: Observational

Clinical Trial Summary

Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. This assessment is known as the ongoing professional practice evaluation (OPPE). There are several methods by which this can be accomplished, including through chart review of clinician practice and by verbally assessing knowledge of the steps required to do a particular procedure. However, for infrequently performed or complicated procedures, these methods may not allow objective evaluation on a regular basis. Simulation using task trainers or manikin models offers an alternative method of objective evaluation in a standardized setting.

The goal of this study is to develop two simulated scenarios to assess physician skill in relatively uncommon procedures and compare the data obtained against the verbal assessment and chart review model previously used at our institution. Two raters will assess each practitioner's verbal and simulated procedures. They will also assess globally their confidence that the physician is capable of performing the procedure safely and correctly based on the verbal or simulated trial.

The investigators anticipate that using a simulated experience for assessment will increase the ability of raters to assess proficiency for the purposes of an OPPE, specifically by increasing the number of critical procedural components that can be objectively evaluated. The investigators hypothesize that there will not be a strong relationship between the scores obtained on the verbal assessment and the scores obtained on the task trainer assessment. The investigators anticipate that there will be a portion of study participants that do not meet a minimum passing standard and may require additional deliberate practice and further testing.

Description:

Physician training has developed over recent years to include simulation-based learning as a method of both formative and summative assessment. Simulation based medical education (SBME) with deliberate practice has been shown to be superior to traditional clinical education with respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar concept with Mastery Learning has shown to increase resident skill in lumbar puncture placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in central line placement reduces hospital wide rates of central line associated bacterial infections.

When used in continuing medical education (CME), simulation has been shown to be feasible, and to increase clinician confidence in patient management. A recent study of nephrology fellows and attendings in practice showed that both groups scored similarly on a pre-test examination, and that all fellows who completed mastery learning subsequently passed the checklist assessment at the competency level and rated the training session highly.

In other work, simulation has been used for ongoing professional practice assessments (OPPE), for example in psychiatry using online simulated patients and assessments as a way of identifying a smaller number of attendings requiring focused assessment in person). Using electronically collected clinical data, anesthesia cases were reviewed for deviation from the standard of care on several key pieces of documentation common to many anesthesia procedures. This method of direct chart assessment for OPPE may be possible in anesthesia, a field where many cases have similarities in procedure, but would not work for a field such as Emergency Medicine, where the type and acuity of patient encounters varies greatly from day to day.

The ability of Simulation to present a standardized, specific procedural need offers an opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a concise way. The investigators' research aims to present one possible approach to this assessment.

Recruitment information / eligibility

• Status: Completed
• Enrollment: 17
• Est. completion date: May 1, 2016
• Est. primary completion date: May 1, 2016
• Accepts healthy volunteers: Accepts Healthy Volunteers
• Gender: All
• Age group: N/A and older

Eligibility

Inclusion Criteria:

• Emergency Medicine physician

Exclusion Criteria:

• Not Emergency Medicine physician

Related Conditions & MeSH terms

• Competence
• Emergencies

Intervention

Other:
• Verbal versus procedural assessment
Faculty were assessed on both verbal and simulated performance of procedures.

Locations

Country	Name	City	State
n/a

Sponsors (1)

Lead Sponsor	Collaborator
Rush University Medical Center

References & Publications (8)

Barsuk JH, Cohen ER, Caprio T, McGaghie WC, Simuni T, Wayne DB. Simulation-based education with mastery learning improves residents' lumbar puncture skills. Neurology. 2012 Jul 10;79(2):132-7. doi: 10.1212/WNL.0b013e31825dd39d. Epub 2012 Jun 6. — View Citation

Barsuk JH, Cohen ER, Nguyen D, Mitra D, O'Hara K, Okuda Y, Feinglass J, Cameron KA, McGaghie WC, Wayne DB. Attending Physician Adherence to a 29-Component Central Venous Catheter Bundle Checklist During Simulated Procedures. Crit Care Med. 2016 Oct;44(10):1871-81. doi: 10.1097/CCM.0000000000001831. — View Citation

Barsuk JH, Cohen ER, Potts S, Demo H, Gupta S, Feinglass J, McGaghie WC, Wayne DB. Dissemination of a simulation-based mastery learning intervention reduces central line-associated bloodstream infections. BMJ Qual Saf. 2014 Sep;23(9):749-56. doi: 10.1136/bmjqs-2013-002665. Epub 2014 Mar 14. — View Citation

Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009 Oct;37(10):2697-701. — View Citation

Kerr B, Hawkins TL, Herman R, Barnes S, Kaufmann S, Fraser K, Ma IW. Feasibility of scenario-based simulation training versus traditional workshops in continuing medical education: a randomized controlled trial. Med Educ Online. 2013 Jul 18;18:21312. doi: 10.3402/meo.v18i0.21312. — View Citation

McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 Jun;86(6):706-11. doi: 10.1097/ACM.0b013e318217e119. Review. — View Citation

McQuillan RF, Clark E, Zahirieh A, Cohen ER, Paparello JJ, Wayne DB, Barsuk JH. Performance of Temporary Hemodialysis Catheter Insertion by Nephrology Fellows and Attending Nephrologists. Clin J Am Soc Nephrol. 2015 Oct 7;10(10):1767-72. doi: 10.2215/CJN.01720215. Epub 2015 Sep 25. — View Citation

OPPE and FPPE: Tools to help make privileging decisions 2013. Wise RA. (Accessed Jul 19, 2017, at http://www.jointcommission.org/jc_physician_blog/oppe_fppe_tools_privileging_decisions/).

Outcome

Type	Measure	Description	Time frame	Safety issue
Primary	Performance on Procedural Checklist	A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance.	Immediate
Secondary	Acceptance of simulated training and assessment format	A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session.	Immediate