Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations

Study Overview
=================
Brief Summary
-----------------
Since 2007, the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) has required evaluation of practicing attending physicians by management to ensure that procedural skills and medical care are meeting the accepted standards of care as assessed by each individual institution. This assessment is known as the ongoing professional practice evaluation (OPPE). There are several methods by which this can be accomplished, including through chart review of clinician practice and by verbally assessing knowledge of the steps required to do a particular procedure. However, for infrequently performed or complicated procedures, these methods may not allow objective evaluation on a regular basis. Simulation using task trainers or manikin models offers an alternative method of objective evaluation in a standardized setting. The goal of this study is to develop two simulated scenarios to assess physician skill in relatively uncommon procedures and compare the data obtained against the verbal assessment and chart review model previously used at our institution. Two raters will assess each practitioner's verbal and simulated procedures. They will also assess globally their confidence that the physician is capable of performing the procedure safely and correctly based on the verbal or simulated trial. The investigators anticipate that using a simulated experience for assessment will increase the ability of raters to assess proficiency for the purposes of an OPPE, specifically by increasing the number of critical procedural components that can be objectively evaluated. The investigators hypothesize that there will not be a strong relationship between the scores obtained on the verbal assessment and the scores obtained on the task trainer assessment. The investigators anticipate that there will be a portion of study participants that do not meet a minimum passing standard and may require additional deliberate practice and further testing.

Detailed Description
-----------------
Physician training has developed over recent years to include simulation-based learning as a method of both formative and summative assessment. Simulation based medical education (SBME) with deliberate practice has been shown to be superior to traditional clinical education with respect to procedural skills in physicians in training (McGaghie,W.C. 2011), and a similar concept with Mastery Learning has shown to increase resident skill in lumbar puncture placement. In addition, it has been shown that Simulation Based Mastery Learning (SBML) in central line placement reduces hospital wide rates of central line associated bacterial infections. When used in continuing medical education (CME), simulation has been shown to be feasible, and to increase clinician confidence in patient management. A recent study of nephrology fellows and attendings in practice showed that both groups scored similarly on a pre-test examination, and that all fellows who completed mastery learning subsequently passed the checklist assessment at the competency level and rated the training session highly. In other work, simulation has been used for ongoing professional practice assessments (OPPE), for example in psychiatry using online simulated patients and assessments as a way of identifying a smaller number of attendings requiring focused assessment in person). Using electronically collected clinical data, anesthesia cases were reviewed for deviation from the standard of care on several key pieces of documentation common to many anesthesia procedures. This method of direct chart assessment for OPPE may be possible in anesthesia, a field where many cases have similarities in procedure, but would not work for a field such as Emergency Medicine, where the type and acuity of patient encounters varies greatly from day to day. The ability of Simulation to present a standardized, specific procedural need offers an opportunity to assess Attending Emergency Physicians on a variety of critical procedures in a concise way. The investigators' research aims to present one possible approach to this assessment.

Official Title
-----------------
Development and Assessment of a Simulation Model of Focused and Ongoing Professional Practice Evaluations

Conditions
-----------------
Competence, Emergencies

Intervention / Treatment
-----------------
* Other: Verbal versus procedural assessment


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: Emergency Medicine physician Exclusion Criteria: Not Emergency Medicine physician
Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| 2016 OPPE<br>All faculty participating in yearly ongoing professional Practice evaluation were eligible for enrollment | Other: Verbal versus procedural assessment<br>* Faculty were assessed on both verbal and simulated performance of procedures.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Performance on Procedural Checklist | A 29-item procedural checklist for Central Line and a 21-item procedural checklist for Lumbar puncture will be used to assess performance by clinicians on Central Venous Catheter insertion and Lumbar Puncture performance. | Immediate | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Acceptance of simulated training and assessment format | A survey tool will be used to query participants on their preferred way of both practicing and being assessed on procedural skill. It will also assess whether participants feel inclined to complete a future simulated procedural training session. | Immediate | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
Simulation, Task Trainer