Faculty Development to Improve Mini-CEX Ratings

Study Overview
=================
Brief Summary
-----------------
We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.

Detailed Description
-----------------
Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy. Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability. Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy. Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields. Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.

Official Title
-----------------
Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial

Conditions
-----------------
Faculty Development

Intervention / Treatment
-----------------
* Behavioral: Rater training workshop


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota Exclusion Criteria: None
Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Treatment
Allocation: Randomized
Intervention Model: Parallel Assignment
Masking: None (Open Label)


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Experimental: Workshop<br>Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion | Behavioral: Rater training workshop<br>* Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion<br>|
| No Intervention: Delayed workshop<br>No specific intervention until after follow-up assessment, then receives same workshop. | |

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Interrater reliability (videotaped resident-patient encounters) |  | April-June 2006 | 

Secondary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores |  | April-June 2006 | 
| Interrater reliability (actual resident-patient encounters) |  | March 2005 (historical) to June 2007 | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
Medical education, faculty development, rater training, clinical competence, assessment