Cognitive Demands and Bias: Challenges Facing Clinical Competency Committees.

T he Clinical Competency Committee is discussing what milestone level of professionalism to assign a resident. The discussion is reaching its end, and the Committee is about to designate a strong Level 4. Then 1 faculty member questions the rating, and mentions that the resident was 10 minutes late to clinic last week. Another member states that the resident missed a didactic session 3 months ago. A third faculty member pipes up and recalls that the resident didn’t read an assigned article last year. A fourth member says, ‘‘Not good. Let’s give her a Level 3. ’’ What happened? In 2013, the Accreditation Council for Graduate Medical Education (ACGME) charged training programs to form a Clinical Competency Committee (CCC) to develop a method to assess residents’ progression toward unsupervised practice, using the ACGME Milestones and based on residents’ evaluations. The CCCs form 1 element of an integrated assessment system and are an essential component of ACGME’s new accreditation system. The CCCs are expected to review residents’ development semiannually and to assign and report milestone levels. Milestones are explicit outcome data of developing competencies. In their determination of appropriate, resident-specific milestone levels, CCCs integrate information gleaned from rotation evaluations, specific competency assessments, tests, and other data. The CCC members deliberate and determine the milestone level to report to the ACGME. The goal of this process is to improve feedback to residents, to ‘‘enhance credibility of judgments about resident/ fellow performance,’’ and to aid in program improvement and faculty development. In CCC deliberations, members often need to make more than 20 milestone-level determinations per resident. This represents a high cognitive load, with the potential for decision-making fatigue, which degrades decision-making processes. During assessment and decision making, bias can and does occur across settings. A study of judicial decisions found that after eating, court judges give more lenient sentences. When asked if this is true, judges denied the tendency. These judges, while striving to be impartial, demonstrate the unconscious operation of biases. Another example comes from the marketing of wine. The list price of a bottle of wine influences the subjective appraisal of its taste. Bias also occurs in elections. People correctly predict election winners in fields of unknown candidates, based solely on their appearance. Even in the appraisal of the scientific merit of journal manuscripts, reviewers display implicit bias. Bias may be hardwired. In the example of wine pricing, the more expensive the wine, the more it is experienced as pleasant, the more there is activation in the orbitofrontal cortex. Just altering the price of wine alters drinkers’ experience and neuronal activity. Bias is normal and common. Bias typically has a negative connotation, but it does help us navigate our environment. Constantly monitoring for bias is not easy, especially during CCC meetings. (See the TABLE for examples of bias that can occur during CCC deliberations.) There are several strategies to help minimize the effect of bias on decision making. Recognition of bias and motivation to change are key initial steps. For CCC members, faculty development exercises may help raise awareness and build a shared vocabulary regarding bias (information provided as online supplemental material). Having CCC members role play scenarios written to elicit bias and label the types of bias is an option. This has been used successfully both within a local CCC and at a national workshop. To optimize the value of the data, many CCCs have turned to commercial resident management systems (RMS) to organize assessments and evaluations. Alternatively, a homegrown RMS can provide a DOI: http://dx.doi. org/10.4300/JGME-D-16-00411.1