A study of decision making: how faculty define competence
The authors studied decision making in evaluation of residents by 34 faculty members in a university training program by simulating the evaluation process. Twenty-seven hypothetical residents were described in terms of six criteria used by the American Board of Anesthesiology: Pre-operative Evaluation, Anesthetic Management, Technical Ability, Scholarship, Conduct in Stress, and Relationships with Others. Each criterion was studied at three performance levels: poor, average, and outstanding. A factorial design dictated how each resident was portrayed on an evaluation form familiar to the faculty. Each faculty member ranked the residents in order of clinical competence. His ranking was transformed by conjoint measurement into the percentage contribution of each criterion to the evaluation decision. Mean values (in percentages) were Pre-operative Evaluation 21, Anesthetic Management 22, Technical Ability 14, Scholarship 16, Conduct in Stress 18, and Relationships with Others 9. Underlying these values were four decision-making patterns differing in the perceived importances of the criteria. Although one pattern (41 per cent of faculty members) used all performance data supplied, the others used only three or five criteria. Only Pre-operative Evaluation and Anesthetic Management were part of all four patterns; three of the four patterns, used by 59 per cent of faculty members, apparently ignored Relationships with Others in their evaluations. Faculty age and subspecialty interest did not discriminate among evaluative patterns. This study suggests that there is no consensus on the operational definition of competence and that conjoint measurement is particularly appropriate for the analysis of complex decisions like the evaluation of competence.