Towards Understanding Why Assessment Centers Work: An Evaluation of the Subtle Criterion Contamination Hypothesis

TOWARDS UNDERSTANDING WHY ASSESSMENT CENTERS WORK: AN EXAMINATION OF THE SUBTLE CRITERION CONTAMINATION HYPOTHESIS Christopher T. Rotolo Old Dominion University. 1997 Director Dr. Robert M. McIntyre The success of the assessment center method in predicting job performance has been one of the most researched efforts in personnel psychology (Thornton, 1992). There is little reported evidence showing that assessment center procedures produce scores that serve as valid representations of separate constructs, or that those constructs are used in evaluation decisions in the manner proposed by assessment center designers (Klimoski & Brickner. 1987). It is perhaps ironic, then, that despite the success stories, we still do not understand why assessment centers "work. " (i.e., predict performance). This study was designed to examine the subtle criterion contamination hypothesis as an explanation to assessment center validity. The subtle criterion contamination hypothesis states that assessment centers predict managerial performance not because the assessors look for behavioral evidence of specific traits or dimensions, but in fact observ e and evaluate participants on the basis of their knowledge of those factors needed to get ahead in the company. These factors, it is argued, may not necessarily be related to performance in the assessment center but are instead relevant to the manager's promotability within the organization (Klimoski & Strickland. 1977). Descriptions of behavior along three job-relevant skill dimensions Analysis, Decisiveness/Decision Making, and Superv isory' Skills as well as two job-extraneous Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. cues ratee physical attractiveness and ratee sex served as the cues in a Brunswik (1955) lens model framework. Twenty-six experienced assessors and 20 supervisors from a county police department rated 32 profiles of fictitious ratees in a 2 (assessor vs. supervisor) x 2 (photo-present vs. Photo-absent) x 2 (high Analysis vs. low Analysis) x 2 (high Decisiveness/Decision Making vs. low Decisiveness/Decision Making) x 2 (high Supervisory Skills vs. low Supervisory Skills) mixed factorial design. For each profile, each rater made two ratings: (1) an evaluation of the ratee's overall performance: and (2) an evaluation of the ratee's future promotability in the organization. Results indicated that extraneous variables did not add significantly to the rating variance accounted for by the dimensions. However, the ratee photograph affected the weight that raters placed on the dimensions when making their evaluations. Further analyses revealed that ratee attractiveness and ratee sex had no impact on rater evaluations of ratee overall performance. However, ratee attractiveness significantly affected rater evaluations of ratee future promotability. Further, assessor decision strategies appeared to match those of their supervisor counterparts. These results suggest that subtle criterion contamination has minimal impact on assessment center validity. However, further research is encouraged to identify other potential extraneous factors that may have an impact on rater judgments. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.