Opinion ID: 771005
Heading Depth: 2
Heading Rank: 2

Heading: Failure to match test items to skills important to the job

Text: 115 As the majority notes, the third step of the Craig test requires employers to demonstrate by `professionally acceptable methods' that the selection device is `predictive of or significantly correlated' with the element of work behavior identified [by the job analysis]. Craig, 626 F.2d at 662 (quoting Albemarle Paper, 442 U.S. at 431). In other words, the employer must demonstrate not only that the skills identified in the job analysis are job related, but also that the test items actually measure those skills. In Contreras v. City of Los Angeles, for example, this requirement was satisfied by an independent examination review phase in which a new group of job experts reviewed each test question and decided whether it tested one of the critical elements identified in the preceding job analysis phase. 656 F.2d 1267, 1282 (9th Cir. 1981); cf. Craig, 626 F.2d at 667 (finding that testimony from the sheriff and a physical training instructor that height was important in commanding respect and applying restraint was insufficient to show by professionally acceptable methods that the height requirements were predictive of the ability to carry out the control functions of the job). As the majority acknowledges, a key requirement of [the] third step, a requirement essential to the proof of job relatedness, is that the validation method be professionally acceptable.  Contreras, 626 F.2d at 1282. Here, even assuming that the job analysis was sufficiently specific to generate skills important for all the positions in question, the CBEST is invalid because the CTC failed to demonstrate in a professionally acceptable manner that the test actually measures the skills identified in the job analysis. 116 The majority appears to concede that neither the Practitioner's Review study nor the Lundquist study attempted to match the test items to the job skills as required under Craig. Maj. op. at 13641. To support the district court's finding of validity, the majority cites evidence describing the process of test development employed by ETS. However, this evidence merely purports to show that the test items were expertly developed; it does not demonstrate, as the third step of the Craig test requires, that the final version of the CBEST administered to job applicants is predictive of or significantly correlated with the skills identified in the job analysis as important for the job. Where an employer seeks to use a test that has a disparate impact on minorities, it must present evidence that the test actually measures the skills that have been identified as important for the job, and may not rely solely on the fact that it used a well known and respected as a developer of standardized tests. Maj. op. at 13640 (quoting the report of Dr. William A. Mehrens). 117 As evidence that the test questions were matched  to the skills they were intended to measure, the majority points only to the Curriculum Matching Project, in which two ETS employees matched CBEST test specifications to material found in textbooks purportedly used in California public schools. The district court criticized this study as unscientific and not particularly helpful, but nevertheless upheld the test. AMAE v. State of California, 937 F.Supp. 1397, 1412 n.21 (N.D. Cal. 1996). We have explicitly held that it is essential that the item validation method be professionally acceptable. Contreras, 195 F.3d at 490.Unscientific and not particularly helpful tests clearly do not meet that standard. Thus, even under a clear error standard, the district court's determination that step three of the Craig test was satisfied requires reversal.