Opinion ID: 805464
Heading Depth: 6
Heading Rank: 2

Heading: Content Relatedness and Representativeness

Text: M.O.C.H.A. also contends that there was no proof that the generic sub-tests “measure[d] anything.” Appellants’ Br. 42. By this, we understand M.O.C.H.A. to be making three arguments, none of which is persuasive. First, M.O.C.H.A. submits that Buffalo failed to introduce statistical evidence demonstrating that an applicant’s success on the generic sub-tests is predictive of success as a fire lieutenant. This argument fails because the district court expressly found that the 1998 examination was content, not construct, validated, and that this content validation was an appropriate method for determining the examination’s job relatedness. See M.O.C.H.A. I, 2009 WL 604898, at –13. M.O.C.H.A. does not challenge these findings on appeal. 31 In Guardians I, we explained that construct validation is “frequently impossible” because it requires “a demonstration from empirical data that the test successfully predicts job performance.” 630 F.2d at 92; see Gulino v. N.Y. State Educ. Dep’t, 460 F.3d at 384 (discussing greater difficulty of construct validation relative to content validation). By contrast, content validation does not require this predictive validation study, and only obligates a test-maker to show that the Guardians I factors were satisfied. See Guardians I, 630 F.2d at 95. Thus, even if a predictive validation study would have been preferable, see 29 C.F.R. § 1607.14(C)(5) (stating that, “[w]henever it is feasible, appropriate statistical estimates should be made of the reliability of the selection procedure” that has been content validated), we cannot conclude that its absence was fatal to Buffalo’s defense in light of the district court’s uncontested finding that content validation was appropriate for determining the 1998 examination’s job relatedness. Second, M.O.C.H.A. asserts that the 1998 examination was not content related. See Guardians I, 630 F.2d at 97–98 (discussing content-relatedness requirement). In this sense, M.O.C.H.A. contends that the generic sub-tests were not reflective of a fire lieutenant’s tasks and, therefore, did not measure an applicant’s ability to serve in that position. The record shows, however, that in performing the job analysis, Steinberg conducted statewide surveys to determine the tasks a fire lieutenant performs and the SKAPs he would be expected to possess on the first day at that position. Based on the results of those surveys, Steinberg statistically ranked and linked the most critical tasks and SKAPs, which revealed that supervision, training, and understanding written material were related to the content of 32 the job. Cf. id. at 98 (describing process of identifying tasks and abilities in order to determine content relatedness). Further, based on those statistical results, consultation with the Fire Advisory Committee, and a comparison of nationwide test plans, Steinberg found that each sub-test area was important to the fire lieutenant position and should carry equal weight to ensure that the examination was content representative. Cf. id. at 99 (defining content-representativeness as proof that “the test measure[s] important aspects of the job, . . . but not that it measure all aspects, regardless of significance, in their exact proportions”). Having already determined that Steinberg’s underlying job analysis was suitable to the Buffalo fire lieutenant position, and in the absence of any evidence impugning Steinberg’s process of ranking and linking the task and SKAP surveys, we conclude that the evidence of Steinberg’s methodology was sufficient to support the district court’s findings that the 1998 examination was content related and representative. Finally, to the extent M.O.C.H.A. posits that questions on the generic sub-tests were unrelated to the sub-test areas, i.e., that questions on the supervision sub-test had nothing to do with supervision, no record evidence demonstrates such disjunction. Once Buffalo showed that the Civil Service Department used reasonably competent means to construct the 1998 examination, the district court was permitted to infer that the generic sub-test questions were, in fact, related to the generic sub-test areas, which had independently been shown to be content related and representative. Further, although M.O.C.H.A. received the 1998 examination questions in discovery, it never contested the relationship between test questions and the subject matters they were supposed to cover. In the absence of any evidence 33 undermining the inference that, based on the Civil Service Department’s reasonably competent test design, the 1998 examination’s questions were related to the areas they were supposed to test, the district court did not err in finding the examination job related and consistent with business necessity. Having identified no merit in M.O.C.H.A.’s various challenges to the finding of job relatedness, we conclude that the district court did not clearly err in determining that Buffalo had carried its burden at the second step of Title VII analysis. Accordingly, the district court properly entered judgment in favor of Buffalo on M.O.C.H.A.’s disparate impact challenge to the 1998 examination.14 14 Judge Kearse suggests that our affirmance of the district court’s ultimate job- relatedness finding will make it “virtually impossible” for municipalities to refuse to certify employment test results that have a proven disparate impact. See Dissenting Opinion, post at 5. Judge Kearse reasons that no municipality ever will have a “strong basis in evidence to believe it will be subject to disparate-impact liability,” the threshold necessary to discard test results and defend against a subsequent disparate treatment claim. Ricci v. DeStefano, 557 U.S. at 585. Our court has only begun to define this “strong basis in evidence” standard. See United States v. Brennan, 650 F.3d 65, 110–14 (2d Cir. 2011) (setting forth factors for determining whether municipality established “strong basis in evidence” defense, including whether there is “objectively strong evidence of non-job-relatedness,” which can be demonstrated by less than preponderance of evidence); id. at 144–45 (Raggi, J., concurring in judgment) (expressing doubt that “strong basis in evidence” can be satisfied by less-thanpreponderance showing). We have no occasion here to explore this issue further. We hold only that the district court, acting as fact finder after a bench trial, did not commit clear error in finding that a preponderance of the evidence showed that the 1998 examination was job related and consistent with business necessity. Whether such a relatively narrow and factdependent determination compels the broader legal conclusion that, at the time it certified the test results, the municipal employer lacked a “strong basis in evidence to believe it [was] subject to disparate-impact liability” is a question we leave for future courts to address. 34