Opinion ID: 3204484
Heading Depth: 2
Heading Rank: 2

Heading: The 2005 and 2008 Exams

Text: Born of these purposes and shaped by decades of Title VII litigation,2 the examinations at issue in this case allowed no room for the subjective grading of applications. The total score of a test-taker who sat for the promotional examination in 2005 or 2008 was determined by two components: an 80-question written examination scored on a 100-point scale and an education and experience (E&E) rating, also scored on a 100-point scale. The written examination counted for 80% of an applicant's final score and the E&E rating comprised the remaining 20%. Applicants needed an overall score of seventy to be considered for promotion. On top of the raw score from these two components, Massachusetts law affords special consideration for certain military veterans, id. § 26, and individuals who have long records of service with the state, id. § 59. The subject matter tested on the 2005 and 2008 examinations can be traced back to a 1991 validation study or job analysis report performed by the state agency responsible for compiling the exam.3 See 29 C.F.R. § 1607.14 (technical 2 The district court offered a detailed summary of this litigious history. See Lopez, 2014 U.S. Dist. LEXIS 124139, at –27. 3 The Officers argue that Boston misrepresented its reliance on the 1991 report and that the City, in fact, used only a lessthorough report conducted in 2000. The Officers' evidence for this consists of a comparison, in a footnote in their appellate - 10 - requirements for a content validity study under the Uniform Guidelines on Employee Selection Procedures); see also Watson v. Fort Worth Bank & Tr., 487 U.S. 977, 991 (1988) (opinion of O'Connor, J.) (Standardized tests and criteria . . . can often be justified through formal 'validation studies,' which seek to determine whether discrete selection criteria predict actual onthe-job performance.). That 1991 report was prepared by the Massachusetts Department of Personnel Administration (DPA), the predecessor to HRD. In preparing the report, DPA surveyed police officers in thirty-four jurisdictions nationwide, issuing a questionnaire that sought to ascertain the kinds of knowledge[], skills, abilities and personnel characteristics that police officers across the country deemed critical to the performance of a police sergeant's responsibilities. The report's authors distilled the initial results from this survey and their own knowledge regarding professional best practices into a list of critical police supervisory traits. They then distributed this list in a second survey to high-ranking police officers in Massachusetts, who were asked to rank these traits according to how important they felt brief, between three tested skill areas out of fifteen total areas on the 2008 outline of exam questions and similar language from the 2000 job analysis. We decline to find that this perfunctory, post-judgment sampling demonstrates that the district court committed clear error. - 11 - each was to a Massachusetts police sergeant's performance of her duties. DPA further refined the ranking of key skills and traits through focused small-group discussions with police sergeants and conducted a testability analysis of which skills could likely be measured through the written examination or the E&E component. In 2000, HRD engaged outside consultants to refresh the findings of the 1991 examination through a process similar to, though less thorough than, DPA's approach in 1991. The written question and answer component of the examination consisted of multiple choice questions that covered many topic areas, including the rules governing custodial interrogation, juvenile issues, community policing, and firearm issues, to name a few.4 The text of individual questions was often closely drawn from the text of materials identified in a reading list provided by the Boston Police Department (BPD) to testtakers in advance of the exams. For example, one question on the 2008 promotional exam asked applicants to accurately complete the following statement: According to [a criminal investigations textbook on the reading list], a warrantless search and seizure is acceptable: A. after stopping a vehicle for a traffic violation and writing a citation. 4 Boston supplemented the HRD-produced examination with additional jurisdiction-specific questions that sought to probe a candidate's knowledge of Boston-specific rules, orders, and regulations. - 12 - B. after obtaining the consent of the person, regardless of whether obtained voluntarily or nonvoluntarily.