Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 1 of 9

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

___________________________________

)

BRUCE SMITH, PAUL JOSEPH,

)

JOHN M. JOHNSON, ROBERT TINKER, )

MARTIN JOSEPH, KIM GADDY,

)

BRIAN KEITH LATSON, MARWAN )

MOSS, LEIGHTON FACEY, and

)

LATEISHA ADAMS,

)

)

Plaintiffs,

)

)

v.

)

)

CITY OF BOSTON, MASSACHUSETTS )

)

Defendant.

)

)

Civil Action No.: 12-10291-JLT

SECOND AMENDED COMPLAINT, COMPENSATORY, INJUNCTIVE AND DECLARATORY RELIEF REQUESTED

I. INTRODUCTION

1. This is an action by minority police sergeants for the Boston Police Department

who have taken the 2005 and/or 2008 Boston Police Lieutenants’ promotional exam and who

claim that such examinations are unlawful under Title VII of the Civil Rights Act of 1964 and

Mass. Gen. Laws ch. 151B because they have a severe disparate impact on minority test takers

and cannot be shown to be job related under applicable federal Equal Employment Opportunity

Commission guidelines. The Plaintiffs seek appropriate injunctive and compensatory relief as

described herein.

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 2 of 9
II. JURISDICTION 2. This Court has jurisdiction over this case under 28 U.S.C §§ 1331 and 1343 in
that this case arises under the laws of the United States of America, including particularly, Title VII of the Civil Rights Act of 1964, 42 U.S.C. §§ 2000(e), as amended. The Court has supplemental jurisdiction over pendent claims pursuant to 28 U.S.C. § 1367(a). III. PARTIES
3. Plaintiff Bruce Smith is an adult resident of Randolph, Massachusetts. He has been a Boston Police Officer since 1989 and a Police Sergeant since May of 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 78.
4. Plaintiff Paul M. Joseph is an adult resident of Boston, Massachusetts and is a Sergeant on the Boston Police force. He has held this position since 1997, and he has been a uniformed member of the Boston Police Department since 1989. In 2005 he took the Boston Police Lieutenants’ promotional exam and received a passing score of 70. In 2008 he took the promotional examination and received a score of 87.
5. Plaintiff John M. Johnson is an adult resident of Milton, Massachusetts. He is currently a Boston Police Sergeant, a position he has held since 1997. He has been a uniformed member of the Boston Police force since 1986. In 2005 he took the Boston Police Lieutenants’ promotional exam and received a score of 86.
6. Plaintiff Robert Tinker is an adult resident of Weymouth, Massachusetts. He has been a Boston Police Sergeant since 1988, and has been a uniformed member of the Boston Police Department since 1979. He took the Boston Police Lieutenants’ promotional exam in 2005 and received a score of 86.
2

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 3 of 9
7. Plaintiff Martin Joseph is an adult resident of Dorchester, Massachusetts. He is currently a sergeant on the Boston Police Department and has held this position since 2005. He has been a uniformed member of the police force since 1986. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a passing score of 77.
8. Plaintiff Kim Gaddy is an adult resident of Jamaica Plain, Massachusetts. She has been a police officer for the City of Boston since 1985, and she has been a sergeant since 1992. Plaintiff Gaddy took the 2008 Boston Police Lieutenants’ promotional exam and received a passing score in the 80’s.
9. Plaintiff Brian Keith Latson is an adult resident of Sharron, Massachusetts. He has been a Boston Police officer since 1985 and a Boston Police Sergeant since 2001. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 76.
10. Marwan Moss is an adult resident of Roslindale, Massachusetts. He has been a police officer since 1995 and a Police Sergeant since April of 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 87.
11. Leighton Facey is an adult resident of Boston, Massachusetts. He has been a Boston Police Officer since June 1998 and a Police Sergeant since September 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 88.
12. LaTeisha Adams is an adult resident of Dorchester, Massachusetts. She has been a Boston Police Officer since March 1995 and a Police Sergeant since June 2007. In 2008 she took the Boston Police Lieutenants’ promotional exam and received a score of 84.
13. The Defendant, City of Boston, is a duly incorporated municipality of the Commonwealth of Massachusetts. It runs and operates a large urban Police Department, and it
3

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 4 of 9
makes promotions in the Boston Police Department pursuant to M.G.L. ch. 31, the state Civil Services Law. IV. CLASS ALLEGATIONS
14. The named Plaintiffs, all of whom are African American, bring this action individually and on behalf of a class of individuals similarly situated. The class of individuals similarly situated is as follows: All black police sergeants for the Boston Police Department, who have taken the 2005 and/or 2008 Boston Police Lieutenant Promotional Examination or who plan to take such examination when next given.
15. The claims of the class plaintiffs are similar if not identical to the claims of the named plaintiffs in that each has challenged the validity of the Boston Police Lieutenants’ examination and promotional procedure and do so under the same legal theory – disparate impact.
16. The class meets all of the numerosity requirements of Rule 23 of the Federal Rules of Civil Procedure, as it would be impractical to name each individual plaintiff. In addition, the claims of the named plaintiffs are representative and typical of the claims of the class plaintiffs.
17. There are common questions of law and fact which predominate over any specific individual factual claims in this case, and plaintiffs’ counsel are eminently qualified to represent the claims of the named and class plaintiffs. This class is primarily certifiable under Rule 23(b)(2) of the Federal Rules of Civil Procedure in that Plaintiffs seek declaratory and injunctive relief generally applicable to all plaintiffs. The plaintiffs also seek damages in the form of back pay and other damages allowable by Title VII of the Civil Rights Act of 1964, but those claims are ancillary to the injunctive and declaratory relief sought by the plaintiffs.
4

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 5 of 9
18. This class should also be certified under Rule 23(b)(3) of the Federal Rules of Civil Procedure in that there are questions of law or fact which are common to the class which predominate over any questions affecting only individual members, and a class action is a far superior method of fairly and efficiently adjudicating this controversy. V. STATEMENT OF FACTS
19. All of the named plaintiffs are African-American police sergeants for the City of Boston Police Department, and each is well qualified for promotion to the position of Lieutenant on the Boston Police Department.
20. Each of the plaintiffs took either the 2005 or 2008 Boston Police Lieutenants’ promotional examination, or both, and each received a passing score on said examination, but has not been promoted to the position of lieutenant.
21. The promotional process for becoming a lieutenant on the Boston Police Department is as follows: First, the Boston Police Department, in conjunction with the Commonwealth of Massachusetts, Human Resources Division announces that there will be a Boston Police Lieutenants’ promotional exam consisting of approximately 100 multiple choice questions which are taken almost verbatim from texts and other written materials which relate to police work or supervision.
22. Each individual taking the examination receives a score based on the number of correct answers, scaled to 100 possible points. In order to be considered at all for a promotion, one must attain a passing score of 70. The final score consists of the results of the written multiple choice exams (which are based upon rote memory and taken directly from texts), and several points may be added or subtracted based upon one’s years of service and educational background, but no points are added for relevant forms of experience, including military service.
5

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 6 of 9
23. The final score is then used to rank each candidate in order of their score on the examination.
24. Promotions to the position of Boston Police Lieutenant are then made by the defendant in strict rank order based upon one’s rank on the promotional list.
25. For at least 30 years, the Boston Police Department, as well as all experts in the field, have been aware that such multiple choice rote memory examinations have a severe disparate impact on minority test takers, including African-Americans. In addition, the evidence is overwhelming that such examinations are not valid under appropriate federal and state EEOC guidelines, in that they are not job related – or predictive of actual ability to do the job. Further, there are many and multiple less discriminatory testing procedures which are equally or more valid as predicting job abilities, and which have a far less discriminatory impact on minority candidates.
26. Because the City of Boston has predominately administered such multiple choice police promotional examinations for the position of lieutenant over the last 25 years, the result is that currently despite the fact that more than 30% of the entire Boston Police Department is made up of minority police officers, there are only, on information and belief, 2 African-American Police Lieutenants on the entire Boston Police Department out of nearly 50 lieutenants, or less than 5% of the entire police lieutenants on the Boston Police Department. In addition as of March 1, 2012 the Boston Police Department will have no minority police captains.
27. In the fall of 2005, the Boston Police Department conducted a lieutenants’ promotional examination, which results were certified in the early spring of 2006. Promotions from that list continued to be made until early 2009. On information and belief, of all the lieutenant promotions made from that list, only one African-American was promoted.
6

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 7 of 9
28. In the fall of 2008, the Boston Police Department administered another promotional exam for the position of lieutenant, which list became certified in early 2009. Although many lieutenant promotions have been made from that list between 2009 and the present, to date there have been no minority promotions from that list despite the fact that there are many African-American police sergeants who passed the exam and were well qualified for the position.
29. Thus, as a direct result of the Boston Police Department’s use of written multiple choice examinations, the number of Boston Police Lieutenants who are African-American is far below the number of African-American Police Lieutenants that would be expected to be promoted had there been a fair lieutenants’ promotional examination procedure that did not have such a severe disparate impact upon minority officers. VI. EXHAUSTION OF ADMINISTRATIVE REMEDIES
30. Each of the named plaintiffs have filed complaints of discrimination relating to their failure to be promoted to the position of Boston Police Lieutenant with the Massachusetts Commission Against Discrimination and the Equal Employment Opportunities Commission, and each plaintiff has received right-to-sue letters.
COUNT I (Title VII of Civil Rights Act of 1964) 31. The conduct of the defendant, City of Boston, in conducting, administering, and utilizing a promotional examination system for the position of lieutenant in the Boston Police Department that has a severe disparate impact on minority candidates, cannot be shown to be job related and is not valid in accordance with EEOC guidelines is unlawful. In addition, there are many less discriminatory examination procedures which would have less discriminatory impact.
7

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 8 of 9
Thus the Defendant’s actions violate Title VII of the Civil Rights Act of 1964, 42 U.S.C. §2000(e).
COUNT II (Mass. Gen. Laws ch. 151B) 32. The conduct of the defendant, City of Boston, in conducting, administering, and utilizing a promotional examination system for the position of lieutenant in the Boston Police Department that has a severe disparate impact on minority candidates, cannot be shown to be job related and is not valid in accordance with EEOC guidelines is unlawful. In addition, there are many less discriminatory examination procedures which would have less discriminatory impact. Thus the Defendant’s actions violate Mass. Gen. Laws ch. 151B. WHEREFORE, Plaintiffs request this Honorable Court to issue appropriate injunctive, declaratory, and compensatory relief which 1) requires the Boston Police Department to utilize a nondiscriminatory promotional examination procedure for the position of lieutenant, 2) grants injunctive and declaratory relief preventing defendants from using their current promotional procedure, 3) grants plaintiffs injunctive and compensatory relief including back pay and other damages, including reasonable attorneys’ fees and costs, and 4) grants such other and further relief as this Court deems just and proper.
8

Case 1:12-cv-10291-WGY Document 14 Filed 09/04/12 Page 9 of 9

DATED: September 4, 2012

Respectfully submitted,
BRUCE SMITH, et al.
By their attorneys,
/s/ Stephen S. Churchill Harold L. Lichten, BBO # 549689 Stephen S. Churchill, BBO # 564158 Sara Smolik, BBO # 661341 Lichten & Liss-Riordan, P.C. 100 Cambridge Street, 20th Floor Boston, MA 02114 617-994-5800

CERTIFICATE OF SERVICE

I certify that a copy of the foregoing document has been served on counsel for all parties through the ECF system on this date.

DATED: September 4, 2012

/s/ Stephen S. Churchill Stephen S. Churchill

9

 
 Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 1 of 140
1

1

UNITED STATES DISTRICT COURT

2

DISTRICT OF MASSACHUSETTS (Boston)

3

No. 12-cv-10291-WGY

4

5 BRUCE SMITH, et al Plaintiffs
6

7 vs.

8

9 CITY OF BOSTON, Defendant
10

11 *********
12

13

For Trial Before:

14

Judge William G. Young

15 Bench Trial
16

17

United States District Court

District of Massachusetts (Boston)

18

One Courthouse Way

Boston, Massachusetts 02210

19

Monday, December 22, 2014

20

21

********

22

REPORTER: RICHARD H. ROMANOW, RPR

23

Official Court Reporter

United States District Court

24

One Courthouse Way, Room 5510, Boston, MA 02210

bulldog@richromanow.com

25

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 2 of 140
2

1

A P P E A R A N C E S

2

3 HAROLD L. LICHTEN, ESQ.

BENJAMIN WEBER, ESQ.

4

Lichten & Liss-Riordan, P.C.

100 Cambridge Street, 20th Floor

5

Boston, MA 02114

Email: Hlichten@llrlaw.com

6 and

STEPHEN S. CHURCHILL, ESQ.

7

Fair Work, P.C.

192 South Street, Suite 450

8

Boston, MA 02111

Email: Steve@fairworklaw.com

9

For plaintiffs

10

GEOFFREY R. BOK, ESQ.

11 KAY H. HODGE, ESQ.

JOHN M. SIMON, ESQ.

12

Stoneman, Chandler & Miller

99 High Street

13

Boston, MA 02110

Email: Gbok@scmillp.com

14

For defendant

15

16

17

18

19

20

21

22

23

24

25

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 3 of 140
3

1

I N D E X

2

3 WITNESS

DIRECT CROSS REDIRECT RECROSS

4

5 MICHAEL A. CAMPION (Continued.)

6

By Mr. Simon:

7

By Mr. Lichten:

5

8

9

10

11 E X H I B I T S
12

13

14 EXHIBIT 82........................................ 109

15

16

********

17 EXHIBIT J......................................... 25
18 EXHIBIT K......................................... 30
19

20

21

22

23

24

25

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 4 of 140
4

1

P R O C E E D I N G S

2

(Begins, 9:15 a.m.)

3

THE COURT: Good morning, counsel. And you may

4 remind the witness.

5

THE CLERK: I'd like to remind you that you're

6 still under oath.

7

THE WITNESS: Yes.

8

THE COURT: And, Mr. Lichten, you may continue --

9 actually you may commence.

10

MR. LICHTEN: Your Honor, as the Clerk may have

11 told you, there's a small issue still with this

12 deposition. Do you want to take that up?

13

THE COURT: What is it?

14

MR. LICHTEN: Dr. Hough filed her report at 7:00

15 this morning. She's arranged to be available by

16 teleconferencing deposition that we will pay for on the

17 28th, 29th, and 30th, in the middle of her vacation in

18 Colorado. Alternatively she's agreed to cut short her

19 vacation and get here on the 5th, that would be if they

20 want to do it in person, then she can testify a day or

21 two later. They interpret your order as not permitting

22 either of those and the defendants won't agree to either

23 of those.

24

THE COURT: Well, on reflection it seems to me you

25 can take her by skype, if you want, that's okay, if she

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 5 of 140
5

1 can do it by teleconference, she can do it by skype, but

2 I'm not going to require her to come to Boston. That's

3 satisfactory. So that resolves that.

4

You've reminded the witness?

5

THE CLERK: Yes, your Honor.

6

THE COURT: Go ahead, Mr. Lichten.

7

8 CROSS-EXAMINATION BY MR. LICHTEN:

9 Q.

Good morning, Dr. Campion. I hope you've had a

10 good weekend.

11 A.

Thank you.

12 Q.

Dr. Campion, you have a consultant resume as well

13 as a professional resume, is that correct?

14 A.

Well, I have an academic and a consulting resume.

15 Q.

Right. And some of the things you list on your

16 consultant resume are "employment interviewing" and

17 "assessment testing," is that correct?

18 A.

Oh, yes.

19 Q.

"Performance management"?

20 A.

Yes.

21 Q.

"Litigation support"?

22 A.

Yes.

23 Q.

"Skill career development"?

24 A.

Yes.

25 Q.

"Compensation and reward systems"?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 6 of 140
6

1 A.

Yes.

2 Q.

"Surveys"?

3 A.

Yes.

4 Q.

"Validation and legal defensibility"?

5 A.

Right.

6 Q.

"Work team and organizational design"?

7 A.

Yes.

8 Q.

"Training and management development"?

9 A.

Yes.

10 Q.

"Turnover management," is that right?

11 A.

Yes.

12 Q.

And "Recruiting services"?

13 A.

That's right.

14 Q.

Okay. And then you list -- on the third page you

15 list some of your consultive clients and you've

16 consulted for a wide array of private companies in the

17 United States, very large companies, is that right?

18 A.

Yes.

19 Q.

Now, you have never before been an expert in a

20 police promotional case, is that right?

21 A.

Yes, that's right.

22 Q.

And you have never before constructed a police

23 promotional process, is that correct?

24 A.

Not police specifically, no.

25 Q.

Okay. Well, whether it's state police or local

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 7 of 140
7

1 police in a city, you've never constructed a police

2 promotional process for any local or state police, is

3 that correct?

4 A.

Not police specifically, no.

5 Q.

Okay. And you've never before been involved in a

6 police case, is that right?

7 A.

Do you mean a case that is in -- a law case?

8 Q.

Yes.

9 A.

No.

10 Q.

No, that is correct or it is not correct?

11 A.

I'm sorry. That is correct.

12 Q.

Okay. Now, you have had some involvement with --

13 let's see if I understand this, the Department of State

14 and the arm of the Department of State that's

15 responsible for security as well as many other positions

16 at the Department of State, is that correct?

17 A.

Yes.

18 Q.

And you've overseen, um, the way they do

19 promotional -- promotions at the state department with

20 respect to that arm in the State Department that does

21 law enforcement for the State Department, is that

22 correct?

23 A.

Yes.

24 Q.

Okay. And I think you described in your

25 deposition, I think you said those are the people with

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 8 of 140
8

1 the earplugs or the earphones in their ears and the big

2 sports coats over them?

3 A.

To hide the gun, right.

4 Q.

Yes, to hide the gun.

5 And the promotional process that you've overseen for the

6 State Department involves -- is a highly-structured

7 process, would you agree with that?

8 A.

Oh, yes.

9 Q.

Okay. And it -- and the reason it's

10 highly-structured is to ensure fairness, is that right?

11 A.

Yes, consistency as well as transparency.

12 Q.

Got you. And during the time you've overseen this

13 there's probably been hundreds of such promotions, is

14 that fair?

15 A.

Yes, not just diplomatic security, but more

16 broadly, yes.

17 Q.

Well, let's just talk about diplomatic security,

18 hundreds or less?

19 A.

Well, I don't directly oversee their promotion

20 boards, but I work with the Department in the State

21 Department that manages the promotion board process.

22 Q.

Okay. And there are no written examinations in

23 the State Department law enforcement arm promotional

24 process, is that correct?

25 A.

Yes, not for promotion, but for hiring.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 9 of 140
9

1 Q.

All right. I'm just talking about promotions

2 right now. But not for promotions, is that correct?

3 A.

Not for promotions, no.

4 Q.

Okay. And were you involved -- but it does have

5 structured oral interviews, is that correct?

6 A.

No.

7 Q.

Okay. Now, you've been involved over your career

8 in the design of structured oral interviews, is that

9 correct?

10 A.

Yes.

11 Q.

And in fact you published on the subject, is that

12 correct?

13 A.

Yes.

14 Q.

And it's actually listed as one of your primary --

15 I don't know if "life's work" is the way to say it, but

16 one of your primary interests is the structured oral

17 interview, is that correct?

18 A.

I believe so.

19 Q.

Okay. And as I understand it, for many years,

20 maybe 25, 30 years ago, scientists such as yourself were

21 concerned about using interviews to promote people

22 because of the possibility of subjectively or cronyism,

23 is that right?

24 A.

I'm sorry. Could you restate that?

25 Q.

Sure. Would you agree that before you began

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 10 of 140
10

1 publishing in this field about those highly-structured

2 interviews there was concern that oral interviews might

3 not be a good way to promote people because of the

4 possibility of bias and cronyism?

5 A.

I don't know that.

6 Q.

Okay.

7 A.

The interview had been used previously, but I

8 don't know that there were concerns specifically about

9 the interview.

10 Q.

Okay. Well, how about the rating of the scores

11 from the interview?

12 A.

I think the concern was with cronyism, but it

13 wasn't specifically that interviews are good or bad.

14

THE COURT: I'm not clear how you're parsing it.

15 One would think that an interview process, um, however

16 structured, has certain risks of subjectivity that a

17 written exam would not have, right?

18

THE WITNESS: Absolutely, yes.

19

THE COURT: All right. And among those, though

20 you try to get independent people and the like, are

21 cronyism?

22

THE WITNESS: Oh, for sure. It could be all kinds

23 of biases.

24

THE COURT: Right. Political partisanship?

25

THE WITNESS: Sure.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 11 of 140
11

1

THE COURT: Covert bias or unrecognized bias to be

2 attracted to the like and turned off by those who are

3 different from us, those things?

4

THE WITNESS: Surely hypothetically that's true.

5

THE COURT: All right. Go ahead.

6

MR. LICHTEN: Thank you, your Honor.

7 Q.

But then along came yourself and others and you

8 began, um, stating in literature that you believed that

9 if interviews were highly-structured and subject to

10 various conditions they could become -- they could have

11 validity, is that correct?

12 A.

Oh, yes.

13 Q.

Okay. In fact you developed, for a number of your

14 clients, highly-structured interviews that you believe,

15 because of the precautions taken in the structural

16 interview, have good validity, is that right?

17 A.

Yes.

18 Q.

(Pause.) I just wanted to ask you very quickly

19 about something.

20 On your resume you say that your GRE scores were "fairly

21 average." You say that. That's not me trying to impugn

22 you in any way, because I'm not. But you say that, is

23 that correct?

24 A.

No.

25 Q.

You don't say that your -- you haven't published

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 12 of 140
12

1 something that says your GRE scores were fairly average?

2 A.

That's not my resume.

3 Q.

What?

4 A.

That's not my resume.

5 Q.

What is it on?

6 A.

Well, when I was president of my professional

7 association they asked us to write an autobiography to

8 sort of inspire or historically document something about

9 our lives, and so that was actually an essay that I

10 wrote after I was president of my professional

11 association, I was describing my education and graduate

12 career and I made that statement in that essay.

13

THE COURT: Is it true?

14

THE WITNESS: Well, it wasn't as high as I wanted

15 it to be. Let's put it that way.

16 Q.

And in fact for graduate school you went to the

17 University of Acron, is that correct?

18 A.

For my master's, yes.

19 Q.

Okay. Now, I don't mean anything bad about this

20 because I understand you're now a top scientist in the

21 field of industrial organizational psychology, is that

22 correct?

23 A.

I've had a very good career.

24 Q.

So you would agree that the GRE was not a great

25 predictor of how you would do in the field of

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 13 of 140
13

1 organizational psychology, is that correct?

2

MR. SIMON: Objection, your Honor.

3

THE COURT: Yeah, sustained. I don't think that's

4 relevant. Any more relevant than my musings that the

5 LSATs are very good at predicting law school work and

6 not at all good at predicting what happens after law

7 school. Sustained.

8

MR. LICHTEN: Sure, your Honor.

9 Q.

Now, at your deposition you mentioned that one of

10 the recent promotional processes that you designed is

11 for Walgreens, which I think we all know. Is that

12 correct?

13 A.

Yes.

14 Q.

And Walgreens is a huge company that has

15 pharmacies all over the country, is that right?

16 A.

Yes.

17 Q.

And you've designed, um -- you've been involved in

18 the design of the promotional process for store managers

19 and regional managers and then higher-level officials,

20 is that correct?

21 A.

Yes.

22 Q.

And that promotional system involves an appraisal

23 system, is that correct?

24 A.

In part, yes.

25 Q.

It involved an interview process, is that right?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 14 of 140
14

1 A.

Yes.

2 Q.

And it involves past job performance, is that

3 correct?

4 A.

Yes, among other things.

5 Q.

And then it involves some test for math skills,

6 isn't that right?

7 A.

Yes.

8 Q.

And you designed that process to, Number 1, try to

9 make it valid or more valid, is that right?

10 A.

Yes.

11 Q.

And to reduce adverse impact, is that right, that

12 was your purpose?

13 A.

Well, no, mainly it was to ensure its validity.

14 Q.

But you have reduced adverse impact with that

15 process, is that correct?

16 A.

I don't know that historically it has reduced

17 adverse impact, no, I don't know that.

18 Q.

Okay. Now, if you think about all the promotional

19 procedure that you've been involved in designing in the

20 last 10 years, none of them involved the use solely or

21 exclusively of a multiple choice job knowledge test, is

22 that correct?

23 A.

No, that's incorrect.

24 Q.

Do you remember testifying in a deposition?

25 A.

Do I remember testifying?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 15 of 140
15

1 Q.

Yes.

2 A.

Yes.

3 Q.

(Hands deposition to witness.) Okay. Do you

4 remember me asking you, "In the last 10 years" -- and

5 I'm quoting now from the deposition, -- "is there any

6 promotional process that you have designed anywhere that

7 has only, as its component, a written multiple choice

8 job knowledge test." Do you see that?

9 A.

No. Where am I looking?

10 Q.

Page 28 at the bottom. I'm sorry.

11 A.

(Looks.) Well, okay, so, I'm sorry, but what --

12 I'm on 28?

13 Q.

At the bottom of Page 28 I asked you, did I not,

14 and I'll try to quote, "In the last 10 years is there

15 any promotional process that you have designed anywhere

16 that has only, as its component, a written multiple

17 choice job knowledge test." Do you see that?

18 A.

I'm sorry I don't. What line number is it?

19 Q.

And just so we're clear I'm using the page numbers

20 at the bottom of the page.

21 A.

Oh, I'm sorry, there are page numbers at two

22 locations. You're at the bottom. Okay. There are

23 those other ones. So 28 at the bottom. Yes, I do see

24 that.

25 Q.

Well, I'll say it one more time. "In the last 10

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 16 of 140
16

1 years is there any promotional process you have designed

2 anywhere that has only, as its component, a written

3 multiple choice job knowledge test." Do you see that?

4 A.

Yes.

5 Q.

And your answer was "Probably not, but you

6 remember I have done a lot, but I have not done a police

7 promotion." Was that your answer?

8 A.

Yes, but you'll see later in the page that I

9 clarify that I did develop promotional exams for patent

10 and trademark examiners that was based solely on a

11 written exam.

12 Q.

Okay. And was that a hiring process or a

13 promotional process?

14 A.

No, that's promotion to what they call "signature

15 authority" where they can give early patent approval and

16 it's a promotional exam that they must pass.

17 Q.

So your testimony is that the only promotional

18 examination process that you've been involved in in the

19 last 10 years that uses exclusively a written multiple

20 choice test is patent examiners?

21 A.

That's the only one I can remember right now, but

22 remember I've done 900 projects, it's hard to keep them

23 all straight. I don't believe there are many others,

24 but that one I think is -- was just a job knowledge test

25 that we used.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 17 of 140
17

1 Q.

Okay. And with respect to that, the patent

2 examiners, you used an open book test process, is that

3 correct?

4 A.

Well, what I think I testified here is I could not

5 recall precisely, but that they do -- the nature of the

6 work is such that if they need to look something up they

7 can and that that was the key consideration. But I

8 don't actually remember the exam instructions. But I

9 seem to recall that they could. But again I'm not 100

10 percent.

11 Q.

So as you sit here today your best belief is that

12 in this patent examiner process, um, that you were

13 involved in the design of a, um -- that people taking

14 the exam were allowed to use an open book, is that

15 right?

16 A.

No, I'm saying I don't remember. But it is

17 possible because on the job they can look things up and

18 so that would then reflect how the job is performed.

19 But I honestly don't remember. If I could add, at the

20 time --

21 Q.

I'm not asking you to add anything right now. I

22 just want to get that clear.

23

So is it your testimony that if someone, um,

24 looking for a promotional job, um, could answer

25 questions by looking them up in some quick fashion, that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 18 of 140
18

1 would suggest to the industrial organizational

2 psychologist designing the exam that you would use an

3 open book format, is that correct?

4 A.

All I said was that was a consideration. There

5 are other considerations as well.

6 Q.

Well, are you aware -- let me ask you this way, of

7 other police jurisdictions that utilize an open book

8 examination process?

9 A.

No, I have not studied that. No.

10 Q.

So you don't know one way or another, is that

11 correct?

12 A.

No, I do not know.

13 Q.

But that is an alternative to have an open book

14 process, is that correct?

15 A.

Yes, but it changes the skill that you're

16 measuring.

17 Q.

So what is the skill that you're measuring if

18 you're utilizing an open book examination process?

19 A.

Well, it changes it from measuring knowledge to

20 measuring the ability to find information quickly, which

21 is a different -- it's a different kind of aptitude.

22 Q.

But when you're measuring on a multiple choice

23 test job knowledge that's taken directly out of text and

24 rules, you're measuring memorization, are you not?

25 A.

Well, certainly you're measuring a person's

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 19 of 140
19

1 knowledge and you can call it "memorization" if you'd

2 would like, but it's their possession of facts.

3 Q.

Were you aware that on this test -- and I think

4 you said this already, the answers came right out of the

5 text or rules that people were asked to study on the

6 reading list, is that correct?

7 A.

Yes. Uh-huh.

8 Q.

So those that are good at memorization would have

9 a better chance of memorizing what came out of that text

10 when they see the answer of one of the possible multiple

11 choice answers, is that right?

12 A.

It's one of the many aptitudes that would play a

13 role, yes.

14 Q.

(Pause.) Now, as you sit here today or as you sat

15 there at the deposition a couple of days ago, you're not

16 aware of any large police jurisdiction in the United

17 States, city or state, that uses only a multiple choice

18 job knowledge test to promote its officers to the

19 position of lieutenant, is that correct?

20 A.

I have not studied that, no.

21 Q.

So the answer to my question is you're not aware

22 of any such jurisdictions, is that correct?

23 A.

I don't know one way or the other.

24 Q.

Okay.

25

(Pause.)

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 20 of 140
20

1

MR. LICHTEN: May I approach, your Honor?

2

THE COURT: You may.

3

(Hands to witness.)

4 Q.

Very quickly, Doctor, you provided a report

5 regarding a case involving the Dallas Morning News, is

6 that correct?

7 A.

Yes.

8 Q.

Okay. And I'm showing you that report, it's

9 Plaintiffs' Exhibit A, "Declaration of Dr. Michael

10 Campion, PhD," and attachments. Do you see that?

11 A.

Yes.

12 Q.

It looks like that was filed in court on August

13 30th, 2010, is that right?

14 A.

I don't remember the date.

15 Q.

Now, if you turn to Page 30.

16 A.

(Turns.) Which 30?

17 Q.

Page 30 at the top right-hand corner.

18 A.

All right.

19 Q.

And Paragraph 48. Do you see that?

20

THE COURT: Not on Page 30, it would be on 29.

21

MR. LICHTEN: I'm sorry, your Honor, I have bad

22 eyesight. It's Paragraph 49.

23 A.

(Turns.)

24 Q.

Do you have that?

25 A.

Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 21 of 140
21

1 Q.

And you talk about the results of job performance

2 "goes from a marginally significant or just short of

3 marginally significant to a nonsignificant predictor of

4 termination decisions," is that correct?

5

MR. BOK: Objection, your Honor, um, just a note

6 on the record that they're now moving to one area which

7 was not covered by this expert. So previously you did

8 not allow the objection but that you warned counsel that

9 they were opening doors.

10

THE COURT: Noted and thank you. He may have the

11 question.

12

Do you see that, Dr. Campion?

13

THE WITNESS: Oh, yes, your Honor.

14

THE COURT: Go ahead, Mr. Lichten.

15 Q.

The marginally significant number you were

16 referring to is the T equals .055, is that correct?

17 A.

No. Oh, yes.

18 Q.

What was the marginally significant number you

19 were referring to?

20 A.

No, I'm sorry, I've not read this in four years.

21 I need a moment to study it.

22 Q.

Dr. Campion, I asked you about this in your

23 deposition a couple of days ago, did I not?

24

THE COURT: Well, he's on the stand now and if he

25 needs a moment before he answers, I'm going to allow

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 22 of 140
22

1 that. What he said in his deposition he said.

2

MR. LICHTEN: Sure.

3 A.

(Reads.) Okay, sir.

4

THE COURT: Go ahead, Mr. Lichten.

5 Q.

This marginally significant finding that you were

6 referring to is the .055 finding, is that correct?

7 A.

Yes. Uh-huh.

8

(Pause.)

9

MR. LICHTEN: May I approach, your Honor?

10

THE COURT: You may.

11

(Hands to witness.)

12 Q.

You also provided an expert report in a case

13 called Carlo vs. Pittsburgh Glass Works, is that

14 correct?

15 A.

Yes.

16 Q.

And that was filed with the court in August of

17 2013, is that right?

18 A.

Yes.

19 Q.

Okay. And if you go to Page 10. On the top right

20 it says "Page 10," do you see that?

21 A.

Do I see a Page 10?

22 Q.

Yes.

23 A.

Yes. Uh-huh.

24 Q.

And if you go to the second paragraph you say --

25 this is the last sentence of the second paragraph, "The

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 23 of 140
23

1 ratio is .66." And, by the way, when you say the ratio

2 is .66, you're talking about the adverse impact ratio,

3 is that correct?

4 A.

Yes.

5 Q.

And so that ratio would have been -- that would

6 have been in violation of a four-fifths rule, is that

7 correct?

8 A.

Yes.

9 Q.

Okay. And then you say, "But the standard

10 deviation at the 40 level falls just short of 2.0 at

11 1.72, which is significant at the 9 percent level

12 two-tailed test," is that correct?

13 A.

Yes.

14 Q.

And if you go up to the last sentence of the

15 paragraph before that, you say, "The ratio is .68," and

16 that's the adverse impact ratio, is that correct?

17 A.

Yes.

18 Q.

-- "but the standard deviation at the 40 level

19 falls just short of 2.0 at 1.54, which is significant at

20 the 13 percent level for a two tailed test," is that

21 correct?

22 A.

Yes.

23

MR. LICHTEN: Your Honor, do you want me to try to

24 put these reports in or just leave them as standing in

25 the record?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 24 of 140
24

1

THE COURT: It's your case to try. I can't --

2

MR. LICHTEN: Just for completeness, I would move

3 to put them in.

4

THE COURT: Right. Any objection?

5

MR. BOK: Yes, your Honor. These should only be

6 marked for identification, they're otherwise hearsay.

7

THE COURT: Yeah, if he's going to object, they

8 seem to be hearsay, and unless I can see an exception,

9 we'll leave them marked for identification. But we will

10 mark them for identification.

11

And the next letters are what?

12

THE CLERK: H.

13

THE COURT: H. And we will mark Appendix A in

14 this case 06cv1 -- 06cv01960-BF as in Exhibit H, and the

15 document from 10cv01283-TFM as Exhibit I for

16 identification. Go ahead, Mr. Lichten.

17

MR. LICHTEN: May I approach, your Honor?

18

THE COURT: You may.

19

MR. LICHTEN: Madam Clerk.

20

(Hands up.)

21

(Exhibits H and I, marked.)

22 Q.

You also provided expert testimony or an expert

23 report in the case of Brand vs. Comcast, is that

24 correct?

25 A.

Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 25 of 140
25

1 Q.

And you were the expert for the plaintiffs in that

2 case, is that correct?

3 A.

Yes.

4 Q.

Actually in this case, Brand, and in two cases

5 I've just provided to you, you were the expert for the

6 plaintiffs in those cases, is that correct?

7 A.

Yes.

8 Q.

Okay. So the plaintiffs would have an interest in

9 finding statistical significance of hiring differences

10 or promotional differences, is that correct? The

11 plaintiffs want to win their cases.

12 A.

Um, I don't know. I never met any of the

13 plaintiffs.

14 Q.

Okay.

15

THE COURT: No, but you understood that in each of

16 these engagements a finding of disparate impact would be

17 in the plaintiff's interest?

18

THE WITNESS: Oh, sure.

19

THE COURT: Yeah.

20

We'll mark this J for identification, this next

21 one.

22

Go ahead, Mr. Lichten.

23

MR. LICHTEN: Okay.

24

(Exhibit J, marked.)

25 Q.

If you can go to Page 22 and Paragraph C at the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 26 of 140
26

1 bottom.

2 A.

(Turns.) Yes.

3 Q.

And you say here that for -- now, in this case

4 you're looking at an issue of hiring or promotions

5 regarding blacks and whites, which is it?

6 A.

I believe these were promotions.

7 Q.

Promotions.

8

THE COURT: Forgive me, but I need the page again.

9

MR. LICHTEN: It's Page 22, your Honor.

10

THE COURT: Thank you.

11 Q.

So this is a case involving promotions, is that

12 correct?

13 A.

Yes.

14 Q.

Between blacks and whites?

15 A.

Yes.

16 Q.

And what you said in Paragraph C is, the second

17 sentence, "The ratio is less than .80," so again you're

18 addressing the fact that there's a violation of the

19 four-fifths rule because the rate is below .80, is that

20 correct?

21 A.

Yes, that's one of the statistics we will report.

22 Q.

Got you. And you said the standard deviation

23 exceed 2 for the promotions, from C-3 to C-4, and then

24 you say the ratio is less than .80 and the standard

25 deviation is marginally significant at 1.63 P equals

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 27 of 140
27

1 .103 for the promotions from C-4 to 5, is that correct?

2 A.

Yes.

3 Q.

Now, in your deposition do you recall me asking

4 you and you told me that marginally significant can

5 occur between P .05 and P .10?

6 A.

Generally, although sometimes around.

7 Q.

Okay. And in this case you rounded the .03 to 1,

8 is that correct, the 1.03 to 1?

9 A.

Yes, as we discussed in my deposition.

10 Q.

Okay. And then if you go to the next page, Page

11 23.

12 A.

(Turns.)

13 Q.

Again at the bottom of Paragraph B, you say, at

14 the last sentence, "Also the promotion rates for blacks,

15 from C-4 to 5, is 35 percent of the promotion rates for

16 whites, so that would be in violation of the four-fifths

17 rule," is that correct?

18 A.

Yes.

19 Q.

This is marginally significant at the 1.95,

20 parens, P equals .051, is that correct?

21 A.

Yes.

22 Q.

Okay. Now, you also provided an expert report in

23 the case of Ernst vs. The City of Chicago, is that

24 correct?

25 A.

Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 28 of 140
28

1 Q.

And again you were hired by the plaintiffs in that

2 case, is that right?

3 A.

Yes.

4 Q.

And, as I understand it, Ernst vs the City of

5 Chicago was a hiring case involving females who claimed

6 that the physical agility test that the City of Chicago

7 was making them pass had an adverse impact on them and

8 was not valid, does that capture what the case was

9 about?

10 A.

Yes.

11 Q.

And you opined both on the adverse impact issue

12 and you also opined on the validity issue and you also

13 opined on the less discriminatory alternative issue, is

14 that correct?

15 A.

Yes.

16 Q.

And you believed that the test was not valid and

17 that there were other less discriminatory alternatives,

18 is that correct?

19 A.

Yes.

20 Q.

Okay. Now, in that case the physical agility test

21 had actually been designed for the City of Chicago by a

22 woman named Dr. Debra Gephart, is that right?

23 A.

Yes.

24 Q.

And Dr. Gephart was fairly well-known in the field

25 of designing physical agility tests for public safety

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 29 of 140
29

1 positions, is that right?

2 A.

Um, I suppose.

3 Q.

And, um, she purported to justify the test not

4 based on content validity but a type of criterion

5 validity, is that right?

6 A.

No.

7 Q.

How did she purport to justify the tests?

8 A.

Well, she correlated her tests with other tests.

9 Q.

That she had done?

10 A.

That she had done.

11 Q.

Okay.

12 A.

So it's correlating tests with tests, which does

13 not constitute criterion validity or content validity.

14 Q.

Okay. Now, what you opined in that case is that

15 you didn't believe the test was valid because she only

16 tested for certain physical attributes of the job that

17 would disproportionately impact women such as large

18 muscle group activities, is that right?

19 A.

No.

20 Q.

That's not right?

21 A.

No.

22 Q.

Okay. So can I -- well, I'll give you the report.

23

MR. LICHTEN: May I approach, your Honor?

24

THE COURT: You may.

25

(Gives report.)

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 30 of 140
30

1

THE COURT: We'll mark this K for identification.

2

(Exhibit K, marked.)

3 Q.

If I could draw your attention to Page 8, please.

4 A.

(Turns.) Yes.

5 Q.

Okay. First, it, um -- if you'd go to the second

6 sentence on Page 8, you say, quote, "It appears that the

7 job analysis of the CFD paramedic job by Dr. Gephart was

8 conducted in a traditional manner," is that correct?

9 A.

Yes.

10 Q.

So she did perform a job analysis, is that

11 correct?

12 A.

Yes.

13 Q.

You then say: "However, Dr. Gephart made a number

14 of decisions that may have influenced the findings and

15 likely overestimated the physical ability requirements

16 for the job which led to the development of a selection

17 procedure, i.e. the physical agility test, that

18 increased adverse impact against women."

19 Is that correct?

20 A.

Right.

21 Q.

And then you list some of the issues you have and

22 you say, Number 1, "The physical tests are

23 overemphasized in the job analysis," is that correct?

24 A.

Right.

25 Q.

Okay. And if we were to take an analogy to that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 31 of 140
31

1 to the present case, in this case only job knowledges

2 were tested for from the Boston Police Department in

3 2008, is that right?

4 A.

I don't see the analogy.

5 Q.

But other aspects of the job that weren't tested

6 for just, in the Chicago Fire Department EMT case, were

7 those abilities and skills that might not have had such

8 an adverse impact on minorities, isn't that correct?

9 A.

No.

10 Q.

But in this fire department case didn't you opine

11 that they should have tried to test for those attributes

12 that might have had less discriminatory impact on women

13 such as small muscle activities such as motor skills and

14 things like that, that had a lesser adverse impact,

15 didn't you say that?

16 A.

I'm sorry, I thought we were talking validity and

17 then you had changed it to job analysis and now you're

18 talking alternatives. I'm sorry but I'm confused as to

19 what the question is that I'm trying to answer.

20 Q.

I'm just reading from your report, Dr. Campion.

21 You said on Page 8: "The physical requirements were set

22 based on the most physically demanding tasks and only a

23 small portion of all the tasks of the job," didn't you

24 say that in Paragraph 2?

25 A.

Yes, but I don't understand how it relates to the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 32 of 140
32

1 current question.

2 Q.

Well, in the Boston Police Department case you

3 understand that there was no attempt to test for skills

4 and abilities that were important to being a police

5 lieutenant but that might have had a less discriminatory

6 impact upon minority candidates, isn't that right, you

7 understood that, didn't you?

8 A.

Yes, but I don't think see the link between the

9 two. I'm sorry.

10 Q.

Okay?

11 A.

It would be like if the job knowledge exam was

12 based on the most complex knowledge you would ever have

13 to know on the job, then that would be an analogy to

14 what happened here in Chicago.

15 Q.

Well, that's not really true, is it? The test

16 involved certain physical tests that the candidate had

17 to perform which had been studied as being part of the

18 job of an EMT, isn't that right?

19 A.

No, I'm saying you'd be wrong, you don't

20 understand what they did in the job analysis.

21 Q.

So you deny that Dr. Gephart studied the job and

22 then tried to simulate portions of the job that were

23 necessary to perform the job?

24 A.

But she focused on the most -- the most difficult

25 physical aspects that were a very small part of the job.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 33 of 140
33

1 She didn't focus on all of it, she only focused on the

2 most difficult. And that's why it would be, in the

3 context of Boston, focusing on just the most complex job

4 knowledge you'd ever have to know in developing the exam

5 around that, which is not what I understand occurred.

6 Q.

And Boston did not focus on all of the job, right?

7 A.

Well, as I say it's a separate question.

8 Q.

Well, you just said that in Chicago they did not

9 focus on all of the job, they focused on the hardest

10 physical tasks?

11 A.

Within a domain of aptitude they focused on the

12 most physical demanding of the physical attributes, they

13 didn't look at any other attributes, is that what you're

14 asking?

15 Q.

Well, let's move on. Let me see if I can --

16

THE COURT: Let me just see if I can understand

17 the testimony.

18

Your complaint about Chicago is, and I'll use this

19 hypothetical because it's simple, um, but rarely, but as

20 part of the job, you're going to have to transport an

21 extremely obese individual off the upper floor of a

22 tenement down to the lower floor to get him to the

23 hospital. Now, an EMT is going to have to do that and

24 that's going to be very difficult for a woman. Take

25 that hypothetical. And you're -- and you say that the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 34 of 140
34

1 problem with the analysis is it focused on that aspect

2 of the job rather than the 80 percent or more, the

3 percentage I'm making up, which are more routinely where

4 a woman can do it fine.

5

Have I got it?

6

THE WITNESS: In part. The other part is they

7 always send a fire department -- 80 percent of the time

8 they have a fire department company at the scene and if

9 there's not one there they can call one immediately.

10 And it never was a problem forever that women couldn't

11 do the job.

12

THE COURT: So you give my hypothetical but then

13 you say real world that hypothetical is so rare to occur

14 as not to be particularly significant?

15

THE WITNESS: Not significant enough to select out

16 half of all women.

17

THE COURT: And that was your judgment?

18

THE WITNESS: Yes.

19

THE COURT: Then he's pushing you for an analogy

20 here and you're saying that for your Chicago analysis to

21 be germane here the focus would have to be on those

22 equally rare bits of knowledge that a lieutenant would

23 have to know?

24

THE WITNESS: That's exactly what I'm saying.

25

THE COURT: All right. Go ahead.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 35 of 140
35

1 Q.

Do you know how often a Boston police lieutenant

2 has to know Maslow's theory?

3 A.

No, I don't, for a fact.

4 Q.

Moving on in your report.

5

On Paragraph 5 on Page 8, you say: "The job

6 analysis did not consider other abilities that are

7 important to the paramedic job where there would not be

8 gender differences," is that correct?

9 A.

Yes.

10 Q.

And what were you referring to in that paragraph?

11 A.

Other than what's stated there, you mean like a

12 specific example?

13 Q.

Yeah, a specific example of other abilities that

14 would not have had such a disparate impact on women.

15 A.

Yes, I was referring or as an example to what we

16 call "psychomotor abilities" which are, like for

17 example, hand -- manual dexterity, finger dexterity,

18 because unlike mental abilities big muscle group

19 physical abilities does not correlate well with small

20 muscle group physical ability. So giving up -- which is

21 unlike knowledge, the knowledge correlates with anything

22 that's mental ability, which is most everything. So

23 it's a particular problem with physical abilities

24 because they don't generalize across other physical

25 abilities.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 36 of 140
36

1 Q.

Are you saying -- I just want to be clear on this.

2 Are you saying that the physical abilities that

3 Dr. Gephart was testing for were not physical abilities

4 necessary for the job?

5 A.

The physical abilities that she was testing for

6 were -- I'm sorry, I'm -- notwithstanding the caveat

7 that she focused on, the most difficult of those

8 physical abilities, the physical abilities she focused

9 on are part of the job, I believe, in some way.

10 Q.

Well, let me move on. If you would go to Page 9

11 of your report.

12 A.

(Turns.)

13 Q.

The first paragraph, Paragraph Number 1. You say

14 "The components of the EMT were apparently chosen with

15 little consideration of their adverse impact on women

16 applicants in mind," is that correct?

17 A.

Yes, that's right.

18 Q.

And again if we try to analogize it to this case,

19 you were aware, were you not, that the Boston Police

20 Department was aware by 2008 that if they concentrated

21 only on a job knowledge test it would have a significant

22 adverse impact on minority candidates, is that correct?

23 A.

So -- yes, in Boston I think they tried to

24 understand the adverse impact.

25 Q.

Okay. And moving on to Page 12. I think this may

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 37 of 140
37

1 be what you were trying to say. On Page 12 under

2 "Alternatives to the Physical Ability Test." Do you see

3 this?

4 A.

Yes.

5 Q.

It says -- you're stating here, "The Uniform

6 Guidelines requires a search be conducted of alternative

7 selection procedures that meet the business needs of the

8 organization but have lesser adverse impact," and you go

9 on to say "No such search was documented in the Gephart

10 report."

11

THE COURT: Where are you reading from just then?

12

MR. LICHTEN: Sorry, your Honor, it's Page 12

13 under "Alternatives to Physical Ability."

14

THE COURT: Yes, thank you.

15 Q.

Do you see that?

16 A.

Yes.

17 Q.

And you agree with that statement, is that

18 correct?

19 A.

Oh, yes, I wrote that.

20 Q.

Okay. In 2008 -- oh, strike that.

21

There is no validity report for 2008, is there?

22 A.

Oh, sure there is.

23 Q.

Where is it?

24 A.

Well, the 1991 report describes for me the process

25 that was used for --

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 38 of 140
38

1 Q.

No, no, no, Dr. Campion --

2

Dr. Campion, I'm asking you about the validity

3 report for 2008, was there a validity report for 2008?

4 A.

I'm sorry, I thought I answered that.

5 Q.

Well, I guess I'm not clear.

6 You know there was a validity report for 2002, don't

7 you?

8 A.

They had a different firm develop a different

9 procedure.

10 Q.

That's not my question, Dr. Campion. Wasn't, at

11 the end of the testing, didn't they produce a validity

12 report for 2002?

13 A.

Um, I don't know, I didn't read a report, but --

14 Q.

You've never seen the validity report for 2002?

15 A.

Not for 2002.

16

(Pause.)

17

MR. LICHTEN: May I approach, your Honor?

18

THE COURT: You may.

19

(Hands over report.)

20

THE COURT: Are these in evidence already?

21

MR. LICHTEN: No, your Honor.

22

THE COURT: You're offering them?

23

MR. LICHTEN: Yes, your Honor.

24

THE COURT: Any objection?

25

MR. SIMON: Objection, your Honor.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 39 of 140
39

1

THE COURT: Grounds?

2

MR. SIMON: Relevance.

3

THE COURT: Relevance to -- from a November of

4 2002 validity report? I -- it would seem to me to be

5 relevant. There's nothing more germane with respect to

6 2008, is there?

7

MR. SIMON: Well, the point, your Honor, is this

8 is a different test, this was the Morris and McDaniel

9 test with the job assessment center attached to it.

10

THE COURT: I follow that.

11

MR. SIMON: So relevance.

12

THE COURT: No, I can think of various reasons why

13 the validity report in 2002 would be relevant.

14

I'm not clear why you're also proffering the

15 draft, Mr. Lichten, I don't see why that's relevant?

16

MR. LICHTEN: Your Honor, we have searched in

17 vain, we have not been able to locate anything but that

18 draft, and that was produced by the City in the Lopez

19 litigation.

20

THE COURT: But that draft with respect to -- he

21 says "I object on the grounds of relevance," well, no, I

22 think it's relevant. One, it's pretty good evidence of

23 what was known in 2008, if it was known back in 2002.

24 So for that ground alone I'll admit the, um, the

25 validity report.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 40 of 140
40

1

MR. LICHTEN: Oh, your Honor, I know what it is.

2

THE COURT: Oh, I see this other is for the

3 written examination validity report.

4

MR. LICHTEN: Right.

5

THE COURT: And your answer is you can only find

6 the draft, you can't find the final?

7

MR. LICHTEN: They have the --

8

THE COURT: I follow.

9

So what's the objection to the written, Mr. Simon,

10 there's no objection to that?

11

MR. SIMON: Well, again, your Honor, it would be

12 the same objection, in the context of a different test.

13

THE COURT: I understand. They're both overruled.

14

And so I'm going to take the draft written report

15 and give it the next number, which is --

16

MR. SIMON: And if I may, your Honor, just one

17 more point on that? You know HRD is the one creating

18 the test. It's a different entity creating the test.

19

THE COURT: I'm not insensitive to that.

20

MR. SIMON: Okay. Thank you.

21

THE COURT: But what's the next number?

22

THE CLERK: 80.

23

THE COURT: 80. The draft written examination

24 validity report is 80 and the assessment center validity

25 report is 81 in evidence.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 41 of 140
41

1

Go ahead.

2

(Exhibits 80 and 81, marked.)

3 Q.

So, Dr. Campion, I'm showing you what's been

4 marked as Exhibit 80 and 81 which purport to be validity

5 reports, one in draft form for the written and the oral

6 assessment center -- or the assessment center for the

7 2002 Boston police lieutenant examination.

8 Is it your testimony that you've never seen these before

9 today?

10 A.

No, they were, um -- for the reasons that counsel

11 explained, it's a different company developing a

12 different assessment.

13 Q.

I'm sorry. You weren't shown it because --

14

THE COURT: But his question was -- have you seen

15 them?

16

THE WITNESS: No.

17

THE COURT: You've never seen them. All right.

18 Q.

I'm sorry, Dr. Campion, were you about to say that

19 you know why counsel for the City of Boston didn't show

20 it to you because it was a different company?

21 A.

Well, I understood that, um, what I was studying

22 was the HRD exam, not the occasional other exam put

23 together by different contractors.

24 Q.

Well, weren't you asked to look at less

25 discriminatory alternatives and you opined on that, did

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 42 of 140
42

1 you not?

2 A.

Well, that doesn't require me to read this report.

3 Q.

Well, how would you know less discriminatory

4 alternatives had been tried and how they fared and how

5 relevant they were and how well they tested for skills

6 and abilities if you were never shown the report?

7 A.

Because there's a whole huge science out there

8 with thousands of studies that I know a lot about. I

9 don't have to read this one to be aware of a new

10 alternative.

11 Q.

Doctor, you don't know of any studies of police

12 promotional exams and how individuals had fared on

13 police promotional exams by race, um, whether it's oral

14 assessment centers or written exams, do you, you don't

15 know one study?

16 A.

I know a whole lot of studies, I've read many many

17 published studies of --

18 Q.

Tell me one study --

19

THE COURT: Wait. Wait. Let him finish his

20 answer, "I've read many published studies" --

21 A.

I've read many published studies on a wide range

22 of occupations including police and fire and all other

23 types.

24 Q.

Just tell me one police promotional exam study

25 that you relied on in coming to your conclusions today,

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 43 of 140
43

1 published or unpublished?

2 A.

I'm sorry I just can't think of a single specific

3 reference, but it's part of a broad literature that I

4 try to keep up with and have reviewed many times for a

5 wide range of purposes, whether it be interviewing or

6 testing or --

7 Q.

Could you give me any cite to a police promotional

8 study that was done, either published or nonpublished,

9 that tested various alternative mechanisms and looked at

10 the adverse impact that they had?

11 A.

I did not do that review on the eve of this trial,

12 so I don't remember specifics.

13 Q.

You never did that review. It's not just the eve

14 of this trial, you've never done that review, have you?

15 A.

Oh, no, you'd be wrong. We routinely in, for

16 example, graduate classes we review the literature on

17 different professions and I have many times had projects

18 with students where they reviewed the literature on

19 police and fire many times and that's just a "for

20 instance."

21 Q.

Can you give me one?

22 A.

Well, I can't remember the name of the student.

23 Q.

Give me the name of the study or the paper.

24 A.

Well, as I say these are course projects where we

25 routinely review those literature bases.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 44 of 140
44

1 Q.

All right.

2 A.

Plus I was a journal editor for seven years and

3 I've read thousands of papers. I'm sorry that I can't

4 bring back a single reference, but I didn't just review

5 that literature for this case and document the

6 references. But I could.

7 Q.

Let me see if I understand it. You think this

8 report is not relevant because it's produced by another

9 professional firm and only deals with the Boston Police

10 Department?

11 A.

I wanted to make sure I could fully understand the

12 HRD testing process and the materials there were my

13 primary focus.

14 Q.

Okay. Let me ask you this, are you aware of any

15 papers that Dr. Jacobs' firm has published on their

16 experience with multitesting devices for police

17 promotional exams in other cities and what the increased

18 validity and lesser adverse impact, if any, is in

19 combining such techniques?

20 A.

Um, as complicated as that question was, I guess

21 the answer would be, no, I'm familiar that Jacobs and

22 Company are -- do work in this domain and I'm familiar

23 in a general sense but not with respect to the specific

24 validity data or adverse impact data.

25 Q.

When you were hired in this case back in late

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 45 of 140
45

1 November, did you make any attempt to survey any

2 studies, published or not, or call any other large

3 jurisdiction police departments to find out what the

4 most updated police promotional processes were around

5 the country?

6 A.

I did not review other city's processes, no.

7 Q.

Let me move on.

8 All right. Would you turn to, um, Page 81, that's the

9 police lieutenant assessment center validity report for

10 the City of Boston in 2002. Would you turn to that

11 report and turn to Page 4.

12 A.

(Turns.) Okay.

13

THE COURT: 4?

14

MR. LICHTEN: Yes, at the bottom, your Honor.

15 Q.

Okay. Now, they're talking in here about the

16 assessment center method and I understand you haven't

17 read this before, um, but let me ask you.

18 It says in the Paragraph 1 that: "A job analysis of

19 relevant behaviors must be conducted to determine the

20 dimensions, attributes, characteristics, qualities,

21 skills, abilities, motivations, knowledge or tests that

22 are necessary for effective job performance and to

23 identify what should be evaluated by the assessment

24 center."

25

Do you see that?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 46 of 140
46

1 A.

Yes.

2 Q.

And you would agree with that, is that correct?

3 A.

Yeah, as I've stated previously a job analysis is

4 usually where we start because we want to ensure

5 job-related procedure.

6 Q.

And Morris and McDaniel did that, did they not, or

7 do you know?

8 A.

I don't know. I'm only reading this sentence here

9 with you.

10 Q.

So as you sit here today do you know that they

11 concluded that --

12 A.

I'm sorry. I'm sorry. I misspoke. I did read

13 their 2000 job analysis study and to the extent that

14 that's what they're describing here, yes, I did read

15 that. I'm sorry.

16 Q.

And that based upon that job-analysis study, they

17 had to design a testing program to test those

18 knowledges, skills, and abilities that they found were

19 essential, is that right?

20 A.

I don't know. As I said, I have not read this

21 report.

22 Q.

Do you know that when Morris and McDaniel studied

23 the job knowledges, skills, and abilities of the Boston

24 Police Department lieutenant's position, they concluded

25 that more than 50 percent of the test should be devoted

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 47 of 140
47

1 to certain skills and abilities that could be only

2 tested through the assessment center and not through a

3 written multiple choice job-knowledge test?

4 A.

I don't know.

5 Q.

You're unaware of what was done?

6 A.

Yes.

7

THE COURT: That's a little different, at least as

8 I'm following. No, it is a little different. But he

9 says he does not know what was done.

10

MR. LICHTEN: Okay.

11 Q.

Now, if you turn to the next page, Page 5 of this

12 report.

13 A.

(Turns.)

14 Q.

Now, they identify a number of skills and

15 abilities that they think can be tested by the

16 assessment center. It's titled on Page 5,

17 "Identification of Assessment Center Dimensions." Do

18 you see that there?

19 A.

Yes.

20 Q.

And they list oral communications as one of those

21 skills and abilities, is that right?

22 A.

Yes, that's what it says.

23 Q.

And you would agree, would you not, that having

24 good oral communications skills is very important for a

25 Boston police lieutenant, wouldn't you?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 48 of 140
48

1 A.

Um, I suppose. It's not a public speaking role.

2 It's not a role that requires a level of the oral

3 communication of an attorney, for example.

4 Q.

Well, that's interesting.

5 A.

But I'm sure it requires some, yes.

6 Q.

Well, let me ask you this. It sounds like you

7 haven't really studied what the essential knowledges,

8 skills, and abilities are of a Boston police lieutenant,

9 is that accurate?

10 A.

No, but when I'm testifying I want to be very

11 careful about what I claim to know and so --

12 Q.

But what do you --

13 A.

-- so you've asked me to read this here and I've

14 read it and I believe it seems reasonable even though

15 they don't need a high level of oral communication, but

16 --

17 Q.

Did you even read the job description for a Boston

18 police lieutenant?

19 A.

Sure.

20 Q.

You did. Did you talk to anyone at the Boston

21 Police Department about what the essential duties are

22 for a Boston police lieutenant?

23 A.

Well, there was the detailed job analysis reports

24 which are usually the scientific document you use to

25 understand the job as opposed to a casual conversation

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 49 of 140
49

1 with someone.

2 Q.

And that would be the so-called "145 critical

3 knowledges, skills, and abilities necessary for the

4 job," as contained in the 2000 job analysis from Morris

5 and McDaniel, is that right?

6 A.

Well, I would consider the job tasks more relevant

7 to the idea of a job description whereas the knowledge,

8 skills and abilities, we usually call that the "job

9 specifications."

10 Q.

The knowledge, skills, and abilities, there were

11 149 of them identified as "critical" in the Morris and

12 McDaniel 2000 job analysis, is that right?

13 A.

No, you just said 145. They started with 149, but

14 after they were reviewed by subject matter experts, a

15 subset of those were found to be the most important.

16 Q.

All right. And only about a third of those were

17 knowledges, the rest were skills and abilities, is that

18 correct?

19 A.

I don't remember the exact numbers.

20 Q.

And over 100 of those critical skills and

21 abilities were never tested for in the 2008 exam, were

22 they, Doctor?

23 A.

No, I don't think that's the best way to describe

24 it.

25 Q.

You didn't see the section that said these are the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 50 of 140
50

1 things that are not being tested for and they included

2 100 skills and abilities?

3 A.

Well, they tested for 13 very broad knowledges and

4 they showed that those --

5 Q.

13 broad knowledges --

6 A.

-- and they showed that those 13 were related to

7 all the knowledge, skills and abilities and they also

8 showed that those 13 were related to all the important

9 job tasks. So the 13 were very broad and they related

10 to the entire job.

11 Q.

Doctor, there was no attempt to test for certain

12 critical skills and abilities such as oral

13 communications on the 2008 test, was there?

14 A.

I'm sorry, you'd have to restate that.

15 Q.

There was no attempt to test for oral

16 communications skills in the 2008 exam, was there?

17 A.

No, no, the knowledge exam didn't test for oral

18 communications skills.

19 Q.

And there was no attempt to test for interpersonal

20 skills, is that correct?

21 A.

Um, when I read the test questions I don't recall

22 if any of them dealt with knowledge around interpersonal

23 behaviors, but it may have. I just don't recall. It's

24 not uncommon that we will ask questions, knowledge

25 questions about how they interrelate to people or work

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 51 of 140
51

1 on a team, for example. So you can have knowledge

2 questions related to interpersonal, but I don't recall

3 if they had those here because I can't remember all 100

4 questions.

5 Q.

Let's just see if I understand this. Do you

6 remember in your deposition I asked you about Maslow and

7 Maslow was a well-known psychologist of some repute from

8 sometime ago?

9 A.

Yes.

10 Q.

Are you aware of -- well, let me ask you this.

11 If a question asks about Maslow's theory of something,

12 that's not the same as knowing that the person has the

13 skills and ability to practice that technique, is it?

14 A.

No, but you bring up a very good point.

15 Understanding that --

16 Q.

I think you said "No," is that correct?

17

MR. SIMON: Objection, your Honor.

18

THE COURT: Oh, no, overruled. It was -- it could

19 be framed that way. This is cross-examination. You'll

20 have a chance to inquire of him further.

21

Go ahead.

22 Q.

Now, were you aware that when the assessment

23 center was conducted testing for certain skills and

24 abilities which Morris and McDaniel deemed critical for

25 the job of lieutenant the result was no statistically

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 52 of 140
52

1 significant differences which existed between the mean

2 overall assessment scores of African American,

3 Hispanics, and Caucasians?

4

THE COURT: Could you ask the question again? And

5 that's my fault.

6

MR. LICHTEN: I'll put it a better way, your

7 Honor.

8

THE COURT: All right.

9 Q.

If you would turn to Page 18.

10 A.

(Turns.)

11 Q.

Do you see under Section 5?

12 A.

Do I see Section 5? Yes.

13 Q.

And they say, do they not: "No statistically

14 significant differences exist between the mean overall

15 assessment scores of Caucasians and African Americans."

16 Do you see that?

17 A.

Yes.

18 Q.

So following up on what you said in Ernst, if the

19 Uniform Guidelines require a search for less

20 discriminatory alternatives and in 2008 the Boston

21 Police Department gave an assessment center that tested

22 for critical knowledges, skills, and abilities that was

23 valid, that is it was properly constructed by a

24 professional firm and it had a significant reduction on

25 adverse impact, that would have been something that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 53 of 140
53

1 should have been looked at in 2008, is that right?

2 A.

No.

3 Q.

No?

4 A.

I don't know that it was valid and as a

5 statistician I can see that there were race differences

6 and a 1.86 would likely be marginally significant and I

7 see they only had 30 people in the sample. So if they

8 added a small number of additional people it would be

9 significant. And I see that the differences is about 40

10 percent of the standard deviation, maybe a little more,

11 in Table 3.

12 Q.

Okay.

13 A.

And so I wouldn't draw the conclusion you just

14 did.

15

THE COURT: Well, let me -- I have a basic

16 misunderstanding here and I think you can help me out a

17 little, Doctor.

18

Even though you haven't seen Exhibit 81 before,

19 apparently am I correct in thinking this is a validation

20 report, um, for the City of Boston on an assessment

21 center process that had taken place, is that right?

22

THE WITNESS: Apparently.

23

THE COURT: Yeah, okay, that's how I read it. I

24 thought going in, and just so counsel can follow, that

25 this was the report about the wisdom of using such an

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 54 of 140
54

1 assessment center, but this appears to be a report on

2 how this particular use of such an assessment center

3 worked. And that's how you read it anyway?

4

THE WITNESS: Yes.

5

THE COURT: I'm seeing it as quickly as you.

6

All right. Go ahead, Mr. Lichten.

7

(Pause.)

8

THE COURT: And you're just cautioning us, to get

9 your normative testimony, not to read too much into this

10 because of the sample size?

11

THE WITNESS: That's right, yes, because small

12 samples are --

13

THE COURT: Well, he's examining. Go ahead.

14 Q.

Now let's pursue that for a second, because you

15 just said it was marginally significant at 1.86, is that

16 correct?

17 A.

Yes, I believe that would be accurate.

18 Q.

That's a P value of .056?

19 A.

Um, I don't know that.

20 Q.

Okay.

21 A.

I can say it's over .05, but it's not too too far

22 over.

23 Q.

But, Dr. Campion, do you understand that what's

24 being looked at here are not promotion rates but mean

25 score differences?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 55 of 140
55

1 A.

Yes.

2 Q.

And do you understand that the mean score

3 differences for the 2005 and 2008 Boston Police

4 Department written job knowledge tests were way, many

5 times higher than .056?

6 A.

Um --

7 Q.

They were .000-something, weren't they?

8

MR. SIMON: Objection, your Honor.

9

THE COURT: Well, he can put the question. It's

10 not evidence of anything. We'll see if he knows.

11

MR. SIMON: Okay. Thank you, your Honor.

12 A.

Um, a job knowledge test, if it's more valid, will

13 show a larger race difference.

14 Q.

Well, I'm just asking you now if you've looked at

15 the difference, the mean score differences in this case,

16 for 2005 and 2008, would you agree that the statistical

17 significance was huge, it was .000-something?

18 A.

I was actually not asked to testify about

19 statistics for this case.

20 Q.

(Pause.) Now, you've also mentioned the --

21 several times you've mentioned the 1991 job analysis, is

22 that correct?

23 A.

I haven't --

24 Q.

Oh, the validation report. Excuse me, the

25 validation report.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 56 of 140
56

1 A.

Frankly it was a job analysis and a validation

2 report in one document, yes.

3 Q.

And I just want to ask you, when you looked at the

4 1991 validation report, did you look at all the

5 appendices?

6 A.

Um, yes, I've skimmed them all, I didn't study

7 them all in great detail.

8 Q.

So "A" down to "UU," or wherever it ended?

9 A.

It went through and then we had double letters, so

10 there must have been 30 something. But I didn't look at

11 all of them.

12 Q.

Okay. So I'm -- I'm out to kill a couple more

13 trees.

14

(Hands to witness.)

15

MR. LICHTEN: This is Exhibit 41 from Lopez, your

16 Honor.

17

(Hands up.)

18

THE COURT: And again I take it there's no dispute

19 about this, but these appendices that have just been

20 handed up, Exhibit 41 from Lopez, which is in evidence

21 in this case, are the appendices to the validation

22 report that I have admitted in evidence as Exhibits 80

23 and 81, is that what the representation is?

24

MR. LICHTEN: Almost. 71, your Honor.

25

THE COURT: 71?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 57 of 140
57

1

MR. LICHTEN: This is the 1991, not the 2002.

2

THE COURT: Thank you. All right. So these are

3 the appendices to the validation report admitted in this

4 case as Exhibit 71? (Silence.) Everyone seems content

5 with that. All right. Fine.

6

MR. SIMON: That's correct, your Honor.

7

THE COURT: Fine. Thank you.

8 Q.

And you've seen this before, is that correct?

9 A.

Well, an electronic version of these, yes.

10 Q.

And you've reviewed it, is that correct?

11 A.

Well, as I say I reviewed those that seem relevant

12 to the validity or job analysis of those specific topics

13 that I was interested in.

14 Q.

Okay. Well, let me ask you to turn to, um, Page

15 82.

16 A.

(Turns.)

17 Q.

Oh, I'm sorry, Page 82 of Exhibit 71, which is the

18 validity report, sorry, not the appendix, and you have,

19 um -- Exhibit 71 should be in one of those notebooks.

20 A.

(Looks.) What page was that again, sir?

21

THE COURT: Page 82 of Exhibit 71.

22 A.

(Turns.) Yes, sir. I'm there.

23

THE COURT: Well, you're faster than me.

24

MR. LICHTEN: Excuse me, your Honor?

25

THE COURT: He's faster than I am. Now I'm there

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 58 of 140
58

1 too. Go ahead.

2 Q.

Okay. So looking at Page 82 at the bottom,

3 there's a little chart. Do you see that?

4 A.

Yes.

5 Q.

And do you know what that chart is?

6 A.

Um, it appears to be an adverse impact analysis of

7 cutting scores for sergeant and lieutenant in previous

8 examination years.

9 Q.

Okay. These are pass/fail cut-off scores, is that

10 correct?

11 A.

That was my interpretation, yes.

12 Q.

Now, there's been some testimony, and we'll get to

13 it in a minute, that in 1985 and 1987 there may have

14 been a different type of test, but do you see the result

15 for 1991?

16 A.

Yes, it looks like they looked at two different

17 passing points.

18 Q.

Right. And what this says is if they used the

19 passing point of 70, which I represent to you is the

20 traditional passing point for these civil service

21 promotional exams, the adverse impact ratio on

22 lieutenants would have been .21 and for sergeants it was

23 .16, is that correct?

24 A.

That's what I'm reading along with you here.

25 Q.

So that's severe adverse impact and that's just

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 59 of 140
59

1 the pass/fail complaints, right?

2 A.

Well, according to that statistic, I -- again I'm

3 not studying the statistics. I don't know how many

4 minorities were in this analysis. If the numbers were

5 very small, you can often get average results that seem

6 extreme in either direction. So --

7

THE COURT: Well, putting aside the explanation

8 and at a level of simply looking at this report at face

9 value, on this chart that appears at the bottom of Page

10 82, higher is better at that level, is that right?

11

THE WITNESS: Yes.

12

THE COURT: All right.

13

Put your questions, Mr. Lichten.

14

MR. LICHTEN: I'm not sure I understood what the

15 Court just said, but --

16

THE COURT: Well, well, that a .16 is more adverse

17 impact than a .30 and for lieutenants a .21 is higher

18 adverse impact than a .68.

19

MR. LICHTEN: Yes.

20

THE COURT: I recognize it's simplistic, but I'm

21 trying to keep up. He agrees and that's my

22 understanding and now put your questions.

23

MR. LICHTEN: All right.

24 Q.

And again just to reiterate, and that's only the

25 AI impact on the pass/fail rate, this doesn't have the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 60 of 140
60

1 report as to what the, um -- as to what the adverse

2 impact, if any, would have been with respect to

3 selection rates, is that right?

4 A.

Yes, that would be my interpretation.

5 Q.

And it doesn't have what the adverse impact is of

6 any mean score differences, is that correct?

7 A.

Well, that question is not correct. I can't

8 answer it.

9 Q.

It doesn't address whether there's statistically

10 significant mean score difference in the scores, is that

11 right?

12 A.

Yes, that's right because adverse impact and mean

13 differences are very different.

14 Q.

Thank you. I'm going to take statistics when I'm

15 done with this trial but I'm not able to quite yet.

16

(Laughter.)

17 Q.

Okay. And then if you go to, um -- if you go to,

18 um, Page 133 of this report --

19 A.

(Turns.)

20 Q.

Do you have that?

21 A.

Yes. Uh-huh.

22 Q.

This gives you some information on the mean score

23 differences of the 1991 written exam that was given for

24 the Boston Police Department, is that right?

25 A.

Yes, it appears to.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 61 of 140
61

1

THE COURT: And just so -- look at the first chart

2 there on Page 133. This chart seems to confirm the

3 testimony I've heard that while a written exam will --

4 there's going to be a disparate impact between blacks

5 and whites, that disparate impact is lessened as the

6 passing score is lessened?

7

THE WITNESS: Yes.

8

THE COURT: But of course the validity of the exam

9 as a promotional tool is correspondingly lessened, is

10 that right?

11

THE WITNESS: Yes, although we usually refer to

12 the latter as the "utility" because if you remember

13 "validity" is kind of the inference you can make from

14 test scores. So you can have a valid test that has no

15 use to you because you have a cutting score so low it's

16 irrelevant.

17

THE COURT: I'll stand corrected, so the

18 "utility."

19

Why he suggested, and the evidence seems to

20 confirm this, that over the years 70 has been the usual

21 cut-off. Why is that the optimal utility, if you know?

22

THE WITNESS: I do have opinions of that, your

23 Honor.

24

THE COURT: Well, I'm asking a "why" question, so

25 tell me why.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 62 of 140
62

1

THE WITNESS: Well, in the, um -- tests are used

2 for a range of purposes. The big distinctions are what

3 we call "norm-referenced tests" and, oddly, "criterion-

4 referenced," and what it means -- "norm-referenced"

5 tests, the scores only have meaningfulness in the sense

6 of comparing to norms, so "This candidate is better than

7 this candidate," but you don't know what the scores, do

8 they absolutely mean the person is competent, for

9 example? "Criterion-referenced" testing, which is like

10 the bar exam, tests in college, licensure, you're -- the

11 exam scores are supposed to represent that you have

12 possession of a body of knowledge, and, you know, how

13 much depends but over the years 70 percent of the

14 knowledge is often thought of as demonstrating that you

15 possess, you know, enough of the domain of knowledge.

16 So that 70 has really come around through historic

17 reasons like that.

18

THE COURT: Thank you. Go ahead, Mr. Lichten.

19

MR. LICHTEN: Yes.

20 Q.

Well, just to pursue that, here the pass/fail rate

21 is actually not as important as it might be in some

22 situations because you understand that selections are

23 being done on a rank-order basis, is that correct?

24 A.

I'm sorry, I -- the question took a twist at the

25 end. Can you repeat that maybe?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 63 of 140
63

1 Q.

You understand that the selections for the Boston

2 Police Department are not done by just simply being in a

3 pool by having a passing score sort of like in the

4 Chicago Fire Department EMT test that you did, but here

5 they're selected by rank order, is that correct?

6 A.

That's my understanding, yes.

7 Q.

Right. So in fact you're not aware if the Boston

8 Police Department has ever gotten down to 70 or 75 or

9 even 78 making promotional decisions, but people have to

10 score high on the test in order to really have a chance

11 of getting promoted, is that right?

12 A.

I understand it's a competitive process, yes.

13 Q.

And do you have any idea, as you sit here today,

14 what have been the scores at which people actually have

15 gotten promoted over the years?

16 A.

No, as a fact I don't. No.

17 Q.

Now, if we go to, um, this chart on the bottom of

18 Page 133, it shows, does it not, that the mean -- I

19 think it's the mean score for lieutenant, on this

20 particular exam, was 72.66. Do you see that?

21 A.

Yes.

22 Q.

Okay. And then if we go to the last, it's 61.53,

23 is that correct?

24 A.

62.53?

25 Q.

Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 64 of 140
64

1

THE COURT: You misspoke. You said 61. But

2 62.53.

3

MR. LICHTEN: I'm sorry. I'm trying to make my

4 case better.

5

(Laughter.)

6 Q.

It's a 9-point difference, is that right?

7 A.

Yes, a little more than 10 points.

8 Q.

That's a huge difference, isn't it?

9 A.

Um, I don't know. We only have 20 blacks here,

10 it's -- I don't know if that's significant even. But

11 it's 10 points.

12 Q.

A 10-point spread indicates to you that if you

13 keep giving this test, a multiple choice job knowledge

14 test, you're going to have discriminatory impact on

15 minority candidates, am I correct, with a 10-point mean

16 score difference?

17 A.

Actually I don't know. I don't know that.

18 Q.

And for Hispanics it's even lower, their average

19 score was 57 -- the mean score was 57.59, is that

20 correct?

21 A.

There's only two Hispanics. I'm sorry, these

22 numbers are so small that from a statistical point of

23 view you have to be very very cautious.

24 Q.

If 20 minorities taking the exam is not small

25 enough to -- you can still have statistical significance

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 65 of 140
65

1 with a group of 20 if there are significant differences,

2 is that correct, with a group of 20 verse however many

3 whites there were that took the exam, you can get

4 statistical significance out of that, can't you?

5 A.

Um, if the difference is large enough it could

6 reach the level at which it's big enough to be unlikely

7 to be by chance, which is what significance is --

8 Q.

But if you were an industrial organizational

9 psychologist being hired by the City of Boston in 1991

10 or subsequently, in subsequent years, and you were

11 looking at this data, the adverse impact ratio of the

12 pass/fail score, the historical knowledge you have of

13 differences on these type of tests between minorities

14 and nonminorities and then looking at this mean score

15 difference on the 1991 test, it wouldn't take even a

16 scholar such as yourself to come to the conclusion that

17 if you keep giving just this test you're going to have

18 significant adverse impact against minority candidates

19 for lieutenant, isn't that right?

20 A.

No, no, see that's the fallacy, don't ever assume

21 small samples behave like big samples, that is

22 incorrect. Small samples are not a good predictor of

23 what's going to happen in the future or with larger

24 samples. That's the whole notion of statistics. So --

25

THE COURT: Well, let me try this. You're -- as

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 66 of 140
66

1 I'm listening to you, your quarrel with his question is

2 that he uses the word "significant." If he said, "If

3 you keep giving this test you're going to observe a

4 disparate impact between blacks and whites," you'd agree

5 with that?

6

THE WITNESS: No, no, see that's the problem, with

7 only 20 there is a very large band of error around the

8 estimate of their mean score.

9

THE COURT: And when we come to the numbers. But

10 I had thought you, and apparently other witnesses in

11 this case, do take the position that a written test, for

12 reasons we don't understand, is whites are going to

13 outperform blacks to some degree?

14

THE WITNESS: That I would agree with, but this

15 sample, because of its small size, is not a good

16 estimator for the future.

17

THE COURT: I get it. That's your point. Go

18 ahead, Mr. Lichten.

19

MR. LICHTEN: Sure.

20 Q.

Now, if you can go to the appendix for a moment.

21

THE COURT: Now, this is the document -- the bulky

22 document you just handed up?

23

MR. LICHTEN: Yes.

24

THE COURT: All right.

25 Q.

Okay. I want to ask you a couple of questions

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 67 of 140
67

1 about this appendix.

2

I believe your testimony was that you reviewed the

3 appendix but maybe not in the fashion that you normally

4 would as a social scientist, do I have that right?

5 A.

No, I don't think I said that.

6 Q.

Okay, I didn't mean to put words in your mouth.

7 How would you characterize your review of the appendix

8 for thoroughness?

9 A.

Well, I looked at those appendices that were

10 related to issues as I read the report. So if there was

11 something that was relevant to my understanding, I would

12 go to the appendices. And then at the end of my reading

13 of the materials, I did do kind of a skim through the

14 appendices, um, just to make sure I laid eyes on each

15 one of them.

16 Q.

Well, one of the things you testified about, when

17 you were under direct testimony, is certain constraints

18 that the Boston Police Department might have been under

19 that would have prevented them from doing other

20 components to the test, I think you talked about civil

21 service requirements as one of those, is that right?

22 A.

Um, yes.

23 Q.

I think you told the Court that actually in

24 response to a question.

25 A.

That I understood that there were civil service

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 68 of 140
68

1 requirements that were applicable in this case?

2 Q.

Yeah, right.

3 A.

Yeah, that was my understanding. Yes.

4 Q.

Okay. Well, did you happen to look at Appendix C

5 to this document?

6

THE COURT: While he's looking for it, let me ask

7 you if you have much more for this witness?

8

MR. LICHTEN: Oh, yes, your Honor.

9

THE COURT: All right, then that implies that a

10 break will be in order.

11

MR. LICHTEN: Sure.

12

THE COURT: We'll take the morning recess at this

13 time for one half hour. We'll recess.

14

(Recess, 10:45 a.m.)

15

(Resumed, 11:20 a.m.)

16

THE COURT: Proceed, Mr. Lichten.

17 Q.

So in Exhibit --

18

THE COURT: We had looked at a page of these

19 appendices and you had directed him to C.

20

MR. LICHTEN: Yes, I've got it, your Honor.

21 Thanks. And for the record this was Exhibit 41 in

22 Lopez.

23

And, um, so it's not necessary to offer it, is

24 that right, your Honor?

25

THE COURT: No, it is of record in this case,

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 69 of 140
69

1 Lopez and all of its exhibits.

2

MR. LICHTEN: Okay.

3 Q.

So my question, um, just to recap, is whether you

4 --

5

You said you may have skimmed the appendices, you

6 may not have read them, is that correct?

7 A.

Yes, that's correct.

8 Q.

Okay. Well, did you skim or read this decision by

9 the Civil Service Commission in a case called Carr vs.

10 the Department of Personnel Administration?

11 A.

No, I did not, and this would have been why I just

12 opened it and observed that it was something not

13 relevant, so I don't think as to the validity. So I did

14 not read it.

15 Q.

You didn't think it went to validity?

16 A.

Um, and still don't understand how it does. I'm

17 sorry.

18 Q.

Well -- we'll pursue that.

19

Did anyone at the Boston Police Department or

20 anyone prior to today inform you of this decision in

21 Carr vs. The Department of Personnel Administration?

22 A.

No, they were very hands-off.

23 Q.

Okay. And you have no idea that this was affirmed

24 by the appeals court either, you wouldn't know that,

25 right?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 70 of 140
70

1 A.

No, no, I would not know that.

2 Q.

Okay. Now, are you aware that the civil -- that

3 our Massachusetts statute has its own requirement for

4 promotional exams?

5 A.

Well, I only saw reference to something to that

6 effect in the Lopez decision and that's all the

7 knowledge I have of it.

8 Q.

Okay. And so if I could turn your attention to,

9 um -- I don't see a number. I'm sorry about that.

10 It's, um, about nine pages from the back.

11

THE COURT: Of Exhibit C?

12

MR. LICHTEN: Yes, of C.

13 A.

(Turns.)

14

THE COURT: They have like Bates numbers on the

15 bottom, my copy. 3000 and specific numbers. For

16 instance, my last page is 3606 and so if we go back 9,

17 that would be 35 --

18

MR. LICHTEN: I have a different version, your

19 Honor. I'm sorry.

20

THE COURT: -- 97. So if it is 9 pages, it's on a

21 page, the first full paragraph begins "After reviewing

22 the second set of statistics, Dr. Wiesen."

23

Is that where you want me to be?

24

MR. LICHTEN: No, it's Roman Numeral III -- and

25 let me see if I can get to that in this document. I'm

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 71 of 140
71

1 sorry about that.

2

(Pause.)

3 Q.

Okay. If you could go to -- it's Bates page

4 Number 3594.

5 Do you have that?

6 A.

Yes, I do.

7 Q.

And you see at the bottom they're just quoting

8 this Massachusetts statute which says "Examinations

9 shall fairly test the knowledge" --

10

THE COURT: Right, you don't need to read it. I

11 see it there.

12

MR. LICHTEN: Okay.

13 Q.

And you're not aware that in this case the Civil

14 Service Commission found that the test that was

15 administered in this case, which threw out the

16 assessment center and the video exercises, was found to

17 violate this statutory provision because it was not a

18 fair test of the knowledge, skills, and abilities

19 required for the job, you're not aware of that?

20

MR. SIMON: Objection.

21

THE COURT: Yeah, the fact is the fact, whether

22 he's aware of it or not, why is that relevant?

23

MR. LICHTEN: Sure, your Honor, because in his

24 testimony he claimed that there were civil service

25 requirements that might have precluded the City from

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 72 of 140
72

1 selecting alternative techniques because of restrictions

2 in municipalities and public sector civil service and

3 this shows just the opposite, your Honor.

4

THE COURT: Well, it shows what it shows. I think

5 the point is made by reference to the facts within

6 Massachusetts.

7

Anything else for this witness?

8

MR. LICHTEN: Yes, your Honor.

9 Q.

Now, I want to see if I understand this question

10 of the search for alternatives.

11 Is it your testimony or understanding in this case that

12 the City, in 2007 or 2008, made a search for alternative

13 selection procedures as we discussed the Uniform

14 Guidelines provide for?

15 A.

It was my understanding that in the 1991 report,

16 when they began the process that describes the 2008 and

17 2005 exams, that they did a search at that time, and

18 then as I mentioned I understood that there were

19 alternatives examined over the years, and then my

20 reading of the Lopez case was that there was information

21 that the City had considered other alternatives and that

22 those were the three pieces of information that caused

23 me to conclude that they had considered alternatives.

24 Q.

Okay. Putting aside 91, we'll get to that in a

25 second, what were the alternatives that you believed

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 73 of 140
73

1 were considered after 1991, and with respect to the 2008

2 examination, what did they consider, what did they do to

3 consider these alternatives?

4 A.

I don't know what was in their minds. As I said,

5 I --

6 Q.

Well, do you know what they did, do you know what

7 they actually did?

8 A.

Well, it appears as though they did this 2002

9 study with the Morris and McDaniel firm which

10 experimented with an alternative and I thought there was

11 another instance as well.

12 Q.

As you sit here today can you tell me anything,

13 between let's say 2004 and 2008, that the Boston Police

14 Department or HRD or anyone else did to explore

15 alternative selection procedures in preparation for the

16 2005 and 2008 Boston police lieutenant's examination, if

17 you know?

18 A.

I don't believe I know of any single document in

19 that time frame on this topic.

20 Q.

The truth is you don't know what they did? As you

21 sit here today you can't tell me what they did between

22 2004 and 2008, can you?

23

MR. SIMON: Objection, your Honor.

24

THE COURT: No, he may have the question in that

25 form.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 74 of 140
74

1 A.

Well, I told you what I know.

2 Q.

Okay. But you believe they did make such a search

3 in 1991, is that correct?

4 A.

Yes.

5 Q.

Okay. Now, the field of your science has evolved

6 to a huge extent between 1991 and 2007, a period of 16

7 years, is that correct?

8 A.

Well, certainly a lot of progress has been made.

9 Q.

You would expect that it would be appropriate,

10 under the Uniform Guidelines, for a community such as

11 Boston to make a search for alternative selection

12 procedures in 1991 and then use that as a defense for

13 not doing so 16 years later, would you?

14 A.

I'm not saying anything about a defense. I'm

15 sorry. That's not my role to say how they need to

16 defend themselves.

17 Q.

Well, it is your role -- like you testified in

18 your expert report in the Ernst case, to see if the

19 Uniform Guidelines were complied with, that's one of the

20 things you've done in this case, isn't that right?

21 A.

Well, sure.

22 Q.

And one of the things the Uniform Guidelines

23 provide for is that there should be -- when you know

24 you're using a selection procedure that may likely have

25 adverse impact, you're supposed to make a search for

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 75 of 140
75

1 alternative selection procedures, that's right in the

2 guidelines, right?

3 A.

That's right.

4 Q.

Okay. Assuming that to be the case, you would

5 agree that relying on a search for alternatives 16 years

6 before, given the evolution of social science in this

7 field, would not meet that guideline, isn't that right?

8 A.

Well, I -- a lot has happened in those years, but

9 not so fundamentally as the 20 years preceding that.

10 Um, like, for example, we discovered meta analysis in

11 the '70s. Since the '90s there have been a number of

12 refinements towards selection procedures, but we still

13 have not, you know, "cracked the nut," so to speak,

14 there has still not been invented a selection procedure

15 that is highly valid that shows no race differences.

16 That has evaded us.

17

THE COURT: Actually I'm interested in that. But

18 we have -- if I understand your testimony, we have

19 evolved. Well, let me put it to you this way.

20

What I'm hearing is that in terms of test validity

21 you -- and we'll make it simple because I think I've

22 heard this from you, more is better, it tends to capture

23 a greater array of the skill set of the job, especially

24 a supervisory job like a police lieutenant.

25

You have so testified, right?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 76 of 140
76

1

THE WITNESS: Sure, more is always better,

2 usually.

3

THE COURT: Usually.

4

THE WITNESS: Except for adverse inference.

5

THE COURT: Right, and then we got into a colloquy

6 where you forcefully explained that more does not at all

7 suggest that disparate impact will be reduced.

8

That was your testimony and we'll stop there,

9 right?

10

THE WITNESS: Absolutely right.

11

THE COURT: Okay. So have we, over this time --

12 you answered him that we haven't "cracked the nut," we

13 haven't got testing procedures that will eliminate a

14 disparate impact and indeed I understood you to say that

15 when your testimony started, but have we developed

16 procedures which reliably will reduce disparate racial

17 impact?

18

THE WITNESS: Um, with less -- with the same

19 validity but less adverse impact?

20

THE COURT: Correct.

21

THE WITNESS: It all depends. It would not be at

22 the same cost, it would be more costly.

23

THE COURT: More costly, and as I get from your

24 earlier testimony, you'd have to have a good

25 correlation?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 77 of 140
77

1

THE WITNESS: Uh-huh. Uh-huh.

2

THE COURT: "Yes" is better.

3

THE WITNESS: Yes. I'm sorry. The answer is

4 "Yes."

5

THE COURT: Okay. All right. Thank you.

6

Go ahead, Mr. Lichten.

7 Q.

Okay. Now, I've read all the attachments and the

8 1991, um, validity report that you allude to and I have

9 to confess that I don't see anywhere in there any data

10 on what alternatives were looked at by the Boston Police

11 Department in 1991. Can you help me out on that? You

12 said they made this search, but where is this search?

13 A.

Um, in the -- can I look at the report?

14 Q.

Absolutely, you can look at the report or you can

15 look at the attachments, whichever one you want.

16 A.

(Looks.) All right. Are you ready, sir?

17 Q.

Yes. Sure.

18 A.

One place that they referred to it is in the

19 information-gathering stage of the job analysis where

20 they did a literature review and a survey of other

21 jurisdictions on Page 5, and in the second paragraph

22 under "Information Gathering," about the middle maybe --

23 well, 10 lines down it measures -- it says "Selection

24 Procedures." So the whole sentence reads --

25 Q.

What page are you on?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 78 of 140
78

1 A.

I'm on Page 5, the paragraph centered in the

2 middle of the page.

3 Q.

Right.

4 A.

Maybe starting on Line 8 it said, "In the

5 questionnaire DPA sought information about job analysis

6 methodologies, reading scales, selection procedures,

7 reading lists, and validity evidence pending ongoing

8 litigation and a ranking of candidates." So that

9 suggested to me that they gathered information about

10 selection procedures and about the validity data and

11 about alternative uses such as ranking.

12 Q.

But did you ever see what they got back, whether

13 they got back one other selection procedure?

14 A.

Well, no, they didn't have a section of the report

15 where they said, "Here's everything we got back."

16 Q.

Well, they have Attachment E, "Survey of Other

17 Jurisdictions, List of Jurisdictions Surveyed," and in

18 the appendix, "Summary of Survey of Other

19 Jurisdictions," that's Attachment F, um, G -- I'm sorry,

20 under E, F, and G. If you look at E, F, and G, I don't

21 see anything that shows that any jurisdiction sent back

22 one piece of information on what selection procedures

23 they were using, do you?

24 A.

No, but it doesn't make me feel like they didn't.

25 You can't document everything. The report is already

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 79 of 140
79

1 100-and-some pages single-spaced plus several hundred

2 pages of appendices.

3 Q.

Doctor --

4 A.

I'm sorry, I don't think necessarily that they

5 have to provide an analysis of it to believe that they

6 conducted it.

7 Q.

But you don't know that they got back any

8 information because it's nowhere to be found and you're

9 a scientist and certainly you believe that you should

10 only believe something if there's documentation that it

11 occurred, right?

12 A.

I can see where they describe their search. And

13 the second place I see is on Page 15 where the second

14 full paragraph of 15 they explain, um, why they selected

15 this selection procedure, that would be the sort of

16 conclusions that would come out of the search for

17 alternatives and they list six of those there. We can

18 read them, if you'd like, into the record, but --

19 Q.

Dr. Campion, there are a thousand pages or more in

20 this 1991 survey with the appendices, I mean you've

21 testified to that in your direct examination, and I

22 don't see, and I'm asking you if you see one piece of

23 evidence, a document that shows "We contacted Los

24 Angeles, this is what Los Angeles told us," "We

25 contacted Chicago, this is what Chicago does"? I see

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 80 of 140
80

1 they got KSAs from those towns and cities, I see that

2 they asked them questions about what skills and

3 abilities they were looking for, but I see not one

4 document, not one word that says, "This city told us

5 that they do the following," and I'm asking you if

6 you've seen such a document?

7 A.

No, but it doesn't again lead me to conclude that

8 they lied in these two sections of the report.

9

THE COURT: No one's suggesting they lied or at

10 least I don't hear that.

11

You haven't seen it?

12

THE WITNESS: I haven't seen it.

13

THE COURT: All right.

14 Q.

If you were retained by the plaintiffs in a case

15 such as this you wouldn't rely on such supposition that

16 something might have happened, you would have brought or

17 put that in your report, that this is a deficiency,

18 right?

19 A.

Well, no, and actually I've written something like

20 300 technical reports and I understand that you need to

21 be -- you know, you're selective in what you put in,

22 every single bit of data that you collect or thought

23 that you have or scrap bit of paper doesn't go into the

24 report, necessarily so. And then who would know, 20

25 years later, that there would be some need for that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 81 of 140
81

1 thing that you could have documented better. It's so

2 easy after the fact to say, "Well, they didn't document

3 it, so it didn't happen."

4 Q.

But, Dr. Campion, the only reason that I raised it

5 is because you testified yesterday, or Friday in your

6 direct testimony, that they've made a search for

7 alternatives, they complied with the guideline, yet you

8 can't point out to me where that -- where's a piece of

9 evidence that showed that they got back information from

10 other jurisdictions about what was being done in 1991,

11 that's right, isn't it, you can't show me anything?

12 A.

Um, maybe that's more important to you than it is

13 to me. I've told you the data that I relied upon and we

14 have at least two sections of the report here and we

15 also have other sections of the report that discuss

16 alternative uses and particularly they looked at, if you

17 remember earlier today, the cutting scores, so they

18 considered alternative cutting scores, and they also

19 discussed ranking. And so they considered both

20 alternative procedures and alternative uses in this '91

21 report, and then I've mentioned the other bit of data

22 that I have relied upon already.

23 Q.

Have you completed your answer?

24 A.

Yes.

25 Q.

Okay. Let me move on to the final thing about

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 82 of 140
82

1 1991.

2 When you looked at the 1991 report, do you remember, um,

3 something about looking at "critical incident

4 techniques"?

5 A.

Yes.

6 Q.

Now, can you tell us what "critical incident

7 techniques" are, please?

8 A.

Well, it's a type of data gathering, um, perhaps

9 considered a type of job analysis technique where

10 information is gathered on extreme examples of either

11 especially-effective job performance or especially-

12 ineffective job performance, and that information is

13 often used then to understand the consequences of job

14 performance or differences in job performance in terms

15 of important organizational outcomes and it's not

16 uncommon to use that information to even write the test

17 questions.

18 Q.

Okay. And is that a -- that's a technique that

19 you, um, support, is that right?

20 A.

Oh, sure, I mean it can be used, but it's not very

21 efficient. Per amount of hour you don't get as big a

22 pay-out from it, so we don't use it a lot unless we're

23 developing test questions. But as a form of job

24 analysis it's not -- it's not very time intensive and

25 it's not very thorough because you're just getting the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 83 of 140
83

1 ends of the distribution of performance, not everything

2 in the middle. So it's usually a supplementary thing

3 that is done in the service of writing tests or other

4 assessment questions.

5 Q.

But if you do it it can be helpful in

6 differentiating the essential duties of a lieutenant

7 from those of a sergeant, is that right?

8 A.

Perhaps. There's lots of ways of getting that

9 kind of information, but that's one of them you might

10 use.

11 Q.

All right. You mentioned in your report, your

12 expert report, that they had done these critical

13 incident techniques, is that correct?

14 A.

Yes, it's described in the -- in the '91 job

15 analysis report.

16 Q.

Doctor, as you sit here today do you know whether

17 or not there were any critical incident techniques done

18 for lieutenants as opposed to sergeants?

19 A.

I don't recall reading the incidents but I do

20 recall that they had a panel of twelve subject matter

21 experts.

22 Q.

That's not my question. I'm asking you a very

23 simple question. Do you know if they did any critical

24 incidents from the position of lieutenant as opposed to

25 sergeant?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 84 of 140
84

1 A.

I, um -- we'd have to look at what the report says

2 because that's what I --

3 Q.

Did you look at the critical incidents when you

4 wrote your expert report to see if they in fact -- if we

5 had a group of critical incidents for lieutenants in the

6 appendices?

7 A.

Which appendix?

8 Q.

Well, let's go to, um -- I can help you on that.

9 Um, V.

10 A.

I'm sorry, did you say "Z" or "V"?

11 Q.

"V."

12 A.

(Turns.)

13 Q.

Dr. Campion, I've gone through this and I have

14 found one critical incident and one alone that says

15 "lieutenant," the rest -- the other few that are in here

16 all say "sergeant." Do you see otherwise? You're free

17 to look at them.

18 A.

(Looks.) Um, are you saying that since the

19 observation was of a lieutenant, that the critical

20 incident does not apply -- excuse me, to a sergeant,

21 that the critical incident does not apply to a

22 lieutenant's job?

23

THE COURT: No, I don't understand. If you're

24 asking him to clarify his question, I -- he claims to

25 have looked this over thoroughly, he claims to have

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 85 of 140
85

1 found one critical incident where it pertained

2 specifically to a lieutenant and a handful that pertain

3 to sergeants. Now, that's the asserted premise for his

4 question.

5

MR. LICHTEN: That's right, your Honor.

6

THE COURT: And the question is what, again,

7 Mr. Lichten, what's the question?

8 Q.

Is that accurate?

9

THE COURT: All right.

10 Q.

Or do you know?

11 A.

Well, I guess -- I, um -- let me make sure that I

12 agree with your observation. (Looks.)

13 Q.

If you look at the top right of each critical

14 incident, some say "sergeant," and I see several that

15 say "sergeant," and then it has -- and I see one of

16 those, and only one, that says "lieutenant."

17 A.

(Looks.) I'm sorry. This is just taking a

18 moment. I just want to make sure -- (Turns pages.)

19 Q.

Sure.

20 A.

Okay. So the one that you say applies to

21 lieutenants starts on Page 4029?

22 Q.

Well, let me get to that. (Turns pages.)

23 Exactly. (Turns pages.) Hold on. (Looks.) Oh, no,

24 4038, it says "lieutenant" at the top right.

25 A.

Um -- and this is very confusing because on 4029

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 86 of 140
86

1 it says "the title of the actor," that's the person

2 being described, it says "patrol supervisor-LT," but the

3 incident was apparently provided -- the title of this

4 SME was a sergeant. So apparently this is a sergeant

5 giving not one but two critical incidents on a

6 lieutenant.

7 Q.

Well, aren't you aware that in the Boston Police

8 Department patrol supervisors are sergeants not

9 lieutenants?

10 A.

It says "-LT".

11 Q.

Are you aware that in Boston patrol supervisors

12 are sergeants? You're not aware of that, are you?

13 A.

Okay. So it says "the patrol supervisor-LT" --

14

THE COURT: Well, let's go question by question

15 because I have to follow it.

16

MR. LICHTEN: Yes.

17 Q.

Are you aware that --

18

THE COURT: Go ahead.

19 Q.

Are you aware that in the Boston Police Department

20 a patrol supervisor is a sergeant position, it's not a

21 lieutenant position?

22 A.

I haven't a clue.

23 Q.

All right.

24 A.

But I know what I read here.

25 Q.

Okay. Now, a final question on this and I'll be

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 87 of 140
87

1 done with this part.

2

Is it your testimony that somehow these critical

3 incident technique reports made their way in the

4 analysis and construction of the 2008 Boston police

5 lieutenant's examination?

6 A.

I don't know.

7 Q.

You mentioned in your expert report that there

8 were these critical incident techniques that the Boston

9 Police Department did, but you don't actually have any

10 evidence before you that these ever found their way into

11 the 2008 Boston police lieutenant's examination, is that

12 right?

13 A.

Well, that's not exactly -- what I think I

14 described is the process by which these exams are

15 normally conducted or developed and how a job analysis

16 is used to drive the knowledges, skill requirements --

17 Q.

Dr. Campion, I'm just asking you whether you have

18 any evidence before you that these critical incidents

19 that we just went over in 1991 somehow made their way

20 into questions that were used in 2008, do you know or

21 don't you know?

22 A.

Well, no, I guess I don't know, but I wouldn't

23 expect that.

24 Q.

Okay. In fact you would agree that the 145 or 149

25 KSAs, knowledge skills and abilities, that were found to

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 88 of 140
88

1 be important that were eventually selected for some of

2 them to be used as questions for the Boston police

3 lieutenant's 2008 examination, those KSAs came from the

4 2002 Morris and McDaniel job analysis, that's right,

5 isn't it?

6 A.

I believe so.

7 Q.

Okay. Now, you also said that you reviewed the

8 sergeant exam because you were interested in that

9 because you saw that there seemed to be a lot of

10 similarity between the sergeant's position and the

11 lieutenant's position, is that right?

12 A.

Yes, as part of the comparison between the two

13 jobs I reviewed both of the exams for, I believe, both

14 years.

15 Q.

And you looked at the 2002 job analysis report --

16 job analysis report for sergeant to confirm your view

17 that that -- did that 2000 job analysis report confirm

18 your view that that testing was done properly, is that

19 right?

20 A.

Well, no, the report didn't describe testing, it

21 only described the job analysis --

22 Q.

The job analysis.

23 A.

-- and so I did look at the job analyses for the

24 two jobs in 2000, yes.

25 Q.

Okay.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 89 of 140
89

1

MR. LICHTEN: May I approach, your Honor?

2

THE COURT: You may.

3

MR. LICHTEN: Your Honor, this is 42 from Lopez.

4

(Hands up.)

5

MR. LICHTEN: Mr. Weber is happy every time I get

6 these documents out, it's a little less he has to carry

7 back.

8 Q.

Okay. I'm going to show you what's been marked as

9 Exhibit 42, it's in evidence from the Lopez case, and

10 ask you if you've seen this before?

11 A.

Yes, it appears to be the report from which I drew

12 a table to compare the two jobs.

13 Q.

Now, Doctor, did you notice anything unusual or

14 suspicious about this report, anything at all, when you

15 looked at it?

16 A.

No.

17 Q.

Well, you were aware that part of it contained

18 SMEs ranking certain tasks for certain scales such as

19 how important they are and what dimensions they require,

20 is that correct?

21 A.

Yes, I believe they used a panel of subject matter

22 experts. I read the lieutenant's report most carefully.

23 Q.

All right. And they were supposed to -- each

24 subject matter expert was supposed to answer each

25 question as to taking a particular KSA and how important

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 90 of 140
90

1 it was and how frequently they did it, is that right?

2 A.

Um, yes, I believe I'd have to study it in more

3 detail but that was -- yes, it does show here, "How

4 often do you perform this task?" "How important?" "Is

5 it necessary for" --

6 Q.

And these individuals were taking from a variety

7 of different job duties, is that right, some were in

8 administration, some were in specialty units, some were

9 in districts, are you aware of that?

10 A.

No, I see their names on Table 13 but I believe

11 that they are described -- again if the report models a

12 lieutenant report, then that's what they did.

13 Q.

What is the likelihood by chance that every one of

14 those 11 SMEs answered every question exactly the same,

15 what is the statistical probability of that?

16 A.

Um, I don't know, but one of the problems we have

17 with job analysis --

18 Q.

No, I'm just asking if you know what the

19 statistical probability is of that? You don't know, is

20 that your answer?

21 A.

No, I'm trying to explain it to you. May I

22 explain?

23 Q.

Sure.

24 A.

Okay. The job analysis surveys that get to, um,

25 subject matter experts are usually only the tasks that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 91 of 140
91

1 are found to be relevant to the job already, so you

2 often find that they're all or nearly all rated as

3 "important" because otherwise they wouldn't even be in a

4 survey. So it could be a high probability that they

5 would all respond positively.

6 Q.

Dr. Campion, that makes no sense. What's the

7 purpose of doing these ratings if everybody's going to

8 rate everyone as being "important," doesn't that destroy

9 the whole purpose of the job analysis?

10 A.

No -- no, it's a confirmatory step.

11 Q.

Okay.

12 A.

And so if you've done a good job in developing

13 your tasks and your KSAs, then when you're given the

14 subject matter experts, they confirm that they are

15 important.

16 Q.

So when you give a bunch of KSAs, 149, to subject

17 matter experts, you ask them each to rate them and they

18 all come out with the exact same score, then you say,

19 "Well, that's not surprising," and it doesn't affect, in

20 your opinion, the integrity of the process at all?

21 A.

I'm sorry, I've just explained that to you. I can

22 do it again if you'd like?

23 Q.

Well, let's first go to page -- let's start on

24 Page 24. You can start on whatever page.

25 You see in here that there's absolute agreement on every

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 92 of 140
92

1 knowledge, skill, and ability in this document, isn't

2 there?

3 A.

(Looks.) Um, I'd have to spend some time studying

4 it.

5 Q.

Doesn't that concern you, as a very reputable and

6 highly-regarded industrial organizational psychologist,

7 that you would rely on a KSA job analysis by subject

8 matter experts that has 100 percent agreement on every

9 part?

10 A.

Um, did they do it as a group?

11 Q.

Well, you tell me?

12 A.

Well, I'd have to look.

13 Q.

Do you understand that the instructions were they

14 were each to rate it individually?

15 A.

It was a task-rating session, which implies that

16 they may have met as a group. The session implies that.

17 Q.

Dr. Wiesen testified that the amount of time

18 allotted for that session could not possibly, just from

19 a mathematical calculation, have given them enough time

20 to do all of those together in a joint session through

21 some sort of an agreement process, do you agree with

22 that or do you know?

23 A.

Um, I haven't a clue of what Dr. Wiesen talked

24 about.

25 Q.

But it doesn't raise your hackles at all that

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 93 of 140
93

1 there's complete agreement on every KSA for every

2 dimension that's looked at, that doesn't concern you as

3 a social scientist?

4 A.

What are you implying, that somebody filled it out

5 for them and they didn't actually participate?

6 Q.

That's exactly what I'm -- I'm implying that this

7 was not a valid construction of a job analysis, that's

8 exactly what I'm implying, because how could it be

9 proper if everyone agreed on every facet, how could that

10 possibly be?

11 A.

Well, I'm just trying to explain it to you, it's

12 in part because we only present them with tasks and KSAs

13 that we know to be important or they wouldn't be on the

14 survey, and second of all it appears that they

15 participated in a rating session. I certainly would

16 think it's more reasonable that that would explain it

17 than a fraud.

18 Q.

Do you know?

19 A.

Well, again, Jesus, you might be --

20

THE COURT: Wait a minute. Just -- just answer

21 the question.

22 A.

I -- I know what I read here and it doesn't look

23 so odd that I want to question its validity.

24 Q.

Is there anything that you've seen that shows that

25 the instructions given to the SMEs were to meet and then

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 94 of 140
94

1 go over each one in a group fashion and come to a

2 consensus, is there anything that shows this was done by

3 consensus?

4

MR. SIMON: Objection.

5

THE COURT: Grounds?

6

MR. SIMON: Asked and answered, your Honor.

7

THE COURT: All right. He may answer it.

8 A.

All I can see is that they participated in a task-

9 rating session on page -- Bates 1561.

10 Q.

Okay. So I guess the answer to my question is

11 you're not surprised or concerned, as a reputable social

12 scientist, about the fact that there was complete

13 unanimity, is that your testimony, it doesn't concern

14 you at all?

15 A.

Not so much so that I would throw it out. I mean

16 it does seem high, I may ask some questions, but I

17 wouldn't think it was so extreme.

18 Q.

Well, did you ask questions?

19 A.

No, I don't think it's that extreme that I believe

20 it requires that.

21 Now, do you have the lieutenant's exam, I looked at that

22 one a little more closely.

23 Q.

In fact there was variation for the lieutenant's?

24 A.

Well, see, there you go.

25 Q.

Yeah, that's right, there you go, one seems to

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 95 of 140
95

1 have variation and one has no variation at all.

2 A.

So your point is?

3 Q.

I'll move on to something else.

4 I'd like to ask you about the training and experience

5 rating in this case. So as I understand it you made no

6 attempt to figure out what the actual nominal value was

7 of the T and E rating, is that correct, that is whether

8 it actually accounts for 20 percent or whether it

9 effectively could only count for 4 or 5 percent, you

10 don't know, is that right?

11 A.

Um, I can't answer that question because it's

12 illogical.

13 Q.

Well, you never studied the documents that showed

14 how the T and E is constructed in the City of Boston or

15 in the Commonwealth of Massachusetts, is that correct?

16 A.

No, I thought I looked at the point system and how

17 it works.

18 Q.

You did look at it?

19 A.

I thought so.

20 Q.

Do you know how it works?

21 A.

You get a certain number of points for work

22 experience and education and the nominal weight is 20

23 percent.

24 Q.

The nominal weight is 20 percent. But do you know

25 that you get 14 points just for showing up?

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 96 of 140
96

1 A.

No, you'd have to show me what you're talking

2 about.

3 Q.

Well, I'm asking you. You were the one who

4 studied this case and came out with an opinion that the

5 test is valid and I'm asking you whether you actually

6 studied the effective weight of the training and

7 experience on the entire examination?

8 A.

I didn't do a statistical analysis, no.

9 Q.

So whether it counts for 2 or 3 points effectively

10 or 18 or 19 points or 4 or 5 points, you don't know, you

11 haven't done the analysis, is that correct?

12 A.

I have not done an analysis of that.

13 Q.

And you weren't here at trial when Dr. Wiesen

14 testified that you get 14 points just for showing up for

15 the examination of the 20, so that reduces its actual

16 effective percent weight in the overall examination

17 process, you weren't here for that, is that right?

18 A.

I was not here for his testimony, no.

19 Q.

Did you read it in his report?

20 A.

No.

21 Q.

All right. Now, you're aware of some literature

22 on training and experience ratings, aren't you?

23 A.

Sure.

24 Q.

(Silence.)

25 A.

"Sure," I said. I'm sorry.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 97 of 140
97

1 Q.

I'm sorry, I didn't hear your answer. I'm sorry.

2 A.

I said "Sure."

3 Q.

Okay. And, um, you would agree that training and

4 experience ratings, using a point system like this, have

5 one of the lowest correlations of any type of testing or

6 promotional mechanism, is that correct?

7 A.

No, that would not be correct.

8 Q.

That would be incorrect?

9 A.

That would be incorrect.

10 Q.

(Hands document to clerk and witness.) Are you

11 familiar with this article?

12 A.

Yes.

13 Q.

Okay. In fact you're not just familiar with this

14 article, but you cited this article in one of your

15 footnotes to your expert report, is that correct?

16 A.

That's right.

17 Q.

And you know Frank Schmidt, is that right?

18 A.

Yes.

19 Q.

And he's quite renown in the field, is that right?

20 A.

Oh, yes.

21 Q.

All right. And you respect his work, is that

22 correct, in fact you cited this study in your expert

23 report?

24 A.

Um, yes, and it has limitations, but it makes some

25 contributions to the literature.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 98 of 140
98

1 Q.

Okay. Well, if you go to Page 265, Table 1.

2 A.

(Turns.)

3 Q.

Now, just so we're clear, this is a, um -- this is

4 an article published by the American psychological

5 Association, is that right?

6 A.

Yes.

7 Q.

And it was published in 1998, is that right?

8 A.

That's what it says, right.

9 Q.

Okay. And if you go to Page 265, it has a table,

10 does it not?

11 A.

Yes, there is a table.

12 Q.

And in the first column in the table under

13 "Validity," it has -- based on published papers and

14 studies, it has validity quotients for various types of

15 performance or measures of what one would look for

16 either in hiring or promotion, is that right?

17 A.

Yes.

18 Q.

Okay. And validity is the higher the number the

19 more validity there is, is that correct?

20 A.

Yes.

21 Q.

Okay. And just so we're clear, these studies are

22 based upon criterion-related studies not content-

23 validity studies, is that correct?

24 A.

Oh, yes, that's right.

25 Q.

Right. So when we look at this published

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 99 of 140
99

1 literature, generally the published literature of this

2 criterion validity, that is they've actually studied

3 this issue of how well people do with certain tests with

4 respect to their job performance using actual subject

5 matter people that they look at, is that right?

6 A.

Um, actually I think you really messed that up.

7 Q.

I think I did mess that up.

8 Criterion validity is actually a study of how -- what

9 people -- how people actually perform and how they did

10 on these tests, is that right?

11 A.

Yes, criterion-related validity is a statistical

12 correlation between performance on the test and

13 performance on some criterion of interest and that may

14 be job performance or it may be something else.

15 Q.

Got you. Okay. I got the second one better.

16 Okay. And if you look at that, this said that the

17 highest of what GMA -- I'm guessing that's "General

18 Mental Ability" as in Point 51, is that correct?

19 A.

Yes, that's right.

20 Q.

But then below that they say "Work sample tests

21 are a Point 54," that's a high validity, is that right?

22 A.

Well, you're looking now at "work sample tests"?

23 Q.

"Work sample tests."

24 A.

That's a bigger number, you're right.

25 Q.

So "work sample tests" could be like an in-basket

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 100 of 140
100

1 where you sit down and they say, "Okay, this problem

2 just occurred, do something," and they judge how you've

3 done on it, and that could be a work sample test, is

4 that right?

5 A.

Um, no, more often a work sample is an actual

6 piece of the work. I mean an in-basket is more of a

7 simulation. But as I explained in my deposition the

8 other day, these are only -- only the very best studies

9 can get published and also we don't know how many

10 studies underlies each of these. That's what I was

11 saying. There's real limitations to these --

12 Q.

I haven't asked you that question, Dr. Campion,

13 and I'm sure your counsel will ask it, but just to

14 follow up, you're saying that these are based on good

15 studies, is that correct?

16 A.

I'm saying that usually we get published studies

17 which are generally stronger studies with better

18 results. And so it creates an upward bias in the

19 estimates in the literature.

20 Q.

But wouldn't that be an argument for if you're

21 doing a promotional test, if you do the best possible

22 job, you're likely to get stronger correlations than if

23 you really go out and try to test for certain

24 knowledges, skills, and abilities, and you do it right,

25 then you're likely to get a higher correlation or

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 101 of 140
101

1 validity quotient than if you do a poor job, isn't that

2 right?

3 A.

It's easy to do a good job with job knowledge.

4 It's hard to do a good job with --

5 Q.

I didn't ask you that. (Pause.) Let me move on.

6 If you continue to look at this, "job knowledge tests"

7 are listed and they have high validity of .48, is that

8 correct? "Job knowledge tests" .48 validity?

9 A.

That's what it says here.

10 Q.

Right, just below "work sample tests," right, .54?

11 A.

I'm sorry, I thought you were saying "job

12 knowledge tests" and that has a 48.

13 Q.

.48 validity, that's good, right, it's what you've

14 been saying in this case?

15 A.

Yes, there's a lot of evidence in job knowledge

16 tests, there's not really much in job samples.

17 Q.

But, Dr. Campion, that .48, that's just based on

18 the best studies, that's not a fair number, you just

19 said that with respect to work sample tests, right? So

20 you don't think that .48 really holds up under your

21 analysis that these are only the best cherry-picked

22 studies?

23 A.

No, I didn't say that.

24 Q.

Oh, so the .48 is accurate but the .54 is not?

25 A.

There's probably multiple times more studies for

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 102 of 140
102

1 the job knowledge tests in general mental ability and

2 employment interviews than there is for these other

3 selection procedures and that's why -- I warned you of

4 that during my deposition that this study is based on

5 the accumulation of other meta analyses and so it

6 doesn't even show the number of primary studies. So you

7 have to be very very careful here in terms of how to

8 interpret these results.

9 Q.

Well, in their report Morris and McDaniel, in

10 2002, say that they believe the validity for an

11 assessment center such as they ran has been shown, and

12 this is in 2002, to be at the .38 level, and I won't go

13 back to that, but it's in there. Do you think that's

14 accurate that a good assessment center could have .38

15 validity?

16 A.

That was based on the study by Daubler, et al in

17 '87.

18 Q.

And do you think they're right or wrong?

19 A.

Well, I'm saying that it's a very old meta

20 analysis and so --

21 Q.

I'm just asking you what your opinion is, do you

22 think that a good job assessment center could have

23 validity about .38?

24 A.

Could?

25 Q.

Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 103 of 140
103

1 A.

Well, sure it could.

2 Q.

Okay. Now -- just a long way of getting down to

3 the T and E. They have -- towards the bottom, the "T

4 and E point method." Do you see that?

5 A.

Yes.

6 Q.

And that's what we're talking about in this case,

7 we're talking about a "Training and Experience point

8 method," is that correct?

9 A.

Um, can you help me find a copy of the T and E

10 rating system, so I can review it, because I can't

11 answer this question without that.

12 Q.

You know it's a -- what is unclear about the

13 question? You know it's a point system, don't you?

14 A.

Well, no, um, there's several different kinds of

15 point systems and if you looked there's also something

16 on this chart that's called a "T and E behavioral

17 consistency method."

18 Q.

And that's a behavioral consistency method and

19 that's not what was used in the Commonwealth of

20 Massachusetts, is it?

21 A.

Can you show me there?

22 Q.

I'm asking you if you know. You're the expert.

23 Do you know, as you sit here today, which method was

24 used for the training and experience?

25 A.

I'd like to look at the methods.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 104 of 140
104

1 Q.

So you don't -- as you sit here today, you don't

2 know?

3 A.

Well, I've read a lot of documents or I don't have

4 them all memorized. If you can show me that, perhaps we

5 can clarify this.

6 Q.

In any event you would agree that the training and

7 experience method of setting a training and experience

8 score has a very low validity, .11, which is in the very

9 low field of validity, is that right?

10 A.

No, I wouldn't agree with that at all.

11 Q.

So you disagree with that analysis in a report

12 that you cited in your --

13 A.

Right, I would disagree with that. We don't know

14 how many primary studies went into this and we don't

15 know whether this is an exact match with what we have

16 here in Boston. So --

17 Q.

Can you cite it? Can you cite me a study, as you

18 sit here now, that you relied on that has a description

19 of what validity they assigned, if any, to a T and E

20 point system in a promotional process, is there any

21 study that you have in mind that shows that there's

22 higher validity?

23 A.

Well, I'd have to go back and look but I think

24 there was an article by McDaniel and others and then

25 there was another article by Neil Schmidt.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 105 of 140
105

1 Q.

Well, what do you think the validity is of the T

2 and E point system?

3 A.

Well, I don't know here, we don't have

4 criterion-related validity data. I mean this is not the

5 sort of thing you could just guess about.

6 Q.

Okay. Now, if you continue on with this article

7 again that you've cited in your report, they have a

8 section about the increase in validity that one can get

9 by adding together certain tests. This is what I think

10 the judge has been asking about, that if you add certain

11 tests together you increase the validity because you're

12 testing for more of the job domain, do you agree with

13 that?

14 A.

Um, hold on. The question was is that possible

15 hypothetically?

16 Q.

No, no, no -- um, yes, let's start there.

17 A.

Well, sure, it's possible hypothetically that

18 adding additional selection procedures may give you what

19 we call incremental validity.

20 Q.

Right. And this article, this table again from a

21 report, an article that you cite in your expert report,

22 this tries to quantify the increased validity that you

23 can get, isn't that right?

24 A.

Yes, using meta analytic techniques, yes.

25 Q.

Yeah, using meta analytic techniques would be

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 106 of 140
106

1 looking at big good studies that have been done in the

2 field?

3 A.

No, that's the problem and I tried to explain this

4 also in my deposition. These are different studies so

5 the studies that -- for one selection procedure are not

6 the same studies for another selection procedure. So

7 the incremental validity cannot be calculated with

8 certainty. It is estimated based on the validities

9 here. And that's what makes it such a "crap shoot."

10 And I don't mean to use technical terms.

11

THE COURT: No, but I -- no, I need to understand

12 it. But this Schmidt paper, I think fairly read,

13 purports to say, for example, that if you added work

14 sample tests you'd have a 24 percent increase in

15 validity. It says that, doesn't it?

16

THE WITNESS: That's what it says and I'm saying

17 that --

18

THE COURT: Well, wait a minute, that's what it

19 says and the -- I have not read the report, but that's

20 not some sort of straw man, the report purports to

21 validate that, isn't that right? I'm not asking whether

22 you agree with it, that's what this report purports to

23 say?

24

THE WITNESS: It is an estimate, it is not proof.

25

THE COURT: Right.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 107 of 140
107

1

THE WITNESS: And it's based on --

2

THE COURT: Right, it's an estimate. But this

3 report purports to make those estimates?

4

THE WITNESS: Well, it is, yeah. It does.

5

THE COURT: Yeah, they would make those estimates?

6

THE WITNESS: Uh-huh.

7

THE COURT: All right.

8

Go ahead, Mr. Lichten.

9

MR. LICHTEN: Your Honor, I would ask to move that

10 report in not for the truth of the matters asserted

11 therein, but for some indication of what literature was

12 out in the field by, um, you know, 2006 and 2007 with

13 respect to if searches had been made.

14

MR. BOK: Well, your Honor, he is trying to get at

15 it for the truth of the matter, he's --

16

THE COURT: Well, I'm not -- I'm able to sort it

17 out. He wants it for the limited purpose that it was

18 said on or about the date of publication. That's all.

19 I can handle that as a factfinder. I question how

20 relevant it is and he sought to answer that. He says

21 it's relevant to know that at least reputable scientists

22 were saying these things back at about that date. Not

23 that it's in as a learned treatise, he can't, with this

24 witness, establish that basis. And isn't it at least

25 relevant that the statements were being made back in

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 108 of 140
108

1 1998?

2

MR. BOK: All right, your Honor, if -- as to

3 whether it's relevant, your Honor, let me explain why.

4

THE COURT: Yeah.

5

MR. BOK: First of all, I believe that it's

6 hearsay, but putting that aside --

7

THE COURT: He doesn't offer it for a hearsay

8 reason. It is hearsay. There's no doubt about it.

9 It's hearsay. I cannot make any findings based on

10 accepting the truth of this document as the record now

11 stands. But as we frequently have in cases, it may be

12 relevant that competent scientists were saying this back

13 then. This is not some new concern. And I'll go

14 further. It appears from this witness's testimony and

15 that of others that this problem of the effect of

16 aggregating different methods of testing has been a

17 persistent problem and a very naughty one, it appears.

18

MR. BOK: Well, then I would argue, your Honor,

19 that even if you do find it to be relevant and not

20 excludable under the hearsay rule, that its probative

21 value is really limited because of the fact that you're

22 taking it in as one article in a sea of literally

23 thousands of articles --

24

THE COURT: It may be. It may be. That's right.

25 But I am going to admit it, Exhibit 82, but it's limited

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 109 of 140
109

1 not for the truth, but for the fact that it was said on

2 or about the date it was said. Now, it may be they'll

3 provide some other evidence, but for now it's limited.

4

Go ahead, Mr. Lichten.

5

MR. LICHTEN: Thank you, your Honor.

6

(Exhibit 82, marked.)

7 Q.

Okay. Going back to your Ernst report for a

8 second, Ernst vs. the City of Chicago, do you recall --

9 and I'm just trying to speed things up.

10 Do you recall making the assertion I asked you about in

11 your deposition that because these EMTs had already

12 become EMTs and therefore had to pass a physical and a

13 practical test to actually get the certificate to begin

14 with, that it was less likely that they needed to be

15 tested for these physical attributes when they were

16 going on the Chicago Fire Department, or trying to,

17 because they had already shown competence in the area?

18 A.

Yes.

19 Q.

Okay. Now let's try to apply that here.

20 In this case you've testified, under oath, that you

21 found great similarity of -- it sounds like close to the

22 same job, according to you, between the sergeant's

23 position and the lieutenant's position, is that right?

24 A.

Well, if you recall, looking at the knowledge

25 areas, that the knowledge areas were 80 percent in

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 110 of 140
110

1 common, but based on the job analyses that were

2 conducted.

3 Q.

All right. And in this case you have, as the pool

4 of candidates, a group of sergeants who have already

5 taken and not only passed the sergeant's exam, which is

6 very similar to the lieutenant's exam, but have actually

7 scored high enough to get themselves in the ranks where

8 they could be reached for promotion, isn't that correct?

9 A.

Yes.

10 Q.

So in this case you don't have a pool of

11 applicants or a pool that haven't shown themselves

12 competent to know the technical knowledge which a

13 sergeant needs to know, the pool of candidates is

14 someone who's sort of self-selected, people who have

15 already shown themselves to have the technical knowledge

16 and competence necessary to go on and be a lieutenant,

17 isn't that right?

18 A.

Um, I don't know that. I've lost when you were

19 saying that people have shown they have the technical

20 knowledge and competence to be a lieutenant, that's

21 where I didn't --

22 Q.

They've already passed a test, 80-which questions

23 which were the same as the questions on the lieutenant's

24 exam, and they've not only passed such a test in the

25 past but they've done sufficiently high to be promoted

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 111 of 140
111

1 to the rank of sergeant, isn't that right?

2 A.

Oh, yes, they're sergeants.

3 Q.

Right. So that would justify -- and the Court

4 asked you about this, I think, on Friday, that you could

5 make an argument, could you not, that having shown the

6 ability to know the technical knowledge needed for the

7 job, you could say these candidates have shown that they

8 have the ability and therefore they should go on, in a

9 multiple hurdle kind of situation, to take other tests

10 that might differentiate themselves with respect to

11 skills and abilities that are critical for a police

12 lieutenant to know such as oral communication,

13 situational judgment, reasoning, things of that nature?

14

THE COURT: That's a long question, Mr. Lichten.

15

MR. LICHTEN: Sure.

16

THE COURT: Allow me to rephrase it and then if

17 you want to go back, fine.

18

MR. LICHTEN: It's the last time I'll ask.

19

THE COURT: I think this is what he's asking you.

20 These are already sergeants so at sometime they've

21 passed a sergeant's exam plus I think there's some time

22 in grade so they've been working as sergeants and I have

23 some evidence that some of them have been working as

24 acting lieutenants.

25

So against that background, doesn't it make sense

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 112 of 140
112

1 either to weight the written portion of the lieutenant's

2 exam less or to add more experiential -- um, valid

3 experiential exam points for lieutenant?

4

THE WITNESS: I suppose you could.

5

THE COURT: Well, it makes sense, but does it

6 commend itself to you?

7

THE WITNESS: Well, whether it would be more valid

8 than a competitive knowledge exam is not at all clear.

9 The fact that they have established that they were

10 competitive for a sergeant means they have some, but,

11 you know, over time people accumulate more knowledge,

12 some people --

13

THE COURT: -- burn out.

14

THE WITNESS: They burn out. So when it's time

15 for the competition for lieutenants, normally in most

16 organizations it's a new what we call "tournament," it's

17 a tournament model of promotion where in order to be in

18 the competition for lieutenant they have to have gotten

19 to sergeant already, so your knowledge has already been

20 considered up to that point, and then it's a competitive

21 and limited opening competition for the next level. So

22 that's what we have here.

23

THE COURT: All right. I tried my question. You

24 go ahead with yours, Mr. Lichten.

25

MR. LICHTEN: Okay.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 113 of 140
113

1 Q.

Well, that competition that you've described, that

2 could actually be a competition to test for those

3 abilities and skills that were never tested for in the

4 sergeant's exam namely the kinds of judgment and

5 abilities and reasoning that you can get at with these

6 alternative selection procedures such as an in-basket,

7 video scenarios and the like, isn't that right?

8 A.

I suppose. I don't know that it would be more

9 valid, but, you know, it's a possible idea.

10 Q.

But it might be equally valid because assuming two

11 things -- assume two things in this hypothetical.

12 Assuming that you had to pass and get a high score on

13 the sergeant's exam in order to be promoted to sergeant

14 and then you take and at least get a passing score at

15 some cut-off level that shows you're competent on the

16 lieutenant's exam to get on the list, wouldn't that --

17 and then you add to that process these assessment center

18 exercises, that could actually be equally or more valid

19 than a lieutenant's exam that only tests for job

20 knowledges in a multiple choice fashion given that these

21 sergeants have already taken such a test and passed it,

22 isn't that right?

23 A.

(Laughs.) Wow, I'm not sure I can follow all of

24 that, but the piece that I did hear that I would not

25 support would be to use their sergeant's score years

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 114 of 140
114

1 later for lieutenant promotions.

2 Q.

That's not what I said. No, I'm saying that --

3 maybe my hypothetical wasn't clear enough, sorry I was

4 so longwinded.

5

What if you just required sergeants to take and

6 pass, with a reasonable cut-off score, the lieutenant's

7 multiple choice written knowledge test and then you

8 added to that these other tests, the assessment centers

9 that test for these other skills and abilities not

10 tested on the multiple choice job knowledge test, you

11 would likely have a more valid exam, isn't that correct?

12 A.

You know I don't know, I don't know because these

13 other things you may add in might water it down. So

14 using a very good selection procedure, averaging it in

15 with a selection procedure that's not so good, could

16 very well dilute the validity and it all depends on a

17 lot of factors including how you've weighted them and

18 whether there were cut-offs and whether high scores for

19 one could compensate for low scores on another, so it's

20 trickier than it sounds. And I honestly don't know,

21 without much more detail, how to -- well, I can estimate

22 it mathematically but I will need more details to do so.

23 Q.

But you would agree -- you've written about

24 something called "range restriction," is that correct?

25 A.

I'm familiar with "range restriction" research

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 115 of 140
115

1 issues and I've authored an article recently on that.

2 Q.

All right. I'm just trying to cut to the chase

3 here. Some studies are not as valid -- and I'm using

4 this in a different way, as they might be "because of

5 range restrictions on the people who are being studied,"

6 is that correct?

7 A.

That's right, that's one of the big ones, that and

8 unreliability tend to reduce our ability to detect

9 criterion validities.

10

THE COURT: What's -- "range restrictions within

11 the group of people being tested," what did you mean by

12 that?

13

MR. LICHTEN: Your Honor, I think I'm going to get

14 at that, if you'll indulge me.

15

THE COURT: I will. I'll withdraw. Go ahead.

16 Q.

So, for example, and this is just one back-up

17 question to that.

18

You talked about this meta analysis, the judge

19 asked you some questions and you said, "Well, look, your

20 Honor, job knowledge tests are known to have a high

21 validity based upon all this meta analysis that's out

22 there," is that correct?

23 A.

Well, um, based on the article you showed me, yes,

24 and job knowledge tests have a long history with lots

25 and lots of studies.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 116 of 140
116

1 Q.

Based upon meta analysis, is that right?

2 A.

Yes, these data here that you've showed me are

3 based on meta analytical results which average across

4 studies.

5 Q.

Right. But where you have range restriction,

6 which I'll get to in a second, that could impair the

7 integrity of the study that you're looking at, is that

8 right?

9 A.

Yes, that's one of the critical issues.

10 Q.

Okay. So in this case if you have -- if you're

11 looking at -- if you actually study the Boston Police

12 Department promotional exam, here you have sergeants who

13 have already taken and passed and have received a

14 high-enough score on the job knowledge test to be

15 considered -- um, to be promoted, you would have "range

16 restriction," that is you would have a group that's not

17 a group just out in the stratosphere who are all types

18 of persuasions and all types of abilities, you would

19 have a group that has already sort of self-identified

20 themselves as being proper candidates for the position,

21 and that's "range restriction," isn't that right?

22 A.

Um, no, "range restriction" would be if you, um,

23 only promoted a small number of lieutenant candidates

24 and then correlated their scores with job performance

25 because they don't now any longer represent the full

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 117 of 140
117

1 range of all the sergeants who took the exam. That's

2 "range restriction."

3 Q.

But here you wouldn't be testing all the people

4 who are police officers, you'd only be testing --

5

THE COURT: Wait, I need to get --

6

MR. LICHTEN: Sure.

7

THE COURT: We have to deal with his testimony.

8 Now he gave a definition of "range restriction" and let

9 me give it back to you so I understand it.

10

You say "range restriction" is, in conducting

11 validation studies, it would be to look only at the

12 people who were selected, for example, and say if the

13 test is valid on whatever basis -- and as I get the

14 sense of your testimony, "range restriction" in that

15 sense would be a mistake because it doesn't look at the

16 body of people who took the exam, both those who pass

17 and those who don't pass and also those who were

18 selected and those who were not selected, have I got

19 your testimony right?

20

THE WITNESS: Yes.

21

THE COURT: So "range restriction," as you're

22 using it, is not a valid way of corroborating the

23 validity of a test?

24

THE WITNESS: That's right, it makes the validity

25 smaller.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 118 of 140
118

1

THE COURT: Yes. All right.

2

Now, Mr. Lichten --

3

MR. LICHTEN: Well, that's the point I'm making.

4

THE COURT: Well, go ahead. It may well be.

5 Q.

The point is that with respect to the Boston

6 police lieutenant exam the meta data that you testified

7 to that shows the high correlation of job knowledge

8 tests may not be as applicable if you actually tested

9 that hypothesis in the Boston Police Department because

10 the sergeants taking the lieutenant's job knowledge test

11 have already shown themselves to be superior individuals

12 with respect to having some job knowledge, isn't that

13 right, that's what "range restriction" is?

14 A.

No, I'm sorry, in the context of validation we

15 don't use it for the fact that the candidate population

16 may not be at a random draw of the general population,

17 that is always the case, and that's not when we use the

18 term "range restriction" in a validity context, it is

19 not referring to the fact that the candidates, as a

20 total group, may be somewhat restricted because they've

21 gotten promoted up to that level. I mean that may be

22 important to you to think about that, but usually when

23 we refer to "range restriction" it's looking at the

24 correlation between test scores and job performance and

25 the restriction is that those who are selected don't

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 119 of 140
119

1 have as wide a range of test scores as that whole

2 candidate population.

3 Q.

Okay. And I'll leave this in a moment, but

4 wouldn't you expect this population not to have as big a

5 range in test scores as the general population, just as

6 you just said, on a lieutenant's exam because they've

7 already passed the technical knowledge and done well on

8 the technical knowledge of the sergeant's exam which has

9 many of the same or similar questions?

10 A.

Yeah, but that's a given in the situation. I mean

11 we're not testing people off the street for the

12 lieutenant's exam, they would probably show much lower

13 job knowledge. So the context is one where the

14 candidates themselves are, you know, not a random draw,

15 but that's the way most validation studies are, unless

16 they look at entry-level hiring.

17 Q.

Well, let me ask it this way.

18 You told the judge that you believe that technical

19 knowledge, a job knowledge test, have generally, based

20 on this meta data, high validity, is that correct?

21 A.

It's got a very strong history as being a good

22 selection procedure, perhaps one of the most commonly

23 used historically across, you know, considering

24 employment, education, licensure. I mean it's --

25 Q.

Do you know one police promotional exam study,

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 120 of 140
120

1 whether it's published, whether it's not studied, any

2 data that you can cite to me that shows that there's a

3 high correlation between how you do on a multiple choice

4 job knowledge test and how you perform, using a

5 criterion-validity study, in the job of a police

6 supervisor, can you cite me one study of any type?

7 A.

I have not reviewed the literature on that, but I

8 would be real surprised if we couldn't find literature

9 on that.

10 Q.

Okay. Now, you were hired to be an expert witness

11 in this case, is that correct?

12 A.

Yes.

13 Q.

And you didn't try to find whether this

14 hypothesis, that a job knowledge multiple choice test

15 would have high validity with respect to a police

16 promotional exam, you made no attempt to look for such

17 literature?

18 A.

Um, there's so much evidence on certain types of

19 selection procedures --

20 Q.

Doctor, did you make an attempt?

21 A.

No, I didn't think it was necessary.

22 Q.

Okay. So you don't know of any study?

23 A.

No, I didn't think it was necessary. That's a

24 different answer.

25 Q.

Well, let me move on.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 121 of 140
121

1 Now, with respect to the issue of feasibility, you did

2 not do anything in this case to systematically look at

3 the question of whether adding an assessment center or a

4 structured oral interview or some of these other

5 assessments that look at these other characteristics

6 other than job knowledge would or would not be feasible

7 in this case, is that correct?

8 A.

I did not do an analysis of the cost, but it's

9 obvious.

10 Q.

Well, you say it's obvious. You didn't do a

11 feasibility study, is that correct?

12 A.

Well, I know a lot about the costs of assessment

13 centers. I've developed many.

14 Q.

You didn't do a feasibility study, is that

15 correct?

16 A.

What do you mean by "feasibility"?

17 Q.

Do you remember me asking you at your deposition

18 about this and you said, quote, "I haven't done anything

19 systematically on this on the issue of feasibility for

20 this case," do you remember saying that?

21 A.

I think so.

22 Q.

And was that true?

23 A.

Well, I guess it depends on what we're talking

24 about, but I haven't put pencil and paper and actually

25 outlined the project proposal and what it would cost.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 122 of 140
122

1 But I know it would be a very meaningful cost.

2 Q.

Well, you didn't call Landry Jacobs -- you didn't

3 call EB Jacobs or Morris and McDaniel, you didn't call

4 Dr. Wiesen or anyone like that, and try to find out what

5 these things would cost, is that right?

6 A.

Well, I did understand the 2002 cost 2.2 million.

7 Q.

Well, that's not my question, my question is did

8 you make any calls?

9 A.

I wouldn't need to, sir, I do this for a living

10 and I know what these things cost.

11 Q.

Well, you've never done these promotional exams,

12 you've never done one in your life?

13 A.

But I could estimate the cost without ever having

14 done one.

15 Q.

(Pause.) Now, it's also true that you don't know

16 whether it would have been feasible to do a structured

17 oral interview in this case, is that correct?

18 A.

In the sense of would it have been possible to do

19 it?

20 Q.

Yes, exactly.

21 A.

Again it would have cost, but perhaps it could

22 have been done. There are administrative aspects, but

23 they're not insurmountable I wouldn't think.

24 Q.

(Pause.) I'd like you to turn to Exhibit 55.

25 A.

(Turns.)

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 123 of 140
123

1 Q.

Okay, before we get to that, we've established,

2 have we not, that there was a validity report done in

3 1991 for the sergeant examination, is that correct, a

4 validity report done by HRD?

5 A.

Yes, a content validity study in 1991. Yes.

6 Q.

Well, it's actually called a "validity report,"

7 isn't that right?

8 A.

Yes.

9 Q.

Okay. And that is typical in your trade that

10 after you construct an examination of this type you do

11 something at the end called a "validity report" to

12 report on what happened, is that right?

13 A.

Sometimes, yes.

14 Q.

All right. And in 2002 we now know, because we've

15 looked at it earlier today, that there was a validity

16 report, is that correct?

17 A.

Yes, that's what you showed me today.

18 Q.

But you haven't seen any validity report for 2005

19 and 2008, is that correct?

20 A.

I have not.

21 Q.

If you can turn to -- and the validity report --

22 and I just want to be clear on this if it's not already

23 clear, that you do a validity report after you've given

24 the examination, not before, is that correct, because

25 one of the things you're doing is reporting on the

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 124 of 140
124

1 administration, isn't that right?

2 A.

Um, not exactly. Really there is no specific

3 right or wrong. You sometimes have validity reports,

4 you sometimes do not, it depends on the budget, it

5 depends on whether there's likely to be litigation, it

6 depends on how big a project it is, and how you write it

7 again is stylistic. Again I write reports as the

8 project is conducted so I don't get to the end and have

9 to remember everything. So I, you know, kind of clean

10 as I go.

11 Q.

In 1991 the validity report was written after the

12 fact, is that correct?

13 A.

Oh, I don't know that.

14 Q.

But you do know that because their scores were in

15 there, so it was written after the fact, wasn't it?

16 A.

Well, how do you know it wasn't written in part

17 while they were doing it and they just happened to

18 complete it after the facts and put the date of

19 completion on there.

20 Q.

Well, do you know?

21 A.

Well, that's a -- again I've written hundreds of

22 these things and I've read thousands.

23 Q.

And 2002 was written after the test was given,

24 isn't that right?

25 A.

As I say, I don't know for the reasons I've

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 125 of 140
125

1 explained.

2 Q.

Okay. Now, you testified that you have reviewed

3 all these materials and you mentioned -- one of the

4 exhibits you mentioned was Exhibit 55 and you believe

5 that, um, HRD did a sufficiently good job with respect

6 to constructing this examination, that you found that it

7 was valid, is that correct?

8 A.

Well, I need to do it. What am I looking at here?

9 Q.

Yes, you mentioned that these documents that you

10 had looked at, it showed that HRD had constructed a

11 multiple choice exam in 2005 and 2008 that you thought

12 was valid?

13 A.

Well, what I think I testified to is that in '91

14 there was a large validation study and a process

15 established for developing these exams and that every

16 three years or so, with a couple of exceptions when the

17 City was trying alternative selection procedures, it

18 basically did the same process that was laid out in '91,

19 and they also did one kind of major update to the job

20 analysis which was the 2000 job analysis study. And

21 then right before the exams they did just a quick check

22 to make sure the job didn't change by having a small

23 sample of subject matter experts look at the tasks and

24 the KSAs, which is what we have in front of us. But

25 that was a confirmatory step, though, the primary

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 126 of 140
126

1 validation data was all the earlier material.

2 Q.

Well, isn't it your testimony in this case that

3 you believe that HRD, in 2005 and 2008, constructed a

4 multiple choice job knowledge lieutenant's promotional

5 exam for the Boston Police Department that you believe

6 was valid under various guidelines?

7 A.

Yes, I reviewed it compared to the Uniform

8 Guidelines and the SIOP principles.

9 Q.

Okay. So can you tell me, if you can, what

10 Exhibit 55 is?

11 A.

Well, I believe Exhibit 55 is a review of the

12 tasks of the job, um, to ensure that they are still

13 applicable at the time that the 2008 exam was developed.

14 And these are the task ratings. I also believe there

15 are -- yes, the next tab has the KSA ratings.

16 Q.

So what do we make of where it says "RT SME 2.53

17 RT SME PRY 1," a bunch of "1s," after that "GK" a bunch

18 of "3s," what is this telling us?

19 A.

Well, what I believe that it was was the ratings

20 by subject matter experts as to the extent to which

21 these tasks are still applicable to the job.

22 Q.

And do you know who these people are?

23 A.

No, I was thinking that those letters reflected

24 perhaps initials of the people, but I actually -- I do

25 not know who they are.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 127 of 140
127

1 Q.

And do you know why one SME, on the second column,

2 "RT SME PRY," has nearly all "1s" and why if you go to

3 the column "GK," that's nearly all "2s," do you know?

4 A.

No, I -- when that happens it's very often that

5 they have different jobs and so they see the tasks and

6 requirements differently.

7 Q.

(Pause.) Now, you were the editor of "Personnel

8 Psychology," is that correct?

9 A.

Yes.

10 Q.

Okay. So that's a rigorous journal, can we agree

11 on that, because you're the editor of it?

12 A.

It's considered to be one of the A-class journals

13 in my field.

14 Q.

Absolutely. And you're also familiar with two

15 gentlemen, Bopko and Ruaf, is that correct?

16 A.

Oh, sure.

17 Q.

They're heavyweights in the field?

18 A.

Well, I was going to say Ruaf will want to hear

19 that. I can't wait to tell him.

20 Q.

Well, do you think he's a heavyweight? It doesn't

21 matter what I think.

22 A.

I think they're both real solid people. I do.

23 Q.

All right.

24

MR. LICHTEN: May I approach, your Honor?

25

THE COURT: You may.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 128 of 140
128

1

MR. SIMON: Your Honor, if I may bring up a

2 problem that I think we're running into here?

3

THE COURT: Okay.

4

MR. SIMON: The parties had agreed that

5 Dr. Campion would be concluded today, um, and he's not

6 available the week of the 5th or the first week in

7 January that we'd be coming back. Um, I'm not sure

8 where we're at in Attorney Lichten's --

9

THE COURT: Well, if he's agreed he'll be

10 concluded, then he'll be done by 1:00. Are you

11 satisfied with that?

12

MR. SIMON: Well, I would like a chance for

13 redirect.

14

THE COURT: Well, I know you would, but apparently

15 you haven't worked that out. I don't know what you want

16 to do, do you want me to come back this afternoon? I

17 can.

18

MR. SIMON: If necessary. That would be

19 acceptable, your Honor.

20

THE COURT: Are you okay with that?

21

MR. LICHTEN: Um, I have a flight to visit my

22 family, um, and it's 5:00.

23

THE COURT: I mean I thought you people had worked

24 this out.

25

MR. LICHTEN: Well, your Honor, I will accommodate

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 129 of 140
129

1 them in any way, I will agree to a videotape of it, I

2 will agree to teleconferencing. I don't want to

3 prejudice them. I have taken a long time, but I think

4 I've been relevant and I don't think I've wasted time.

5 But nevertheless I will accommodate them in any way. I

6 don't want to prejudice Mr. Simon.

7

THE COURT: We'll assume that it can be worked out

8 and we'll run to 1:00. He'll be done by 1:00, in view

9 of his agreement, and it seems to me when I say "work it

10 out," I've got to, in some fashion, receive your now

11 redirect, so it will have to be within the scope of the

12 cross. But I do want to hear your redirect. And I'll

13 accept that.

14

Go ahead, Mr. Lichten.

15

MR. SIMON: Thank you, your Honor.

16

MR. LICHTEN: Thank you, your Honor.

17 Q.

Okay. So this is an article from 1999 from "The

18 Journal of Personnel Psychology." Were you the editor

19 at the time?

20 A.

No, my editorial period was '90 to '97.

21 Q.

Okay.

22

THE COURT: But just to cut to the chase here, I

23 take it you think this is authoritative?

24

THE WITNESS: Oh, sure, it's a -- it's a meta

25 analysis so it has those same limitations we were

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 130 of 140
130

1 talking about before.

2

THE COURT: Right, I'm not speaking to what it

3 says, it's just that you would consider it authoritative

4 in your field?

5

THE WITNESS: Yes, it's a very good study.

6

THE COURT: All right. Go ahead, Mr. Lichten.

7

MR. LICHTEN: Okay.

8 Q.

Now, if you could go to Table 1. I'll try to pick

9 this up.

10 A.

(Looks.)

11 Q.

You see where -- this is a table and do you see

12 where, um, the authors say that based upon the meta data

13 out there, cognitive ability tests have a one standard

14 deviation -- I'm sorry a 1.00 D, which I think we've

15 already discussed, is that correct?

16 A.

Yes. Yes. I'd have to read it here. But the

17 fact that it's exactly 1.00, they may have specified it

18 to be that in the creation of this matrix.

19 Q.

But this doesn't surprise you, this is what you

20 testified to on direct in answer to the judge, you'd

21 expect, on a cognitive ability test, you're going to see

22 about a 1.00 D between scores for minorities and scores

23 for Caucasians, right?

24 A.

Well, um, these data are kind of old. I'd, you

25 know -- I think as I told you in my deposition, I would

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 131 of 140
131

1 estimate, you know, a .4 to maybe a 1.2. I don't know

2 that the overall average these days is a 1.0, because

3 people are trying to develop cognitive ability tests

4 that are less difficult and so it has made the

5 difference smaller. In fact we often focus on a

6 literacy level to avoid problems. So I don't know that

7 this estimates, you know, the grand average going

8 forward even though it might estimate the older data in

9 our field.

10 Q.

Okay. This also has the D on other types of

11 tests, alternative types of tests, and it says the

12 structured interview has only a .3 D. Do you agree with

13 that?

14 A.

Um, it looks like it says 23.23.

15 Q.

.23. I'm sorry, that's what I said.

16 A.

Oh, okay, I thought you said .3.

17 Q.

.23.

18

I'm just asking if you agree with that?

19 A.

Oh, I'm sorry, yes, I'm reading along with you.

20 Uh-huh.

21 Q.

And you do agree with that?

22 A.

Well, I'm reading along with you, yes.

23 Q.

No, I'm asking you a different question. I'm

24 asking you whether you agree that that is a fair

25 analysis, based upon the meta data, of the smaller D

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 132 of 140
132

1 that occurs with structured interviews?

2 A.

Well, again, it all depends. It's like saying,

3 you know, you've got 1.7 kids, you know, what's a .7

4 kid? You know, an average doesn't necessarily mean it's

5 a good estimate. So the .23 has an interval around it

6 of variation due to differences between studies, and it

7 doesn't show it here in this table but it would be plus

8 or minus that size. I guess the D -- I think the value

9 in parentheses may be an estimate of the interval, but

10 I'd have to read the article again more carefully. I

11 have not read it in a few years.

12 Q.

Dr. Campion, I don't get something. Every time I

13 show you this data in articles you quarrel with the

14 data, the D or the meta analysis that's done for other

15 types of assessment tools, but yet your whole testimony

16 Friday to the Court was that the job knowledge test had

17 this high validity ratio based upon this meta data and

18 that data is okay.

19

So is it your belief that all the data is okay

20 except the one related to job knowledge tests?

21 A.

Well, I know this research very deeply. I was a

22 formal journal editor and one of the most published in

23 my field and I see these data in a lot deeper way. And

24 so I know that these meta analyses cannot be interpreted

25 at face value, that some are based on very large samples

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 133 of 140
133

1 and some are not, and that that needs to be considered.

2 And I'm only trying to bring forth every bit of

3 knowledge I have about the topic, but I try not to be --

4 Q.

Well, I appreciate that, but doesn't that same

5 analysis hold true for job knowledge in the Boston

6 Police Department and how much of a predictor it is,

7 it's speculating just like this other data, isn't it?

8 A.

Well, there's a lot more data on job knowledge

9 than there is on virtually all other selection

10 procedures except for maybe cognitive ability and often

11 what's called "cognitive ability" may be a job knowledge

12 test. So there's much much more data there.

13 Q.

But not on police departments that you know about,

14 right?

15 A.

I will be certain to do a review of the literature

16 when I get out of here today.

17 Q.

Well, I don't know about that, that will be up to

18 the Court.

19 A.

It will be too late to do it anyway.

20 Q.

Okay. Now, if you go to Number 5 -- and this was

21 actually my purpose in putting this up, is you have job

22 performance and that's a .5, okay, do you see that?

23 .45. Excuse me, .45.

24 A.

Yes.

25 Q.

Okay. Now, I assume there's a lot of data on this

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 134 of 140
134

1 because this is measuring actually how you do on the

2 job, right?

3 A.

Yeah, I don't know the sample size used for job

4 performance.

5 Q.

Dr. Campion, you're not serious? Certainly

6 there's as much data on how certain groups do on the job

7 as there is as to how they do on a cognitive ability

8 test, right, I mean job performance measures have been

9 around for a long time?

10 A.

Yes, but they're not published. Just because

11 somebody gathers data on differences in job performance

12 doesn't mean it gets into the literature. But what I

13 meant is I don't know where these data come from. I

14 don't know what the job performance data here derives

15 from because it's normally not used as a selection

16 procedure like these others are. So I don't know --

17 Q.

Well, that's what I'm trying to establish. Let me

18 ask it more directly.

19 This would suggest that minorities do better on the job

20 by two-fold than they do on the cognitive ability tests

21 that they maintained, that there's half the difference

22 --

23 A.

Uh-huh.

24 Q.

-- on a performance, that is between whites and

25 blacks it's half as much as they do on cognitive ability

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 135 of 140
135

1 tests, you would agree with that, wouldn't you?

2 A.

Yes, and I would say that that is always the case

3 in every prediction situation anywhere.

4 Q.

Well, let's just see what that means. That means

5 that you can't -- if you were to be testing people based

6 upon a cognitive ability test, you're going to see a

7 much bigger difference on the performance on the

8 cognitive ability test than you're going to see if you

9 actually judged blacks and whites on how they performed

10 the job, is that correct?

11 A.

Yes, and it's because there's always prediction

12 loss unless the validity is 1.0, and that's true of

13 every science and every phenomenon.

14 Q.

So to apply that in this case, if you only use a

15 multiple choice job knowledge test, you are going to see

16 bigger differences in scores than you would expect to

17 see if you actually looked at how such minorities were

18 doing on the job at the Boston Police Department as

19 lieutenants, is that right?

20 A.

That's what I said, it is a characteristic of

21 every prediction situation ever anywhere.

22

MR. LICHTEN: Your Honor, I still have some more,

23 but do you want me to continue or --

24

THE COURT: Well, look, by the way this is played

25 out with your agreements, already we're beyond the end

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 136 of 140
136

1 of next week -- um, not next week, but January 5th, and

2 as I sit here I accept this, but I had thought we were

3 striving to get these matters resolved. Now it's very

4 clear I can't resolve it, cannot, until I have at least

5 got in the record the redirect of this witness.

6

MR. LICHTEN: Right.

7

THE COURT: So while I'm going to count the time

8 against the total 10 days, it's clear I'm going to take

9 the thing under advisement whenever I finish up on the

10 5th of January.

11

Well, so be it. I do find the testimony adduced

12 both by the plaintiffs and the defense important, and

13 with some wasted motion, um, germane, and I've come to

14 consider this a profoundly important case, really one

15 that evokes the finest of our nation's aspirations in

16 giving everyone equal opportunity and a fair shot. And

17 so I don't care whether you continue or not, it's 5 more

18 minutes, I'm counting the time.

19

Mr. Simon?

20

MR. SIMON: Yes, your Honor, I'm sorry. I had

21 thought that you said that we were going to be done on

22 the 5th, but we have until the 8th is our understanding.

23

THE COURT: What's that? I didn't say you were

24 done. You do.

25

MR. SIMON: Yes.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 137 of 140
137

1

THE COURT: If I misspoke, you have 10 days, and

2 I'm counting out the time.

3

MR. SIMON: Yes.

4

THE COURT: But now, whenever you're done, if he's

5 not available the week of the 5th, I'm not going to see

6 your redirect -- or read it, at least, your directed of

7 this witness. You've agreed to that and I have to say I

8 think I've been very accommodating in terms of time.

9

So now I'm sort of at sea about how we're going to

10 finish with this witness. We'll go on to another

11 witness on the 5th. I didn't say we're going to be done

12 on the 5th, we're going to be done the week of the 5th,

13 when 10 days have elapsed, and I will count whatever

14 time you've taken in redirect against you. So as we get

15 closer -- you seem to have plenty of time, but as we get

16 closer, um, I'll be asking you for an estimate of time,

17 et cetera, and I can take it from the deposition or

18 however you work it out in order to be fair.

19

I am -- unless you people resolve this, I am now

20 eager to get going in resolving it myself. Though I

21 express no opinion. This is an extraordinarily

22 difficult case and we're not done with the evidence.

23

I keep saying that it would be wise to settle it,

24 and I continue to think that, and I will say that after

25 we've had closing arguments. I've told you how I was

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 138 of 140
138

1 going to do this, if after the closing arguments you

2 want me to stay my hand, well, I'll stay it. Now I've

3 got to stay it until I see your redirect.

4

MR. SIMON: Well, your Honor, can we actually have

5 a minute to talk about whether or not we can resolve

6 this right now?

7

THE COURT: Well, why don't you.

8

MR. LICHTEN: Well, your Honor, I'm still

9 confident that we're going to get done by the time you

10 allotted. We're really -- we have like, I think, two

11 short witnesses after this that are going to take like a

12 half hour apiece and we've got Dr. Hough who is going to

13 be much shorter.

14

THE COURT: And who are your short witnesses?

15

MR. LICHTEN: Um, Commissioner Davis, who we've

16 accommodated the defendants on, and a Thomas Nolan.

17

THE COURT: Right.

18

MR. LICHTEN: He's a retired Boston police

19 officer.

20

THE COURT: That's right. So you're done but he's

21 not here the week of the 5th.

22

MR. LICHTEN: So that I think -- we've been

23 discussing this, I will accommodate them in any way, and

24 we may be able to do it by teleconference testimony or

25 something. And I will accommodate them in any way.

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 139 of 140
139

1

THE COURT: We've done that. We've done it.

2

MR. SIMON: Okay.

3

MR. LICHTEN: So I'm confident we can get done by

4 the 8th.

5

MR. SIMON: Well, we obviously need to talk about

6 those kinds of options, but we will do our best to try

7 to work it out.

8

THE COURT: All right. I think we're at 1:00.

9 We'll stop now.

10

As we now stand, the plaintiff has used up 3 days,

11 2 hours, 5 minutes, the defense has used up 2 days, 1

12 hour, 25 minutes. And I most genuinely wish you all a

13 very fine holiday. And we'll resume at 9:00 a.m. on

14 Monday the 5th of January unless of course you have

15 resolved it, in which case a telephone call to

16 Ms. Gaudet is all that's necessary. Have a good

17 holiday. We'll recess.

18

(Adjourned, 1:00 p.m.)

19

20

21

22

23

24

25

Case 1:12-cv-10291-WGY Document 167 Filed 02/19/15 Page 140 of 140
140

1

C E R T I F I C A T E

2

3

4

I, RICHARD H. ROMANOW, OFFICIAL COURT REPORTER,

5 do hereby certify that the foregoing record is a true

6 and accurate transcription of my stenographic notes

7 before Judge William G. Young, on Friday, December 19,

8 2014, to the best of my skill and ability.

9

10

11

12 /s/ Richard H. Romanow 02-19-15 _______________________
13 RICHARD H. ROMANOW Date

14

15

16

17

18

19

20

21

22

23

24

25

 
 Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 1 of 82

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

)

BRUCE SMITH, PAUL JOSEPH, JOHN M. )

JOHNSON, ROBERT TINKER, MARTIN

)

JOSEPH, KIM GADDY, BRIAN KEITH

)

LATSON, LEIGHTON FACEY and MARWAN )

MOSS,

)

)

Plaintiffs, )

)

v.

)

)

CITY OF BOSTON,

)

)

Defendant.

)

)

CIVIL ACTION NO. 12-10291-WGY

YOUNG, D.J.

November 16, 2015

FINDINGS OF FACT, RULINGS OF LAW, AND ORDER

I. INTRODUCTION

In this action, ten black police sergeants (the

“Plaintiffs”) employed by the Boston Police Department (the

“Department”) brought suit against the City of Boston (“Boston”

or the “City”) under Title VII of the Civil Rights Act of 1964,

alleging that the multiple-choice examinations the Department

administered in 2005 and 2008 to select which sergeants to

promote to the rank of lieutenant had a racially disparate

impact on minority candidates and were insufficiently job-

[1]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 2 of 82
related to pass muster under Title VII.1 The Plaintiffs also asserted a pendent claim under Massachusetts General Laws Chapter 151B (“Chapter 151B”).2 The City disputes that the exams had a disparate impact on minority candidates and claims that, even if they did, the exams were sufficiently job-related to survive a Title VII challenge.
This is a profoundly important case, one that evokes the finest of our nation’s aspirations to give everyone equal opportunity and a fair shot. In deciding this case, the Court first emphasizes what this case is not about: this is not a case about conscious racial prejudice. Rather, the Plaintiffs’ case is rooted in their allegation that the seemingly benign multiple-choice examination promotion process, while facially neutral, was slanted in favor of white candidates.
1 Minority police officers include black and Hispanic officers. Ex. 47, Adverse Impact Evaluation: 2008 and 2005 Exams Promotion Lieutenant, BPD (“Adverse Impact Evaluation”) 6. This demographic information is self-reported. 12/15/14 Bench Trial Tr. (“Tr.”) 61:6-20, ECF No. 161. Self-reporting makes sense. This Court struggled with the concept of race in Cotter v. City of Boston, 193 F. Supp. 2d 323, 328 (D. Mass. 2002) aff'd in part, rev'd in part, 323 F.3d 160 (1st Cir. 2003).
2 The legal analysis of disparate impact claims under Massachusetts General Laws Chapter 151B and Title VII is the same. The discussion here will therefore focus on the better developed principles of Title VII jurisprudence, which apply to both claims. See, e.g., White v. Univ. of Mass., 410 Mass. 553, 557 (1991).
[2]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 3 of 82
The parties engaged in a ten-day bench trial and submitted exhaustive post-trial briefs. The long trial involved substantial and dense discussions of statistical analysis. Consequently, the decision that follows is admittedly complex, but its conclusion is simple: the Department’s lieutenantselection process -- ranking candidates for promotion based on their scores on an exam administered in 2008 (“2008 exam”) -had a racially disparate impact and was not sufficiently jobrelated to survive Title VII scrutiny. Accordingly, the Court imposes liability on the City.3 II. PROCEDURAL HISTORY
The Plaintiffs initiated this case in federal court in February 2012. Compl., ECF No. 1. Judge Tauro, to whom this case was originally assigned, dismissed without prejudice the claims of two of the Plaintiffs (John Johnson and Robert Tinker) for their failure to exhaust administrative remedies. Mem., ECF No. 28. Once this case was transferred to this Session on December 26, 2013, Mem., ECF No. 56, this Court denied the Plaintiffs’ motion to reconsider the dismissal, Elec. Order, ECF No. 67, and subsequently denied without prejudice the Plaintiffs’ motion to certify a class, Elec. Clerk’s Notes, ECF
3 The Court does so without deciding whether the Plaintiffs have satisfactorily proven a less discriminatory alternative. See infra note 43 (explaining then rejecting the City’s argument that the Court must decide the issue as a prerequisite to imposing liability on the City).
[3]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 4 of 82
No. 70. Two years of discovery ensued, followed by the virtually inevitable cross-motions for summary judgment. Def.’s Mot. Summ. J., ECF No. 89; Pls.’ Mot. Summ. J., ECF No. 94. This Court denied summary judgment on all claims due to genuine disputes of material fact. Elec. Clerk’s Notes, ECF No. 120. In December 2014, the Court ruled that the remaining eight Plaintiffs had no viable disparate impact liability claim arising from their taking the 2005 lieutenant promotional exam (the “2005 exam”) due to their failure to exhaust administrative remedies. Elec. Order, ECF No. 150. Although no longer formally the subject of this litigation, the Court did consider evidence regarding the 2005 exam for background and context in evaluating the 2008 exam.
At the pre-trial conference, the Court bifurcated the case into separate liability and damages phases. Elec. Clerk’s Notes, ECF No. 98. The liability phase was tried before the Court between December 15, 2014 and January 7, 2015. See 12/15/14 Tr. 3:3-4, ECF No. 161; 01/07/15 Tr. 3:9-11, ECF No. 160. The following witnesses testified for the Plaintiffs: Dr. Joel Wiesen, PhD., industrial organizational psychologist (expert witness), 12/15/14 Tr. 3:6-12, Department Sergeant and Plaintiff Bruce Smith (fact witness), 12/17/14 Tr. 3:11-13, ECF No. 163, former Department Commissioner Edward Davis (fact witness), 01/05/15 Tr. 3:9-11, ECF No. 158, and Leatta M. Hough,
[4]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 5 of 82
PhD, industrial organizational psychologist (expert witness), 01/06/15 Tr. 3:5-7, ECF No. 159. The following witnesses testified for the City: Dr. Jacinto Silva, PhD, industrial organizational psychologist (expert witness), 12/17/14 Tr. 3:1517, Dr. Michael Campion, PhD, industrial organizational psychologist (expert witness), 12/19/14 Tr. 3:5-7, ECF No. 166, Department Chief of the Bureau of Administration and Finance Edward P. Callahan (fact witness), 01/06/15 Tr. 3:9-11, and Department Commissioner William E. Evans (fact witness), 01/07/15 Tr. 3:5-7. III. LEGAL CONTEXT
A. Title VII It is the goal of Title VII “that the workplace be an environment free of discrimination, where race is not a barrier to opportunity.” Ricci v. DeStefano, 557 U.S. 557, 580 (2009). The statute is designed to “‘promote hiring on the basis of job qualifications, rather than on the basis of race or color.’” Id. at 582 (quoting Griggs v. Duke Power Co., 401 U.S. 424, 434 (1971)). Title VII, codified at 42 U.S.C. § 2000e, provides two theories of liability for discrimination in the employment context: disparate treatment and disparate impact. Ricci, 557 U.S. at 577-78. A disparate treatment claim accuses an employer of intentionally basing employment decisions on an improper
[5]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 6 of 82

classification, such as race. See id. at 577. A disparate

impact claim, by contrast, challenges an employment decision

that is facially neutral, but which falls more harshly on those

in a protected class. See id. at 577-78.

This is a disparate impact case. Second Am. Compl.,

Compensatory, Injunctive & Declaratory Relief Requested (the

“Complaint”) ¶ 1, ECF No. 14. Section 2000e-2(k)(1)(A) outlines

the burden of proof in a disparate impact case:

An unlawful employment practice based on disparate impact is established under this subchapter only if —

(i)

a complaining party demonstrates that a

respondent uses a particular employment

practice that causes a disparate impact on the

basis of race, color, religion, sex, or

national origin and the respondent fails to

demonstrate that the challenged practice is job

related for the position in question and

consistent with business necessity; or

(ii)

the complaining party makes the demonstration described in subparagraph (C) with respect to an alternative employment practice and the respondent refuses to adopt such alternative employment practice.

42 U.S.C. § 2000e-2(k)(1)(A).

Under First Circuit case law, the plaintiff bears the

burden of establishing a prima facie case of discrimination

which consists of identification of an employment practice (in

this case, the 2008 exam and promotions flowing therefrom),4

4 Identifying an employment practice can be a tricky proposition in the context of Title VII. As is typical in cases
[6]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 7 of 82
disparate impact, and causation. Bradley v. City of Lynn, 443 F. Supp. 2d 145, 156 (D. Mass. 2006) (Saris, J.) (quoting EEOC v. Steamship Clerks Union, Local 1066, 48 F.3d 594, 601-02 (1st Cir. 1995)).
If the Plaintiff meets this burden, the employer may either debunk the Plaintiff’s prima facie case, or alternatively, may demonstrate that the challenged practice is “job-related and consistent with business necessity.” Bradley, 443 F. Supp. 2d at 157; see also Ricci, 557 U.S. at 578. If the employer demonstrates the latter, the ball bounces back into the plaintiff’s court to demonstrate that “some other practice, without a similarly undesirable side effect, was available and would have served the defendant’s legitimate interest equally well.” Bradley, 443 F. Supp. 2d at 157.
The law of disparate impact has become a powerful tool for ensuring equal opportunity. It is both balanced and nuanced. Each step in its three-part doctrine serves a valuable function. Consider the following.
involving a battle of statistics, the parties argue that the statistic more favorable to them is the appropriate one. The Plaintiffs here argue that the Court may and should consider promotion rates, pass-fail rates, average scores, and delays in promotion stemming from the 2008 exam. The City argues that promotion rates are the only pertinent statistic. As is explained below, the Court sides with the Plaintiffs on this disagreement. See infra Part V-A-1.
[7]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 8 of 82
All testing, hiring, and promotion schemes are necessarily discriminatory. These programs exist because there are more applicants than there are jobs. Under the first prong, the Plaintiffs must make a significant showing of actual disparate impact upon an identified protected minority. This is as it should be: no one wants federal courts acting as super personnel agencies.
If the plaintiff can, however, make this showing, then under the second prong, the employer gets a chance to demonstrate that the test in question is both job-related and consistent with business necessity. Again, this step is sensible: courts ought not defer excessively to employers, but neither should they ignore the realities of particular jobs. The current debate over the exhaustive testing to determine the capabilities of women to engage in military ground combat comes immediately to mind as exemplifying the difficult issues encountered in prong two.
Even if the employer succeeds, however, the case is not over. Under the third prong, the plaintiff gets one more shot. If the plaintiff can demonstrate the availability of a testing program equally determinative of job performance, yet resulting in less disparate impact, the Court should fashion a remedy to secure the greatest degree of equal opportunity. In other words, to produce more equality of opportunity, Title VII
[8]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 9 of 82
empowers courts to impose on employers an equally effective means of evaluating applicants.
B. The Commonwealth’s Statutory and Administrative Framework
Under Massachusetts law, police sergeants seeking to be promoted to lieutenant are subject to the state civil service statutory promotion regime. See Mass. Gen. Laws ch. 31, § 51. The purpose of the examination regime is to “guard against political considerations, favoritism, and bias in governmental employment decisions . . . and to protect efficient public employees from political control.” Cambridge v. Civil Serv. Comm’n, 43 Mass. App. Ct. 300, 304 (1997). Under this regime, to become a lieutenant, Boston police sergeants must first pass a competitive civil service examination. Mass. Gen. Laws ch. 31, § 59.
The Commonwealth of Massachusetts Personnel Administrator of the Human Resources Division (“HRD”), Mass. Gen. Laws ch. 31, § 1, is responsible for “conduct[ing] examinations for purposes of establishing eligible lists” for promotion. Id. § 5(e). HRD is obligated by statute to “fairly test the knowledge, skills and abilities which can be practically and reliably measured and which are actually required” to perform the job. Id. § 16. To achieve this end, HRD develops the examination, id. §§ 5(e), 16, posts notices of the exams, id. §§ 18-19, and determines the
[9]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 10 of 82
passing requirements, id. § 22. Promotional examinations within the Department are typically administered every two or three years. 12/15/14 Tr. 64:4-10, ECF No. 161. Unlike other jurisdictions, Massachusetts requires candidates for each supervisory rank (sergeant, lieutenant, and captain) to take a test at each promotional level, even if there is substantial overlap in the questions that appear on the tests for each rank. Id. at 129:19-130:5.
The Department has the option of either using the exams developed by HRD, or seeking a delegation agreement with HRD by which HRD agrees to oversee the Department creating its own exam. Mass. Gen. Laws ch. 31, §§ 5(l), 59, 65; 01/06/15 Tr. at 91-93. Under this delegation regime, municipalities must still comply with civil service law and regulations, but may decide themselves how to satisfy these requirements. Lopez v. Massachusetts, 588 F.3d 69, 76 (1st Cir. 2009). When the City enters into such a delegation agreement, it must bear the costs of developing and administering the tests. 12/17/14 Tr. 49:2124. The City does not incur these costs when it uses an HRD test. 01/06/15 Tr. 112:16-21.
In a competitive examination, “an applicant shall be given credit for employment or experience in the position for which the examination is held.” Mass. Gen. Laws ch. 31, § 22. Such credit is known as an education and experience score (“E&E”),
[10]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 11 of 82
and is calculated using biographical information provided by the applicants. Id.; Ex. 85, Affidavit Edward P. Callahan (“Callahan Aff.”), ECF No. 177; Ex. 3, Education Experience Sheet Instructions (“E&E Instructions”) 1, ECF No. 177-3. Pursuant to Massachusetts statute, veterans and long-service employees receive preference points. Mass. Gen. Laws ch. 31, §§ 26, 59.
After combining the exam scores with the E&E scores, HRD issues an eligibility list for specific positions ranked in order of an applicant’s total score. Id. §§ 25, 27.5 An eligibility list remains in effect until replaced by a new eligibility list from a subsequent exam. Id. § 25; Callanan v. Pers. Adm’r for the Commonwealth, 400 Mass. 597, 601-02 (1987). The City has promoted police officers based on promotional examination results in strict rank order since at least 1977. 12/15/14 Tr. 63-64.
To hire for a vacancy, the Department submits a request to HRD, which certifies from the larger eligibility list a smaller list of names of persons for consideration in rank order. Mass. Gen. Laws ch. 31, § 6. Under HRD’s Personnel Administration
5 Section 26 seems to mandate that other categories of qualified applicants (meaning those who “pass[ed the relevant] examinations”), including disabled veterans, veterans, and widows or widowed mothers of veterans who were killed in action, be placed above even the highest scorers on the test. Mass. Gen. Laws ch. 31, § 26.
[11]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 12 of 82
Rules, the number of candidates appearing on the smaller list is determined by the formula 2n+1, with n representing the number of vacancies. 12/15/14 Tr. 63:5-16. For example, if there are two job vacancies corresponding to one applicable list, the list would be comprised of the candidates with the five highest scores (2x2+1=5).
The Department must then make selections from that list based on strict rank order, based on the candidates’ performance on the promotional exam. 01/06/15 Tr. 102:12-18. If the candidates have tied scores, the Commissioner may consider factors such as past work history and diversity. 01/07/15 Tr. 9:25-11:19. The statutory framework allows a municipal employer to “bypass” a candidate on the list -- that is, to step out of strict rank order -- but the employer must have a defensible reason for the bypass, such as a history of disciplinary infractions. Mass. Gen. Laws ch. 31, § 27; City of Cambridge v. Civil Service Comm’n, 43 Mass. App. Ct. 300, 305 (1997). Former Commissioner Davis testified that as a practical matter, bypassing is difficult. 01/05/15 Tr. 100-101.
Dissatisfied candidates may challenge the results of the examination with the HRD by claiming that the examination was not a “fair test of the applicant’s fitness actually to perform the primary or dominant duties of the position for which the examination was held.” Mass. Gen. Laws ch. 31, § 22. The
[12]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 13 of 82
Massachusetts Civil Service Commission oversees the administrative appeals process for candidates to air their grievances with the hiring process and the initial review by the HRD. Id. § 24. Judicial review is available after the candidate has exhausted his or her administrative remedies. Id. § 44.6 IV. FINDINGS OF FACT
This is not the first case in which Department employees or potential employees have challenged the Department’s hiring or promotional procedures. In fact, a case raising similar issues to this one was brought before Judge O’Toole in 2007. Judge O’Toole’s findings of fact and conclusions of law issued in September 2014. Findings Fact, Conclusions Law, Order J., Lopez v. City of Lawrence (“Lopez”), ECF No. 347 (D. Mass. Sept. 5, 2014). An appeal is pending. Lopez, appeal docketed, No. 1401952 (1st Cir. Sept. 17, 2014). As there is some factual overlap, on the first trial day in this case, without objection, this Court admitted in evidence all the trial testimony and exhibits from Lopez. 12/15/14 Tr. 32:4-16.7
6 The Plaintiffs remaining in this case have done so. 7 In Lopez, various minority police officers across the Commonwealth who were vying for a promotion to become sergeants challenged the promotional exams under state and federal law. This Court’s conclusions differ from the Lopez decision in one crucial respect. See infra note 42 and accompanying text.
[13]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 14 of 82
More broadly, over the past few decades, candidates who are members of a racial minority and candidates who are not have challenged the hiring and promotional procedures employed by the Department as unlawfully discriminatory. The extensive litigation history is well documented elsewhere, see Lopez at 12-14, and the Court will not repeat it here. The Court mentions it only for the purpose of noting that, similarly to police departments in other jurisdictions, this pendulum of litigation has pressured the City constantly to re-tool its examination procedures in an effort to appease job applicants and courts alike. See Barnhill v. City of Chicago, Police Dep't, 142 F. Supp. 2d 948, 949 (N.D. Ill. 2001).
A. The Role of a Boston Police Department Lieutenant Boston Police Department lieutenants are second-line supervisors, meaning that they supervise sergeants, who themselves supervise police officers out in the field. 01/05/15 Tr. 102:1-10. Lieutenants are also in charge of station houses and are responsible for the proper arrest of suspects and for the safety of prisoners. Id. at 128:3-9, 131:4-8. The job involves a significant amount of desk work, 12/17/14 Tr. 104:912, as well as work outside of the station, including talking with citizens at community meetings, id. at 105:14-25, and taking control of scenes of major incidents, 01/05/15 Tr. 126:23-127:12. The job requires good management skills,
[14]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 15 of 82
including the ability to motivate employees, and to communicate information between ranks. Id. at 102:11-23.
The official Department job description for lieutenant has not changed since 1979, and current Boston Police Department Commissioner William Evans testified that it remains accurate today. Ex. 23, Boston Police Department Rules Procedures, Rule 105; 01/07/15 Tr. 16:15-17:9. Just prior to 2006, however, Boston began to shift its policing philosophy towards that of community policing, where police officers engage more directly with the community they serve. 01/05/15 Tr. 109:16-110:15. The skills required for a Boston Police Department lieutenant evolved with this shift, differing from what was needed in the early 1990s when the prevailing policing philosophy involved responding to, rather than preventing, emergencies. Id. at 110:9-12.
B. Job Analyses and Validation Studies Pre-Dating 2005 The first step in developing a valid civil service promotional exam is to create a job analysis, which identifies “important work behavior(s) required for successful performance and their relative importance.” 29 C.F.R. § 1607.4(C)(2) (“Uniform Guidelines”).8
8 Chapter 29 of the Code of Federal Regulations, section 1607, was published under the name of Uniform Guidelines on Employee Selection Procedures in 1978 by several government agencies to interpret how selection and testing and assessment
[15]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 16 of 82
The development of the 2005 and 2008 exams began in 1991 with the creation of a job analysis and validity report, which HRD incorporated into the 2008 exam. 12/19/14 Tr. 14:5-15. HRD also incorporated into the 2008 exam a job analysis from 2000, which was in part based on the 1991 job analysis.9 Id. at 14:515:25, 32:13-16. The documents are somewhat dense, and the Court will discuss each in turn.
1. The 1991 Job Analysis In 1991, the Massachusetts Department of Personnel Administration (“DPA”), the predecessor to HRD, prepared a state-wide validation report for the ranks of sergeant, lieutenant, and captain.10 Ex. 71, Validation Report 1991 Police Promotional Selection Procedure (“1991 Validation Report”) at 00247. DPA relied on the Uniform Guidelines, as well as the Society for Industrial and Organizational Psychology Principles for the Validation and Use of Personnel Selection Procedures (“SIOP Principles”). Id. at 00247, 00250.
should be conducted in accordance with the Civil Rights Act of 1964. 12/19/14 Tr. 23:6-18.
9 The parties disagree as to whether the exams were based on both job analyses, or on the 2000 job analysis alone. The 2000 report explicitly states that it reviewed “past job analyses” to prepare the 2000 report. Ex. 39, Job Analysis Report Police Lieutenant City Boston 9. The Court regards both job analyses as relevant to this case.
10 In the Department, captains supervise lieutenants, and lieutenants supervise sergeants. Ex. 46, Boston Police Organizational Structure 2.
[16]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 17 of 82
DPA began the job analysis by gathering information about the positions of police sergeant, lieutenant, and captain. It did so by surveying other jurisdictions and reviewing various documents including job analysis studies, articles, and the Uniform Guidelines. 1991 Validation Report at 00253-54. Based on this research, DPA created a list of 136 potentially critical tasks that sergeants, lieutenants, and captains perform. Id. at 00256. DPA sent surveys to municipal police departments across the Commonwealth asking incumbents of the positions to identify which tasks were critical to successful job performance and then rate them accordingly. Id. Police officers were asked to provide information regarding how often they performed the tasks, and to identify the fifteen tasks most critical to their jobs. 1991 Validation Report, App. H at 3638.
DPA then used these task ratings to create a list of 187 Knowledge, Skills, and Abilities (“KSAs”) necessary to carry out the critical tasks. 1991 Validation Report at 00257-58. In a survey administered to subject matter experts (“SMEs”) -superior officers serving in Massachusetts police departments -DPA asked the SMEs to evaluate two things: the importance of the KSAs, and whether the candidate needed the KSA at the time of appointment to the position or could acquire the KSA on the job. Id. Only KSAs that were determined to be important and
[17]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 18 of 82
necessary upon starting the job were considered for inclusion in the test for applicants. Id. at 00258.
In addition to these various surveys, DPA developed additional KSAs by holding “critical incident” technique discussions with the SMEs. Id. at 00252. These discussions involved SMEs providing narrative explanations or anecdotes about the tested positions. Id. The purpose was to allow DPA to design situational examination questions evaluating supervisory abilities. Id.
The next step was to link the KSAs with the critical tasks. Nine SMEs were tasked with this assignment. Id. at 00258. The SMEs identified, by group consensus, which KSAs had a direct relationship to identified clusters of critical tasks. Id. at 00259.11 Of the 187 KSAs, 60 were ultimately incorporated in the written test. 1991 Validation Report, App. EE.
The 1991 report acknowledged that some of the skills identified as important by SMEs could not be evaluated by a written test, such as the “ability to establish rapport with persons from different ethnic, cultural, and/or economic backgrounds.” Id. at 00265. The 1991 report also noted that “assessment of the performance of these skills and abilities would require the use of selection devices outside the scope of
11 DPA also conducted structured discussions with SMEs to discuss the E&E component. Id. at 00260.
[18]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 19 of 82
the written, multiple choice format.” Id. at 00265. Nevertheless, DPA decided to proceed exclusively with a written examination, offering eighty questions common to all three positions; an additional twenty questions for lieutenants and captains testing knowledge of police supervision, administration, and management; and another twenty-five questions to test captains on their knowledge of police administration. Id. at 00266-67. For the 1991 exam, the written portion accounted for 80% of an applicant’s final score, and the E&E portion for 20%. Id. at 00263.
2. 2000 Job Analysis and the Corresponding 2002 Exam The 2000 job analysis (the “2000 report”) was prepared at the request of the City of Boston by Morris & McDaniel, Inc., a consulting firm that specializes in the development of promotional systems. Ex. 39, Job Analysis Report Police Lieutenant City Boston (“2000 Job Analysis Report”). The 2000 report at issue in this case concerned Boston Police Department lieutenants only. Id. Morris & McDaniel came up with a list of 302 possibly relevant tasks that Boston police lieutenants perform, as well as KSAs necessary to carry out those tasks. Id. at 65; id., App. A, Task Inventory Police Lieutenant Boston Police Department. Morris & McDaniel then had twelve SMEs, consisting of Department employees holding the rank of Lieutenant or higher,
[19]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 20 of 82
rate the tasks for frequency, importance, necessity of performing the task upon starting the job, and how correlative successful performance of the task was to successful job performance. 2000 Job Analysis Report at 10-14. If ten of the SMEs rated a task as “very important” or “important,” necessary upon entry to the job, and agreed that performance of that task clearly separated the best workers or better workers from inferior workers, then it satisfied Morris & McDaniel’s test criteria. Id. at 14. Of the initial 302 tasks, 281 fulfilled the criteria. Id. Morris & McDaniel also asked the SMEs to determine which of the following dimensions were required for each task: oral communication, interpersonal skills, problem identification and analysis, judgment, and planning and organizing. Id. at 15. Morris & McDaniel then composed a list of 149 KSAs potentially necessary to perform the 281 tasks. See id. at 48-49. Next, the SMEs were asked whether the KSAs related to the job of police lieutanant, when the KSA was learned (before or after assignment to the job), how long it took to learn the KSA, how the KSA differentiated performance, and whether the KSA was required to perform the job effectively. Id. For a KSA to be important enough to be tested, nine of the twelve SMEs must have rated the KSA as related to the job, learned before assignment to the job, requiring more training than a brief orientation period, capable of distinguishing
[20]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 21 of 82
performance to a high or moderate degree, and required or desirable to perform the job effectively. Id. at 49. Of the 149 KSAs rated by the SMEs, 145 were deemed sufficiently important to be tested. Id.
Based on its 2000 job analysis, Morris & McDaniel created a promotional test for the Department, to be administered in 2002. Ex. 80, Draft Police Lieutenant Written Examination Validity Report (“Validity Report”) 1.12 The 2002 exam consisted of three components: a written exam, an assessment center, and E&E. Id. at 2.13 Of a possible 100 points, Morris & McDaniel, after consulting with SMEs, assigned 30% to the written examination, 50% to the assessment center, and 20% to E&E. Id.; Ex. 81, Police Lieutenant Assessment Center Validity Report (“Assessment Center Report”) 17; 12/22/14 Tr. 46-47, ECF No. 167; 01/06/15 Tr. 110:7-13.
12 A draft of the validity report for the written examination was admitted in evidence because counsel for the Plaintiffs indicated that they could not locate a final copy. 12/22/14 Tr. 39:14-40:8. Neither party objected to its admission or argued that the final report contained conclusions different from the draft.
13 As part of the 2002 exam, the Department attempted to introduce a performance review. In response, the police officers’ union brought a complaint to the Civil Service Commission. 01/06/15 Tr. 88:17-21. The Department ultimately excluded the performance review system from the 2002 exam. Id. at 88:23-25.
[21]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 22 of 82
Morris & McDaniel composed the written test questions, which SMEs reviewed for accuracy and clarity. Validity Report 11-12. Based on the responses, Morris & McDaniel determined that the questions were internally consistent and reliable. Id. at 13.
The assessment center was designed to test oral communication skills, interpersonal skills, ability to quickly identify a problem and analyze it, ability to make sound decisions promptly, and ability to break work down into subtasks and prioritize them.14 Assessment Center Report 5-6. The assessment center consisted of an in-basket exercise (a simulated written exercise), and a situational exercise in which candidates were videotaped offering verbal responses to hypothetical scenarios that a lieutenant may encounter. Id. at 7-8. The assessment center exercises were evaluated by outside assessors. 01/06/15 Tr. 110:14-23.
After the 2002 exam had been administered, Morris & McDaniel prepared validity reports of the written examination and the assessment center. See Validity Report; Assessment Center Report. The validating report was written to comply with the Uniform Guidelines. Validity Report 1. Morris & McDaniel
14 Testimony in the Lopez case indicated that the use of the assessment center was in part the result of an effort by the Department to increase diversity among its promotional ranks. Lopez 07/27/10 Trial Tr. 100.
[22]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 23 of 82
concluded that both portions of the examination were valid. Id. at 18.
This process cost $1,300,000, which included consulting fees and travel costs for outside assessors. 01/06/15 Tr. 112113. HRD certified a list for promotion, from which one black sergeant was selected for promotion. Ex. 38; 01/06/15 Tr. 112:2-4.
C. Development and Administration of the 2008 Exam Based in part on financial constraints and a perceived lack of improvement in diversity resulting from the 2002 exam, the Department elected to use HRD exams (conventional, written exams) in 2005 and 2008.15 Lopez 07/28/10 Tr. 17, 30; 01/06/15 Tr. 93:9-12, 118-19.16 HRD consulted with the outside firm EB Jacobs for the 2008 exam. 12/16/15 Tr. 55:17-56:3. HRD did not create a comprehensive job analysis for the 2008 exam, but instead conducted “mini job analyses” for all three ranks, which were more or less updates or overlays to the 2000 report. HRD asked SMEs to rate tasks and KSAs. Ex. 55;
15 The development and administration of the 2005 and 2008 exams were substantially similar. In an effort to avoid redundancy, the Court focuses on the 2008 exam because it forms the crux of this dispute.
16 Then-police commissioner Kathleen O’Toole wanted to include an assessment center component in the 2005 exam, a request that was ultimately rejected because of funding concerns. 01/06/15 Tr. 113-114.
[23]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 24 of 82
Ex. 56; Tr. 01/07/15 at 20-36. The KSAs were pulled from the 2000 job report. 12/19/14 Tr. 38:9-19.
Commissioner Evans, who was a fact witness in this case, served as an SME for the 2008 exam. 01/06/15 Tr. 126:6-126:10.17 He testified that the SME review of the KSAs and tasks was “meticulous,” 01/07/15 Tr. 30:25-31:4, and that the purpose of the 2008 exam was not to test memorization of facts, but to test situational judgment, interpersonal relations, communication ability, and knowledge of rules and regulations. 01/07/15 Tr. 35-36.
Based on the mini job analyses, the Department’s consultant, EB Jacobs, created a test outline for the 2008 exam. Ex. 54; Ex. 60.18 The Civil Service then compiled a list of 100 questions for the exam. 01/07/15 Tr. 31:5-10. The SMEs reviewed the test questions for suitability for the different ranks, difficulty, and readability, and then indicated whether they recommended the question for the exam. Ex. 60; 01/07/15 Tr. 31:11-17.
HRD announced the 2008 exam and provided a corresponding reading list to members of the Department. Exs. 1, 5, 6, 8, 17.
17 The other SMEs for the 2008 exam were Captain Purvis Ryan, Captain Mark Hayes, and Captain Genevieve King. 01/06/15 Tr. 125:6-126:10.
18 Trial exhibits without clearly identifiable titles are referred to simply by their numbering.
[24]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 25 of 82
The Department provided “substantial amount[s] of tutorial information” for all candidates taking the promotional exams, including taped lectures and practice questions. 01/06/15 Tr. 128. The questions that appear on the written portion of the exam are taken directly from the reading list. 12/15/14 Tr. 51:19-52:12.19
The 2008 exam consisted of two elements: a written, closedbook exam consisting of 100 multiple-choice questions, and an E&E rating. 12/15/14 Tr. 49:7-14. Out of 100 possible points on the written examination, a candidate needed to score 70 to pass. Id. at 62:21-23. The E&E Score is calculated only for candidates who passed the written exam. Id. at 62:24-63:2. The written portion accounted for 80% of the final score; the E&E component for 20%. Id. at 50:9-50:15.
D. Development and Administration of the 2014 Exam The Department typically offers promotional exams every two or three years. Id. at 64:4-7. The 2008 exam did not follow this trend: Commissioner Davis requested that the promotional certifications be extended due to the Lopez litigation. Id. at 64:11-18; Ex. 59; Ex. 61. Thus, it was not until 2014 that the Department developed and administered a new promotional exam;
19 The topics covered by the reading list for the 2008 exam included police supervision, management, administration, and other aspects of Massachusetts law. 12/17/14 Tr. 134-36.
[25]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 26 of 82
the list from the 2008 exam was still in effect at the time of this trial. 01/06/14 Tr. 143:25-144:3.
The Department, in part due to an improved economic climate, and in part due to a desire to increase diversity, elected to go beyond a written examination and E&E for the 2014 promotional process. 01/05/2015 Tr. 139-140:8. The Department again retained the firm of EB Jacobs, this time to design and administer the 2014 exam. Callahan Aff. ¶ 1. At the firm’s recommendation, the Department in 2013 approved the development of a “very comprehensive” job analysis in anticipation of the 2014 exam (“2013 job analysis”). 01/06/15 Tr. 109:11-19. This was the first full job analysis performed since 2000. Id. at 131:5-19. Based on the 2013 job analysis, EB Jacobs recommended the use of an assessment center in addition to a written examination. Id. at 133:12-23. After securing funding from Boston for an assessment center, the Department secured a delegation from HRD to develop its own promotional exam. Id. at 134:5-17.
The Department posted an announcement of the exam, and indicated that the exam would be weighted as follows: technical knowledge written exam (36%); in-basket test (where candidates provide written essay-style responses to various job situations) (20%); oral board test (where candidates provide oral responses to hypothetical incidents and personnel issues) (24%); and E&E
[26]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 27 of 82
(20%). Callahan Aff. ¶¶ 4, 7; id., Ex. 2, Lieutenant Promotion Examination Candidate Preparation Guide In-Basket Oral Board Tests 3, 5, ECF No. 177-2.
Unlike the 2005 and 2008 exams, there was no cut-off score for the written portion of the 2014 exam. 12/17/14 Tr. 111:25112:2. Much else was the same, however. Outside assessors evaluated each candidate’s performance in the assessment centers, id. at 114:22-115:4, the E&E component was based on self-reporting of candidates’ education, training and work experience, E&E Instructions 1, and veterans received an additional two points, Callahan Aff. ¶ 9. The full development of the promotional exam process (testing for promotion of sergeant, lieutenant, and captain) cost over $1,600,000. Id. ¶ 12.
E. Results of the 2005 and 2008 exams While all of this history is informative and helpful to gain an understanding of the Department’s promotional process, it is the results of the 2005 and 2008 exams that form the crux of this dispute. By and large, the parties agree on all of the numbers in this section. The crux of the dispute is which of these numbers are important, and which methodology is the most appropriate for analyzing these numbers. One hundred and twenty seven candidates reported for the 2005 lieutenant promotional exam: 104 were white, 22 black, and
[27]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 28 of 82
1 Hispanic.20 Ex. 47, Adverse Impact Evaluation: 2008 and 2005 Exams Promotion Lieutenant, BPD (“Wiesen Report”) 8; Ex. 72, Report Jacinto M. Silva (“Silva Report”) 2. The passing rate of the written exam for minorities was 50%, and for whites, 88%. Wiesen Report 12. The mean score for minorities on the 2005 exam was 69.9, and for whites, 78.7. Id. at 16. Of the 127 sergeants who sat for the exam, 27 were promoted: 25 were white (out of 104 white applicants), one was black (out of 22), and one was Hispanic (who was the only Hispanic candidate). Id. at 8; Silva Report 2.
Ninety-one sergeants sat for the 2008 promotional exam: 65 were white, 25 were black, and one was Hispanic. Wiesen Report 7; Silva Report 7. Of the 91 candidates who took the exam, the passing rate for minorities was 69%, and for whites was 94%. Wiesen Report 11. The mean score for minorities was 76.6, and for whites was 83.2. Id. at 14. Of the 91 candidates, 33 were promoted: 28 were white (out of 65), and 5 were black (out of 25). Silva Report 7.
20 The expert reports on disparate analysis (Wiesen Report, Silva Report) disagree as to whether the number of people who took the 2005 promotional exam was 126 or 127. The parties agree that the Court should adopt 127, the number more favorable to the Plaintiffs, and also agree that the difference will have only a minor impact. Proposed Findings Fact City Boston (“PreTrial City Proposed Findings”) ¶ 25, ECF No. 141; Silva Report 2.
[28]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 29 of 82
After reviewing the final scores from the 2008 exam, the Department’s consultant, EB Jacobs, recommended “banding” the results in nine-point increments (meaning that scores within nine-point ranges would be deemed equivalent). Lopez Exs. 70, 71.
The background and raw numbers for this case are straightforward and self-explanatory. Their statistical analysis and corresponding legal conclusions, less so. V. CONCLUSIONS OF LAW
A. Disparate Impact (Prong 1) The use of the 2008 exam is the employment practice subject to challenge under Title VII. The parties agree that minority test-takers passed the 2008 exam and were promoted to lieutenant at a lower rate when compared to white candidates. But a lower rate of passage or promotion is not, by itself, sufficient to establish a prima facie case of discrimination. Statistical disparities must be significant enough to “raise an inference of causation.” Bradley, 443 F. Supp. 2d at 157 (citation omitted). The Supreme Court has stated: [T]he plaintiff must offer statistical evidence of a kind and degree sufficient to show that the practice in question has caused the exclusion of applicants for jobs or promotions because of their membership in a protected group. Our formulations, which have never been framed in terms of any rigid mathematical formula, have consistently stressed that statistical disparities must be sufficiently substantial that they raise such an inference of causation.
[29]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 30 of 82
Watson v. Fort Worth Bank & Trust, 487 U.S. 977, 994-95 (1988) (plurality opinion); see also Texas Dep't of Hous. & Cmty. Affairs v. Inclusive Communities Project, Inc., 135 S. Ct. 2507, 2523 (2015) (noting that “[a] robust causality requirement . . . protects defendants from being held liable for racial disparities they did not create”). In other words, the Plaintiffs must show that any disparity between races is not the result of mere chance. See Jones v. City of Boston, 752 F.3d 38, 43 (1st Cir. 2014).
There is no “single test” to demonstrate disparate impact. Langlois v. Abington Hous. Auth., 207 F.3d 43, 50 (1st Cir. 2000). Plaintiffs in Title VII disparate impact cases often demonstrate causation by presenting evidence that the disparity in outcomes between white and minority candidates is “statistically significant,” meaning that statistical analysis reflects that the odds of the disparity occurring by mere coincidence are less than 5%. Jones, 752 F.3d at 43-44. This is demonstrated when a statistician determines that the “pvalue,” which stands for “probability,” is less than .05, meaning that the probability of the result occurring by chance is less than 5%. Jones, 752 F.3d at 46-47; 12/15/14 Tr. 74:1975:7; Wiesen Report 5.
Another way to express this same mathematical calculation is to utilize the statistical measure of “standard deviation,”
[30]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 31 of 82
which measures how dispersed a set of data is (the more dispersed the data, the higher the standard deviation). See 12/15/14 Tr. 75:19-25. One “can calculate the standard deviation . . . for any [p-value].” Id. at 77:1-9. For a socalled one-tailed test,21 the relevant standard deviation is 1.645. 12/15/14 Tr. 77:1-9. For a two-tailed test, it is 1.96. Id. In other words, if one utilized a two-tailed test and found that the mean test scores of minority candidates were located 1.96 standard deviations away from the overall mean, there would be only a 5% probability that such difference was due to chance. (And if their mean score was more than 1.96 standard deviations from the mean, the probability that it was due to chance would be even lower.)
Parties alleging disparate impact also sometimes rely on what is known as the “four-fifths rule,” articulated in the Uniform Guidelines, a non-binding set of guidelines authored by the Equal Employment Opportunity Commission to help employers comply with Title VII. The thrust of the “four-fifths rule” is that a selection rate for any racial group that is less than four-fifths (or 80%) of the rate of the group with the highest rate is evidence of adverse impact. 29 C.F.R. § 1607.4(D). The
21 The Court discusses below at some length the distinction between one-tailed tests and two-tailed tests. See infra Part V-A-3.
[31]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 32 of 82
four-fifths rule is “widely used” among organizational psychologists. 12/15/14 Tr. 72:25-73:6. The First Circuit has acknowledged, however, that it is not decisive. Jones, 752 F.3d at 51.
Reports and testimony from experts play a crucial role in evaluating a disparate impact claim. The Plaintiffs’ expert witness regarding disparate impact was Dr. Joel Peter Wiesen, an industrial organizational psychologist. 12/15/14 Tr. 34:3. Dr. Wiesen holds his PhD in psychology. Id. at 34:11. He worked for HRD from 1977-1992, during which time he was the chief of test development and validation. Id. at 36:13-25. Since leaving HRD, Dr. Wiesen has been consulting in the area of test development, and has served as an expert witness. Id. at 38:1016.22 The City rebutted Dr. Wiesen’s testimony with that of Dr. Jacinto M. Silva, who holds a PhD in industrial organizational psychology. 12/17/14 Tr. 141:19-142:11; Silva Report 1. Dr. Silva is currently a senior managing consultant at EB Jacobs, which is the firm that developed the 2014 lieutenants’ exam for the Department and consulted on the 2008 exam. Id. at 143:25144:11.
Drs. Wiesen and Silva agreed on many issues: the raw data underlying each other’s analysis (although there were some minor
22 The Plaintiffs also offered Dr. Wiesen’s testimony on the issue of test validation. 12/15/14 Tr. at 42:17-22.
[32]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 33 of 82
discrepancies based on the timing of their reports); that each other’s mathematical calculations were correct; and that the Fisher Exact Test was the appropriate test for this analysis. Def. City Boston’s Post-Trial Proposed Findings Fact & Conclusions Law (“Post-Trial City’s Proposed Findings”) 38 n.21, ECF No. 190. The experts disagreed as to which statistics were relevant (among promotion rates, mean scores, pass-fail rates, or delay in promotion); whether the results from the 2005 and 2008 tests should be aggregated; and whether a one-tailed or a two-tailed Fisher Exact Test was the appropriate methodology. The Court will address these issues in turn.
1. The Relevant Data Points Before beginning any analysis, statistical or otherwise, the Court must determine the proper scope of its inquiry. The Plaintiffs argue the Court should cast a wide net, looking to several statistics regarding the 2008 exam. The City, however, suggests that one number, promotion rates, provides all the necessary information. As explained more fully below, the Court largely agrees with the Plaintiffs. Dr. Wiesen examined various aspects of the lieutenant promotional procedure employed by the City in an effort to determine whether there was disparate impact. Specifically, he compared the numbers between minority and non-minority candidates for: (1) promotion rates; (2) passing rates, (3)
[33]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 34 of 82
average exam scores; and (4) delays in promotion. 12/15/14 Tr. 81:23-82:11; Wiesen Report 3.
Dr. Wiesen, acknowledging that the first measurement, promotion rates, is the “most important[],” Wiesen Report 5, nevertheless argued that the other measurements were also important for various reasons. He argued that the passing rates and average scores were relevant because of the current system in which candidates are promoted in strict rank order. Id. He posited that passing rates and average scores were more “sensitive” or “statistical[ly] power[ful]” than promotion rates. Id. at 5-6. Dr. Wiesen opined that delays in promotion in the Department -- meaning the time between becoming eligible for a promotion and actually receiving that promotion -- were important because the timing of promotion from sergeant to lieutenant affects how job tasks are assigned. Id. at 6. Dr. Wiesen ultimately concluded that the 2005 and 2008 exams “had fairly severe adverse impact on minority candidates, black and Hispanic.” 12/15/14 Tr. 48:21-24.
In contrast to the Plaintiffs’ consideration of four sets of data, Dr. Silva and the City argued that promotion rates were the only appropriate measurement for determining adverse impact. Post-Trial City’s Proposed Findings 44-45; see Silva Report 13. They argued essentially that the Court should ignore average exam scores and pass-fail rates because the only value they have
[34]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 35 of 82
is in predicting promotion rates, which can measured directly. See Silva Report 13; Post-Trial City’s Proposed Findings 45. Dr. Silva argued that a delay in promotion is an inappropriate measuring device because “the number of days between promotions is not a function of the test, it is a function of when the positions open up. The test is only responsible for the order in which the promotions are made.” Silva Report 6. Analyzing only the promotion rates in 2005 and 2008 using a two-tailed test, the City argues, one cannot conclude that the 2008 exam resulted in a statistically significant adverse impact, and thus, judgment should enter in the City’s favor. Silva Report 7-10; Post-Trial City’s Proposed Findings 41.
The Court agrees with the Plaintiffs that statistics other than promotion rates are relevant in evaluating disparate impact. The City’s argument that promotion rates are the only relevant factor is a “bottom line” defense which the Supreme Court has rejected. In the seminal case Connecticut v. Teal, the Supreme Court stated:
In considering claims of disparate impact under [Title VII], this Court has consistently focused on employment and promotion requirements that create a discriminatory bar to opportunities. This Court has never read § 703(a)(2) as requiring the focus to be placed instead on the overall number of minority or female applicants actually hired or promoted. Teal, 457 U.S. 440, 450 (1982). In Teal, a pass-fail test had an adverse impact on minorities but, due to an affirmative
[35]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 36 of 82
action program, no adverse impact on promotion rates. Id. at 443-44. The Supreme Court rejected the agency’s “bottom-line” defense, admonishing that “[t]he suggestion that disparate impact should be measured only at the bottom line ignores the fact that Title VII guarantees these individual respondents the opportunity to compete equally with white workers on the basis of job-related criteria.” Id. at 451. In other words, “individual components of a hiring process may constitute separate and independent employment practices subject to Title VII even if the overall decision-making process does not disparately impact the ultimate employment decisions involving a protected group.” Bradley, 443 F. Supp. 2d at 158-59. Under the progeny of Teal, even in the absence of adverse impact on promotion rates, an exam can lead to liability for an employer if it functions as “a gateway that has a disparate impact on minority hiring.” Id. at 159. Promotion rates -- the “bottom line” -- are thus not the only relevant inquiry in this Court’s disparate impact analysis.
The 2005 and 2008 exams served two functions: they were used as pass-fail hurdles and they accounted for 80% of the final score that determined candidates’ rank on an eligibility list from which they were promoted in rank order. Under such a scheme, this Court cannot rule that passing rates and average scores are irrelevant. Indeed, the Second Circuit has ruled
[36]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 37 of 82
that where an exam is used as both a pass-fail hurdle as well as a mechanism for ranking candidates for a promotion (as is the case here), courts should consider the disparate impact in the pass-fail rates, as well as the placement of ethnic groups on the ranking list. Waisome v. Port Auth. of New York & New Jersey, 948 F.2d 1370, 1377 (2d Cir. 1991). The average scores and passing rates are relevant to the Court’s determination of whether the Plaintiffs have met their burden on prong one.23
The City argued that the timing of promotions is largely determined by the timing of vacancies, so delays are more relevant to the damages phase of the litigation than to the liability phase. See Post-Trial City’s Proposed Findings 78. Neither argument persuades the Court to exclude delays in promotion from its analysis. The first argument falls flat because promotion rates themselves are determined, at least in part, by vacancies, and the City nowhere argues that promotion rates are irrelevant. Regarding the damages argument, the Court acknowledges that the delay in promotion will be relevant at the damages phase of the litigation, but it can also constitute disparate impact in the form of loss of pay, benefits, and seniority. See Bradley, 443 F. Supp. 2d at 168 (ranking by exam score disproportionately precluded minority candidates from
23 In the City’s own validation study of the 2002 exam, Morris & McDaniel based its statistical analysis on mean scores. Draft Validity Report 14; Assessment Center Report 18.
[37]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 38 of 82
earning an earlier promotion, thus constituting an adverse impact); Guinyard v. City of New York, 800 F. Supp. 1083, 1089 (E.D.N.Y. 1992) (same). This comports with common sense, with case law, and with the spirit of Teal -- that employers may not circumvent Title VII “by merely showing that eventually they may hire some members of the disadvantaged minority group.” Bridgeport Guardians, Inc. v. City of Bridgeport, 933 F.2d 1140, 1147-48 (2d Cir. 1991) (citing Teal, 457 U.S. at 455-56).
The Court will therefore consider all of the factors that Dr. Wiesen statistically analyzed: promotion rates, pass-fail rates, average scores, and delays in promotion.
2. To Aggregate or Not to Aggregate? For two of the four aspects of the promotional procedure, promotional rates and passing rates, Dr. Wiesen aggregated the data for the 2005 and 2008 exams, Wiesen Report 10, 13, properly taking care to account for people who took both exams. Id. at 5. Dr. Wiesen stated that aggregation yields a “more powerful statistical test [because] you have a larger sample size and you see . . . the big picture.” 12/15/14 Tr. 83:10-17. See also Wiesen Report 5. Dr. Jacobs, a colleague of Dr. Silva’s, argued in a pretrial affidavit that aggregation was inappropriate. He posited that aggregation “only increases the sample size without a strong underlying logic as to the appropriateness of treating
[38]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 39 of 82
candidates from 2005 and 2008 as competing for the same jobs.” Aff. Rick R. Jacobs, PhD (“Jacobs Aff.”) ¶ 15, ECF No. 93. Dr. Silva opined in his expert report that aggregating the data between 2005 and 2008 is inappropriate because of Simpson’s Paradox, a phenomenon by which an “effect exists in two separate data sets but disappears when the data sets are combined or vice versa.” Silva Report 10. During trial, Dr. Silva testified that aggregation creates a risk of distortion, even if Simpson’s Paradox is not present. 12/18/14 Tr. 33:6-19, ECF No. 165.
Dr. Wiesen does not offer a sound basis for aggregating the data, other than its favoring the case of the party who hired him. He implies in his report that he only aggregated data if his original analysis did not produce statistically significant and practically important numbers for individual exam years. See Wiesen Report 16. In other words, he aggregated when the results using individual exams were not strong enough to help the Plaintiffs. The Court is not persuaded that this provides a sufficient basis to aggregate two data sets.24
This Court rules that aggregation is inappropriate in this case. The 2005 and 2008 exams presented different questions,
24 In the Lopez case, Dr. Wiesen aggregated data across exam years and across jurisdictions within the Commonwealth. Lopez, at 20. Judge O’Toole found such aggregation inappropriate in part because of Simpson’s Paradox, but also for other reasons. Id. at 18-20.
[39]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 40 of 82
had different mean scores, and tested different candidates. 12/16/14 Tr. 129-131, ECF No. 162. Moreover, the Court has already ruled that the 2005 exam is not actionable; combining the 2005 scores with the 2008 scores would allow the Plaintiffs to circumvent this ruling. It would also raise so-called slippery slope concerns: why not aggregate with exams from the 1990s? Why not the 1980s? The Court sees no legitimate basis for aggregating the statistics on the facts of this case and therefore declines to do so.
3. One-Tailed vs. Two-Tailed The next dispute the Court must resolve involves statistical methodology. It is a familiar one in disparate impact cases: whether to use a one- or two-tailed test when testing for statistical significance. Drs. Wiesen and Silva both used “Fisher Exact Tests” to compare the exam results for candidates who are members of a minority group and white candidates. 12/17/14 Tr. 148:8-20. Fisher Exact Tests produce a bell curve with a tail on either end representing the lowest probability events. Id. at 149:24150:9. When conducting a Fisher Exact Test, one can use a onetailed or a two-tailed test. The terms “one-tailed” and “twotailed” reflect whether statistical significance is determined from one or both the tails of the sampling distribution.
[40]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 41 of 82
A two-tailed test assumes that any result could come from the test: in this case, in determining whether a given result was due to random chance, a two-tailed test would entertain three possibilities: that minorities would outperform nonminorities, non-minorities would outperform minorities, or that their performances would be equal. 12/17/14 Tr. 150:10-19. In contrast, a one-tailed test assumes only two possibilities: in this case, the performance between the groups was equal, or minorities performed worse on the test than non-minorities. Id. at 150:20-151:1; see Palmer v. Shultz, 815 F.2d 84, 94-95 (D.C. Cir. 1987).
In Dr. Wiesen’s original report, he conducted all of his analyses using a two-tailed test, stating that although the onetailed approach is more logically defensible, the two—tailed approach is more conservative. Wiesen Report 7 n.4; 12/16/14 Tr. 119:25-120:2. Subsequent to his initial report, and before the City offered its expert report, two additional sergeants were promoted to lieutenant, one white and one black. 12/15/14 Tr. 108:19-109:3.
When analyzing the data with the two new hires using a twotailed test, Dr. Jacobs, the City’s expert, concluded that the adverse impact for the promotion rates for the 2008 exam was not statistically significant. Jacobs Aff. ¶ 14. Dr. Jacobs found a p-value of .052, a hair above the .05 threshold. Id. ¶ 10.
[41]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 42 of 82
Dr. Silva arrived at the same statistical conclusion. Silva Report 8.
Dr. Wiesen acknowledged this lack of statistical significance using a two-tailed test to examine the new data set. Second Aff. Joel P. Wiesen (“Wiesen Second Aff.”) ¶ 6, ECF No. 103. He subsequently switched his analysis from a twotailed test to a one-tailed test, defending this approach by explaining that the question asked in this litigation is whether there was adverse impact on minorities, and thus, the “onetailed test is more appropriate.” 12/16/14 Tr. 122; Wiesen Second Aff. ¶¶ 1, 7. Dr. Wiesen concluded that, even accounting for the two new hires, the “proper statistical conclusion is that there was adverse impact in promotions, both for the 2008 and the 2005 exams.”25 Id. ¶ 10.
The City bristled at Dr. Wiesen’s flip-flop from the twotailed test to the one-tailed test, arguing that the two-tailed test is the appropriate one because it “accepts it is possible on a promotional examination that minorities could outscore nonminorities or that non-minorities could outscore minorities[.]” Post-Trial City’s Proposed Findings 41. Dr. Silva argued that a
25 The p-value for the one-tailed test of the 2008 promotional rates was .027, which is less than .05 and is thus statistically significant. Wiesen Second Aff. ¶ 8. After running some additional analyses, Dr. Wiesen stated that the increase of the p-value above .05 using a two-tailed test was “a blip, not a pattern.” Id. ¶ 6.
[42]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 43 of 82
one-tailed approach was inappropriate because it “implies that there is no chance that the direction of a promotion rate difference will ever favor minorities,” which contradicted Dr. Silva’s professional experience. Silva Report 2-3.
Whether to use a one-tailed or a two-tailed test is a common point of contention in disparate impact cases. Title VII defendants often argue that both minority and majority groups are protected from discrimination, and “it is therefore inequitable to disregard the probability of outcomes that may favor either group.” Palmer, 815 F.2d at 95. Defendants are also aware, of course, that it is easier for plaintiffs to prove significance and thus disparate impact with a one-tailed test. See, e.g., Brown v. Delta Airlines, Inc., 522 F. Supp. 1218, 1228 n.14 (S.D. Texas 1980).
The weight of the case law appears to favor two-tailed tests. In Palmer, the D.C. Circuit Court of Appeals favored the two-tailed test for Title VII cases, 815 F.2d at 95-96, and that court continues to do so today. Csicseri v. Bowsher, 862 F. Supp. 547, 564 (D.D.C. 1994) aff'd, 67 F.3d 972 (D.C. Cir. 1995). Other courts have agreed. See, e.g., Dicker v. Allstate Life Ins. Co., No. 89 C 4982, 1997 WL 182290, at *41 (N.D. Ill. Apr. 9, 1997) (two-tailed is especially appropriate in disparate impact claims involving facially neutral employment selection procedures).
[43]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 44 of 82
Courts recognize, however, that a one-tailed test can be appropriate in some Title VII circumstances, such as when “one population is consistently over-selected over another.” Stender v. Lucky Stores, Inc., 803 F. Supp. 259, 323 (N.D. Cal. 1992); see Brunet v. City of Columbus, 642 F. Supp. 1214, 1230 (S.D. Ohio 1986) rev'd on other grounds, 1 F.3d 390 (6th Cir. 1993) (stating that the one-tailed test is appropriate where “the raw numbers indicate that women are selected at a lesser rate than men. In these circumstances, the question being asked is whether this apparent difference is real or a statistical artifact.”). A one-tailed test can also be appropriate if “[t]here is little chance, from a facial review of the evidence, that applicants” in the plaintiff class were “treated statistically better” than those in the other group. Csicseri, 862 F. Supp. at 564-65. The First Circuit has overtly avoided choosing between the two. Jones, 752 F.3d at 43 n.5.
There are two good arguments for using a one-tailed test in this case. The first is broader, relying on current inequalities in our society to put this case in context, and the second is narrower, implicating the history between this particular defendant, and this particular class of plaintiffs.
Experts for both sides testified to the phenomenon of written multiple-choice tests producing high levels of adverse impact on minority candidates. 12/15/14 Tr. 113-114; 12/18/14
[44]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 45 of 82
Tr. 45-46, 70; Lopez, 07/13/10 Tr. 82-85; Lopez, 07/14/10 Tr. 43-48, 55, 59-60; Lopez, 07/26/10 Tr. 30; Lopez, 09/15/10 Tr. 58-59; Lopez, 09/16/10 Tr. 110. Judge O’Toole in Lopez also recognized this phenomenon. Lopez, slip op. at 14. Experts in the seminal case of Ricci v. DeStefano similarly testified. 557 U.S. at 570, 572 (2009). Why the difference in performance between members of minority groups and white applicants on these tests? Neither expert addressed this question, other than Dr. Wiesen suggesting that “there are . . . dozens of reasons, each of which account[] for just a small amount of that difference.” 12/15/14 Tr. 114:10-13. Without wading into social-scientific debates, the Court agrees with Dr. Wiesen that there are likely several factors driving this disparity (e.g., legacies from historical discrimination, economic inequality, current explicit and implicit biases). Whatever the causes, the so-called “achievement gap” is real,26 and might recommend adopting a onetailed test as more rooted in reality.
26 That it is an uncomfortable truth does not rob it of its current empirical foundation. There are many examples of the differences in academic achievement between white students and members of the minority groups at issue in this case (black and Hispanic).
This gap is present early. By fourth grade, white public school students outperform Hispanic students in both math and reading by the equivalent of roughly two grade levels. See U.S. Dep’t of Educ., Nat’l Cent. for Educ. Statistics (“NCES”), How Hispanic and White Students in Public Schools Perform in Mathematics and Reading on the National Assessment of Educational Progress 10-11, 36-37 (June 2011) (describing point-
[45]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 46 of 82
In addition, Boston has a history of discrimination against minority police officers. See Boston Police Superior Officers Fed'n v. City of Boston, 147 F.3d 13, 20 (1st Cir. 1998). This history might suggest that viewing white police-officerapplicants as equally likely to over- and under-perform minority applicants on an exam developed by Boston is overly idealistic.
The Court is hesitant, however, to analyze a disparate impact case, a case in which no one has accused the Department of any malfeasance or conscious desire to discriminate, under the assumption that minorities could only underperform and not also possibly outperform their white peers on a promotional exam. Cf. Parents Involved in Cmty. Sch. v. Seattle Sch. Dist. No. 1, 551 U.S. 701, 748 (2007) (Roberts, C.J.) (plurality opinion) (“The way to stop discrimination on the basis of race is to stop discriminating on the basis of race.”); id. at 789 (Kennedy, J., concurring) (suggesting that “facially raceneutral means” or considering race as part of “a more nuanced,
differential on tests; providing scores for grades four and eight for estimating grade-level gains). The same goes for white public school forth-graders as compared with black fourthgraders. See NCES, How Black and White Students in Public Schools Perform in Mathematics and Reading on the National Assessment of Educational Progress iii (July 2009).
The gap also manifests itself in later measures of academic success. For one, the four-year high school graduation rate for white students is currently 84%, for black students 69%, and for Hispanic students, 71%. NCES, Public High School Four-Year OnTime Graduation Rates and Event Dropout Rates: School Years 2010-2011 and 2011-12 4 (April 2014).
[46]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 47 of 82
individual” evaluation of applicants is permissible, whereas
broad-based classifications based on it are subject to strict
scrutiny).27 Moreover, although the First Circuit and the
Supreme Court have remained relatively quiet on the issue of one
versus two tails, the Supreme Court has previously suggested
that a protected class’s treatment that falls two or three
standard deviations beyond the mean is evidence of disparate
impact. See Castaneda v. Partida, 430 U.S. 482, 496 n.17
(1977). This requirement is closer to that of a two-tailed test
(which requires a result be more than 1.96 standard deviations
from the mean to reach statistical significance) than of a one-
tailed test (1.65). The Court is inclined to agree with the
City that a two-tailed test is the more appropriate methodology
for evaluating statistical significance in this case.
The debate over the one versus two-tailed test, while
important and fascinating, is not dispositive in this case. As
the Court explains in the next section, the Plaintiffs have met
27 Parents Involved, of course, has been the subject of considerable scholarship, with the Chief Justice’s quip, quoted above, raising particularly strong objections, see, e.g., James E. Ryan, The Supreme Court and Voluntary Integration, 121 Harv. L. Rev. 131, 156-57 (2007) (characterizing the pronouncement as saying “we know how to end race discrimination in this country and you [a Seattle school district, in that case,] who are closer to the issue than we will ever be and have been working in good faith toward the same end, do not.”) (internal footnote omitted). The Court does not comment on the merits of the Supreme Court’s analysis in Parents Involved but simply notes that applying a one-tailed test would appear to be in tension with its reasoning.
[47]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 48 of 82
their burden of demonstrating disparate impact, regardless of
whether the Court accepts the one or two-tailed approach for the
2008 promotion rates.
4. Conclusions Regarding Disparate Impact (Prong 1)
In his expert report, Dr. Wiesen concluded that:
• The adverse impact ratio for promotions from the 2008 exam was .45 (meaning that minorities were promoted at a rate of .45 as compared to the rate at which whites were promoted, well below the 4/5 rule, which would be satisfied by a rate of up to .80), Wiesen Second Aff. 7, and .38 for the 2005 exam. Wiesen Report 8. He concluded that the p-value for the 2008 promotion rates was .052 for a two-tailed test, and .027 for a one-tailed test. Wiesen Second Aff. 7. The ratio for the 2005 exam was not statistically significant. Wiesen Report 9.
• The adverse impact ratio for passing scores was .74 for the 2008 exam and .57 for the 2005 exam, both of again satisfy the 4/5 rule, and both of which were statistically significant at a p-value of .004 for the 2008 exam and .00005 for the 2005 exam, using a twotailed test. Wiesen Report 11-12; 12/15/14 Tr. 88:14.
• The adverse impact for average scores was 6.6 points on the 2008 exam (meaning that minority applicants scored, on average, 6.6 points lower than white applicants) and 8.8 points on the 2005 exam, both of which were “highly statistically significant” at a pvalue for the 2008 exam of .0015 and for the 2005 exam of .00003 (both using a two-tailed test). Wiesen Report 3, 15-16.28
• The adverse impact for delay in promotions for the 2008 exam was an average of 750 additional days, which was statistically significant at a p-value of .001 (using a two-tailed test). Wiesen Second Aff. 4-5.
28 This analysis was based on the scores from the multiple choice exam alone. Id. at 13.
[48]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 49 of 82
There was no adverse impact for promotion dates from the 2005 exam. Wiesen Report 16-17. The Court concludes that the Plaintiffs have met their burden of establishing disparate impact stemming from the 2008 exam. The fact that the p-value for the 2008 promotion rates was .052 using a two-tailed test, a breath above the .05 threshold, is not enough to persuade the Court that the Plaintiffs failed to meet their burden of establishing a prima facie case of disparate impact. This is so for various reasons: First, while the .05 cut-off for a finding of statistical significance is generally accepted, such thresholds are not hard-line rules in disparate impact cases. E.g., Jones, 752, F.3d at 46-47 (explicitly declining to rule on whether establishing a p-value of .05 was a requirement to show disparate impact); Little v. Master-Bilt Products, Inc., 506 F. Supp. 319, 333 (N.D. Miss. 1980) (rejecting a hard-line cut-off of a certain p-value to make out a prima facie Title VII case based on statistical evidence alone). Second, the City’s expert on test validity acknowledged that when significance is between .05 and .10, the results are “marginally significant.” 01/05/2015 Tr. 53:2-15. He further acknowledged that a marginally significant analysis, combined with other analyses that are statistically significant, can lead to a finding that an adverse impact has been demonstrated. Id.
[49]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 50 of 82
As explained above, the Plaintiffs have presented evidence of statistically significant disparate impact on pass-fail rates, averages scores, and delay in promotions.
Third, the First Circuit has acknowledged that when evaluating statistical significance, “no single test controls in measuring disparate impact.” Langlois, 207 F.3d at 50 (citing Watson, 487 U.S. at 995-96 n.3) (noting that a case-by-case approach is appropriate); see also Waisome, 948 F.2d at 1376 (noting that courts should “consider[] not only statistics but also all the surrounding facts and circumstances”); Police Officers for Equal Rights v. City of Columbus, 644 F. Supp. 393, 432 n.13 (S.D. Ohio 1985) (stating that even if plaintiff fell short of cut-offs for statistical significance, “other evidence submitted in this case . . . tends to support the inference of disparate impact suggested by plaintiff’s statistical data”). The Plaintiffs bolster their evidence on statistical significance by presenting evidence that the 2005 and 2008 exams violated the four-fifths rule.29 As this Court has already held,
29 Using Monte Carlo simulations, Dr. Silva concluded that Dr. Wiesen’s adverse impact ratio calculations (which form the basis of the four-fifths rule) had a high error rate, and the Court should therefore disregard them. Silva Report 5, 10. The Court might find this argument more persuasive if the only evidence Plaintiffs had put forth was a violation of the fourfifths rule. Such is not the case here. Moreover, the City points to no case law addressing a statistical analysis of error rates for the four-fifths rule, and the Court sees no obvious reason for such an analysis. The four-fifths rule, unlike a
[50]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 51 of 82
“a violation of the four-fifths rule . . . may demonstrate
adverse impact, particularly when coupled with other statistical
evidence of adverse impact.” Cotter, 193 F. Supp. 2d at 348
n.12; see also Bradley, 443 F. Supp. 2d at 163 (holding that the
plaintiffs established a prima facie case of statistical
significance by using the four-fifths rule combined with a chi-
square analysis).30
The Court also considers the fact that if the eligibility
list from the 2008 exam had not been extended due to the Lopez
litigation, but instead had expired three years after its
measurement of statistical significance, is a rough and ready rule of guidance which is not given decisive weight. It is a “rule of thumb” only, and the Court regards it as such.
30 Both parties, citing to Jones, agree that the four-fifths rule cannot, by itself, be given decisive weight. 12/15/14 Tr. 47:2-14. In Jones, the First Circuit held that defendants in disparate impact cases could not use the four-fifths rule to trump a statistically significant showing of disparate impact. Jones, 752 F.3d at 52. The City cites to this case law in an effort to convince this Court to disregard the four-fifths rule altogether. 12/15/14 Tr. at 46-47; Def.’s Mot. In Limine Exclude Use “Four-Fifths Rule” Prove Disprove Adverse Impact Based Race, ECF No. 131. The First Circuit did not hold that the four-fifths rule has no place in disparate impact jurisprudence; the First Circuit specifically noted that the regulation establishing the four-fifths rule shows “that the commission views practical significance, along with statistical significance, as relevant in identifying a disparate impact.” Jones, 752 F.3d at 50 (citing 29 C.F.R. § 1607.4(D)). The Court went on to say that the “four-fifths rule may serve important needs,” is a helpful “rule of thumb,” and “has some practical utility.” Jones, 752 F.3d at 52. The Supreme Court cited to the four-fifths rule in the very case to which the City so extensively cites in its post-trial briefing. Ricci, 557 U.S. at 586-87; Post-Trial City’s Proposed Findings 4.
[51]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 52 of 82
creation, as is typical, not a single black sergeant would have been promoted to lieutenant. 01/06/15 Tr. 144-45. Lastly, the Court views as likely the probability that the 2008 figures underestimate the disparate impact considering that at least some of these test-takers had presumably passed the 2005 sergeants’ exam, which Boston conceded as having a disparate impact on minority candidates. Lopez, at 20; see also Nash v. Consol. City of Jacksonville, Duval Cty., Fla., 895 F. Supp. 1536, 1544 (M.D. Fla. 1995) aff'd sub nom., 85 F.3d 643 (11th Cir. 1996) (holding that the plaintiff made out a prima facie case of disparate impact despite a lack of statistical significance based on past discrimination which reduced the number of African-Americans eligible to sit for the promotional exam).
All of this evidence combined is enough for this Court to rule that the Plaintiffs have met their burden of raising an inference of causation and demonstrating a prima facie case of disparate impact. The burden thus shifts to the City to defend its promotional process as a valid selection tool.
B. Job-Related and Consistent with Business Necessity (Prong 2) 1. Introduction
To pass muster under Title VII once disparate impact has been shown, the City must convince the Court that the 2008 exam
[52]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 53 of 82
was both “job related” for the position of Boston Police Department lieutenant and consistent with “business necessity.” Jones, 752 F.3d at 53(quoting 42 U.S.C. § 2000e–2(k)(1)(A)(i)). The purpose of this second prong is not for the Court to substitute its judgment for the City’s or for that of experienced police officers, but to ensure that the City took sufficient care to ensure that its employment practice was consistent with business necessity. Cf. Texas Dep't of Hous., 135 S. Ct. at 2512 (holding that “[p]olicies, whether governmental or private, are not contrary to the disparateimpact requirement unless they are artificial, arbitrary, and unnecessary barriers”) (internal quotation marks omitted); Langlois, 207 F.3d at 54 (Stahl, J., dissenting) (stating, in the discriminatory housing context, that a practice fails a similar standard if, “without demonstrably advancing the interest asserted in justification, [it] somehow impedes persons of color from competing on an equal footing with others”).
The First Circuit has instructed that defendants whose employment practices produce a disparate impact must establish two elements to satisfy this second prong: “First, the [defendant] must show that its program aims to measure a characteristic that constitutes an ‘important element of work behavior.’” Jones, 752 F.3d at 54 (quoting Albemarle Paper Co. v. Moody, 422 U.S. 405, 431 (1975)). Second, the defendant
[53]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 54 of 82
“must show that the outcomes of [its challenged practice] are ‘predictive of or significantly correlated with’ the characteristic described above.” Id. (quoting Albermarle Paper Co., 422 U.S. at 431). This framework directly mirrors that in the Uniform Guidelines, compare 29 C.F.R. § 1607.5(B) (“[T]he selection procedure [must be] predictive of or significantly correlated with important elements of job performance.”).
The Guidelines provide a sensible way of evaluating whether a given test, like the one in this case, measures an important work characteristic, and whether the outcomes of that test are actually correlated with the characteristic measured. Many courts have utilized the Guidelines in exactly this way, and have re-phrased the prong two inquiry (following the Guidelines’ language) as a determination of whether a given test is “valid.” See, e.g., M.O.C.H.A. Soc'y, Inc. v. City of Buffalo, 689 F.3d 263, 274 (2d Cir. 2012) (discussing validity as goal of prong 2 inquiry; utilizing the Uniform Guidelines in its analysis); Bryant v. City of Chicago, 200 F.3d 1092, 1094 (7th Cir. 2000) (same); Williams v. Ford Motor Co., 187 F.3d 533, 539 (6th Cir. 1999) (same); Lopez, at 28 (same); Bradley v. City of Lynn, 443 F. Supp. 2d 145, 161 (D. Mass. 2006) (Saris, J.) (same). This Court follows their lead, and will look to the Uniform Guidelines throughout its prong 2 analysis.
[54]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 55 of 82
2. Did the Department Aim to Measure a Characteristic that Constitutes an Important Element of Work Behavior?
Beginning the First Circuit’s two-part inquiry for prong two, the Court must determine whether the 2008 exam measures “characteristic[s] that constitute[] . . . important element[s] of work behavior.” Jones, 752 F.3d at 54 (internal citation omitted). In short, the Court holds that it does.
The development of most promotional examinations begins with a job analysis. A job analysis “includes an analysis of the important work behavior(s) required for successful performance and their relative importance.” 29 C.F.R. § 1607.14(C)(2). It need not be developed concurrently with an exam to form the basis of the exam’s validation. See Rudder v. District of Columbia, 890 F. Supp. 23, 42 (D.D.C. 1995) (a job analysis created earlier which is itself based on an even older job analysis retains relevance when officials were interviewed to ensure the job analysis was still relevant and the job had not changed significantly). A job analysis typically involves Subject Matter Experts (again, SMEs) identifying important tasks for the job, followed by identifying Knowledge, Skills, and Abilities (again, KSAs) necessary to perform those tasks, followed by linking the KSAs back to the tasks. E.g., United States v. City of New York, 731 F. Supp. 2d 291, 302 (E.D.N.Y. 2010).
[55]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 56 of 82
Dr. Campion testified that the 1991 and 2000 job analyses, on which the 2008 exam was based, were conducted in great detail and reflected good linkage between KSAs and tasks. 12/19/14 Tr. 32:13-16, 36-37. He testified that the age of the 1991 job analysis did not undercut its usefulness, id. at 15:3-23, and the 2005 and 2008 mini-job analyses, while not as thorough as the 1991 and 2000 job analyses, were nonetheless sufficient because the job of lieutenant had not changed much over time. Id. at 67:14-68:1; Campion Report 21-22.
Dr. Wiesen testified that the 2000 job analysis –– which was the focus of his report -- was insufficient. He opined that the SMEs’ rankings were not done with sufficient care or attention, reflected by the fact that the SMEs unanimously agreed that they performed the tasks of reporting homicides and disciplining subordinates on a daily basis, which, based on the statistics, is impossible. See Wiesen Rebuttal 15-16. He noted that SME ratings for the second half of the list of 302 tasks became much more uniform and cursory, likely reflecting rater fatigue. Id. at 87. He complained that mis-numbering of tasks and KSAs likely led to inaccurate ratings. Id. at 88. He also complained about the wording of the tasks in the final task list. Id. at 89.
While acknowledging that the 2000 job analysis may have some errors and is not a model of perfection, the Court
[56]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 57 of 82
concludes that the robust job analyses performed in 1991 and 2000, and the mini-job analyses performed in 2005 and 2008, were adequate. As detailed above, for the 1991 job analysis, DPA identified important work behaviors by gathering information from various sources, created a list of tasks, asking police officers to rate the tasks, creating a list of potentially important KSAs, and asking SMEs to link KSAs to tasks. The same was true for the 2000 job analysis for which Morris & McDaniel, in part using the 1991 report, created a list of 302 possibly relevant tasks and KSAs, which SMEs rated. In 2005 and 2008, HRD asked SMEs to again rank tasks and KSAs that had been identified in the older reports.
Although there may have been some errors or snags in the review system –– for instance, there seems to be no evidence that the SMEs in 2008 linked the important KSAs back to tasks –– the lists of important tasks and KSAs emerging from the various job analyses strike the Court as adequately capturing the role of a Boston Police lieutenant. Dr. Wiesen nowhere claims that the lists of important tasks and KSAs have glaring gaps, and Plaintiffs’ expert Dr. Hough opined that the 145 KSAs identified in the 2000 job analysis were consistent with the studies she had done in the field. 01/06/15 Tr. 19:5-10. The Court finds that the job analyses were sufficiently thorough and current so as to form solid ground on which to build a valid test; in other
[57]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 58 of 82
words, they measure “an important element of work behavior.” Jones, 752 F.3d at 54 (internal citation omitted). The City therefore has satisfied its burden for the first element of prong two.
3. Were the Exam Results Predictive of or Correlated with the Important Work Behaviors?
Dr. Michael A. Campion, who holds a PhD in industrial psychology, served as the City’s expert on the issue of validity. 12/19/14 Tr. 5:21-6:4. In his expert opinion, the 2008 exam was content valid (more on that below) because it complied with the Uniform Guidelines “well enough,” 12/19/14 Tr. 54:22-55:2, complied with the Society for Industrial Organizational Principles, Ex. 73, Report Michael A. Campion, PhD (“Campion Report”) 21, and conformed with best practices in test development, id. at 25.
Dr. Wiesen again served as the Plaintiffs’ expert to discuss validation. He concluded that the 2008 exam was not content valid. In doing so, he criticized nearly every step of the City’s validation process, including its job analyses, Wiesen Rebuttal 15, the construction of the exam, id. at 18, the content of the exam, id. at 26, and how the exam was used (here, to rank candidates), id. at 28-32. Although the Court is not persuaded by all of Dr. Wiesen’s criticisms, the Court ultimately agrees with Dr. Wiesen that the evidence does not
[58]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 59 of 82
support the necessary inference that those who perform better on the exam will be better performers on the job, primarily because the exam did not test a sufficient range of KSAs, and there was no evidence that the exam was reliable enough to justify its use for rank ordering.
a. Methodology Employers are not required to submit formal validation studies to meet this standard, but neither are courts required to take a leap of faith and simply accept an employer’s claim to validity. Bradley, 443 F. Supp. 2d at 171 (citing Watson, 487 U.S. at 998; Steamship Clerks, 48 F.3d at 607). Validation is not primarily a legal subject, and requires expertise that courts lack. Guardians Ass'n of New York City Police Dep't, Inc. v. Civil Serv. Comm'n of City of New York, 630 F.2d 79, 89 (2d Cir. 1980). There are two methods commonly used for determining validity: content validity and criterion validity. 29 C.F.R. § 1607.5(B). Criterion validity is a statistical analysis in which an analyst correlates the selection procedure with job performance. 12/19/14 Tr. 19:25-20:21. Content validity, by contrast, is not statistical; it is an attempt to link the important KSAs of the job with the selection procedure. Id. at 21:3-10. Under either approach, the goal is to make inferences from the test scores about future job performance. Id. at
[59]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 60 of 82
22:10-16. Here, the City attempted to shoulder its burden on
job relatedness using content validity. Id. at 25:14-19;
Campion Report 5-6; Wiesen Rebuttal 8.
In evaluating content validity, experts in the field31 rely
on three authorities: the Uniform Guidelines, mentioned above,
the “Standards for Educational Psychological Tests,” referred to
as “the Standards,” and the “Society for Industrial
31 Fact witnesses for both sides offered their opinions regarding the ability of the Department’s promotional procedures to predict job performance. Commissioner William Evans testified that he had observed a direct relationship between how well a person did on a Department knowledge exam and how well they performed on the job. 01/07/15 Tr. 41-42, 46. Commissioner Evans’ predecessor, Commissioner Davis, disagreed. He thought that communication and interpersonal skills are very important for a superior officer, 01/05/15 Tr. 91:3-11, and the inclusion of assessment centers (in addition to knowledge tests) improved the Department’s promotional procedures. Id. at 91:1792:1. In his opinion, past job performance is likely one of the best indicators for core lieutenant skills. Id. at 113:5-15. Commissioner Davis’s predecessor, Commissioner Paul Evans (coincidentally, the brother of the current Commissioner) expressed his opinion in a memo that “the best indicator of future performance is past performance . . . . The best supervisors cannot always be identified solely by their performance on a written test and an hour in an assessment center. . . . We must become willing to reward police work, not memorization skills.” Lopez, Ex. 194.
The named plaintiff in this case, Bruce Smith, testified that the 2008 exam did not ask questions that would differentiate his ability to serve as a lieutenant as opposed to a sergeant. 12/17/14 Tr. 109:7-15. Although the Court regards the fact witness’ testimony as important for certain aspects of this case, and cites to it accordingly, the fact witnesses’ personal opinions on the effectiveness of the testing procedures do not aid the Court on this prong of the framework, which requires the Court to rely on testing expertise. Guardians, 630 F.2d at 89.
[60]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 61 of 82
Organizational Psychology” principles, referred to as “the Principles.” 12/15/14 Tr. 118:4-19; Ex. 49, Rebuttal Dr. Campion’s Report Validity Alternatives 2008 and 2005 Exams (“Wiesen Rebuttal”) 10.
b. The Scope of the Analysis The 2008 exam consisted of two components: the written, multiple-choice component (weighted at 80%), and the E&E component (weighted at 20%). 12/15/14 Tr. 49:25-50:17. In analyzing the validity of the 2008 exam, the Court will exclude the E&E portion for the reasons detailed below. Every candidate that sits for the exam is automatically awarded fourteen of the twenty possible E&E points. Id. at 50:24-51:5. Dr. Wiesen posits that, because of this practice, the E&E portion in actuality has very little bearing on where the candidate ranks on the eligibility list. Id. In his report, Dr. Wiesen correlated candidates’ scores on the written exam with their final exam score, calculating a correlation coefficient of .95, an almost perfect positive correlation. Wiesen Report 20.32 In other words, the E&E component and the bonus points awarded for things such as veteran’s status had virtually “no impact on the final exam scores.” 12/15/14 Tr. 58:19-21. It is the score on the written exam, Dr. Wiesen
32 For the 2005 exam, the coefficient was .96. Wiesen Report 21.
[61]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 62 of 82
argues, that drives a candidate’s placement on the eligibility list.
The City counters this point by arguing that the fourteen points automatically awarded to every candidate should not be discounted because they measure characteristics important to serving as a lieutenant. By way of analogy, the City offers that medical degrees, while held by every doctor, are made no less important in a promotional context simply because every doctor holds one. Post-Trial City’s Proposed Findings 58-59; 01/05/15 Tr. at 56:11-20.
The City’s argument is not persuasive. The Court does not deny that the length of employment and nature of experience of a sergeant in the Department, much like a doctor’s medical degree, is an important input to a candidate’s performance on the job. But the medical degree, while an important requirement for a doctor, would not aid the hospital in selecting which doctors to promote. Dr. Campion acknowledged as much when he testified in the context of this doctor analogy that a medical degree “may not differentiate [the doctor candidates] very much.” 01/05/15 Tr. at 56:12-20. Here, the Court is evaluating the Department’s method for selecting whom to promote within a pool of candidates who all have earned fourteen points. In short, differentiation is the crux of this case. Dr. Wiesen’s analysis persuades this Court to find that the E&E portion of the exam had a de minimis
[62]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 63 of 82
impact on the candidates’ final scores. The Court will therefore focus on whether the 2008 written exam was content valid.
In doing so, the Court will evaluate the adequacy of the test construction process used by the City, the exam’s content (the extent to which it matches up with the job), and how the Department used the exam (meaning its system of scoring the exam and then using those scores to rank candidates). Cf. Guardians, 630 F.2d at 95 (“distill[ing] five attributes” from the Guidelines relevant to assessing content validity covering “the quality of the test’s development[,]” the test’s content, and the test’s grading).
c. Adequacy of Test Construction Having explained the overall framework and appropriately narrowed the scope of its inquiry, the Court finally begins its analysis of the second element of prong two, looking first to test construction. The Court finds that the City fell short at many stages of the test construction process. In his expert report, Dr. Campion addresses many factors that go into test development. The Court will address those that strike it as most relevant to deciding the instant case. When using a multiple choice exam, the developer must convert the job analysis result into a test plan to “ensure a direct and strong relationship between the job analysis and the
[63]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 64 of 82
exam.” Campion Report 24. As Dr. Campion points out, the City created test plans for the 2008 exam. Ex. 54; Ex. 60. The Court cannot find, however, that the test plan ensured a strong relationship between the job analysis and the exam. As discussed above, the test outlines reflect that the 2008 exam was written to test knowledge; the outlines reveal that only two abilities appeared on the 2008 exam. Because the Court has found that the E&E component had very little bearing on the final score, the Court cannot find that the City met its burden on this step: too many skills and abilities were missing from the 2008 test outline.
Next, an employer should use validation analyses through a variety of measures, including “ratings of test items, [and] linkages between test content and job tasks or KSAs.” Campion Report 24. HRD conducted a robust validation analysis in 1991 after administering the 1991 exam. As part of this process, HRD asked SMEs to again link tasks with KSAs, link tasks with examination subjects, and link KSAs with examination subjects, and evaluate the manner in which each was covered on the 1991 written test. 1991 Validation Report at 00350. While the process was not as robust in 2008, it does appear as though the SMEs reviewed the test questions, identified which KSAs matched
[64]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 65 of 82
the questions,33 and evaluated the questions for difficulty, readability, and recommendation for use. Ex. 60. The City has met its burden on this requirement.
The next step is for the test developer to “conduct[] various statistical analyses to ensure the quality of the test scores,” such as item analyses, reliability analyses, and adverse impact analyses. Campion Report 24-25. The record does not present evidence that HRD conducted such analyses following the 2008 exam. Dr. Campion cites exclusively to the 1991 report to demonstrate that the City conducted the necessary analyses to ensure the quality of the test scores. Campion Report 25. Considering the updates to the tasks and KSAs in 2000, 2005 and 2008, and the lack of evidence in the record that the 2008 exam was highly similar to the 1991 exam, the Court finds this citation insufficient. The Court thus finds that the City failed to conduct statistical analyses to ensure the quality of the test scores for the 2008 exam.
Lastly, the test developer must recommend proper administration of the test, including cut-off scores, rankings,
33 The Court notes that it is very difficult to tell from the exhibits exactly what the SMEs did in 2008 to support validity. Dr. Wiesen complained that HRD insufficiently explained how it prepared the 2008 exam. Wiesen Rebuttal 18. This lack of explanation, he asserted, flew in the face of published professional standards for testing. Id. The City would do well to improve its procedures for documenting its validation processes.
[65]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 66 of 82
bands, and weighting. Campion Report 25. There was no indication in the record that HRD analyzed any of these things in 2008. Dr. Campion, again, points to the 1991 report to support his conclusion that HRD fulfilled its obligation to properly construct the test. Id. Again, the Court is troubled by this because the 2008 and 1991 exams were different.
Further, the Court is troubled by the substance of the 1991 report on these points. For instance, to explain its decision in 1991 to weight the written portion of the exam at 80% and the E&E at 20%, HRD looked to the 1985 and 1987 sergeant promotional exams, for which SMEs recommended weights for the various components. 1991 Validation Report at 00349. Those exams included an interactive component in addition to the written component. For the 1991 exam, which did not include the interactive component, HRD chose to weight the written portion at 80%, stating that “if the same SMEs were asked to split the full examination between only two components [as opposed to three components] . . . the education and experience component would receive more weight than previously.” See 1991 Validation Report at 00349. This explanation cannot alone support the City’s decision, seventeen years later, to weight the written portion of the exam at 80%. The Court also fails to see how the 1991 report supports the 2008 cut-off score of 70. The Uniform Guidelines state that “[w]here cutoff scores are used, they
[66]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 67 of 82
should normally be set so as to be reasonable and consistent
with normal expectations of acceptable proficiency within the
work force.” 29 C.F.R. § 1607.5(H). The only criteria HRD
seemed to use in 1991 to set a cut-off score was adverse impact;
there is no mention of proficiency. 1991 Validation Report at
00381. The City has not explained why it chose 70 as a cut-off
score for the 2008 exam.
Considering all of these factors,34 the Court finds that the
test construction process was inadequate to support the
heightened validity requirement necessary to rank candidates.35
34 Dr. Campion also argued that, when developing a test, “[t]he test developer should evaluate alternative test methods that would most validly measure the KSAs needed at time of hire; if this is in the employment context, this will also usually consider the potential adverse impact of the test method.” Campion Report 23. There is indeed no evidence in the record that the City in 2008 evaluated alternative test methods. In defense of the City, Dr. Campion cites to HRD’s conclusion in the 1991 report that “(1) job knowledge tests have been shown by the research to be generally valid for a very wide variety of jobs, (2) reading and writing are required on the job so a written test is a reasonable format, and (3) a multiple-choice format is feasible to administer to a large applicant population.” Campion Report 23. If the City did in fact have such an obligation to consider alternatives in 2008 (an issue the Court need not decide, as its prior analysis of factors is sufficient to decide the issue), the City’s use of this prior decision, which did not explicitly consider relative validity, would not excuse that obligation.
35 For a discussion of the heightened standards for scores that rank candidates, see infra Part V-B-3-e.
[67]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 68 of 82
d. Content of the Exam
The Court, proceeding with its own validity analysis,36
continues to rely on expert testimony and the Uniform Guidelines
to structure its inquiry, which now shifts to the content of the
2008 exam.
The Uniform Guidelines state that “[a] selection procedure
can be supported by a content validity strategy to the extent
that it is a representative sample of the content of the job.”
29 C.F.R. § 1607.14(C)(1). If the selection procedure purports
to measure knowledge, skills, and abilities -– as is the case
here –- the employer “should show that (a) the selection
procedure measures and is a representative sample of that
knowledge, skill, or ability.” Id. § 1607.14(C)(4).
The Second Circuit has stated that the purpose of this
requirement is
to prevent either the use of some minor aspect of the job as the basis for the selection procedure or the needless elimination of some significant part of the job's requirements from the selection process entirely; . . . . Thus, it is reasonable to insist that the test measure important aspects of the job, at least those for which appropriate measurement is
36 The Court does so because the City did not conduct a formal validity study of the 2008 exam. The City cannot establish validity by relying on the 1991 validation study; the study was conducted seventeen years before the 2008 exam and experts generally agree that validity studies should be conducted every five to eight years. Bradley, 443 F. Supp. 2d at 172. Seventeen years “is too long a hiatus under the standards in the industry.” Id. Further, the 1991 validity study did not validate the 1991 exam for strict rank order.
[68]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 69 of 82
feasible, but not that it measure all aspects, regardless of significance, in their exact proportions. Guardians, 630 F.2d at 99. From the 1991 job analysis, DPA determined that 187 KSAs were necessary to carry out the tasks critical to performing the role of a Boston police lieutenant. 1991 Validation Report at 00257-58. Of these 187 KSAs, 108 were skills and abilities. 1991 Validation Report, App. EE. The 2000 report concluded that 145 KSAs were critical to the role. 2000 Job Analysis 49. Of these 145 critical KSAs, 91 were skills and abilities. Id. at 59-64. The Court finds that the thirteen knowledge categories were very broadly worded (much more so than the knowledge categories emerging from the 1990 and 2000 reports), and is therefore satisfied that the 2008 exam tested a sufficient range of the critical knowledge areas.37 Yet that is not enough to demonstrate content validity: the 2008 test outline indicated that HRD decided to test none of the critical skills, and to test only two abilities categories: the ability to read, understand, interpret, and explain material in written form, and
37 The test outline for the 2008 exam reflects that HRD decided to test only thirteen knowledge categories. Ex. 54; Ex. 60. The categories were, however, worded more broadly than those appearing on the 1991 and 2000 job analyses. In fact, Dr. Campion opined that the knowledge categories were so broad that roughly 80% of the knowledge areas from the 2000 report could fall under these thirteen categories. 01/05/15 Tr. 60:5-16.
[69]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 70 of 82
the ability to read and interpret documents such as maps and charts, and make basic arithmetical calculations. Ex. 54; Ex. 60.
The Court concludes that the 2008 exam did not sufficiently test for a representative sample of the critical KSAs. The job analyses reflect that many skills and abilities are necessary to perform the job of lieutenant, yet the 2008 written exam tested knowledge almost exclusively. Dr. Campion’s report does little to convince the Court otherwise. He stated that the examinations “are representative measures of the knowledge areas” (but no mention of the skills or abilities), Campion Report 10, and that the exam “measured a large number of knowledge areas,” (again, without mention to the skills and abilities), id. at 14. His report discussed at some length how well knowledge was tested, again, with no mention of the skills or abilities. Id. at 14-15. Dr. Campion in fact acknowledged during trial that there was no attempt to test for skills and abilities that may be important to a lieutenant, 12/22/14 Tr. 32:2-9. Despite these deficiencies, he opined that the work behaviors selected for measurement were important and constituted most of the job. Campion Report 13.
Because job knowledge is only a limited part of the job analyses for the role of lieutenant, the Court agrees with Dr. Wiesen that the 2008 exam skipped over critical skills and
[70]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 71 of 82
abilities, including interpersonal skills, presentation skills, reasoning and judgment skills, oral communication skills, analytical skills, ability to give constructive criticism, ability to speak in front of groups, ability to counsel subordinates, ability to counsel and comfort families of victims, and ability to make sound decisions quickly. Wiesen Rebuttal 28. The Court therefore also agrees with Dr. Wiesen’s opinion that, as a result of HRD’s decision not to test many critical skills, a high score on the 2008 exam simply was not a good indicator that a candidate would be a good lieutenant. Id. While the Court acknowledges that an exam need not test for every relevant KSA to be content valid, and gives credence to Commissioner Evans’ statement that “knowledge is power,” 01/07/15 Tr. at 20,38 the near total absence of the 2008 exam’s test of critical skills and abilities leads the Court to conclude that the City has not demonstrated validity.39 This is
38 Commissioner Evans also testified that in his experience, police officers who scored higher on the written exam were better performers. 01/07/15 Tr. 41:18-20. Such anecdotal accounts, however, “cannot substitute for actual evidence of validity.” United States v. City of New York, 637 F. Supp. 2d 77, 131 (E.D.N.Y. 2009). In any event, former Department Commissioners evidently placed less weight on the written exam. See supra notes 16, 31 (advocating for an assessment center).
39 The Court also notes that the City nowhere substantiated its use of 70 as a cut-off score. There should be an independent basis for choosing a cutoff score, such as a determination that the cut-off point separates those who can do the job from those who cannot, or a way of locating a logical
[71]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 72 of 82
consistent with Judge O’Toole’s opinion in Lopez in which he held that the written portion of the 2008 sergeants’ exam could not alone support validity “because it could not measure some skills and abilities (as distinguished from knowledge) essential to the position, such as leadership, decision-making, interpersonal relations, and the like.” Lopez, at 35-36;40 cf. Boston Police Superior Officers Fed’n v. Civil Serv. Comm’n, 35 Mass. App. Ct. 688, 695 (1993) (ruling that the Civil Service Commission properly concluded that the 1987 Boston lieutenants’ exam, which consisted of a multiple-choice written exam and an E&E component after other components were thrown out, “failed to constitute a fair test of supervisory skills and ability”).
e. Use of Exam Results to Rank Candidates The Court analyzes how the exam results were used because “evidence of both the validity and utility of a selection procedure should support the method the user chooses for operational use of the procedure, if that method of use has a
break-point in the pool of candidates. United States v. City of New York, 637 F. Supp. 2d 77, 124 (E.D.N.Y. 2009).
40 Dr. Campion points out that in the 2000 report, SMEs rated the extent to which all of the KSAs differentiated performers, and only those KSAs that differentiated between good and bad performers were used to develop the exam. Campion Report 15-16. The Court agrees with Dr. Campion that this would support the City’s ability to distinguish among candidates’ command of the requisite knowledge areas, but it does not address the utter lack of skills and abilities tested on the 2008 exam.
[72]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 73 of 82
greater adverse impact than another method of use.” 29 C.F.R. § 1607.5(G). Even were the Court able to find that the 2008 exam was sufficiently valid as a screening tool, the Court would still hold the 2008 exam invalid as the sole mechanism for ranking candidates.
The Uniform Guidelines impose a higher standard for finding validity when a test is used to rank candidates as compared to when a test is used simply as a screening tool, stating that “[e]vidence which may be sufficient to support the use of a selection procedure on a pass/fail (screening) basis may be insufficient to support the use of the same procedure on a ranking basis.” Id.; cf. Boston Chapter, N.A.A.C.P., Inc. v. Beecher, 504 F.2d 1017, 1026 (1st Cir. 1974) (“Even if the test is minimally valid, we might also doubt its use as an absolute cutoff. A test seemingly should receive no more weight in the selection process than its validity warrants. Use of a minimally valid test as an absolute cutoff is questionable even if more limited uses of the test are acceptable.”).
The Uniform Guidelines specifically allow employers to demonstrate validity for rank-order exams using content validity. To do so, the employer must show “by a job analysis or otherwise, that a higher score on a content valid selection procedure is likely to result in better job performance.” Id. § 1607.14(C)(9). Although the Court may infer a relationship
[73]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 74 of 82
between higher test performance and higher job performance, when the test scores reveal adverse impact that is greater at the higher end of the ranked list, “the appropriateness of inferring that higher scores closely correlated with better job performance must be closely scrutinized.” Guardians, 630 F.2d at 100 (citing to the Uniform Guidelines). The Court cannot find that the City has met this heightened validity standard. The City failed both to test a sufficient range of critical KSAs in the 2008 exam, and to produce evidence sufficient for the Court to conclude that the exam was valid.
Dr. Campion testified that the 2008 exam was sufficiently valid to use for rank ordering. He cited to the 2000 job analysis, in which SMEs, through the ranking process, concluded that the “more knowledge you have the better the lieutenant you could be.” 12/19/14 Tr. 47:8-18. He also testified that thousands of studies called meta-analyses revealed, “based on a statistical relationship,” that “higher scores [on knowledge tests] lead to higher job performance.” Id. at 47:20-48:5. Both arguments are unpersuasive.
First, the fact that the 2008 exam measured only knowledge areas that differentiate among performers cannot compensate for the 2008 exam’s failure to test for critical non-knowledge skills and abilities. It is possible that someone might excel on a portion of an exam testing knowledge, but tank on a portion
[74]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 75 of 82
of an exam testing skills and abilities, and vice versa. If a test only examines knowledge (even if limited to knowledge areas that differentiate performance) while ignoring a broad swath of necessary skills and abilities, it hardly seems plausible that a higher score is likely to result in higher job performance, or even that the procedure measures aspects that differentiate among levels of job performance. What the Court can conclude from the 2008 exam is that those who excelled at the exam would exhibit superior levels of knowledge on the job, and that the 2008 exam differentiated among levels of candidates’ knowledge levels. The Court agrees with Dr. Wiesen, however, that this is insufficient for predicting who will be a good police lieutenant. Wiesen Rebuttal 28.
Second, testimony as to the criterion validity of knowledge tests in general is insufficient to support the “refined use” of this exam as necessary for rank ordering. United States v. City of New York, 637 F. Supp. 2d 77, 131 (E.D.N.Y. 2009); see also Bradley, 443 F. Supp. 2d at 173 (noting that although cognitive ability is correlated with job performance, there was “no persuasive evidence in [the] record that the use of the written cognitive examination as the sole basis for rank ordering entrylevel firefighter candidates [was] a valid selection procedure”). In other words, Dr. Campion’s testimony on this
[75]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 76 of 82
point was far too general to support the City’s claim that the 2008 exam was valid enough to be used to rank candidates.
The Court also concludes that the City cannot use the 2008 exam to rank candidates because the City has failed to demonstrate that the 2008 exam was reliable. Reliability measures “the extent to which the exam would produce consistent results if applicants repeatedly took it or similar tests.” Guardians, 630 F.2d at 101. This is especially important when a test is used to rank candidates. 12/16/14 Tr. 75-76; 12/19/14 Tr. 18. Courts do not require perfect reliability, but “[w]ithout some substantial demonstration of reliability it is wholly unwarranted to make hiring decisions, with a disparate racial impact, for thousands of applicants that turn on onepoint distinctions among their passing grades.” Guardians, 630 F.2d at 101. One way of demonstrating reliability is by showing how skillfully the questions on the exam have been formulated, which can be done by twice giving a sample of an exam to the same population. Id. at 102. Employers can also use split-half correlation, observing how consistently an individual scores on each half of the test. Id. Employers can do such analyses before or after a test is administered. Id.
The only evidence the City presented on reliability was HRD’s determination in the 1991 report that the 1991 exam had a reliability rating of .79. 12/19/14 Tr. 42-44; 1991 Validation
[76]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 77 of 82
Report at 00382. Dr. Campion claimed that the reliability of the 2008 exam was likely comparable to the 1991 exam because the exams were highly similar. Campion Report 15. The 1991 validation report is insufficient to demonstrate reliability. In addition to its age, it was evaluating a different test. The City’s suggestion that the 2008 and 1991 exams were highly similar is inconsistent with the City’s reliance on the 2000 job analysis during the trial, and with the 2005 and 2008 mini-job analyses. The City seems to suggest that despite all of these updates to the 1991 report, the exam hardly changed at all. The Court does not find this argument persuasive, and concludes that the City has not demonstrated the reliability of the 2008 exam. Further undercutting a finding of reliability is the fact that Dr. Jacobs (of EB Jacobs), recommended in 2009 that the City “band” the results of the 2008 exam in nine-point increments because “a given candidate’s score will vary from one administration of a test to another.” Lopez, Exs. 70-71. The Court cannot find that the 2008 exam was sufficiently reliable for use as a ranking mechanism.
4. Conclusions Regarding Job Relatedness and Consistency with Business Necessity (Prong 2)
The City has failed to demonstrate that “a higher score [on the 2008 exam] is likely to result in better job performance.” 29 C.F.R. § 1607.14(C)(9). The state statute that the City
[77]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 78 of 82
argues requires civil service employers to promote in strict rank order cannot serve as an affirmative defense for the City: Title VII relieves employers of state hiring requirements “which purport[] to require or permit” any discriminatory practice. Guardians, 630 F.2d at 104-05 (quoting 42 U.S.C. § 2000e-7); Bridgeport Guardians, Inc., 933 F.2d at 1148 (quoting 42 U.S.C. § 2000e-7); Vanguard Justice Soc., Inc. v. Hughes, 592 F. Supp. 245, 268 (D. Md. 1984) (citing Guardians, 630 F.2d at 104). Moreover, current Chief Judge Saris noted in Bradley that it was not clear that the statutory scheme prohibits banding.41 443 F. Supp. 2d at 174 (noting that “[w]hile the attorneys have not briefed the issue, banding based on scores that have no statistical difference to diminish the adverse impact of a rankorder system seems consistent with the statutory scheme and applicable caselaw under Title VII”).
41 The Commonwealth attempted to use band scoring in 2009. 01/06/15 Tr. 98:16-99:7. HRD declined to band because of a challenge by the Civil Service Commission. Id. at 99:18-100:2. In that same year, a Massachusetts Superior Court issued a preliminary injunction enjoining the Department from banding scores, pending HRD utilizing its rule-making authority to establish the banding process. See Mem. Decision Order Pls.’s Mot. Prelim. Inj. Order, Pratt v. Dietl, No. SUCV2009-1254 (Suffolk Sup. Ct. April 15, 2009), ECF No. 7. The Superior Court decision does not justify the City’s rank ordering of candidates based on the 2008 exam; the ruling was a preliminary injunction, not a final judgment on the merits, and, in any event, it cannot override the requirements of Title VII.
[78]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 79 of 82
This Court holds that even were the 2008 exam valid enough
to be used as a screening tool,42 the City has failed to meet its
burden of showing that the 2008 exam was sufficiently valid to
be used as a basis for ranking candidates. The City has
42 Judge O’Toole held in Lopez that the 2005 and 2008 sergeant exams were “minimally valid” and the City had therefore met its burden of demonstrating that the 2005 and 2008 sergeant promotional exams were “job related” and “consistent with business necessity.” Lopez, at 35-36 (internal quotation marks omitted). Considering the substantial overlap between the questions appearing on the 2008 sergeants’ exam and those appearing on the 2008 lieutenants’ exam (sixty-eight questions in common), Wiesen Rebuttal 8, a decent respect for the views of a colleague compels this Court to comment on the divergent holdings.
Judge O’Toole’s findings are not so different from this Court’s. Agreeing with the City’s expert, Judge O’Toole held that, but for the E&E component, the 2005 and 2008 sergeants’ exams would not have passed muster under Title VII. Lopez, at 35-36. This Court arrives at a different holding in large part because the Court does not place as much stock in the E&E component as did Judge O’Toole. First, as discussed above, the automatic award of fourteen of the possible twenty points on this component reduces its usefulness. Second, the City bears the burden of establishing the validity of the E&E component, meaning it must establish a relationship between the content of the training/experience that is rewarded, and the content of the job. 29 C.F.R. § 1607.14(C)(6). The Court could possibly infer such a relationship but, as the case law indicates, this is an area for which the Court looks to expertise. Dr. Wiesen points out, and this Court agrees, that the City has presented no evidence on this point. Wiesen Rebuttal 63. While SMEs did assist DPA in evaluating and developing the E&E component, there is no evidence linking the E&E inputs with tasks or KSAs from the job analyses. Lopez, Ex. 41 Apps. W and X. Lastly, E&E scores were calculated only for candidates who passed the exam. 12/15/14 Tr. 62:24-63:2. There was disparate impact in the pass rates for the 2008 exam. In light of Title VII’s purpose, the Court is skeptical that a component of the test that was disproportionately unavailable to minority candidates can rescue an otherwise invalid exam.
[79]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 80 of 82
therefore failed to meet its burden on the second prong of the
legal framework: it has not convinced the Court that the 2008
exam was “job related for the position in question and
consistent with business necessity.” Jones, 752 F.3d at 54
(internal citation omitted).
The Court need not proceed to step three of the disparate
impact analysis: the Plaintiffs have won their case.43
43 The City’s post-trial submission argued that language in a recent Supreme Court case changed the usual disparate inquiry, imposing a new rule: “a Court cannot reject a governmental defendant’s Prong 2 justification if a plaintiff has not . . . presented enough evidence to meet its Prong 3 burden[.]” Def. City Boston’s Supp. Post-trial Filing 5-6, ECF No. 196 (citing Texas Dep't of Hous. & Cmty. Affairs v. Inclusive Cmtys. Project, Inc., 135 S. Ct. 2507, 2518 (2015)). Admittedly, the Supreme Court’s statement in dictum, read in isolation, does indeed appear to state that rule. See Texas Dep't of Hous. & Cmty. Affairs, 135 S. Ct. at 2518 (“[B]efore rejecting a business justification -- or, in the case of a governmental entity, an analogous public interest -- a court must determine that a plaintiff has shown that there is an available alternative practice that has less disparate impact and serves the entity's legitimate needs.”) (internal citation omitted).
The Court notes that the Supreme Court majority favorably cited Title VII precedent and made no indication that it was changing a decades-old three-prong doctrine. See, e.g., id. (noting “cases interpreting Title VII . . . provide essential background” in construing other federal antidiscrimination statutes). More importantly, although it is far from obvious, the best reading of its opinion is that the Supreme Court was actually discussing the proper application of Prong 3 when it made that statement.
The Supreme Court affirmed the Fifth Circuit’s reversal of the trial court. See id. at 2526. The Fifth Circuit had held that the trial court had conflated prongs 2 and 3 by placing the burden on the defendant to “prove that there are no other less discriminatory alternatives to advancing their proffered interests[.]” Id. at 2514 (internal citation omitted). Thus the Court’s statement can be interpreted as merely reiterating
[80]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 81 of 82
VI. CONCLUSION
Judgment shall enter in favor of the Plaintiffs regarding
liability under Title VII and Chapter 151B for the 2008
lieutenant promotional exam. By order dated November 10, 2014,
the Court bifurcated this trial into a liability phase and a
damages phase. The first part is complete.
The Court and the parties must now proceed to the remedial
phase of the lawsuit. The Court invites the parties to reach an
agreement concerning an appropriate remedy. If they cannot, the
Plaintiffs shall propose a remedy within thirty days of the date
of this order, and the Defendants shall respond within thirty
days thereafter.
that, if the defendant has met its burden on prong 2, the plaintiff still bears the burden on prong 3. Cf. Abril-Rivera v. Johnson, 795 F.3d 245, 253-54, 255-56 (1st Cir. 2015) (stating the familiar three-prong burden-shifting framework for disparate impact cases; discussing Texas Dep't of Hous. & Cmty. Affairs without revisiting the framework).
That is not to say the City’s argument lacks force. In fact, it might actually ensure Title VII litigation helps counteract unjustifiable disparate impact, instead of merely negating it. Forcing Title VII plaintiffs to identify a workable alternative that would achieve more equal opportunity before invalidating an employer’s practices under prong 2 would ensure that an employer’s impermissible employment practice is replaced by one that is just as effective but which offers more equal opportunity. But under the current scheme, the plaintiff bears no burden of demonstrating the existence of equally effective alternative practices unless the defendant has successfully met the requirements of prong two.
[81]

Case 1:12-cv-10291-WGY Document 199 Filed 11/16/15 Page 82 of 82

SO ORDERED.

/s/ William G. Young WILLIAM G. YOUNG U.S. DISTRICT JUDGE

[82]

 
 Case 1:12-cv-10291-WGY Document 228 Filed 07/09/16 Page 1 of 2

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

BRUCE SMITH, PAUL JOSEPH, MARTIN JOSEPH, KIM GADDY, BRIAN KEITH LATSON, MARWAN MOSS, LEIGHTON FACEY, and LATEISHA ADAMS,
Plaintiffs,

Civil Action No.: 12-1029I-WGY

CITY OF BOSTON, MASSACHUSETTS
Defendant.

Young, D.J.

AMENDED ORDER

The plaintiffs in this case alleged that the 2008 Boston police promotional examination

for lieutenants had a disparate impactracial discrimination on them and violated Title VII of the

Civil Rights Act of 1964 and M.G.L. c. 151B. In December 2014 and January 2015, the parties

engaged in a ten-day benchtrial and submitted exhaustive post-trial briefs on the issue of

liability under Title VII. On November 16, 2015, this Court ruled that the 2008 lieutenants'

examination had a raciallydisparate impactand was not sufficiently job-related to survive Title

VII scrutiny and issued an order of liability against the City of Boston. ECF No. 199.

Prior to a trial on damages, this Court administratively closed this case without prejudice

on February 1,2016, so that the parties could engage in Alternative Dispute Resolution

proceedings before the magistrate. ECF No. 212. The parties met for mediation on April 26,

2016 and were set to meet again on June 8, 2016. ECF No. 217.

Case 1:12-cv-10291-WGY Document 228 Filed 07/09/16 Page 2 of 2
Meanwhile, on May 18, 2016, the First Circuit Court of Appeals issued its opinion in the related case of Lopez v. City of Lawrence, et al.. Appeal No. 14-1952 and upheld the district court's findings that the 2005 and 2008 police promotional examinations for sergeants did not violate Title VII. As a result of the First Circuit's opinion, the parties were unable to make any progress on a mediated settlement and Magistrate Collings, who was overseeing the mediation, suggested that the parties seek interlocutory review of this Court's decision, in Smith, and notified this Court that he had terminated the mediation to facilitate interlocutory review of this Court's November 16, 2015 ruling on liability in the First Circuit. ECF No. 223.
On June 21, 2016, the parties filed a joint motion to permit interlocutory review of the Court's November 16, 2015 order and decision under 28 U.S.C. § 1292(b). ECF No. 224. On June 22,2016, this Court granted the parties joint motion to permit interlocutory review. ECF
No. 225.
Accordingly, the Court now amends its November 16, 2015 order and decision by certifying this matter for interlocutory review before the First Circuit pursuantto 28 U.S.C. § 1292(b). For the reasons stated in the parties' joint motion to permit interlocutory review, I am of the opinion that the issue for which interlocutory review is sought involves a controlling question of lawas to whichthere is substantial ground for difference of opinion and that an immediate appeal from this order as authorized by 28 U.S.C. § 1292(b) may materially advance the ultimate termination of the litigation. Proceedings in this case shall be stayed until the interlocutory appeal issue is resolved.
Hon. William GCroiml
United Stated Ijfstrict ^dge

 
 Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 1 of 30

@UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

___________________________________

)

BRUCE SMITH, PAUL JOSEPH,

)

MARTIN JOSEPH, KIM GADDY,

)

BRIAN KEITH LATSON, MARWAN )

MOSS, LEIGHTON FACEY, and

)

LATEISHA ADAMS,

)

)

Plaintiffs,

)

)

v.

)

)

CITY OF BOSTON, MASSACHUSETTS )

)

Defendant.

)

)

Civil Action No.: 12-10291-WGY

PLAINTIFFS’ POST-TRIAL BRIEF ON EQUALLY VALID BUT LESS DISCRIMINATORY ALTERNATIVES AND INCORPORATING FINDINGS OF
FACTAND CONCLUSIONS OF LAW

	   	  

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 2 of 30
TABLE OF CONTENTS I. INTRODUCTION................................................................................................................ 1 II. PLAINTIFFS’ PROPOSED FINDINGS OF FACT REGARDING EQUALLY VALID AND LESS DISCRIMINATORY ALTERNATIVES................................................................... 1
A. Equally or More Valid Alternatives. .................................................................................... 1 B. Use Of The Multiple Choice Test As A Hurdle/Qualifier, Not As A Strict Rank Order
Selection Method.................................................................................................................. 3 C. The Valid Scientific Evidence Demonstrates That The Use Of Alternative Selection
Procedures Has Less Adverse Impact On Minority Candidates. ......................................... 6 D. Alternative Types Of Selection Procedures Having Less Adverse Impact........................ 14
1. Performance review systems. .......................................................................................... 14 2. Structured Oral Interviews .............................................................................................. 16 3. Situational Judgment Tests.............................................................................................. 18 III. PROPOSED CONCLUSIONS OF LAW ON LESS DISCRIMINATORY ALTERNATIVES......................................................................................................................... 20 A. The City’s Attempted Anecdotal Evidence That Utilizing Multi-Components Does Not
Reduce Adverse Impact Is Not Based Upon Any Valid Evidentiary Showing. ................ 22 B. The Plaintiffs Have Demonstrated Equally Valid And Less Discriminatory Alternatives
That Could Have Been Used. ............................................................................................. 24 CONCLUSION............................................................................................................................. 26
ii

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 3 of 30
TABLE OF AUTHORITIES Cases
	  
Allen v. City of Chicago, 351 F.3d 306 (7th Cir. 2003) ............................................. 12, 15, 24, 26 Banos v City of Chicago, 2004 WL 626154 (N.D. Ill. 2004)......................................................... 5 Barnhill v City of Chicago, 2001 WL 243410 (N.D. Ill. 2001)...................................................... 5 Bradley v. City of Lynn, 443 F. Supp. 2d 145 (D.Mass. 2006) ……………………………………..21 Bryant v. City of Chicago, 200 F.3d 1092 (7th Cir. 2000) ............................................................ 22 Johnson v. City of Memphis, 770 F.3d 464 (2014) .......................................................... 12, 22, 24 Massachusetts Ass’n of Minority Law Enforcement Officers v. Abban, 434 Mass. 256, 748
N.E.2d 455 (2001)..................................................................................................................... 11
ii

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 4 of 30
I. INTRODUCTION
	  
The Plaintiffs submit their brief now on the issue of “equally valid and less discriminatory alternatives.” The Court granted the Plaintiffs additional time to submit this brief so that the Plaintiffs could conduct discovery on the Defendant’s recently-admitted evidence on the 2014 promotional exam. ECF No. 176. However, to date the Defendant has not produced any of the requested discovery, and therefore this brief is submitted without the benefit of the discovery that this Court granted the Plaintiffs the right to seek. The Defendant’s motion for a protective order on this discovery remains pending with this Court. ECF Nos. 181-82.
II. PLAINTIFFS’ PROPOSED FINDINGS OF FACT REGARDING EQUALLY VALID AND LESS DISCRIMINATORY ALTERNATIVES A. Equally or More Valid Alternatives.
1. In this case, and in Lopez, there was universal agreement by all experts that adding other components to a written multiple choice job knowledge test significantly increases – and never decreases – validity. Thus, every expert agreed that the adding of job-related components, such as a structured oral interview, or performance review, to the exam process increases validity significantly. (12/17/14, p. 80-83; 12/18/14, p. 89; 12/22/14, p. 16, 106; Lopez Tr. 6:114-15; 16:67; 17:102-105, 111-112, 120-21, 12930).
2. Dr. Campion testified that he helped develop close to 900 hiring and promotional examinations and used only a written job knowledge test only once, and that was an open book exam for trademark and patent examiners, because he freely admitted that adding components increases validity. (12/22/14, p. 16, 106).
1	  
	  

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 5 of 30
3. Defendant’s expert Dr. Silva testified in this trial, and in Lopez, that adding components to an exam typically reduces mean score differences and therefore adverse impact, while increasing validity. (12/18/14, p. 89; Lopez Tr. 17:102-105, 111-112, 120-21, 129-30). His firm, E.B. Jacobs, promotes the use of multi-component tests as leading to “reduced adverse impact” while “increasing validity.” (12/18/14, p. 89; Lopez Tr. 17:103-105).
4. Dr. Wiesen and Dr. Hough also testified that adding other components to a multiple choice job knowledge test significantly and dramatically increases validity, particularly since there is relatively low validity, if any, between a multiple choice job knowledge test and performance as a police sergeant or lieutenant.(12/17/14, p. 80-83; 1/6/15, p. 37-38, 67-68). Dr. Outtz testified to the same effect in Lopez. (Lopez Tr. 16:67)
5. The reasons that adding job-related components to a promotional process increases validity, are well documented in the scientific literature. Studies in peer-reviewed journals show that validity increases when an additional component is added, because when one tests for skills and abilities of the job that the written M/C exam cannot assess validity significantly increases. (Ex. 82 at Table 1; 12/17/14, p. 80-81; 12/22/14, p. 105-06; Lopez Tr. 6:114-15 (Dr. Fields); 15:104-106 (Dr. Outtz); 17:111112 (Dr. Silva)).
6. Indeed, several reputable studies have documented a direct relationship between increased validity and the addition of job-related components, such as the components which plaintiffs proffer in this case. (Ex. 82; Ex. L).
7. Accordingly, on this record, there is simply no dispute that if the Defendants had added additional components to the promotional process for lieutenant, there would have
2

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 6 of 30

been a direct proportional increase in validity, and therefore Plaintiffs would satisfy that part of Prong III which requires them to show that alternative procedures would result in an “equally valid” testing process. 8. Additionally, with respect to the multiple choice job knowledge test at issue in this case, the City, Dr. Silva’s firm, E.B. Jacobs, and Dr. Outtz all determined that the multiple choice test should have been “banded” within at least a nine-point spread because there was no way to tell a better-performing candidate. (12/16/14, p. 69; 1/6/15, p. 99; Lopez Tr. 15:59-62; 16:13-14; Lopez Exs. 70-73). Therefore the Defendant must concede that the use of additional components along with at least the banding of the written exam scores rather than strict rank order, was an equally (and likely more valid) valid method of selecting police Lieutenants.

	  
9.

B. Use Of The Multiple Choice Test As A Hurdle/Qualifier, Not As A Strict Rank Order Selection Method.
As stated earlier, all of the individuals competing for the position of lieutenant would

have already taken and successfully passed (with a reasonably high score) a previous

multiple choice job knowledge test, in which 80% of the questions were questions

deemed appropriate for the lieutenants’ exam and utilized for that exam. (1/7/15, p. 37;

Ex. 8; Lopez Ex. 45).

10. Since all plaintiffs had already been promoted to the position of sergeant, each would

have previously scored high enough on that multiple choice exam to have been reached

for selection to the position of sergeant. In addition, all have worked as sergeants for at

least two years or more and would have learned on the job many of the job knowledges

necessary for promotion. (12/15/14, p. 50-51; Ex. 22)

3

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 7 of 30
11. In addition, all of the officers who took the lieutenants’ exam would have also previously scored relatively high (in the 90’s) on the BPD entry-level written cognitive ability test. (1/6/15, p. 29-30).
12. Thus, all of the lieutenant candidates had adequate job knowledge coming into the lieutenants’ exam. Moreover, assuming that the pass/fail score was appropriately set, all individuals passing the lieutenants’ exam would have demonstrated reasonably proficient job knowledge sufficient to qualify as a police lieutenant.
13. Significantly, no criterion validity study was ever conducted by BPD or its experts to demonstrate that there was any correlation between higher performance on the multiple choice job knowledge test, and performance on the job. Conversely, Dr. Hough testified that in a study she co-authored for the LEAA in the late 1970’s, their criterion validity study demonstrated that there was very little correlation between scores on a written job knowledge test and performance as a police officer. (1/6/15, p. 11-15, 2830). This was confirmed by several studies from reputable journals used to cross examine Dr. Campion. (1/5/15, p. 47-50).
14. Many other cities have used a M/C job knowledge test as a pass/fail device, not as a ranking device, and E.B. Jacobs does this on many occasions, as did Dr. Cassie Fields in her exams, and Dr. Wiesen testified, along with Dr. Outtz and Dr. Campion that this is common. (Lopez Tr. 3:62-63; 6:63-64;17:93-94; 18:14-15, 62-63). See also Vanguard Justice Soc., Inc. v. Hughes, 592 F. Supp. 245, 248 (D. Md. 1984) (Baltimore); Isabel v. City of Memphis, 404 F.3d 404, 408 (6th Cir. 2005). Indeed for the last twenty years the City of Chicago has successfully used the written M/C test as a qualifying, not ranked component, with each candidate who achieved a qualifying
4

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 8 of 30
score being permitted to proceed in the process and then be ranked based on other components, including a merit and assessment component. Barnhill v City of Chicago, 2001 WL 243410 *1 (N.D. Ill. 2001); Banos v City of Chicago, 2004 WL 626154 *1(N.D. Ill. 2004). Indeed, the EEOC Guidelines suggest that such written tests, which are not well documented to show validity, should be used in this fashion. 29 C.F.R. § 1607.5, 1607.14. 15. In addition, Dr. Outtz, Dr. Silva, as well as Dr. Wiesen, testified that they frequently use written multiple choice job knowledge tests as a qualifier for promotional exam, not as a scored device. (12/16/14, p. 86-88; 12/19/14, p. 119; Lopez Tr. 18:13-15)). 16. Further, as described above, since all of the individuals taking the Lieutenants’ exam have already shown proficiency in a job knowledge test when becoming Sergeant, and 80% of the questions were the same as on the Lieutenants’ exam, it would be reasonable to only use the written multiple choice test, that is known to have such high adverse impact, as a qualifier to establish the pool of candidates. 17. After establishing the pool, selections could be made utilizing a combination of performance review, structured oral interview, and other type assessments. The scientific literature establishes that it is almost certain that this will reduce adverse impact while not impairing validity at all, because by adding these additional components, the test will be more job-related than only one component (i.e. the written test), and since the written test is a non-scored component, any small decrease in validity because multiple choice test results are not being used in rank order fashion, will be far outweighed by the increase in validity by testing for other attributes important to a police lieutenant, but not testable by a written exam.
5

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 9 of 30
18. Moreover, the practice of banding is similar to the use of a written test as a hurdle as it allows police department to use other factors beyond written test scores to rank officers within the band.
19. Even the Defendant’s own experts agree that within certain point spreads it is impossible to discern any difference in performance. Dr. Outtz testified that candidates should have been ranked in bands, because the test scores did not have sufficient reliability to use strict ranking. (15:59-62; 16:13-14). Dr. Jacobs, meanwhile, instructed the City to band the 2008 results. (Lopez Ex. 70). Thus, the City could have banded the scores allowing many more minorities to be considered for each round of hiring, and other components such as performance review could have been used to pick the better candidates. There is significant evidence that such a system would have less adverse impact.
20. Dr. Outtz testified that the reduction of adverse impact when banding can be improved by adding other selection criteria within the band by, for example, including a special interview process for the candidates within the band and by using prior performance in the department to break ties. (Lopez Tr. 15:19-20).
21. Dr. Fields also testified that adverse impact could be reduced through the use of banding. (Lopez Tr. 6:111).
C. The Valid Scientific Evidence Demonstrates That The Use Of Alternative Selection Procedures Has Less Adverse Impact On Minority Candidates.
22. All of the experts who testified in this case agree that there are a significant number of studies, social science papers, and validity reports which have carefully studied the question of whether certain selection techniques have more or less adverse impact on
6

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 10 of 30
minority candidates. These studies are based upon sound statistically significant scientific evidence. 23. There is no question that the consensus among industrial psychiatrists is that mean score difference between non-minorities and minorities is much higher on multiplechoice tests than on alternative testing components such as assessment centers, structures interviews, and oral boards. (12/16/14, p. 83). 24. Dr. Wiesen testified that alternatives such as structured interviews and assessment centers had from one-third up to one-half less adverse impact on minorities. (12/17/14, p. 84). 25. Dr. Fields testified that she had successfully reduced adverse impact in most of the examinations she developed which utilized multiple components. (Lopez Tr. 6:114) 26. Likewise, Dr. Outtz admitted that in the two places where he has devised police promotional exams using multiple components in the last 20 years – Detroit and Bridgeport – he has successfully reduced adverse impact. (Lopez Tr. 15:104-106). 27. Dr. Hough testified that she has devoted her whole career of 40 years to the “development and evaluation of personnel selection in promotion systems that have criterion- related validity with reduced or less adverse impact against minorities or protected groups.” 1/6/15, p. 16. She testified that she has developed promotional systems for, among others, Nynex, Microsoft, the Library of Congress, and the Federal Trade Commission and was successful in reducing adverse impact on minorities in each instance. (1/6/15, p. 17). 28. Dr. Wiesen testified about a chapter in a 2009 book called “Adverse Impact” which was written by Neal Schmitt and edited by Dr. Outtz which found that when
7

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 11 of 30
researchers gave a multiple-choice math test and then an open-ended math test, the D, or score difference, on Latinos was reduced by .66. ((Lopez Tr. 3:53-54). 29. Dr. Outtz testified in Lopez that the general expectation among industrial psychologists is that the use of an assessment center will reduce adverse impact on minorities. (Lopez Tr. 14:103). 30. Even when testing only for job knowledge, adverse impact can be reduced by using testing modes other than written, multiple-choice tests. (Lopez Tr. 3:51-55; 6:45-49). 31. Plaintiffs’ expert Dr. Fields, who has developed more police promotional exams than the other experts in both Lopez and this case (Lopez Tr. 6:30, 34), testified that she has used well-developed multi-component testing to increase validity and decrease adverse impact in most jurisdictions where she has worked. (Lopez Tr. 6:114-15). 32. Dr. Fields testified that adverse impact could be reduced further through the use of weighting. Specifically, Dr. Fields testified that industrial psychologists had shown that overall adverse impact could be reliably reduced by giving less weight to components with the most adverse impact. (Lopez. Tr. 6:83) 33. According to a presentation given by Dr. Rick Jacobs to the International Personnel Management Association Assessment Council in 2000, Dr. Jacobs was able to reduce adverse impact in police examinations by adding testing components beyond a written cognitive ability test. 1/5/15, p. 40-43. 34. Dr. Outtz, in an article titled “The Role of Cognitive Ability Tests in Employment Selection” concluded: “Given that (a) cognitive ability tests can be combined with other predictors such that adverse impact is reduced while overall validity is increased; and (b) alternative predictors with less adverse impact can produce validity coefficients
8

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 12 of 30
comparable to those obtained with cognitive ability tests alone, sole reliance on cognitive ability tests when alternatives are available is unwarranted.” (Lopez Tr. 16:63-64). Dr. Outtz confirmed that opinion at the Lopez trial. (Lopez Tr. 16:64). 35. Adding components to an exam – i.e., expanding the range of critical KSA’s that are tested—tends to reduce adverse impact without any sacrifice to validity. (3:45-48; 16:67-68, 74-76; 17:129-30). More specifically, the evidence establishes that test procedures such as assessment centers, structured interviews, and video situational judgment tests tend to result in mean score differences that range from 0 to 0.55, while the mean score differences on multiple-choice job-knowledge tests range from 0.45 to over 1.0. (Lopez Tr. 3:57-60; 6:43-44; 16:80-81). 36. Dr. Outtz wrote an article concluding that there does not have to be a tradeoff between reducing adverse impact and increasing validity and that tests can achieve both goals at the same time by increasing the components and the dimensions tested for the job. (Lopez Tr. 16:67) 37. Dr. Outtz described what a meta-analysis is in Lopez, stating: A meta-analysis is an accumulation of all the existing research data on a particular topic that you can find, that’s been published in any reputable source. It is the aggregation of that data such as you can draw conclusions as to the true state of a given phenomenon that you're trying to study as opposed to relying on one or two individual studies from time to time in different journals. In the field of industrial organizational psychology we perform meta-analysis periodically to see where we are based on the accumulated data up to that point and then it's published. It also allows you to accurately make an estimate of the amount of error or variance associated with whatever phenomenon I am looking at. How far off am I? What's the range in which I could possibly be off in making an estimate, for example, of the potential adverse impact of an assessment center? So it’s quite a valuable tool. (Lopez Tr. 14:104).
9

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 13 of 30
38. Dr. Outtz testified that short-term memory tests have lower adverse impact. (Lopez Tr. 14:134)
39. In a 1997 article published in Personnel Psychology, the publication edited by Dr. Campion, by Dr. Paul Sackett and Dr. Jill Ellingson titled “The Effects of Forming Multi-Predictor Composites of Group Differences and Adverse Impact,” demonstrated that an exam with multiple components will reduce adverse impact if the components are properly correlated. (Ex. L; Lopez Tr. 15:7)
40. As Dr. Campion admitted, the meta-analysis performed by Schmidt and Hunter showed that adding components, such as assessment centers and work sample tests would increase the criterion-related validity of an exam that included a written test. (12/22/14, p. 106; Ex. 82)
41. Dr. Campion testified about a 1999 article by industrial psychologists Philip Bobko and Philip Roth published in The Journal of Personnel Psychology. (12/22/14, p. 129-30). The article included a meta-analysis that showed that the mean score difference between minorities and non-minorities on cognitive ability tests was 1.0 or, in other words, minorities scored, on average, one standard deviation worse than nonminorities. (12/22/14, p. 130).
42. The Bobko-Roth meta-analysis also found that the difference in job performance for minorities was only .45. (12/22/14, p. 133-34). Dr. Campion testified that this study showed, and all other studies have predicted, that the job performance of minorities is two-fold better than non-minorities based on their written test scores would indicate. (12/22/14, p. 134-35).
10

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 14 of 30
43. That is, as Dr. Campion agreed, “if you only use a multiple choice job knowledge test, you are going to see bigger differences in scores than you would expect to see if you actually looked at how such minorities were doing on the job at the Boston Police Department as lieutenants.” (12/22/14, p. 137).
44. In addition, there is evidence from Boston that non-multiple choice components have had significantly less adverse impact on minority candidates.
45. In 1985, Boston’s multi-component test had no adverse impact at the pass/fail level. (Ex. 71 (1991 Validation Report at p. 82); Lopez Tr. 8:100).
46. In addition, in 2002, the non-multiple choice assessment center components had significantly less adverse impact than the written exam, (Ex. 81) In fact the 2002 Lt.’s exam validity report authored by Dr. David Morris, concluded that there was no statistically significant adverse impact from the assessment center component. (12/22/14, p. 52; Ex. 81, p. 18, ¶ 5.1).)
47. Furthermore, Dr. Wiesen found that the D, or difference between minorities and nonminorities, on the 2002 written multiple choice exam was .55, while the D on the oral component was only .26. (Lopez Tr. 3:57). See also Exs. 80-81. In other words, the mean score difference between minorities and non-minorities was cut in half on the oral component. (Lopez Tr. 3:57-58).
48. In addition, even in 2014, the structured oral interview had significantly less adverse impact than the written test. (See Attachment to Callahan Affidavit).
49. Even without expert statistical analysis, it is easy to see that the oral component greatly reduced the adverse impact on minorities. The mean score for minorities on the 2014 written test (out of a possible high score of 36) was 25.58, while the mean score for
11

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 15 of 30
non-minorities was 28.15, for a mean score difference on the written test of 2.57 points. The mean score for minorities on the oral board (out of a possible 24) was 19.13, while the mean score for non-minorities was 20.19, for a mean score difference of 1.06. The mean score difference between minorities and non-minorities was, therefore, reduced by more than 50 percent on the oral board. 50. To demonstrate that the reduction in adverse impact is not simply due to the smaller maximum point total for the oral board, the reduction in adverse impact is clearly demonstrated when the scores for both components are converted to being out of 100 (by converting the scores into percentages of the total maximum score for each component). Out of 100, the mean score for minorities on the written test was 71.06, while the mean score for non-minorities on the written test was 78.19, for a difference of 7.13 points. Out of 100, the mean score for minorities on the oral board was 79.71, while the mean score for non-minorities was 84.13, for a mean score difference of 4.41. The mean score difference for the written test (7.13 points) was nearly two-times higher than the mean score difference on the oral board (4.41 points). 51. Further, a review of the briefs in Massachusetts Ass’n of Minority Law Enforcement Officers v. Abban, 434 Mass. 256, 262, 748 N.E.2d 455, 460 (2001), shows that in 1992, when the City of Boston used an exam that included a structured oral interview, there was a significant reduction in adverse impact. In fact, by the time that the City attempted to promote several minority candidates with scores of 84 over several white candidates with scores of 85, there had been no violation of the 4/5’s rule. Abban, Brief of Boston Police Patrolmen’s Association, 2000 WL 34607839, *7 (Mass.), at *7.
12

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 16 of 30
52. Many court cases have shown that the use of alternative testing procedures has significantly reduced adverse impact. For instance, Memphis achieved significant reduction in adverse impact in the 1990’s by the use of its multi-component exam, and Chicago significantly reduced adverse impact by making 30% of its promotions based on a highly structured merit system. Johnson v. City of Memphis, 770 F.3d 464 at 475 (2014); Allen v. City of Chicago, 351 F.3d 306, 310 (7th Cir. 2003)
53. Dr. Rick Jacobs, in a presentation to the International Personnel Management Association Assessment Council in 2000 on his experience administering a promotional examination for Alabama state troopers, demonstrated that he was able to reduce adverse impact by adding multiple test components beyond a written test for cognitive ability. (1/5/15, p. 41).
54. In 2007, Dr. Outtz gave a presentation in South Africa on how to reduce adverse impact in governmental and private agencies based on data relating to various promotional tests given for fire lieutenants. (Lopez Tr. 16:77-78, 80). Dr. Outtz stated that African-American candidates suffered substantial adverse impact (a D of 1.3) on the written multiple-choice exams for fire lieutenant. (Lopez Tr. 16:81). Furthermore, Dr. Outtz concluded that the adverse impact on report writing was only one half of the impact on the multiple-choice test. Id. Dr. Outtz further found that the adverse impact on situational judgment tests was only one-quarter of that for written multiple-choice tests where there is one correct answer. Id. He also found that video situational judgment tests further reduced adverse impact by one half. Id.
55. In sum, there is overwhelming evidence that the use of alternative promotional testing mechanisms, which deemphasize M/C job knowledge will better assess the real skills
13

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 17 of 30
and abilities necessary to be a successful Lieutenant, while significantly reducing adverse impact.
D. Alternative Types Of Selection Procedures Having Less Adverse Impact.
	  
56. There are a multitude of alternative selection procedures that have been shown to have less adverse impact while being predictive of good job performance as a police superior officer.
57. These include the following:
1. Performance review systems. 58. In 2002, the Boston Police Department decided to use a structured performance review
system that was designed by their expert consultant firm at the time, Morris & McDaniel. (1/6/15, p. 88-90). The Department subsequently abandoned the idea solely because the unions challenged it before the Civil Service Commission. When he did abandon it, Commissioner Evans wrote an impassioned letter describing how essential a performance review system was and why it would have benefited the Boston Police Department. The letter stated:
It was my intention to provide candidates an opportunity to receive credit for the work they do day in and day out. . . . I believe that the best indicator of future performance is past performance, that what you do when you come into work counts. The best supervisors cannot always be identified solely by their performance on the written test and an hour in an assessment center. I wanted to give candidates credit for the work that they do on a daily basis. Quite simply, it is fairer to officers to factor in what they do on the job when they come to work. (Lopez Ex. 194). 59. Commissioner Evans’ lament was borne out by Sergeant Bruce Smith, who testified in this case about winning the Shattuck Award for his community involvement, his growing up on the streets of Boston, all of the excellent work that he had been doing,
14

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 18 of 30
his commendations, working as acting Lieutenant, but was never promoted to lieutenant. (12/17/14, p. 94-99). 60. Indeed, Commissioner Davis, and even current Commissioner Evans, gave testimony fully supportive of performance review systems. Commissioner Edward Davis testified that he sought to include a structured review system that would take account of past performance, which would measure abilities that were not measured on a written test, such as the ability to communicate. (1/5/15, p. 113). Commissioner Davis testified both in Lopez and in this case that being able to use past performance was extremely important in picking the best candidates, that many minority candidates would benefit from such a process because they were excellent minority candidates who simply didn’t score well on a test. (1/5/15, p. 96-101, 108-09; Lopez Tr. 9:41-42). Commissioners Davis and Evans testified how they used performance review in selecting the 20 or so non-Civil Service Deputy and Assistant Deputy positions that they are allowed to select outside of the Civil Service system. (1/5/15, p. 96; 1/7/15, p. 12-13). Davis testified that approximately 40% of his appointments were minorities using this system, that he has had excellent results, and that he has chosen people that would not have necessarily scored well on a test, but who have performed great at their job as Deputy or Assistant Deputy, a position over and above a Lieutenant. (1/5/15, p. 96) 61. Even current Commissioner Evans praised the fact that he had appointed a minority police Sergeant William Gross to be his second-in-command – Superintendent-InChief of the entire Boston Police Department and he had performed the job very well,
15

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 19 of 30
even though he had not even taken a lieutenant’s promotional exam or had taken it and not done well enough to be promoted out of his sergeant’s position. (1/7/15, p. 67-70). 62. Further, in a series of cases in Chicago, Illinois, the Chicago Police Department had agreed to a process where 30% of all of their promotions were based upon a structured performance review system and that process had resulted in “no” adverse impact. Allen v. City of Chicago, 351 F.3d 306, 310 (7th Cir. 2003). 63. Dr. Hough testified at length that certain systems like a performance review system that look at the individual’s qualities, accomplishments, etc., are highly valid ways to make selections, and are known to have little or no adverse impact and are more valid than multiple choice tests. (1/6/15, p. 34-37).
2. Structured Oral Interviews 64. Another highly effective method of selecting promotional candidates is through the
“structured oral interview” which is a well-recognized technique in the literature. 65. As was explained by many of the experts, under a structured oral interview, the
questions are all arranged in advance and given to every candidate, the assessors and the experts debate what the appropriate answers would be, and the responses by the candidates, either by video or in person, are then evaluated based on a set group of standards. Further, as evidenced from the 2014 exam, structured oral interviews had significantly less adverse impact than did the written exam, and in 2002, the video scenarios/interviews were demonstrated to have no statistically significant adverse impact on lieutenant candidates. (Ex. 81, p. 18). 66. Significantly, Defendant’s expert, Dr. Campion, is one of the leading proponents of the validity of the highly structured oral interview, and, in 1988, authored an article titled
16

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 20 of 30
“Structured Interviewing” which was the landmark study calling for the use of structured interviews and which described what steps could be taken to provide jobrelated structure to an interview. (12/15/14, p. 134; 12/16/14, p. 81). And since then, Dr. Campion has continued to published extensively on the subject with articles including “The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature” published in Personnel Psychology in 2014 and “Are Highly Structured Job Interviews Resistant to Demographic Similarity Effects?” published in Personnel Psychology in 2010. (Ex. 73 at Ex. A). 67. Dr. Campion believes that a highly structured oral interview process has significant validity in selecting candidates, and he utilizes such a procedure in the attempt to reduce adverse impact. Indeed, in Dr. Campion’s 2010 article on structured interviews, he concluded that structured interviews could be conducted in a way that would reduce adverse impact that would be expected from racial similarities between interviewers and interviewees. (1/5/15, p. 40). 68. Dr. Wiesen, Dr. Outtz, and Dr. Fields all testified that these types of structured oral interviews are known to significantly reduce adverse impact when properly conducted. (Lopez Tr. 3:57-60; 6:43-44; 16:80-81). 69. The meta-analysis by Bobko and Roth found that structured interviews had a mean score difference of only .23. 12/22/14, p. 131. 70. Other studies have shown that, traditionally, the D on structured interviews is 0.1, much less than the D usually found on multiple-choice tests, which is closer to 1.0. Lopez Tr. 3:59- 60). Dr. Campion testified that an article by Neal Schmitt and Elaine Pulakos found that the D for tests for “verbal ability [such as cognitive ability test] was
17

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 21 of 30
1.03 while the D for structured interviews was 0.12 and the D for other measures, such as bio data and situational judgment tests had a D of 0.23. (1/5/15, p. 31, 33-34). 71. According to a 2009 article from the Human Resource Management Review by industrial psychologist Terese Macan, “Notably numerous meta analyses have revealed that structured interviews can display relatively high levels of validity without the adverse impact typically found in cognitive ability tests.” 1/5/15, p. 30. Dr. Campion agreed with this statement from the article. Id.
3. Situational Judgment Tests 72. Dr. Hough described situational judgment tests as “an assessment [in] which you
describe a situation . . . and then you ask the applicant to describe what they would do, what they should do, a variety of response options.” (1/6/15, p. 23). Dr. Hough testified that situational judgment tests result in less adverse impact where if you ask the candidates to “describe what they should do” rather than what “would” they do. Id. Dr. Hough said that the reduction in adverse impact from situational judgment tests was clear from both her personal experience administering promotional examinations and also found to be the case by meta-analyses conducted by other industrial psychologists. (1/6/ 15, p. 24). 73. Dr. Hough further testified that the scientific research showed that situational judgment tests have even less adverse impact where the scenario is presented by videotape. 1/6/15, p. 24-26. 74. In Lopez, Dr. Outtz agreed that video-based tests resulted in significant reductions in adverse impact. (Lopez Tr. 16:64-65).
18

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 22 of 30
75. Dr. Wiesen, meanwhile, testified that the Pulakos-Schmitt article found that, where a situational judgment test was presented in a paper-and-pencil test and then was presented by video, the D, or discriminatory impact, was reduced by .74, which Dr. Wiesen concluded was a “huge reduction.” (Lopez Tr. 3:53-54).
76. Dr. Outtz described how the video test could be administered: You can have the video and then have a person give an oral response to a panel as to how they would handle that particular situation, or you can have a video and have them choose between a number of strategies as to how they would handle that particular scene on the video, or you can have a video and have them speak into an audiotape saying what they would do, or you can have them watch the video and have a videotape of what they say they would do. (Lopez Tr. 16:65-66).
77. In another scholarly article by several respected industrial psychologists, including Britt De Soete, Filip Lievens, Janneke Oostrom, and Lena Westerveld, published in the International Journal of Selection and Assessment in 2013, concluded: “At a practical level the present findings combined with the predictive validity evidence found in previous studies suggest that the constructive response multimedia test may be a valuable alternative predictor in [diverse] applicant settings.” And they further concluded that “This might be particularly relevant for police force selections as previous studies have revealed rather low predictive validity coefficients for cognitive ability tests in police contexts.” 1/5/15, p. 48.
III. PROPOSED CONCLUSIONS OF LAW ON LESS DISCRIMINATORY ALTERNATIVES
	  
78. Even if somehow the court were to find the Lieutenants’ exam job-related and consistent with business necessity, even when used as a strict rank order device, the plaintiffs may still prevail by showing that they were equally valid and less 19

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 23 of 30
discriminatory alternatives available. Bradley v. City of Lynn, 443 F. Supp.2d at 156. The burden of proving equally valid less discriminatory alternatives is subject to the same standard of proof as in civil cases – plaintiffs must show by a preponderance of the evidence; meaning that it is more likely than not that there are equally valid selection processes that would have likely had lower levels of disparate impact. Id. at 175. Plaintiffs “have no obligation to provide the exact floorplan, or to have proposed the alternative before the exam was given.” Id. It is the plaintiffs' burden to prove that there was “another available method of evaluation which was equally valid and less discriminatory.” Bryant v. City of Chicago, 200 F.3d 1092, 1094 (7th Cir. 2000). The Plaintiffs are not required to have proposed the alternative, rather they need only show that “the alternative was available.” This has been interpreted as meaning that the Defendants knew or should have known of such alternatives. Johnson v. City of Memphis, 770 F.3d 464, 473 (6th Cir. 2014). It is undisputed that by 2008, all of the alternatives discussed herein were well known to social scientists, and had been carefully studied. 79. Preliminarily, the plaintiffs emphasize that when the City decided to administer the 2008 exam based upon a strict multiple choice job knowledge test, it knew with virtual certainty that the test would have severe disparate impact. Here and in Lopez, experts for both sides repeatedly made the point that a multiple-choice exam would result in the most adverse impact on minorities. nt. (12/15/14, pp.113-114; 12/18/14, pp.45-46, 70; Lopez Tr. 2:82-84; Lopez Tr. 3:43-47, 55, 60; Lopez Tr. 10:30; Lopez Tr. 16:59; Lopez Tr. 17:110). Dr. Outtz stated in writing that “the multiple-choice cognizability, or job-knowledge test, has the highest level of adverse impact.” (Lopez Tr. 3:44-46)
20

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 24 of 30
(emphasis added). Another expert emphasized, “[o]ne of the effects of using a written multiple-choice job-knowledge test is that you’re virtually assured of adverse impact.” (Lopez Tr. 3:47). Moreover, in Lopez, Boston actually conceded that the 2008 sergeant’s exam had adversely impacted minority candidates. The City knew, therefore, that, of all the various test designs that it could possibly utilize, a rank-order multiple-choice job knowledge test that accounted for virtually all of the exam score would have the highest and most severe adverse impact of any test and promotional process imaginable. 80. Thus, it is hard to understand how a court could conclude that there are no alternatives that would lessen the adverse impact of the promotional process, when the process used by the City was known in advance to have the highest level of adverse impact possible and rather low validity because it only tested for some job knowledge, not the critical skills and abilities, and attributes necessary to be a police lieutenant. 81. Because every expert that testified readily agreed that the addition of job-related components which test for additional skills and abilities always increases, not decreases, and because the multiple choice test has never been shown to have any significant correlation with job performance as a police superior officer, and does not test for most of the skills necessary for the job, Plaintiffs have clearly met their simple burden of demonstrating that the utilization of a multi-component promotional process would have been equally valid, if not more so. 82. Moreover, Plaintiffs have demonstrated that given the already documented proficiency of the candidate to obtain a high score on a job knowledge test when they became Sergeants, the fact of their performing on the job for many years (since that is a
21

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 25 of 30
prerequisite to sit for a lieutenants’ exam) and the “range restriction” that results from this fact, the Plaintiffs have demonstrated by a preponderance of the evidence that utilizing a written multiple choice examination as a hurdle/qualifying device, and not as a scored rank order device, if combined with utilizing other components that test for skills and abilities not testable by a written exam, would increase not decrease validity. At the very least it would have equal validity while reducing adverse impact.
A. The City’s Attempted Anecdotal Evidence That Utilizing Multi-Components Does Not Reduce Adverse Impact Is Not Based Upon Any Valid Evidentiary Showing.
83. Presumably, the City here will do attempt what it tried to do in Lopez and claim that because it had not obtained better results in 2002 and 2014 with respect to utilizing multi-components, the plaintiffs cannot demonstrate that alternative testing procedures would have less adverse impact. The problem with this argument is that it has been explicitly rejected by at least two circuit courts of appeal, based on evidentiary principles. In both Allen v. City of Chicago, 351 F.3d 306, 315-16 (7th Cir. 2003), and Johnson v. City of Memphis, 770 F.3d 464, 475 (2014), the courts held that the parties could not rely on the isolated results of a past examination process, in an effort to persuade the court that such evidence would predict future results for a test. Thus in both cases (Allen and Johnson), the plaintiffs attempted to demonstrate that their proposals would be “equally valid and less discriminatory” by relying on the past success those processes had had in prior tests in the Chicago and Memphis police departments respectively.
84. In Johnson, the officers proposed eliminating the written job skills test from the process so as to give full weight to merit review boards. The district court had accorded
22

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 26 of 30
“considerable significance” to the testing process used in 1996 that involved a job simulation which resulted in no adverse impact. The 1996 process was found to have “no adverse impact.” Id. at 475. But the Sixth Circuit held:
Yet as the Seventh Circuit explained in Allen – and we agree – past practice alone does not suffice…the past success of a specific testing process predicts but does not establish success in future applications. Id. at 475. 85. Here, the City of Boston will claim that because the results of the 2002 exam were not as they had hoped, they would have no reason to believe that in 2008 there were less discriminatory alternatives. This argument suffers from the same defect that the Second and Seventh Circuit rejected since anecdotal evidence of one exam constructed in 2002 is not evidence. Indeed, as plaintiffs have shown, the non-written components of the exam did in fact have dramatically less adverse impact. Indeed, the 2002 assessment center had no statistically significant adverse impact on minorities. (Ex. 81, p. 18). The same is also true of the 2014 exam. In 2014, the structured oral interview had significantly less adverse impact than the written exam. 86. Accordingly, the City cannot purport to demonstrate that there were no less discriminatory alternatives by relying on one prior instance and one future instance. It is simply anecdotal evidence and not having any evidentiary value. Rather, the Court must value empirical social science studies of the effect of various test components and their relationship to adverse impact. On this question, virtually every study out there, and discussed in Lopez and here, demonstrates that utilizing non-written components will decrease adverse impact. Further, since using the written test as a qualifying device has been endorsed by many of the experts in this case, and has not been shown
23

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 27 of 30
to decrease validity, a final testing process that would have these other less discriminatory components, while still keeping the written test as a qualifying device, is to a reasonable likelihood (indeed is almost certain), going to have less adverse impact.
B. The Plaintiffs Have Demonstrated Equally Valid And Less Discriminatory Alternatives That Could Have Been Used.
87. There was testimony in this case that even when there are components of a promotional process that have lesser adverse impact, if they are combined with scores from a written multiple choice test that has significant adverse impact, the final result may not result in a substantial improvement of the adverse impact selection ratio. Essentially that is because the known adverse impact effect of the written M/C test will not likely be substantially ameliorated as long as that written score is being used as a substantial component, when scored on a rank order basis. Indeed, the 2002 exam demonstrates this point as there is only a slight improvement in the selection ratio for minorities because the written test was still accounted for over 40% of the score, and had severe adverse impact. Likewise, in 2014, there was severe adverse impact in the written exam, and although the structured oral interview had significantly less adverse impact, it is apparent that it will not result in a substantially reduced adverse impact because of the heavy weight that the City continues to place on the written test.
88. The Plaintiffs are not arguing for the elimination of the written M/C test as was the case in Allen v. City of Chicago. While that might substantially reduce adverse impact, assuming the selections are made from another component known to have lesser adverse impact, it does not permit a court to conclude that the final result was “equally valid.” Allen v. City of Chicago, 351 F.3d at 313. 24

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 28 of 30
89. Rather, Plaintiffs assert that the obvious solution is to combine a written multiple choice job knowledge test – used as a hurdle/ qualifying test with an appropriately set cut off score and carefully constructed by qualified experts in such a way as to reduce adverse impact, and to utilize and score other important components that test for the supervisory skills and abilities necessary to be a good police lieutenant, like communication, situational judgment, ability to take command, etc., all of which cannot be tested by a written exam, but all of which can be tested by structured oral interviews, assessment centers and the like.
90. If the written multiple choice test had been a hurdle, not a rank order device, many minorities would have been in the promotion pool. There would be a huge increase in minorities who could now be considered for promotion. But when the written M/C test is used as a rank order device, minorities generally sit at the bottom of the written test rank order list and never get real consideration. If that written hurdle then allowed passing candidates to take additional components such as the assessment center or structured oral interview, performance review, and other similar types of tests, the result would clearly be an equally or significantly more valid test (since more attributes were being examined), without any decrease in validity by reason of the written test being used on a qualifying basis. This is because, as stated above, there is no real evidence that police lieutenants who score higher on a written test perform better on the job than others who simply are determined to have successfully passed such an exam. Moreover, since these are lieutenants who have already successfully passed the sergeants’ exam, they have already shown that competence. The non-written components would have significantly less adverse impact, as demonstrated by study
25

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 29 of 30
after study, and the inclusion of the additional components, and the different attributes they would test, ensures that there would be no less validity. 91. Based on the above, the plaintiffs have demonstrated that if in 2008 the City of Boston had used the written test as a hurdle, not a rank, and then combined it with welldesigned components that are known to have less adverse impact, and test for the interpersonal and organizational attributes needed for the job, they would be equally valid and less discriminatory.
CONCLUSION For these reasons, and the reasons stated in their earlier brief on adverse impact and validity, the Plaintiffs respectfully request that the Court issue findings of fact and rulings of law that the Defendant has engaged in disparate impact racial discrimination, in violation of Title VII and M.G.L. c. 151B.
Respectfully submitted, BRUCE SMITH, et al. By their attorneys,
/s/ Benjamin Weber Harold Lichten, BBO#549689 Benjamin Weber, BBO#673736 Lichten & Liss-Riordan, P.C. 729 Boylston Street, Suite 2000 Boston, MA 02114 (617) 994-5800 hlichten@llrlaw.com bweber@llrlaw.com
Stephen Churchill Fair Work, P.C. 192 South Street, Suite 450 Boston, MA 02111
26

Case 1:12-cv-10291-WGY Document 191 Filed 05/04/15 Page 30 of 30

DATED: May 4, 2015

(617) 607-3262 steve@fairworklaw.com

CERTIFICATE OF SERVICE
I certify that a copy of the foregoing document has been served on counsel for all parties through the ECF system on this date.

DATED: May 4, 2015

/s/ Benjamin Weber_____________ Benjamin Weber

27

 
 Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 1 of 8

UNITED STATES DISTRICT COURT FOR THE DISTRICT OF MASSACHUSETTS

)

BRUCE SMITH, PAUL JOSEPH,

)

JOHN M. JOHNSON, ROBERT TINKER,)

MARTIN JOSEPH, KIM GADDY,

)

BRIAN KEITH LATSON,

)

LEIGHTON FACEY and

)

MARWAN MOSS

)

)

Plaintiffs,

)

)

v.

)

)

CITY OF BOSTON, MASSACHUSETTS )

)

Defendant.

)

_ _ _ _ _ _ _ _ _ _ _ _)

Civil Action No.: · - - - - - -

COMPLAINT, COMPENSATORY, INJUNCTIVE AND DECLARATORY RELIEF REQUESTED

I.

INTRODUCTION

1. This is an action by minority police sergeants for the Boston Police Department

who have taken the 2005 and/or 2008 Boston Police Lieutenants' promotional exam and who

claim that such examinations are unlawful under Title VII of the Civil Rights Act of 1964

because they have a severe disparate impact on minority test takers and cannot be shown to be

job related under applicable federal Equal Employment Opportunity Commission guidelines.

The Plaintiffs seek appropriate injtmctive and compensatory relief as described herein.

II. JURISDICTION

2. This Court has jurisdiction over this case under 28 U.S.C §§ 1331 and 1343 in that

this case arises under the laws of the United States of America, including particularly, Title VII

of the Civil Rights Act of 1964, 42 U.S.C. §2000(e), as amended.

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 2 of 8
III. PARTIES 3. Plaintiff Bruce Smith is an adult resident of Randolph, Massachusetts. He has
been a Boston Police Officer since 1989 and a Police Sergeant since May of2005. In 2008 he took the Boston Police Lieutenants' promotional exam and received a score of 78.
4. Plaintiff Paul M. Joseph is an adult resident of Boston, Massachusetts and is a Sergeant on the Boston Police force. He has held this position since 1997, and he has been a uniformed member of the Boston Police Department since 1989. In 2005 he took the Boston Police Lieutenants' promotional exam and received a passing score of 70. In 2008 he took the promotional examination and received a score of 87.
5. Plaintiff John M. Johnson is 811 adult resident of Milton, Massachusetts. He is currently a Boston Police Serge811t, a position he has held since 1997. He has been a uniformed member of the Boston Police force since 1986. In 2005 he took the Boston Police Lieutenants' promotional exam and received a score of 86.
6. Plaintiff Robert Tinker is an adult resident of Weymouth, Massachusetts. He has been a Boston Police Sergeant since 1988, and has been a uniformed member of the Boston Police Dep81tment since 1979. He took the Boston Police Lieutenants' promotional exam in 2005 and received a score of 86.
7. Plaintiff Martin Joseph is an adult resident of Dorchester, Massachusetts. He is currently a sergeant on the Boston Police Dep81tment and has held this position since 2005. l-Ie has been a uniformed member of the police force since 1986. In 2008 he took the Boston Police Lieutenants' promotional exam and received a passing score of 77.
8. Plaintiff Kim Gaddy is 811 adult resident of Jamaica Plain, Massachusetts. She has been a police officer for the City of Boston since 1985, 811d she has been a sergeant since 1992.
2

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 3 of 8
Plaintiff Gaddy took the 2008 Boston Police Lieutenants' promotional exam and received a passing score in the 80's.
9. Plaintiff Brian Keith Latson is an adult resident of Shanon, Massachusetts. He has been a Boston Police officer since 1985 and a Boston Police Sergeant since 200 l. In 2008 he took the Boston Police Lieutenants' promotional exam and received a score of 76.
I0. Plaintiff Leighton Facey is an adult resident of Boston, Massachusetts. He began working as a Boston Police Officer in 1998 and has been a Sergeant since September of2005. In 2008 he took the Boston Police Lieutenants' promotional exam and received a score of 88.
II. Marwan Moss is an adult resident of Roslindale, Massachusetts. l-Ie has been a Police officer since 1995 and a Police Sergeant since April of2005. In 2008 he took the Boston Police Lieutenants' promotional exam and received a score of 87.
12. The Defendant, City of Boston, is a duly incorporated municipality of the Commonwealth of Massachusetts. It runs and operates a large urban Police Department, and it makes promotions in the Boston Police Department pursmmt to M.G.L. ch. 31, the state Civil Services Law. IV. CLASS ALLEGATIONS
13. The named Plaintiffs, all of whom are African American, bring this action individually and on behalf of a class of individuals similarly situated. The class of individuals similarly situated is as follows: All black police sergeants for the Boston Police Department, who have taken the 2005 and/or 2008 Boston Police Lieutenant Promotional Examination or who plan to take such exan1ination when next given.
14. The claims of the class plaintiffs are similar if not identical to the claims of the named plaintiffs in that each has challenged the validity ofthe Boston Police Lieutenants'
3

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 4 of 8
examination and promotional procedure and do so under the same legal theory- disparate impact.
15. The class meets all of the numerosity requirements of Rule 23 of the Federal Rules of Civil Procedure, as it would be impractical to name each individual plaintiff. In addition, the claims of the named plaintiffs are representative and typical of the claims of the class plaintiffs.
16. There are common questions of law and fact which predominate over any specific individual factual claims in this case, and plaintiffs' counsel are eminently qualified to represent the claims of the named and class plaintiffs. This class is primarily certifiable under Rule 23(b)(2) of the Federal Rules of Civil Procedure in that Plaintiffs seek declaratory and injunctive relief generally applicable to all plaintiffs. The plaintiffs also seek damages in the fonn of back pay and other damages allowable by Title VII of the Civil Rights Act of 1964, but those claims are m1cillary to the injunctive and declaratory relief sought by the plaintiffs.
17. This class should also be certified lillder Rule 23(b)(3) of the Federal Rules of Civil Procedure in that there are questions oflaw or fact which are common to the class which predominate over any questions affecting only individual members, and a class action is a far superior method of fairly and efficiently adjudicating this controversy. V. STATEMENT OF FACTS
18. All of the named plaintiffs are Afi'ican-American police sergeants for the City of
Boston Police Department, and each is well qualified for promotion to the position of Lieutenant on the Boston Police Department.
4

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 5 of 8
19. Each of the plaintiffs took either the 2005 or 2008 Boston Police Lieutenants' promotional examination, or both, and each received a passing score on said examination, but has not been promoted to the position of lieutenant.
20. The promotional process for becoming a lieutenant on the Boston Police Department is as follows: First, the Boston Police Department, in conjunction with the Commonwealth of Massachusetts, Human Resources Division announces that there will be a Boston Police Lieutenants' promotional exam consisting of approximately 100 multiple choice questions which are taken almost verbatim from texts and other written materials which relate to police work or supervision.
21. Each individual taking the examination receives a score based on the number of correct answers, scaled to 100 possible points. In order to be considered at all for a promotion, one must attain a passing score of70. The final score consists of the results of the written multiple choice exams (which are based upon rote memory and taken directly from texts), and several points may be added or subtracted based upon one's years of service and educational background, but no points are added for relevant forms of experience, including military service.
22. The final score is then used to rank each candidate in order of their score on the examination.
23. Promotions to the position of Boston Police Lieutenant are then made by the defendant in strict rank order based upon one's rank on the promotional list.
24. For at least 30 years, the Boston Police Department, as well as all experts in the field, have been aware that such multiple choice rote memory examinations have a severe disparate impact on minority test tal<ers, including African-Americans. In addition, the evidence is overwhelming that such examinations are not valid under appropriate federal and state EEOC
5

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 6 of 8
guidelines, in that they are not job related- or predictive of actual ability to do the job. Further, tl1ere are many and multiple less discriminatory testing procedures which are equally or more valid as predicting job abilities, and which have a far less discriminatory impact on minority candidates.
25. Because the City of Boston has predominately administered such multiple choice police promotional examinations for the position of lieutenant over tl1e last 25 years, the result is that currently despite the fact that more than 30% of the entire Boston Police Department is made up of minority police officers, iliere are only, on information and belief, 2 African-American Police Lieutenants on the entire Boston Police Department out of nearly 50 lieutenants, or less than 5% of the entire police lieutenants on ilie Boston Police Department. In addition as of March 1, 2012 the Boston Police Department will have no minority police captains.
26. In the fall of2005, the Boston Police Department conducted a lieutenants' promotional examination, which results were certified in fue early spring of 2006. Promotions from that list continued to be made tmtil early 2009. On information and belief, of all the lieutenant promotions made from that list, only one African-American was promoted.
27. In ilie fall of2008, the Boston Police Department administered another promotional exam for the position oflieutenant, which list became certified in early 2009. Although many lieutenant promotions have been made from that list between 2009 and the present, to date there have been no minority promotions from that list despite the fact that there are many African-American police sergeants who passed the exam and were well qualified for tl1e position.
28. Thus, as a direct result of the Boston Police Department's use of written multiple choice examinations, the number of Boston Police Lieutenants who are African-American is far
6

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 7 of 8
below the number of African-American Police Lieutenants that would be expected to be promoted had there been a fair lieutenants' promotional examination procedure that did not have such a severe disparate impact upon minority officers. VI. EXHAUSTION OF ADMINISTRATIVE REMEDIES
29. Each of the named plaintiffs have filed complaints of discrimination relating to their failure to be promoted to the position of Boston Police Lieutenant with the Massachusetts Commission Against Discrimination and the Equal Employment Opportunities Commission, and more than 180 days has elapsed since the filing of these complaints.
COUNT I 30. The conduct of the defendant, City of Boston, in conducting, administering, and utilizing a promotional examination system for the position oflieutenant in the Boston Police Department that has a severe disparate impact on minority candidates, cannot be shown to be job related and is not valid in accordance with EEOC guidelines is unlawful. In addition, there are many less discriminatory examination procedures which would have less discriminatory impact. Thus the Defendant's actions violate Title VII of the Civil Rights Act of 1964, 42 U.S.C. §2000(e). WHEREFORE, Plaintiffs request this Honorable Court to issue appropriate injunctive, declaratory, and compensatory relief which 1) requires the Boston Police Department to utilize a nondiscriminatory promotional examination procedure for the position of lieutenant, 2) grants injunctive and declaratory relief preventing defendants from using their current promotional procedure, 3) grants plaintiffs injunctive and compensatory relief including back pay and other damages, including reasonable attorneys' fees and costs, and 4) grants such other and further relief as this Court deems just and proper.
7

Case 1:12-cv-10291-WGY Document 1 Filed 02/14/12 Page 8 of 8

Respectfully submitted, PAUL JOSEPH, eta!. By their attorneys,

DATED: February 11/,2012

Harold L. Lichten, BBO # 549689 StephenS. Churchill, BBO # 564158 Sara Smolik, BBO # 661341 Lichten & Liss-Riordan, P.C. 100 Cambridge Street, 20th Floor Boston, MA 02114 617-994-5800

8

 
 Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 1 of 31

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

)

BRUCE SMITH, PAUL JOSEPH, JOHN M. )

JOHNSON, ROBERT TINKER, MARTIN

)

JOSEPH, KIM GADDY, BRIAN KEITH

)

LATSON, LEIGHTON FACEY, MARWAN

)

MOSS, and LATEISHA ADAMS,

)

)

Plaintiffs, )

)

v.

)

)

CITY OF BOSTON,

)

)

Defendant.

)

)

CIVIL ACTION NO. 12-10291-WGY

YOUNG, D.J.

July 26, 2017

MEMORANDUM & ORDER

I. INTRODUCTION

To understand why the Court here revisits and reconsiders

rulings it made earlier in Smith v. City of Boston, 144 F. Supp.

3d 177 (D. Mass. 2015), it is necessary to understand the timing

of my decision in Smith and how that decision may or may not

conform to two other related yet distinct decisions -- Judge

O’Toole’s thorough opinion in Lopez v. City of Lawrence (Lopez

I), No. 07-11693-GAO, 2014 U.S. Dist. LEXIS 124139 (D. Mass.

Sept. 5, 2014) (O’Toole, J.), and its affirmance by the First

Circuit, Lopez v. City of Lawrence (Lopez II), 823 F.3d 102 (1st

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 2 of 31
Cir. 2016), cert. denied, 137 S. Ct. 1088 (2017). The latter decision, of course, controls this Court’s analysis.
All three decisions seek accurately to apply the law of disparate impact. At the most superficial level, the jurisprudence of disparate impact seeks fairly to ensure that employment decisions are made on genuine merit.
Recognizing that all employment tests are, by their very nature, discriminatory (after all, that’s the whole purpose of testing -- to choose the few from the many), the plaintiffs must (first prong) prove that the test reveals a significantly disparate impact upon a lawfully protected minority -significantly disparate impact because we don’t want federal judges messing around with every employment test.
If the plaintiffs prove the first prong, the employer has the chance (second prong) to prove that the test vindicates itself through the business necessity of choosing on the basis of merit the best persons for the job.
Even if the employer prevails on the second prong, the plaintiffs get one last chance (third prong) -- to prove that there existed a test equal or better at identifying the best person for the job thus satisfying the employer’s business necessity, which test was available to the employer and which test had a less disparate impact.
[2]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 3 of 31
This is an elegant and nuanced matrix. The devil, of course, is in the details.
Lopez I, the first of these three related cases, commenced on September 11, 2007, with the filing of a complaint by a number of black and Hispanic patrolmen from various municipalities (including the City of Boston (“Boston”)) challenging the civil service examination procedures for promotion to the rank of sergeant (“2008 sergeants’ exam”). Drawn to Judge George O’Toole, this case came on for an eighteen-day bench trial commencing on July 12, 2010. When the trial concluded, Judge O’Toole took the case under advisement.
In February 2012, ten black police sergeants (the “Plaintiffs”) in Boston commenced a substantially similar case before Judge Joseph Tauro. This case, the Smith case, challenged the police promotional exam from sergeant to lieutenant. When Judge Tauro took senior status, the case was transferred to this session on December 26, 2013.
In the meantime with Lopez I under advisement and Smith pending, Boston substantially revamped its police promotional testing procedures, adopting -- at significant expense -- many of the improvements for which both the Lopez I and Smith plaintiffs were contending.
On September 5, 2014, Judge O’Toole issued his full written opinion in Lopez I, finding that the 2008 sergeants’ exam
[3]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 4 of 31
imposed a significantly disparate impact on minority applicants, 2014 U.S. Dist. LEXIS 124139, at *48, and that the written portion of that exam could not alone support its validity “because it could not measure some skills and abilities (as distinguished from knowledge) essential to the position, such as leadership, decision making, interpersonal relations, and the like,” id. at *60-61. Judge O’Toole went on to find that the Education and Experience portion of the examination saved it, albeit just barely. Id. The plaintiffs promptly appealed.
In Smith, the Plaintiffs alleged that the multiple-choice examination used by the Boston Police Department in 2008 to select and rank candidates for promotion from the rank of sergeant to lieutenant (“2008 lieutenants’ exam”) had a disparate impact on racial minorities and was invalid under Title VII of the Civil Rights Act of 1964. Smith, 144 F. Supp. 3d at 181. Boston responded that the exam did not have a disparate impact and, even if it did, was sufficiently jobrelated to be held valid. Id. at 180.
On December 15, 2014, at the outset of what proved to be a ten-day bench trial, the parties commendably moved into evidence the full trial record and exhibits from Lopez I. Then, for ten days, the Court heard lay and expert witnesses proffered by both sides, some of whom had not testified in Lopez I. See id. at 181. On November 26, 2015, this Court issued its opinion
[4]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 5 of 31
concluding that the 2008 lieutenants’ exam had a racially disparate impact and was insufficiently job-related to survive the Plaintiffs’ challenge. Id. at 180-81. The Court thus imposed liability on Boston. Id. at 181.
Before engaging in extensive hearings concerning remedy, all parties sought time to explore settlement. After all, the challenged 2008 lieutenants’ exam had long been out of use and the real nub of contention appeared to be the attorneys’ fees due the Plaintiffs’ counsel as prevailing parties.
Then, in a comprehensive opinion issued on May 18, 2016, the First Circuit affirmed Lopez I. Lopez II, 823 F.3d 102. As that court itself summarized: “[f]inding that the district court applied the correct rules of law and that its factual findings were not clearly erroneous, we affirm.” Id. at 107.
Naturally, I read Lopez II with great interest. I was gratified to see that the First Circuit had unanimously concluded, as did Judge O’Toole -- and as had I with respect to the 2008 lieutenants’ exam -- that the 2008 sergeants’ exam had a significantly disparate impact on racial minorities. Id. at 111. On the sole issue where I had parted company with Judge O’Toole -- finding on different and additional evidence that business necessity could not justify use of the 2008 lieutenants’ examination for the rank ordering of candidates for promotion -- the Court of Appeals had split 2-1 in reviewing
[5]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 6 of 31
Judge O’Toole’s findings as to the 2008 sergeants’ exam. Id. at 122 (Torruella, J., concurring and dissenting). Most important, I detected no shift in the governing law in Lopez II from that I had applied to the facts I found in Smith. Nor would any shift be expected. Absent intervening Supreme Court precedent or legislative change, it is the practice in the First Circuit faithfully to adhere to the decisions of earlier panels of that court. See, e.g., Peralta v. Holder, 567 F.3d 31, 35 (1st Cir. 2009) (“‘We have held, time and again, that in a multi-panel circuit, prior panel decisions are binding upon newly constituted panels in the absence of supervening authority sufficient to warrant disregard of established precedent.’” (quoting Muskat v. United States, 554 F.3d 183, 189 (1st Cir. 2009))).
Whatever the potential legal effect of Lopez II on Smith, its practical effect was immediate. Settlement negotiations ceased. Now the parties sought an interlocutory appeal to settle once and for all the propriety of this Court’s ruling on prong 2. This Court readily acceded to their wishes.
On October 11, 2016, the Court of Appeals gently but firmly rebuffed this gambit:
The district court issued its findings on liability in this case without the benefit of our subsequently-issued opinion in the case of Lopez v. City of Lawrence, 823 F.3d 102 (1st Cir. 2016). Since then, the district court has not yet purported to apply Lopez to the facts of this case.
[6]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 7 of 31
For example, it has not stated whether and how its assessment of validity has taken into consideration the guidance we provided. Id. at 116-17. We therefore deny the petition without prejudice to renewal, if otherwise appropriate, after the district court has itself applied Lopez to this case. J. United States Ct. Appeals 1, ECF No. 229. In one sense, this order is both generous and courteous. It gives me first crack at applying Lopez II to my earlier legal analysis in Smith and making such analytic adjustments as may be necessary. Its tenor, however, suggests I may have missed something. Boston certainly thinks so. In light of the First Circuit’s order, this Court promptly held a status conference with the parties, Electronic Clerk’s Notes, ECF No. 232, who subsequently briefed their positions on the effect of Lopez II on this Court’s previous ruling in Smith, Pls.’ Br. Ct. Appeal J., ECF No. 235; Pls.’ Reply Br. Regarding Ct. Appeals J., ECF No 241; City of Boston’s Br. Affect Lopez Ct.’s Liability Decision (“Def.’s Br.”), ECF No. 236; City of Boston’s Reply Br. Lopez’s Affect Ct.’s Liability Decision (“Def.’s Reply”), ECF No. 242. Boston argues that Lopez II requires this Court to change its previous ruling by: (1) applying different legal standards to its prong 2 analysis, thus necessitating a different outcome, and (2) reaching prong 3 of the disparate impact inquiry.
[7]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 8 of 31
II. THIS COURT’S PREVIOUS RULING
In these circumstances, this Court first conducts a brief,
albeit rigorous and reflective review of what it has already
done. In Smith, this Court examined the Plaintiffs’ challenges
to Boston’s use of the 2008 lieutenants’ exam to select and rank
candidates for promotion from the rank of sergeant to
lieutenant. 144 F. Supp. 3d at 180. The Court imposed
liability on Boston, after concluding that the 2008 lieutenants’
exam had a racially disparate impact and was insufficiently job-
related to withstand the Court’s disparate impact inquiry. Id.
at 181.
In examining the evidence, this Court set forth the legal
framework:
Under First Circuit case law, the plaintiff bears the burden of establishing a prima facie case of discrimination which consists of identification of an employment practice (in this case, the 2008 [lieutenants’] exam and promotions flowing therefrom), disparate impact, and causation.
If the Plaintiff meets this burden, the employer may either debunk the Plaintiff’s prima facie case, or alternatively, may demonstrate that the challenged practice is “job-related and consistent with business necessity.” If the employer demonstrates the latter, the ball bounces back into the plaintiff’s court to demonstrate that “some other practice, without a similarly undesirable side effect, was available and would have served the defendant’s legitimate interest equally well.”
. . . . . . . Under the first prong, the Plaintiffs must make a significant showing of actual disparate impact upon an identified protected minority . . . . If the plaintiff can, however, make this showing, then under the second prong, the employer gets a chance to
[8]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 9 of 31
demonstrate that the test in question is both job-related and consistent with business necessity. . . .
Even if the employer succeeds, however, the case is not over. Under the third prong, the plaintiff gets one more shot. If the plaintiff can demonstrate the availability of a testing program equally determinative of job performance, yet resulting in less disparate impact, the Court should fashion a remedy to secure the greatest degree of equal opportunity. Id. at 182-83 (citations omitted). The Court then issued its findings of fact, id. at 185-91, thoroughly discussing the role of a Boston Police Department Lieutenant, id. at 185, pre-2005 job analyses and validation studies, id. at 185-88, the development and administration of the 2008 lieutenants’ exam, id. at 188-89, the development and administration of the 2014 exam, id. at 189-90, and the results of the 2005 and 2008 lieutenants’ exams, id. at 190-91. Next the Court set forth its conclusions of law. Id. at 191-211. A. Prong 1 In its discussion of disparate impact (prong 1), id. at 191-200, the Court addressed the relevant data to consider in determining whether the Plaintiffs showed a significant disparate impact, id. at 192-94, whether or not to aggregate the data, id. at 194-95, and whether to use a one-tailed or a twotailed test of statistical significance, id. at 195-98. The Court concluded that it would consider promotion rates, passfail rates, average scores, and delays in promotion to assess disparate impact, id. at 194, that it would not aggregate the
[9]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 10 of 31
data, id. at 195, and that the Plaintiffs established disparate impact regardless of the Court’s use of a one or two-tailed approach, id. at 198. The Court thus ruled that the Plaintiffs met their prong 1 burden to raise an inference of causation and demonstrate a prima facie case of disparate impact. Id. at 200.
B. Prong 2 The Court then assessed prong 2 of the disparate impact framework: job-relatedness and consistency with business necessity. Id. at 200-11. The Court noted that to prevail on prong 2, “[Boston] must convince the Court that the 2008 [lieutenants’] exam was both ‘job related’ for the position of Boston Police Department lieutenant and consistent with ‘business necessity.’” Id. at 200 (quoting Jones v. City of Boston (Jones I), 752 F.3d 38, 53 (1st Cir. 2014) (quoting 42 U.S.C. § 2000e-2(k)(1)(A)(i))). The Court cited First Circuit precedent, stating: to satisfy this second prong . . . “the [defendant] must show that its program aims to measure a characteristic that constitutes an ‘important element of work behavior’” . . . . [and] “that the outcomes of [its challenged practice] are ‘predictive or significantly correlated with’ the characteristic described above.” Id. at 201 (quoting Jones I, 752 F.3d at 54 (quoting Albemarle Paper Co. v. Moody, 422 U.S. 405, 431 (1975))). Noting that the
[10]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 11 of 31
Uniform Guidelines1 “provide a sensible way of evaluating whether
a given test . . . measures an important work characteristic,
and whether the outcomes of that test are actually correlated
with the characteristic measured,” the Court “look[ed] to the
Uniform Guidelines throughout its prong 2 analysis.” Id.
Proceeding through its inquiry, this Court first held that
the 2008 lieutenants’ exam measured characteristics that are
important elements of work behavior, id. at 201-02, because the
job analyses on which the test was based “were sufficiently
thorough and current so as to form solid ground on which to
build a valid test.” Id. at 202.
The Court then addressed if the exam results were
predictive of or correlated with the important work behaviors.
Id. at 202-10. The Court held that they were not, explaining
that it:
ultimately agrees with [the Plaintiffs’ expert] Dr. Wiesen that the evidence does not support the necessary inference that those who perform better on the exam will be better performers on the job, primarily because the exam did not test a sufficient range of [knowledge, skills, and abilities (“KSAs”)], and there was no evidence that the exam was reliable enough to justify its use for rank ordering.
1 29 C.F.R. § 1607.4(C)(2). The Court explained that “Chapter 29 of the Code of Federal Regulations, section 1607, was published under the name of Uniform Guidelines on Employee Selection Procedures in 1978 by several government agencies to interpret how selection and testing and assessment should be conducted in accordance with the Civil Rights Act of 1964.” Smith, 144 F. Supp. at 186 n.8 (citing 12/19/14 Tr. 23:6-18, ECF No. 166).
[11]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 12 of 31
Id. at 203. Boston attempted to show job-relatedness through content validity, “an attempt to link the important KSAs of the job with the selection procedure.” Id. The Court noted that the 2008 lieutenants’ exam had two components: a multiple choice component, worth eighty percent of an examinee’s score, and an Education and Experience score (“E&E”), worth twenty percent. Id. at 204. Relying on Dr. Wiesen’s report, which correlated candidates’ scores on the multiple choice section of the exam almost perfectly with their final score, the Court concluded that the E&E “had virtually ‘no impact on the final exam scores,’” and thus excluded the E&E from the remainder of the Court’s validity analysis. Id. (quoting 12/15/14 Tr. 58:19-21, ECF No. 151).
The Court then looked to test construction, id. at 204-06, and “f[ound] that the test construction process was inadequate to support the heightened validity requirement necessary to rank candidates,” id. at 206. In reaching this conclusion, the Court noted that the test outlines showed that only two abilities appeared on the 2008 lieutenants’ exam; Boston had sufficiently evaluated the test questions and their linkage to KSAs, but had “failed to conduct statistical analyses to ensure the quality of the test scores for the 2008 [lieutenants’] exam”; and that the record failed to address whether the test developer properly
[12]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 13 of 31
recommended cut-off scores, rankings, bands, and weighting. Id. at 205.
Next, the Court addressed the 2008 lieutenants’ exam’s content. Id. at 206-08. Using the Uniform Guidelines’ representative sample test, the Court found that the 2008 lieutenants’ exam tested a sufficient range of the critical knowledge areas, but that the exam’s near exclusion of any critical skills and abilities meant that “a high score on the 2008 [lieutenants’] exam simply was not a good indicator that a candidate would be a good lieutenant.” Id. at 207-08.
Lastly, the Court discussed the use of the exam results to rank candidates. Id. at 208-10. The Court relied on the Guidelines’ statements that “‘evidence of both the validity and utility of a selection procedure should support the method the user chooses for operational use of the procedure, if that method of use has a greater adverse impact than another method of use,’” and that “‘[e]vidence which may be sufficient to support the use of a selection procedure on a pass/fail (screening) basis may be insufficient to support the use of the same procedure on a ranking basis.’” Id. at 208 (quoting 29 C.F.R. § 1607.5(G)). Noting that the Uniform Guidelines allow employers to validate rank-order exams with content validity, by establishing “‘that a higher score on a content valid selection procedure is likely to result in better job performance,’” id.
[13]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 14 of 31
(quoting 29 C.F.R. § 1607.14(C)(9)), the Court found that Boston failed to meet this standard because it neither tested a sufficient range of critical KSAs nor convinced the Court that the exam was valid, id. at 208-09. Accordingly, this Court concluded that Boston failed to show that a higher score on the 2008 lieutenants’ exam would likely result in better job performance, id. at 210, and “h[eld] that even were the 2008 [lieutenants’] exam valid enough to be used as a screening tool, [Boston] failed to meet its burden of showing that the 2008 [lieutenants’] exam was sufficiently valid to be used as a basis for ranking candidates,” id. at 211. Thus, the Court held that Boston did not meet its burden on prong 2 of the disparate impact inquiry, and the Plaintiffs won the case. Id. III. ANALYSIS
Boston raises several challenges to this Court’s prong 2 finding, essentially arguing that Lopez II effected changes in disparate impact law that mandate a reconsideration of this Court’s previous decision. Boston also challenges this Court’s failure to reach prong 3, arguing that the Court could not reject the exam without the Plaintiffs showing an equally valid, less discriminatory alternative. Id. at 12-18. The Court concludes that none of Boston’s arguments have merit and upholds the original decision in Smith.
[14]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 15 of 31
A. 2008 Lieutenants’ Exam Validity (Prong 2 Challenges) Boston argues that this Court ought reconsider its prong 2
analysis because Lopez II: (1) disposes of the Guidelines’
representative sample inquiry in favor of a better than random
selection test, (2) mandates that this Court include the E&E
component in its assessment, and (3) holds that there is not a
heightened validity requirement for Boston to use rank ordering.
Def.’s Reply 5-11. These arguments are not convincing.
Further, even were this Court to follow Boston’s proposed test,
Smith’s conclusions would lead to the same result.
1. Reliance on the Guidelines Boston argues that this Court used the wrong legal standard
by relying on the Guidelines’ representativeness inquiry, rather
than Lopez II’s purported “better than random selection”
standard to determine the exam’s validity. Def.’s Br. 6; Def.’s
Reply 4.2 The First Circuit certainly did not ban the use of the
2 In advancing this argument, Boston comes close willfully to misreading Lopez II. Standing alone, the written multiplechoice lieutenants’ examination is pretty clearly better than random selection yet the hard truth is that Boston’s own experts recognize that such an examination disfavors minority applicants and Boston knows it. See Smith, 144 F. Supp. at 197 (citing 12/15/14 Tr. 113-14, ECF No. 161; 12/18/14 Tr. 45-46, ECF No. 165; Lopez I, 07/13/10 Tr. 82-85; Lopez I, 07/14/10 Tr. 43-48, 55, 59-60; Lopez I, 07/26/10 Tr. 30; Lopez I, 09/15/10 Tr. 5859; Lopez I, 09/16/10 Tr. 110). Surely Boston is not here arguing that such an examination, standing alone, would pass muster. Every judge who has considered the issue has held to the contrary. Smith, 144 F. Supp. at 208, 210-11 (Young, J.); Lopez I, 2014 U.S. Dist. LEXIS 124139, at *60-61 (O’Toole, J.);
[15]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 16 of 31
Uniform Guidelines’ representative sample test. It would be surprising if it had since the Guidelines come from the Equal Employment Opportunity Commission and are due an appropriate degree of deference.
Still, Boston argues that this Court erred by applying the Guidelines as “binding legal standards” -- Boston asserts that Lopez II makes clear that the representative sample test cannot be used because the Guidelines do not have a quantitative measure for deciding whether or not a selection procedure tests a representative number of KSAs. Def.’s Br. 6-7.
This Court disagrees. Lopez II noted that although the Guidelines do not provide a quantitative measure to draw the line between representative and nonrepresentative samples of job performance, the Guidelines do “point to the qualitative understandings of these concepts generally accepted by professionals who evaluate [selection procedures].” 823 F.3d at 112. Accordingly, although there may not be a bright line to which reference can be made, expert testimony can still highlight on which side of a blurry line a selection device falls. In fact, Lopez II recognized that the testimony of Dr. James Outtz did just this -- “[Outtz] opined that the exams were
see also Lopez II, 823 F.3d at 113-15 (Lynch & Kayatta, JJ.); id. at 124-25 (Torruella, J., concurring in part and dissenting in part).
[16]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 17 of 31
based on job analyses that validly identified the critical skills used by actual police sergeants and that the tests covered a ‘representative sample’ of the content of the job.” Id. (quoting 29 C.F.R. § 1607.14(C)(4)). Here, in contrast, a different expert opining on a different exam did not convince this Court that the 2008 lieutenants’ exam measured a representative sample of relevant KSAs. Smith, 144 F. Supp. at 207-08.
Boston also argues that Lopez II declined to follow the Guidelines’ technical requirements and instead established a lessened burden for the employer: a showing that the challenged exam is more job-related than random selection. Def.’s Reply 3. Boston posits that Lopez II “makes clear, ‘in the absence of any quantitative measure of “representativeness” provided by the law,’ the proper inquiry is not about representativeness, but whether the exam overall is more job-related than random selection would be.” Def.’s Br. 6 (quoting Lopez II, 823 F.3d at 116). Indeed, Lopez II emphasized that “[t]he Guidelines quite understandably provide no quantitative measure for drawing the line between ‘representative,’ and nonrepresentative samples of job performance and behaviors.” 823 F.3d at 112 (quoting 29 C.F.R. § 1607.5(B)). As Boston notes, the Lopez II court went on to reject the appellants’ arguments, stating:
[17]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 18 of 31
None of the remaining arguments advanced by the [appellants] seriously support any claim that the exams are not materially better predictors of success than would be achieved by the random selection of those officers to be promoted to sergeant. The parties’ arguments, instead, focus on how much better the exams were. Do they test enough skills and knowledge? Do they weigh the answers in an appropriate, valid way? In finding Outtz persuasive on these points, the district court as factfinder did not clearly err.
Id. at 116-17.
Boston interprets the First Circuit’s decision as a
rejection of the argument that a test can be invalidated if it
fails either to test a sufficient representative sample of
skills and abilities or to meet a heightened standard of
validity for rank ordering. Def.’s Reply 3-4. This, however,
may well not be a fair interpretation -- in the quoted passage
the First Circuit was merely deferring to the district court’s
role as the finder of fact on those points. Further, Lopez II
upheld the district court’s use of the representative sample
test, noting:
The district court’s opinion as a whole thus makes clear that the court trained its focus on critical and important knowledge, skills, and abilities called for by the job, and it did not clearly err by finding that a test that measured a large percentage of such critical and important KSAs was a test that was sufficiently “representative of important aspects of performance on the job.” Our conclusion to this effect finds further support in the absence of any quantitative measure of “representativeness” provided by the law. Rather, the relevant aim of the law, when a disparate impact occurs, is to ensure that the practice causing the impact serves an important need of the employer, in which case it can be used unless there is another way to meet that need with lesser disparate impact.
[18]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 19 of 31
We cannot see how it is an error of law to find that an exam that helps determine whether an applicant possesses a large number of critical and necessary attributes for a job serves an important need of the employer. Lopez II, 823 F.3d at 115-16. Fairly read, this passage condones the district court’s use of the representative sample test. It does not go so far as either to mandate or disapprove of that use as matter of law. Nearly all of the circuits have utilized a representative sample test in examining content validity. See, e.g., Johnson v. City of Memphis, 770 F.3d 464, 478 (6th Cir. 2014), cert. denied sub nom. Johnson v. City of Memphis, Tenn., 136 S. Ct. 81 (2015); M.O.C.H.A. Soc’y, Inc. v. City of Buffalo, 689 F.3d 263, 281 (2d Cir. 2012); Equal Emp’t Opportunity Comm’n v. Dial Corp., 469 F.3d 735, 742 (8th Cir. 2006); Allen v. City of Chicago, 351 F.3d 306, 309 n.3 (7th Cir. 2003); Zottola v. City of Oakland, 32 F. App’x 307, 311 (9th Cir. 2002); Nash v. Consolidated City of Jacksonville, Duval Cty., Fla., 837 F.2d 1534, 1539 (11th Cir. 1988), cert. granted, judgment vacated, 490 U.S. 1103 (1989), and opinion reinstated, 905 F.2d 355 (11th Cir. 1990). One imagines that, had the First Circuit been adopting a legal rule it would have acknowledged this body of law. Boston argues that to satisfy its prong 2 burden, it need only establish that its test is a materially better predictor of
[19]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 20 of 31
success than random selection. Def.’s Reply 6. This proposed standard, however, is not irreconcilable with the representative sample test. The aim of the representative sample test is to ensure that an exam tests for success in a specific job. Applying the representative sample test ensures that “the content of the selection procedure is representative of important aspects of performance on the job for which the candidates are to be evaluated.” 29 C.F.R. § 1607.5(B). In other words, a court ensures that a selection device evaluates characteristics important to job performance, rather than random attributes that may not correlate with success in that job. To be materially better than random at choosing applicants who will excel at a job, this Court can only imagine that the selection device would necessarily examine a large proportion of the KSAs needed to succeed at the position.
As discussed above, Lopez II did not reject the representative sample test as matter of law; and to assume that the First Circuit would change disparate impact law without so much as a comment seems somewhat at odds with reality. Accordingly, this Court declines to reconsider its use of the representative sample test from the Guidelines.
2. Rejection of the E&E Component Boston argues that because Lopez II found that the E&E was useful for qualities important to a sergeant’s daily
[20]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 21 of 31
responsibilities, this Court should apply the E&E as part of its
validity analysis. Def.’s Br. 9-10. In fact, this Court did
consider the E&E in its ruling, but held the E&E did not rescue
an otherwise invalid written exam. Smith, 144 F. Supp. 3d at
204.
The First Circuit discussed the E&E component of the 2008
sergeant’s exam:
In Outtz’s opinion, however, the addition of the E&E component effectively pushed the selection device as a whole across the finish line to show validity. It did this, according to Outtz, because the level and extent of work and educational experience and accomplishments listed by each applicant served as a useful, if imperfect, proxy for the kinds of qualities that were deemed to be important to a sergeant’s daily responsibilities, yet were insufficiently tested by the examination’s question and answer component. Outtz recognized that the gain in validity from the E&E component was, on its own, only marginal or “incremental.” As the Officers stress, many of the attributes for which the E&E assigned points . . . were shared by all or most applicants. . . . when weighted to provide only 20% of the combined final score, it accounted for a range of only about 5% to 7% of a candidate’s total score. Nevertheless, we cannot see how a rational factfinder could ignore the impact of the E&E, small or not, in evaluating the exam overall.
Lopez II, 823 F.3d at 113.
The evidence presented in Smith, however, varied from that
available in Lopez I. In reaching its decision, this Court:
(1) relied on expert testimony that the E&E component failed to
differentiate among candidates or demonstrate the KSAs necessary
in a lieutenant, Smith, 144 F. Supp. 3d at 203-04; (2) had no
evidence that incumbent lieutenants performed better on the
[21]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 22 of 31
written exam, see generally id. at 207-10 (discussing the evidence presented to demonstrate exam validity); and (3) had no evidence to show that the E&E component was valid on its own, id. at 211 n.42.
These differences are crucial. In Lopez II, the E&E inched the exam over the line of validity due to its measuring “the kinds of qualities that were deemed to be important to a sergeant’s daily responsibilities.” 823 F.3d at 113. Here, however, the testimony does not establish that the E&E measured qualities important to a lieutenant’s daily responsibilities. Further, even if the E&E assessed a lieutenant’s important KSAs, Dr. Wiesen’s testimony that the E&E “had virtually no impact on the final exam scores,” Smith, 144 F. Supp. 3d at 204 (internal quotation marks omitted), persuades this Court that the E&E had so minimal an effect that it could not uphold the 2008 lieutenants’ exam’s validity. Accordingly, this Court declines further to address the E&E in its analysis.
3. Rank Ordering Boston argues that this Court inappropriately applied a heightened validity requirement for rank ordering and that Lopez II holds that rank ordering furthers Boston’s interest in eliminating patronage and intentional racism. Def.’s Br. 11. The First Circuit’s statement is, however, dicta. In Lopez II, the First Circuit stated:
[22]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 23 of 31
Rank ordering furthers the City’s interest in eliminating patronage and intentional racism under the guise of subjective selection criteria. Such a goal is itself a reasonable enough business need so as to provide some weight against a challenge that is unaccompanied by any showing that rank order selection itself caused any disparate impact in this case. 823 F.3d at 119. Boston asserts that this statement binds this Court, Def.’s Br. 11, and so Boston’s business need to rank order supports its meeting the prong 2 burden in light of the Plaintiff’s failure to set for evidence that rank ordering results in disparate impact, id. at 11-12. This Court, however, did find that rank ordering based on the 2008 lieutenants’ exam had a disparate impact on minority applicants. Smith, 144 F. Supp. 3d at 199-200. In Smith, this Court stated, “the Plaintiffs have presented evidence of statistically significant disparate impact on . . . delay in promotions”; “if the eligibility list from the 2008 [lieutenants’] exam had not been extended due to the Lopez litigation, but instead had expired three years after its creation, as is typical, not a single black sergeant would have been promoted to lieutenant”; and “[a]ll of this evidence combined is enough for this Court to rule that the Plaintiffs have met their burden of raising an inference of causation and demonstrating a prima facie case of disparate impact.” Id. Although these statements do not use the precise language “rank order selection caused disparate impact,” this Court made it sufficiently clear -- through its
[23]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 24 of 31
discussion of disparate impact in relation to delayed promotions -- that rank ordering resulted in disparate impact.
Boston goes on to argue that a prong 2 analysis does not depend on whether rank order selection increases disparate impact and that neither Title VII nor the Guidelines provide a quantitative requirement about how job-related a selection device has to be or how much better it need be for rank ordering. Def.’s Reply 9-10. “Employers should tailor a discriminatory hiring practice to a job-related risk, making sure to proportionally weigh the costs and benefits of accommodating that risk.” Jake Elijah Struebing, Note, Reconsidering Disparate Impact Under Title VII: Business Necessity as Risk Management, 34 Yale L. & Pol’y Rev. 499, 507 (2016). Where a selection procedure not only has a disparate impact on a pass-fail basis, but also compounds that effect through use of rank ordering, each hiring decision carries an increased risk of a discriminatory result. Such heightened risk merits applying a more stringent validity requirement to ensure that the exam is sufficiently job-related to warrant the cost of potential discrimination. Accordingly, this Court did not err in applying a heightened validity requirement for rank ordering.
Further, even if Boston faced a lower bar to establish validity, it still failed to show that it met its burden. Although, as Boston argues, Def.’s Reply 10, this Court stated
[24]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 25 of 31
that “[w]hat the Court can conclude from the 2008 [lieutenants’] exam is that those who excelled at the exam would exhibit superior levels of knowledge on the job, and that the 2008 [lieutenants’] exam differentiated among levels of candidates’ knowledge levels,” Smith, 144 F. Supp. 3d at 209, the Court also noted “that this is insufficient for predicting who will be a good police lieutenant,” id. In particular, this Court emphasized that testing only knowledge, rather than including other necessary skills and abilities, could not persuade this Court that those who did well would be better performers on the job. See id. Put another way, even were this Court to apply the “better than random” standard Boston advocates, Def.’s Reply 10, the Court concluded that it cannot presume that testing only knowledge will result in a better than random procedure for selecting candidates for promotion. As this Court stated, “the evidence does not support the necessary inference that those who perform better on the exam will be better performers on the job, primarily because the exam did not test a sufficient range of KSAs, and there was no evidence that the exam was reliable enough to justify its use for rank ordering.” Smith, 144 F. Supp. 3d at 203. Additionally, the Court later emphasized, “the Court cannot find that [Boston] met its burden on [adequacy of test construction]: too many skills and abilities were missing from the 2008 test outline,” id. at 205, and “[t]he Court
[25]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 26 of 31
concludes that the 2008 [lieutenants’] exam did not sufficiently test for a representative sample of the critical KSAs,” id. at 207. This goes to show that the Court held that even if Boston did not need to meet a heightened standard for rank ordering, it still failed to carry its burden to establish test validity.
B. Equally Valid, Less Discriminatory Alternative (Prong 3 Challenges)
Boston last argues that this Court cannot reject Boston’s business justification unless there is some showing that there “exists an available alternative with less disparate impact that serves [Boston’s] legitimate needs.” Def.’s Br. 12. This Court, however, is confident in its understanding of the shifting burdens of the disparate impact framework: if the defendant fails to meet its burden of proof on prong 2, then the defendant loses, regardless of the plaintiffs’ showing of an alternative.
Boston argues that Jones v. City of Boston (Jones II), 845 F.3d 28 (1st Cir. 2016), affirms Lopez II’s lowered prong 2 standards and emphasizes the importance of prong 3. Def.’s Reply 1-2. This contention is hardly convincing. In Jones II, a number of police department employees brought a disparate impact challenge to the Boston Police Department’s hair drug test, which they claimed was racially discriminatory. 845 F.3d at 30-31. Having held that the police department employees met
[26]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 27 of 31
the first prong of the disparate impact inquiry in Jones I, the First Circuit examined the district court’s grant of summary judgment on prongs 2 and 3, upholding the former and vacating the latter. See id. The First Circuit first turned to prong 2: whether the challenged test was job-related and consistent with business necessity. See id. at 32. Noting that “[t]he parties agree[d] that ‘abstention from drug use is an important element of police behavior,’ and is thus job related . . . . [and] that selecting police officers for retention or discharge based on that job-related behavior is consistent with business necessity,” the court turned its analysis to whether the drug test was so unreliable that a reasonable juror could find that the test “did not meaningfully further the [Boston Police] Department’s legitimate need for a drug-abstaining police force.” Id. The court emphasized that the test had a high degree of accuracy, but that
there is no reason why a test need be anything near 100% reliable (few tests are) to be consistent with business necessity (keeping in mind that the presence of an alternative method that would have had less of a disparate impact will still be relevant under the third prong of the inquiry). Id. at 33. The court then ruled that the Boston Police Department had clearly met its prong 2 burden to establish that the test was job-related and consistent with business necessity. Id. at 33-34. Turning to prong 3 of the inquiry, the court
[27]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 28 of 31
eventually held that police department employees could potentially succeed in showing that the Boston Police Department refused to adopt an alternative test with less of a disparate impact. See id. at 38. Accordingly, the circuit vacated the district court’s grant of summary judgment on prong 3. Id.
Boston latches onto the Jones II court’s focus on reliability, arguing that it evidences the First Circuit lessening the burden on employers in prong 2 while increasing the importance of prong 3. Def.’s Reply 2. Jones II, however, is distinguishable from the instant case on numerous grounds. First and foremost, in Jones II, both parties had essentially agreed that, if reliable, the drug test was job-related and consistent with business necessity. See 845 F.3d at 32. Here, in contrast, this Court specifically held that Boston failed to show that the 2008 lieutenants’ exam was job-related and consistent with business necessity. Smith, 144 F. Supp. 3d at 211. Accordingly, an emphasis on reliability would be inappropriate. Second, Jones II addressed the validity of a test designed to look only at one thing -- drug abstention -rather than a complex examination designed to test for the KSAs of promotion candidates. For this reason, the Jones II court’s validity examination is distinguishable from this Court’s business necessity analysis in Smith.
[28]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 29 of 31
Further, Lopez II itself is consistent with the traditional burden-shifting framework of disparate impact. In Lopez II, the court summarizes the law of disparate impact, explicitly stating that a plaintiff who satisfies prong 1 will prevail either by the employer failing to meet their burden on prong 2, or by the plaintiff meeting their burden on prong 3. See Lopez II, 823 F.3d at 110-11. This outright statement of law warns against heeding Boston’s call to collapse the inquiry of prongs 2 and 3. IV. CONCLUSION
Both Lopez I and Smith are fact intensive cases. In Lopez I, Boston persuaded Judge O’Toole, albeit barely, that it ought prevail on prong 2. Smith is a different case, with a different evidentiary record, involving different expert testimony about a different and more demanding senior officer position. In Smith, Boston failed to convince me that it ought prevail on prong 2, an aspect of the case on which it bears the burden of proof. The Court of Appeals affirmed the district court’s decision in Lopez I.
In view of this affirmance, Boston seems to be arguing that it is now legal error to reach a contrary result in a significantly different case.3 That’s not how it works. Fact
3 Indeed, Boston apparently seeks to conjure up a hitherto unrecognized species of offensive issue preclusion, cf. BlonderTongue Labs., Inc. v. University of Ill. Found., 402 U.S. 313 (1971), based on the overlapping portions of the sergeants’ and
[29]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 30 of 31
finding is the province of the district courts. In Lopez II, the Court of Appeals did what it always does -- it carefully scrutinized the evidentiary record, giving due deference to the fact-finding role of the district judge, to see whether any of his conclusions were clearly erroneous. That’s what it said it was doing. See Lopez II, 823 F.3d at 107-08. That’s what it did. Should this Smith case be appealed, it will do the same for me.
Boston fails to offer any convincing argument as to why this Court ought disrupt its previous ruling in Smith. Accordingly, it declines to do so. The Court also takes this opportunity to note that Boston is not left without any useable test. Boston utilized an updated selection procedure in 2014. Def.’s Br. 4. Although Boston has represented to this Court that the 2014 exam resulted in a greater disparate impact and already faces legal challenges, id. at 4 & n.3, none of these issues are properly before this Court.
What is clear is that this case has gone on far too long. It is nearly a decade since the original Boston patrolmen brought suit in Lopez I. The tests that so engross us here are long out of date. In its order rejecting an interlocutory appeal, the First Circuit indicated it was amenable to
lieutenants’ written exams and the fact that the same plaintiffs’ counsel appears in both cases.
[30]

Case 1:12-cv-10291-WGY Document 245 Filed 07/26/17 Page 31 of 31 entertaining such an appeal once this Court had analyzed the effect of Lopez II on its earlier decision in Smith. It has now done so. Should the parties jointly move, within 14 days of the date of this memorandum, for an order authorizing application for a second interlocutory appeal, this Court will allow such motion. If not, this Court will promptly schedule hearings on remedy, settlement talks or no.
SO ORDERED. /s/ William G. Young WILLIAM G. YOUNG DISTRICT JUDGE
[31]

 
 Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 1 of 10

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

___________________________________

)

BRUCE SMITH, PAUL JOSEPH,

)

JOHN M. JOHNSON, ROBERT TINKER )

MARTIN JOSEPH, KIM GADDY,

)

BRIAN KEITH LATSON, MARWAN )

MOSS, LEIGHTON FACEY, LATEISHA )

ADAMS, KENNETH SOUSA, and

)

WILLIAM WOODLEY,

)

)

Plaintiffs,

)

)

v.

)

)

CITY OF BOSTON, MASSACHUSETTS )

)

Defendant.

)

)

C. A. No.: 1:12cv-10291-WGY
LEAVE TO FILE GRANTED ON JULY 9, 2019 (ECF NO. 276)

THIRD AMENDED COMPLAINT, COMPENSATORY, INJUNCTIVE AND DECLARATORY RELIEF REQUESTED

I. INTRODUCTION

1. This is an action by minority police sergeants for the Boston Police Department

who have taken the 2005 and/or 2008 Boston Police Lieutenants’ promotional exam and who

claim that such examinations are unlawful under Title VII of the Civil Rights Act of 1964 and

Mass. Gen. Laws ch. 151B because they have a severe disparate impact on minority test takers

and cannot be shown to be job related under applicable federal Equal Employment Opportunity

Commission guidelines. The Plaintiffs seek appropriate injunctive and compensatory relief as

described herein.

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 2 of 10
II. JURISDICTION 2. This Court has jurisdiction over this case under 28 U.S.C §§ 1331 and 1343 in
that this case arises under the laws of the United States of America, including particularly, Title VII of the Civil Rights Act of 1964, 42 U.S.C. §§ 2000(e), as amended. The Court has supplemental jurisdiction over pendent claims pursuant to 28 U.S.C. § 1367(a). III. PARTIES
3. Plaintiff Bruce Smith is an adult resident of Randolph, Massachusetts. He has been a Boston Police Officer since 1989 and a Police Sergeant since May of 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 78.
4. Plaintiff Paul M. Joseph is an adult resident of Boston, Massachusetts and is a Sergeant on the Boston Police force. He has held this position since 1997, and he has been a uniformed member of the Boston Police Department since 1989. In 2005 he took the Boston Police Lieutenants’ promotional exam and received a passing score of 70. In 2008 he took the promotional examination and received a score of 87.
5. Plaintiff John M. Johnson is an adult resident of Milton, Massachusetts. He is currently a Boston Police Sergeant, a position he has held since 1997. He has been a uniformed member of the Boston Police force since 1986. In 2005 he took the Boston Police Lieutenants’ promotional exam and received a score of 86.
6. Plaintiff Robert Tinker is an adult resident of Weymouth, Massachusetts. He has been a Boston Police Sergeant since 1988, and has been a uniformed member of the Boston Police Department since 1979. He took the Boston Police Lieutenants’ promotional exam in 2005 and received a score of 86.
2

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 3 of 10
7. Plaintiff Martin Joseph is an adult resident of Dorchester, Massachusetts. He is currently a sergeant on the Boston Police Department and has held this position since 2005. He has been a uniformed member of the police force since 1986. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a passing score of 77.
8. Plaintiff Kim Gaddy is an adult resident of Jamaica Plain, Massachusetts. She has been a police officer for the City of Boston since 1985, and she has been a sergeant since 1992. Plaintiff Gaddy took the 2008 Boston Police Lieutenants’ promotional exam and received a passing score in the 80’s.
9. Plaintiff Brian Keith Latson is an adult resident of Sharron, Massachusetts. He has been a Boston Police officer since 1985 and a Boston Police Sergeant since 2001. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 76.
10. Marwan Moss is an adult resident of Roslindale, Massachusetts. He has been a police officer since 1995 and a Police Sergeant since April of 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 87.
11. Leighton Facey is an adult resident of Boston, Massachusetts. He has been a Boston Police Officer since June 1998 and a Police Sergeant since September 2005. In 2008 he took the Boston Police Lieutenants’ promotional exam and received a score of 88.
12. LaTeisha Adams is an adult resident of Dorchester, Massachusetts. She has been a Boston Police Officer since March 1995 and a Police Sergeant since June 2007. In 2008 she took the Boston Police Lieutenants’ promotional exam and received a score of 84.
13. Kenneth Sousa is an adult resident of Bridgewater, Massachusetts. He has been a Boston police officer since November 1989 and a Police sergeant since September 2006. He took the Boston Police Lieutenant’s promotional exam in 2008 and received a score of 78.
3

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 4 of 10
14. William Woodley is an adult resident of Brockton, Massachusetts. He served as a Boston police officer from June 1986 until his retirement in January 2017. He was promoted to the rank of Police sergeant in February 1995. He took the Boston Police Lieutenant’s promotional exam in 2008 and received a score of 82.
15. The Defendant, City of Boston, is a duly incorporated municipality of the Commonwealth of Massachusetts. It runs and operates a large urban Police Department, and it makes promotions in the Boston Police Department pursuant to M.G.L. ch. 31, the state Civil Services Law. IV. CLASS ALLEGATIONS
16. The named Plaintiffs, all of whom are African American, bring this action individually and on behalf of a class of individuals similarly situated. The class of individuals similarly situated is as follows: All black police sergeants for the Boston Police Department, who have taken the 2005 and/or 2008 Boston Police Lieutenant Promotional Examination or who plan to take such examination when next given.
17. The claims of the class plaintiffs are similar if not identical to the claims of the named plaintiffs in that each has challenged the validity of the Boston Police Lieutenants’ examination and promotional procedure and do so under the same legal theory – disparate impact.
18. The class meets all of the numerosity requirements of Rule 23 of the Federal Rules of Civil Procedure, as it would be impractical to name each individual plaintiff. In addition, the claims of the named plaintiffs are representative and typical of the claims of the class plaintiffs.
4

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 5 of 10
19. There are common questions of law and fact which predominate over any specific individual factual claims in this case, and plaintiffs’ counsel are eminently qualified to represent the claims of the named and class plaintiffs. This class is primarily certifiable under Rule 23(b)(2) of the Federal Rules of Civil Procedure in that Plaintiffs seek declaratory and injunctive relief generally applicable to all plaintiffs. The plaintiffs also seek damages in the form of back pay and other damages allowable by Title VII of the Civil Rights Act of 1964, but those claims are ancillary to the injunctive and declaratory relief sought by the plaintiffs.
20. This class should also be certified under Rule 23(b)(3) of the Federal Rules of Civil Procedure in that there are questions of law or fact which are common to the class which predominate over any questions affecting only individual members, and a class action is a far superior method of fairly and efficiently adjudicating this controversy. V. STATEMENT OF FACTS
21. All of the named plaintiffs are African-American who served as police sergeants for the City of Boston Police Department, and each is well qualified for promotion to the position of Lieutenant on the Boston Police Department.
22. Each of the plaintiffs took either the 2005 or 2008 Boston Police Lieutenants’ promotional examination, or both, and each received a passing score on said examination, but has not been promoted to the position of lieutenant.
23. The promotional process for becoming a lieutenant on the Boston Police Department is as follows: First, the Boston Police Department, in conjunction with the Commonwealth of Massachusetts, Human Resources Division announces that there will be a Boston Police Lieutenants’ promotional exam consisting of approximately 100 multiple choice
5

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 6 of 10
questions which are taken almost verbatim from texts and other written materials which relate to police work or supervision.
24. Each individual taking the examination receives a score based on the number of correct answers, scaled to 100 possible points. In order to be considered at all for a promotion, one must attain a passing score of 70. The final score consists of the results of the written multiple choice exams (which are based upon rote memory and taken directly from texts), and several points may be added or subtracted based upon one’s years of service and educational background, but no points are added for relevant forms of experience, including military service.
25. The final score is then used to rank each candidate in order of their score on the examination.
26. Promotions to the position of Boston Police Lieutenant are then made by the defendant in strict rank order based upon one’s rank on the promotional list.
27. For at least 30 years, the Boston Police Department, as well as all experts in the field, have been aware that such multiple choice rote memory examinations have a severe disparate impact on minority test takers, including African-Americans. In addition, the evidence is overwhelming that such examinations are not valid under appropriate federal and state EEOC guidelines, in that they are not job related – or predictive of actual ability to do the job. Further, there are many and multiple less discriminatory testing procedures which are equally or more valid as predicting job abilities, and which have a far less discriminatory impact on minority candidates.
28. Because the City of Boston has predominately administered such multiple choice police promotional examinations for the position of lieutenant over the last 25 years, the result is that currently despite the fact that more than 30% of the entire Boston Police Department is made
6

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 7 of 10
up of minority police officers, there are only, on information and belief, 2 African-American Police Lieutenants on the entire Boston Police Department out of nearly 50 lieutenants, or less than 5% of the entire police lieutenants on the Boston Police Department. In addition as of March 1, 2012 the Boston Police Department will have no minority police captains.
29. In the fall of 2005, the Boston Police Department conducted a lieutenants’ promotional examination, which results were certified in the early spring of 2006. Promotions from that list continued to be made until early 2009. On information and belief, of all the lieutenant promotions made from that list, only one African-American was promoted.
30. In the fall of 2008, the Boston Police Department administered another promotional exam for the position of lieutenant, which list became certified in early 2009. Although many lieutenant promotions have been made from that list between 2009 and the present, to date there have been no minority promotions from that list despite the fact that there are many African-American police sergeants who passed the exam and were well qualified for the position.
31. Thus, as a direct result of the Boston Police Department’s use of written multiple choice examinations, the number of Boston Police Lieutenants who are African-American is far below the number of African-American Police Lieutenants that would be expected to be promoted had there been a fair lieutenants’ promotional examination procedure that did not have such a severe disparate impact upon minority officers. VI. EXHAUSTION OF ADMINISTRATIVE REMEDIES
32. Each of the named plaintiffs have filed complaints of discrimination relating to their failure to be promoted to the position of Boston Police Lieutenant with the Massachusetts
7

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 8 of 10
Commission Against Discrimination and the Equal Employment Opportunities Commission, and each of the named plaintiff has received right-to-sue letters.
COUNT I (Title VII of Civil Rights Act of 1964) 33. The conduct of the defendant, City of Boston, in conducting, administering, and utilizing a promotional examination system for the position of lieutenant in the Boston Police Department that has a severe disparate impact on minority candidates, cannot be shown to be job related and is not valid in accordance with EEOC guidelines is unlawful. In addition, there are many less discriminatory examination procedures which would have less discriminatory impact. Thus the Defendant’s actions violate Title VII of the Civil Rights Act of 1964, 42 U.S.C. §2000(e).
COUNT II (Mass. Gen. Laws ch. 151B) 34. The conduct of the defendant, City of Boston, in conducting, administering, and utilizing a promotional examination system for the position of lieutenant in the Boston Police Department that has a severe disparate impact on minority candidates, cannot be shown to be job related and is not valid in accordance with EEOC guidelines is unlawful. In addition, there are many less discriminatory examination procedures which would have less discriminatory impact. Thus the Defendant’s actions violate Mass. Gen. Laws ch. 151B. WHEREFORE, Plaintiffs request this Honorable Court to issue appropriate injunctive, declaratory, and compensatory relief which 1) requires the Boston Police Department to utilize a nondiscriminatory promotional examination procedure for the position of lieutenant, 2) grants injunctive and declaratory relief preventing defendants from using their current promotional procedure, 3) grants plaintiffs injunctive and compensatory relief including back pay and other damages, including reasonable attorneys’ fees and costs, and 4) grants such other and further relief as this Court deems just and proper.
8

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 9 of 10 9

Case 1:12-cv-10291-WGY Document 289 Filed 09/24/19 Page 10 of 10

Respectfully submitted,
BRUCE SMITH, et al. By their attorneys,

/s/ Harold Lichten Harold Lichten, BBO#549689 Benjamin Weber, BBO#673736 Zach Rubin, BBO#704485 Lichten & Liss-Riordan, P.C. 729 Boylston Street, Suite 2000 Boston, MA 02114 (617) 994-5800 hlichten@llrlaw.com bweber@llrlaw.com

DATED: September 24, 2019

Stephen Churchill, BBO# 564158 Fair Work, P.C. 192 South Street, Suite 450 Boston, MA 02111 (617) 607-3262 steve@fairworklaw.com

CERTIFICATE OF SERVICE I certify that a copy of the foregoing document has been served on counsel for all parties through the ECF system on this date.

DATED: September 24, 2019

/s/ Harold L. Lichten Harold L. Lichten

10

 
 Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 1 of 29

UNITED STATES DISTRICT COURT DISTRICT OF MASSACHUSETTS

)

BRUCE SMITH, PAUL JOSEPH, JOHN M. )

JOHNSON, ROBERT TINKER, MARTIN

)

JOSEPH, KIM GADDY, BRIAN KEITH

)

LATSON, LEIGHTON FACEY, KENNETH )

SOUSA, WILLIAM WOODLEY, MARWAN

)

MOSS, and LATEISHA ADAMS,

)

)

Plaintiffs, )

)

v.

)

)

CITY OF BOSTON,

)

)

Defendant.

)

)

CIVIL ACTION NO. 12-10291-WGY

YOUNG, D.J.

May 13, 2020

MEMORANDUM & ORDER

I. INTRODUCTION

This litigation has spanned years and prompted the Court to

issue two published opinions on the merits ruling in favor of

the police-officer plaintiffs (“the Officers”) against the City

of Boston (“Boston”) on their Title VII disparate impact claim

arising from the 2008 lieutenants’ promotional exam. Smith v.

City of Boston (Smith I), 144 F. Supp. 3d 177 (D. Mass. 2015);

Smith v. City of Boston (Smith II), 267 F. Supp. 3d 325 (D.

Mass. 2017). Having determined that the test had an unlawful

disparate impact, the Court must now decide the remedy. The

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 2 of 29
issues in dispute are whether there is a binding presumption that entitles the plaintiffs to back pay and, if so, whether Boston has successfully rebutted that presumption by introducing evidence of scores from the new exam given in 2014. The parties also debate the start and end dates of any back pay award.
For the reasons that follow, the Court rules that there is a binding presumption of back pay (notwithstanding the contrary dictum in a footnote in a recent opinion of the First Circuit in a related case) and that Boston has not rebutted this presumption. Per the stipulation of the parties, back pay is limited to each plaintiff’s “loss of chance” of being promoted as calculated by an agreed-upon formula. The Court adopts the Officers’ proposed start date and awards the Officers back pay in keeping with the stipulated calculation. The Court also adopts the Officers’ end dates, except for those plaintiffs who failed to mitigate damages by taking the 2014 exam and whose back pay therefore terminates on Boston’s proposed end date.
A. Procedural History After the Court ruled in favor of the Officers in Smith I in 2015, the First Circuit rejected a similar claim from a group of officers challenging Boston’s 2008 sergeants’ promotional exam. Lopez v. City of Lawrence, 823 F.3d 102 (1st Cir. 2016), cert. denied, 137 S. Ct. 1088 (2017). When the parties sought interlocutory appeal of Smith I, the First Circuit demurred on
[2]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 3 of 29
October 11, 2016, with the suggestion that this Court take another look with the benefit of the Lopez case. Smith II, 267 F. Supp. 3d at 328 (quoting ECF No. 229). Finding that Lopez addressed a somewhat different set of facts and did not change the relevant law, this Court reaffirmed its decision on July 26, 2017. Id. at 337. The parties again sought interlocutory review and, on April 17, 2019, the First Circuit again denied it, this time without comment. ECF No. 250.
From October 28-30, 2019, the Court held a three-day bench trial on damages. ECF Nos. 292-294. The Officers declined to seek any remedy other than back pay (potentially along with attorneys’ fees and costs once a remedy is decided). Trial Tr. Day 1 at 7:8-25, ECF No. 292; Pls.’ Br. Remedy (“Pls.’ Br.”) 7, ECF No. 303. Furthermore, the parties stipulated to a “loss of a chance” calculation of back pay, which fixes the lost chance ratio at 36.26% of the total value of each Officer’s back pay (the promotion rate of the 2008 exam). Joint Trial Stipulations, Trial Ex. 86; Joint Suppl. Trial Exs., Ex. A, Pls.’ Calculations, ECF No. 301-1; id., Ex. B, Def.’s Calculations, ECF No. 301-2 (jointly, “Stipulated Calculations”); Def. City Boston’s Damages Trial Br. (“Def.’s Br.”) 24-26, ECF No. 302; Pls.’ Br. 5-7.
[3]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 4 of 29
B. Factual Background This opinion presumes familiarity with the underlying facts of liability discussed in the Court’s two previous published opinions in this case, Smith I and Smith II. The crucial facts to recall from those opinions are that Boston’s 2008 lieutenants’ exam was deemed to have a disparate impact on minority candidates and to lack job-relatedness, in violation of Title VII and its Massachusetts analogue, Chapter 151B. Smith I, 144 F. Supp. 3d at 212. This Court did not reach the question whether any alternative selection tool might have had less discriminatory impact. Id. at 211. With that in mind, the Court turns to the factual questions aired at the damages trial. The focus of the damages trial was the import, if any, of Boston’s next iteration of the lieutenants’ exam, administered in 2014. There is no doubt that the 2014 exam was qualitatively far superior to the 2008 exam. It included not only a written technical knowledge test (45% of the grade), but also an “inbasket” exercise (weighted at 25%) and two oral board exercises (the final 30% together). Trial Tr. Day 1 at 78:2-79:8. The 2014 exam was, in brief, a test significantly changed for the better, “in part due to a desire to increase diversity.” Smith I, 144 F. Supp. 3d at 190. Yet, to the “surpris[e]” of both experts, the 2014 exam turned out to have had a greater adverse impact on black candidates than did the 2008 exam. Trial Tr.
[4]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 5 of 29
Day 1 at 100:15-23; Trial Tr. Day 2 at 130:4-5, ECF No. 293. Whereas five black candidates were promoted as a result of the 2008 exam, the 2014 exam landed only one black candidate in the top forty spots. Def.’s Br. 23 (citing Trial Ex. 93 at 15-18).
Of the plaintiffs here, only four -- Paul Joseph, Martin Joseph, Bruce Smith, and Kenneth Sousa -- sat for the 2014 exam. Pls.’ Br. 18-19; Aff. Edward P. Callahan –- Trial Ex. 85, Ex. 4, Boston PD 2014: Lieutenant Final Score Report (“2014 Results”), ECF No. 177. Sergeant Paul Joseph received a score of 87 on the 2008 test and thought he would be promoted from that list. Trial Tr. Day 1 at 119:15-121:9. He took the 2014 test anyway because, in his words, he “just wanted to see what the test was going to be about” -- but he “[a]bsolutely [did] not” study for it. Id. at 122:3-6. He scored a 71, next to last in the class. 2014 Results 3. The other three officers in fact did marginally better on the 2014 exam than they had done in 2008, though still too low to be in line for a realistic promotion. Sergeant Martin Joseph got an 80 (up from 77.49 in 2008), Sergeant Bruce Smith a 78 (compared with a 77.75 in 2008), and Sergeant Kenneth Sousa an 83 (78.41 in 2008). Def.’s Br. 22 ns. 36-39. II. ANALYSIS
The Officers’ argument for back pay stems from the “Albemarle presumption,” derived from the Supreme Court’s holding “that, given a finding of unlawful discrimination,
[5]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 6 of 29
backpay should be denied only for reasons which, if applied generally, would not frustrate the central statutory purposes of eradicating discrimination throughout the economy and making persons whole for injuries suffered through past discrimination.” Albemarle Paper Co. v. Moody, 422 U.S. 405, 421 (1975). Thus, the Officers contend, back pay is presumptively warranted, and Boston has done nothing to rebut that presumption. In response, Boston offers two arguments: (1) that the Albemarle presumption does not really exist, at least according to the First Circuit’s recent decision in Lopez; and (2) alternatively, that Boston effectively rebutted the presumption with its evidence from the 2014 exam.
A. Is There a Presumption -- and What Does It Mean? The term “presumption” does not appear in Albemarle itself, except in a footnote’s passing reference to analogous case law under the National Labor Relations Act. Id. at 420 n.12 (“The finding of an unfair labor practice and discriminatory discharge is presumptive proof that some back pay is owed by the employer.” (quoting NLRB v. Mastro Plastics Corp., 354 F.2d 170, 178 (2d Cir. 1965))). Nonetheless, the Supreme Court subsequently stated that “[t]he Albemarle presumption in favor of retroactive liability can seldom be overcome, but it does not make meaningless the district courts’ duty to determine that such relief is appropriate.” City of Los Angeles, Dep’t of
[6]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 7 of 29
Water & Power v. Manhart, 435 U.S. 702, 719 (1978); see Johnson v. Spencer Press of Maine, Inc., 364 F.3d 368, 382 (1st Cir. 2004) (“Albemarle taught that backpay is a presumptive entitlement of a victim of discrimination and that the discriminating employer is responsible for all wage losses that result from its unlawful discrimination, at least until the time of judgment.”).
The Supreme Court and First Circuit speak of a “presumption” in favor of back pay, and this word is at the core of the quarrel between Boston and the Officers. What precisely does this “presumption” entail? “No term has been more frequently or variously defined,” one learned author bemoaned, to the point that “the whole subject seems an entanglement of definition and explanation, which leaves the mind in a hopeless state of bewilderment.” John Jay McKelvey, Handbook of the Law of Evidence § 39, at 93-94 (3d ed. 1924). “Is it possible,” another scholar wondered, “that for a century or longer judges, lawyers and jurists have not known what they are talking about when using the word ‘presumption’?” Otis Harrison Fisk, Presumptions in the Law: A Suggestion 1 (1921). One more century of legal rumination behind us, here we are.
Boston contends that the Albemarle presumption is a mirage. See Def.’s Br. 17. Instead of “some ‘substantive rule of law,’” Boston argues, “the so-called ‘Albermarle presumption’ boils
[7]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 8 of 29
down to little more than the rather unremarkable proposition
that back pay is an available Title VII remedy that should be
awarded where appropriate.” Id. at 18 (citing Rosario-Torres v.
Hernandez-Colon, 889 F.2d 314, 322 (1st Cir. 1989) (en banc),
which recharacterized the “presumptive remedy” for violations of
42 U.S.C. § 1983 as “not a substantive rule of law, but merely
an attempt at reformulation of the basic principle that . . .
courts should weigh and balance the equities”). In support of
this view, Boston relies heavily on a footnote in the First
Circuit’s recent Lopez decision, which is worth quoting in full:
The Officers’ failure to explain how a particular alternative would have reduced disparate impact in 2005 and 2008 -- and by how much -- is particularly odd given the obvious mootness of their claim for injunctive relief. Consequently, had the remedy phase of trial proceeded as the Officers would have hoped, each officer would have needed to show that, more likely than not, he or she would have been promoted had Boston used an equally or more valid selection tool with less impact. See 42 U.S.C. § 2000e–5(g)(1) (authorizing “back pay” remedy for Title VII violation); Azimi v. Jordan’s Meats, Inc., 456 F.3d 228, 235 (1st Cir. 2006) (“Injuries allegedly caused by the violation of Title VII . . . must be proven to the factfinder . . . which may reasonably find, within the law, that while there has been [injury], the plaintiff has not been injured in any compensable way by it.”). How any officer could have made such a showing without first securing a liability finding predicated on a specific alternative selection tool that would have been equally or more valid and produced less adverse impact is entirely unclear.
823 F.3d at 121 n.16 (alteration in original).
[8]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 9 of 29
Boston asserts that Lopez’s footnote 16 (“the Lopez footnote”) teaches that “Title VII disparate impact plaintiffs must prove their back pay (and other) damages by showing that ‘but for’ the discrimination they would have obtained some quantifiable financial gain.” Def.’s Br. 16. This demonstrates, Boston argues, that the First Circuit does not read Albemarle as conjuring a substantive “presumption” that would entitle plaintiffs to back pay in the absence of loss proven by a preponderance of the evidence. More pointedly, the Lopez footnote specifies what each individual officer must prove in this case: “that, more likely than not, he or she would have been promoted had Boston used an equally or more valid selection tool with less impact.”
It is understandable that Boston would latch onto this footnote in Lopez. That footnote, however, is plainly dictum, since it addresses a counterfactual scenario. Moreover, because Boston convinced the Lopez Court that its test was sufficiently job-related, the footnote refers to a different theory of liability than the one before this Court in Smith I. Accordingly, the Lopez footnote is dictum that does not bind this Court.1 Still, a district court ordinarily gives great
1 The Lopez footnote cites Azimi, 456 F.3d at 235, but that case was about harassment (not a refusal to hire or promote) and the quoted passage deals with compensatory damages, see 42 U.S.C. § 1981a(a)(1), not Title VII’s equitable remedy of back
[9]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 10 of 29
weight to the dicta of its court of appeals, and that is
especially appropriate when the dictum is uttered in a case so
closely related to the one before the district court. In a
typical scenario, this Court would readily apply the dictum from
Lopez to the present case. The sticking point here is that the
Lopez footnote is contrary to controlling authority. Since this
Court must follow precedent of the Supreme Court that is
pay. The distinction is made clear in an Eighth Circuit case that Azimi relied upon, Bailey v. Runyon, 220 F.3d 879, 882 n.2 (8th Cir. 2000) (“The 1991 amendments [to Title VII], 42 U.S.C. § 1981a, added compensatory and punitive damages relief to the already existing equitable relief . . . .”). Were this a case of intentional discrimination, the Lopez footnote may have correctly stated the burden of proof for compensatory damages under § 1981a(a)(1) -- though it fails to acknowledge the availability of compensation for the lost chance of promotion, even if each plaintiff’s chance was below 50% likelihood. Cf. Alexander v. City of Milwaukee, 474 F.3d 437, 449-51 (7th Cir. 2007). The equitable remedy sought in this case, however, stands on different footing than compensatory damages.
Boston argues that “the Azimi citation makes perfect sense as it reaffirms the First Circuit’s general requirement that damages be proved and the nature of the proof required.” Def.’s Suppl. Mem. 5. Yet, like the Lopez footnote, this argument fails to appreciate two critical distinctions: (1) between harassment and denial of promotion (a probabilistic injury); and (2) between compensatory damages and equitable remedies. Here, back pay is needed to compensate the Officers for their lost chance of being promoted, even if none can prove she would likely have received that promotion. See Biondo v. City of Chicago, 382 F.3d 680, 688 (7th Cir. 2004). Moreover, as an equitable remedy back pay serves other purposes aside from compensation. Indeed, the Supreme Court has explained that back pay is in part a deterrent and thus may sometimes be awarded even when no harm was inflicted (as when after-acquired evidence of an employee’s wrongdoing would otherwise justify the adverse decision). See McKennon v. Nashville Banner Pub. Co., 513 U.S. 352, 362 (1995). The reliance on Azimi in the Lopez footnote is a head-scratcher.
[10]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 11 of 29
inconsistent with the Lopez footnote, it cannot rely on the dictum in Lopez.
The Supreme Court has repeatedly rejected “the basic contention” of the Lopez footnote and Boston here -- namely, “that if the [plaintiff] has not . . . already brought forth specific evidence that each individual was discriminatorily denied an employment opportunity, it must carry that burden at the second, ‘remedial’ stage of trial.” International Bhd. of Teamsters v. United States, 431 U.S. 324, 361 (1977). The Supreme Court repudiated this view, explaining that “proof of the [discriminatory] pattern or practice supports an inference that any particular employment decision, during the period in which the discriminatory policy was in force, was made in pursuit of that policy.” Id. at 362. Thus, a plaintiff “need only show that an alleged individual discriminatee unsuccessfully applied for a job and therefore was a potential victim of the proved discrimination,” at which point “the burden then rests on the employer to demonstrate that the individual applicant was denied an employment opportunity for lawful reasons.” Id. (footnote omitted) (citing Franks v. Bowman Transp. Co., 424 U.S. 747, 773 n.32 (1976)). Applying this reasoning, the Supreme Court expressly stated that “[o]n remand . . . every . . . minority group applicant for a line-driver position will be presumptively entitled to relief, subject to a
[11]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 12 of 29
showing by the company that its earlier refusal to place the
applicant in a line-driver job was not based on its policy of
discrimination.” Id. (footnote omitted) (emphasis supplied).
Since the Officers are “presumptively entitled to relief,”
Albemarle demands that this relief must – once again,
presumptively -- include back pay.2
It is true that Teamsters was a pattern-or-practice
disparate treatment case, not a disparate impact case like the
one before this Court. Thus, it might be thought that disparate
impact plaintiffs would not enjoy this presumption that their
missed promotions are attributable to the unlawful employment
practice. Yet the reasoning of Teamsters undercuts this
argument. The Supreme Court offered three justifications for
this presumption: (1) the plaintiffs’ “prima facie case . . .
create[s] a greater likelihood that any single decision was a
component of the overall pattern”; (2) “the finding of a
2 Somewhat confusingly, there are two separate “presumptions” at work here: Teamsters establishes a presumption that a particular adverse employment decision was the result of a general discriminatory policy and thus warrants individual relief, 431 U.S. at 362, while Albemarle creates a presumption that such relief must include back pay, City of Los Angeles, 435 U.S. at 719. The Officers in this case must rely on the combination of these two presumptions. Though Boston directs its fire at the “so-called Albemarle presumption,” Def.’s Br. 18, its volley would more aptly be aimed at the Teamsters presumption regarding the hiring decision, for that is the presumption that sinks Boston’s boat. Despite filing two briefs on the presumptive damages question, Boston fails to reckon with or even mention Teamsters.
[12]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 13 of 29
[discriminatory] pattern or practice changed the position of the employer to that of a proved wrongdoer”; and (3) “the employer was in the best position to show why any individual employee was denied an employment opportunity.” Teamsters, 431 U.S. at 359 n.45.
All three reasons apply to disparate impact cases such as this one. First, the Officers’ showing that the 2008 exam was unlawful and that they were not promoted after that exam increases the likelihood that each particular promotional failure is a result of that discriminatory exam. Second, Boston has violated the law (though it is admittedly far less culpable as “a proved wrongdoer” than are employers who intentionally discriminated). Finally, Boston has access to the trove of police department records and personnel, leaving it in the best position to proffer some lawful explanation for why an individual Officer would not have been promoted under a lawful selection process. Indeed, the weight of authority points to the conclusion that the Teamsters presumption applies in determining individual relief in disparate impact cases. See, e.g., Chin v. Port Auth. of N.Y. & N.J., 685 F.3d 135, 151-52 (2d Cir. 2012); Isabel v. City of Memphis, 404 F.3d 404, 414-15 (6th Cir. 2005); Crum v. Alabama, 198 F.3d 1305, 1315-16 (11th Cir. 1999); Sledge v. J. P. Stevens & Co., 585 F.2d 625, 637 (4th Cir. 1978); 2 Larson on Employment Discrimination § 31.02
[13]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 14 of 29
(2d ed.) (noting that “it appears to be the rule that the [successful Title VII disparate-impact] plaintiff is entitled to back pay” unless the defendant “can show that the plaintiff was otherwise not qualified, or for some other legitimate reason would not have received the job or the promotion even in the absence of discrimination,” and stating that “[t]his rule is parallel to that found in pattern or practice cases and class action disparate treatment cases”).
Neither can Albemarle itself be reconciled with the Lopez footnote. In the first place, the Lopez footnote appears to draw a significant distinction between injunctive relief and back pay. Albemarle specifically rejected such a theory, for that would “open an enormous chasm between injunctive and backpay relief under Title VII” and “[t]here is nothing on the face of the statute or in its legislative history that justifies the creation of drastic and categorical distinctions between those two remedies.” 422 U.S. at 423.
More fundamentally, the defendant-friendly evidentiary burden that the Lopez footnote seeks to place on successful Title VII plaintiffs, “if applied generally, would . . . frustrate the central statutory purposes of eradicating discrimination throughout the economy and making persons whole for injuries suffered through past discrimination.” Id. at 421. That is precisely what Albemarle forbade courts from doing. The
[14]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 15 of 29
Second Circuit has cogently argued this point, explaining that “to relieve an employer of the burden to show that it would not have hired the plaintiff absent discrimination would contravene part of Congress’s intent in enacting Title VII, in that an employer would be less likely to avoid or eliminate discriminatory employment practices if it knew in advance that it would never have to answer for those practices when faced with a claim for back pay.” EEOC v. Joint Apprenticeship Comm. of Joint Indus. Bd. of Elec. Indus., 186 F.3d 110, 123 (2d Cir. 1999); see Albemarle, 422 U.S. at 417-18 (“If employers faced only the prospect of an injunctive order, they would have little incentive to shun practices of dubious legality. It is the reasonably certain prospect of a backpay award that provide[s] the spur or catalyst which causes employers and unions to selfexamine and to self-evaluate their employment practices . . . .” (citation and internal quotation marks omitted)). Nor would the rule of the Lopez footnote make the plaintiffs whole, as Albemarle requires, since the plaintiffs would need to prove that they would in fact have been promoted -- not simply that they could have been promoted, which is the “position [the employee] would have been in absent the discrimination.” McKennon v. Nashville Banner Pub. Co., 513 U.S. 352, 362 (1995) (citing Franks, 424 U.S. at 764). Under Title VII, the diminished opportunity for employment is a distinct and
[15]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 16 of 29
“concrete injury,” even though it is “speculative” how the
ultimate employment hiring or firing decision will shake out.
Lorance v. AT&T Techs., Inc., 490 U.S. 900, 907 n.3 (1989)
(citing Delaware State Coll. v. Ricks, 449 U.S. 250, 258
(1980)), superseded on other grounds by statute, Civil Rights
Act of 1991, Pub. L. No. 102–166, § 112, 105 Stat. 1071, 1078–
79); cf. Alexander v. City of Milwaukee, 474 F.3d 437, 449 (7th
Cir. 2007) (“[W]hat [the plaintiffs] in fact lost was a chance
to compete on fair footing, not the promotion itself.”). Since
each plaintiff suffered at least a diminished chance of
employment as a result of the discriminatory test, they are all
entitled to make-whole relief for the value of that job
insecurity as a standalone injury.
In short, the test articulated in the Lopez footnote
conflicts with Supreme Court precedent that this Court is not
free to ignore.3 Accordingly, the Court rules that the Officers,
3 The Officers seek further support from a First Circuit case like this one, where disparate impact Title VII liability was predicated on a police-officer selection tool (a height requirement) that was not job-related, in which the panel held “that once a violation of title VII is proven, the successful plaintiff is presumptively entitled to back pay.” Costa v. Markey, 706 F.2d 1, 6 (1st Cir. 1982), reversed on other grounds, 706 F.2d at 10 (1st Cir. 1983) (en banc); Pls.’ Br. 32 (citing Costa). Yet Costa is not particularly helpful here, since the defendants in that case conceded “that plaintiff was rejected for appointment because of the application of the [unlawful] height requirement to her,” and the district court found that “but for application of the height requirement . . . plaintiff would have been on the police force under a valid
[16]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 17 of 29
in showing they were denied promotions on the basis of an unlawful exam, are presumptively entitled to back pay.
B. Boston Has Not Rebutted the Presumption The next question is whether Boston has rebutted the presumption that the Officers would have been promoted but for the unlawful exam and are therefore entitled to back pay. The standard for rebutting the presumption is mired in a decades-old circuit split. See Cohen v. West Haven Bd. of Police Comm’rs, 638 F.2d 496, 502 n.10 (2d Cir. 1980) (citing cases). Some circuits require “the employer to prove by clear and convincing evidence that the plaintiff, even absent the discrimination, would not have been promoted to that particular position.” Johnson v. Brock, 810 F.2d 219, 224 (D.C. Cir. 1987) (citing Day v. Mathews, 530 F.2d 1083, 1085 (D.C. Cir. 1976) (per curiam)). The First Circuit appears to require only “proof by a preponderance of the evidence that the same [employment] decision would have been made absent the discrimination.” Fields v. Clark Univ., 817 F.2d 931, 937 (1st Cir. 1987). Yet the Fields case was discussing establishing liability for the underlying violation, not the issue of damages. Since Fields nevertheless cited the D.C. Circuit’s opinion in Day, which concerned damages, it is possible that Fields did not see a
appointment.” LeBoeuf v. Ramsey, 503 F. Supp. 747, 756, 762 (D. Mass. 1980) (Keeton, J.).
[17]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 18 of 29
meaningful distinction between liability and remedy in setting the standard for rebutting the presumption.4 In this case, it is not necessary to pin down the precise method of rebutting the presumption because Boston has failed under any conceivable standard.
Boston argues that it has rebutted the presumption that the Officers would have been promoted with evidence from the 2014 exam. Def.’s Br. 23-24. Boston’s argument is that the 2014 exam was qualitatively better and yet, surprisingly, “had a much greater adverse impact on black/African American candidates than the 2008 exam.” Id. at 23. On this basis, Boston’s expert “Dr. Silva offered his considered opinion that had a content-valid, multi-component examination like that given in 2014 been used in 2008, it would still have had adverse impact, perhaps significantly so.” Id. at 24. Boston contends that “this fact rebuts any presumption . . . that Plaintiffs suffered any harm whatsoever” from the 2008 promotional exam. Id.
The fatal blow for Boston, however, is a statement that both sides’ expert witnesses agreed with: “One cannot say that any specific individual plaintiff or group of plaintiffs who
4 In Johnson, the D.C. Circuit insisted that the distinction between liability and remedy mattered for the presumption, such that “the clear and convincing proof requirement of Day comes into play only after the plaintiff has established a statutory violation.” 810 F.2d at 224.
[18]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 19 of 29
scored too low to be appointed on the 2014 exam would have scored too low, based on a similar exam, had such an exam been given in 2008.” Trial Tr. Day 2 at 115:1-17. Since the crucial question is whether the 2014 exam reveals how these plaintiffs would have performed on a counterfactually similar exam in 2008, and both experts agreed that the 2014 exam reveals nothing about how these plaintiffs would have performed, the presumption has not been rebutted.5 For argument’s sake, the Court credits the testimony of Dr. Silva, Boston’s expert, that one may infer from the 2014 exam that, had a similar exam been given in 2008, “it would still have [had] adverse impact.” Trial Tr. Day 2 at 120:3-8. Yet that fact does not rebut the presumption, because the critical question is whether these plaintiffs would have done as poorly as they did, and Dr. Silva agrees that the 2014 exam does not answer that question. The 2014 exam results are thus neither here nor there.
Indeed, the Second Circuit confronted a similar issue in Cohen, in which female police officers successfully challenged a physical agility test on the grounds that it had a discriminatory impact and was not job related. 638 F.2d at 499. The district court denied backpay, in part, because the female
5 The Officers further observe that “a majority of the Plaintiffs did not take the 2014 exam,” so its relevance (if any) is limited to those few plaintiffs who took it. Pls.’ Br. 28.
[19]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 20 of 29

police officers failed a newly devised physical agility 17
months after taking the first one. Id. at 499-500. The Second
Circuit reversed, holding that “[t]here was no basis for
concluding, as the district court did, that plaintiffs would
have failed the new test at the earlier time.” Id. at 503. Yet
the Second Circuit also mused about the circumstances in which
this sort of backward projection might be tenable:
The reasonableness of such an inference will vary, depending principally on the nature of the qualities to be measured and the length of time between the refusal and the new test. Thus although a test of general intelligence or knowledge, for example, may have retrospective validity for a substantial period of time, we have no such confidence in a test of physical agility.
Id. at 503 (footnote omitted).
Arguably -- indeed Boston so argues, Def.’s Suppl. Br. 7 --
the Second Circuit might have reached a different result in this
case, since the 2014 exam was largely one of knowledge and
intelligence, rather than physical agility. Yet that seems
highly doubtful. It is eminently plausible that the Officers
could have performed better on the updated exam had they taken
it six years earlier. Julius Caesar was “constant as the Northern Star,”6 and the biblical God is said to “change not,”7
but Title VII does not assume imperial or divine plaintiffs.

6 William Shakespeare, Julius Caesar act 3, sc. 1.

7 Malachi 3:6.

[20]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 21 of 29
People change, as do their circumstances. Illnesses grave and mild, cognitive slippage, dips in professional motivation (perhaps due to earlier disappointments), familial duties, social distractions, and even last night’s sleep or this morning’s breakfast8 may explain why an employee might flunk a test she would have aced years earlier. There is little basis in logic or fact to infer from the 2014 exam that these Officers would have achieved similar mediocre results had they taken it in 2008. Boston has failed to rebut the presumptions described by the Supreme Court in Albemarle and Teamsters. Accordingly, the Court rules that the Officers are entitled to back pay.
C. Chapter 151B Also Entitles the Officers to Back Pay As an alternative ground for an award of back pay, the Officers argue that the Commonwealth’s anti-discrimination statute, Mass. Gen. Laws ch. 151B, entitles them to back pay even were such relief foreclosed under Title VII by the Lopez footnote. Pls.’ Br. 41; see Smith I, 144 F. Supp. 3d at 212 (ruling that Boston violated both Title VII and Chapter 151B). Boston contends, as it did with respect to Title VII, that “any
8 See, e.g., Rachel Galioto & Mary Beth Spitznagel, The Effects of Breakfast and Breakfast Composition on Cognition in Adults, 7 Advances in Nutrition 576S, 580S-81S (Suppl. 2016), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4863263/pdf/an01023 1.pdf (reviewing studies and noting that “the preponderance of work examining delayed recall tasks shows a benefit of eating breakfast”).
[21]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 22 of 29
back pay damages under c. 151B must be proved and not just presumed.” Def.’s Supp. Br. 9.
The Court agrees with the Officers that Chapter 151B independently entitles them to back pay in order to make them whole. As the First Circuit has explained, “Mass. Gen. Laws ch. 151B . . . mandates ‘make-whole relief,’ which encompasses damages which are ‘the natural and probable consequences of the illegal conduct.’” McMillan v. Massachusetts Soc’y for Prevention of Cruelty to Animals United, 140 F.3d 288, 306 (1st Cir. 1998) (quoting Conway v. Electro Switch Corp., 402 Mass. 385, 388, 523 N.E.2d 255 (1988)). Here, the Officers have demonstrated a lost chance of promotion as a result of the discriminatory 2008 exam and they ought be made whole through an award of back pay. The Court so rules.
D. Calculation of the Back Pay Helpfully, the parties have stipulated to the calculation of back pay in Joint Trial Exhibits 86(b)-(c), fixing the lost chance calculation at 36.26% of the total value of each Officer’s back pay. Stipulated Calculations; Def.’s Br. 24-26; Pls.’ Br. 5-7. The stipulation relies on the “loss of a chance doctrine.” Alexander, 474 F.3d at 449; Biondo v. City of Chicago, 382 F.3d 680, 688 (7th Cir. 2004) (“[I]f a person would have had a 25% chance of promotion from lieutenant to captain, then preventing that person from becoming a lieutenant should
[22]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 23 of 29
lead to a remedy equal to 100% of the benefits of being a lieutenant plus 25% of the incremental benefits of being a captain.”). The Court honors the parties’ stipulation.
They did not reach an agreement as to the start and end dates, however, since each side’s expert witness chose different dates. For the following reasons, the Court generally adopts the Officers’ proposed start and end dates, excepting the end dates for those Officers who failed to mitigate damages by taking (and studying for) the improved 2014 exam. The cut-off point for those Officers is the date of the 2014 exam.
1. Start Date “[U]nder a disparate-impact theory of liability, injury occurs and back-pay calculations commence when the employer uses the discriminatory practice for the first time.” Howe v. City of Akron, 801 F.3d 718, 745 (6th Cir. 2015) (citing Lewis v. City of Chicago, 560 U.S. 205, 212 (2010)). The parties have agreed back pay ought begin from the average date of candidate promotion. Trial Tr. Day 1 at 35:1-12; Trial Tr. Day 2 at 112:3-8.9 Yet they disagree as to the pool of candidates from which the average is drawn. The opposing experts differed on this point. Trial Tr. Day 1 at 35:4-12.
9 But see Isabel, 404 F.3d at 410 (affirming back pay award starting from the “date that the first pool of candidates was promoted to lieutenant”).
[23]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 24 of 29
The Officers (following Dr. Weisen) contend that the start date should be fixed at the average promotional date of white candidates only (November 18, 2011), arguing that “[i]t would be patently illogical to adopt Dr. Silva’s approach because the average promotion date of all candidates (including minorities) is premised on a process infected with discrimination.” Pls.’ Br. 40. In contrast, Boston (following Dr. Silva) urges the Court to adopt the average promotional date of all candidates irrespective of race (February 16, 2012). Def.’s Br. 25-26. Dr. Silva indicated that including white and black candidates captured the entire sample and would not “penalize” Boston for promoting black candidates. Trial Tr. Day 2 at 113.
Courts have wide discretion in exercising equitable powers in order to fashion the “most complete relief possible” for a Title VII plaintiff. See Albemarle, 422 U.S. at 421 (quoting 118 Cong. Rec. 7168); see also Franks, 424 U.S. at 764. Exercising this authority, this Court follows Dr. Weisen’s approach in calculating the start date and rules that back pay starts from November 18, 2011. Smith I was clear in finding that the 2008 test had a discriminatory impact on black candidates. 144 F. Supp. 3d at 199. By excluding the lagging promotion dates of black candidates from the calculated average, this Court eliminates the test’s adverse impact on the population of black candidates. Pooling all candidates will
[24]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 25 of 29
perpetuate the effects of the unlawful 2008 exam on black candidates, which the Court declines to do.
2. End Date The end date is when the plaintiff’s economic injury ceased. See Howe, 801 F.3d at 745. The Officers argue that the end dates should be the “retirement date, the trial date for the 8 non-promoted Plaintiffs, and the promotion dates of Plaintiffs Facey and Moss (the 2 Plaintiffs who were promoted).” Pls.’ Br. 16. Boston urges this Court to determine July 4, 2014 to be the end date, because that is the date when the Officers “got the remedy they sought throughout this case -- a content-valid, multicomponent, assessment-center-style exam.” Def.’s Br. 26.10 The 2014 exam was in part a remedial measure to the 2008 exam, but that does not mean that the discriminatory effects of the 2008 exam ended when the new exam was used. A black candidate denied promotion due to the unlawful 2008 exam would continue to be paid a lower salary, and not receive other associated benefits, even after the 2014 exam. It is the duty of this Court to remedy the economic inequity created by the 2008 exam between black and white candidates, and this inequity did not cease on July 4, 2014. See Berkman v. New York, No. CV-
10 “The actual exam administration began on June 28, 2014. [Boston] used the later July 4, 2014 date as the end date because that was end of pay period, greatly simplifying its calculations.” Def.’s Br. 26 n.45.
[25]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 26 of 29
79-1813, 1986 U.S. Dist. LEXIS 25570, at *8-9 (E.D.N.Y. May 13, 1986) (concluding that “the date on which defendants offered to administer the qualifying test to class members” does not necessarily coincide with the end of discrimination because the effects of the discrimination could continue past the remedial qualifying test).
The Court thus agrees with the Officers -- with one caveat explained below -- and rules that back pay ends with either the Officer’s promotion date, retirement date if not promoted, or (if neither promoted nor retired before the trial) the trial date of October 25, 2019.11 Yet for those Officers who did not mitigate damages by taking the 2014 exam, the end date is July 4, 2014. The same goes for Paul Joseph, who did badly on the new exam and, when asked whether he studied for it, testified: “Absolutely not.” Trial Tr. Day 1 at 122:5-6. These Officers failed to mitigate damages and thus their award of back pay will be “offset by any wages that could have been earned with reasonable diligence.” Johnson, 364 F.3d at 379 (citing 42 U.S.C. § 2000e-5(g)(1); Quinones Candelario v. Postmaster Gen. of United States, 906 F.2d 798, 801-02 (1st Cir. 1990)).
11 Though the trial began on October 28, 2019, the parties have agreed to use the date of October 25th for “administrative practicality.” Pls.’ Br. 16 n.9.
[26]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 27 of 29
Accordingly, drawing on the stipulated information in Trial Exhibit 86, Ex. A, this Court determines the following end dates:
1. Brian Latson’s end date is July 4, 2014, for failure to take the new exam. 2. Bruce Smith’s end date is January 2, 2018, when he retired. 3. Kenneth Sousa’s end date is October 25, 2019. 4. Kim Gaddy’s end date is July 4, 2014, for failure to take the new exam. 5. LaTeisha Adams’s end date is March 22, 2015, her date of retirement. Though she did not take the 2014 exam, that was not a failure of reasonable diligence. She reasonably opted not to take the exam because she was on the verge of retirement due to a heart condition. See Trial Tr. Day 2 at 7-8. 6. Martin Joseph’s end date is October 25, 2019. 7. Paul Joseph’s end date is July 4, 2014, for failure to exercise reasonable diligence in mitigating damages by not studying for the new exam. 8. Marwan Moss’s end date is November 8, 2014, when he was promoted. 9. Leighton Facey’s end date is January 26, 2013, the date of promotion.
[27]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 28 of 29
10. William Woodley’s end date is July 4, 2014, for failure to take the new exam.
3. Overtime and Detail Pay and Prejudgment Interest Overtime and detail pay are awarded according to the stipulated calculation through each plaintiff’s end date as determined above, except for Bruce Smith, whose overtime and detail pay end when he was placed on administrative leave on June 28, 2017. See Pls.’ Br. 16 n.9; Trial Ex. 86, Ex. A. The Court further awards the Officers prejudgment interest at a rate of 12% per annum starting from the commencement of the suit. See Mendoza v. Union St. Bus Co., 876 F. Supp. 8, 12 (D. Mass. 1995) (Lasker, J.). III. CONCLUSION For the foregoing reasons, the Court awards back pay to the Officers in accordance with the stipulated calculation using the Officers’ proposed dates, as modified above due to the failure of some plaintiffs to mitigate. It behooves the Court to address one final matter. Though it has properly ruled in favor of the Officers, the Court commends Boston’s efforts in redesigning the promotional exam. Since the 2014 exam turned out to have a greater disparate impact than the 2008 exam, however, Boston argues that awarding back pay here “would cause the City to face a perpetual cycle of potential liability and damages for discrimination without any
[28]

Case 1:12-cv-10291-WGY Document 306 Filed 05/13/20 Page 29 of 29
reasonable alternative selection process.” Def.’s Suppl. Br. 2. The Court rejects the premise that Boston faces a perpetual cycle of liability. Now six years removed, no plaintiff has attacked the 2014 exam and Boston elsewhere asserts that “[t]he content-valid 2014 exam plainly passes muster under Title VII.” Def.’s Br. 26. That being the case -- and the Court has no reason (nor occasion) to disagree -- it is highly unlikely that Boston faces “an endless cycle of damages.” Def.’s Suppl. Br. 3. If ever there emerges a disparate impact challenge to the 2014 exam, Boston would be on much sturdier legal ground than it was in this litigation.
Boston adds that the higher-quality 2014 exam cost more to run than did the 2008 multiple-choice exam, without comparable gains. Id. The law sometimes imposes costs, and it is indeed vexing that the diversity goals of the 2014 exam were not realized here. Yet it does not follow that the 2014 exam also violated the law. Nor does it follow that there was no gain. The benefit of the improved promotional system ultimately redounds to Boston, including its police officers and all its citizens who seek their protection.
SO ORDERED. /s/ William G. Young_ WILLIAM G. YOUNG DISTRICT JUDGE
[29]

 
 Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 1 of 28
United States District Court District of Massachusetts (Boston) CIVIL DOCKET FOR CASE #: 1:12−cv−10291−WGY

Smith et al v. City of Boston Assigned to: Judge William G. Young related Case: 1:19−cv−12170−WGY Cause: 42:2000e Job Discrimination (Employment)
Plaintiff

Date Filed: 02/14/2012 Jury Demand: Plaintiff Nature of Suit: 442 Civil Rights: Jobs Jurisdiction: Federal Question

Bruce Smith

represented by Benjamin Weber Lichten & Liss−Riordan, P.C. 729 Boylston Street Suite 2000 Boston, MA 02116 617−994−5800 Email: bjweber@llrlaw.com ATTORNEY TO BE NOTICED

Harold L. Lichten Lichten & Liss−Riordan, P.C. 729 Boylston Street Suite 2000 Boston, MA 02116 617−994−5800 Fax: 617−994−5801 Email: hlichten@llrlaw.com ATTORNEY TO BE NOTICED

Sara E. Smolik EEOC John F. Kennedy Federal Building Government Center Room 475 Boston, MA 02203−0506 617−565−3207 Fax: 617−565−3196 Email: sara.smolik@eeoc.gov TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill Fair Work, P.C. 192 South Street, Suite 450 Boston, MA 02111 617−607−3260 Email: steve@fairworklaw.com ATTORNEY TO BE NOTICED

Zachary L. Rubin Lichten & Liss−Riordan, P.C. 729 Boylston Street Suite 2000 Boston, MA 02116 914−994−5800 Email: zrubin@llrlaw.com ATTORNEY TO BE NOTICED

Plaintiff Paul Joseph

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 2 of 28

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff John M Johnson

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Robert Tinker

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Martin Joseph

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 3 of 28

(See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Kim Gaddy

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Brian Keith Latson

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Leighton Facey

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 4 of 28

ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Marwan Moss

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill (See above for address) ATTORNEY TO BE NOTICED

Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

Plaintiff Kenneth Sousa

represented by Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Plaintiff William Woodley

represented by Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Plaintiff LaTeisha Adams

represented by Benjamin Weber (See above for address) ATTORNEY TO BE NOTICED

Harold L. Lichten (See above for address) ATTORNEY TO BE NOTICED

Sara E. Smolik (See above for address) TERMINATED: 04/16/2014 ATTORNEY TO BE NOTICED

Stephen S. Churchill

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 5 of 28
(See above for address) ATTORNEY TO BE NOTICED
Zachary L. Rubin (See above for address) ATTORNEY TO BE NOTICED

V. Defendant City of Boston Massachusetts
Third Party Witness

represented by Amy E Condon Bowditch & Dewey, LLP 200 Crossing Boulevard, Suite 300 Framingham, MA 01702 617−757−6522 Fax: 508−872−1492 LEAD ATTORNEY ATTORNEY TO BE NOTICED
John M. Simon Stoneman, Chandler & Miller, LLP 99 High Street Boston, MA 02110 617−542−6789 Fax: 617−556−8989 Email: jsimon@scmllp.com LEAD ATTORNEY ATTORNEY TO BE NOTICED
Kay H. Hodge Stoneham, Chandler & Miller LLP 99 High Street Boston, MA 02110 617−542−6789 Fax: 617−556−8989 Email: KHodge@scmllp.com LEAD ATTORNEY ATTORNEY TO BE NOTICED
Geoffrey R. Bok 1938 East Craftsbury Rd Craftsbury, VT 05826 802−586−2334 Email: geoffreybok@gmail.com TERMINATED: 05/09/2019 ATTORNEY TO BE NOTICED
Nicole I. Taub Boston Police Department One Schroeder Plaza Boston, MA 02120 617−343−4550 ATTORNEY TO BE NOTICED
Thomas H. Costello Stoneham, Chandler & Miller LLP 99 High Street Boston, MA 02110 617−542−6789 Email: tcostello@scmllp.com ATTORNEY TO BE NOTICED

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 6 of 28

Massachusetts Human Resources Division

represented by Robert L. Quinan , Jr. Attorney General's Office One Ashburton Place Room 2019 Boston, MA 02108−1698 617−727−2200 Fax: 617−727−5785 Email: Robert.Quinan@state.ma.us ATTORNEY TO BE NOTICED

Mediator
Judge Robert B. Collings TERMINATED: 06/09/2016

Date Filed 02/14/2012
02/15/2012
02/15/2012
02/15/2012 02/15/2012 03/20/2012 03/20/2012 03/20/2012 03/21/2012 03/21/2012 03/23/2012 03/23/2012

# Docket Text
1 COMPLAINT , COMPENSATORY, INJUNCTIVE AND DECLARATORY RELIEF REQUESTED against City of Boston, Massachusetts Filing fee: $ 350, receipt number 0101−3808749 (Fee Status: Filing Fee paid), filed by Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Civil Cover Sheet, # 2 Exhibit Category form)(Lichten, Harold) (Entered: 02/14/2012)
ELECTRONIC NOTICE of Case Assignment. Judge Joseph L. Tauro assigned to case. If the trial Judge issues an Order of Reference of any matter in this case to a Magistrate Judge, the matter will be transmitted to Magistrate Judge Robert B. Collings. (Abaid, Kimberly) (Entered: 02/15/2012)
2 Summons Issued as to City of Boston. Counsel receiving this notice electronically should download this summons, complete one for each defendant and serve it in accordance with Fed.R.Civ.P. 4 and LR 4.1. Summons will be mailed to plaintiff(s) not receiving notice electronically for completion of service. (Paine, Matthew) (Entered: 02/15/2012)
3 NOTICE of Appearance by Stephen S. Churchill on behalf of Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker (Churchill, Stephen) (Entered: 02/15/2012)
4 NOTICE of Appearance by Sara E. Smolik on behalf of Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker (Smolik, Sara) (Entered: 02/15/2012)
5 NOTICE of Appearance by Amy E Ambarik on behalf of City of Boston (Ambarik, Amy) (Entered: 03/20/2012)
6 NOTICE of Appearance by Nicole I. Taub on behalf of City of Boston (Taub, Nicole) (Entered: 03/20/2012)
7 First MOTION for Extension of Time to April 27, 2012 to File Responsive Pleading by City of Boston.(Taub, Nicole) (Entered: 03/20/2012)
Judge Joseph L. Tauro: ELECTRONIC ORDER entered ALLOWED 7 Motion for Extension of Time. City of Boston answer due by 4/27/2012. (Geraldino−Karasek, Clarilde) (Entered: 03/21/2012)
Reset Answer Deadlines for City of Boston. Answer due by 4/27/2012. (Geraldino−Karasek, Clarilde) (Entered: 03/21/2012)
8 AMENDED COMPLAINT against City of Boston, filed by Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Churchill, Stephen) (Entered: 03/23/2012)
9 SUMMONS Returned Executed All Defendants. (Lichten, Harold) (Entered: 03/23/2012)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 7 of 28

04/20/2012 04/27/2012 04/30/2012 04/30/2012 04/30/2012 05/01/2012 09/04/2012
09/04/2012 09/04/2012 10/02/2012 10/02/2012
10/02/2012 10/15/2012
10/17/2012
10/22/2012
05/29/2013 05/29/2013 05/29/2013

10 Joint MOTION to Stay Proceedings by City of Boston.(Taub, Nicole) (Entered: 04/20/2012)
11 Second MOTION for Extension of Time to 05/12/12 to File Responsive Pleading by City of Boston.(Taub, Nicole) (Entered: 04/27/2012)
Judge Joseph L. Tauro: ELECTRONIC ORDER entered granting 11 Motion for Extension of Time (Lovett, Zita) (Entered: 04/30/2012)
Reset Answer Deadlines for City of Boston. Answer due by 05/12/2012 (Geraldino−Karasek, Clarilde) (Entered: 04/30/2012)
12 Judge Joseph L. Tauro: ENDORSED ORDER entered ALLOWED 10 Motion to Stay Proceedings (Geraldino−Karasek, Clarilde) (Entered: 05/01/2012)
Filing of the Responsive Pleadings due in 30 Days after receipt of the Plaintiffs' Notice of Right to Sue Letters. (Geraldino−Karasek, Clarilde) (Entered: 05/01/2012)
13 NOTICE by Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith of Amended Complaint (Attachments: # 1 Exhibit A (Second Amended Complaint))(Churchill, Stephen) (Entered: 09/04/2012)
14 AMENDED COMPLAINT against All Defendants, filed by Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Churchill, Stephen) (Entered: 09/04/2012)
21 Judge Joseph L. Tauro: ORDER entered. ORDER LIFTING STAY as 14 Amended Complaint; 17 Answer filed. (Geraldino−Karasek, Clarilde) (Entered: 05/29/2013)
15 First MOTION TO DISMISS FOR FAILURE TO STATE A CLAIM for Plaintiffs Lateisha Adams, Leighton Facey, Robert Tinker and John M. Johnson by City of Boston.(Taub, Nicole) (Entered: 10/02/2012)
16 MEMORANDUM in Support re 15 First MOTION TO DISMISS FOR FAILURE TO STATE A CLAIM for Plaintiffs Lateisha Adams, Leighton Facey, Robert Tinker and John M. Johnson filed by City of Boston. (Attachments: # 1 Exhibit, # 2 Exhibit, # 3 Exhibit, # 4 Exhibit, # 5 Exhibit, # 6 Exhibit, # 7 Exhibit, # 8 Exhibit)(Taub, Nicole) (Entered: 10/02/2012)
17 ANSWER to 14 Amended Complaint by City of Boston.(Taub, Nicole) (Entered: 10/02/2012)
18 Assented to MOTION for Extension of Time to October 22, 2012 to File Response/Reply as to 15 First MOTION TO DISMISS FOR FAILURE TO STATE A CLAIM for Plaintiffs Lateisha Adams, Leighton Facey, Robert Tinker and John M. Johnson by LaTeisha Adams, Leighton Facey, John M Johnson, Robert Tinker.(Churchill, Stephen) (Entered: 10/15/2012)
19 Judge Joseph L. Tauro: ELECTRONIC ORDER entered ALLOWED 18 Motion for Extension of Time to File Response/Reply re 15 First MOTION TO DISMISS FOR FAILURE TO STATE A CLAIM for Plaintiffs Lateisha Adams, Leighton Facey, Robert Tinker and John M. Johnson Responses due by 10/22/2012 (Geraldino−Karasek, Clarilde) (Entered: 10/17/2012)
20 Opposition re 15 First MOTION TO DISMISS FOR FAILURE TO STATE A CLAIM for Plaintiffs Lateisha Adams, Leighton Facey, Robert Tinker and John M. Johnson filed by LaTeisha Adams, Leighton Facey, John M Johnson, Robert Tinker. (Churchill, Stephen) (Entered: 10/22/2012)
22 NOTICE of Scheduling Conference set for 6/19/2013 11:00 AM in Courtroom 22 before Judge Joseph L. Tauro. (Geraldino−Karasek, Clarilde) (Entered: 05/29/2013)
23 Judge Joseph L. Tauro: RULE 26 DISCOVERY ORDER entered. (Geraldino−Karasek, Clarilde) (Entered: 05/29/2013)
24 NOTICE of Appearance by Benjamin Weber on behalf of LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker (Weber, Benjamin) (Entered: 05/29/2013)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 8 of 28

06/12/2013 06/12/2013
07/24/2013
07/24/2013 07/30/2013
07/30/2013 07/30/2013 08/02/2013 08/02/2013 08/06/2013
08/07/2013
08/30/2013 09/17/2013 09/17/2013 09/17/2013 09/17/2013 09/17/2013

25 NOTICE of Appearance by John M. Simon on behalf of City of Boston (Simon, John) (Entered: 06/12/2013)
26 ELECTRONIC NOTICE OF RESCHEDULING: Scheduling Conference set for 8/6/2013 11:30 AM in Courtroom 22 before Judge Joseph L. Tauro. The deadlines for the filings called for in the scheduling order are extended accordingly.(Lovett, Zita) Modified on 6/12/2013 (Lovett, Zita). (Entered: 06/12/2013)
27 Judge Joseph L. Tauro: ORDER entered. For the reasons stated in the accompanying memorandum, this court hereby orders that Defendant's Partial Motion to Dismiss 15 is ALLOWED IN PART and DENIED IN PART. The claims of Plaintiffs Johnson and Tinker are DISMISSED WITHOUT PREJUDICE for failure to exhaust their administrative remedies. IT IS SO ORDERED (Geraldino−Karasek, Clarilde) (Entered: 07/24/2013)
28 Judge Joseph L. Tauro: ORDER entered. MEMORANDUM and ORDER (Geraldino−Karasek, Clarilde) (Entered: 07/24/2013)
29 STATEMENT OF COUNSEL of Harold Lichten Joint Statement by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 07/30/2013)
30 NOTICE by City of Boston Proposed List of Deponents (Simon, John) (Entered: 07/30/2013)
31 NOTICE by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker List of Proposed Deponents (Weber, Benjamin) (Entered: 07/30/2013)
32 ELECTRONIC NOTICE of Hearing. Status Conference set for 8/14/2013 12:00 PM in Courtroom 22 before Judge Joseph L. Tauro. (Lovett, Zita) (Entered: 08/02/2013)
33 ELECTRONIC NOTICE Canceling Hearing of 8/14/2013, wrong case. Thank you for the call. (Lovett, Zita) (Entered: 08/02/2013)
34 ELECTRONIC Clerk's Notes for proceedings held before Judge Joseph L. Tauro: Scheduling Conference held on 8/6/2013. An order will issue. (Court Reporter: Carol Scott at 617−330−1377.)(Attorneys present: Lechten,Churchill,Webb,plts; Simon,Taub,deft.t) (Lovett, Zita) (Entered: 08/07/2013)
35 Judge Joseph L. Tauro: SCHEDULING ORDER entered. After a Scheduling Conference on August 6, 2013, this court hereby orders that: The parties may conduct the following depositions included in this ORDER. All depositions shall be completed by December 31, 2013 ( Discovery to be completed by 12/31/2013). No further discovery is permitted without leave of this court. Plaintiffs shall file their MOTION for class certification by October 11, 2013 ( Motions due by 10/11/2013). A Further Conference set for 1/9/2014 11:00 AM in Courtroom 22 before Judge Joseph L. Tauro. IT IS SO ORDERED. (Geraldino−Karasek, Clarilde) (Entered: 08/07/2013)
36 CERTIFICATION pursuant to Local Rule 16.1 by City of Boston.(Simon, John) (Main Document 36 replaced on 8/30/2013 to correct PDF document attached ) (Geraldino−Karasek, Clarilde). (Entered: 08/30/2013)
37 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Brian Keith Latson. (Weber, Benjamin) (Entered: 09/17/2013)
38 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Bruce Smith. (Weber, Benjamin) (Entered: 09/17/2013)
39 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Kim Gaddy. (Weber, Benjamin) (Entered: 09/17/2013)
40 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Lateisha Adams. (Weber, Benjamin) (Entered: 09/17/2013)
41 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Leighton Facey. (Weber, Benjamin) (Entered: 09/17/2013)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 9 of 28

09/17/2013 09/17/2013 09/17/2013 10/09/2013 10/16/2013 10/18/2013
10/31/2013 11/04/2013
11/15/2013 11/25/2013 11/26/2013
11/26/2013
12/18/2013 12/26/2013 12/30/2013 01/14/2014

42 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Martin Joseph. (Weber, Benjamin) (Entered: 09/17/2013)
43 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Marwan Moss. (Weber, Benjamin) (Entered: 09/17/2013)
44 CERTIFICATION pursuant to Local Rule 16.1 of Plaintiff Paul Joseph. (Weber, Benjamin) (Entered: 09/17/2013)
45 Assented to MOTION for Extension of Time to October 18, 2013 to File Plaintiffs' Motion for Class Certification by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Weber, Benjamin) (Entered: 10/09/2013)
46 Judge Joseph L. Tauro: ELECTRONIC ORDER entered granting 45 Motion for Extension of Time to File (Lovett, Zita) (Entered: 10/16/2013)
47 MOTION to Certify Class and Memorandum in Support Thereof by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D, # 5 Exhibit E)(Weber, Benjamin) (Entered: 10/18/2013)
48 Assented to MOTION for Extension of Time to November 15, 2013 to File Response/Reply as to 47 MOTION to Certify Class and Memorandum in Support Thereof by City of Boston.(Simon, John) (Entered: 10/31/2013)
49 Judge Joseph L. Tauro: ELECTRONIC ORDER entered granting 48 Motion for Extension of Time to File Response/Reply re 48 Assented to MOTION for Extension of Time to November 15, 2013 to File Response/Reply as to 47 MOTION to Certify Class and Memorandum in Support Thereof Responses due by 11/15/2013 (Lovett, Zita) (Entered: 11/04/2013)
50 Opposition re 47 MOTION to Certify Class and Memorandum in Support Thereof filed by City of Boston. (Simon, John) (Entered: 11/15/2013)
51 Assented to MOTION for Leave to File Reply on Class Certification by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit 1, # 2 Exhibit A, # 3 Exhibit B)(Weber, Benjamin) (Entered: 11/25/2013)
52 Judge Joseph L. Tauro: ELECTRONIC ORDER entered granting 51 Motion for Leave to File Document ; Counsel using the Electronic Case Filing System should now file the document for which leave to file has been granted in accordance with the CM/ECF Administrative Procedures. Counsel must include − Leave to file granted on (date of order)− in the caption of the document. (Lovett, Zita) (Entered: 11/26/2013)
53 REPLY to Response to 47 MOTION to Certify Class and Memorandum in Support Thereof filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Weber, Benjamin) (Entered: 11/26/2013)
54 ELECTRONIC NOTICE Canceling Hearing of 1/9/2014. A new date will be announced for a hearing on the pending motions. (Lovett, Zita) (Entered: 12/18/2013)
56 ELECTRONIC NOTICE of Reassignment. Pursuant to Judge Joseph L. Tauro's taking of senior status, this case is randomly reassigned to Judge William G. Young.(Abaid, Kimberly) (Entered: 12/26/2013)
57 ELECTRONIC NOTICE of Hearing. Status Conference set for 1/14/2014 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/30/2013)
58 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Status Conference held on 1/14/2014. The Court sets the pending motion 47 MOTION to Certify Class for hearing on 2/11/2014 at 2:00 PM before Judge William G. Young. This hearing will take place at Boston College Law School, 885 Centre Street, East Wing, Room 115, Newton, MA. Bench Trial is placed on the running trial list for

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 10 of 28

01/16/2014 01/30/2014 02/03/2014 02/03/2014 02/06/2014 02/07/2014 02/07/2014 02/13/2014 02/14/2014 02/27/2014 03/25/2014
04/15/2014 08/14/2014
08/14/2014

December 2014. A final pretrial conference will be set during the month of November 2014. A joint final pretrial memorandum is due the first business day of November 2014. Any motions for summary judgment are due on or before the first business day of September 2014. (Ready for Trial on 12/1/2014 09:00 AM in Courtroom 18 before Judge William G. Young.). (Court Reporter: Donald Womack at womack@megatran.com.)(Attorneys present: Churchill, Lichten for the plaintiffs, Simon for the defendant) (Gaudet, Jennifer) (Entered: 01/15/2014)
59 MOTION for Reconsideration of Dismissal by Bruce Smith.(Lichten, Harold) (Entered: 01/16/2014)
60 Assented to MOTION for Extension of Time to February 6, 2014 to File Opposition to Motion for Reconsideration by City of Boston.(Simon, John) (Entered: 01/30/2014)
61 Judge William G. Young: ELECTRONIC ORDER entered granting 60 Assented to MOTION for Extension of Time to February 6, 2014 to File Opposition to Motion for Reconsideration (Paine, Matthew) (Entered: 02/03/2014)
62 Reset Deadlines as to 59 MOTION for Reconsideration of Dismissal. Responses due by 2/6/2014 (Paine, Matthew) (Entered: 02/03/2014)
63 Joint MOTION for Extension of Time to February 13, 2014 to File Opposition to Motion for Reconsideration and to Continue Class Certification Hearing by City of Boston.(Simon, John) (Entered: 02/06/2014)
64 Judge William G. Young: ELECTRONIC ORDER entered granting 63 Joint Motion for Extension of Time to February 13, 2014 to File Opposition to Motion for Reconsideration and to Continue Class Certification Hearing by City of Boston. (Gaudet, Jennifer) (Entered: 02/07/2014)
65 ELECTRONIC NOTICE Resetting Hearing on Motion 47 MOTION to Certify Class and Memorandum in Support Thereof : Motion Hearing set for 3/6/2014 02:00 PM before Judge William G. Young. This hearing will take place at Suffolk Law School, 120 Tremont Street, 4th Floor, Boston, MA. (Gaudet, Jennifer) (Entered: 02/07/2014)
66 Opposition re 59 MOTION for Reconsideration of Dismissal filed by City of Boston. (Simon, John) (Entered: 02/13/2014)
67 Judge William G. Young: ELECTRONIC ORDER entered denying 59 MOTION for Reconsideration of Dismissal (Paine, Matthew) (Entered: 02/14/2014)
68 ELECTRONIC NOTICE Setting Hearing on Motion 47 MOTION to Certify Class and Memorandum in Support Thereof : Motion Hearing set for 3/25/2014 02:00 PM before Judge William G. Young. This hearing will take place at Suffolk Law School, 120 Tremont Street, 4th Floor, Boston, MA. (Gaudet, Jennifer) (Entered: 02/27/2014)
70 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Motion Hearing held at Suffolk Law School on 3/25/2014 re 47 MOTION to Certify Class and Memorandum in Support Thereof filed by John M Johnson, Robert Tinker, Bruce Smith, Paul Joseph, Kim Gaddy, Brian Keith Latson, LaTeisha Adams, Leighton Facey, Martin Joseph, Marwan Moss. After hearing from counsel, the Court enters an Order denying without prejudice 47 Motion to Certify Class. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Liss−Riordan and Churchill for the plaintiff, Simon for the defendant) (Gaudet, Jennifer) Modified on 4/11/2014 (Gaudet, Jennifer). (Entered: 04/11/2014)
71 NOTICE of Withdrawal of Appearance by Sara E. Smolik (Smolik, Sara) (Entered: 04/15/2014)
72 MOTION to Compel and Enforce Deposition and Document Subpoenas by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit Affidavit of Ben Weber, # 2 Exhibit Deposition Subpoenas, # 3 Exhibit Proofs of service)(Lichten, Harold) (Entered: 08/14/2014)
73 CERTIFICATE OF SERVICE pursuant to LR 5.2 by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker re 72 MOTION to Compel and Enforce

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 11 of 28

08/28/2014 08/29/2014 09/10/2014 09/11/2014 09/17/2014 10/10/2014 10/23/2014 10/31/2014 10/31/2014 11/03/2014 11/04/2014 11/10/2014 11/10/2014 11/10/2014
11/17/2014

Deposition and Document Subpoenas . (Lichten, Harold) (Entered: 08/14/2014)
74 Assented to MOTION for Extension of Time to 9/11/2014 to Respond to Paper No. 72 (Plaintiffs' Discovery Motion directed at HRD) by Massachusetts Human Resources Division.(Quinan, Robert) Modified on 8/29/2014 (Gaudet, Jennifer). (Entered: 08/28/2014)
75 Judge William G. Young: ELECTRONIC ORDER entered granting 74 Assented to Motion for Extension of Time to File Response/Reply re 72 MOTION to Compel and Enforce Deposition and Document Subpoenas. Responses due by 9/11/2014. (Gaudet, Jennifer) (Entered: 08/29/2014)
76 Assented to MOTION for Extension of Time to 09/18/2014 to File Response/Reply as to 72 MOTION to Compel and Enforce Deposition and Document Subpoenas by Massachusetts Human Resources Division.(Quinan, Robert) (Entered: 09/10/2014)
77 Judge William G. Young: ELECTRONIC ORDER entered granting 76 Motion for Extension of Time to File Response/Reply re 72 MOTION to Compel and Enforce Deposition and Document Subpoenas Responses due by 9/18/2014 (Paine, Matthew) (Entered: 09/11/2014)
78 NOTICE of Change of Address or Firm Name by Harold L. Lichten (Lichten, Harold) (Main Document 78 replaced on 9/18/2014) (Gaudet, Jennifer). (Entered: 09/17/2014)
79 Judge William G. Young: ELECTRONIC ORDER entered withdrawing as moot 72 Motion to Compel. Counsel represents the motion is withdrawn as moot. (Gaudet, Jennifer) (Entered: 10/10/2014)
80 Judge William G. Young: ORDER entered. PROCEDURAL ORDER re pretrial/trial. Final Pretrial Conference set for 11/10/2014 02:00 PM in Courtroom 18 before Judge William G. Young.(Gaudet, Jennifer) (Entered: 10/23/2014)
81 Emergency MOTION to Continue PreTrial Conference by City of Boston. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Simon, John) (Entered: 10/31/2014)
82 Opposition re 81 Emergency MOTION to Continue PreTrial Conference filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 10/31/2014)
83 PRETRIAL MEMORANDUM by Bruce Smith. (Attachments: # 1 Exhibit A)(Lichten, Harold) (Entered: 11/03/2014)
84 Judge William G. Young: ELECTRONIC ORDER entered denying 81 Emergency Motion to Continue PreTrial Conference by City of Boston. (Gaudet, Jennifer) (Entered: 11/04/2014)
85 Objection by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker to Defendants' Witnesses and Exhibits. (Lichten, Harold) (Entered: 11/10/2014)
86 Objection by City of Boston to Plaintiffs' Witnesses and Exhibits. (Simon, John) (Entered: 11/10/2014)
98 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Final Pretrial Conference held (lobby) on 11/10/2014. The Court inquires as to settlement. Any summary judgment motions shall be filed within one week. No response to motions is necessary. A hearing on motion(s) for summary judgment is set for 12/1/2014 at 2:00 PM. Case is bifurcated. Jury waived trial expected to begin December 10th or 11th and is expected to last 10 days (2 weeks). If plaintiff's counsel's case in another district goes forward, this case will be postponed. The Court goes over the requirements of the session regarding time limits, exhibits, etc. (Court Reporter: None.)(Attorneys present: Churchill, Lichten, Webber for the plaintiff, Bok and Simon for the defendants) (Gaudet, Jennifer) (Entered: 11/18/2014)
87 NOTICE of Appearance by Kay H. Hodge on behalf of City of Boston (Hodge, Kay) (Entered: 11/17/2014)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 12 of 28

11/17/2014 11/17/2014 11/17/2014 11/17/2014 11/17/2014 11/17/2014 11/17/2014 11/17/2014
11/17/2014
11/17/2014 11/18/2014 11/21/2014
11/24/2014 11/24/2014
11/24/2014
11/24/2014
11/24/2014

88 NOTICE of Appearance by Geoffrey R. Bok on behalf of City of Boston (Bok, Geoffrey) (Entered: 11/17/2014)
89 MOTION for Summary Judgment by City of Boston.(Simon, John) (Entered: 11/17/2014)
90 MEMORANDUM in Support re 89 MOTION for Summary Judgment filed by City of Boston. (Simon, John) (Entered: 11/17/2014)
91 Statement of Material Facts L.R. 56.1 re 89 MOTION for Summary Judgment filed by City of Boston. (Simon, John) (Entered: 11/17/2014)
92 AFFIDAVIT in Support re 89 MOTION for Summary Judgment by John M. Simon. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Simon, John) (Entered: 11/17/2014)
93 AFFIDAVIT in Support re 89 MOTION for Summary Judgment by Rick R. Jacobs. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Simon, John) (Entered: 11/17/2014)
94 MOTION for Summary Judgment by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 11/17/2014)
95 MEMORANDUM in Support re 94 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A)(Lichten, Harold) (Entered: 11/17/2014)
96 Statement of Material Facts L.R. 56.1 re 94 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 11/17/2014)
97 AFFIDAVIT in Support re 96 Statement of Material Facts L.R. 56.1, . (Attachments: # 1 Exhibit 1)(Lichten, Harold) (Entered: 11/17/2014)
99 Set/Reset Deadlines as to 94 MOTION for Summary Judgment , 89 MOTION for Summary Judgment . Motion Hearing set for 12/1/2014 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 11/18/2014)
100 MOTION to Strike 93 Affidavit in Support of Defendant's Motion for Summary Judgment by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A−1, # 2 Exhibit A−2, # 3 Exhibit A−3)(Weber, Benjamin) (Entered: 11/21/2014)
101 Opposition re 89 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Churchill, Stephen) (Entered: 11/24/2014)
102 Counter Statement of Material Facts L.R. 56.1 re 89 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Churchill, Stephen) (Entered: 11/24/2014)
103 Second AFFIDAVIT of Joel P. Wiesen in Opposition re 89 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Churchill, Stephen) (Entered: 11/24/2014)
104 AFFIDAVIT of Stephen Churchill in Opposition re 89 MOTION for Summary Judgment filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D, # 5 Exhibit E, # 6 Exhibit F, # 7 Exhibit G, # 8 Exhibit H)(Churchill, Stephen) (Entered: 11/24/2014)
105 Opposition re 94 MOTION for Summary Judgment filed by City of Boston. (Simon, John) (Entered: 11/24/2014)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 13 of 28

11/24/2014 11/24/2014 11/24/2014 11/25/2014
11/25/2014 11/25/2014 11/26/2014 11/26/2014 11/26/2014 11/26/2014 11/26/2014 11/29/2014 11/29/2014
12/01/2014 12/01/2014

106 Counter Statement of Material Facts L.R. 56.1 re 94 MOTION for Summary Judgment by Plaintiffs filed by City of Boston. (Simon, John) (Entered: 11/24/2014)
107 AFFIDAVIT in Support re 105 Opposition to Motion for Summary Judgment by Plaintiffs. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Simon, John) (Entered: 11/24/2014)
108 AFFIDAVIT in Support re 105 Opposition to Motion for Summary Judgment by Plaintiffs. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D)(Simon, John) (Entered: 11/24/2014)
109 ELECTRONIC NOTICE issued requesting courtesy copy for 106 Statement of Material Facts L.R. 56.1, 102 Statement of Material Facts L.R. 56.1, 105 Opposition to Motion, 103 Affidavit in Opposition to Motion, 108 Affidavit in Support, 104 Affidavit in Opposition to Motion, 101 Opposition to Motion, 107 Affidavit in Support. Counsel who filed this document are requested to submit a courtesy copy of this document (or documents) to the Clerk's Office attention Matt Paine − Docket Clerk − Judge Young. These documents must be clearly marked as a Courtesy Copy and reflect the document number assigned by CM/ECF. (Paine, Matthew) (Entered: 11/25/2014)
110 MOTION to Strike Affidavit of Jacintho Silva by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A)(Weber, Benjamin) (Entered: 11/25/2014)
111 MOTION to Strike Affidavit of Michael A. Campion by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A)(Weber, Benjamin) (Entered: 11/25/2014)
112 MOTION to Strike Plaintiffs' Expert Witness by City of Boston.(Simon, John) (Entered: 11/26/2014)
113 Opposition re 100 MOTION to Strike 93 Affidavit in Support of Defendant's Motion for Summary Judgment filed by City of Boston. (Simon, John) (Entered: 11/26/2014)
114 Opposition re 110 MOTION to Strike Affidavit of Jacintho Silva filed by City of Boston. (Simon, John) (Entered: 11/26/2014)
115 Opposition re 111 MOTION to Strike Affidavit of Michael A. Campion filed by City of Boston. (Simon, John) (Entered: 11/26/2014)
116 MOTION to Amend 83 Pretrial Memorandum re Defendant's Trial Witness List by City of Boston.(Simon, John) (Entered: 11/26/2014)
117 Opposition re 112 MOTION to Strike Plaintiffs' Expert Witness filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 11/29/2014)
118 AFFIDAVIT re 117 Opposition to Motion, of Benjamin J. Weber by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D, # 5 Exhibit E, # 6 Exhibit F, # 7 Exhibit G, # 8 Exhibit H, # 9 Exhibit I, # 10 Exhibit J, # 11 Exhibit K)(Lichten, Harold) (Entered: 11/29/2014)
119 MOTION to Strike Report of Michael A. Campion by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A)(Lichten, Harold) (Entered: 12/01/2014)
120 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Motion Hearing held on 12/1/2014 re 94 MOTION for Summary Judgment and 89 MOTION for Summary Judgment. The Court addresses the trial schedule. Due to counsel on trial in another district this matter is continued to Monday, December 15, 2014 at 9:00. After hearing from counsel, the Court enters an Order tentatively denying the pending motions for summary judgment. (Bench Trial Day One set for

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 14 of 28

12/02/2014 12/03/2014 12/05/2014
12/05/2014 12/05/2014 12/05/2014 12/05/2014 12/05/2014
12/05/2014 12/07/2014 12/07/2014 12/07/2014
12/09/2014
12/09/2014

12/15/2014 09:00 AM in Courtroom 18 before Judge William G. Young., Bench Trial Day Two set for 12/16/2014 09:00 AM in Courtroom 18 before Judge William G. Young., Bench Trial Day Three set for 12/17/2014 09:30 AM in Courtroom 18 before Judge William G. Young., Bench Trial Day Four set for 12/18/2014 09:00 AM in Courtroom 18 before Judge William G. Young., Bench Trial Day Five set for 12/19/2014 09:00 AM in Courtroom 18 before Judge William G. Young.). (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Churchill, Lichten for the plaintiffs, Simon, Bok for the deft.) (Gaudet, Jennifer) (Entered: 12/02/2014)
121 Judge William G. Young: ELECTRONIC ORDER entered denying 119 MOTION to Strike Report of Michael A. Campion. (Paine, Matthew) (Entered: 12/02/2014)
122 Joint MOTION for Order to Postpone Trial To Allow Parties To Mediate by City of Boston.(Simon, John) (Entered: 12/03/2014)
123 Judge William G. Young: ELECTRONIC ORDER entered: denied. Mediation is always desirable but it may take place during the trial if not before re 122 Joint MOTION for Order to Postpone Trial To Allow Parties To Mediate. (Paine, Matthew) (Entered: 12/05/2014)
124 Emergency MOTION to Permit Counsel's Attendance at Deposition by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith.(Churchill, Stephen) (Entered: 12/05/2014)
125 MOTION Opposition to Emergency Motion and Cross Motion for Protective Order re 124 Emergency MOTION to Permit Counsel's Attendance at Deposition of Dr. Jacinto Silva by City of Boston.(Bok, Geoffrey) (Entered: 12/05/2014)
126 Letter/request (non−motion) from Harold Lichten re: Case Status. (Lichten, Harold) (Entered: 12/05/2014)
127 Emergency MOTION for Protective Order Regarding Plaintiffs' Demand to Depose Rick Jacobs, Ph.D − An Expert Witness Who Is Not Testifying At Trial For Either Party by City of Boston.(Bok, Geoffrey) (Entered: 12/05/2014)
128 Emergency MOTION to Take Deposition of Dr. Rick Jacobs by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit Affidavit of Harold Lichten)(Lichten, Harold) (Entered: 12/05/2014)
129 MOTION Protection From Trial Being Held From December 23, 2014 to Janaury 4, 2015 Due to Trial Counsels' Previously Scheduled Family Vacations (Unopposed by Plainitffs) by City of Boston.(Bok, Geoffrey) (Entered: 12/05/2014)
130 MEMORANDUM in Opposition re 128 Emergency MOTION to Take Deposition of Dr. Rick Jacobs filed by City of Boston. (Bok, Geoffrey) (Entered: 12/07/2014)
131 MOTION in Limine to Exclude Any Use of the "Four−Fifths Rule" to Prove or Disprove Whether There Was an Adverse Impact Based on Race by City of Boston.(Bok, Geoffrey) (Entered: 12/07/2014)
132 MOTION in Limine to Exclude From Trial Any Argument That Any of the Eight Current Plaintiffs Have a Viable Disparate Impact Liability Claim Based on Their Taking the City's 2005 Lieutenant Promotional Examination by City of Boston. (Attachments: # 1 Exhibit A)(Bok, Geoffrey) (Entered: 12/07/2014)
133 Judge William G. Young: ELECTRONIC ORDER entered: The deposition of Jacinto Silva shall take place live in Miami on Thursday, December 11, 2014. All counsel may be present and may interrogate in person. re 124 Emergency MOTION to Permit Counsel's Attendance at Deposition (Paine, Matthew) (Entered: 12/09/2014)
134 Judge William G. Young: ELECTRONIC ORDER entered: Dr. Jacobs need not be deposed. re 127 Emergency MOTION for Protective Order Regarding Plaintiffs' Demand to Depose Rick Jacobs, Ph.D − An Expert Witness Who Is Not Testifying At Trial For Either Party (Paine, Matthew) (Entered: 12/09/2014)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 15 of 28

12/09/2014 12/09/2014
12/09/2014 12/11/2014 12/12/2014 12/12/2014
12/12/2014 12/14/2014 12/14/2014
12/14/2014 12/15/2014 12/15/2014

135 Emergency MOTION to Bifurcate the Trial on Liability in Accordance with Rule 26(a) by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 12/09/2014)
136 MEMORANDUM in Support re 135 Emergency MOTION to Bifurcate the Trial on Liability in Accordance with Rule 26(a) filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit Email with Silva Report, # 2 Exhibit Email with Campion Report)(Lichten, Harold) (Entered: 12/09/2014)
137 Opposition re 135 Emergency MOTION to Bifurcate the Trial on Liability in Accordance with Rule 26(a) filed by City of Boston. (Simon, John) (Entered: 12/09/2014)
138 Judge William G. Young: ELECTRONIC ORDER entered denying 135 Emergency MOTION to Bifurcate the Trial on Liability in Accordance with Rule 26(a). (Paine, Matthew) (Entered: 12/11/2014)
139 Judge William G. Young: ELECTRONIC ORDER entered granting 129 Motion for Protection From Trial Being Held From December 23, 2014 to Janaury 4, 2015 Due to Trial Counsels' Previously Scheduled Family Vacations (Unopposed by Plainitffs) by City of Boston. (Gaudet, Jennifer) (Entered: 12/12/2014)
140 Set/Reset Hearings: Bench Trial Day One set for 12/15/2014 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Two set for 12/16/2014 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Three set for 12/17/2014 09:30 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Four set for 12/18/2014 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Five set for 12/19/2014 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Six set for 12/22/2014 09:00 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/12/2014)
141 Proposed Findings of Fact by City of Boston. (Simon, John) (Entered: 12/12/2014)
142 Opposition re 131 MOTION in Limine to Exclude Any Use of the "Four−Fifths Rule" to Prove or Disprove Whether There Was an Adverse Impact Based on Race filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A)(Churchill, Stephen) (Entered: 12/14/2014)
143 Opposition re 132 MOTION in Limine to Exclude From Trial Any Argument That Any of the Eight Current Plaintiffs Have a Viable Disparate Impact Liability Claim Based on Their Taking the City's 2005 Lieutenant Promotional Examination filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Weber, Benjamin) (Entered: 12/14/2014)
144 Interim Proposed Findings of Fact by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Churchill, Stephen) (Entered: 12/14/2014)
145 Set/Reset Hearings: Bench Trial Day Two set for 12/16/2014 09:00 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/15/2014)
146 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day One held on 12/15/2014. The Court and counsel review preliminary and housekeeping matters. Opening statements are made. All exhibits int he Lopez case (J O'Toole) are admitted in this case. Plaintiffs' evidence commences with P−1, Joel Wiesen (sworn). The Court Orders that Ms. Hough can only testify if the report is filed by noon on 12/22/2014. Defendant may depose Ms. Hough by 1/5/2015. Bench trial continued to 12/16/2014 at 9:00 AM. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/16/2014)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 16 of 28

12/16/2014 12/16/2014 12/16/2014 12/16/2014 12/17/2014 12/18/2014
12/19/2014 12/22/2014
01/05/2015 01/06/2015

147 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Two held on 12/16/2014. P−1, Joel Wiesen resumes the stand. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/16/2014)
148 Judge William G. Young: ELECTRONIC ORDER entered denying 128 Emergency MOTION to Take Deposition of Dr. Rick Jacobs (Paine, Matthew) (Entered: 12/16/2014)
149 Judge William G. Young: ELECTRONIC ORDER entered denying 131 MOTION in Limine to Exclude Any Use of the "Four−Fifths Rule" to Prove or Disprove Whether There Was an Adverse Impact Based on Race. (Paine, Matthew) (Entered: 12/16/2014)
150 Judge William G. Young: ELECTRONIC ORDER entered granting 132 MOTION in Limine to Exclude From Trial Any Argument That Any of the Eight Current Plaintiffs Have a Viable Disparate Impact Liability Claim Based on Their Taking the City's 2005 Lieutenant Promotional Examination (Paine, Matthew) (Entered: 12/16/2014)
151 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Three held on 12/17/2014. P−1, Wiesen resumes the stand. P−2, Bruce Smith (sworn). A witness is taken out of order. D−1, J. Silva (sworn). Court adjourned and continued to Thursday, December 18th at 9:00 AM. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge) (Gaudet, Jennifer) (Entered: 12/19/2014)
152 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Four held on 12/18/2014. D−1, J. Silva resumes the stand. Testimony of Silva concludes at 12:30. The Court Orders that Ms. Hough shall be available for deposition on or before January 2, 2015 and that the location of the deposition should be worked out among counsel. Trial suspended early and continued to Friday, December 19th at 9:00 AM. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/19/2014)
153 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Five held on 12/19/2014. D−2, Dr. Michael Campion (sworn) direct examination. Court adjourned at 12:25 PM. Cross examination of D−2 to begin at 9:00 AM on Monday, December 22, 2014. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant.) (Gaudet, Jennifer) (Entered: 12/30/2014)
154 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Six held on 12/22/2014. Cross examination of D−2, Michael Campion. Court adjourned and continued to Monday, January 5, 2015 at 9:00 AM. (Bench Trial Seven set for 1/5/2015 09:00 AM, Bench Trial Day Eight set for 1/6/2015 09:00 AM, Bench Trial Day Nine set for 1/7/2015 09:00 AM, Bench Trial Day Ten set for 1/8/2015 09:00 AM in Courtroom 18 before Judge William G. Young.) (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant.) (Gaudet, Jennifer) (Entered: 12/30/2014)
155 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Seven held on 1/5/2015. D−2, Michael Campion resumes the stand by video conference. P−3, Edward Davis (sworn). Court adjourned and continued to 9:00 AM on Tuesday, January 6, 2015. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 01/09/2015)
156 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Eight (8) held on 1/6/2015. P−4, Hough (sworn); P−5, Callahan (sworn). Court adjourned and continued to Wednesday, January 7th at 9:00 AM. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 17 of 28

01/07/2015 01/07/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015 02/19/2015

defendant) (Gaudet, Jennifer) (Entered: 01/09/2015)
157 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Nine (9) held on 1/7/2015. D−3, William B. Evans (sworn). The Court hears closing arguments and takes the matter under advisement. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill for the plaintiffs, Bok, Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 02/13/2015)
221 Exhibit and Witness List. (Gaudet, Jennifer) (Entered: 06/06/2016)
158 Transcript of Bench Trial held on January 5, 2015, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
159 Transcript of Bench Trial held on January 6, 2015, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
160 Transcript of Bench Trial held on January 7, 2015, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
161 Transcript of Bench Trial held on December 15, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
162 Transcript of Bench Trial held on December 16, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
163 Transcript of Bench Trial held on December 17, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
164 SEALED Transcript of Bench Trial held on December 17, 2014, before Judge William G. Young. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com. (Scalfani, Deborah) (Entered: 02/19/2015)
165 Transcript of Bench Trial held on December 18, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
166 Transcript of Bench Trial held on December 19, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 18 of 28

02/19/2015
02/19/2015 03/02/2015 03/09/2015 03/09/2015 03/10/2015 03/20/2015 03/25/2015
03/27/2015 03/27/2015

and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
167 Transcript of Bench Trial held on December 22, 2014, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 3/12/2015. Redacted Transcript Deadline set for 3/23/2015. Release of Transcript Restriction set for 5/20/2015. (Scalfani, Deborah) (Entered: 02/19/2015)
168 NOTICE is hereby given that an official transcript of a proceeding has been filed by the court reporter in the above−captioned matter. Counsel are referred to the Court's Transcript Redaction Policy, available on the court website at http://www.mad.uscourts.gov/attorneys/general−info.htm (Scalfani, Deborah) (Entered: 02/19/2015)
169 NOTICE by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker of Filing of First Circuit Brief in Lopez v. City of Lawrence, No. 14−1952 (Attachments: # 1 Exhibit Lopez Appeals Brief)(Lichten, Harold) (Entered: 03/02/2015)
170 NOTICE by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker of Filing of Amicus Brief with First Circuit in Lopez v. City of Lawrence (Attachments: # 1 Exhibit Exhibit A − DOJ Amicus Brief)(Lichten, Harold) (Entered: 03/09/2015)
171 NOTICE by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker of Filing of Amicus Brief with First Circuit in Lopez v. City of Lawrence (Attachments: # 1 Exhibit Exhibit A − MAMLEO Amicus Brief)(Lichten, Harold) (Entered: 03/09/2015)
172 NOTICE by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker of Filing of Amicus Brief with First Circuit in Lopez v. City of Lawrence (Attachments: # 1 Exhibit NAACP Amicus Brief)(Lichten, Harold) (Entered: 03/10/2015)
173 MOTION to Seal Document and Supplement the Trial Record by Receiving the Results of 2014 Promotional Examination as an Impounded Trial Exhibit by City of Boston.(Bok, Geoffrey) (Entered: 03/20/2015)
174 MEMORANDUM in Opposition re 173 MOTION to Seal Document and Supplement the Trial Record by Receiving the Results of 2014 Promotional Examination as an Impounded Trial Exhibit filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4, # 5 Exhibit 5)(Weber, Benjamin) (Entered: 03/25/2015)
175 MOTION for Leave to File Affidavit of Devin Taylor in Support of its Pending Motion to Supplement the Trial Record by City of Boston. (Attachments: # 1 Affidavit Affidavit of Devin Taylor)(Bok, Geoffrey) (Entered: 03/27/2015)
176 Judge William G. Young: ELECTRONIC ORDER entered re 173 MOTION to Seal Document and Supplement the Trial Record by Receiving the Results of 2014 Promotional Examination as an Impounded Trial Exhibit.

Allowed. The plaintiffs may, of course, seek such additional discovery as may be appropriate.

03/30/2015

(Paine, Matthew) (Entered: 03/27/2015)
177 AFFIDAVIT of Edward Callahan − Trial Exhibit (85). (FILED UNDER SEAL) (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4)(Paine, Matthew) (Entered: 03/31/2015)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 19 of 28

04/03/2015
04/03/2015 04/07/2015 04/15/2015
04/17/2015
04/19/2015 04/27/2015
04/28/2015 04/28/2015 04/29/2015 04/30/2015 04/30/2015 05/04/2015 05/04/2015 05/05/2015 05/06/2015 06/01/2015

178 MOTION to Bifurcate Briefing and Consideration of the Less Discrimnatory Alternatives Prong by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith.(Weber, Benjamin) (Entered: 04/03/2015)
179 Opposition re 178 MOTION to Bifurcate Briefing and Condiseration of the Less Discrimnatory Alternatives Prong filed by City of Boston. (Bok, Geoffrey) (Entered: 04/03/2015)
180 Judge William G. Young: ELECTRONIC ORDER entered: denied. Further briefing will be filed by April 30, 2015. re 178 Motion to Bifurcate (Paine, Matthew) (Entered: 04/07/2015)
181 Emergency MOTION for Protective Order Regarding Plaintiffs' Post−Trial Discovery Requests Seeking to Litigate All Aspects of the 2014 Police Promotional Examination by City of Boston. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3)(Bok, Geoffrey) (Entered: 04/15/2015)
182 Opposition re 181 Emergency MOTION for Protective Order Regarding Plaintiffs' Post−Trial Discovery Requests Seeking to Litigate All Aspects of the 2014 Police Promotional Examination filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 04/17/2015)
183 Proposed Findings of Fact by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Lichten, Harold) (Entered: 04/19/2015)
184 Emergency MOTION to Strike 177 Sealed document by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A, # 2 Exhibit B, # 3 Exhibit C, # 4 Exhibit D)(Lichten, Harold) (Entered: 04/27/2015)
185 Joint MOTION for Extension of Time to 5/4/2015 to File Post−Trial Briefs by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith.(Lichten, Harold) (Entered: 04/28/2015)
186 Opposition re 184 Emergency MOTION to Strike 177 Sealed document filed by City of Boston. (Attachments: # 1 Exhibit A)(Bok, Geoffrey) (Entered: 04/28/2015)
187 Judge William G. Young: ELECTRONIC ORDER entered granting 185 Joint MOTION for Extension of Time to 5/4/2015 to File Post−Trial Briefs. (Paine, Matthew) (Entered: 04/29/2015)
188 ELECTRONIC NOTICE of Hearing. Status Conference set for 5/5/2015 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 04/30/2015)
189 Judge William G. Young: ELECTRONIC ORDER entered denying 184 Motion to Strike (Paine, Matthew) (Entered: 04/30/2015)
190 Proposed Findings of Fact by City of Boston. (Bok, Geoffrey) (Entered: 05/04/2015)
191 Proposed Findings of Fact by All Plaintiffs. (Weber, Benjamin) (Entered: 05/04/2015)
192 ELECTRONIC NOTICE OF RESCHEDULING. Status Conference reset for 5/6/2015 10:00 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 05/05/2015)
193 ELECTRONIC Clerk's Notes for proceedings held before Judge William G. Young: Status Conference held on 5/6/2015. Colloquy re: discovery motion. Order to issue. (Court Reporter: No Court Reporter Used.) (Folan, Karen) (Entered: 05/06/2015)
194 Plaintiffs' Response by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker to 190 Proposed Findings of Fact . (Lichten, Harold) (Entered: 06/01/2015)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 20 of 28

06/15/2015 08/28/2015 09/01/2015
09/14/2015 11/16/2015 12/16/2015 12/18/2015 12/22/2015 01/06/2016 01/15/2016 01/19/2016
01/22/2016
01/22/2016

195 Notice of Supplemental Authorities by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker re 191 Proposed Findings of Fact (Attachments: # 1 Exhibit A)(Lichten, Harold) (Modified on 6/16/2015 to Correct Docket Text) (Paine, Matthew). (Entered: 06/15/2015)
196 MEMORANDUM OF LAW by City of Boston. (Attachments: # 1 Exhibit 1, # 2 Exhibit 2, # 3 Exhibit 3, # 4 Exhibit 4)(Bok, Geoffrey) (Entered: 08/28/2015)
197 ELECTRONIC NOTICE issued requesting courtesy copy for 196 Memorandum of Law. Counsel who filed this document are requested to submit a courtesy copy of this document (or documents) to the Clerk's Office attention Matthew Paine − Docket Clerk Judge Young. These documents must be clearly marked as a Courtesy Copy and reflect the document number assigned by CM/ECF. (Paine, Matthew) (Entered: 09/01/2015)
198 MOTION to Strike 196 Memorandum of Law by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A, # 2 Exhibit B)(Lichten, Harold) (Entered: 09/14/2015)
199 Judge William G. Young: ORDER entered; FINDINGS OF FACT, RULINGS OF LAW, AND ORDER.(Gaudet, Jennifer) (Entered: 11/16/2015)
200 MEMORANDUM OF LAW by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit Expert Report)(Lichten, Harold) (Entered: 12/16/2015)
201 ELECTRONIC NOTICE of Hearing. Status Conference set for 1/12/2016 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/18/2015)
202 ELECTRONIC NOTICE OF RESCHEDULING. Status Conference reset for 1/19/2016 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/22/2015)
203 ELECTRONIC NOTICE OF RESCHEDULING. Status Conference reset for 1/19/2016 11:00 AM in Courtroom 18 before Judge William G. Young. PLEASE NOTE: TIME CHANGE ONLY. (Gaudet, Jennifer) (Entered: 01/06/2016)
204 MEMORANDUM OF LAW by City of Boston. (Attachments: # 1 Affidavit of Jacinto M. Silva, Ph.D)(Bok, Geoffrey) (Entered: 01/15/2016)
205 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Status Conference held on 1/19/2016. After hearing from counsel, the Court sets schedules the matter for a jury waived trial which is expected to last five days. The first three days are set for 2/29/2016, 3/1/2016 and 3/2/2016. Further days if necessary will be set for later in the month. The trial will not cover attorney's fees. The Court welcomes motions in limine and sets a final pretrial conference. (Bench Trial Day One set for 2/29/2016 09:00 AM in Courtroom 18 before Judge William G. Young, Bench Trial Day Two set for 3/1/2016 09:00 AM in Courtroom 18 before Judge William G. Young, Bench Trial Day Three set for 3/2/2016 09:00 AM in Courtroom 18 before Judge William G. Young, Final Pretrial Conference set for 2/11/2016 02:00 PM in Courtroom 18 before Judge William G. Young.) (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Churchill, Lichten and Weber for the plaintiffs, Simon, Hodge and Bok for the defendant) (Gaudet, Jennifer) (Entered: 01/20/2016)
206 Transcript of Status Conference held on January 19, 2016, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 2/12/2016. Redacted Transcript Deadline set for 2/22/2016. Release of Transcript Restriction set for 4/21/2016. (Scalfani, Deborah) (Entered: 01/22/2016)
207 NOTICE is hereby given that an official transcript of a proceeding has been filed by the court reporter in the above−captioned matter. Counsel are referred to the Court's

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 21 of 28

01/28/2016 01/28/2016 02/01/2016

Transcript Redaction Policy, available on the court website at http://www.mad.uscourts.gov/attorneys/general−info.htm (Scalfani, Deborah) (Entered: 01/22/2016)
208 ELECTRONIC NOTICE OF RESCHEDULING. Final Pretrial Conference reset for 2/11/2016 10:00 AM in Courtroom 18 before Judge William G. Young. TIME CHANGE ONLY. (Gaudet, Jennifer) (Entered: 01/28/2016)
209 Joint MOTION for Order to For Referral of Case to Mediation Before the Honorable Robert B. Collings and for Recheduling of Pretrial Conference and Damages Trial to May 2016 to Allow for Mediation by City of Boston.(Bok, Geoffrey) (Entered: 01/28/2016)
210 Judge William G. Young: ELECTRONIC ORDER entered re 209 Joint MOTION for Order to For Referral of Case to Mediation Before the Honorable Robert B. Collings and for Recheduling of Pretrial Conference and Damages Trial to May 2016 to Allow for Mediation.

Motion allowed and the case is referred to Magistrate Judge Collings for mediation. The case is ordered administratively closed. It may be reopened upon motion of any party should the mediation succed or fail.

02/01/2016 02/01/2016 02/02/2016 02/17/2016
04/26/2016 04/26/2016 05/06/2016 05/11/2016 06/09/2016

(Paine, Matthew) (Entered: 02/01/2016)
211 REFERRING CASE to Alternative Dispute Resolution as of February 2016.(Paine, Matthew) (Entered: 02/01/2016)
212 Judge William G. Young: ORDER entered. ORDER FOR CLOSURE (Paine, Matthew) (Entered: 02/01/2016)
213 Notice of assignment to ADR Provider. Judge Robert B. Collings appointed.(Garvin, Brendan) (Entered: 02/02/2016)
214 ELECTRONIC NOTICE of ADR Conference. Alternative Dispute Resolution Hearing set for APRIL 26, 2016 AT 9:00 AM in Courtroom 5 before Magistrate Judge Robert B. Collings. INSTRUCTIONS − COUNSEL ARE DIRECTED TO BE PRESENT WITH THEIR CLIENTS AND REPRESENTATIVES THEREOF WITH FULL SETTLEMENT AUTHORITY. AN ASSENTED−TO MOTION MUST BE FILED FOR ANY REASON WHEN REQUESTING ATTENDANCE BY PHONE AND WILL BE TAKEN UNDER CONSIDERATION. A brief mediation memoranda (5−6 pages, including exhibits) addressing both the merits of the case and parties settlement position should be submitted by email to yvonne_franklin@mad.uscourts.gov no later than APRIL 20, 2016 and marked "Confidential − Not for Docketing" and is not to be served upon opposing Parties. Counsel are requested to try and accommodate this date. In the event that you believe the case is not ripe for mediation at this time, or the date poses a serious conflict, counsel are to confer with opposing counsel and FILE AN ASSENTED−TO MOTION WITH THE COURT to continue the mediation as soon as possible. (Entered: 02/17/2016)
217 NOTICE: Reference for Alternative Dispute Resolution. There was some progress case retained for further mediation session. (See report for further details) (Franklin, Yvonne) (Entered: 04/26/2016)
218 ELECTRONIC NOTICE of Continued ADR Conference. Alternative Dispute Resolution Hearing set for JUNE 8, 2016 AT 11:30 AM in Courtroom 5 before Magistrate Judge Robert B. Collings.(Franklin, Yvonne) (Entered: 04/26/2016)
219 Mail Returned as Undeliverable. Mail sent to Harold L. Lichten (Paine, Matthew) (Entered: 05/06/2016)
220 NOTICE Resetting a Hearing. The 6/8/16 hearing is canceled. Alternative Dispute Resolution Hearing Reset for JUNE 9, 2016 at 11:30 AM in Courtroom 5 before Magistrate Judge Robert B. Collings. (Franklin, Yvonne) (Entered: 05/11/2016)
223 REPORT of Alternative Dispute Resolution Provider. Magistrate Judge Robert B. Collings reports case did not settle. See report for further details. (Franklin, Yvonne) (Main Document 223 replaced on 6/14/2016)to correct ADR report. (Franklin, Yvonne). (Entered: 06/09/2016)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 22 of 28

06/21/2016 06/22/2016 06/27/2016
07/09/2016 07/09/2016 10/11/2016

224 Joint MOTION to Permit Interlocutory Review by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 06/21/2016)
225 Judge William G. Young: ELECTRONIC ORDER entered granting 224 Joint MOTION to Permit Interlocutory Review (Paine, Matthew) (Entered: 06/22/2016)
226 Joint MOTION Entry of Proposed Amended Order re 224 Joint MOTION to Permit Interlocutory Review by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit Proposed Order)(Lichten, Harold) (Entered: 06/27/2016)
227 Judge William G. Young: ELECTRONIC ORDER entered granting 226 Joint MOTION Entry of Proposed Amended Order. (Paine, Matthew) (Entered: 07/11/2016)
228 Judge William G. Young: ORDER entered: AMENDED ORDER re 199 FINDINGS OF FACT, RULINGS OF LAW, AND ORDER. (Paine, Matthew) (Entered: 07/11/2016)
229 JUDGMENT of the USCA (16−8034):

The district court issued its findings on liability in this case without the benefit of our subsequently−issued opinion in the case of Lopez v. City of Lawrence, 823 F.3d 102 (1st Cir. 2016). Since then, the district court has not yet purported to apply Lopez to the facts of this case. For example, it has not stated whether and how its assessment of validity has taken into consideration the guidance we provided. Id. at 116−117. We therefore deny the petition without prejudice to renewal, if otherwise appropriate, after the district court has itself applied Lopez to this case.

10/17/2016 10/18/2016 10/20/2016
11/17/2016 11/21/2016 12/13/2016 12/13/2016 12/20/2016 12/22/2016

(Paine, Matthew) (Entered: 10/12/2016)
230 Letter/request (non−motion) from Plaintiff's Counsel . (Attachments: # 1 Exhibit A)(Lichten, Harold) (Entered: 10/17/2016)
231 ELECTRONIC NOTICE of Hearing. Status Conference set for 10/20/2016 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 10/18/2016)
232 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Status Conference held on 10/20/2016. The parties notify the Court of their positions regarding the COA Judgment (#229). Briefs are due on or before Tuesday, November 29, 2016. Briefs may not exceed 20 pages. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Churchill, Lichten and Webber for the plaintiffs, Simon, Bok and Taub for the defendant) (Gaudet, Jennifer) (Entered: 10/24/2016)
233 MOTION for Extension of Time to December 13, 2016 to File Briefs On Effect of Lopez Decision (Unopposed) by City of Boston.(Simon, John) (Entered: 11/17/2016)
234 Judge William G. Young: ELECTRONIC ORDER entered granting 233 MOTION for Extension of Time to December 13, 2016 to File Briefs On Effect of Lopez Decision (Unopposed). (Paine, Matthew) (Entered: 11/21/2016)
235 BRIEF by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker to 232 Status Conference, Regarding Court of Appeal Judgment. (Lichten, Harold) (Entered: 12/13/2016)
236 BRIEF by City of Boston On Affect of Lopez on Court's Liability Decision. (Hodge, Kay) (Entered: 12/13/2016)
237 Joint MOTION for Leave to File Responses by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 12/20/2016)
238 Judge William G. Young: ELECTRONIC ORDER entered granting 237 Joint MOTION for Leave to File Responses. Response Due On or Before January 20, 2017. (Paine, Matthew) (Entered: 12/22/2016)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 23 of 28

01/18/2017 01/20/2017 01/27/2017 01/27/2017 02/07/2017 03/01/2017
07/26/2017 08/01/2017 08/08/2017
08/11/2017 03/26/2018 04/17/2019 04/24/2019 05/02/2019 05/06/2019
05/07/2019

239 Assented to MOTION for Extension of Time to January 27, 2017 to File Response/Reply re: Affect of Lopez Decision by City of Boston.(Simon, John) (Entered: 01/18/2017)
240 Judge William G. Young: ELECTRONIC ORDER entered granting 239 Assented to MOTION for Extension of Time to January 27, 2017 to File Reply Briefs (Paine, Matthew) (Entered: 01/20/2017)
241 Response by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker to 236 Brief Regarding Court of Appeal Judgment in Lopez. (Lichten, Harold) (Entered: 01/27/2017)
242 Response by City of Boston to 235 Brief, of Plaintiffs Regarding Court of Appeals Judgment. (Attachments: # 1 Exhibit A)(Hodge, Kay) (Entered: 01/27/2017)
243 ELECTRONIC NOTICE of Hearing.Hearing Re: Briefs set for 3/1/2017 09:30 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 02/07/2017)
244 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Hearing re Re: Briefs held on 3/1/2017. Court hears arguments from counsel on the briefs. Court will prepare a further memorandum and order consistent with the mandate of the Court of Appeals. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Weber, Lichten, Churchill, Simon, Hodge, Bok) (Lima, Christine) (Entered: 03/01/2017)
245 Judge William G. Young: ORDER entered. MEMORANDUM AND ORDER(Sonnenberg, Elizabeth) (Entered: 07/26/2017)
246 MOTION to Amend 245 Memorandum & ORDER by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Declaration of Harold Lichten)(Lichten, Harold) (Entered: 08/01/2017)
247 Joint MOTION to Permit Interlocutory Review by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Judgment Entered 10.11.2016, # 2 Text of Proposed Order)(Lichten, Harold) (Entered: 08/08/2017)
248 Judge William G. Young: ORDER entered re 247 Joint MOTION to Permit Interlocutory Review (Paine, Matthew) (Entered: 08/16/2017)
249 Judge William G. Young: ELECTRONIC ORDER entered denying 246 Motion to Amend (Paine, Matthew) (Entered: 03/26/2018)
250 JUDGMENT of the USCA (17−8030): The petition for permission to appeal is DENIED. (Paine, Matthew) (Entered: 04/18/2019)
251 ELECTRONIC NOTICE of Hearing. Status Conference set for 5/6/2019 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 04/24/2019)
252 Letter/request (non−motion) from Plaintiffs re Status Conference. (Lichten, Harold) (Entered: 05/02/2019)
253 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Status Conference held on 5/6/2019. Counsel update the Court as to status of the case. Jury waived trial is placed on the running trial list for July 2019. A final pretrial conference will be held in June. A joint pretrial memo is due the first Monday in June. (Ready for Trial on 7/1/2019 09:00 AM in Courtroom 18 before Judge William G. Young.) (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Churchill, Lichten and Weber for the plaintiffs, Attorneys Simon, Hodge and Bok for the defendant) (Gaudet, Jennifer) (Entered: 05/07/2019)
Case reopened. (Garvin, Brendan) (Entered: 05/07/2019)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 24 of 28

05/08/2019 05/22/2019 06/03/2019 06/05/2019 06/05/2019 06/06/2019 06/11/2019 06/12/2019 06/13/2019 06/13/2019 06/13/2019
06/18/2019

254 NOTICE of Withdrawal of Appearance by Geoffrey R. Bok (Bok, Geoffrey) (Entered: 05/08/2019)
255 MOTION to Certify Class (Renewed) by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Affidavit of Larry Ellison, # 2 Exhibit B − Lopez Order, # 3 Exhibit C − Raghai Order)(Lichten, Harold) (Entered: 05/22/2019)
256 PRETRIAL MEMORANDUM by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Lichten, Harold) (Entered: 06/03/2019)
257 Judge William G. Young: ORDER entered. PROCEDURAL ORDER re pretrial/trial. Final Pretrial Conference set for 6/13/2019 02:00 PM in Courtroom 18 before Judge William G. Young.(Gaudet, Jennifer) (Entered: 06/05/2019)
258 ELECTRONIC NOTICE Setting Hearing on Motion 255 RENEWED MOTION to Certify Class (Renewed) : Motion Hearing set for 6/13/2019 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 06/05/2019)
259 Opposition re 255 MOTION to Certify Class (Renewed) filed by City of Boston. (Simon, John) (Entered: 06/06/2019)
260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A − April 17, 2019 Judgment, # 2 Exhibit B − Declaration of Paul Joseph)(Lichten, Harold) (Entered: 06/11/2019)
261 Assented to MOTION for Leave to File Reply in Support of Pls. Renewed Mot. for Class Cert. (ECF No. 255) by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A − Reply in Support of Pls.' Renewed Mot. for Class Cert. (ECF No. 255))(Lichten, Harold) (Entered: 06/12/2019)
262 ELECTRONIC NOTICE OF RESCHEDULING. Final Pretrial Conference reset for 6/13/2019 12:00 PM in Courtroom 18 before Judge William G. Young. Time change only. (Gaudet, Jennifer) (Entered: 06/13/2019)
263 Set/Reset Deadlines as to 255 MOTION to Certify Class (Renewed). Motion Hearing reset for 6/13/2019 12:00 PM in Courtroom 18 before Judge William G. Young. Time change only. (Gaudet, Jennifer) (Entered: 06/13/2019)
264 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Motion Hearing held on 6/13/2019 re 255 MOTION to Certify Class (Renewed) filed by John M Johnson, Robert Tinker, Bruce Smith, Paul Joseph, Kim Gaddy, Brian Keith Latson, LaTeisha Adams, Leighton Facey, Martin Joseph, Marwan Moss. After hearing from counsel, the Court enters an order denying 255 Motion to Certify Class and expresses no opinion as to whether others may join as plaintiffs. Final Pretrial Conference held on 6/13/2019 in chambers. The bench trial is set tentatively for Monday, July 8, 2019 at 9:00 AM and shall take five (5) days. The Court explains the calendar for July − if the criminal trial set for July 8, 2019 goes forward, this matter will proceed to trial on Monday, July 15, 2019. The Court reviews the joint pretrial memo with counsel. If counsel would like to mediate this matter, this Court will try to arrange ADR with a magistrate judge. The Court answers questions of counsel and sets hearing as to 260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam for Monday, July 1, 2019 at 2:00 PM. The City of Boston's response to motion in limine is due on or before 6/20/2019; reply brief due on or before 6/26/2019. The Court instructs counsel to contact the clerk if the case were to settle. (Motion Hearing set for 7/1/2019 02:00 PM in Courtroom 18 before Judge William G. Young.) (Court Reporter: None.)(Attorneys present: Attorneys Lichten and Weber for the plaintiffs, Attorneys Hodge, Simon and Costello for the defendant) (Gaudet, Jennifer) (Entered: 06/17/2019)
265 NOTICE of Appearance by Thomas H. Costello on behalf of City of Boston (Costello, Thomas) (Entered: 06/18/2019)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 25 of 28

06/20/2019 06/21/2019
06/24/2019 06/26/2019 06/27/2019 06/27/2019 06/28/2019 07/01/2019 07/01/2019 07/01/2019
07/02/2019 07/09/2019

266 Opposition re 260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam filed by City of Boston. (Simon, John) (Entered: 06/20/2019)
267 MOTION to Intervene Partially Assented−to by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Proposed Am. Compl., # 2 Exhibit B − 2008 Rank Order List, # 3 Exhibit C − MCAD Finding of Probable Cause, # 4 Exhibit D − Woodley EEOC Charge)(Lichten, Harold) (Entered: 06/21/2019)
268 MEMORANDUM OF LAW by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Decl. of Paul Joseph, # 2 Exhibit B − MCAD Decision I, # 3 Exhibit C − MCAD Decision II)(Lichten, Harold) (Entered: 06/24/2019)
269 Assented to MOTION for Extension of Time to June 27, 2019 to File Response/Reply in Support of Pls.' Motion in Limine (ECF No. 260) by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 06/26/2019)
270 Judge William G. Young: ELECTRONIC ORDER entered granting 269 Assented to MOTION for Extension of Time to June 27, 2019 to File Response re 260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam Response due by 6/27/2019 (Paine, Matthew) (Entered: 06/27/2019)
271 REPLY to Response to 260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith. (Attachments: # 1 Exhibit A − Declaration of Harold Lichten)(Lichten, Harold) (Entered: 06/27/2019)
272 Joint MOTION to Modify Order Tentatively Setting Trial Dates by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − UnitedHealth Group Order, # 2 Exhibit B − Leidos Order)(Lichten, Harold) (Entered: 06/28/2019)
273 Judge William G. Young: ELECTRONIC ORDER entered granting 272 Joint Motion to Modify Order Tentatively Setting Trial Dates and to Exclude Proceedings on the Week of July 8, 2019. This case is continued to the September 2019 running trial list to commence after the conclusion of the jury trial in civil matter 15−30148−WGY. (Gaudet, Jennifer) (Entered: 07/01/2019)
274 Set/Reset Hearings: Ready for Trial on 9/3/2019 09:00 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 07/01/2019)
278 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Motion Hearing held on 7/1/2019 re 260 MOTION in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam filed by Bruce Smith, Paul Joseph, Kim Gaddy, Brian Keith Latson, Leighton Facey, LaTeisha Adams, Martin Joseph, Marwan Moss. After hearing arguments, the Court enters an order denying 260 Motion in Limine to Preclude Evidence and Testimony Related to the Defendant's 2014 Promotional Exam for the reasons stated on the record. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Churchill, Lichten and Weber for the plaintiffs and Attorneys Hodge, Simon and Costello for the defendant) (Gaudet, Jennifer) (Entered: 07/17/2019)
275 Opposition re 267 MOTION to Intervene Partially Assented−to filed by City of Boston. (Simon, John) (Entered: 07/02/2019)
276 Judge William G. Young: ELECTRONIC ORDER entered granting 267 MOTION to Intervene (Partially Assented−To).

To the extent the defendants assent, the motion is allowed. The remainder remains under advisement.

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 26 of 28

07/16/2019
07/17/2019
07/17/2019
07/29/2019 08/15/2019 09/05/2019 09/06/2019 09/09/2019 09/18/2019 09/19/2019 09/24/2019 09/24/2019 10/08/2019 10/16/2019

(Paine, Matthew) (Entered: 07/09/2019)
277 Assented to MOTION for Leave to File Reply in Further Support of Motion to Intervene by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Proposed Reply)(Lichten, Harold) (Entered: 07/16/2019)
279 Judge William G. Young: ELECTRONIC ORDER entered granting 277 Assented to MOTION for Leave to File Reply in Further Support of Motion to Intervene ; Counsel using the Electronic Case Filing System should now file the document for which leave to file has been granted in accordance with the CM/ECF Administrative Procedures. Counsel must include − Leave to file granted on (date of order)− in the caption of the document. (Paine, Matthew) (Entered: 07/17/2019)
280 REPLY to Response to 267 MOTION to Intervene Partially Assented−to filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − Desmesmin MCAD Charge)(Lichten, Harold) (Entered: 07/17/2019)
281 Judge William G. Young: ORDER entered. re 267 MOTION to Intervene Partially Assented−to filed by John M Johnson, Robert Tinker, Bruce Smith, Paul Joseph, Kim Gaddy, Brian Keith Latson, LaTeisha Adams, Leighton Facey, Martin Joseph, Marwan Moss (Sonnenberg, Elizabeth) (Entered: 07/29/2019)
282 NOTICE of Appearance by Zachary L. Rubin on behalf of LaTeisha Adams, Leighton Facey, Kim Gaddy, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith (Rubin, Zachary) (Entered: 08/15/2019)
283 MOTION to Intervene (Renewed) by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Attachments: # 1 Exhibit A − MCAD Letter, # 2 Exhibit B − Desmesmin MCAD Charge)(Lichten, Harold) (Entered: 09/05/2019)
284 Joint MOTION to Continue Damages Trial to October 28, 2019 by City of Boston.(Simon, John) (Entered: 09/06/2019)
285 Judge William G. Young: ELECTRONIC ORDER entered granting 284 Joint Motion for Trial Date of October 28, 2019. Motion allowed. The Court will do its best to honor this date but makes no guarantees. In any event, the remedy phase of this trial will not commence before October 28, 2019. (Gaudet, Jennifer) (Entered: 09/09/2019)
286 PRETRIAL MEMORANDUM by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker. (Lichten, Harold) (Entered: 09/18/2019)
287 Opposition re 283 MOTION to Intervene (Renewed) filed by City of Boston. (Simon, John) (Entered: 09/19/2019)
288 Judge William G. Young: ELECTRONIC ORDER entered denying 283 MOTION to Intervene (Renewed) (Paine, Matthew) (Entered: 09/24/2019)
289 AMENDED COMPLAINT , Compensatory, Injunctive, and Declaratory Relief Requested against City of Boston, filed by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Robert Tinker.(Lichten, Harold) (Entered: 09/24/2019)
290 ANSWER to 289 Amended Complaint, by City of Boston.(Simon, John) (Entered: 10/08/2019)
291 ELECTRONIC NOTICE of Hearing. Bench Trial Day One set for 10/28/2019 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Two set for 10/29/2019 09:00 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Three set for 10/30/2019 09:30 AM in Courtroom 18 before Judge William G. Young. Bench Trial Day Four set for 10/31/2019 09:00 AM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 10/16/2019)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 27 of 28

10/28/2019
10/29/2019
10/30/2019
11/05/2019 11/05/2019 11/05/2019 11/05/2019 12/10/2019 12/10/2019 12/16/2019 01/15/2020

296 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day One held on 10/28/2019. Opening statements made. Plaintiffs' evidence commences with: P−1, Joel Wiesen (sworn); P−2, Paul Joseph (sworn). Court adjourned and continued to October 29, 2019. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Lichten, Weber, Rubin and Churchill for the plaintiffs; Attorneys Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/10/2019)
297 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Two held on 10/29/2019. Plaintiffs' evidence continues. P−3, Latisha Adams (sworn); P−4, Leighton Facey (sworn); P−5, Marwan Moss (sworn); P−6, Dr. Kim Gaddy (sworn). Plaintiffs rest subject to rebuttal. Defendant's evidence commences: D−1, Jacinto Manuel Silva (sworn). Court adjourned until October 30, 2019. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Lichten, Weber, Rubin and Churchill for the plaintiffs; Attorneys Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/10/2019)
298 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Bench Trial Day Three held on 10/30/2019. D−1, Jacinto Manuel Silva resumes the stand. D−2, William Evans (sworn). Defendant rests. Briefs to be filed on or before January 15, 2020. Final arguments to be promptly set for hearing. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Lichten, Weber, Rubin and Churchill for the plaintiffs; Attorneys Simon and Hodge for the defendant) (Gaudet, Jennifer) (Entered: 12/10/2019)
292 Transcript of Bench Trial: Damages held on October 28, 2019, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 11/26/2019. Redacted Transcript Deadline set for 12/6/2019. Release of Transcript Restriction set for 2/3/2020. (Scalfani, Deborah) (Entered: 11/05/2019)
293 Transcript of Bench Trial: Damages held on October 29, 2019, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 11/26/2019. Redacted Transcript Deadline set for 12/6/2019. Release of Transcript Restriction set for 2/3/2020. (Scalfani, Deborah) (Entered: 11/05/2019)
294 Transcript of Bench Trial: Damages held on October 30, 2019, before Judge William G. Young. The Transcript may be purchased through the Court Reporter, viewed at the public terminal, or viewed through PACER after it is released. Court Reporter Name and Contact Information: Richard Romanow at bulldog@richromanow.com Redaction Request due 11/26/2019. Redacted Transcript Deadline set for 12/6/2019. Release of Transcript Restriction set for 2/3/2020. (Scalfani, Deborah) (Entered: 11/05/2019)
295 NOTICE is hereby given that an official transcript of a proceeding has been filed by the court reporter in the above−captioned matter. Counsel are referred to the Court's Transcript Redaction Policy, available on the court website at http://www.mad.uscourts.gov/attorneys/general−info.htm (Scalfani, Deborah) (Entered: 11/05/2019)
299 Exhibit and Witness List. (Gaudet, Jennifer) (Entered: 12/10/2019)
300 ELECTRONIC NOTICE of Hearing.Closing arguments set for 1/29/2020 02:00 PM in Courtroom 18 before Judge William G. Young. (Gaudet, Jennifer) (Entered: 12/10/2019)
301 Joint Supplemental Trial Exhibits by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Kenneth Sousa, Robert Tinker, William Woodley. (Attachments: # 1 Exhibit A − Joint Trial Exhibit 86 (b), # 2 Exhibit B − Joint Trial Exhibit 86 (c), # 3 Exhibit C − Joint Trial Exhibit 95)(Lichten, Harold) (Modified on 12/17/2019 to Correct Docket Text) (Paine, Matthew). (Entered: 12/16/2019)
302 TRIAL BRIEF Re: Damages by City of Boston. (Simon, John) (Entered: 01/15/2020)

Case: 1:12-cv-10291 As of: 05/24/2020 10:18 AM EDT 28 of 28

01/15/2020 01/29/2020
02/25/2020 05/13/2020

303 TRIAL BRIEF on Remedy by LaTeisha Adams, Leighton Facey, Kim Gaddy, John M Johnson, Martin Joseph, Paul Joseph, Brian Keith Latson, Marwan Moss, Bruce Smith, Kenneth Sousa, Robert Tinker, William Woodley. (Attachments: # 1 Exhibit A − MCAD Decision in Harris v. Worcester Police Dept., # 2 Exhibit B − Appeals Court Decision in Worcester v. MCAD)(Lichten, Harold) (Entered: 01/15/2020)
304 Electronic Clerk's Notes for proceedings held before Judge William G. Young: Closing arguments held on 1/29/2020. The parties make their closing arguments. The Court takes the matter under advisement and notifies counsel that he will accept any additional briefing, though he will begin working on this matter immediately. (Court Reporter: Richard Romanow at bulldog@richromanow.com.)(Attorneys present: Attorneys Lichten, Churchill, Weber and Rubin for the plaintiffs; Attorneys Hodge and Simon for the defendant) (Gaudet, Jennifer) (Entered: 01/30/2020)
305 TRIAL BRIEF ON DAMAGES (SUPPLEMENTAL) by City of Boston. (Simon, John) (Entered: 02/25/2020)
306 Judge William G. Young: ORDER entered. MEMORANDUM AND ORDER(Sonnenberg, Elizabeth) (Entered: 05/13/2020)