1 RECOMMENDED FOR FULL-TEXT PUBLICATION Pursuant to Sixth Circuit I.O.P. 32.1(b) File Name: 14a0071p.06 

### UNITED STATES COURT OF APPEALS

FOR THE SIXTH CIRCUIT

_________________

E QUAL E MPLOYMENT O PPORTUNITY C OMMISSION , Plaintiff-Appellant ,

v

.

K

APLAN H IGHER E DUCATION C ORPORATION , K

APLAN , I NC ., and I OWA C OLLEGE A CQUISITION C ORPORATION , d/b/a Kaplan University, Defendants-Appellees

. ┐ │ │ │ │ │ │ │ │ │ ┘ No. 13-3408

Appeal from the United St ates District Court for the Northern District of Ohio at Cleveland No. 1:10-cv-02882—Patricia A. Gaughan, District Judge. Argued: March 20, 2014 Decided and Filed: April 9, 2014
Before: KEITH, COOK, and KETHLEDGE, Circuit Judges.

_________________ COUNSEL

ARGUED: Kate Northrup, UNITED STATES EQUAL EMPLOYMENT OPPORTUNITY COMMISSION, Washington, D.C., for Appellant. Gerald L. Maatman, Jr., SEYFARTH SHAW LLP, Chicago, Illinois, for Appellees. ON BRIEF: Kate Northrup, UNITED STATES EQUAL EMPLOYMENT OPPORTUNITY COMMI SSION, Washington, D.C., for Appellant. Gerald L. Maatman, Jr., David J. Rowland, Pamela Q. De vata, Jennifer A. Riley, Matthew J. Gagnon, SEYFARTH SHAW LLP, Chicago, Illinois, for Appellees. Anastasia P. Boden, PACIFIC LEGAL FOUNDATION, Sacramento, Ca lifornia, for Amicus Curiae. 

### >No. 13-3408 EEOC v. Kaplan, et al. Page 2

_________________ OPINION

_________________ KETHLEDGE, Circuit Judge. In this case the EEOC sued the defendants for using the same type of background check that the EEOC itself uses. The EEOC’s personnel handbook recites that “[o]verdue just debts increase temp tation to commit illegal or unethical acts as a means of gaining funds to meet financial obligat ions.” Because of that concern, the EEOC runs credit checks on applicants for 84 of the agency ’s 97 positions. The defendants (collectively, “Kaplan”) have the same concern; and thus Kaplan runs credit checks on applicants for positions that provide access to students’ financial-loan information, among other positions. For that practice, the EEOC sued Kaplan.
Specifically, the EEOC alleges that Kaplan’s us e of credit checks causes it to screen out more African-American applicants than white app licants, creating a disparate impact in violation of Title VII of the federal Civil Rights Act. See 42 U.S.C. § 2000e-2(a)(1), (a)(2), (k). Proof of disparate impact is usually statistical proof in the form of expert testimony; and here the EEOC relied solely on statistical data compiled by Ke vin Murphy, who holds a doctorate in industrial and organizational psychology. For two reasons, however, the district court excluded Murphy’s testimony on grounds that it was unreliable. First, the EEOC presente d “no evidence” that Murphy’s methodology satisfied any of the factors th at courts typically consider in determining reliability under Federal Rule of Evidence 702; and second, as Murphy himself admitted, his sample was not representative of Kaplan’s applican t pool as a whole. The district court therefore granted summary judgment to Kaplan. The EEOC now argues that the district court “erred”—a telling, oft-repeated, and mistaken choice of wo rd here—when it excluded Murphy’s testimony. We reject the EEOC’s arguments and affirm.
Kaplan offers undergraduate and graduate de grees to students acro ss the country. Some of Kaplan’s students obtain financial aid through programs operated by the United States Department of Education; and consequently, some of Kaplan’s employees have access to those students’ financial information. The Department has regulations that circumscribe the manner inNo. 13-3408 EEOC v. Kaplan, et al. Page 3
which Kaplan can access and use students’ inform ation. Violations of those regulations can bring severe penalties.
Kaplan’s concerns became reality about a decad e ago, when it discovered that some of its
financial-aid officers had stolen payments that belonged to student s. Kaplan also learned that some of its executives had engaged in self-dealing, by hiring relatives as vendors. In response, Kaplan implemented a number of measures to pr event these abuses. One of those measures was to run credit checks on applicants for senior-e xecutive positions, accounting and other positions with access to company financials or cash, and positions with access to student financial-aid information. The credit checks are performe d by a third-party vendor, which reports, among other things, whether the applicant has ever f iled for bankruptcy, is delinquent on child-support payments, has any garnishments on earnings, has outstanding civil judgments exceeding $2,000, or has a social-security number that does not match th e number the credit bureau has on file. If an applicant’s credit history includes any of the enumerated items, the vendor flags the applicant’s file for “review.” At that point, Kaplan typically reviews the file and makes an ad
hoc decision as to whether to move forward with the application. The credit-check process is racially blind: the vendor does not report the applicant’s ra ce with her other information.
Kaplan has used several vendors for its credit checks, but Murphy focused upon applications screened by one vendor, General Information Serv ices (“GIS”). Murphy obtained GIS data for 4,670 applicants. That data, as di scussed above, did not include the applicant’s race, so the EEOC subpoenaed records from the de partments of motor vehicles. Eleven states provided records that identified an applicant’s race. Thirty-six states and the District of Columbia provided color copies of drivers’ license photos for approximately 900 applicants.
The dispute in this case concerns the reliabil ity, or lack thereof, of the process—which the EEOC calls “race rating”—by wh ich Murphy purported to identif y the race of each person in those drivers’ license photos. The process wa s crafted by Murphy himself, specifically for purposes of litigation—though the r ecord contains no indication th at Murphy has any particular expertise in constructing methodologies to identify race by visual means. In any event, Murphy assembled a team of five “race raters,” each of whom has experience in what the EEOC calls “multicultural, multiracial, treatment outcome research”—a term undefined by the EEOC here.No. 13-3408 EEOC v. Kaplan, et al. Page 4
But that term assuredly does not refer to the raters’ experience with methodologies to identify race by visual means—since, undisputedly, they have none. Mur phy directed each rater separately to review each appl icant’s drivers’ license photograph and then classify the person’s race in one of five ways: “African-American,” “Asian,” “Hispanic,” “White,” or “Other.” If four of five raters agreed upon a particular applicant’s race, the applicant was so classified for purposes of Murphy’s statistics. For 11.7% of the photogr aphs, the raters failed to reach that consensus. For some reason Murphy also provi ded the raters with each applicant’s name— which, the EEOC concedes, the raters were s upposed to disregard when classifying an applicant’s race.
Murphy filed his expert report on May 1, 2012 and then a revised report on August 17, 2012, both per the district court’s scheduling order. The revised report included the putative race and credit-check results for a total of 1,090 appli cants, of whom 803 had been racially classified per Murphy’s “rating” process. In that sample of 1,090 applic ants (out of a total of 4,670 applicants for whom GIS provided data), the per centage of black appli cants who were flagged for review, based upon their credit hi stories, was higher than the pe rcentage of wh ite applicants who were flagged. (That is essentially the ba sis upon which the EEOC claims disparate impact here.) But Murphy’s sample overrepresented “fa ils” generally: 23.8% of the applicants in his sample of 1,090 were rejected because of their credit history, whereas only 13.3% of the total GIS pool of 4,670 were.
Murphy then proceeded to file additional repor ts, contrary to the terms of the district court’s scheduling order. On September 5, 2012—in re sponse to a critical analysis of his work by Kaplan’s expert—Murphy submitted a third repor t, which Kaplan moved to strike, but which the district court reluctantly permitted in an Oc tober 5 order, with the admonition that “[n]o further expert reports are allo wed.” Yet Murphy filed anothe r report on November 8, 2012, two weeks before summary-judgment briefing was due. In that report, Mur phy provided what the
EEOC describes as “anecdotal co rroboration” of the re liability of his race -rating process. Murphy filed yet another report on December 21, 2012, this time in response to Kaplan’s motion specifically to exclude his tes timony as unreliable under Rule 702.No. 13-3408 EEOC v. Kaplan, et al. Page 5

The district court thereafter excluded Mu rphy’s testimony in a meticulously reasoned opinion. We review that ex clusion deferentially, for an abuse of discretion. Gen. Elec. Co. v.
Joiner , 522 U.S. 136, 139 (1997).
Rule 702 provides that “[a] witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if[,]” among other requirements, the testimony “is based on su fficient facts and data” and “is the product of reliable principles and methods [.]” Fed. R. Evid. 702(b), (c). As the proponent of expert testimony, the EEOC bears the burden of proving its admissibility. Nelson v. Tenn. Gas Pipeline
Co ., 243 F.3d 244, 251 (6th Cir. 2001).
In determining whether an expert’s methodology is reliable, courts frequently consider the factors set forth by the Supreme Court in Daubert v. Merrell Dow Pharmaceuticals, Inc. , 509 U.S. 579 (1993). But the district court has “broad latitude” as to which factors to consider in a particular case. Kumho Tire Co. v. Carmichael , 526 U.S. 137, 142 (1999).

Here, the district court considered every one of the Daubert factors—and found that Murphy’s methodology flunked them all. Two factor s we consider together. “Ordinarily, a key question to be answered” in dete rmining whether a technique is reliable is “whether it can be (and has been) tested.” Daubert , 509 U.S. at 593. Similarly, “i n the case of a particular scientific technique, the court or dinarily should consider the known or potential rate of error.”
Id . at 594. The district court found that the EEOC “wholly fail[ed]” to provide evidence in support of either of these factors. Op. at 13. In response, the EEOC argues that we can find that support in Murphy’s “anecdotal corroboration”—set forth in his November 8 report, submitted a
month after the district court sa id there would be no more reports . We consider it now only
because the district court did. Mu rphy says that, as to 47 applicants, he cross-checked his raters’
classifications with racial id entifications provided by a DMV, finding 95.7% agreement between the two. Murphy also says that, for another 10 applicants, he cross-checked his raters’ classifications with racial information provi ded by PeopleSoft, a personnel-software program used internally by Kaplan. That cross-ch eck yielded an 80% match—an unimpressive correlation in case where a few percentage poin ts (in credit-check fail rates for blacks and whites) might make the difference between significan t liability and none. But more to the point,No. 13-3408 EEOC v. Kaplan, et al. Page 6
as Murphy himself candidly conceded, a mere 57 instances of anecdotal corroboration is not enough “to establish the reliability of my photo rating methodology.” The district court was well within its “broad latitude[,]” Kumho , 526 U.S. at 142, to find these factors unmet.
The EEOC’s case goes downhill from there. In determining reliability, “[a]nother pertinent consideration is whether the theory or technique has been subjected to peer review and publication.” Daubert , 509 U.S. at 593. Again th e district court found no evidence in support of this factor, and undisputedly there is none. Before us, the EEOC simply argues that the district
court should not have considered the factor. The argument is meritless: “submission to the scrutiny of the scientific community is a com ponent of ‘good science,’ in part because it
increases the likelihood that substantive flaws in methodology will be detected[,]” id .; and the district court had good reason to think that su ch scrutiny might have detected flaws here.
The district court also found that Murphy’ s methodology lacked “standards controlling the technique’s operation.” Id. at 594. The EEOC responds that the relevant standard was Murphy’s requirement that four of five raters agree on an app licant’s race. But that response overlooks Murphy’s own concession that the raters themselves had no particular standard in classifying each applicant; instea d, they just eyeballed the DMV photos. The district court was also troubled by an affirmative breakdown in Murphy’ s controls: “the ‘race raters’ were provided the names of the applicants,” which—for a me thodology purportedly based exclusively on visual identification of race—might “create an unintended bi as on the part of the panel.” Op. at 16. The EEOC countered at oral argumen t that there is “no evidence” th at the raters considered any of the applicants’ names in classifying them by race. But that argument merely illustrates a fallacy that pervades the agency’s entire argumen t on appeal, to wit: that it was Kaplan’s (or the district court’s) burden to s how that Murphy’s testimony was in admissible, rather than the EEOC’s burden to show that his testimony was admissible. The law says the contrary. See
Nelson , 243 F.3d at 251; Fed. R. Evid. 702 Advisory Comm. Notes, 2000 Amendments (“the proponent has the burden of establishing that th e pertinent admissibility requirements are met a by preponderance of the evidence”).

The district court also f ound that the EEOC “present[ed] no evidence” that Murphy’s race-rating methodology “is generally accepted in the scien tific community.” Op. at 16; seeNo. 13-3408 EEOC v. Kaplan, et al. Page 7
generally Daubert , 509 U.S. at 593. On this point the EEOC offers no response—perhaps because, as the district court observed, “the EE OC itself discourages employers from visually identifying an individual by race a nd indicates that visual identifica tion is appropriate ‘only if an employee refuses to self identify.’” Op. at 16.
Finally, as an independent ground for exclud ing Murphy’s testimony, the district court found that “[t]here is no indication” that Murp hy’s group of 1,090 applicants is in any “way ‘representative’ of the applicant pool as a whole.” Op. at 18. Inst ead there is a strong indication to the contrary: Murphy’s group had a fail rate of 23.8%, whereas the GIS applicant pool had a fail rate of only 13.3%. On this point, suffice it to say that an unrepresentative sample by definition might not be representative of the resp ective fail rates of black and white applicants in the larger pool—and thus is not a reliable means to demonstrate disparate impact.

We need not belabor the issue further. Th e EEOC brought this case on the basis of a homemade methodology, crafted by a witness with no particular expertise to craft it, administered by persons with no particular expe rtise to administer it , tested by no one, and accepted only by the witness himself. The district court did not abuse its discretion in excluding Murphy’s testimony.
The district court’s judgment is affirmed.