Opinion ID: 523563
Heading Depth: 2
Heading Rank: 1

Heading: Significance of Plaintiffs' Statistical Evidence

Text: 25 At trial, the plaintiffs herein contended that the statistical evidence alone was sufficient to establish a prima facie case of discrimination. According to plaintiffs, female faculty members were clearly treated less favorably than their male counterparts, and that unfavorable, disparate treatment was due solely to gender bias. The district court, however, found that the plaintiffs' statistical evidence was not statistically significant enough to establish a prima facie case of discrimination. For the reasons that follow, we conclude that the district court did not clearly err in ruling that the plaintiffs' proffered statistical evidence was not dispositive of their Title VII claims. 26 As discussed earlier, plaintiffs in a disparate treatment case frequently rely on statistical evidence to establish that there is a disparity between the predicted and actual treatment of employees who are members of a disadvantaged group, and to argue that such disparities exist because of an unlawful bias directed against those employees. Not all disparities, however, are probative of discrimination. Before a deviation from a predicted outcome can be considered probative, the deviation must be statistically significant. 27 Statistical significance is a measure of the probability that a disparity is simply due to chance, rather than any other identifiable factor. See Segar, 738 F.2d at 1282; see also Castaneda v. Partida, 430 U.S. 482, 496 n. 17, 97 S.Ct. 1272, 1281 n. 17, 51 L.Ed.2d 498 (1977); Mister v. Illinois Cent. Gulf R. Co., 832 F.2d 1427, 1430-31 (7th Cir.1987), cert. denied, --- U.S. ----, 108 S.Ct. 1597, 99 L.Ed.2d 911 (1988). Because random deviations from the norm can always occur, see Frazier v. Consolidated Rail Corp., 851 F.2d 1447, 1451-52 (D.C.Cir.1988); Palmer, 815 F.2d at 91; Mister, 832 F.2d at 1430-31; Fisher, supra, at 705, statisticians do not consider slight disparities between predicted and actual results to be statistically significant. See Eubanks v. Pickens-Bond Constr. Co., 635 F.2d 1341, 1347-48 (8th Cir.1980); Fisher, supra, at 706. As the disparity between predicted and actual results becomes greater, however, it becomes less likely that the deviation is a random fluctuation. When the probability that a disparity is due to chance sinks to a certain threshold level, statisticians can then infer from the statistical evidence, albeit indirectly, that the deviation is attributable to some other cause unrelated to mere chance. See D. Baldus & J. Cole, supra, Sec. 9.42, at 191-93 (Supp.1987); Berger, 843 F.2d at 1412. 28 One unit of measurement used to express the probability that an observed result is merely a random deviation from a predicted result is the standard deviation. D. Baldus & J. Cole, supra, Sec. 2.224, at 63; id. Sec. 9.03, at 294-95; see also Berger, 843 F.2d at 1412; Coates v. Johnson & Johnson, 756 F.2d 524, 536 n. 11 (7th Cir.1985). The standard deviation is a measure of spread, dispersion or variability of a group of numbers. D. Baldus & J. Cole, supra, at 359. Generally, the fewer the number of standard deviations that separate an observed from a predicted result, the more likely it is that any observed disparity between predicted and actual results is not really a disparity at all but rather a random fluctuation. Conversely, [t]he greater the number of standard deviations, the less likely it is that chance is the cause of any difference between the expected and observed results. Coates, 756 F.2d at 536 n. 11; see Woodbury, 832 F.2d at 770; Segar, 738 F.2d at 1282. A finding of two standard deviations corresponds approximately to a one in twenty, or five percent, chance that a disparity is merely a random deviation from the norm, and most social scientists accept two standard deviations as a threshold level of statistical significance. See Castaneda, 430 U.S. at 496 n. 17, 97 S.Ct. at 1281 n. 17; Melani v. Board of Higher Educ., 561 F.Supp. 769, 774 (S.D.N.Y.1983); Cooper v. University of Tex. at Dallas, 482 F.Supp. 187, 194 (N.D.Tex.1979) (citing N. Nie, C. Hull, J. Jenkins, K. Steinbrenner & D. Bent, Statistical Package for the Social Sciences 222 (2d ed. 1975)), aff'd, 648 F.2d 1039 (5th Cir.1981) (per curiam). When the results of a statistical analysis yield levels of statistical significance at or below the 0.05 level, chance explanations for a disparity become suspect, and most statisticians will begin to question the assumptions underlying their predictions. 29 Cognizant of the important role that statistics play in disparate treatment cases, the Supreme Court has held that [w]here gross statistical disparities can be shown, they alone may in a proper case constitute prima facie proof of a pattern or practice of discrimination. Hazelwood School Dist. v. United States, 433 U.S. 299, 307-08, 97 S.Ct. 2736, 2741-42, 53 L.Ed.2d 768 (1977). The threshold question in disparate treatment cases, then, is: [A]t what point is the disparity in selection rates ... sufficiently large, or the probability that chance was the cause sufficiently low, for the numbers alone to establish a legitimate inference of discrimination[?] Palmer, 815 F.2d at 92 (emphasis added); see Frazier, 851 F.2d at 1452. In answer to this question, most courts follow the conventions of social science which set 0.05 as the level of significance below which chance explanations become suspect. D. Baldus & J. Cole, supra, Sec. 9.02, at 291; see, e.g., Castenada, 430 U.S. at 496 n. 17, 97 S.Ct. at 1281 n. 17; Frazier, 851 F.2d at 1452; Berger, 843 F.2d at 1412; Segar, 738 F.2d at 1262. The existence of a 0.05 level of statistical significance indicates that it is fairly unlikely that an observed disparity is due to chance, and it can provide indirect support for the proposition that disparate results are intentional rather than random. 6 By no means, however, is a five percent probability of chance (or approximately two standard deviations) considered an exact legal threshold. Palmer, 815 F.2d at 92. 30 In the present case, the three salary studies which the district court considered most probative of a pattern or practice of discrimination produced a range of standard deviations between approximately one and five, and of the total thirty-three standard deviation measures cited, twenty-four exceeded two standard deviations. 7 Significantly, however, nine of the measures cited fell below two standard deviations. Also, the negative residuals associated with being female were not significant in every year of the liability period. 31 Given the range of standard deviations associated with their salary regressions, the plaintiffs contended that the statistical evidence clearly gave rise to a presumption of discrimination. As discussed earlier herein, however, although the district judge found the studies to be persuasive, she nevertheless held that these levels of statistical significance alone were not sufficiently high to support a prima facie claim of salary discrimination. Ottaviani, 679 F.Supp. at 309. 32 On appeal, appellants argue inter alia that, as a matter of law, a finding of two standard deviations should be equated with a prima facie case of discrimination. According to appellants, the district court therefore erred in finding that they had not met their burden of establishing a prima facie case. In support of this argument, appellants point out that several courts have accepted two standard deviations as prima facie proof of discrimination. See, e.g., Berger, 843 F.2d at 1412 (if the likelihood that a fluctuation from expected results occurred by chance is five percent or less, a statistically significant difference is proved, and a prima facie case of discrimination is established) (citing Segar, 738 F.2d at 1282-83); Eldredge v. Carpenters 46 N. Cal. Counties JATC, 833 F.2d 1334, 1340 n. 8 (9th Cir.1987) (. 045 level of statistical significance (approximately two standard deviations or 1 chance in 22) sufficient to give rise to an inference that discriminatory system rather than chance is responsible for women's lower admission rates to apprenticeship program), cert. denied, --- U.S. ----, 108 S.Ct. 2857, 101 L.Ed.2d 894 (1988); Dalley v. Michigan Blue Cross/Blue Shield, Inc., 612 F.Supp. 1444, 1451 n. 18 (E.D.Mich.1985) (Most courts and commentators have accepted the .05 level, or 1 in 20 probability, as indicative of statistical significance). While appellants' argument that a finding of two standard deviations should be equated with a prima facie case of discrimination under Title VII is not without initial appeal, we are constrained to reject such a formal litmus test for assessing the legitimacy of Title VII claims. 33 It is certainly true that a finding of two to three standard deviations can be highly probative of discriminatory treatment. See Segar, 738 F.2d at 1282-83. As tempting as it might be to announce a black letter rule of law, however, recent Supreme Court pronouncements instruct that there simply is no minimum threshold level of statistical significance which mandates a finding that Title VII plaintiffs have made out a prima facie case. See, e.g., Watson v. Fort Worth Bank & Trust, --- U.S. ----, 108 S.Ct. 2777, 2789 n. 3, 101 L.Ed.2d 827 (1988) (We have emphasized the useful role that statistical methods can have in Title VII cases, but we have not suggested that any particular number of 'standard deviations' can determine whether a plaintiff has made out a prima facie case in the complex area of employment discrimination.); see also Palmer, 815 F.2d at 92 (noting that Supreme Court has not established an exact legal threshold at which statistical evidence, standing alone, establishes an inference of discrimination); Coser v. Moore, 739 F.2d at 754 n. 3 (a significance level of 5% probability of chance has no talismanic importance); EEOC v. American Nat'l Bank, 652 F.2d 1176, 1192 (4th Cir.1981) (courts of law should be extremely cautious in drawing any conclusions from standard deviations in the range of one to three), cert. denied, 459 U.S. 923, 103 S.Ct. 235, 74 L.Ed.2d 186 (1982); D. Baldus & J. Cole, supra, Sec. 9.4, at 188-89 (Supp.1987) (courts should use tests of statistical significance only as an aid to interpretation and not as a rule of law). Accordingly, in accordance with Supreme Court pronouncements, we must reject appellants' suggestion that this court announce a rule of law with respect to what level of statistical significance automatically gives rise to a rebuttable presumption of discrimination. 34 Moreover, as a practical matter, the issue of whether or not two or more standard deviations establish a prima facie case of discrimination is not pivotal in this case. It is well-established that once a Title VII case has been 'fully tried on the merits,' the question whether the plaintiff has established a prima facie case 'is no longer relevant.'  Mitchell v. Baldrige, 759 F.2d 80, 83 (D.C.Cir.1985) (quoting Aikens, 460 U.S. at 714-15, 103 S.Ct. at 1481-82). As the Supreme Court stated in Bazemore v. Friday, the only issue to be decided at that point is whether the plaintiffs have actually proved discrimination. 478 U.S. at 398, 106 S.Ct. at 3008 (emphasis added). Despite appellants' efforts to deemphasize on appeal what transpired at trial, the district court herein actually proceeded as though the plaintiffs had made out a prima facie case of discrimination, and it cannot be gainsaid that the plaintiffs' claims were fully litigated on the merits. 35 At the close of the plaintiffs' case-in-chief, the defendants moved to dismiss the action specifically on the grounds that plaintiffs had not met their burden of establishing a prima facie case. The court denied the motion without explanation, however, and directed the defendants to proceed with their case. Later, at the close of defendants' rebuttal case, the court also allowed the plaintiffs to respond to the defendants' rebuttal evidence. Accordingly, the question remaining for the district court at the close of the entire trial was whether the plaintiffs should ultimately prevail on their Title VII claims. See Bazemore, 478 U.S. at 398, 106 S.Ct. at 3008. Since the plaintiffs fully litigated those claims, any arguments on appeal directed at whether or not plaintiffs established a prima facie case are arguably misleading and misplaced. See EEOC v. Sears, Roebuck & Co., 839 F.2d at 312 n. 9. 36 The net import of Judge Kram's rulings regarding the significance of plaintiffs' statistical evidence is that she found the evidence to be persuasive but not dispositive. Contrary to appellants' assertions, it is clear from the district judge's rulings that she did not simply ignore the statistical evidence of discrimination presented by plaintiffs. The court found this evidence sufficient to cause her to deny the defendants' motion to dismiss at the end of plaintiffs' case, and to accept rebuttal evidence from the defendants. On rebuttal, however, the defendants were able to successfully undermine the plaintiffs' case by attacking the validity of the plaintiffs' statistical evidence, and by introducing statistical evidence of their own to negate the inference of discrimination that had been raised. Cf. Berger, 843 F.2d at 1416 (Mere conjecture or general assertions of inadequacies in the opponent's statistical case, without demonstrating their effect on the results, will not suffice.); Sobel, 839 F.2d at 34 (same). 37 Specifically, the defendants criticized the plaintiffs' most probative studies for excluding one factor which they claimed exerted a highly significant positive influence[ ] on current salary, namely, whether a faculty member had held a prior, full-time administrative position at SUNY New Paltz before returning to full-time teaching. The defendants also criticized these studies because the salary regressions were fitted only to male faculty members, i.e., they used independent variables that were derived only from the male population. The district court noted in its opinion that a males only regression based exclusively on values existing only in the male population might have tended to overestimate the predicted salaries of certain female faculty members, because it might not have taken into account legitimate factors existing solely in the female population which could have affected the rate of pay for women teachers at the University. See Ottaviani, 679 F.Supp. at 307. If the predicted salary for a female faculty member was overestimated, this type of regression arguably would have overestimated the discrepancies between male and female salaries at the University. Finally, the defendants criticized these studies because they inappropriately aggregated Instructors and Assistant Professors into a single rank. The defendants pointed out at trial that when the two ranks were combined into a single rank, the predicted salary of a female Instructor would essentially be based on the higher salary of an Assistant Professor, and hence the net residual difference between the predicted and actual salaries of a female Instructor would be overstated. Apart from these criticisms of plaintiffs' statistical evidence, the defendants also offered persuasive anecdotal evidence to negate the plaintiffs' claims of discriminatory animus. After considering all of the evidence presented, both statistical and anecdotal, the district court simply found that plaintiffs had failed to preponderate on their claims. 38 Recent Supreme Court precedent has made it clear that this court can reverse such a factual determination only if it is clearly erroneous in light of all the evidence in the record or if it rests on legal error. Palmer, 815 F.2d at 101 (citing, inter alia, Bazemore, 478 U.S. 385, 106 S.Ct. 3000). Especially in cases where statistical evidence is involved,  'great deference is due the district court's determination of whether the resultant numbers are sufficiently probative of the ultimate fact in issue.'  EEOC v. Sears, Roebuck & Co., 839 F.2d at 310 (citation omitted); see Griffin v. Board of Regents of Regency Univ., 795 F.2d 1281, 1289-90 (7th Cir.1986). As the Supreme Court cautioned in the Teamsters case, statistics are not irrefutable; they come in infinite variety and, like any other kind of evidence, they may be rebutted. In short, their usefulness depends on all of the surrounding facts and circumstances. 431 U.S. at 340, 97 S.Ct. at 1857. The district judge herein gave due consideration to all of the evidence presented, and after reviewing the record, we do not perceive a convincing basis for finding her interpretation of that evidence to be clearly erroneous. Accordingly, we affirm her rulings with respect to the statistical evidence presented.