Detecting Adverse Impact: The Four-Fifths Rule Versus Significance Testing

Over the years, numerous landmark Supreme Court cases have clarified many selection discrimination issues (e.g., Griggs v. Duke Power, 1971). However, one issue for which the Supreme Court failed to provide clear guidance is regarding the appropriate decision criteria for adverse impact. Regarding the determination of adverse impact, the Uniform Guidelines on Employee Selection Procedures (U.S. Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and Department of Justice, 1978) include the well-known four-fifths rule and statistical significance testing. Practitioners, compliance officers, and organizational leaders initially gravitated toward the four-fifths rule because it was easier to use than significance tests (Meier, Sacks, & Zabell, 1984). However, the use of significance testing in adverse impact cases became more prevalent in the 1990s, and in a review of recent adverse impact cases, Esson and Hauenstein (2006) concluded that significance testing is now used more often than the four-fifths rule in determining adverse impact. Although the courts look at all available evidence for prima facia cases beyond the four-fifths rule and significance testing, understanding the consequences of choosing either criterion provides guidance for their use. It is well known that the four-fifths rule and statistical significance testing frequently disagree regarding the adverse impact decision (Bobko & Roth, 2004; Morris, 2001; Morris & Lobsenz, 2000; Roth, Bobko, & Switzer, 2006; Sobol & Ellard, 1988). It is further known that the pattern of disagreement is systematic. More specifically, for small sample size and/or small selection ratio (SR) situations, the four-fifths rule is more likely to detect adverse impact than the significance test. For large sample size and/or small SR situations, the significance test is more likely to detect adverse impact than the four-fifths rule. Sobol and Ellard (1988) referred to this change where the significance test is more likely to indicate adverse impact than the four-fifths rule as the "cross-over point." In the present study, we examine this cross-over phenomenon more closely to clarify issues about choosing an adverse impact criterion. Most importantly, we demonstrate that the cross-over point where the significance test is as likely to indicate adverse impact as the four-fifths rule is an excellent criterion for when concerns about small sample sizes and small SRs diminish. Furthermore, we examine the practical consequences of choosing one adverse impact criterion over the other. The ultimate point of this demonstration is to show that although neither adverse impact criteria performs well in small sample size/small SR situations, the practical consequence of this choice is not that great. In contrast, in large sample size/large SR situations, although the significance testing is superior to the four-fifths rule as a criterion for detecting adverse impact, the four-fifths rule may be preferred. Simulation Overview Sobol and Ellard (1988), in their comparison of the four-fifths rule with the binomial test, referred to the "cross-over point" as the point where the significance test becomes more sensitive than the four-fifths rule. However, to describe the change over as a point of inflection is misleading, because to do so requires that somewhere in the process people are represented as "less then whole" (e.g., hiring 122.83 minorities). By limiting the discussion to integers that reflects that only "whole people" change the notion of a cross-over point to that of lower and upper boundaries. For our purposes, we will characterize these boundaries based on the size of the minority group applicant pool for a given majority group applicant pool and majority SR. For a given majority group SR and sample size, the lower boundary is the minority group applicant pool sample size that establishes the largest minority applicant pool where the four-fifths rule is more likely to indicate adverse impact than the significance test being used. …