Psychological Bulletin
1984, Vol96, No. 1,72-98

Copyright 1984 by theAmerican Psychological Association, Inc.

Validity and Utility of Alternative Predictors of Job Performance
John E. Hunter

Ronda F. Hunter

Michigan State University

College of Education
Michigan State University

Meta-analysis of the cumulative research on various predictors of job performanceshows that for entry-level jobs there is no predictor with validity equal to that ofability, which has a mean validity of .53. For selection on the basis of current job
performance, the work sample test, with mean validity of .54, is slightly better. For
federal entry-level jobs, substitution of an alternative predictor would cost from
$3.12 billion (job tryout) to $15.89 billion per year (age). Hiring on ability has autility of $15.61 billion per year, but affects minority groups adversely. Hiring on
ability by quotas would decrease this utility by 5%. A third strategy—using a low
cutoff score—would decrease utility by 83%. Using other predictors in conjunction
with ability tests might improve validity and reduce adverse impact, but there isas yet no data base for studying this possibility.
A crucial element in the maintenance of that over a 10-year period, with a 50% selection
high productivity in both government and pri- ratio, the federal government could save $376
vate industry is the selection of people with million by using a cognitive ability test to selecthigh ability for their jobs. For most jobs, the computer programmers rather than by seonly presently known predictive devices with lecting at random.1
high validity are cognitive and psychomotor
Hunter and Schmidt (1982b) applied a utilability tests. Recent work on validity gener- ity model to the entire national work force.
alization (Schmidt, Gast-Rosenberg, & Hunter, Gains from using cognitive tests rather than1980; Schmidt & Hunter, 1977; Schmidt & selecting at random are not as spectacular at
Hunter, 1980; Schmidt, Hunter, Pearlman, & the national level as might be predicted by
Shane, 1979) has shown that most findings of findings from single organizations. This is below validity are due to artifacts, mainly sta- cause high-ability people who are selected fortistical error due to small sample size. Similar crucial top-level jobs will not be available forand broader conclusions follow from reana- lower-level jobs, where they would bring higher
lyses of past meta-analyses by Lent, Aurbach, productivity. However, even considering thisand Levin (1971), as noted by Hunter (1980b) cancellation of effect, Hunter and Schmidt esand from inspection of unpublished large- timated in 1980 that productivity differences
sample studies done in the United States Army between complete use of cognitive ability tests
by Helm, Gibson, and Brogden (1957), in and no use of the tests would amount to a
Hunter (1980c), and in Schmidt, Hunter, and minimum of $80 billion per year. That is, proPearlman (1981). High selection validity ductivity differences due to use or nonuse oftranslates into considerable financial savings present tests would be, in the current state offor most organizations. Hunter (1979a) esti- the economy, about as great as total corporate
mated that if the Philadelphia, Pennsylvania, profits, or about 20% of the total federal
Police Department were to drop its use of a budget.
cognitive ability test to select entry-level ofTo replace cognitive ability tests with any
ficers, the cost to the city would be more than instrument of lower validity would incur very
$170 million over a 10-year period. Schmidt, high costs, because even minute differences in
Hunter, McKenzie, and Muldrow (1979) show1
Random selection, though rarely if ever practiced,
provides a base line for comparison of utility gains andlosses that best demonstrates the potential magnitude ofthe differences.

Requests for reprints should be sent to John E. Hunter,
Department of Psychology, Michigan State University, East
Lansing, Michigan 48824.

72

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

validity translate into large dollar amounts.
These costs would be shared by everyone, regardless of sex or group affiliation, because
failure to increase productivity enough tocompensate for inflation and to compete inforeign markets affects the entire economy.
Adverse Impact and Test Fairness
Unfortunately, the use of cognitive ability
tests presents a serious problem for American
society; there are differences in the mean ability scores of different racial and ethnic groups
that are large enough to affect selection outcomes. In particular, blacks score, on the average, about one standard deviation lower thanwhites, not only on tests of verbal ability, but
on tests of numerical and spatial ability as well(e.g., see U.S. Employment Service, 1970).
Because fewer black applicants achieve highscores on cognitive ability tests, they are more
likely to fall below selection cutoff scores than
are white applicants. For example, if a test is
used to select at a level equivalent to the top
half of white applicants, it will select only the
top 16% of the black applicants. This differenceis adverse impact as denned by the courts.
Fifteen years ago the elimination of adverse
impact seemed a straightforward but arduous
task: It was a question of merely amending
the tests. Testing professionals reasoned that
if there were no mean difference between racial
groups in actual ability, the mean differencein test scores implied that tests were unfair toblack applicants. If tests were unfair to black
applicants, then it was only necessary to remove all test content that was culturally biased.
Not only would adverse impact then vanish,
but also the validity of the test would increase.
Moreover, if a test were culturally unfair toblacks, it would probably prove to be culturally
unfair to disadvantaged whites as well, andreforming the tests would eliminate that bias
also.
However, the empirical evidence of the last
15 years has not supported this reasoning(Schmidt & Hunter, 1981). Evidence showingthat single-group validity (i.e., validity for one
group but not for others) is an artifact of small
sample sizes (Boehm, 1977; Katzell & Dyer,
1977; O'Connor, Wexley, & Alexander, 1975;
Schmidt, Berner, & Hunter, 1973) has shownthat any test that is valid for one racial group73

is valid for the other. Evidence of differential
validity (i.e., validity differences between
subgroups) to be an artifact of small sample
size (Bartlett, Bobko, Mosier, & Hannan, 1978;
Hunter, Schmidt, & Hunter, 1979) suggeststhat validity is actually equal for blacks andwhites. Finally, there is an accumulation ofevidence directly testing for cultural bias,
showing results that are consistently oppositeof that predicted by the test bias hypothesis.
If test scores for blacks were lower than theirtrue ability scores, then their job performance
would be higher than that predicted by theirtest scores. In fact, however, regression lines
for black applicants are either below or equal
to the regression lines for white applicants (seereview studies cited in Hunter, Schmidt, &Rauschenberger, in press; Schmidt & Hunter,
1980, 1981). This finding holds whether job
performance measures are ratings or objective
measures of performance, such as production
records or job sample measures.
The evidence is clear: The difference inability test scores is mirrored by a corresponding difference in academic achievement andin performance on the job. Thus, the difference
in mean test scores reflects a real difference
in mean developed ability. If the difference isthe result of poverty and hardship, then it will
vanish as poverty and hardship are eliminated.
However, because the difference currently represents real differences in ability, constructionof better tests will not reduce adverse impact.
In fact, better tests, being somewhat more reliable, will have slightly more adverse impact.
Adverse impact can be eliminated from theuse of ability tests, but only by sacrificing theprinciple that hiring should be in order of
merit. If we hire solely on the basis of probablejob performance, then we must hire on ability
from the top down, without regard to racialor ethnic identification. The result will be lower
hiring rates for minority applicants. If we decide to moderate adherence to the merit principle so as to apply the principle of racial andethnic balance, then the method that loses theleast productivity in the resulting work force
is that of hiring on the basis of ability fromthe top down within each ethnic group separately. This method produces exact quotas
while maximizing the productivity of theworkers selected within each ethnic group.
Many organizations are presently using a

74

JOHN E. HUNTER AND RONDA F. HUNTER

third method. They use ability tests, but witha very low cutoff score. This method does reduce adverse impact somewhat, but it reduces
the labor savings to the organization by far
more (Hunter, 1979a, 198 la; Mack, Schmidt,& Hunter, undated). The low-cutoff method
is vastly inferior to quotas in terms of either
productivity or ethnic balance. This point will
be considered in detail in the section on economic analysis.

increase validity and decrease adverse impact.
That is, the most likely successful approachto reduced adverse impact is not through thediscovery of substitutes for ability tests—there
is no known test that approaches cognitive
tests in terms of validity for most jobs—but
through the discovery of how best to use alternative measures in conjunction with cognitive ability tests.
Distinguishing Method From Content

Adverse Impact Gap and the Promise of
Alternative Predictors
As long as there is a true difference in ability
between groups, there will be a gap in theirrelative performance that can never be completely closed. The fact that tests are fair meansthat differences in mean test score between
groups are matched by differences in meanjob performance. However, differences in testscore are not matched by equivalent differencesin job performance, because the differences
are affected by the validity of the test. Figure
1 illustrates this point: If the predicted criterion
score (e.g., performance rating) for any testscore is the same for all applicants, and if test
validity is .50 (the validity for most jobs after
correction for restriction in range and unreliability of criterion measures), then a difference of one standard deviation on the mean
test score corresponds to half a standard deviation's difference on performance.
If the test score cutoff is set to select the top
half of the white applicants, 50% will performabove the mean on the criterion. At the same
cutoff, only 16% of the black applicants will
be selected, although 31% would perform at
or above the mean for white applicants. Improving the test cannot entirely overcome thisdifference. If the test is improved so that thereis half a standard deviation difference on bothtest and criterion, 50% of whites as against
31% of blacks will be selected. (This is thetrue meaning of Thorndike's, 1971, claim thata test that is fair to individuals might be unfair
to groups.)
Although the gap cannot be entirely closed,
it can be narrowed by increasing the validityof the selection process. If we could find predictors of determinants of performance otherthan ability to add to the prediction suppliedby ability tests, then we could simultaneously

In considering the use of alternatives, especially their use in conjunction with ability tests,
it is helpful to distinguish the means of measurement (method) from the specification ofwhat is to be measured (content). Most contemporary discussion of alternative predictorsof job performance is confused by the failure
to make this distinction. Much contemporary
work is oriented toward using ability to predict
performance but using something other thana test to assess ability. The presumption is thatan alternative measure of ability might findless adverse impact. The existing cumulative
literature on test fairness shows this to be afalse hope. The much older and larger literatureon the measurement of abilities also suggested
that this would be a false hope. All large-sample
studies through the years have shown that paper-and-pencil tests are excellent measures ofabilities and that other kinds of tests are usuallymore expensive and less valid. Perhaps an investigation of characteristics other than ability
would break new ground, if content were to
be considered apart from method.
An example of confusion between content
and method is shown in a list of alternative
predictors in an otherwise excellent recent review published by the Office of Personnel
Management (Personnel Research and Development Center, 1981). This review lists
predictors such as reference checks, self-assessments, and interviews as if they were mutually exclusive. The same is true for work
samples, assessment centers, and the behavioral consistency approach of Schmidt, CapIan, et al., 1979). However, reference checks
often ask questions about ability, as tests do;about social skills, as assessment centers do;or about past performance, as experience ratings do. Self-assessments, interviews, and tests
may all be used to measure the same char-ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

acteristics. A biographical application blankmay seek to assess job knowledge by asking
about credentials, or it may seek to assess social
skill by asking about elective offices held.
What characteristics could be assessed thatmight be relevant to the prediction of future
job performance? The list includes: past performance on related jobs; job knowledge; psychomotor skills; cognitive abilities; social skills;
job-related attitudes such as need for achievement (Hannan, 1979), locus of control (Hannan, 1979), or bureaucratic value orientation75(Gordon, 1973); emotional traits such as resistance to fear or stress or to enthusiasm. Suchcharacteristics form the content to be measured in predicting performance.
What are existing measurement methods?
They come in two categories: methods entailing observation of behavior, and methods
entailing expert judgment. Behavior is observed by setting up a situation in which theapplicant's response can be observed andmeasured. The observation is valid to the extent that the count or measurement is related

Criterion

Figure 1. This shows that a fair test may still have an adverse impact on the minority group. (From "Conceptsof Culture Fairness" by R. L. Thorndike, 1971, Journal of Educational Measurement, 8, pp. 63-70. Copyright1971 by the National Council on Measurement in Education. Reprinted by permission.)

76

JOHN E. HUNTER AND RONDA F. HUNTER

to the characteristic. Observation of behavior data base for studies of judgment is largest forincludes most tests, that is, situations in which self-judgments, second largest for judgments
the person is told to act in such a way so as by knowledgeable others, and smallest for
to produce the most desirable outcome. The judgments by strangers. However, the likeliresponse may be assessed for correctness, hood of distortion and bias in such studies is
speed, accuracy, or quality. Behavior may also in the same order.
be observed by asking a person to express a
Figure 2 presents a scheme for the classichoice or preference, or to make a judgment fication of potential predictors of job perforabout some person, act, or event. A person's mance. Any of the measurement procedures
reports about memories or emotional states along the side of the diagram can be used to
can also be considered as observations, al- assess any of the characteristics along the top.
though their validity may be more difficult to Of course the validity of the measurement may
demonstrate. If judgment is to be used as the vary from one characteristic to another, andmeasurement mode, then the key question is the relevance of the characteristics to particular"Who is to judge?" There are three categories jobs will vary as well.
for judges: self-assessments, judgments by
Meta-Analysis and Validity Generalization
knowledgeable others such as supervisor, parent, or peer, and judgments by strangers such
In the present study of alternative predictors
as interviewers or assessment center panels. of job performance, the results of hundreds
Each has its advantages and disadvantages. The of studies were analyzed using methods called

METHOD(Means of Measurement)

Category

Informal,on thejob
Observation

MethodInspection &Appraisal
Incidence
CountsCoding of Company Records
Speed

CONTENT(Aspect of Behavior or Thought)

Perfor- Knowmance
ledge

Psychomotor
SocialAbility Skill
Skill

Attitude

Emotional
Trait

Accuracy
Controlled
Measurement(Tests)

Quality
Adaptiveness& Innovation
ChoiceSelf
JudgmentofTraits
Judgment

Informed
Other
Stranger
Self

Judgmentof Relevant Experience

Informed
Other
Stranger

Figure 2. Measurement content and method: A two-dimensional structure in which various procedures areapplied to the assessments of various aspects of thought and behavior.

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

validity generalization in the area of personnel
selection and called meta-analysis in education
or by people using the particular method recommended by Glass (1976). The simplest formof meta-analysis is to average correlations
across studies, and this was our primary analytic tool. However, we have also used themore advanced formulas from personnel selection (Hunter, Schmidt, & Jackson, 1982),
because they correct the variance across studies
for sampling error. Where possible we havecorrected for the effects of error of measurement and range restriction. Because validity
generalization formulas are not well known
outside the area of personnel selection, we review the basis for such corrections in this section.
The typical method of analyzing a large
number of studies that bear on the same issue
is the narrative review. The narrative review
has serious drawbacks, however (Glass,
McGaw, & Smith, 1981; Hunter, Schmidt, &Jackson, 1982; Jackson, 1980). Much better
results can be obtained by quantitative analysisof results across studies. Glass (1976) coined
the word meta-analysis for such quantitative
analysis. A variety of meta-analysis methods
have been devised, including the counting of
ppsitive and negative results, the counting ofsignificance tests, and the computing of meansand variances of results across studies. The
strengths and weaknesses of the various methods are reviewed in Hunter, Schmidt, andJackson (1982).
The most common method of meta-analysis
is that which was proposed by Glass (1976),
and was presented in detail in Glass,
McGaw, and Smith (1981). The steps in thismethod are:
1. Gather as many studies on the issue as
possible.
2. Code the data so that the outcome measure in each study is assessed by the same
statistic.
3. Assess the general tendency in the findings by averaging the outcome statistic acrossstudies.
4. Analyze the variance across studies bybreaking the studies down into relevant
subgroups and comparing the means of thesubgroups.
In the area of personnel selection, the Glass(1976) method has been in use for at least 50

77

years (Dunnette, 1972; Ghiselli, 1966, 1973;
Thorndike, 1933). Ghiselli gathered hundredsof validation studies on all methods of predicting job performance. Each validation study
was coded to yield a validity coefficient, that
is, a Pearson correlation coefficient between
the predictor of job performance (such as anability test score) and the measure of job performance (such as a rating of performance by
the worker's immediate supervisor). Ghiselli
then averaged validity coefficients across studies (although he used the median instead ofthe mean). He analyzed the variance of validity
across studies by breaking the studies down
into families of jobs and comparing averagevalidity across job families.
The Glass (1976) method has one main deficiency. If the variance across studies is taken
at face value, then the problem of sampling
error is neither recognized nor dealt with adequately (Hedges, 1983; Hunter, 1979b;
Hunter,-Schmidt, & Jackson, 1982; Schmidt& Hunter, 1977). Schmidt and Hunter (1977)
showed that ignoring sampling error leads todisastrous results in the area of personnel selection. Because he ignored the effect of sampling error on the variance of findings acrossstudies, Ghiselli (1966, 1973) concluded thattests are only valid on a sporadic basis, thatvalidity varies from one setting to another because of subtle differences in job requirementsthat have not yet been discovered. He therefore
concluded that no predictor could be used forselection in any given setting without justifying
its use with a validation study conducted inthat exact setting.
The following example shows how different
conclusions may be drawn when sampling error is considered. In the present study we foundthat the average population correlation for theinterview as a predictor of supervisor ratingsof performance is. 11 (. 14 if corrected for errorof measurement in ratings). For simplicity, let
us suppose that the population correlation for
the interview in every setting is .11. Let us
also suppose that all studies on the interview
were done with a sample size of 68 (Lent,
Aurbach, & Levin, 1971, found that the average sample size across more than 1,500
studies was 68), Then the average observed
correlation across studies would be close to. 11 (depending on the total sample size acrossstudies), but the standard deviation would be

78

JOHN E. HUNTER AND RONDA F. HUNTER

far from zero; the observed standard deviation
would be. 12. The correlations would be spreadover a range from —.25 to +.47. A little over
16% of the correlations would be negative, anda little over 16% would be .20 or greater.
The correct conclusion from this example
is, "The validity of the interview is . 11 in all
settings." However, if the variance of results
across studies is taken at face value, as it is in
Glass's (1976) method, then the conclusion is,"The validity of the interview varies enormously from one setting to the next. In about
one sixth of settings, however, the validity
reaches a reasonable level of .20 or better." If
the number of studies is small, then some feature of the studies would by chance be correlated with the variation in observed correlations, and the reviewer would say something
like, "Studies done with female interviewers
were more likely to produce useful levels ofvalidity." If the number of studies is large, thenno study feature would be correlated with outcome and the conclusion would probably read,"The validity of the interview varies according
to some subtle feature of the organization
which is not yet known. Further research isneeded to find out what these moderator variables are."Use of the significance test in this context
adds to the confusion. The correlation would
be significant in only 14% of the studies (thatis, it would be wrong in 86% of the studies,reflecting the usual low level of power is psychological studies in which the null hypothesis
is false), leading to the false conclusion, "Theinterview was found to be valid in only 14%
of studies—9% greater than chance—and thus
cannot be used except in that 9% of settings
in which a local validation study has shown
it to be valid."
It is possible to correct the variance acrossstudies to eliminate the effect of sampling error.
Schmidt and Hunter (1977) devised a formulafor doing so under the rubric of validity generalization, though that first formula had anerror that was corrected in Schmidt, Hunter,
Pearlman, and Shane (1979). Schmidt andHunter and others (Callender & Osburn, 1980;
Hedges, 1983; Raju & Burke, in press) havedeveloped a variety of formulas since then.
The historical basis for these formulas is reviewed in Hunter, Schmidt, and Pearlman(1982), and the mathematical basis for com-paring them is given in Hunter, Schmidt, andJackson (1982). In the context of personnel
selection, the formulas all give essentially
identical results.
Schmidt and Hunter (1977) noted that study
outcomes are distorted by a number of artifacts
other than sampling error. They presented
meta-analysis formulas that correct for errorof measurement and range restriction as well
as sampling error. The comparable formulasof Hunter, Schmidt, and Jackson (1982) wereused when applicable in this study. However,there are artifacts that are not controlled byeven these formulas, the most important of
which are bad data, computational errors,
transcriptional errors, and typographical errors. Thus, even the corrected standard deviations reported in the present study areoverestimates of the actual variation in validity
across jobs.
Productivity Gains and Losses
A further advantage of meta-analysis is thatthe validity information from the meta-analysis can be combined with estimated costs toform a utility analysis for the use of alternative
predictors in place of written tests. Dollar impact figures can be generated in a form appropriate for use by individual firms or organizations. The cost of testing is a significant
part of this impact. Because the cost of paperand-pencil tests is negligible in comparison
with the gains to be obtained from a selection
process, this factor has usually been ignoredin the literature. Cost is a more significant
factor, however, for assessment centers, miniature training and evaluation, and probationary periods; taking cost into consideration
adds to the accuracy of utility estimates for
all predictors.
Design of This Study
This study was begun with the intent ofusing meta-analyses to study the cumulative
research on alternative predictors in four ways:
validity of the alternative predictor consideredalone and in conjunction with ability tests,
and adverse impact of the predictor consideredalone and in conjunction with ability tests.
However, we could discover a data base for
only one of the four: validity of the predictor

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

79

considered alone. Few studies consider more matches that for cognitive ability, then the corthan one predictor on the same data set. Fur- rect correlation would be .19.
thermore, most of the studies of multiple preNo correction for restriction in range was
dictors do not present the correlations between needed for biographical inventories because it
the predictors, so that multiple regression can- was always clear from context whether the
not be done. The lack of data on adverse im- study was reporting tryout or follow-up validpact has to do with the age of the studies. ity, and follow-up correlations were not used
Because early studies found low validity for in the meta-analyses.
alternative predictors, research on alternative
Studies analyzed. This study summarizes
predictors tended to die out by the time that results from thousands of validity studies.
analysis by racial group began to become rou- Twenty-three new meta-analyses were done fortine. The sparse literature that does exist is this review. These analyses, reported in Tablesummarized in Reilly and Chao (1982).
8, review work on six predictors: peer ratings,
In all the new meta-analyses conducted for biodata, reference checks, college grade point
this study, it was possible to correct the vari- average (GPA), the interview, and the Strong
ance of validity for sampling error. Corrections Interest Blank. These meta-analyses were
for reliability and range restriction were made based on a heterogeneous set of 202 studies:when the necessary .data were available, as 83 predicting supervisor ratings, 53 predicting
noted below. Correction for error of mea- promotion, 33 predicting training success, andsurement in job performance was done using 33 predicting job tenure.
the upper bound interrater reliability of .60
Table 1 reanalyzes the-meta-analysis done
for ratings (King, Hunter, & Schmidt, 1980) by Ghiselli (1966, 1973). Ghiselli summarized
and a reliability of .80 for training success as thousands of studies done over the first 60
measured by tests. Except for ability, corre- years of this century. His data were not availlations are not corrected for restriction in able for analysis using state-of-the-art methods.
range. Empirical values for range restriction
Tables 1 through 7 review meta-analyses
are known only for ability tests, and most of done by others. Table 2 contains a state-ofthe alternative predictors considered here are the-art meta-analysis using ability to predict
relatively uncorrelated with cognitive ability. supervisor ratings and training success. This
Also, when Hunter (1980c) analyzed the study analyzed the 515 validation studies carUnited States Employment Service data base, ried out by the United States Employment
he found incumbent-to-applicant standard Service using its General Aptitude Test Battery
deviation ratios of .67, .82, and .90 for cog- (GATE). Tables 3 through 6 review comparnitive, perceptual, and psychomotor ability, ative meta-analyses by Dunnette (1972), Reilly
respectively. The figure of .90 for psychomotor and Chao (1982), and Vineberg and Joyner
ability suggests that restriction in range on al- (1982), who reviewed 883, 103, and 246 studternative predictors stems largely from indirect ies, respectively. Few of the studies reviewed
restriction due to selection using ability.
by Reilly and Chao date as far back as 1972;
There are two predictors for which restric- thus, their review is essentially independent oftion in range may be a problem: the interview Dunnette's. Vineberg and Joyner reviewed
and ratings of training and experience. For only military studies, and cited few studies
example, some of the studies of the interview found by Reilly and Chao. Thus, the Vineberg
might have been done in firms using the in- and Joyner meta-analysis is largely indepenterview for selection, in which case restriction dent of the other two. Reilly's studies are inin range was present. The key finding for the cluded in the new meta-analyses presented ininterview is a mean validity of .14 (in pre- Table 8.
dicting supervisor ratings of performance,
Tables 6 and 7 review meta-analyses of acorrected for error of measurement). If the more limited sort done by others. Table 6 prerestriction in range is as high as that for cog- sents unpublished results on age, education,
nitive ability, then the true mean validity of and experience from Hunter's (1980c) analysis
the interview is .21. The observed average cor- of the 515 studies done by the United States
relation for ratings of training and experience Employment Service. Table 6 also presents
is .13. If the extent of restriction in range meta-analyses on job knowledge (Hunter,

80

JOHN E. HUNTER AND RONDA F. HUNTER

1982), work sample tests (Asher & Sciarrino, methods of job classification to data on train1974), and assessment centers (Cohen, Moses, ing success; they found no method of job anal& Byham, 1974), reviewing 16, 60, and 21 ysis that would construct job families for which
studies, respectively.
cognitive ability tests would show a significantly different mean validity. Cognitive ability
has a mean validity for training success of
Validity of Alternative Predictors
about .55 across all known job families
This section presents the meta-analyses that (Hunter, 1980c; Pearlman, 1982; Pearlman &cumulate the findings of hundreds of valida- Schmidt, 1981). There is no job for whichtion studies. First, the findings on ability tests cognitive ability does not predict training sucare reviewed. The review relies primarily on cess. Hunter (1980c) did find, however, thata reanalysis of GhiseUi's lifelong work (Ghiselli, psychomotor ability has mean validities for1973), and on the first author's recent meta- training success that vary from .09 to .40 acrossanalyses of the United States Employment job families. Thus, validity for psychomotor
Service data base of 515 validation studies ability can be very low under some circum(Hunter, 1980c). Second, past meta-analyses stances.
are reviewed, including major comparative reRecent studies have also shown that ability
views by Dunnette (1972), Reilly and Chao tests are valid across all jobs in predicting job(1982), and Vineberg and Joyner (1982). proficiency. Hunter (1981c) recently reanaThird, new meta-analyses are presented using lyzed GhiseUi's (1973) work on mean validity.
the methods of validity generalization. Finally, The major results are presented in Table 1,
there is a summary comparison of all the pre- for GhiseUi's job families. The families havedictors studied, using supervisor ratings as the been arranged in order of decreasing cognitive
measure of job performance.
complexity of job requirements. The validityof cognitive ability decreases correspondingly,
with a range from .61 to .27. However, even
Predictive Power of Ability Tests
the smallest mean validity is large enough to
There have been thousands of studies as- generate substantial labor savings as a result
sessing the validity of cognitive ability tests. of selection. Except for the job of sales clerk,
Validity generalization studies have now been the validity of tests of psychomotor ability inrun for over 150 test-job combinations (Brown, creases as job complexity decreases. That is,
1981; Callender & Osburn, 1980, 1981; Lil- the validity of psychomotor abUity tends to be
ienthal & Pearlman, 1983; Linn, Harnisch, & high on just those job families where the vaDunbar, 1981; Pearlman, Schmidt, & Hunter, lidity of cognitive ability is lowest. Thus, mul1980; Schmidt et al., 1980; Schmidt & Hunter, tiple correlations derived from an optimal
1977; Schmidt, Hunter, & Caplan, 198la, combination of cognitive and psychomotor
1981b; Schmidt, Hunter, Pearlman, & Shane, ability scores tend to be high across all job
1979; Sharf, 1981). The results are clear: Most families. Except for the job of sales clerk, whereof the variance in results across studies is due the multiple correlation is only .28, the multo sampling error. Most of the residual variance tiple correlations for ability range from .43
is probably due to errors in typing, compu- to .62.
tation, or transcription. Thus, for a given testThere is a more uniform data base forjob combination, there is essentially no vari- studying validity across job families than Ghiation in validity across settings or time. This seUi's heterogeneous collection of validationmeans that differences in tasks must be large studies. Over the last 35 years, the United
enough to change the major job title before States Employment Service has done 515 valthere can be any possibility of a difference in idation studies all using the same aptitude batvalidity for different testing situations.
tery. The GATE has 12 tests and can be scored
The results taken as a whole show far less for 9 specific aptitudes (U.S. Employment Servariability in validity across jobs than had been vice, 1970) or 3 general abilities. Hunter
anticipated (Hunter, 1980b; Pearlman, 1982; (1980a, 1980b; 1981b, 1981c) has analyzed
Schmidt, Hunter, & Pearlman, 1981). In fact this data base and the main results are prethree major studies applied most known sented in Table 2. After considering 6 major

81

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

Table 1
Hunter's (1981c) Reanalysis ofGhiselli's (1973) Work on Mean Validity
Mean validity

Beta weight

Job families

Cog

Per

Mot

Cog

Mot

R

Manager
Clerk
Salesperson
Protective professions worker
Service worker
Trades and crafts worker
Elementary industrial worker
Vehicle operator
Sales clerk

.53
.54
.61
.42
.48
.46
.37
.28
.27

.43
.46
.40
.37
.20
.43
.37
.31
.22

.26
.29
.29
,26
.27
.34
.40
.44
.17

.50
.50
.58
.37
.44
.39
.26
.14
.24

.08

.53
.55
.62
.43
.49
.50
.47
.46
.28

.1-2

.09
.13
.12
.20
.31
.39
.09

Note. Cog = general cognitive ability; Per = general perceptual ability; Mot = general psychomotor ability; R = multiple
correlation. Mean validities have been corrected for criterion unreliability and for range restriction using mean figures
for each predictor from Hunter (1980c) and King, Hunter, and Schmidt (1980).

methods of job analysis, Hunter found thatthe key dimension in all methods was the dimension of job complexity. This dimension islargely captured by Fine's (1955) data dimension, though Fine's things dimension did define
2 small but specialized industrial job families:
set-up work and jobs involving feeding/offbearing. (Fine developed scales for rating all
job according to their demands for dealing
with data, things, and people.) The present
analysis therefore groups job families by levelof complexity rather than by similarity of tasks.
Five such job families are listed in Table 2,
along with the percentage of U.S. workers ineach category. The principal finding for theprediction of job proficiency was that although
the validity of the GATE tests varies acrossjob families, it never approaches zero. The
validity of cognitive ability as a predictor decreases as job complexity decreases. The validity of psychomotor ability as a predictor
increases as job complexity decreases. Thereis corresponding variation in the beta weights
for the regression equation for each job family
as shown in Table 2. The beta weights forcognitive ability vary from .07 to .58; the beta
weights for psychomotor ability vary from .00
to .46. The multiple correlation varies little
across job families, from .49 to .59 with anaverage of .53.
The results of all cumulative analyses arecongruent. If general cognitive ability alone isused as a predictor, the average validity acrossall jobs is .54 for a training success criterionand .45 for a job proficiency criterion. Regarding the variability of validities across jobs,
for a normal distribution there is no limit as
to how far a value can depart from the mean.
The probability of its occurrence just decreases
from, for example, 1 in 100 to 1 in 1,000 to
1 in 10,000, and so forth. The phrase effective
range is used for the interval that captures themost values. In the Bayesian literature, theconvention is 10% of cases. For tests of cognitive ability used alone, then, the effective
range of validity is .32 to .76 for training success and .29 to .61 for job proficiency. However,
if cognitive ability tests are combined withpsychomotor ability tests, then the average validity is .53, with little variability across job
families. This very high level of validity is thestandard against which all alternative predictors must be judged.

Past Comparisons of Predictors
Three meta-analytic studies (Reilly & Chao,
1982; Dunnette, 1972; Vineberg & Joyner,1982) have compared the validity of different
predictors; two studies were completed after
the present study began. All three studies havebeen reanalyzed for summary here.
Table 3 presents a summary of the findingsof Dunnette (1972). The predictors are presented in three groups. Group 1 shows Dunnette's data as reanalyzed by the present authors in terms of three general abilities tests
rather than in terms of many specific aptitudes.

82

JOHN E. HUNTER AND RONDA F. HUNTER\ oo fi

o ON

2
1
I

2
o

o

Tt

o

H
c
E

o

i —
i in

o<
<4- <

03

—
-^ m
Sm

S

£
•- 8
a a
•s »»

m <N ^t

m

g
£

O
1

as

Tested

t!
s?
a-°.
w^

«
X

o a

«.&

The resulting validities are comparable withthose found in the United States Employment
Service data base as shown in Table 2. Group2 includes three predictors that might be used
for entry-level jobs. Of these, only biodata has
a validity high enough to compare with thatof ability. The interview is a poor predictor,
and amount of education as a predictor did
not work at all for the jobs Dunnette was interested in. Group 3 includes two predictorsthat can be used only if examinees have been
specifically trained for the job in question: job
knowledge and job tryout. The validity of .51
for job knowledge is comparable with the validity of .53 for ability tests. The validity of.44 for job tryout is lower than one might
expect. The most likely cause of the low validity of job tryout is the halo effect in supervisor ratings. Because of the halo effect, if
the performance of the worker on tryout wasevaluated by a supervisor rating, then the upper bound on the reliability of the tryout measurement would be .60 (King et al., 1980). In
that case, error in supervisor ratings reduces
the reliability of the predictor rather than thatof the criterion. If so, then job tryout withperfect measurement of job performancemight have a validity of .57 (corrected using
a reliability of .60) rather than the observed
validity of .44.
Table 4 presents a summary of the findingsof Reilly and Chao (1982). Most of their findings are included in the meta-analyses presented in Table 8. The validity of .38 for biodata is comparable with the .34 found by
Dunnette (1972), and the finding of .23 forthe interview is roughly comparable with the. 16 found by Dunnette. The finding of. 17 foracademic achievement, however, is muchhigher than the zero for education Dunnette
found. Dunnette used amount of education
as the predictor, however, whereas Reilly andChao used grades. One would expect grades
to be more highly correlated with cognitive
ability than amount of education. For thatreason, grades would have higher validity thanamount of education.
Reilly and Qiao's (1982) estimate of somevalidity for self-assessments is probably anoverestimate. Under research conditions people can provide reasonable self-assessments on
dimensions—for example typing speed (Ash,1978, r = .59), spelling (Levine, Flory, & Ash,

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

Table3
Meta-Analysis Derived From Dunnette (1972)
Predictors
Group 1
Cognitive ability

Perceptual ability
Psychomotor abilityGroup 2
Biographicalinventories
Interviews
Education
Group 3
Job knowledge

Job tryout

No. correlations

Averagevalidity

215
97
95

.45
.34
.35

115
30
15

.34

296
20

.51
,44

.16
.00

83

correlated the self-assessments of welfare
counselors with supervisor ratings and founda correlation of-.02 (sample size = 104). Van
Rijn and Payne (1980) correlated self-estimateson 14 dimensions of fire fighting with objective
measures such as work sample tests. Only 2of the 14 correlations were significant, andboth were negative (sample size = 209). Farleyand Mayfield (1976) correlated self-assessments with later performance ratings for 1,128
insurance salespersons and found no significant relation (hence, r = .05 for this samplesize). The average validity of self-assessments
across these 4 studies is zero. Self-assessments
appear to have no validity in operational settings.
Reilly and Chao (1982) also found that projective tests and handwriting analysis are notvalid predictors of job performance.
Table 5 presents the summary results of theVineberg and Joyner (1982) study, corrected
for error of measurement in ratings using a
reliability of .60. These researchers reviewed
only military studies. The most striking featureof Table 5 is the low level of the validity coefficients. The rank order of the correlations forsupervisor rating criteria is about the same asin other studies (i.e., training performance,
aptitude, biodata, education, interest), but thelevel is far lower for all variables except education. This fact may reflect a problem withmilitary operational performance ratings, evidence for which can be found in one of theVineberg and Joyner tables. They found that
if a performance test was used as the criterion,
then the validity was more than twice as highas when performance ratings were used as thecriterion. Many of the ratings were actually

1977, r = .58), and intelligence (DeNisi &Shaw, 1977, r = .35)—on which they received
multiple formal external assessments. However, there is a drastic drop in validity for selfassessments of dimensions on which persons
have not received external feedback. Some examples are typing statistical tables (Ash, 1978,
r = .07), filing (Levine, Flory, & Ash, 1977,
r = .08), and manual speed and accuracy(DeNisi & Shaw, 1977, r - .19). As the assessment shifts from task in school to task onjob, the validity drops to very low levels. Furthermore, these correlations are misleadingly
high because they are only indirect estimatesof validity. For example, if a person's self-assessment of intelligence correlates .35 with hisor her actual intelligence, and if intelligencecorrelates .45 with job performance, then according to path analysis the correlation between self-assessment and job performance
would be about .35 X .45 = .16. This correlation would be even lower if self-assessments
in hiring contexts proved to be less accurate
than self-assessments under research condi- Table 4
Meta-Analysis Derived From Reilly
tions.
We consider four studies that actually used and Chao (1982)
self-assessments in validation research. Bass
No. correand Burger (1979) presented international
Predictor
lations
comparisons of the validity of self-assessments
44of performance in management-oriented sit- Biographical inventory
11
Interview
uational tests measuring such traits as lead- Expert
16
recommendation
ership style and attitude toward subordinates. Reference check7
The average correlation with advancement, Academic achievement
10
7
across a total sample size of 8,493, was .05, Self-assessment
5
testswith no variation after sampling error was Projective
Handwriting analysis
3eliminated. Johnson, Guffey, and Perry (1980)

Averagevalidity

.38
.23
.21
.17
.17
Some
LittleNone

84

JOHN E. HUNTER AND RONDA F. HUNTER

Table 5

Meta-Analysis of Military Studies Using
Supervisory Ratings as the Criterion
Performance measures
Globalrating

Predictor

SuitabilityAllratings

Number of correlations
Aptitude
Training performance
Education
Biographical inventory
Age
Interest

101
51
25
12

11
7
10
4
10

112
58
35
16
10
15

.35
.29
.36
.29
.21

.28
.28
.25
.24
.21
.13

15

Average validity"
Aptitude
Training performance
Education
Biographical inventory
Age
Interest

.21
.27
.14
.20
.13

validity for job knowledge tests is high—.48
with supervisor ratings and .78 with work
sample performance tests. However, the useof job knowledge tests is limited by the factthat they can only be used for prediction if
the examinees are persons already trained forthe job.
The same can be said for the work sample
tests considered by Asher and Sciarrino (1974).
These tests are built around the assumptionthat the examinee has been trained to the task.
In fact, the verbal work sample tests in theirstudy differ little from job knowledge tests,
especially if knowledge is broadly denned toinclude items assessing application of knowledge to the tasks in the job. The motor work
samples, from jobs that include such tasks ascomputer programming and map reading, areTable 6

Other Meta-Analysis Studies
Note. Derived from Vineberg and Joyner (1982).

" Global ratings, suitability, and all ratings were corrected
for error of measurement using a reliability of .60.
suitability ratings (the military term for ratingsof potential ability). The difference between
suitability ratings and performance ratings issimilar to the differences between ratings ofpotential and ratings of performance that has
shown up in assessment center studies, and isdiscussed later.
Table 6 presents the results for four othermeta-analytic studies. The data on age, education, and experience were taken from theUnited States Employment Service data base
(Hunter, 1980c). The validity results for age
in Table 6 were gathered on samples of thegeneral working population. Thus, these samples would have included few teenagers. However, there have been military studies that reported noticeably lower performance in recruits aged 17 and younger. That is, although
there were no differences as a function of age
for those 18 years and older, those 17 years ofage and younger were less likely to complete
training courses. This was usually attributed
to immaturity rather than to lack of ability.
Hunter's (1982) job knowledge study largely
involved studies in which job knowledge wasused as a criterion measure along with work
sample performance or supervisor ratings. The

Study/predictor
Hunter (1980c)
Age
Education
Experience
Hunter (1982)
Content valid

job knowledge test

Meanvalidity

No. correlations

Training
Proficiency
Overall
Training
Proficiency
Overall
Training
Proficiency
Overall

-.01
-.01
-.01

.20
.10
.12
.01
.18
.15

90
425
515
90
425
515
90
425
515

Work
samplePerformance

.78

11

.48

10

Criterion

ratings
Asher &

Sciarrino(1974)
Verbal work
sampleMotor work
sampleCohen et al.
(1974)
Assessment
centerTraining
Proficiency
Training
Proficiency

.55
.45
.45
.62

Promotion
Performance

.63'
.43"

30 verbal
30 motor

Note. For Hunter (1980c), 90 studies used the training
criterion, 425 studies used the proficiency criterion; for
Hunter (1982), a total of 16 studies were used; for Cohenet al. (1974), a total of 21 studies were used.
1
Median correlation.
b
Corrected for attenuation.

85

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

Table 7
Meta-Analyses of Three Predictors
Study/predictor

Criterion

No.
studiesNo.
subjects

MeanvalidityNo.
correlations

Promotion
Supervisor ratings
Training success

13
31
7

6,909
8,202
1,406

.49
.49
.36

13
31
7

Promotion
Supervisor ratings'
TenureTraining success

17
11
2
3

6,014
1,089

.21
.11
.05
.30

17
11
2
3

.13»

65
5

Kane &Lawler( 1978)

Peer ratings

O'Leary (1980)
College grade point average

Schmidt, Caplan, et al. (1979)
and Johnson et al. (1980)
Traditional t & e rating
Behavioral consistency
e rating

181
837

.49

Note, t = training; e = experience.
" Corrected for unreliability of supervisor ratings.

harder to describe, but are apparently distinguished by a subjective assessment of the overlap between the test and the job.
The assessment center meta-analysis is now
discussed.
New Meta-Analyses
Table 7 presents new meta-analyses by theauthors that are little more than reanalyses ofstudies gathered in three comprehensive narrative reviews. Kane & Lawler (1978) missed
few studies done on peer ratings, and O'Leary(1980) was similarly comprehensive in regard
to studies using college grade point average.
Schmidt, Caplan, et al. (1979) completely reviewed studies of training and experience ratings.
Peer ratings have high validity, but it should
be noted that they can be used only in very
special contexts. First, the applicant must already be trained for the job. Second, the applicant must work in a context in which hisor her performance can be observed by others.
Most peer rating studies have been done on
military personnel and in police academies for
those reasons. The second requirement is wellillustrated in a study by Ricciuti (1955), conducted on 324 United States Navy midshipmen. On board the men worked in different
sections of the vessel and had only limited
opportunity to observe each other at work. In

the classroom each person was observed by
all his classmates. Because of this fact, eventhough the behavior observed on summer
cruises is more similar to eventual naval duty
than behavior during the academic term, thepeer ratings for the cruise had an average validity of only .23 (uncorrected), whereas theaverage validity for peer ratings during academic terms was .30 (uncorrected).
Schmidt, Caplan, et al. (1979) reviewed
studies of training and experience ratings. The
researchers contrasted two types of procedures:
traditional ratings, which consider only
amounts of training, education, and experience; and behavioral consistency measures,
which try to assess the quality of performancein past work experience. They reported on
two meta-analytic studies on traditional training and experience ratings: Mosel (1952) found
an average correlation of .09 for 13 jobs, andMolyneaux (1953) found an average correlation of .10 for 52 jobs. If we correct for theunreliability of supervisor ratings, then the average for 65 jobs is .13. Schmidt, Caplan, etal. (1979) reported the results of four studies
that used procedures similar to their proposed behavioral consistency method: Primoff(1958), Haynes (undated), and two studies by
Acuff (1965). In addition, there has since beena study that used their procedure—Johnson
et al. (1980). The meta-analysis of these fivestudies shows an average validity of .49, which

86

JOHN E. HUNTER AND RONDA F. HUNTER

is far higher than the validity of traditional
ratings.
However, the comparison of traditional
training and experience ratings with the behavioral consistency method is to some extent
unfair. The behavioral consistency methoddata reviewed here were from situations inwhich the applicants had already worked in
the job for which they were applying. Traditional ratings have often been applied in entrylevel situations where the person is to be
trained for the job after hiring. Generalized
behavioral consistency scales have been constructed and used, but there have been no cri-

terion-related validation studies of such scales
to date.
Table 8 presents the new meta-analyses done
for this report. Supervisor ratings were corrected for error of measurement. Promotion
in most studies meant job level attained (i.e.,salary corrected for years with the company
or number of promotions), although several
studies used sales. Training success refers totraining class grades, although most studies
used a success-failure criterion. Tenure refers
to length of time until termination.
There is a theoretically interesting comparison between training success findings and su-

Table 8
New Meta-Analyses of the Validity of Some Commonly Used Alternatives to Ability Tests
Alternative measure

Averagevalidity

SDofvalidityNo.
correlations

Total
sample size

31
12
10
11
10
3

8,202
4,429
5,389
1,089
2,694
1,789

13
17
3
17
5
4

6,909
9,024

7
11
1
3
9
2

1,406
6,139
1,553

23
2
2
3
3

10,8002,018

Criterion: supervisor ratings
Peer ratings
Biodata"
Reference checks
College GPA
Interview
Strong Interest Inventory

.49
.37
.26
.11
.14
.10

Peer ratings

.49
.26
.16
.21
.08
.25

.15
.10
.09
.00
.05
.11

Criterion: promotion
Biodata"

Reference checks
College GPA
Interview
Strong Interest Inventory

.06
.10
.02
.07
.00
.00

415
6,014
1,744

603

Criterion: training successPeer ratings

Biodata'
Reference checks
College GPA
Interview
Strong Interest Inventory

.36
.30
.23
.30
.10
.18

.20
.11
.16
.07
.00

837
3,544

383

Criterion: tenure
Peer ratingsb
Biodata"
Reference checks
College GPA
Interview
Strong Interest Inventory
Note. GPA = grade point average.
1
Only cross validities are reported.
b
No studies were found.

.26
.27
.05
.03
.22

.15
.00
.07
.00
.11

181
1,925
3,475

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

pervisor rating findings. Table 2 shows thatfor cognitive ability, the validity is usuallyhigher for training success than for supervisor
ratings. This is not generally true for the otherpredictors in Table 8 (or for psychomotor ability either). This fits with the higher emphasis
on learning and remembering in training.
In Table 8, it is interesting to note that acrossall jobs, the average validity of college GPA
in the prediction of promotion was .21 witha standard deviation of .07. A standard deviation of .07 is large in comparison with amean of .21. A further analysis reveals thereason for this variation. Additional analysis
by the authors shows an average validity of
GPA for managers of .23 across 13 studies
with a total sample size of 5,644. The averagevalidity for people in sales, engineering, andtechnical work was -.02 across four studies
with a sample size of 370. This might meanthat the motivational aspects of achieving highcollege grades are more predictive of motivation in management than in other professions. It might also mean that college grades
are considered part of the management evaluation process. That is, it may be that college
grades are a criterion contaminate in management promotion. These hypotheses could
be tested if there were more studies comparing
college grades with supervisor ratings, but only
one such study was found for managers andit had a sample size of only 29.
The validity coefficients in Table 8 are notfree of sampling error, especially those where
the total sample size is below 1,000. For example, the average correlation of .05 for college
GPA as a predictor of tenure is based on a
total sample size of only 181. The 95% confidence interval is .05 + .15. However, the average correlations for supervisor ratings are
all based on total sample sizes of 1,789 ormore, with an average of 4,753.

87

biodata for ability would mean a loss of 30%of the saving gained by using ability tests rather
than random selection, whereas other predictors would cause larger losses. However, theoperational validity of biodata instruments
may be lower than the research validity forseveral reasons.
Biodata predictors are not fixed in theircontent, but rather constitute a method forconstructing fixed predictors. The data reported in the literature are based on specializedbiodata instruments; each study used a keyedbiographical form where the key is specific tothe organization in which the key is tested.
There is evidence to suggest that biodata keys
are not transportable.
Biodata keys also appear to be specific tothe criterion measure used to develop the key.
For example, Tucker, Cline, and Schmitt(1967) developed separate keys for supervisor
ratings and for tenure that were both checkedin the same population of pharmaceutical scientists. A cross validation (as described below)
showed that the ratings key predicted ratings(r = .32), and that the tenure key predicted
tenure (r = .50). However, the cross correlations between different criteria were different—they were both negative (r = —.07 forthe ratings key predicting tenure, and r = -.09
for the tenure key predicting ratings). That is,in their study the key predicting one criterion
is actually negatively predictive of the other.
Biodata keys often suffer decay in validity
over time. Schuh (1967) reviewed biodata
studies in which follow-up validity studies had
been done. Within these studies, the mean
tryout validity is .66, but in successive followup validity studies this value drops from .66
to .52 to .36 to .07. Brown (1978) claimed tohave found an organization in which this decay
did not occur. However, Brown evaluated hisdata on the basis of statistical significancerather than on the basis of validity. Becausethedata were collected from a sample ofProblems Associated With Two Widely Used
14,000, even small correlations were highly
Alternative Predictors
significant. Thus, all his correlations registered
Biographical data. For entry-level jobs, significant regardless of how small they were
biodata predictors have been known to have and how much they differed in magnitude.
validity second in rank to that of measures of His data showed the validity of a key developedability. The average validity of biodata mea- for insurance salespersons in 1933 on followsures is .37 in comparison with .53 for ability up studies conducted in 1939 and 1970. If wemeasures (with supervisor ratings as the job study validity rather than significance level,
performance measure). Thus, to substitute the validities were .22 in 1939 and .08 in 1970;

88

JOHN E. HUNTER AND RONDA F. HUNTER

this shows the same decay as found in otherstudies.
There is a severe feasibility problem withbiodata. The empirical process of developing
biodata keys is subject to massive capitalization
on chance. That is, evidence may be built fromwhat is only a chance relation. The typical
key is developed on a sample of persons forwhom both biodata and performance ratings
are known. Each biographical item is separately correlated with the criterion, and thebest items are selected to make up the key.
However, on a small sample this procedure
picks up not only items that genuinely predictthe criterion for all persons, but also items
that by chance work only for the particularsample studied. Thus, the correlation between
the keyed items and the criterion will be muchhigher on the sample used to develop the key
than on the subsequent applicant population.
This statistical error can be eliminated by usinga cross-validation research design, in which
the sample of data is broken into two subsamples. The key is then developed on one sample,
but the correlation that estimates the validity
for the key is computed on the other sample.
Cross-validity coefficients are not subject tocapitalization on chance. Only cross validities
are reported in Table 8.
Although the cross-validation research design eliminates the bias in the estimate of thevalidity that results from capitalization on
chance, it does not eliminate the faulty item
selection that results from capitalization on
chance. Sampling error hurts item selection
in two ways: Good items are missed, and poor
items are included. For example, suppose that
we start with an item pool of 130 items madeup of 30 good items, with a population correlation of. 13, and 100 bad items, with a population correlation of zero. Let the averageinteritem correlation be .20. Then, using asample of 100, we would on the average select
11 of the good items and 5 of the bad items,
resulting in a test with a validity of. 18. Usinga sample of 400, we would select 25 of thegood items and 5 of the bad items, resulting
in a test with a validity of .23. That is, thevalidity of the biodata key increases by 28%
when the sample size is increased from 100
to 400.
Thus, the use of biodata requires a large
sample of workers, say 400 to 1,000, for thetryout study, and more large samples approximately every 3 years to check for decay invalidity. Only very large organizations have
any jobs with that many workers, so that thevalid use of biodata tends to be restricted toorganizations that can form a consortium.
One such consortium deserves special mention, that organized by Richardson, Henry,
and Bellows for the development of the Supervisory Profile Record (SPR). The latest
manual shows that the data base for the SPR
has grown to include input from 39 organizations. The validity of the SPR for predicting
supervisor ratings is .40, with no variation overtime, and little variation across organizations
once sampling error is controlled. These two
phenomena may be related. The cross-organization development of the key may tend toeliminate many of the idiosyncratic and trivial
associations between items and performance
ratings. As a result, the same key applies to
new organizations and also to new supervisors
over time.
Richardson, Henry, and Bellows are also
validating a similar instrument called theManagement Profile Record (MPR). This instrument is used with managers above the levelof first-line supervisor. It has an average validityof .40 across 11 organizations, with a total
sample size of 9,826, for the prediction of job
level attained. The correlation is higher forgroups with more than 5 years of experience.
Two problems might pose legal threats tothe use of biographical items to predict work
behavior: Some items might be challenged forinvasion of privacy, and some might be challenged as indirect indicators of race or sex (see
Hunter & Schmidt, 1976, pp. 1067-1068).
Assessment centers. Assessment centers
have been cited for high validity in the prediction of managerial success (Huck, 1973),
but a detracting element was found by Klimoski and Strickland (1977). They noted thatassessment centers have very high validity forpredicting promotion but only a moderate validity for predicting actual performance.
Cohen et al. (1974) reviewed 21 studies andfound a median correlation of .63 predicting
potential or promotion but only .33 (.43 corrected for attenuation) predicting supervisor
ratings of performance. Klimoski and Strickland, interpreted this to mean that assessment
centers were acting as policy-capturing devices

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

which are sensitive to the personal mannerisms
that top management tends to use in promotion. To the extent that these mannerisms
are unrelated to actual performance, the highcorrelation between assessment centers andpromotion represents a shared error in thestereotype of a good manager.
In a follow-up study, Klimoski and Strickland (1981) gathered data on 140 managers.
Their predictors were preassessment supervisor
ratings of promotion potential, supervisor ratings of performance, and the assessment center
rating. Three years later the criterion data weremanagement grade level attained, supervisor
rating of potential, and supervisor rating of
performance. The assessment center rating
predicted grade level with a correlation of .34
and predicted future potential rating with acorrelation of .37, but predicted future performance with a correlation of —.02. Preassessment supervisor ratings of potential predicted future ratings of potential with a correlation of .51 and grade level with acorrelation of .14, but predicted future performance ratings with a correlation of only
.08. Preassessment performance ratings predicted performance ratings 3 years later witha correlation of .38, but predicted ratings ofpotential and grade level with correlations of.10 and .06. Thus, assessment center ratingsand supervisor ratings of potential predict eachother quite highly, but both are much poorer
predictors of performance. Klimoski andStrickland interpreted these data to mean thatmanagerial performance could be better predicted by alternatives to assessment centers,such as current performance ratings.
Klimoski and Strickland's (1981) correlations of .37 and .02 between assessment center
ratings and ratings of potential and performance are lower, though in the same direction,
than the average correlations found by Cohenet al. (1974), which were .63 and .33, respectively. (Note that the sample size in the Klimoski and Strickland study was only 140.)
One might argue that in the Klimoski andStrickland study there was a big change incontent over the 3-year period, which madefuture performance unpredictable. However,current job performance ratings predict future
job performance ratings in the Klimoski andStrickland study with a correlation of .38. Thatis, current ratings of job performance predict89

future ratings of job performance as well as
the assessment center predicts future potential
ratings.
There appear to be two alternative hypotheses to explain the data on assessment
centers. First, it may be that supervisors actually generate better ratings of performance
when asked to judge potential for promotion
than when asked to judge current performance.
Alternatively, it may be that assessment centers, supervisors, and upper-level managers
share similar stereotypes of the good manager.
To the extent that such stereotypes are valid,
the assessment center will predict later performance, as in the Cohen et al. (1974) study
showing an average correlation of .43 in predicting later supervisor performance ratings.
The higher correlations for ratings of potential
against promotion record would then measure
the extent to which they share erroneous aswell as valid stereotypic attributes.
Comparison of Predictors
If predictors are to be compared, the criterion for job performance must be the samefor all. This necessity inexorably leads to the
choice of supervisor ratings (with correction
for error of measurement) as the criterion because there are prediction studies for supervisor ratings for all predictors. The following
comparisons are all for studies using supervisor
ratings as the criterion. The comparisons are
made separately for two sets of predictors—
those that can be used for entry-level jobs
where training will follow hiring, and thoseused for promotion or certification. Table 9shows the mean validity coefficient across alljobs for 11 predictors suitable for predicting
performance in entry-level jobs. Predictors arearranged in descending order of validity. The
only predictor with a mean validity of essentially zero is age. The validities for experience,
the interview, training and experience ratings,
education, or the Strong Interest Inventory are
greater than zero but much lower than thevalidity for ability. Later analysis shows thateven the smallest drop in validity would mean
substantial losses in productivity if any of thesepredictors were to be substituted for ability.
In fact, the loss is directly proportional to thedifference in validity.
Table 10 presents the average validity acrossall jobs for the predictors that can be used in

90

JOHN E. HUNTER AND RONDA F. HUNTER

Table 9Mean Validities and Standard Deviations of Various Predictors for Entry-Level Jobs
for Which Training Will Occur After Hiring
Predictor

Mean validitySD

No. correlations

No. subjects

Ability composite
Job tryout
Biographical inventory
Reference check
Experience
Interview
Training and experience ratings
Academic achievement
Education
Interest
Age

.53
.44
.37
.26
.18
.14
.13
.11
.10
.10

.15

425
20
12
10
425
10
65
11
425
3
425

32,124-.01

situations where the persons selected will bedoing the same or substantially the same jobs
as they were doing before promotion. Indeed,
five of the six predictors (all but ability) areessentially ratings of one or more aspects orcomponents of current performance. Thus,
these predictors are essentially predicting future performance from present or past performance. By a small margin, ability is on the
average not the best predictor; the work sample
is best. The validity of all these predictors is
relatively close in magnitude. The drop fromwork sample test to assessment center is about
the same as the drop from ability to the second
best predictor for entry-level jobs (job tryout).
Even so, the following economic analysis showsthat the differences are not trivial.
Combinations of Predictors

.10
.09
.05
.00
.11

4,429
5,389
32,124
2,694
1,089
32,124
1,789
32,124ability is at most the square of its validity.
Second, we note that if a second predictor isused with ability, then it will increase validity
only if the second predictor is given the small
weight that it deserves.
Consider first the maximum extent to which
the multiple correlation for ability and an alternate predictor used together can exceed thevalidity of ability used alone (i.e., can exceed
an average of .53). In the few studies using
more than one type of predictor, the correlations between alternative predictors tend to
be positive. More importantly, the beta weights
are all positive. That is, there is no evidenceof suppressor effects in predictor combinations. If we can assume that there are no suppressor effects, then we can derive an extremely
useful inequality relating the multiple correlation R to the validity of the better predictor(which we will arbitrarily label Variable 1),that is, a comparison between R and r\. Assuming R is the multiple correlation, and H
and r2 are the zero-order correlations for theindividual predictors, then if there are no suppressor effects,

For entry-level hiring, it is clear that ability
is the best predictor in all but unusual situations (those in which biodata might predict
as well as ability). However, there is an important second question: Can other predictors
be used in combination with ability to increase
validity? There are two few studies with mulR
tiple predictors to answer this question directlywith meta-analysis. However, there are mathematical formulas that place severe limits on We can then use Taylor's series to obtain an
the extent to which the currently known al- inequality in which the square root is elimiternatives to ability can improve prediction. nated:
This section discusses two such limits.
First, there is a mathematical inequality thatshows that the increase in validity for an alternate predictor used in combination with

91

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

Table 10
Mean Validities and Standard Deviations of Predictors to be Used for Promotion or Certification,
Where Current Performance on the Job is the Basis for Selection
Predictor

Mean validitySD

No. correlations

No. subjects

Work sample test
Ability composite
Peer ratings
Behavioral consistency experienceratings
Job knowledge test
Assessment center

.54
.53
.49

.15
.15

425
31

32,124
8,202

.49
.48
.43

.08
.08

5
10

3,078

Combining these inequalities yields

We denote the validity of the ability composite
as rA and the validity of the alternate predictor
as rx. Then the validity of the composite is at
most

For the average case in which rA = .53, thismeans that the composite validity is at mostThat is, the increase in validity due to thealternate predictor is at most the square of its
validity. For example, the validity of the interview for an average job is .14. If the interview were used in a multiple regression equation with ability, the increase in validity would
be at most from .53 to .55.
Second, if an additional predictor is usedwith ability, it will increase validity only if thesecond predictor is given the small weight it
deserves. Many companies now use several
predictors for selection. However, the predictors are not being combined in accordancewith the actual validities of the predictors but
are weighted equally. Under those conditions
the validity of the total procedure can be lower
than the validity of the best single predictor.
For example, consider an average job in which
the interview is given equal weight with theappropriate ability composite score, instead ofbeing given its beta weight. In that case, although the validity of ability alone would be.53, the validity of the inappropriate combination of ability and interview would be at
most .47.

Economic Benefits of Various
Selection Strategies
Differences in validity among predictors
only become meaningful when they are translated into concrete terms. The practical resultof comparing the validities of predictors is anumber of possible selection strategies. Thenthe benefit to be gained from each strategy
may be calculated. This section presents ananalysis of the dollar value of the work-force
productivity that results from various selection
strategies.
Three strategies are now in use for selection
based on ability: hiring from the top down,
hiring from the top down within racial or ethnic groups using quotas, and hiring at random
after elimination of those with very low ability
scores. There is also the possibility of replacing
ability measures by other predictors. These
strategies can be compared using existing data.
The comparison is presented here using examples in which the federal government is theemployer; but equations are presented for
doing the same utility analysis in any organization.
An analysis of the results of using alternative
predictors in combination cannot be made because current data on correlations between
predictors are too sparse. However, in the caseof the weaker predictors only severely limited
improvement is possible.
Computation of Utility
Any procedure that predicts job performance can be used to select candidates forhiring or promotion. The work force selectedby a valid procedure will be more productive
than a work force selected by a less valid procedure or selected randomly. The extent of

92

JOHN E. HUNTER AND RONDA F. HUNTER

improvement in productivity depends on theextent of accufacy in prediction. If two predictors differ in validity, then the more valid
predictor will generate better selection andhence produce a work force with higher productivity. The purpose of this section is toexpress this difference in productivity inmeaningful quantitative terms.
A number of methods have been devised toquantify the value of valid selection (reviewed
in Hunter, 198la). One method is to express
increased productivity in dollar terms. The
basic equations for doing this were first derived
more than 30 years ago (Brogden, 1949; Cronbach & Gleser, 1965), but the knowledge necessary for general application of these equations was not developed until recently (Hunter,
198 la; Hunter & Schmidt, 1982a, 1982b;
Schmidt, Hunter, McKenzie, & Muldrow,1979). The principal equation compares thedollar production of those hired by a particular
selection strategy with the dollar productionof the same number of workers hired randomly. This number is called the (marginal)
utility of the selection procedure. Alternative
predictors can be compared by computing thedifference in utility of the two predictors. For
example, if selection using ability would mean
an increased production of $ 15.61 billion andselection using biodata would mean an increased production of $10.50 billion, then theopportunity cost of substituting biodata forability in selection would be $5.11 billion.
The utility of a selection strategy depends
on the specific context in which the strategy
is used. First, a prediction of length of service(tenure) must be considered. The usual baseline for calculating utility is the increase inproductivity of people hired in one year. Assume that N persons are to be hired in thatyear. The gains or losses from a selection decision are realized not just in that year, but
over the tenure of each person hired. Moreover,this tenure figure must include not only thetime each person spent in their particular jobs,
but also the time they spent in subsequent jobs
if they were promoted. Persons who can servein higher jobs are more valuable to the organization than persons who spend their entire
tenure in the jobs for which they were initiallyselected. In the utility equation, we denote theaverage tenure for those hired as T years.
Second, the effect of selection depends onthe dollar value of individual differences in

production. This is measured by the standard
deviation of performance in dollar terms, denoted here as ay.
Third, the effect of selection depends on theextent to which the organization can be selective; the greater the number of potential
candidates, the greater the difference between
those selected and those rejected. If everyone
is hired, then prediction of performance is irrelevant. If only one in a hundred of those
who apply is hired, then the organization canhire only those with the highest qualifications.
Selectiveness is usually measured by the selection ratio, the percentage of those considered who are selected. In utility equations, theeffect of the selection ratio is entered indirectly,
as the average predictor score of those selected:
The more extreme the selection ratio, thehigher the predictor scores of those hired, andhence the higher the average predictor score.
In the utility equation, M denotes the average
predictor score of those selected, whereas foralgebraic simplicity the predictor is expressedin standard score form (i.e., scored so that for
applicants the mean predictor score is 0 andthe standard deviation is 1). If the predictor
scores have a normal distribution, then thenumber M can be directly calculated from theselection ratio. Conversion values are presented in Table 11.
This completes the list of administrative
factors that enter the utility formula: N = thenumber of persons hired, T = the average tenure of those hired, ay = the standard deviationof performance in dollar terms, and M = theexpression of the selection ratio as the average
predictor score in standard score form.
One psychological measurement factor enters the utility equation: the validity of thepredictor, rxy, which is the correlation between
the predictor scores and job performance, forthe applicant or candidate population. If thisvalidity is to be estimated by the correlationfrom a criterion-related validation study, thenthe observed correlation must be corrected forerror of measurement in job performance andcorrected also for restriction in range. Validity
estimates from validity generalization studies
incorporate these corrections.
The formula for total utility then computes
the gain in productivity due to selection for
one year of hiring:
U =NTrxyffyx.

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

Table 11
Average Standard Score on the Predictor Variablefor Those Selected at a Given Selection Ratio
Assuming a Normal Distribution on the Predictor
Selection ratio

Average predictor
scored/

100
90
80
70
60
50
40
30
20
10
5

0.00
0.20
0.35
0.50
0.64
0.80
0.97
1.17
1.40
1.76
2.08

93

top 10% of applicants (i.e., from Table 11,
M- 1.76). Thus, the productivity gain for
one year due to hiring on the basis of ability
rather than at random is
U = NTrxy<ryx = (460,000)(6.52)(.55)
X (40% of $13,598)(1.76) = $15.61 billion.

More recent figures from the Office of Personnel Management show higher tenure anda corresponding lower rate of hiring, but generate essentially the same utility estimate.
The utility formula contains the validity ofthe predictor rxy as a multiplicative factor.
Thus, if one predictor is substituted for another, the utility is changed in direct proportion
to the difference in validity. For example, if
biodata measures are to be substituted for
ability measures, then the utility is reduced in
The standard deviation of job performance in the same proportion as the validities, that is,dollars, ay, is a difficult figure to estimate. Most .37/.5S or .67. The productivity increase overaccounting procedures consider only total random selection for biodata would be
production figures, and totals reflect only mean .67(15.61), or $10.50 billion. The loss in subvalues; they give no information on individual stituting biodata for ability measures would
differences in output. Hunter and Schmidt be the difference between $15.61 and $10.50(1982a) solved this problem by their discovery billion—a loss of $5.11 billion per year.
Table 12 presents utility and opportunityof an empirical relation between ay and annual
wage; ay is usually at least 40% of annual wage. cost figures for the federal government for all
This empirical baseline has subsequently been the predictors suitable for entry-level jobs. The
explained (Schmidt & Hunter, 1983) in terms utility estimate for job tryout is quite unrealof the relation of the value of output to pay, istic because (a) tryouts cannot be used withwhich is typically almost 2 to 1 (because of jobs that require extensive training, and (b)overhead). Thus, a standard deviation of per- the cost of the job tryout has not been subformance of 40% of wages derives from a stan- tracted from the utility figure given. Thus, for
dard deviation of 20% of output. This figure most practical purposes, the next best predictor
can be tested against any study in the literature after ability is biodata, with a cost for substithat reports the mean and standard deviation tuting this predictor of $5.11 billion per year.
of performance on a ratio scale. Schmidt and The cost of substituting other predictors is even
Hunter (1983) recently compiled the results higher.
Table 13 presents utility and opportunityof several dozen such studies, and the 20%
cost figures for the federal government for prefigure came out almost on target.
To illustrate, consider Hunter's (1981a) dictors based on current job performance.
analysis of the federal government as an em- Again, because of the higher average comployer. According to somewhat dated figures plexity of jobs in the government, the validityfrom the Bureau of Labor Statistics, as of mid- of ability has been entered in the utility for1980 the average number of people hired in mula as .55 rather than .53. A shift from ability
one year is 460,000, the average tenure is 6,52 to any of the other predictors means a loss inyears, and the average annual wage is $ 13,598. utility. The opportunity cost for work sample
The average validity of ability measures for tests ($280 million per year) is unrealisticallythe government work force is .55 (slightly low because it does pot take into account thehigher than the .53 reported for all jobs, be- high cost of developing each work sample test,cause there are fewer low-complexity jobs in the high cost of redeveloping the test every
government than in the economy as a whole). time the job changes, or the high cost of adThe federal government can usually hire the ministering the work sample test. Because the94

JOHN E. HUNTER AND RONDA F. HUNTER

range of ability is smaller for predictors basedon current performance than for entry-level
predictors, the range of opportunity costs issmaller as well.
Productivity and Racial Balance
The best known predictor for entry-leveljobs is ability. Therefore, any departure fromthe optimal selection strategy of hiring fromthe top down on ability will necessarily leadto loss of productivity. It has been shown thatability tests are fair to minority group membersin that they correctly predict job performance,
indicating that the differences between racial
groups in mean performance on ability tests

Table 12
Utility of Alternative Predictors That can be Used
for Selection in Entry-Level Jobs ComparedWith Utility of Ability as a Predictor
Predictor/strategy

Utility"

Opportunity
costs"

Ability composite
Ability used with quotas
Job tryout
Biographical inventory
Reference check
Experience
Interview
Training and experienceratingAcademic achievement(college)
Education
Interest
Ability with low cutoff
score"

15.61
14.83
12.49b
10.50
7.38
5.11
3.97

-0.78
-3.12
-5.11
-8.23
-10.50-11.64

3.69-11.92

3.12
2.84
2.84

-12.49-12.77-12.772.50-0.28-13.11-15.89Age

Note. Economic benefits or losses in productivity are shownfor 1 year of hiring, with the federal government as theemployer, if all personnel decisions are based on the useof a single predictor. Utility indicates amount saved overrandom selection; opportunity costs indicate amount lost
if a predictor other than ability is used. The use of ability
with quotas and ability as a screen (i.e., the use of ability
measures with very low cutoff scores) are shown as if they
were alternative predictors rather than alternative strategies
for using ability as a predictor.
* In billions of dollars.
b
Does not include the cost of transporting applicant toworkplace, paying for training and tryout period, or social
cost of firing some applicants after they have been on thejob.
c
Cutoff score was set to reject only the bottom 20% of
applicants.

Table 13
Utility of Predictors That can be Used for
Promotion or Certification ComparedWith the Utility of Ability as a Predictor
Predictor/strategy

Utility"

Opportunity
costs"

Ability: hiring top down
Work sample test
Ability: using quotas
Peer ratings
Behavioral consistency
experience ratings
Job knowledge test
Assessmentvcenter
Ability with low cutoff
score"

15.61
15.33
14.83
13.91

-.28
-.78
-1.70

13.91
13.62
12.20

-1.70
-1.99
-3.41

2.50

-13.11

Note. Economic benefits or losses in productivity are shownfor 1 year of hiring or promoting, with the federal government as the employer, if all personnel decisions are
based on the use of a single predictor. Utility indicatesamount saved over random selection; opportunity costs
indicate amount lost if a predictor other than ability isused. The use of ability with quotas and the use of ability
as a screen (i.e., the use of ability measures with very low
cutoff scores) are shown as if they were alternative predictors
rather than alternative strategies for using ability as a predictor.

"b In billions of dollars.
Cutoff score was set to reject only the bottom 20% of
applicants.
are real. Therefore, the use of ability tests forselection will inevitably mean a lower hiring
rate for minority applicants. Many would argue that racial balance in the work force is so
important that the merit principle should beabandoned in favor of affirmative action. We
do not debate this ethical position here. (See
Hunter & Schmidt, 1976, for a review of suchdiscussions.) However, there are utility implications for hiring on a basis other than merit,which is considered here.
There are two strategies that can be usedwhich base hiring on ability but adjust hiring
rates in the direction of affirmative action: using quotas and using ability tests with very
low cutoff scores. The hiring strategy thatmeets affirmative action goals with the least
loss in productivity is hiring on the basis ofability with quotas. For example, if an organization is to hire 10% of all applicants, it willhire from the top down on ability within eachgroup separately until it has hired the top 10%of white applicants, the top 10% of black ap-

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE

95

plicants, the top 10% of Hispanic applicants,
Current interest in predictors other thanand so on.
ability has been spurred primarily by the desire
The other affirmative action strategy is to to find an equally valid predictor with much
use ability to select, but to set a very low cutoff less adverse impact. The alternative predictors
score and hire randomly from above that cutoff for entry-level jobs are all much less valid thanpoint. This strategy has two disadvantages in are ability measures, and their use implies
comparison with quotas: It leads to the selec- considerable economic loss. Also, they aretion of low-ability applicants from all groups, not without adverse impact. For example,
and it does not completely satisfy affirmative Schmidt, Greenthal, Hunter, Berner, and Seaaction goals.
ton (1977) constructed a work sample test to
The utility implications of affirmative-action be used for apprentice machinists in the auto
strategies can be quantified (Hunter, Schmidt, industry. They found racial differences in mean
& Rauschenberger, 1977). Once this had been production time as large as the racial differdone for the low-cutoff strategy, the disas- ences in mean cognitive ability. The sparse
trous implications were immediately apparent literature on predictors other than ability is(Hunter, 1979a, 198la; Mack, Schmidt, & reviewed in Reilly and Chao (1982).
Hunter, undated). These comparisons were ilFuture Research
lustrated in Tables 12 and 13. Table 12 showsthat for entry-level jobs, using ability withWe have shown that, for entry-level jobs,
quotas is the second-best strategy, whereas the predictors other than ability have validity solow-cutoff strategy is the worst predictor except much lower that substitution would mean
for age, which has an average validity of less massive economic loss. It is likely, however,than zero. Table 13 shows that for promotion that some of the alternative predictors measure
or certification the quota method is third and social skills or personality traits that are relthe low-cutoff method is far poorer than any evant to job performance and are not assessed
other.
by cognitive, perceptual, or psychdmotor abilIt is evident that the use of low cutoff scores ity tests. If so, then validity could be increased
negates most of the gain derived from the pre- by using such predictors in conjunction withdictive power of ability measures. This method ability tests. However, the current research baseis also vastly inferior to quotas for purposes is completely inadequate for setting up suchof racial balance. Quotas guarantee that the programs, because alternative predictors havehiring rate will be the same in each group, but been studied in isolation. Only a handful oflow cutoff scores do not equalize hiring rates. of studies have considered more than one preConsider those jobs in which the discrepancy dictor at a time. In particular, the correlationsin ability is greatest, jobs of high complexity between alternative predictors are unknown;
for which cognitive ability is the sole predictor. hence, generalized multiple regression cannot
If the cutoff is set to hire 10% of the majority be done.
In the introduction to this article, we notedof white people, it will select only 1.1% of theblack applicants—a relative minority hiring that adverse impact would be reduced if therate of 11%. However, if the cutoff is set so validity of prediction could be increased. Vathat 80% of applicants pass, the relative mi- lidity cannot be increased by replacing ability
nority hiring rate is still only 52%. The use of tests by any of the now known alternative prea low cutoff score raises the relative hiring rate dictors. However, validity could be increased
from 11% to 52%, but this is far short of the in some jobs by adding the appropriate second
100% that would be produced by the use of predictor and using it with the appropriate
quotas.
weight. If that second predictor has less adverse
Comparison of the quota method with the impact, then the composite selection strategy
low-cutoff method reveals a stark difference. would have more validity and less adverse imIt would cost the government only $780 mil- pact than the use of ability alone. Unfortulion to achieve racial balance using ability with nately, there is no research base for suggesting
quotas. It would cost $13.11 billion (17 times a generally useful strategy of that type.-Instead,as much) to use the low-cutoff method, which the use of a second predictor with ability would
achieves only a poor approximation of racial require a large sample local or consortiumbased multivariate validation study.
balance.

96

JOHN E. HUNTER AND RONDA F. HUNTER

Referencesof the interaction of ability, work values, and the perceived
environment (Research Report No. 22). College Park:
Acuff, R. (1965). A validity study of an aircraft electrician
University of Maryland.
electrical worker job element examination. Pasadena, Haynes, E. (undated). Tryout of job element examination
for shipfitter. Washington, DC: U.S. Civil Service ComCA: U.S. Naval Laboratories in California, Test Demission, Personnel Research and Development Center.
velopment and Personnel Research Section.
Ash, R. A. (1978, August). Self assessments of five types Hedges, L. V. (1983). A random effects model for effectsizes. Psychological Bulletin, 93, 388-395.
of typing ability. Paper presented at the annual meetingof the American Psychological Association, Montreal. Helm, W. E., Gibson, W. A., & Brogden, H. E. (1957).
Asher, J. J., & Sciarrino, J. A. (1974). Realistic work sample
An empirical test of shrinkage problems in personnel
tests: A review. Personnel Psychology, 27, 519-533.
classification research (Personnel Board Technical ReBartlett, C. J., Bobko, P., Mosier, S. B., & Hannan, R.
search Note 84). Arlington, VA: Adjutant General's Of(1978). Testing for fairness with a moderated multiple
fice, Personnel Research Branch, U.S. Army.
regression strategy: An alternative to differential analysis. Huck, J. R. (1973). Assessment centers: A review of the
Personnel Psychology, 31, 233-241.
external and internal validities. Personnel Psychology,
Bass, B. M., & Burger, P. (1979). Assessment of managers:
26, 191-212.
An international comparison. New York: Free Press.
Hunter, J. E. (1979a). An analysis of validity, differential
Boehm, V. R. (1977). Differential prediction: A methvalidity, test fairness, and utility for the Philadelphia
odological artifact? Journal of Applied Psychology, 62,
Police Officers Selection Examination prepared by the
146-154.
Educational Testing Service. (Report to the Philadelphia
Brogden, H. E. (1949). When testing pays off. Personnel
Federal District Court, Alvarez v. City of Philadelphia)
Psychology, 2, 171-183.
Hunter, J. E. (1979b). Cumulating results across studies:
Brown, S. H. (1978). Long-term validity of a personnel
A critique of factor analysis, canonical correlation, MANhistory item scoring procedure. Journal of Applied PsyOVA, and statistical significance testing. Paper presented
chology, 63, 673-676.
at the annual meeting of the American Psychological
Brown, S. H. (1981). Validity generalization in the life
Association, New York City.
insurance industry. Journal of Applied Psychology, 66, Hunter, J. E. (1980a). The dimensionality of the General664-670.
Aptitude Test Battery (GATB) and the dominance ofCallender, J. C., & Osburn, H. G. (1980). Development
general factors over specific factors in the prediction ofand test of a new model for validity generalization.
job performance. Washington, DC: U.S. Employment
Journal of Applied Psychology, 65, 543-558.
Service, U.S. Department of Labor.
Callender, J. C., & Osburn, H. G. (1981). Testing the con- Hunter, J. E. (1980b). Validity generalization and constructsistency of validity with computer-generated sampling
validity. In Construct validity in psychological measurement: Proceedings of a colloquium on theory anddistributions of the multiplicative model variance esapplication in education and measurement (pp. 119timate; Results of petroleum industry validation re130). Princeton, NJ: Educational Testing Service.
search. Journal of Applied Psychology, 66, 274-281.
Cohen, B., Moses, J. L., & Byham, W. C. (1974). The Hunter, J. E. (1980c). Test validation for 12,000 jobs: An
validity of assessment centers: A literature review. Pittsapplication of synthetic validity and validity generalburgh, PA: Development Dimensions Press.
ization to the General Aptitude Test Battery (GATB).
Washington, DC: U.S. Employment Service, U.S. DeCronbach, L. J., & Gleser, G. C. (1965). Psychologicaltests and personnel decisions. Urbana, IL: University ofpartment of Labor.
Hunter, J. E. (1981a). The economic benefits of personnel
Illinois Press.
DeNisi, A. S., & Shaw, J. B, (1977). Investigation of theselection using ability tests: A state-of-the-art review inuses of self-reports of abilities. Journal of Applied Psycluding a detailed analysis of the dollar benefit of U.S.
chology, 62, 641-644.
Employment Service placements and a critique of theDunnette, M. D. (1972). Validity study results for jobs
low-cutoff method of test use. Washington, DC: U.S.
relevant to the petroleum refining industry. Washington,
Employment Service, U.S. Department of Labor.
DC: American Petroleum Institute.
Hunter, J. E. (1981b). Fairness of the General Aptitude
Farley, J. A., & Mayfield, E. C. (1976). Peer nominations
Test Battery (GATB): Ability differences and their impact
without peers? Journal of Applied Psychology, 61, 109on minority hiring rates. Washington, DC: U.S. Employment Service, U.S. Dept. of Labor.
111.
Fine, S. A. (1955). A structure of worker functions. Per- Hunter, J. E. (1981c, April). False premises underlying
sonnel and Guidance Journal, 34, 66-73.
the 1978 Uniform Guidelines on Employee Selection
Ghiselli, E. E. (1966), The validity of occupational aptitude
Procedures: The myth of test invalidity. Paper presented
tests. New York: Wiley.
to the Personnel Testing Council of Metropolitan WashGhiselli, E. E. (1973). The validity of aptitude tests inington, Washington, DC.
personnel selection. Personnel Psychology, 26, 461-477. Hunter, J. E. (1982, May). The validity of content valid
Glass, G. V. (1976). Primary, secondary, and meta-analysis
tests and the basis for ranking. Paper presented at theof research. Educational Researcher, 5, 3-8.
International Personnel Management Association ConGlass, G. V., McGaw, B., & Smith, M. L. (1981). Metaference, Minneapolis, MN.
analysis in social research. Beverly Hills, CA: Sage.
Hunter, J. E., & Schmidt, F. L. (1976). A critical analysis
Gordon, L. V. (1973). Work environment preference schedof the statistical and ethical implications of five defiule manual. New \brk: Psychological Corp.
nitions of test fairness. Psychological Bulletin, 83, 10531071.
Hannan, R. L. (1979). Work performance as a function

ALTERNATIVE PREDICTORS OF JOB PERFORMANCE
Hunter, J. E., & Schmidt, F. L. (1982a)v The economic
benefits of personnel selection using psychological ability
tests. Industrial Relations, 21, 293-308.
Hunter, J. E., & Schmidt, F. L. (1982b). Fitting people tojobs: The impact of personnel selection on national productivity. In M. D. Dunnette & E. A. Fleishman (Eds.),
Human performance and productivity: Human capability
assessment (pp. 233-284). Hillsdale, NJ: Erlbaum.
Hunter, J. E., Schmidt, F. L., & Hunter, R, (1979). Differential validity of employment tests by race: A comprehensive review and analysis. Psychological Bulletin,
86, 721-735.

97

analysis of the prediction of first year grades in law
school. Applied Psychological Measurement, 5, 281-289.
Mack, M. J., Schmidt, F. L., & Hunter, J. E. (undated).
Dollar implications of alternative models of selection:
A case study of park rangers. Unpublished manuscript.
(Available from F. Schmidt, Office of Personnel Management, Washington, DC 20415)
Molyneaux, J. W. (1953). An evaluation of unassembled
examinations. Unpublished master's thesis, George
Washington University, Washington, DC.
Mosel, J. N. (1952). The validity of rational ratings ofexperience and training. Personnel Psychology, I, 1-10.
Hunter, J. E., Schmidt, F. L., & Jackson, G. B. (1982). O'Connor, E. J., Wexley, K. N., & Alexander, R. A. (1975).
Advanced meta-analysis: Quantitative methods for cuSingle-group validity: Fact or fallacy? Journal of Applied
mulating research findings across studies. Beverly Hills,
Psychology, 60, 352-355. ,CA: Sage.
O'Leary, B. S. (1980). College grade point average as anindicator of occupational success: An update (PRR-80Hunter, J. E., Schmidt, F. L., & Pearlman, K. (1982). The
23). Washington, DC: U.S. Office of Personnel Manhistory and accuracy of validity generalization equations:agement, Personnel Research and Development Center.
A response to the Callender and Osburn reply. Journal(NTIS No. PBS 1-121329)
of Applied Psychology, 67, 853-858.
Hunter, J. E., Schmidt, F. L., & Rauschenberger, J. M. Pearlman, K. (1982). The Bayesian approach to validity
generalization: A systematic examination of the ro(1977). Fairness of psychological tests: Implications ofbustness of procedures and conclusions. (Doctoral disfour definitions for selection utility and minority hiring.
sertation, George Washington University, 1982). DisJournal of Applied Psychology, 62, 245-260.
sertation Abstracts International, 42, 4960-B.
Hunter, J. E., Schmidt, F. L., & Rauschenberger, J. M. (inpress). Methodological, statistical, and ethical issues in Pearlman, K., & Schmidt, F. L. (1981, August). Effects ofthe study of bias in psychological tests. In C. R. Reynolds
alternate job grouping methods on selection procedure(Ed.), Perspectives on bias in mental testing. New \brk:validity. In E. L. Levine (Chair), Job analysis/job forPlenum Press.
mulas: Current perspectives on research and application.
Jackson, G. B. (1980). Methods for integrative reviews.
Symposium conducted at the annual meeting of the
Review of Educational Research, 50, 438-460.
American Psychological Association, Los Angeles.
Johnson, J. C., Guffey, W. L., & Perry, R. A. (1980, July). Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980).
When is a T and E rating valid? Paper presented at the
Validity generalization results for tests used to predict
annual meeting of the International Personnel Mantraining success and job proficiency in clerical occuagement Association Assessment Council, Boston, MA.
pations. Journal of Applied Psychology, 65, 373-406.
Kane, J. S., & Lawler, E. E. (1978). Methods of peer as- Personnel Research and Development Center. (1981). Alsessment. Psychological Bulletin, 85, 555-586.
ternative selection procedures. Federal Personnel Manual
Katzell, R. A., & Dyer, F. J. (1977). Differential validity
Bulletin 331-3. Washington, DC: U.S. Office of Personnel
revived. Journal of Applied Psychology, 62, 137-145.
Management.
King, L. M., Hunter, J. E., & Schmidt, F. L. (1980). Halo
Primoff, E. S. (1958). Report on validation of an examiin a multidimensional forced-choice performance evalnationfor electrical repairer, McClellan Field, California.
uation scale. Journal of Applied Psychology, 65, 507Washington, DC: U.S. Civil Service Commission, Per516.
sonnel Research and Development Center.
Klimoski, R, J., & Strickland, W. J. (1977). Assessment
centers: Valid or merely prescient? Personnel Psychology, Raju, N. S., & Burke, M. J. (in press). Two new proceduresfor studying validity generalization. Journal of Applied30, 353-361.
Psychology.
Klimoski, R. J., & Strickland, W. J. (1981). The comparative view of assessment centers. Unpublished Reilly, R. R., & Chao, G. T. (1982). Validity and fairnessof some alternative employee selection procedures. Permanuscript, Department of Psychology, Ohio Statesonnel Psychology, 35, 1-62.
University.
Lent, R. H., Aurbach, H. A., & Levin, L. S. (1971). Re- Ricciuti, H. N. (1955). Ratings of leadership potential at
the United States Naval Academy and subsequent officer
search design and validity assessment. Personnel Psyperformance. Journal of Applied Psychology, 39, 194chology, 24, 247-274.
199.
Levine, E. L., Flory, A. P., & Ash, R. A. (1977). Selfassessment in personnel selection. Journal of Applied Schmidt, F. L., Berner, J G., & Hunter, J. E. (1973).
Racial differences in validity of employment tests: RealPsychology, 62, 428-435,
ity or illusion? Journal of Applied Psychology, 58, 5-9.
Lilienthal, R. A., & Pearlman, K. (1983). The validity ofFederal selection tests for aid technicians in the health, Schmidt, F. L., Caplan, J. R., Bemis, S. E., Decuir, R.,
Dunn, L., & Antone, L. (1979). The behavioral conscience, and engineering fields (OPRD 83-1). Washingsistency method of unassembled examining (TM-79ton, DC: U.S. Office of Personnel Management, Office21). Washington, DC: U.S. Civil Service Commission,of Personnel Research and Development. (NTIS No.
Personnel Research and Development Center. (NTISPB83-202051)
No. PB80-139942)
Linn, R. L., Harnisch, D. L., & Dunbar, S. B. (1981).
Validity generalization and situational specificity: An Schmidt, F. L., Gast-Rosenberg, I., & Hunter, J. E. (1980).

JOHN E. HUNTER AND RONDA F. HUNTER
Validity generalization results for computer program- Schmidt, F. L., Hunter, J. E., Pearlman, K., & Shane,mers. Journal of Applied Psychology, 65, 643-661.
G. S. (1979). Further tests of the Schmidt-Hunter
Schmidt, F. L., Greenthal, A. L., Hunter, J. E., Berner,
Bayesian validity generalization procedure. Personnel
J. G., & Seaton, F. W. (1977). Job sample vs. paperPsychology, 32, 257-281.
and-pencil trades and technical tests: Adverse impact Schuh, A. J. (1967). The predictability of employee tenure:and employee attitudes. Personnel Psychology, 30, 187A review of the literature. Personnel Psychology, 20,
197.
133-152.
Schmidt, F. L., & Hunter, J. E. (1977). Development of Sharf, J. C. (1981, April). Recent developments in the fielda general solution to the problem of validity generalof industrial and personnel psychology. Paper presented
ization. Journal of Applied Psychology, 62, 529-540.
at the conference sponsored by the Personnel Testing
Schmidt, F. L., & Hunter, J. E. (1980). The future ofCouncil of Metropolitan Washington and BNA Systems,
criterion-related validity. Personnel Psychology, 33,
Washington, DC.
41-60.
Thorndike, R. L. (1933). The effect of the interval between
Schmidt, F. L., & Hunter, J. E. (1981). Employment testing:test and retest on the constancy of the IQ. Journal of
Old theories and new research findings. American PsyEducational Psychology, 25, 543-549.
chologist, 36, 1128-1137.
Thorndike, R. L. (1971). Concepts of culture fairness.
Schmidt, F. L., & Hunter, J. E. (1983). Individual differJournal of Educational Measurement, 8, 63-70.
ences in productivity: An empirical test of the estimate Tucker, M. F., Cline, V. B., & Schmitt, J. R. (1967). Prederived from studies of selection procedure utility. Jourdiction of creativity and other performance measures
nal of Applied Psychology, 68, 407-414.
from biographical information among pharmaceutical
Schmidt, F. L., Hunter, J. E., & Caplan, J. R. (1981a).
scientists. Journal of Applied Psychology, 51, 131-138.
Selection procedure validity generalization (transport- U.S. Employment Service. (1970). Manual for the USES
ability) results for three job groups in the petroleum
General Aptitude Test Battery, Section III: Development.
industry. Washington, DC: American Petroleum InstiWashington, DC: U.S. Department of Labor, Manpowertute.
Administration.
Schmidt, F. L., Hunter, J. E., & Caplan, J. R. (1981b). van Rijn, P., & Payne, S. S. (1980). Criterion-related validity
Validity generalization results for jobs in the petroleum
research base for the D. C. firefighter selection test (PRRindustry. Journal of Applied Psychology, 66, 261-273.
80-28). Washington, DC: U.S. Office of Personnel ManSchmidt, F. L., Hunter, J. E., McKenzie, R. C., & Muldrow,agement, Office of Personnel Research and Development.
(NTIS No. PBS 1-122087)
T. (1979). The impact of valid selection procedures onwork force productivity. Journal of Applied Psychology, Vineberg, R., & Joyner, J. N. (1982). Prediction of job
64, 609-626.
performance: Review of military studies. Alexandria,
Schmidt, F. L., Hunter, J. E., & Pearlman, K. (1981). Task
VA: Human Resources Research Organization.
differences and validity of aptitude tests in selection: A
red herring. Journal of Applied Psychology, 66, 166185.
Received January 11, 1984