Industrial and Organizational Psychology, 1 (2008), 333–342.
Copyright ª 2008 Society for Industrial and Organizational Psychology. 1754-9426/08

FOCAL ARTICLE

Stubborn Reliance on Intuition andSubjectivity in Employee Selection
SCOTT HIGHHOUSE
Bowling Green State University

Abstract
The focus of this article is on implicit beliefs that inhibit adoption of selection decision aids (e.g., paper-and-pencil
tests, structured interviews, mechanical combination of predictors). Understanding these beliefs is just as important as understanding organizational constraints to the adoption of selection technologies and may be more usefulfor informing the design of successful interventions. One of these is the implicit belief that it is theoreticallypossible to achieve near-perfect precision in predicting performance on the job. That is, people have an inherent
resistance to analytical approaches to selection because they fail to view selection as probabilistic and subject toerror. Another is the implicit belief that prediction of human behavior is improved through experience. This mythof expertise results in an overreliance on intuition and a reluctance to undermine one’s own credibility by using
a selection decision aid.

Perhaps the greatest technological achievement in industrial and organizational (I–O)
psychology over the past 100 years is thedevelopment of decision aids (e.g., paperand-pencil tests, structured interviews,
mechanical combination of predictors) that
substantially reduce error in the prediction of employee performance (Schmidt &Hunter, 1998). Arguably, the greatest failureof I–O psychology has been the inability toconvince employers to use them. A little over
10 years ago, Terpstra (1996) sampled 201
human resources (HR) executives about theperceived effectiveness of various selection
methods. As the left side of Figure 1 shows,they considered the traditional unstructured
interview more effective than any of thepaper-and-pencil assessment procedures.
Inspection of actual effectiveness of theseprocedures, however, shows that paperand-pencil tests commonly outperform
Correspondence concerning this article should be
addressed to Scott Highhouse. E-mail: shighho@bgsu.edu
Address: Bowling Green State University, BowlingGreen, OH 43403.

unstructured interviews. For example, theright side of Figure 1 shows the results ofa meta-analysis conducted on the actual
effectiveness of these same procedures forpredicting performance in sales (Vinchur
Schippmann, Switzer, & Roth, 1998). Use of
any one of the paper-and-pencil tests alone
outperforms the unstructured interview—aprocedure that is presumed to assess ability,
personality, and aptitude concurrently.
Although one might argue that these data
merely reflect a lack of knowledge about
effective practice, there is considerable evidence that employers simply do not believethat the research is relevant to their own situation (Colbert, Rynes, & Brown, 2005;
Johns, 1993; Muchinsky, 2004; Terpstra &Rozelle, 1997; Whyte & Latham, 1997).
For example, Rynes, Colbert, and Brown(2002) found that HR professionals were
well aware of the limitations of the unstructured interview. Similarly, one of my students conducted a yet-unpublished surveyof HR professionals (n ¼ 206) about theirviews of selection practice. His data indicated that the HR professionals agreed, by

333

334

S. Highhouse

Unstructured
Interview

Unstructured
Interview

Specific
AptitudeTest

Specific
AptitudeTest

Personality
Test

Personality
Test

GMA Test

GMA Test
1

2

3

4

5

Perceived Effectiveness

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Actual Effectiveness (Sales)

Figure 1. Perceived versus actual usefulness of various predictors.
Note. Perceived effectiveness numbers are on a 1–5 scale (1 ¼ not good; 3 ¼ average; 5 ¼
extremely good). Actual effectiveness numbers are correlations corrected for unreliability inthe criterion and range restriction. Because Vinchur, Schippmann, Switzer, and Roth (1998)
did not include interviews, the interview estimate is from Huffcutt and Arthur (1994) level1 interview. GMA ¼ general mental ability; personality ¼ potency; specific aptitude ¼
sales ability.
a factor of more than 3 to 1, that using tests
was an effective way to evaluate a candidate’s suitability and that tests that assess
specific traits are effective for hiring employees. At the same time, however, thesesame professionals agreed, by more than 3
to 1, that you can learn more from an informal discussion with job candidates and thatyou can ‘‘read between the lines’’ to detect
whether someone is suitable to hire. Thisapparent conflict between knowledge andbelief seems loosely analogous to the common practice of preferring brand name cold
remedies to store brand remedies containing
the same ingredients. People know that thestore brands are identical, but they do nottrust them for their own colds.
Some might argue that the tide is turning.
Much has been written on the merits of evidence-based management (Pfeffer & Sutton,
2006; Rousseau, 2006). This approach,much like evidence-based medicine, relieson the best available scientific evidence tomake decisions. At the core of this movement is ‘‘analytics’’ or data-based decision
making (e.g., Ayers, 2007). Discussions of
number crunching in the arena of personnel
selection, however, are almost always limited to anecdotes from professional sports(e.g., Davenport, 2006). Competing withthe analytical point of view are books like

Malcolm Gladwell’s (2005) Blink: The
Power of Thinking Without Thinking andGerd Gigerenzer’s (2007) Gut Feelings:
The Intelligence of the Unconscious, which
extol the virtues of intuitive decision making. Although the assertions of these authors
have little relevance for the prediction ofhuman performance, the popularity of theirwork likely reinforces the common belief
that good hiring is a matter of experienceand intuition.
Implicit Beliefs
My colleagues and I (Lievens, Highhouse, &De Corte, 2005) conducted a policy-capturing
study of the decision processes of retail managers making hypothetical hiring decisions.
We found that the managers placed moreemphasis on competencies assessed by
unstructured interviews than on competencies measured by tests, regardless of what
those competencies were. They placed moreemphasis, for instance, on Extraversion thanon general mental ability when Extraversion
was assessed using an unstructured interview (and general mental ability was assessed using a paper-and-pencil test). The
opposite was found when Extraversion wasassessed using a paper-and-pencil test andgeneral mental ability was assessed usingReliance on intuition and subjectivity

an unstructured interview! Clearly, thesemanagers believed that good old-fashioned‘‘horse sense’’ was needed to accurately size
up applicants (see Phelan & Smith, 1958).
The reluctance of employers to use analytical selection procedures is at least
partially a reflection of broader misconceptions that the general public has about how
to go about assessing and selecting people
for jobs. Consider two high-profile policy
opinions on testing and selection in the
United States.
In 1990, the National Commission on
Testing and Public Policy (1990) issued
eight recommendations for testing inschools and the workplace. Among
those was the statement as follows:‘‘Test scores are imperfect measuresand should not be used alone to makeimportant decisions about individuals’’(National Commission on Testing andPublic Policy, 1990, p. 30). The commission’s chairman, Bernard Gifford ofApple Computer, commented, ‘‘We
just believe that under no circumstances should individuals be denied a job
or college admission exclusively basedon test scores’’ (‘‘Panel Criticizes Standard Testing,’’ 1990).
d In the landmark Supreme Court decision on affirmative action at the University of Michigan, Justice Rehnquist
concluded that consideration of race
as a factor in student admission is
acceptable—but it must be done at
the individual level, with each applicant considered holistically. In concurrence, Justice O’Connor commented,‘‘But the current [student selection] system, as I understand it, is a nonindividualized, mechanical one. As a result, I
join the Court’s opinion . . . .’’ (Gratz v.
Bollinger, 2003, Concurrence 1).
d

Although these positions sound reasonable
on the surface, they represent fundamentally flawed assumptions. No one disputesthat test scores are imperfect measures,
but the testing commission implies thatcombining them with something else will335

correct the imperfections (rather than exacerbate them). The court’s majority opinion
in Gratz suggests that individualized methods of selection are more fair and reliable
than impersonal ‘‘mechanical’’ ones. Bothof these examples illustrate two implicit
beliefs about employee selection: (1) people
believe that it is possible to achieve nearperfect precision in the prediction of
employee success, and (2) people believethat there is such a thing as intuitive expertisein the prediction of human behavior. These
implicit beliefs exert their influence on policy and practice, even though they may not
be immediately accessible (Kahneman,2003). I acknowledge that there are a number of contextual reasons for resistance toselection technologies, including organizational politics, habit, and culture, along withthe existing legal climate (e.g., Johns, 1993;
Muchinsky, 2004). However, whereas contextual issues are often situation specific,
these are universal ‘‘truths’’ about people.
As such, understanding and studying themprovides hope for overcoming user resistance to selection decision aids.
Irreducible Unpredictability
I recently came across an article in a popular
trade magazine for executives, purportedly
summarizing the state of the science on
executive assessment (Sindelar, 2002). I
was struck by a statement made by theauthor: ‘‘For many top-level positions, technical competence accounts for only 20
percent of a successful alignment. Psychological factors account for the rest’’ (pp.
13–14).1 Whether intentional or not, theauthor was clearly implying what is shownon the top of Figure 2—that 80% of the variance in executive success can be explainedby psychological factors (presumably temperament or personality). Reality, however,is much more like the chart on the bottomof Figure 2—showing that most of the variance in executive success is simply not

1. The author identified his affiliation as the ‘‘Institutefor Advanced Business Psychology.’’

336

S. Highhouse
Tech.
Competence20%

Psych. Factors
80%
Tech.
Competence20%

Psych. Factors
10%
Unpredictability
70%

Figure 2. Variance in success accounted forby technical competence and psychologicalfactors.
predictable prior to employment. The business of assessment and selection involves
considerable irreducible unpredictability;
yet, many seem to believe that all failuresin prediction are because of mistakes in theassessment process. Put another way, people
seem to believe that, as long as the applicant
is the right person for the job and the applicant is accurately assessed, success iscertain. The ‘‘validity ceiling’’ has been a continually vexing problem for I–O psychology(see Campbell, 1990; Rundquist, 1969). Enormous resources and effort are focused on thequixotic quest for new and better predictorsthat will explain more and more variance inperformance. This represents a refusal, byknowledgeable people, to recognize thatmany determinants of performance are notknowable at the time of hire. The notion that
it is still possible to achieve large gains in theprediction of employee success reflects a failure to accept that there is no such thing as
perfect prediction in this domain. Campbell
noted that our poor professional self-esteem
is based on an unrealistic notion of what canbe achieved in the prediction of employee

success. Campbell wrote: ‘‘No externalsource imposed this [validity ceiling] standard on the discipline or even argued that
there should be a standard at all’’ (p. 689).
Recall the earlier comment by thenational testing commission, cautioningthat tests are ‘‘imperfect’’ and must be
supplemented with other things. It is remarkably similar to Viteles’ (1925) observationthat ‘‘objective scores of vocational tests
are at best uncertain diagnostic criteria’’(p. 132). This early pioneer of I–O was arguing that standardized methods of assessment
could only fill the proverbial glass halfway.
Intuitive judgment was needed to fill it therest of the way. Viteles wrote: ‘‘It is the opinion of the writer that in the cause of greater
scientific accuracy in vocational selection
in industry the statistical point of view mustbe supplemented by a clinical point of view’’(p. 134). Countering this position was Freyd(1926), who cautioned against allowing intuition to creep into hiring decisions. Freyd,
who represented the analytical viewpointof selection, argued ‘‘allowing selection to
be influenced by personal interpretations
with their unavoidable prejudices instead of
relying upon objective measures gives evenless consideration to the well-being andinterest of the individual worker’’ (p. 354).
History proved Freyd prescient.
Table 1 shows the results of the earliest
study investigating the relative effectivenessof standardized procedures alone versus
supplementing those procedures with intuitive judgment (Sarbin, 1943). As you cansee, academic achievement was better predicted by the standardized scores alone
than by the scores plus clinical judgment.
The notion that analysis outperforms intuition in the prediction of human behavior isamong the most well-established findingsin the behavioral sciences (Grove & Meehl,
1994; Grove, Zald, Lebow, Snitz, & Nelson,
2000).2 Why, therefore, does the intuitive

2. Although few studies in I–O have explicitly madethis comparison, there are a number of examples
where tests alone outpredicted tests 1 intuition(e.g., Borneman, Cooper, Klieger, & Kuncel, 2007;
Huse, 1962; Meyer, 1956).

Reliance on intuition and subjectivity

337

Table 1. Sarbin’s (1943) Investigation of Two Methods for Predicting Success of Universityof Minnesota Undergraduates Admitted in 1939
Predictor composite

Correlation with criterion (r)High school rank 1 college aptitude test
High school rank 1 college aptitude test 1
intuitive judgment of counselors

perspective remain so appealing? Einhorn(1986) observed that a crucial distinction
between the intuitive and the analytical
approaches to human prediction is theworldview of the people making the judgments. According to Einhorn, the intuitive
approach reflects a deterministic worldview, one that rejects the idea that the future
is inherently probabilistic. This is contrasted with the analytical worldview,which accepts uncertainty as inevitable.
Consider the San Diego Chargers professional football team who, despite having
a regular season record of 14-2 in 2006,
fired its head coach following a play-off
loss. The fired coach had a reputation forleading teams to successful regular season
records, only to lose the big games. The
Chargers organization evidently failed toconsider that the contribution of uncertainty to a play-off outcome is much greaterthan to a 16-game season record. Abelson(1985) found that knowledgeable baseballfans overestimated by a factor of 75 the contribution of skill (vs. chance) to the likelihood of a major league baseball player
getting a hit in a given turn at bat.
Intuitive approaches to employee selection make the errors in selection ambiguous.
Analytical approaches make them part ofthe process—hence, visible. Considerable
research suggests that ambiguity about thelikelihood of an outcome (e.g., the operation
has an unknown chance of success) encourages more optimism than a low known probability (e.g., the operation has a 20% chanceof success; see Kuhn, 1997). There is little
room for optimism when a composite of predictors is known to leave 75% of the variance
unexplained. This may explain why selection
procedures that are difficult to evaluate (e.g.,
feelings about ‘‘fit’’) are so attractive. Einhorn

.45
.35

(1986) noted, however, that one must be
willing to accept error to make less error.
Myth of Expertise
I have argued that one of the reasons thatpeople have an inherent resistance to analytical approaches to hiring is that they fail toview selection in probabilistic terms. A
related but different reason for employer reticence to use selection decision aids is thatmost people believe in the myth of selection
expertise. By this I mean the belief that one
can become skilled in making intuitive judgments about a candidate’s likelihood of success. This is reflected in the survey responsesof the HR professionals who believed in‘‘reading between the lines’’ to size up job
candidates. It is also evidenced in the phenomenal growth of the professional recruiteror ‘‘headhunter’’ profession (Finlay & Coverdill, 1999) and the perseverance of theholistic approach to managerial assessment(Highhouse, 2002).
Despite this widespread belief in intuitive
expertise, the data suggest that it is a myth.
For example, the considerable research on
predicting human behavior per se shows thatexperience does not improve predictions
made by clinicians, social workers, paroleboards, judges, auditors, admission committees, marketers, and business planners(Camerer & Johnson, 1991; Dawes, Faust,& Meehl, 1989; Grove et al., 2000; Sherden,
1998). Although it is commonly acceptedthat some (employment) interviewers are
better than others, research on variance ininterviewer validity suggests that differences
are due entirely to sampling error (Pulakos,
Schmitt, Whitney, & Smith, 1996). Existing evidence suggests that the interrater
reliability of the traditional (unstructured)338interview is so low that, even with a perfectly
reliable and valid criterion, interview-based
judgments could never account for morethan 10% of the variance in job performance(Conway, Jako, & Goodman, 1995).3 Thisempirical evidence is troubling for a procedure that is supposed to simultaneously take
into account ability, motivation, and person–organization fit. Keep in mind also that thesefindings are based on interviews that had ratings associated with the interviewers’ judgments. Thus, the unstructured interviews
subjected to meta-analyses are almost certainly unusual and on the high end of rigor.
The data do not paint a sanguine picture of
intuitive judgment in the hiring process.
There are commonly two scholarly rebuttals to the arguments against prediction
expertise. I will consider these in turn. One
response to the limitations of intuitive
approaches to selection is to focus on theability of experts to spot idiosyncrasies ina candidate’s profile (Jeanneret & Silzer,
1998). Meehl (1954) noted that one limitation of analytical formulas was their inability
to incorporate ‘‘broken-leg’’ cues. The termcomes from an anecdotal example in which
one is trying to predict whether or not a person will go to the movie on a particular day.
A mechanical formula might take into
account things like the nature of the movie(e.g., less likely to go to romantic comedy) or
the weather (e.g., more likely to go on a rainyday). The mechanical procedure would nottake into account, however, an event that isextremely rare (e.g., the person has a brokenleg), and thus, the mechanical prediction
will not be as accurate as a prediction basedon a simple intuitive observation. A mechanical approach to selection would not, thelogic goes, consider idiosyncratic characteristics of any particular job candidate—aseasoned expert would.
Another common response to criticismsof intuitive selection is to focus on the

3. Meta-analysis suggests that it accounts for negligible
incremental validity over simple paper-and-pencil
tests of cognitive ability and conscientiousness(Cortina, Goldstein, Payne, Davison, & Gilliland,2000).

S. Highhouse

expert’s ability to interpret configurationsof traits (Prien, Schippmann, & Prien, 2003).
The notion behind this argument is thateach candidate is unique, and one must consider each piece of information about thecandidate in light of all the other pieces of
information. In other words, assessing patterns of traits is more accurate than assessing
traits individually. For example, Prien et al.
noted that executive assessment requires a‘‘dynamic interpretation’’ of applicant data,
one that takes into account interactions
between test scores and other observations(p. 125). This view is reinforced by leadershiptheorists who assert that leader characteristics
exhibit complex configural relations withleadership outcomes (e.g., Zaccaro, 2007).
Even if we do accept that decision makers
incorporate broken-leg cues and configurations of traits, existing evidence suggests thatthese things account for negligible variancein the predicted outcome. For example,
Dawes (1971) modeled admission decisionsof a four-person graduate admissions committee using a bootstrapping procedure. Thisis shown in Figure 3. Dawes found that themodel (i.e., paramorphic representation) of
the admission committee’s judgments outperformed the committee itself. More relevant to this discussion, however, was the fact
that, whereas a linear combination of theexpert cues correlated significantly (r ¼ .25)
with the criterion, the residual—which included configural judgments, broken-leg
cues, and error—was inconsequential (r ¼
.01). Camerer and Johnson (1991) noted
that, despite accounting for a large portionof the error term, broken-leg cues and configural judgments consistently provide little
incremental gain in prediction—even for socalled experts. The problem with broken-leg
cues is that people rely too much on thembecause they present compelling stories.
The tendency to be seduced by detailed
stories causes people to ignore relevant
information and to violate simple rules oflogic (see Highhouse, 1997, 2001). Also, as
one reviewer noted, broken legs are themselves constructs that can and should bemeasured reliably. The problem with trait
configurations, on the other hand, is that

Reliance on intuition and subjectivity

339

Bootstrapped Models of Experts
Predicted
Outcomer = .19

ExpertPredictionsr = .25
r = .01

Model of
Expertlinear combination of cues

configural judgments
Residuals“broken-leg” cues
error

Figure 3. Results from Dawes’ (1971) examination of graduate admissions decisions.
they require feats of information integrationthat contradict current understanding of
human cognitive limitations (Ruscio,
2003). And true real-world examples of predictive interactions between job applicant
characteristics are difficult to find (e.g.,
Sackett, Gruys, & Ellingson, 1998). Hastie
and Dawes (2001) distilled from the vast literature on prediction ‘‘experts’’ the following
stylized facts:
They rely on few pieces of information.
They lack insight into how they arrive at
predictions.
d They exhibit poor interjudge agreement.
d They become more confident in theiraccuracy when irrelevant information
is presented.
dd

The obvious remedy to the limitations ofexpertise is to structure expert intuition andmechanically combine it with other decision
aids, such as paper-and-pencil inventories.
However, there would likely be considerable resistance to structuring or mechanizing the judgment process (e.g., Lievens et al.,2005; van der Zee, Bakker, & Bakker, 2002).
Most people believe that aspects of an applicant’s character are far too complex to beassessed by scores, ratings, and formulas.
An example of the irrationality of this bias
against decision aids is the contempt withwhich most college football fans and com-mentators hold the Bowl ChampionshipSeries, which is a mechanical formula thatincorporates expert ratings (e.g., coachespoll) and computer rankings (e.g., wins andlosses of opponents) into an overall rankingof football teams. The nature of the complaints (‘‘unplug the computers’’) suggeststhat people do not want mechanical formulas making their expert decisions about who
attends bowl games. A University of Oregoncoach infamously declared: ‘‘I liken the BCS
to a bad disease, like cancer’’ (Vondersmith,2001). Another example of this bias against
decision aids is the considerable patientresistance to diagnostic decision aids (Arkes,
Shaffer, & Medow, 2007). Arkes and his colleagues found that physicians who madecomputer-based diagnoses of ankle injuries
were perceived less competent, professional, and thorough than physicians who
made diagnoses without any aids. Indeed,
the idea that (with the appropriate data)a physician might not even need to meet orinteract with a patient to understand his or
her personal health issues would be a hardsell to most people. Physicians, aware of thislay bias against ‘‘cookbook medicine,’’grossly underutilize these valuable technologies in practice (Kaplan, 2001).4 Hastie
and Dawes (2001) noted that relying on
4. This underutilization also results from overconfidence on the part of physicians in their own diagnostic expertise.

340

expertise is more socially acceptable thanrelying on test scores or formulas. Researchon medical decision making supports thiscontention. It is no wonder, therefore, thatHR practitioners would be reluctant toundermine their status by administeringa paper-and-pencil test, structuring an
employment interview, or plugging ratings
into a mechanical formula.
Concluding Remarks
We know quite a bit about applicant reactions to hiring methods (Hausknecht, Day, &Thomas, 2004), but very little attention has
been given to user resistance to selection
decision aids. Campbell (1990) noted: ‘‘We
still do not know much about how to bestcommunicate selection results to people
outside the [I-O] profession’’ (p. 704). Fifteen
years later, Anderson (2005) lamented: ‘‘In
fact, the whole area of practitioner beliefs
about selection methods and processes isa gargantuan one which research has madelittle or no inroads into’’ (p. 19). I haveinferred from the general psychological literature, and the specific selection literature,
two implicit beliefs that likely inhibit thewidespread acceptance of selection technologies. These include the belief that it ispossible to achieve near-perfect precision inpredicting performance on the job and thebelief that intuitive prediction can be
improved by experience. People trust thatthe complex characteristics of applicants
can be best assessed by a sensitive, equallycomplex human being. This does not standup to scientific scrutiny, and I–O psychologists need to begin focusing their efforts onunderstanding how to navigate these waters.
We can begin by drawing from the judgment
and decision making and human factors literatures on how to better communicate
uncertainty and error. We also need to learn
how to better calibrate user expectations.
Consider Muchinsky’s (2004) experience incommunicating a .50 validity coefficient fora mechanical comprehension test:
my pleasure regarding the findings was
highly apparent to the client organi-

S. Highhousezation. It was at this point a senior company official said to me, ‘‘I fail to see thebasis for your enthusiasm.’’ (p. 194)
Research on probability neglect (Sunstein,2002) suggests that people make little distinction between probabilities that they
consider small. In addition, research on
evaluability (Hsee, 1996) has shown thatmost attributes cannot be evaluated without
appropriate context. Perhaps if Muchinsky(2004) had compared his .50 to flippinga coin (.00) or to an unstructured interview(.20), management would have been more
impressed. Perhaps management would
have been more impressed by a commonlanguage effect size indicator or by an
expectancy chart. We simply do not havethe research to guide these communication
decisions.
The traditional unstructured interview has
remained the most popular and widely used
selection procedure for over 100 years(Buckley, Norris, & Wiese, 2000). This is
despite the fact that, during this same period,
there have been significant advancements inthe development of selection decision aids.
Guion (1965) argued that the waste ofhuman resources caused by poor selection
procedures should pain the professional
conscience of I–O psychologists. It is truethat people are not very predictable, but
selection decision aids help.
References
Abelson, R. P. (1985). A variance explanation paradox:
When a little is a lot. Psychological Bulletin, 97,
128–132.
Anderson, N. (2005). Relationship between practiceand research in personnel selection: Does the left
hand know what the right is doing? In A. Evers, N.
Anderson, & O. Voskuijl (Eds.), The Blackwell handbook of personnel selection (pp. 1–24). Malden,
MA: Blackwell.
Arkes, H., Shaffer, V. A., & Medow, M. A. (2007).
Patients derogate physicians who use a computerassisted diagnostic aid. Medical Decision Making,27, 189–202.
Ayers, I. (2007). Super crunchers: Why thinking-bynumbers is the new way to be smart. New York:
Random House.
Borneman, M. J., Cooper, S. R., Klieger, D. M., & Kuncel,
N. R. (2007, April). The efficacy of the admissions
interview: A meta-analysis. In N. R. Kuncel (Chair),Alternative predictors of academic performance:Reliance on intuition and subjectivity
The glass is half empty. Symposium conducted at
the Annual Meeting of the National Council on
Measurement in Education, Chicago, IL.
Buckley, M. R., Norris, A. C., & Wiese, D. S. (2000). A
brief history of the selection interview: May the next
100 years be more fruitful. Journal of Management
History, 6, 113–126.
Camerer, C. F., & Johnson, E. J. (1991). The process-performance paradox in expert judgment: How canexperts know so much and predict so badly? In K.
A. Ericsson & J. Smith (Eds.), Toward a general theoryof expertise: Prospects and limits (pp. 195–217).
Cambridge, UK: Cambridge University Press.
Campbell, J. P. (1990). Modeling the performance
prediction problem in industrial and organizational
psychology. In M. D. Dunnette & L. M. Hough (Eds.),
Handbook of industrial and organizational psychology (Vol. 1, 2nd ed., pp. 687–732). Palo Alto, CA:
Consulting Psychologists Press.
Colbert, A. E., Rynes, S. L., & Brown, K. G. (2005). Who
believes us? Understanding managers’ agreementwith human resource findings. Journal of Applied
Behavioral Science, 41, 304–325.
Conway, J. M., Jako, R. A., & Goodman, D. F. (1995). A
meta-analysis of interrater and internal consistency
reliability of selection interviews. Journal of Applied
Psychology, 80, 565–579.
Cortina, J. M., Goldstein, N. B., Payne, S. C., Davison, H.
K., & Gilliland, S. W. (2000). The incremental validity of interview scores over and above cognitive
ability and conscientiousness scores. Personnel
Psychology, 53, 325–351.
Davenport, T. H. (2006, January). Competing on analytics. Harvard Business Review, 84, 99–107.
Dawes, R. M. (1971). A case study of graduate admissions: Application of three principles of humandecision making. American Psychologist, 26,
180–188.
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinicalversus actuarial judgment. Science, 243, 1668–
1774.
Einhorn, H. J. (1986). Accepting error to make less error.
Journal of Personality Assessment, 50, 387–395.
Finlay, W., & Coverdill, J. E. (1999). The search game:
Organizational conflicts and the use of headhunters.
Sociological Quarterly, 40, 11–30.
Freyd, M. (1926). The statistical viewpoint in vocational
selection. Journal of Applied Psychology, 10, 349–
356.
Gigerzenger, G. (2007). Gut feelings: The intelligenceof the unconscious. London: Penguin Books.
Gladwell, M. (2005). Blink: The power of thinking without thinking. New York: Little, Brown and Company.
Gratz v. Bollinger, 539 U.S. 244 (2003).
Grove, W. M., & Meehl, P. E. (1994). Comparative efficiency of informal (subjective, impressionistic) andformal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. Psychology, Public Policy, and Law, 2, 293–323.
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., &Nelson, C. (2000). Clinical versus mechanical prediction. Psychological Assessment, 12, 19–30.
Guion, R. M. (1965). Industrial psychology as an academic discipline. American Psychologist, 20, 815–
821.
Hastie, R., & Dawes, R. M. (2001). Rational choice in an
uncertain world. Thousand Oaks, CA: Sage.

341
Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004).
Applicant reactions to selection procedures: An
updated model and meta-analysis. Personnel Psychology, 57, 639–683.
Highhouse, S. (1997). Understanding and improving
job-finalist choice: The relevance of behavioral
decision research. Human Resource Management
Review, 7, 449–470.
Highhouse, S. (2001). Judgment and decision making research: Relevance to industrial and organizational psychology. In N. Anderson, D. S. Ones,
H. K. Sinangil, & C. Viswesvaran (Eds.), Handbookof industrial, work and organizational psychology(pp. 314–332). Thousand Oaks, Sage.
Highhouse, S. (2002). Assessing the candidate asa whole: A historical and critical analysis of individual psychological assessment for personnel decision
making. Personnel Psychology, 55, 363–396.
Hsee, C. K. (1996). The evaluability hypothesis: An
explanation for preference reversals between joint
and separate evaluations of alternatives. Organizational Behavior and Human Decision Processes, 67,
242–257.
Huffcutt, A. L., & Arthur, W (1994). Hunter and Hunter(1984) revisited: Interview validity for entry-leveljobs.
Huse, E. F. (1962). Assessments of higher level personnel
IV: The validity of assessment techniques based on
systematically varied information. Personnel Psychology, 15, 195–205.
Jeanneret, R., & Silzer, R. (1998). An overview of psychological assessment. In R. Jeanneret & R. Silzer(Eds.), Individual psychological assessment: Predicting behavior in organizational settings (pp. 3–26).
San Francisco: Jossey-Bass.
Johns, G. (1993). Constraints on the adoption of psychology-based personnel practices: Lessons fromorganizational innovation. Personnel Psychology,
46, 569–592.
Kahneman, D. (2003). A perspective on judgment andchoice: Mapping bounded rationality. American
Psychologist, 58, 697–720.
Kaplan, B. (2001). Evaluating informatics applications:
Clinical decision support systems literature review.
International Journal of Medical Informatics, 64, 15–37.
Kuhn, K. (1997). Communicating uncertainty: Framing
effects on responses to vague probabilities. Organizational Behavior and Human Decision Processes,
71, 55–83.
Lievens, F., Highhouse, S., & De Corte, W. (2005). The
importance of traits and abilities in supervisors’ hirability decisions as a function of method of assessment. Journal of Occupational and Organizational
Psychology,78, 453–470.
Meehl, P. E. (1954). Clinical versus statistical prediction:
A theoretical analysis and a review of the evidence.
Minneapolis, MN: University of Minnesota.
Meyer, H. H. (1956). An evaluation of a supervisory
selection program. Personnel Psychology, 9, 499–
513.
Muchinsky, P. M. (2004). When the psychometrics oftest development meets organizational realities: A
conceptual framework for organizational change,
examples, and recommendations. Personnel Psychology, 57, 175–209.
National Commission on Testing and Public Policy.
(1990). From gatekeeper to gateway: Transforming

342
testing in America. Chestnut Hill, MA: National Commission on Testing and Public Policy, Boston College.
Panel criticizes standard testing. (1990, May 24). The
New York Times. Retrieved December 10, 2007,
from query.nytimes.com
Pfeffer, J., & Sutton, R. I. (2006). Hard facts, dangerous half-truths and total nonsense: Profiting fromevidence-based management. Boston: Harvard
Business School Press.
Phelan, J., & Smith, G. W. (1958). To select executives—combine interviews, tests, horse sense. Personnel Journal, 36, 417–421.
Prien, E. P., Schippmann, J. S., & Prien, K. O. (2003).
Individual assessment: As practiced in industry andconsulting. Mahwah, NJ: Lawrence Erlbaum.
Pulakos, E. D., Schmitt, N., Whitney, D., & Smith, M.
(1996). Individual differences in interviewer ratings:
The impact of standardization, consensus discussion, and sampling error on the validity of a structured interview. Personnel Psychology, 49, 85–102.
Rousseau, D. M. (2006). Is there such a thing as evidence-based management? Academy of Management Review, 31, 256–269.
Rundquist, E. A. (1969). The prediction ceiling. Personnel Psychology, 22, 109–116.
Ruscio, J. (2003). Holistic judgment in clinical practice:
Utility or futility? Scientific Review of Mental Health
Practice, 2, 38–48.
Rynes, S. L., Colbert, A. E., & Brown, K. G. (2002). HR
professionals’ beliefs about effective humanresource practices: Correspondence between
research and practice. Human Resources Management, 41, 149–174.
Sackett, P. R., Gruys, M. L., & Ellingson, J. E. (1998).
Ability-personality interactions when predicting
job performance. Journal of Applied Psychology,
83, 545–556.

S. Highhouse
Sarbin, T. L. (1943). A contribution to the study ofactuarial and individual methods of prediction.
American Journal of Sociology, 48, 598–602.
Schmidt, F. L., & Hunter, J. E. (1998). The validity andutility of selection methods in personnel psychology: Practical and theoretical implications of 85
years of research findings. Psychological Bulletin,
124, 262–274.
Sherden, W. (1998). The fortune sellers: The big business of buying and selling predictions. New York:John Wiley.
Sindelar, S. (2002, October). Executive assessment.
Executive Excellence, 19, 13–14.
Sunstein, C. R. (2002). Probability neglect: Emotions,worst cases, and law. Yale Law Journal, 112, 61–107.
Terpstra, D. E. (1996, May). The search for effective
methods. HR Focus, 73, 16–17.
Terpstra, D. E., & Rozelle, J. (1997). Why some potentially effective staffing practices are seldom used.
Public Personnel Management, 26, 483–495.
van der Zee, K. I., Bakker, A. B., & Bakker, P. (2002). Why
are structured interviews so rarely used in personnel
selection? Journal of Applied Psychology, 87, 176–184.
Vinchur, A. J., Schippmann, J. S., Switzer, F. S., & Roth, P.
L. (1998). A meta-analytic review of predictors of job
performance for salespeople. Journal of Applied
Psychology, 83, 586–597.
Viteles, M. S. (1925). The clinical viewpoint in vocational
selection. Journal of Applied Psychology, 9, 131–138.
Vondersmith, J. (2001, December 11). Bellotti sounds
off, favors playoffs. The Portland Tribune. Retrieved
January 10, 2008, from www.portlandtribune.com
Whyte, G., & Latham, G. (1997). The futility of utility
analysis revisited: When even an expert fails. Personnel Psychology, 50, 601–610.
Zaccaro, S. J. (2007). Trait-based perspectives of leadership. American Psychologist, 62, 6–16.