Number range CHAPTER **[WHAT WAS HE THINKING?]{.underline}**

**[DECISION MAKING AND JUDGING]{.underline}**

**This paper is intended to be a brief summary of concerns about and
solutions to inherent problems associated with decision making and
judgment. This paper was prepared for the Serious Accident Investigation
course.**

Mike Johns, Assistant U.S. Attorney and Senior Litigation Counsel[^1]

September 14, 2006

**I. Introduction to Cognitive Processes and Biases.**

Psychological research demonstrates that a wide variety of cognitive
biases adversely affect our decision making and our ability to judge
decisions made by others.

Psychological research which measures the process of making a decision
in complex circumstances, shows that the mind can make "coherence
shifts" leading to potentially wrong and overconfident decisions.

Attribution errors and dispositionist thinking subconsciously mislead
us, ignoring the effects of the situation both in our mind and in our
environment, on decision making and behavior.

The research also provides some opportunities and strategies to mitigate
many of these adverse effects on our thinking and judging.

The new Foundational Doctrine continues to rely on quality, professional
decision making on the fire ground, so understanding and training in
decision processes seems particularly appropriate.

As an aside, the research also helps explain such things as why it is so
hard to change someone's mind, why history keeps repeating itself,
terrorism and many other social phenomenon.

**II. The Problems and Solutions.**

**[Hindsight Bias]{.underline}**

Research on the human mind has proven over and over again that hindsight
bias is robust and virtually impossible to eliminate. It wreaks havoc on
people caught second-guessing their own actions, as well as others who
try to judge those actions, with outcome knowledge:

> "Consider a decision maker who has been caught unprepared by some turn
> of events and who tries to see where he went wrong by recreating his
> preoutcome knowledge state of mind. If, in retrospect, the event
> appears to have seemed relatively likely, he can do little more than
> berate himself for not taking the action which his knowledge seems to
> have dictated. He might be said to add the insult of regret to the
> injury inflicted by the event itself. When second guessed by a
> hindsightful observer, his misfortune appears to have been
> incompetence, folly, or worse." **"Hindsight ≠Foresight: The effect of
> outcome knowledge on judgment under uncertainty", B. Fischoff 1975.**

\- Hindsight bias naturally results from knowing the outcome of a
situation. The mind uses outcome knowledge to judge or learn from the
past - to make sense of it. People are generally unaware of the effect
outcome knowledge has on their conclusions about predictability. Even if
they are made aware of hindsight bias and attempt to reduce it, it
cannot be eliminated, because the mind cannot ignore the truth of a
known outcome when trying to judge an act, omission or decision in real
time. Hindsight bias creates the illusion that the outcome was
predictable. Worse, it creates in the post-event judge the illusion that
he surely would have predicted it. See e.g., "Hindsight ≠Foresight: The
effect of outcome knowledge on judgment under uncertainty", B. Fischoff
1975 (Links to Abstract and full article):

[[http://qhc.bmjjournals.com/cgi/content/full/12/4/304]{.underline}](http://qhc.bmjjournals.com/cgi/content/full/12/4/304)

[[http://qhc.bmjjournals.com/cgi/reprint/12/4/304]{.underline}](http://qhc.bmjjournals.com/cgi/reprint/12/4/304)

-Hindsight bias has significant implications in determining liability,
fault, or blame, bordering on creation of strict liability for an actor
implicated in a bad outcome, judged by a person with knowledge of that
outcome. Knowledge of subsequent remedial measures can also increase
hindsight bias. Hindsight bias has been found to increase in some group
settings. Most strategies to de-bias people, including judges and juries
are ineffective, but suggestions are made in this article concerning
litigation techniques. Fully informing the decision makers of the cause
and effects of hindsight bias may help reduce its effects, but tests
continue to demonstrate that hindsight bias cannot be eliminated even by
those who understand it. See e.g., "Hindsight Bias and the Subsequent
Remedial Measures Rule: Fixing the Feasibility Exception", K. Eberwine
2005:

http://law.case.edu/student_life/journals/law_review/55-3/eberwine.pdf

-While Eberwine's article suggests that the use of "counterfactuals" (if
only X, there would have been a different outcome) may help reduce
hindsight bias, studies indicate that counterfactuals actually increase
hindsight bias. Counterfactuals ask the mind to disregard knowledge of
what actually happened, the truth, and to put in place a falsehood,
which simply cannot be done for the same reason that hindsight bias
exists in the first place. However, the use of "semifactuals, (even if
X, the outcome would have been the same) does not increase hindsight
bias, because it does not ask the mind to replace the true outcome with
a false outcome - the known outcome stays in place. The issue is
particularly important in efforts to find the cause(s) of a bad outcome.
Use of semifactuals does not eliminate hindsight bias, it just does not
make it worse. See e.g., "Counterfactuals, Causal Attributions, and the
Hindsight Bias: A Conceptual Integration", N. Roese and J. Olson, 1996:

[[http://www.psych.uiuc.edu/\~roese/Roese%20&%20Olson%20(1996).pdf]{.underline}](http://www.psych.uiuc.edu/~roese/Roese & Olson (1996).pdf)

-Attempts to debias hindsight with counterfactuals - forcing people to
think about alternative outcomes, can backfire. One study found that
people can be asked to consider one or two simple alternative outcomes
(counterfactuals) without increasing hindsight bias, but when people
were asked to consider ten alternatives which could have changed the
outcome, hindsight bias was increased. In other words, the harder you
try to debias hindsight the worse you can make it. "When Debiasing
backfires: Accessible Content and Accessibility Experiences in Debiasing
Hindsight", L. Sanna, et al. 2002:

http://www2.gsb.columbia.edu/divisions/management/seminars/papers/schwarz.pdf

-One strategy when asking people to judge past events, is to use the
semifactual "even if" rather than the counterfactual "if only", since
the semifactual method allows the "judge" to consider the antecedent
act, omission or decision without trying to force the mind to accept an
outcome other than the known true outcome. These concepts have the same
impact on self-blame as they do on determinations of fault or liability.
See e.g., "Semifactual "even if" Thinking", R. McCloy and R. Byrne,
2002:

http://www.tcd.ie/Psychology/Ruth_Byrne/pdf/mccloyandbyrne2002.pdf

-Similarly, research found that when people are shown computer animated
re-enactments, hindsight bias is increased rather than reduced.
"Computer animations used in court colored by bias, researchers say", J.
Barlow, 2006:

[[http://www.news.uiuc.edu/NEWS/06/0410animation.html]{.underline}](http://www.news.uiuc.edu/NEWS/06/0410animation.html)

-Testing on 167 U.S. Magistrate Judges confirmed the profound impact
that hindsight bias (and several other biases) has on the quality of
decision making. "Inside the Judicial Mind", C. Guthrie, *et al*., 86
Cornell L.Rev. 777 (2001).

-Another law review article discusses the unavoidable inaccuracies of
decision making, fairness issues, and other aspects of hindsight bias.
"A Positive Psychological Theory of Judging in Hindsight", J.
Rachlinski, 65 U.Chi.L.Rev. 571(1998).

**[Other Cognitive Biases]{.underline}**

There are many other cognitive biases which affect our judgment as well,
which are necessary to understand in order to improve the quality of our
decisions and choices we make based on those decisions. "Inside the
Judicial Mind" *supra*. addresses Anchoring, Framing, Hindsight Bias,
Representativeness Heuristic and Egocentric Biases:

The **[Representativeness Heuristic]{.underline}** (a heuristic is a
rule of thumb or cognitive shortcut). From "Inside the Judicial Mind":

"When people make categorical judgments (e.g., assessing the likelihood
that a criminal defendant is guilty), they tend to base their judgments
on the extent to which the evidence being analyzed (e.g., the
defendant\'s demeanor) is representative of the category. \[FN130\] When
the evidence appears representative of, or similar to, the category
(e.g., defendant is nervous and shifty), people judge the likelihood
that the evidence is a product of that category as high (i.e., evidence
of guilt). When the evidence being analyzed does not resemble the
category (e.g., defendant appears at ease), people judge the likelihood
that the evidence is a product of that category as low (i.e., evidence
of innocence). Psychologists refer to this phenomenon as the
\"representativeness heuristic.\"

"Although the representativeness heuristic is useful, it can lead people
to discount relevant statistical information. In particular, people
undervalue the importance of the frequency with which the underlying
category occurs\--this is known as the \"base-rate\" statistic."

This heuristic impacts decision making by over-reliance on
representativeness evidence and under-reliance on hard evidence such as
statistical evidence. This can lead to the **"[inverse
fallacy]{.underline}"**, which ". . . refers to the [tendency to treat
the probability of a hypothesis given the evidence]{.underline} (for
example, the probability that a defendant was negligent given that the
plaintiff was injured) [as the same as, or close to]{.underline}, [the
probability of the evidence given the hypothesis]{.underline} (for
example, the probability that the plaintiff would be injured if the
defendant were negligent)." (Quoted from "Inside the Judicial Mind").

Test Results from "Inside the Judicial Mind":

"To test whether judges would commit the inverse fallacy, we gave the
judges in our study a res ipsa loquitur problem. In an item labeled
\"Evaluation of Probative Value of Evidence in a Tort Case,\" we
presented all of the judges with a paragraph-long description of a case
based loosely on the classic English case, Byrne v. Boadle: \[FN145\]"

"The plaintiff was passing by a warehouse owned by the defendant when he
was struck by a barrel, resulting in severe injuries. At the time, the
barrel was in the final stages of being hoisted from the ground and
loaded into the warehouse. The defendant\'s employees are not sure how
the barrel broke loose and fell, but they agree that either the barrel
was negligently secured or the rope was faulty. Government safety
inspectors conducted an investigation of the warehouse and determined
that in this warehouse: (1) when barrels are negligently secured, there
is a 90% chance that they will break loose; (2) when barrels are safely
secured, they break loose only 1% of the time; (3) workers negligently
secure barrels only 1 in 1,000 times."

"The materials then asked: \"Given these facts, how likely is it that
the barrel that hit the plaintiff fell due to the negligence of one of
the workers\" ? The materials provided the judges with one of four
probability ranges to select: 0-25%, 26-50%, 51-75%, or 76-100%."

The calculated base-rate probability that the injury resulted from
negligence is actually only 8.3%. The inverse fallacy, according to this
article, causes most people to conclude that the probability that
negligence was the cause is 90 % or at least very high. According to the
article, the inverse fallacy appears to have led 60 % or more of the
judges to the wrong conclusion about the probability that the defendant
was negligent.

The **Representativeness Heuristic - Inverse Fallacy** also creates
erroneous legal doctrine. The Res Ipsa Loquiter doctrine provides that a
jury can infer negligence from an adverse event that is of the kind
which ordinarily does not result in injury in the absence of negligence.
The doctrine is erroneous and has been rejected in recent Restatements
of the law. The fact is, just because no one is usually injured during
an activity, does not mean that negligence caused the injury, there are
often many other causes which explain the fact of injury.

**[Egocentric Biases]{.underline}**

The following article is particularly useful reading for prosecutors and
investigators because it explains how the mind takes an initial
impression and skews the subsequent processing of information during the
investigation and development of the case.

"IMPROVING PROSECUTORIAL DECISION MAKING: SOME LESSONS OF COGNITIVE
SCIENCE", 47 Wm. & Mary L. Rev. 1587, A. Burke, Associate Professor,
Hofstra University School of Law, (2006). Excerpt:

"This Article explores four related but separate aspects of cognitive
bias that can contribute to imperfect theory formation and maintenance:
**[confirmation bias]{.underline}**, **[selective information
processing]{.underline}, [belief perseverance]{.underline}**, and **[the
avoidance of cognitive dissonance]{.underline}**. Confirmation bias is
the tendency to seek to confirm, rather than disconfirm, any hypothesis
under study. \[FN34\] Selective information processing causes people to
overvalue information that is consistent with their preexisting theories
and to undervalue information that challenges those theories. \[FN35\]
Belief perseverance refers to the human tendency to continue to adhere
to a theory, even after the evidence underlying the theory is disproved.
\[FN36\] Finally, the desire to avoid cognitive dissonance can cause
people to adjust their beliefs to maintain existing self-perceptions.
\[FN37\] This Part summarizes the empirical literature regarding each of
these cognitive phenomena."

**[List of Cognitive Biases]{.underline}**

Here is a link to a long list of cognitive biases which is interesting
reading:

[[http://en.wikipedia.org/wiki/List_of_cognitive_biases]{.underline}](http://en.wikipedia.org/wiki/List_of_cognitive_biases)

The list includes the **Planning Fallacy** in which we underestimate
task-completion times, **Neglect of Probability** in which we disregard
probability when making a decision under uncertainty and **Zero-Risk
Bias** -- the preference for reducing small risk to zero over a greater
reduction in a larger risk.

**[Decision Process and Coherence Shifts]{.underline}**

Unfortunately, the cognitive process can lead to "coherence shifts"
which can result in decisions which are skewed to seem "coherent" and
made with overconfidence that the decision is right. This has
significant implications for making firefighting decisions and for
judging those decisions later. Accepting an assignment which appears in
hindsight to have been too dangerous, might be an example of a "can do"
coherence shift. The article includes actions to mitigate against
coherence shifts including "consider the opposite".

"A THIRD VIEW OF THE BLACK BOX: COGNITIVE COHERENCE IN LEGAL DECISION
MAKING", 71 U.Chi.L.Rev. 511, Dan Simon (2004). Excerpts:

Page 511:

Based on a connectionist cognitive architecture, coherence-based
reasoning shows that the decision-making process progresses
bidirectionally: premises and facts both determine conclusions and are
affected by them in return. A natural result of this cognitive process
is a skewing of the premises and facts toward inflated support for the
chosen decision. The Article applies this research to four important
aspects of the trial. It argues that the current doctrine in these areas
is based on misconceptions about human cognition, which lead to
systematic legal errors. By identifying the cognitive phenomena that lie
at the root of these failings, the research makes it possible to devise
interventions and introduce procedures that reduce the risk of trial
error.

Page 513

Coherence-based reasoning posits that the mind shuns cognitively complex
and difficult decision tasks by reconstructing them into easy ones,
yielding strong, confident conclusions. The research reveals an
unconscious transformation of the way decisions are mentally
represented, ultimately leading to a seemingly straightforward choice
between a compelling alternative and a weak one. To date,
experimentation has revealed eight cognitive features of coherence-based
reasoning, which harbor interesting\--and potentially
troubling\--implications for decision making in the legal domain and
beyond.

Page 516:

Coherence-based reasoning applies to mental tasks in which the person
must make a discrete decision or judgment in the face of complexity.
Tasks are said to be complex when their constitutive considerations are
numerous, contradictory, ambiguous, and incommensurate.

\* \* \*

In a nutshell, coherence-based reasoning suggests that decisions are
made effectively and comfortably when based on coherent mental models.
Loosely defined, mental models capture the decision-maker\'s perception
of the task at hand\--that is, the way the considerations of the
decision are represented in her mind. \[FN18\] A mental model of a
decision task is deemed \"coherent\" when the decision-maker perceives
the chosen alternative to be supported by strong considerations while
the considerations that support the rejected alternative are weak.

\*517 The central finding of coherence-based reasoning research is that
the cognitive system imposes coherence on complex decision tasks.
Throughout the decision-making process, the mental representation of the
considerations undergoes gradual change and ultimately shifts toward a
state of coherence with either one of the decision alternatives. Due to
these coherence shifts, at the culmination of the process, the
decision-maker\'s mental model is skewed toward conformity with the
emerging decision. As the hard case morphs into an easy one, the
decision follows easily and confidently. The fact that decisions are
ultimately based on skewed mental models and backed by high levels of
confidence facilitates the making of the decision, but at the same time
it can also harbor problematic implications.

Page 520:

Connectionist Representations. All mental processing starts with a
mental representation of the task. \[FN27\] Decisions are made within
certain problem spaces that contain and delimit the representation of
the task variables, that is, all the factors that are to be included in
the decision at hand. The representations of the variables are called
mental models of the decision task.

Coherence-based reasoning presumes a connectionist architecture of
mental representations, which can be likened to an intricate electrical
network. \[FN28\] Each variable is represented in the mind as a unit,
\*521 which is then connected through the network to all the other
units. \[FN29\] The initial representation of a mental task is
determined foremost by the rich and detailed store of background
knowledge about one\'s physical, social, and conceptual worlds. \[FN30\]
The mental model of a complex decision task contains a myriad of
variables that point in more than one direction and thus do not all fit
into a coherent mental model. One subset of variables (a1, a» »,.» .
.an) supports conclusion A, and the other subset (b1, b2. . .b%l» ,. .
.b»ts the opposite conclusion B.

\* \* \*

In sum, the ultimate state of coherence is essentially a byproduct of
the cognitive system\'s drifting toward either one of two skewed mental
models. Within each of these models, the initially complex and
incoherent mental model has been spread into two subsets, one of which
dominates the other, thereby enabling a relatively easy and confident
\*523 choice. This skewed representation reflects an artificial
polarization between the inflated representation of the variables that
support the chosen conclusion and the deflated ones that support the
rejected conclusion; it differs considerably from the way the task
variables were perceived before the decision-making process got
underway, and it differs also from the way they will be perceived some
time after the completion of the task.

Page 524:

The first finding concerns the fundamental property of coherence-based
reasoning: that in the course of making a complex decision, the mental
model of the task shifts toward coherence with the emerging decision.

Page 533:

The second feature of coherence-based reasoning concerns its
metacognitive dimension\--the degree to which people are aware of the
process, particularly of their shifting perceptions of the task
variables. We hypothesized that coherence shifts transpire without
awareness\--that they are mostly an automatic, rather than a controlled
and conscious, form of cognitive processing. \[FN63\]

\* \* \*

People tend not to appreciate the incompatibility between their initial
and eventual mental models of the task. This finding supports the view
that coherence shifts are governed by automatic cognitive mechanisms
that operate under the level of conscious awareness. Coherence shifts
are part of the many important cognitive processes that take place in
the background of our conscious arena, without which making sense of the
world would be difficult, if at all possible. \[FN106\] This finding is
consistent with a robust body of literature demonstrating that people
have a general tendency to perceive objectivity in their reasoning
processes, \[FN107\] and to view the world through a perspective of
\"naive realism.\" \[FN108\] The lack of awareness gives the
decision-maker a false sense of constancy in his own perception of the
case. The ensuing decision is thus experienced as rationally warranted
by the inherent values of the variables, rather than by an inflated
perception imposed by the cognitive system. The lack of awareness then
helps the decision-maker maintain the phenomenological experience of
rationality and objectivity.

Coherence shifts, consequently, do not represent conscious, strategic,
or deceitful conduct on the part of a decision-maker; rather, they \*546
are the natural consequence of the normal mechanisms of cognitive
processing.

\* \* \*

Coherence effects are not isolated from noncognitive characteristics of
the task. Recently obtained findings strongly support the view that
coherence-based reasoning interacts with motivations. Not only do
people\'s choices comport with their desired goals, but the mental
models of the entire task tend strongly to cohere with the respective
outcome. In a separate set of studies we found that coherence effects
interact with preexisting attitudes to generate mental models that
cohere with those attitudes.

\* \* \*

Coherence shifts occur, for the most part, before the point of the
decision. This finding refutes the view, espoused by cognitive
dissonance theory, that the cognitive changes serve only to justify
decisions by reducing post-decisional regret. It suggests, rather, that
coherence shifts play an operative role in the decision-making
process\--the spreading apart of the considerations into one strong and
one weak set facilitates confident choice. Any observed post-decisional
shifts tended to be much weaker than the predecisional ones, suggesting
that the regret-induced post-decisional distortions are secondary.
\[FN109\]

\* \* \*

In a number of experiments, we found that changing one aspect of the
case triggered changes throughout the mental models: not only did it
alter the verdict distribution, but it also influenced the ratings of
variables that were unrelated to the manipulated variable. Such
circuitous and indirect influences inhere in the connectionist nature of
cognition. This finding becomes particularly acute in combination with
the finding that people have low awareness of the shifts.
Decision-makers are likely to perceive sufficient reason to base a
decision on a particular set of variables, unaware that their perception
of those variables is influenced by an extraneous\--possibly
illegitimate\--variable.

\* \* \*

The experimental evidence illustrates that coherence can be transitory.
Coherence shifts were found to decay one week after the task was
completed. At least in some conditions, then, coherence can be
understood as an ad hoc state that is constructed only to solve the task
at hand. Dissipation of coherence enables decision-makers to approach
new tasks unencumbered by previous coherence shifts. In subsequent
tasks, different, perhaps opposite, pressures may activate the same
variables, causing them to take on different values.

\* \* \*

From a prescriptive point of view, an important experimental finding is
that coherence shifts can be reduced by means of a straightforward
moderating technique. \[FN111\] A moderating instruction to \"consider
the opposite\" reduced coherence shifts by about one-half, so that
participants\' reasons for deciding as they did were closer to their
initial, pre-coherence perception of the case. The fact that the
technique is simple and can be self-administered makes it a potentially
powerful and expedient means of tempering the effects of coherence-based
reasoning.

Undoubtedly, coherence effects have their limits. While the observed
coherence shifts are substantial and highly significant from a
statistical standpoint, they do not reach extreme values. \[FN112\]
Coherence shifts are mediated by task-specific factors, most notably, by
the degree of ambiguity inherent in the task. Ambiguous variables are
more amenable to change; \[FN113\] unambiguous variables, such as
indisputable facts and paramount principles are less likely to shift.
\[FN114\] It is also likely that coherence shifts are mediated by the
idiosyncratic characteristics of the decision-maker. \[FN115\]

\* \* \*

Coherence shifts skew the mental models and generate \*549 an inflated
sense of confidence even in close decisions, but it cannot be said that
the shifts necessarily result in objectively wrong decisions, nor that
they cause cardinal changes in the decision-maker\'s own perceptions of
the case. \[FN116\] They do, however, cause a substantial increase in
the risk of error in certain circumstances.

\* \* \*

According to the story model, people make sense of complicated bodies of
evidence by constructing narratives, rather than by aggregating measures
of the strength of the raw evidence. Stories are structured around
episodes held together by certain types of causal and physical
relationships that explain the actors\' conduct and mental states. A
pervasive type of narrative structure is based on schemas of human
action: initiating events cause characters to respond in a variety of
ways and to form goals that then motivate subsequent actions. The
acceptance of a story is determined by its global strength based on four
certainty principles\--coverage, coherence, uniqueness, and
goodness-of-fit. Of the stories constructed from the evidence presented
at trial, a juror adopts as the best explanation for the events the one
that best meets these certainty principles. \[FN167\]

\* \* \*

Coherence research fits with the story model. Both approaches posit that
evidentiary conclusions are not derived from mathematical computations
of the independent values of raw evidence. Inferences, rather, are based
on constructed representations of coherence, and it is these constructed
representations that ultimately determine the verdicts. Coherence
research overcomes an important limitation of the story model. As
indicated by its name, the story model proposes that the representation
of evidence bears a distinct narrative structure. Pennington and
Hastie\'s results showed that participants\' stories were centered on a
narrative that captured the intentional and causal scheme of the
defendant\'s behavior\--intentions, psychological states, \*564 goals,
and motivations. \[FN176\] But, in a range of evidentiary situations,
the material facts \"may concern a situation or state of affairs rather
than a sequence of events.\" \[FN177\] A narrative of human
intentionality is of little relevance to negligence cases where the
contested issue is a failure in appreciating a risk, to identification
cases, or to cases in which the material facts concern the physical
conduct of the defendant, the quality of a product, the extent of
damages, and the like. \[FN178\]

\* \* \*

Complex decisions are solved rather by nuanced cognitive processes that
progress bidirectionally between premises and facts on the one hand, and
conclusions on the other. Ultimately, people make decisions through what
appears to be a \*584 rational-like choice in which a strong alternative
is straightforwardly preferred over its rival. However, this dominance
is the product of an unconscious cognitive process that reconstructs and
transforms difficult and complex decisions into easy ones by amplifying
one alternative and deflating the other. This transformation of the
mental model of the decision lies at the heart of the decision-making
process.

**[Knowledge Structures, Schemas and Scripts]{.underline}**

This is a brief excerpt of an interesting article on how our mind relies
on and uses knowledge structures, schemas and scripts - what is in our
mind - when confronted with making a decision or judgment. It is useful
information before reading the next section of this paper.

CATEGORICALLY BIASED: THE INFLUENCE OF KNOWLEDGE STRUCTURES ON LAW AND
LEGAL THEORY, 77 S.Cal.L.R. 1103 (2004), Ronald Chen and Jon Hanson.
Excerpts:

Page 1110:

This Article is the beginning of our attempt to understand better not
only those who respond to the laws and those who make the laws, but also
those who devise the theories on which those laws are based. Knowledge
\*1111 structures, as we shall see, influence virtually every feature of
our cognitive processes and every thought we have\--from the trivial and
humorous to the profound and horrific. \[FN26\]

Page 1128:

Schemas do much more than simply constrain or bias our thinking in
humorous or harmful ways. The human system of processing information is,
in many cases, an efficient means of understanding our worlds and
ourselves. Classification of people, objects, and other stimuli is often
both indispensable and ineluctable. Like Polinsky\'s economic schema,
our day-to-day schemas can be very useful in providing us a workable
procedure for solving complex problems.

Still, as social psychologists have demonstrated, \"virtually any of the
properties of schematic functioning that are useful under some
circumstances will be liabilities under others.\" \[FN103\] The
categories and schemas that operate, usually automatically, influence
all aspects of information processing\--from what information we focus
on, to how we encode that information, to which features of that
information we later retrieve and remember, and to how we draw
inferences and solve problems based on that information. Given the
unconscious and biasing influence of our schemas, combined with the fact
that our schemas themselves will often reflect our unconscious motives,
we should be mindful, even distrustful, of our schemas and the
conclusions that they generate.

These effects, the processes that drive them, and the biases they
engender are the primary subject of this Article. A central goal is to
offer a broad understanding of how individuals utilize categories,
schemas, and scripts to help make sense of their worlds. In doing so, we
serve another main objective: to provide a comprehensive (yet
manageable) synthesis of a vast body of social psychology literature.
\[FN104\] This overview should transform how we make sense of our laws
and legal-theoretic world.

Page 1131:

Categories and schemas are critical building blocks of the human
cognitive process. \[FN109\] They allow humans to process or at least
cope with the infinite amount of information in their environs.
Categories and schemas influence every feature of human cognition,
affecting not only what information receives attention, but also how
that information is categorized, what inferences are drawn from it, and
what is or is not remembered. \[FN110\]

Page 1132:

Here, categorization and the use of categories will refer only to the
classification of elements, experiences, instances, or arguments into
groups. Once an element is categorized, an individual can apply a schema
to it in order to draw inferences and derive predictions. \[FN113\]
Where categorization focuses on the classification of instances, schemas
represent knowledge about the categories \[FN114\] and focus on the
\"application of organized generic prior knowledge to the understanding
of new information.\" \[FN115\] Put slightly differently, where
categories are the \*1133 classifications into which we place the
information we encounter, schemas refer to the embodiment of \"our
knowledge about the category and its members.\" \[FN116\]

Page 1139:

As we have been asserting, schemas have powerful effects. They help us
organize, find meaning in, and make predictions about our environs
\*1140 including the actions and behaviors of others. \[FN150\] In doing
so, schemas \"influence the encoding of new information, memory for old
information, and inferences where information is missing.\" \[FN151\]
Indeed, as Hazel Markus and Robert Zajonc explain, the influence of
schemas on our systems for processing information is ubiquitous:

In general, information processing may be seen as consisting of schema
formation or activation, of the integration of input with these schemas,
and of the updating or revision of these schemas to accommodate new
input. Accordingly, it should be \[and is\] possible to observe the
influence of schemas at every stage of information processing (e.g.,
encoding, storage, retrieval, inference), at all levels of processing
(conscious, preconscious), and on all parameters of the response (speed,
confidence, accuracy, etc.). \[FN152\] In short, schemas interact with
our information processing at every step, at all levels, and on every
parameter.

Page 1159:

In light of the significant role that schemas play in attention and
categorization, their dangers become more evident. William von Hippel
described the good and the bad of schemas this way:

Schema\[s\] clearly serve an important conceptual and organizational
role in information processing, and they provide insight into what is
likely to occur and what has probably occurred. For these reasons, they
seem to allow perceivers to process more information with less effort.
Yet herein lies their fault. . . . \[Very often\] schema\[s\] do not
really allow more information processing with less effort. Rather,
schema\[s\] simply allow the perceiver to quickly encode the gist of an
event, thereby causing the perceiver not to attend to the available
perceptual information. So the gist is stored, but the original
perceptual event is lost. In such a fashion, the momentary facilitation
in understanding that schema\[s\] bring about can \*1160 be outweighed
by the long-term loss to memory of the complex, perceptual world.
\[FN224\] More tersely, Kunda explains that \"the seemingly irrelevant
details that \[schemas\] lead us to ignore may actually be important.\"
\[FN225\]

Such dangers\--created by the fact that our categories and schemas may
bias our understanding of the world in numerous ways\--are one of the
key concerns of this Article.

**[Dispositionist Thinking and Judging]{.underline}**

Dispositionist thinking coupled with other cognitive processes discussed
in these articles play a strong role in such decision making as
advocating slavery or practicing genocide. Here is a brief excerpt from
an excellent article on the subject.

THE SITUATION: AN INTRODUCTION TO THE SITUATIONAL CHARACTER, CRITICAL
REALISM, POWER ECONOMICS, AND DEEP CAPTURE,152 U.Pa.L.Rev. 129 (2003),
Jon Hanson;David Yosifon

Page 312:

We all see dispositionally, and this dispositionism had long made people
blind to the situation of slavery, and to the situational influence on
slaves.

\* \* \*

Page 313:

Yet this common sense dispositionist presumption about Africans had not
always been the common sense. In fact, the dispositionism emerged in
America precisely because it was needed, as indicated above, \[FN654\]
to reconcile principles with practices. \[FN655\] And few people, we
suspect, felt the dissonance and the need to reconcile more intensely
than did Thomas Jefferson \[FN656\]\--civil rights theorist, father of
the American Revolution, author of the Declaration of Independence,
President of the United States, and Virginian slaveholder. \[FN657\]
Indeed, \*313 Jefferson presumed to analyze the inherent nature of
slaves\' inferiority through a neutral and scientific approach. In an
extended argument in his Notes on the State of Virginia, Jefferson
enumerated in remarkably dispositionist terms the evidence that he found
to justify the maintenance of slavery. \[FN658\]

\* \* \*

It is important to note that Jefferson does not entirely disregard the
influence of the situational effects of slavery. In fact, he purports to
examine and control for such evidence.

\* \* \*

Jefferson continues his proof by comparing American slaves to Roman
slaves,

\* \* \*

Thomas Jefferson, who announced to the world that \"all men are created
equal,\" \[FN672\] who founded the University of Virginia and carried
the Enlightenment\'s torch through his lifelong emphasis on the
importance of education, who believed he was an objective natural
historian and scientist, still could not see the situation in the case
of slavery. Dispositionism runs deep, and we can know with fair
confidence that, were we living in Virginia at the time that Jefferson
wrote, our position likely would have been even less situationally
sensitive than his was. \[FN673\]

**Excerpts From another article which outlines the importance of taking
into account internal (in the mind) and external situation rather than
merely disposition - motives, preferences, choices and will:**

THE SITUATIONAL CHARACTER: A CRITICAL REALIST PERSPECTIVE ON THE HUMAN
ANIMAL, 93 Geo.L.J. 1 (2004), Jon Hanson and David Yosifon

Page 6:

In The Situation, we highlighted several foundational studies
illustrating both the strength of dispositionism and the extent to which
our dispositionism is \*7 wrong. \[FN7\] Our exposition centered on the
path-breaking work of Yale psychologist Stanley Milgram, who cracked the
dispositionist nut wide open in a dramatic series of experiments in the
1960s.

Milgram arranged an experimental situation in which
subjects\--compensated volunteers\--were led to believe that they were
participating in a study on memory. \[FN8\] In the basic design of the
experiment, the subject first met another \"subject\"\--who was actually
one of Milgram\'s confederates\--and the two drew straws to determine
what part in the experiment they would take. The confederate was
inevitably assigned the role of the \"student,\" and promptly strapped
into a chair with electrodes affixed to his body. The true subject was
(seemingly randomly) assigned the role of the \"teacher,\" and was
instructed to administer an electric shock\--by flipping a switch on a
shock box\--each time the \"student\" incorrectly answered a question
posed by the experimenter. The \"teacher\" was led to believe that the
shocks would be painful, and that their intensity would increase in
fifteen-volt increments with each wrong answer\-- from 15 volts all the
way up to 450 volts, which was labeled \"Danger! XXX!\" on the shock
box. \[FN9\]

Before the experiment was undertaken, Milgram described the protocol to
lay people and psychologists and then asked both groups to estimate how
far most \"teachers\" would go with the shocking before refusing to
continue. Those surveyed believed, as might the reader, that most would
refuse early on. College students predicted that just 1 in 100 subjects
would shock all the way to 450 volts, and professional psychologists
predicted that only 1 in 1000 \-- \"the sadists\"\--would go that far.
\[FN10\]

But we humans do not\--and this is a central theme of critical
realism\-- understand ourselves well. \[FN11\] In the basic design of
the experiment, 100% of the subjects continued with the shocking at
least to 350 volts, and 65% went all the way to 450 volts (\"Danger!
XXX!\"). \[FN12\]

\* \* \*

In our dispositionism we \*8 fail to appreciate the powerful, but
unseen, situational influences over the subjects\' behavior in
Milgram\'s lab. Milgram performed his study in numerous settings on
hundreds of subjects who were, in all respects, typical people. They
were not sadists; they were simply, like all of us, situational
characters who were subject to unappreciated but profound influences in
the situation. Indeed, Milgram was able to alter his subjects\' behavior
by altering the situational influences. By varying the proximity of the
\"teacher\" to the \"student,\" or the \"teacher\" to the
\"experimenter,\" or by altering the prestige of the experimental
setting (by moving the location of the experiment from Yale to
Bridgeport, Connecticut), Milgram discovered he could increase or
decrease the level of shocking that subjects would be willing to
administer. \[FN13\]

Experiments like Milgram\'s, and there are literally hundreds of others,
\[FN14\] have demonstrated that we place far too much emphasis on
disposition\--on an individual\'s perceived motivations, preferences,
choices, and will\--in accounting for her conduct. In so doing we fail
to appreciate the very potent, though often unnoticed, influences of
situation.

Page 8:

At almost every turn, dispositionism defines or biases what we see and
how we construe what we see: behavior is strongly presumed to reflect
freely willed, preference-satisfying individual choice. But as
dispositionists, we are both consistent and consistently wrong.

Page 20:

Our purpose in reviewing a parade of dispositionism is to raise the
stakes of what probably will come as a surprise: the dominant
attributional schema informing our self-conceptions, our lay and social
theories, and our laws, is, in important ways and to significant
degrees, \*21 wrong.

Page 21:

The stakes of dispositionism are huge\--and they are, in our view,
stakes we are losing and will continue to lose if we persist in ignoring
the extent of our self-deception. Our sense of ourselves is wrong not
just in the details or on average, or in some bounded way. \[FN64\]
Rather, it is monumentally wrong, or so suggests the best available
social science.

Page 32:

5\. The Situation of Our Interiors

\"The more we examine the mechanism of thought, the more we shall see
that the automatic, unconscious action of the mind enters largely into
all its processes. Our definite ideas are stepping-stones; how we get
from one to the other, we do not know: something carries us; we do not
take the step.\" \~ Oliver Wendell Holmes, Sr. \[FN103\]

Our concern here is less with what so many of us, laypeople and
theorists alike, find self-evident and more with what we do not. As was
true exteriorly, it is the unavailable or less salient features of our
interiors that often wield the most influence over us. It is, in the
words of the elder Holmes, something \"we do not know\" that \"carries
us.\" Even Descartes\' terse proposition about the unity of thinking and
being is flawed in an illustrative way. The Cartesian insight imagines
thinking as that cognitive process that we are aware of and that is
independent from all else, including even the body in which that
thinking occurs. That is the flaw that neuroscientist Antonio Damasio
has dubbed \"Descartes\' Error\" in his book of the same title.
\[FN104\] Human cognitive processing is not simply the stuff of
conscious thinking. Our thinking is situational, and is influenced
outside of our awareness and control by everything from our bodies to
our social environments: \"Consciousness, the feature at the center of
what makes humans unique, is the culprit \[of our dispositionism\], for
it permits a view of who we are and what we are capable of that is
independent of the knowledge and feelings that may drive beliefs,
attitudes, and behavior.\" \[FN105\] But, \*33 as Damasio states:

\[T\]he comprehensive understanding of the human mind requires an
organismic perspective; that not only must the mind move from a
nonphysical cogitum to the realm of biological tissue, but it must also
be related to a whole organism possessed of integrated body proper and
brain and fully interactive with a physical and social environment.
\[FN106\]

Thinking may imply being, but our thinking is not what we experience it
to be. The problem is, again, one of perceiving disposition and
overlooking situation. That partial vision is, as we\'ve argued, a key
feature of being human and may be part of what Augustine was suggesting
when he wrote: \"Fallor ergo sum\" (I am deceived, therefore I am)
\[FN107\] or, in any event, it is what we mean when we write \"I think
dispositionally, therefore I am deceived.\"

Our point in this discussion has been that there is more to the
\"situation\" than what occurs outside of the human actor. Just as there
is an unseen exterior situation that gives rise to the exterior
fundamental attribution error, there is an interior
situation\--undetected but incredibly powerful\--that gives rise to the
interior fundamental attribution error. Our experiences are wrapped in
two layers of situational influences. To better understand what moves us
requires understanding them both; and to better understand the power of
either requires understanding its relationship with the other.

Page 33:

Situation, as we mean it, includes anything that influences our
attitudes, memories, cognitions, emotions, behaviors, and the like in
ways that we tend not fully to appreciate or control. The situation,
then, is part of the human predicament: it is in and around us, it is
influencing us, and it is doing so in ways that we do not appreciate, do
not understand, do not have a place for in our theories, and do not \*34
individually or dispositionally control.

Page 39:

Most of the best-known choice biases occur within a precise time period
and largely without relation to temporal considerations. In this
subsection, we briefly consider three of them: heuristics, endowment
effects, and framing effects.

i\. Heuristics

People, for good reason, are cognitively frugal. Some say, as we noted
earlier, that humans are cognitive misers. \[FN127\] Because cognitive
capacity is scarce, comer cutting is not just useful, it is necessary.
People engaging in inferential tasks \"virtually always\" rely on
judgmental strategies\--termed heuristics\--that help them reduce
complex problems into manageable ones. \[FN128\] Such strategies
\"probably produce vastly more correct or partially correct inferences
than erroneous ones, and they do so with great speed and little
effort.\" \[FN129\] Still, there are significant problems with such
mental rules of thumb. First, \"\[a\]lthough these heuristics often lead
to effective reasoning, they also \*40 lead to systematic biases and
errors.\" \[FN130\] Second, we normally do not realize we have these
biases, leaving us undefended against their harmful effects. \[FN131\]
And, third, for the same reasons, our cognitive shortcuts leave us
susceptible to exterior situational manipulation. \[FN132\]

Availability, for example, \"is a heuristic that is used to evaluate the
frequency or likelihood of an event on the basis of how quickly
instances or associations come to mind.\" \[FN133\] This shortcut may
not mislead us, but it often does. \"There are many factors uncorrelated
with frequency \... \[that\] can influence an event\'s immediate
perceptual salience, the vividness or completeness with which it is
recalled, or the ease with which it is imagined.\" \[FN134\]

Page 42:

iii\. Framing Effects

And more generally, the way in which an issue is presented to us
significantly influences how we perceive it. Psychologists have dubbed
this the framing effect. Even minor alterations in the presentation of
options that are substantively identical seem to influence our
perceptions and attitudes regarding the options. \[FN147\] Kahneman and
Tversky, the cognitive psychologists who identified and named the
phenomenon, describe it as \"both pervasive and robust.\" It is \"as
common among sophisticated respondents as among naïve ones \.... In
their stubborn appeal, framing effects resemble perceptual illusions
more than computational \*43 errors.\" \[FN148\] As another decision
theorist has explained, the power of the phenomenon results from our
\"tendency to accept problem formulations as they are given \... \[to\]
remain, so to speak, mental prisoners of the frame provided to us by the
experimentalist, or by the \'expert,\' or by a certain situation.\"
\[FN149\] More succinctly, \"framing\" is one identified piece of the
manipulable situation.

Page 43:

iv\. Summary

There is something familiar about the source of these well-documented
heuristics. They are different ways of talking about the same basic
phenomenon. They are all just manifestations in different contexts of
what we have been describing throughout this Article and its companion:
we see the vivid and we miss the pallid. Small pieces of the picture
tend to dominate our assessment of the whole image. We readily see what
is available, anchored, and presently normal, all according to how it
had been framed, and we find it difficult to see much else. As Ziva
Kunda puts it, these choice biases \"may be viewed as a kind of mental
contamination \.... Even though we do not want our judgments to be
contaminated in this manner, it is very difficult to eliminate the
contamination.\" \[FN150\]

That difficulty, we believe, is largely the consequence of our interior
situations. We do not see these biases at work. We do not see, in other
words, that we do not see. This inability to see our interior situation
is the source of the interior fundamental attribution error. And our
interior myopia helps give rise to the exterior fundamental attribution
error. People\'s behavior, like news of a homicide, is available. Their
situation, like statistics on stomach cancer, generally is not. We are
dispositionists because of what comes to mind most easily\--and, once in
our minds, anchors our attributions. \[FN151\] Interiorly and
exteriorly, we humans miss the situational forest for the dispositional
trees.

In part because our exterior dispositionism is causally related to our
interior dispositionism, our interior situation can be easily exploited
through the manipulation of our exterior situation. Each study
demonstrating a choice bias is itself indirect proof of that fact.
Scientists were able to manipulate cognitions by manipulating the
exterior situation. And the interior situation leaves open and unguarded
the gates through which the Trojan horse of exterior situation freely
enters, not as a trophy of our dispositional triumph, but as a hidden
means of influencing our behavior. \[FN152\]

Page 51 re schemas:

Thus, the benefit of such knowledge structures is that they provide us,
often automatically, with a way of understanding our world so that we
can operate reasonably well within it, at the same time that they free
up cognitive capacity to cope with other pressing issues. \[FN198\]
Similarly, the concepts, insofar as they are shared, allow us to
communicate efficiently with those around us. \[FN199\] In short,
without the knowledge structures \"\[w\]e would be unable to extract
meaning from the huge amount of information that surrounds us, unable to
generalize from one experience to another, and unable to communicate
effectively with each other.\" \[FN200\]

But those benefits are not without costs: \"A price is paid for this
mental economy.\" \[FN201\]

Page 52 re schemas:

Some beliefs, theories, and schemas are relatively poor and inaccurate
representations of the external world. Furthermore, objects and events
are not always labeled accurately and sometimes are processed through
entirely inappropriate knowledge structures. \[FN203\] \"Without these
structures stored in memory, life would be a buzzing confusion, but the
clarity they offer is helpful only in proportion to their validity and
to the accuracy with which they are applied to the data at hand.\"
\[FN204\] And when that is not the case, they can be misleading and
harmful. Indeed, that is the main point that this Article is making
about the dominant self-schema: dispositionism. We are not who our
knowledge structures tell us we are.

Indeed, the best known\--indeed, for many, the only known\--example of
the operation of interior schemas is the group-based stereotype.
\[FN206\] \"\[S\]tereotypes \*53 are typically viewed as cognitive
structures that contain our knowledge, beliefs, and expectations about a
social group \....\" \[FN207\] They are described as \"culturally
shared, indeed hackneyed, notions\" \[FN208\] about those groups. Here
is where the human tendency to rely on knowledge structures has come to
be understood as an inherently pernicious process, instead of as a
necessary and often helpful one. \[FN209\] Stereotypes, in other words,
are the stereotype of what we are calling \"knowledge structures.\"

Page 54:

The dynamics of our stereotypical thinking are driven by the interaction
of our interior and exterior situation. Stereotypes that are prominent
in our culture meet with a cognitive situation within us that is poised
to confirm them. We tend to test a hypothesis by asking questions about
whether there is evidence to confirm it and forgetting to ask whether
there is evidence that would disconfirm it. That unbalanced
positive-test strategy and the resultant confirmatory bias \[FN216\]
occurs, not only for hypotheses that we generate ourselves, but also for
any hypotheses or schemas that occupy our interiors. Thus our minds
automatically search for, and disproportionately emphasize, evidence in
the world that will tend to confirm our racial or sexual stereotypes (be
they negative or positive, conscious or implicit). \[FN217\]

The tendency is heightened by the fact that as a consequence of this
bias in our mental processing, social psychologists have suggested that
encouraging reflection about stereotypes, without more, may perversely
result in the search and location of further confirmatory evidence for
the stereotype under review. \[FN218\] More attention, in other words,
may not counteract the confirmation bias; it may simply give it more to
work with. \[FN219\]

Page 63:

i\. Causation, Responsibility, and Blame

There is a second \"fundamental process underlying much of social
perception and action\" that has long been understood by social
psychologists, but not by the rest of us: namely, the process of making
causal attributions. \[FN266\] People are eager to understand the causes
of salient outcomes and actions in their environs and have fairly deeply
ingrained schemas (using the term broadly \[FN267\]) for identifying
those causes. Fritz Heider introduced the concept of attributions in the
middle of the last century, and since then probably no feature of
cognitive process has received more attention from social and cognitive
psychologists. \[FN268\]

First, social psychologists went on to discover a number of systematic
biases in people\'s attributional processes\--including the
actor-observer bias, \[FN273\] the false consensus effect, \[FN274\] the
self-centered bias, \[FN275\] attributional schemas, \[FN276\] \*64 and,
most important, the fundamental attribution error. \[FN277\] There is
little need, in light of our discussion above (and below), to further
describe those sorts of biases or their sources. They are all
manifestations of the more general phenomena that we are highlighting in
this Article: our perceptions and construals are skewed by what we tend
to see, which tends to be only a small piece of the whole picture. For
example, instead of relying on covariation principles of the sort that
Kelley imagined, people too readily rely more on temporal and spatial
contiguity and salience in making causal attributions. \[FN278\]

For those sorts of reasons, Kelley\'s preliminary hypothesis is now
understood to have significant shortcomings, at least as a descriptive
model. This leads to the second significant development in attribution
theory. Numerous scholars have worked to develop more successful
descriptive models of people\'s attributional processes. \[FN279\] For
example, Bernard Weiner, with some of his colleagues, has shown that
people tend to focus on three (or four) causal dimensions: locus,
stability, control, (and, in some models, intent). \[FN280\] In his
early work, which focused on how people made attributions in terms of
these categories with respect to achievement efforts, he focused on
three dimensions: whether the cause was stable or temporary, whether the
locus of the cause was internal or external to the individual, and
whether the person had control over the cause. \[FN281\]

A little introspection\--or a careful read through the newspaper
headlines\-- should confirm that we humans are indeed focused on those
dimensions when examining causation for all sorts of surprising outcomes
that we encounter. Furthermore, as this line of research shows, our
reactions (affective and behavioral) vary significantly depending on how
we perceive a cause along those dimensions. Thus, the destruction caused
by a forest fire seems different to us when it was caused by a person,
rather than a bolt of lightening (locus). Likewise, it matters if the
person had control over the outcome (controllability), whether the
person has created several such fires in the past (stability), and
whether the person was a careless camper or a profiteering arsonist
(intent). Weiner\'s basic attributional model has enjoyed considerable
empirical support and has been expanded to apply in numerous settings.
\[FN282\]

And that leads to the third major development in attribution theory
since Kelley\'s initial effort. Social psychologists, including Weiner,
have constructed more refined theories to capture not just how people
make causal attributions, but also how they assign responsibility and
blame based on those causal \*65 attributions. Kelly Shaver has argued
that people tend to assign responsibility for harmful outcomes when,
roughly, the harm was foreseeable and when the person acted volitionally
and without justification. \[FN283\] And people assign blame, a more
punitive designation, when the harmful outcome was intended. \[FN284\]
In Shaver\'s words, \"An assignment of blame is \... a particular sort
of social explanation. It is the outcome of a process that begins with
an event having negative consequences, involves judgments about
causality, personal responsibility, and possible mitigation.\" \[FN285\]
Again, there is considerable evidence to support those attributional
theories. \[FN286\]

Page 65:

That leads to the most recent development in attribution theory. There
is significant evidence that many of our legal institutions reflect
people\'s attributional impulses (and not, for example, a means of
promoting efficiency or wealth maximization). Put most simply, just as
people want to identify causation, they likewise want to, when
attributionally appropriate, assign responsibility and blame. Moving
from causal attributions to attributions of responsibility or blame, the
lay scientist turns in her lab coat for a judicial robe.

\* \* \*

Just as the habits of the professional scientist reflect the same urges
and biases as the habits of the lay scientist, \[FN288\] so do the
judgments and sentences of the courtroom judge reflect those of the lay
judge. And our legal system appears to reflect and, to a significant
degree, satisfy our impulses to establish causation, assign
responsibility, and lay blame. \[FN289\]

Page 66:

But, if they are not pointed out to us, our attributional processes
begin and remain largely automatic and unconscious \[FN291\]; they are
experienced, if at all, as obvious and natural. In that way, our
attributions manifest themselves more as conclusions than as a process
or analysis that yields a conclusion. Like thinking, preferring and
choosing, our attributional ascriptions appear to us as self-evident
and, in a sense, dispositional. We miss much of the situation out of
which they emerge. \[FN292\]

Page 73:

Social psychologists have given a variety of names to this process\--for
example, confirmatory bias, perseverance bias, hypothesis-based
filtering, elastic justification, and, more generally, motivated
reasoning. \[FN329\] Perhaps unsurprisingly (given the topic),
researchers have found a great deal of evidence suggesting that such
devices are extremely powerful. Richard Nisbett and Lee Ross summarized
the literature in 1980 as follows:

When people already have a theory, before encountering any genuinely
probative evidence, exposure to such evidence (whether it supports the
theory, \*74 opposes the theory, or is mixed), will tend to result in
more belief in the correctness of the original theory than normative
dictates allow.

\... When people approach a set of evidence without a theory and then
form a theory based on initial evidence, the theory will be resistant to
subsequent evidence\.... \[and\]

\... When people formulate a theory based on some putatively probative
evidence and later discover that the evidence is false, the theory often
survives such total discrediting. \[FN330\]

The influence of those biases, and others, \[FN331\] makes clear one of
the great problems with our schemas: we create them too quickly and
maintain them too loyally. And, again, this problem is not limited to
the processes of just lay scientists: \"The tendency of professional
scientists to persist in adhering to theories well past the point at
which such adherence can be justified by the evidence has been observed
by many.\" \[FN332\] We will return to that point briefly below,
\[FN333\] and more thoroughly in subsequent articles. \[FN334\] For now,
the crux of our point is that all of us are subject to the same biasing
process that we just do not see.

To be sure, we often see others as biased, prejudiced, vested,
incoherent, inconsistent, or closed-minded. \[FN335\] But those
experiences are schematized as dispositional quirks, not as a reflection
of their deeper interior situations. \[FN336\] In ourselves, we see what
every person is presumed capable of\--clarity, objectivity, and
open-mindedness. \[FN337\] And we can maintain that self-affirming
view\--as do those who we feel should not\-- in significant part because
we do not see the interior situation. \[FN338\]

As with the other biases, there\'s a critical kicker to this analysis:
exterior \*75 situation, which we began by emphasizing our failure to
appreciate, can wield an immense influence over which schemas we adopt,
which we reject, and how and when we apply them. And that process is
itself guided in part by the operation of interior schemas functioning
below the level of our conscious awareness. Furthermore, once our
schemas are in place, exterior situation can provide us the evidence and
ambiguity we need to sustain them.

Page 75:

No human inference process is without schematic structuring, and none is
insulated from schematic distortions. \[FN342\]

Page 84:

As this discussion helps to demonstrate, professional scientists and lay
scientists are bedeviled by the same situation. Knowledge structures and
schemas are all around us, guiding our every, or most every, thought,
and simultaneously assisting and distorting what we \"know.\" The
coloring concepts and theories we employ can lead us to focus on
irrelevant details in our environment, to overlook the relevant details,
and to misunderstand our world. \[FN387\] As we hope the reader has
already recalled (with the aid of the knowledge structure that we are
attempting to create), that is precisely the mechanism behind the
exterior and interior fundamental attribution errors.

Page 84-85:

Where lay people and economists see \"thinking,\" they vastly overstate
its significance and vastly understate the interior situation of our
thinking\-- that is, our unseen cognitions. Social psychology and
related fields make clear that all of our cognitive processes are more
or less influenced by unseen and distorting influences, from heuristics
and framing effects on one hand to schematic and attributional processes
on the other. And all of those unseen cognitions and cognitive processes
render us more or less vulnerable to outside manipulation\--indeed, the
experiments revealing the cognitive phenomena simultaneously \*85 reveal
the extent to which they can be tapped through exterior situation.

In short, this section has shown some of the ways in which, although it
is true that we experience ourselves thinking, we do not think the way
we think we think. There is more to the situation. And, as the next
section illustrates, we have barely scratched the surface of our
interior situations.

\*91 It is not just that our minds have a mind of their own (as the
previous analysis has indicated), it is also that those inner minds have
a motivation\--actually, a whole set of motivations\--of their own.
\[FN421\]

\* \* \*

four general types of motivations stand out as particularly significant
aspects of the situational character\'s inner life: (a) the motive to
understand; (b) the motive to self-affirm; (c) the motive to simplify;
and (d) the motive to cohere. \[FN422\] As our review of more specific
motives will reveal, motivations can be, and often are, in tension with
one another. The motive to understand, for example, is often in tension
with the motives to self-affirm and simplify. The motive to cohere,
then, pushes us to reconcile our conflicting motivations by altering
exteriorly and interiorly our cognitions, attitudes, or behavior.

\* \* \*

Sometimes, though, our self-affirming motives can conflict with our
dispositionism\--that is, there are instances, such as following a
failed or disappointing performance, that we look to situation for
causal attributions in order to avoid the disheartening conclusion that
that failure reflected our own dispositional shortcomings. \[FN461\] One
experimenter interviewed politicians several months after an election.
The winners attributed their performance largely to dispositional
factors such as hard work, perseverance, skill, planning, and strategy.
The losers, on the other hand, looked to situation, and attributed their
performance to the politics of the district, their opponents\' name
recognition, to their lack of money, and so on. \[FN462\] The groups
thus revealed the two-sided nature of the self-serving attributions: \"a
\'self-enhancing bias\' (attributing success to internal relative to
external causes) and a \'self-protecting bias\' (attributing failure to
external relative to internal causes).\" \[FN463\]

Page 101:

Our self-affirming motivations extend beyond our self and group
identifications. Just as each of us is motivated to believe that \"I am
good,\" and, with respect to our groups, \"we are good,\" so too are we
motivated to believe that \"our world is good.\"

In a pioneering project within social psychology, Melvin Lerner
demonstrated that people seek to confirm a \"just world hypothesis.\"
Through a series of experiments, Lerner demonstrated that, \"we do not
believe that things just happen in our world; there is a pattern to
events which conveys not only a sense of orderliness or predictability,
but also the compelling experience of appropriateness expressed in the
typically implicit judgment, \'Yes, that is the way it should be.\"\'
\[FN479\] And, yes, \"people get what they deserve.\" \[FN480\]

\* \* \*

In our effort to maintain our belief in a \"just world\" hypothesis, we
tend to attribute bad outcomes to individual dispositions, because it is
generally more comforting to presume that it is the person who was bad,
rather than the situation.

Page 107:

We humans seek explanations that are coherent, that we can make sense
of, and that can be supported by reasons. \[FN508\] This coherence
motive animates the relationship and tradeoff among motives. Because we
value coherence, the desire to see it in ourselves dovetails with our
motive for self-affirmation. That powerful driving force in our
self-conception has figured prominently in social psychological
research. Inquiry into \"cognitive dissonance,\" for example, has been a
mainstay of the field for decades. \[FN509\] Often it is the case, as we
have already suggested, that our motivations are in conflict. Though
motivated to view ourselves positively, our behavior can pose problems
for that self-conception. There can be many reasons for such
dissonance\--not least of which is the fact that we are dispositionist
situational characters.

While social psychologists debate some of the details of the cognitive
dissonance dynamic, \[FN510\] it is the basic pattern, about which most
agree, that best illustrates the dispositional illusion. The
dispositionist sees behavior as reflecting little more than thinking,
preferring, and willing. From that conception, the dissonance can be
eliminated by bringing behavior into line with those interior elements.
But it can also be, and commonly is, eliminated, by bringing the latter
into line with the former. \[FN511\] That basic challenge to
dispositionism\--that our behavior influences our beliefs and
attitudes\--has been a recurring theme in the cognitive dissonance
literature: \"Regardless of the exact motivational underpinnings of
dissonance, the evidence clearly indicates that attitudinally discrepant
actions can result in a reanalysis of the reasons why a person engaged
in a certain behavior (or made a certain choice), and cause a person to
rethink the merits of an attitude object.\" \[FN512\] That subconscious
task can be accomplished in a myriad of ways, from changing our opinions
outright to more subtly trivializing \*108 a belief that is incongruent
with our behavior. \[FN513\]

The important critical realist lesson here is not just that behavior may
not reflect a preference or belief. That much we have said before when
describing how situation can powerfully influence behavior\--disposition
notwithstanding. The lesson is also that our preferences, such as they
are, are themselves malleable, constructed, and contingent\--subject to
changes in our behavior and in our situation. The malleability of our
attitudes, combined with our sense that attitudes are stable, assists us
in our motive for coherence.

Page 109:

This desire to see ourselves in a positive light is an important motive
behind what Lee Ross and his co-authors have dubbed \"naïve
realism\"\--the name \*110 given to \"three related convictions about
the relation between \[one\'s\] subjective experience and the nature of
the phenomena that give rise to that subjective experience.\" \[FN523\]
First, we naively believe that we see the world as it really
is\--through objective, unfiltered lenses. Most of us think that we
\"get it\" for the same reason that the vast majority of us believe that
we are above average drivers\--it is self-affirming. Second, it almost
goes without saying that anyone else who is similarly neutral and
intelligent will see the world as we do\--that is, accurately. At times,
though, we are confronted with views that conflict with our own, an
experience that creates a kind of dissonance. That suggests the third
tenet of naïve realism. When our interpretation of a particular
situation apparently conflicts with someone else\'s, something has to
give. Because we presume that we see things as they are, something must
be distorting the perceptions of those who see things otherwise. Social
psychological research shows that an extremely common means of relieving
that dissonance is to attribute the gap between our outlook and theirs
to a lack of objectivity on their part. We assume that there is some
dispositional source of their bias\--lack of intelligence, or laziness,
or corruption. To be sure, we ourselves will admit to having a
particular vantage point and set of experiences that inform our judgment
and perspective\--but as it turns out, our particular background was the
path to authentic insight. \[FN524\]

This is a key source of our biases: we don\'t believe that we are
subject to them (allowing us to trust our own clear vision) and we are
extremely quick to see them in others (allowing us to distrust others\'
obscured vision). \[FN525\] And so it is that we are quick to see
ideological or political bias on the part of our adversaries and
gullibility or vanity on the part of even our friends and family when
they fail to share our worldview. \[FN526\]

Page 110:

In sum, we see bias there, but not here\--and, in either case,
dispositionism. \[FN530\]

Regarding groups:

page 119:

Social scientists have discovered numerous manifestations of our motive
for group coherence. That motive, combined with the self-affirming faith
we place in our own knowledge structures, contributes, for instance, to
a phenomenon that social psychologists have dubbed the \"false
consensus\" effect. \[FN536\]

\* \* \*

This false consensus effect, a tendency to regard one\'s own views to be
commonly held by others, has been demonstrated by more than a hundred
empirical studies, over a wide range of topics from particular food
preferences to broad political and social policy views. \[FN539\]

\* \* \*

The group-coherence motive combined with dispositionism can yield some
troubling and otherwise perplexing phenomena. Because we are
dispositionists, our perception that certain behaviors are common (or
uncommon) leads us to perceive that the attitudes, preferences and
beliefs of others correspond to that common (or uncommon) behavior.
Because we do not ourselves subscribe to those attitudes, we infer from
others\' behavior that our attitudes are exceptional. That dynamic
contributes to the tendencies known in social psychology as
\"pluralistic ignorance\" and \"false uniqueness.\" Both illustrate the
power of the group coherence motive.

\* \* \*

The bad news was that some subjects began to alter their own behavior
and beliefs to more closely correspond with their ignorant perceptions.
\[FN543\] Prentice and Miller attributed that troubling example of
self-fulfilling group perceptions to a basic motive to avoid dissonance
with one\'s key constituencies. Once again, it is crucial to recognize
that the subjects were not faking their new attitudes. These are not
rational actors operating within stable preferences, altering their
conduct through clever performance while remaining true to their core
beliefs. These are situational characters in whom stable preferences and
core beliefs are largely an illusion. Our behavior and attitudes and the
behavior and perceived attitudes of our groups are all mutually
constructed and reconstructed as the situation requires. As each of us
looks out at others seeing disposition and missing situation, we infer
attitudes that do not exist. But, as these studies reveal, our false
perceptions can be tragically powerful as we each seek to bring our own
view and behavior into sync with shared misperceptions. Prentice and
Miller \"believe that group identification is the root cause for many
cases of pluralistic ignorance\--\*115 that individuals often act of a
desire to be good group members but interpret others\' similarly
motivated behavior as reflecting personal beliefs and opinions.\"
\[FN544\]

The problem of pluralistic ignorance and the motive for group coherence
distorts many social norms and would seem to have significant
implications for policy and law. The dynamic seems to be at work in
creating and reinforcing gender and sex roles. \[FN545\] It is also
behind the pervasive, dysfunctional classroom dynamic in which students
do not ask questions because they assume that others\' silence suggests
they are themselves alone in their ignorance, thus contributing to the
silence that encourages others to do the same. \[FN546\] And so it is
that even in most \"learning\" environments, ignorance begets ignorance.

Regarding free will:

Page 124:

And so we come at last to the conscious will, that inarticulable, yet
unmistakable, inner experience of \"oomph\" that is, in many ways, the
crown jewel of our \*125 dispositionist self-conception. \[FN596\] We
have been emphasizing throughout this Article that there is a vast
interior situation that invisibly influences our thoughts, preferences,
and actions, and leaves us vulnerable to exterior situational forces
that do the same. What, then, of the conscious will? Much of the social
psychological material canvassed in this Article has touched on the
issue of conscious will only implicitly, if at all. Nonetheless, such
material has quietly suggested the same conclusion drawn by those
researchers who have examined the issue directly. If we perceive that we
consciously will our actions, even as the best evidence indicates that
our behavior is substantially influenced by interior and exterior
situational factors, then this experience of \"will\" may be a part of
our dispositionist deception. Indeed, as Daniel Wegner concludes in a
book that brings together generations of experimental research on the
felt experience of human will: \"\[C\]onscious will is an illusion. It
is an illusion in the sense that the experience of consciously willing
an action is not a direct indication that the conscious thought has
caused the action.\" \[FN597\] Two other leading researchers of the
will, John Bargh and Tanya Chartrand, have made an extremely compelling,
if unsettling, case that \"most of a person\'s everyday life is
determined not by \[her\] conscious intentions and deliberate choices
but by mental processes that are put into motion by features of the
environment and that operate outside of conscious awareness and
guidance\"\--a thesis that they acknowledge is \"difficult \... for
people to accept.\" \[FN598\]

In part for that reason, we want to be certain that the claim is not
misconstrued. None of the researchers in this field of social science
have concluded, nor do we, that the \"conscious will\" is purely and
totally an illusion. What is asserted\--and what researchers have
demonstrated\--is that the experience of will is far more widespread
than the reality of will. Wegner calls the latter the empirical will
\[FN599\] and argues that our perceived will is often an unreliable and
misleading basis for understanding our behavior. The experience of will
occurs often without empirical will, and thus creates the illusion of
will. Moreover, it contributes to the illusions of choice, preference,
and, more generally, dispositionism.

Page 133:

What is now fairly clear is that the dominant lay and legal theories of
the person (or \"personology\") are wrong\--not just \"too simple,\" but
fundamentally wrong. Social science has clearly demonstrated that we are
not who we think we are. It is true that we experience ourselves
thinking, preferring, acting, and willing, but those comforting
perceptions are often illusory, and they obscure the far more
significant influence of our unseen interior situation.

Page 177:

We hold onto our dispositionist views tenaciously, even well beyond the
point where our claims about the truth of the matter have been
abandoned. We are dispositionists, not because humans are dispositional,
but because there is too much that has already been built upon the
dispositionist foundation to begin building elsewhere, and because
dispositionism helps us to sleep.

As Nobel Laureate Isaac Bashevis Singer elegantly expressed the dilemma:
\"Of course I believe in free will. I have no choice.\" \[FN820\] And so
it is that the illusion of our freedom, our dispositionism, our wills,
are forced upon us by fears and forces in our situation that we do not
see, and would prefer not to. We have no choice but to pretend that we
have a choice.

Page 178:

Taken together, the social psychological findings we have reviewed here
should shake our self-conception at its foundation. What has been
revealed in the studies we reviewed cannot be considered marginal or
anomalous. Situation, it seems, moves us far more than we suspect in our
slumbering, blissful dispositionism. That conclusion is hard to take,
hard even to get our mind around, because of this very dispositionism,
which sees us as responsible for our situations and not the other way
around.

Page 179:

There is a tragic irony in our predicament. By blinding ourselves to the
very forces that impinge upon our freedom, we are surrendering to them.
To be serious about liberty requires that we not unwittingly turn over
the situational strings to whoever has the means and ends to manipulate
them. Indeed, our greatest dispositional act may be to acknowledge that
we are situational characters and to choose to understand and gain some
voice and control over the situation that largely controls us. In that
very important sense, we do have a choice.

**[Intergroup Conflicts]{.underline}**

The following 18 page article sheds more light on inter-group conflict
grounded in cognitive biases and dispositionist thinking. Among the
contributing causes are Self Serving Attributions, Self Interest bias,
Reactive Devaluation of Proposals from Counterparts, Fundamental
Attribution Error, Positive Halo Effect, Perception of Hostile Media
Effect against Group or Cause, Biased Assimilation of New Information to
Preexisting Beliefs, Cognitive Dissonance, Belief Perseverance, Attitude
Polarization, False Polarization Effect, False Consensus Effect,
Illusion of Personal Objectivity, Overconfidence in Social Prediction,
Rationalization, the Planning fallacy, Nonconsious Priming and
Correspondence Bias.

**THEORETICAL NOTE**

**Objectivity in the Eye of the Beholder:**

**Divergent Perceptions of Bias in Self Versus Others**

Psychological Review, Copyright 2004 by the American Psychological
Association, 2004, Vol. 111, No. 3, 781--799, Emily Pronin, Princeton
University; Thomas Gilovich, Cornell University; Lee Ross, Stanford
University. The Summary:

"Important asymmetries between self-perception and social perception
arise from the simple fact that other people's actions, judgments, and
priorities sometimes differ from one's own. This leads people not only
to make more dispositional inferences about others than about themselves
(E. E. Jones & R. E. Nisbett, 1972) but also to see others as more
susceptible to a host of cognitive and motivational biases. Although
this blind spot regarding one's own biases may serve familiar
self-enhancement motives, it is also a product of the phenomenological
stance of naive realism. It is exacerbated, furthermore, by people's
tendency to attach greater credence to their own introspections about
potential influences on judgment and behavior than they attach to
similar introspections by others. The authors review evidence, new and
old, of this asymmetry and its underlying causes and discuss its
relation to other psychological phenomena and to interpersonal and
intergroup conflict."

Page 794:

"Our account of the asymmetry in perceived bias is best appreciated when
considered within the broader framework of naive realism. The essential
component of naive realism---namely, the (false) sense that one sees the
world as it is and not as it is filtered through one's expectations,
needs, or motives, or "constructed" by one's sensory and cognitive
apparatus---is presented in Figure 6. Also presented there are the two
components of naive realism that follow immediately from this core
conviction: the expectation that "objective and reasonable others" will
share one's perceptions and judgments, and the inference that those who
do not share one's perceptions and judgments are therefore either
uninformed, biased, or under the influence of idiosyncratic
dispositions. Finally, Figure 6 presents a number of more specific
attributional and

information-processing phenomena that stem from these three basic
components of naive realism, some of which we shall enlarge upon briefly
here. When others see things differently, when confronted by an
individual whose responses differ from one's own, the naive realist
faces an attributional dilemma: Why does this person see the world so
differently? Sometimes this dilemma is resolved in the fashion
emphasized by Jones and Nisbett (1972): by concluding that the other
person possesses some telling disposition (or set of dispositions) that
is dictating his or her reactions. This inference is particularly likely
when the discrepancy involves not a differing viewpoint or
interpretation but a behavior that differs from the way one would behave
(or anticipates one would behave). The tendency to resolve this
attributional dilemma by drawing a dispositional inference is, in turn,
related to the correspondence bias (Jones, 1990) or fundamental
attribution error (Ross, 1977). Often, this error results from the
incorrect and insufficiently examined assumption that the person who
behaves differently is in fact responding to the same situation one has
in mind oneself. By assuming that the other person is responding
differently to the same "object of judgment" as oneself (Asch, 1952),
one fails to appreciate the true situational constraints governing the
actor's behavior and thus runs the risk of drawing an unwarranted
dispositional inference. As Figure 6 makes clear, the attributional
dilemma set in motion by responses discrepant from one's own is often
resolved in other ways. Most charitably, perhaps, people may assume that
those others are misinformed or have not had sufficient contact with the
pertinent information. This motivates attempts to "set the other person
straight," accompanied by the expectation that such efforts at
persuasion will prove easy and should proceed smoothly to a successful
conclusion. When that expectation is not borne out, less benign
attributions result, including the attribution of bias that we have
emphasized throughout this article."

Page 796:

"The convictions of naive realism can make parties feel that the other
side is irrational or too biased to be reasoned with (rather than merely
subject to the same cognitive and motivational biases that afflict all
human beings---including oneself and one's ideological and political
allies). Moreover, when the parties do air their grievances, they may
conclude that the other side is being "strategic" and doesn't really
believe what it is saying. Or perhaps worse, the conclusion reached may
be that the other side really does believe what it is saying and that a
rational, interest-based negotiation will thus prove fruitless, and that
therefore only the use of force can win one's own side the just outcome
to which it is entitled. Of course, when force is applied, the cycle of
aggression and retaliation is apt to be self-perpetuating, as both sides
see the other as the instigator and their own side as acting in
self-defense or trying to teach the other side a lesson."

Page 797:

"The evidence is all too plain that human inferential
shortcomings---including overconfident prediction and biased
assimilation of the lessons of history, compounded by people's
unwillingness to consider the possibility that they are just as
susceptible to those biases as those they revile---are continually and
ubiquitously making their influence felt. At the very least, these
shortcomings in judgment and insight serve to exacerbate and perpetuate
the historical and economic roots of the conflicts that are
all-too-present elements of the human condition."

**III. CONCLUSION**

Anyone interested in decision making and judging can improve themselves
by better understanding the mental processes involved. Understanding the
human mind also helps us understand ourselves, others and the world as a
whole. The new Foundational Doctrine being developed for wildland
firefighting continues to rely on quality, professional decision making.
Therefore, the wildland fire community should consider the extent to
which knowledge about decision processes, and ways to improve them,
should be included in firefighter training from basic to advanced.

[^1]: The views in this paper are my own, and not those of the
    Department of Justice or the U.S. Attorney for the District of
    Arizona.