Number range CHAPTER **[WHAT WAS HE THINKING?]{.underline}**

**[BEYOND BIAS-]{.underline}**

**[TO DECISION MAKING AND JUDGING]{.underline}**

**This paper was prepared for the 2007 Serious Accident Investigations
course, BLM National Training Center, Phoenix, AZ.**

Mike Johns, Assistant U.S. Attorney and Senior Litigation Counsel

<mike.johns@usdoj.gov> (602) 514-7566.

March 8, 2007.[^1]

**[CONTENTS]{.underline}**

**I. Summary 1**

**II. Cognitive Biases 6**

**III. Outcome Knowledge and Judging Quality of Decisions 21**

**IV. Coherence Based Reasoning 25**

**V. Knowledge Structures and Schemas 34**

**VI. Dispositionist Thinking and Judging - causation, responsibility
and blame 37**

**VII. Interpersonal and Intergroup Conflict 55**

**VIII. Cultural Cognition 58**

**IX. Conclusion 61**

**[I. Summary.]{.underline}**

Psychological research demonstrates that a wide variety of **cognitive
biases** and **heuristics (cognitive shortcuts)** adversely affect
decision making, and the ability to judge decisions made by others.
Coherence-based reasoning can lead to skewed decisions. Improved
knowledge structures and schemas lead to better decisions. Due to
**attribution errors** and **dispositionist** thinking, we tend to
ascribe more to preferences and will, while missing the situation. This
can skew opinions on causation, responsibility and blame, and decrease
effectiveness of remedial measures.

Because there is an infinite amount of information which can be brought
to bear on any decision process, the mind employs simplifying processes
which facilitate timely decisions, with enough confidence to carry them
out. Otherwise the situation could become fatal before a decision is
made. Information is quickly analyzed against one's own knowledge
structures and schemas which operate subconsciously and include
stereotypes and what firefighters call "slides" from past experiences.
**Recognition Primed Decision making (RPD)** facilitates good, quick
decisions in many situations, but the available information is often
ambiguous and conflicting, requiring **Critical Thinking.** RPD is
insufficient for the less experienced, younger firefighters, and can
cause overconfidence in the very experienced firefighters - critical
thinking fills the gap. However, critical thinking is contaminated with
cognitive biases, heuristics (short cuts) and processes which need to be
understood and mitigated for improved decision making.

During the decision process, alternatives begin to emerge in a
bi-directional process which begins to favor one hypothesis for action
over the others. The **confirmation bias** finds within memory,
information which tends to confirm the favored hypothesis, while not
recalling, minimizing or ignoring information which tends to refute the
hypothesis. Temporary **coherence shifts** begin to attribute more
belief that evidence which is ambiguous or conflicting actually supports
the emerging decision, and the process results in a confident decision
so the actor can act - the leader can lead - analysis - paralysis is
overcome. More often than not these cognitive processes lead to correct
or at least acceptable decisions. Unfortunately, poor knowledge
structures and schemas, cognitive shortcuts, biases and coherence shifts
can generate a confident decision which is just flat wrong from a purely
rational analysis, especially in hindsight. Studies have shown that
coherence shifts and many cognitive biases can be significantly reduced.
Some cannot, and need to be mitigated.

The United States Navy has developed and implemented a program called
**Tactical Decision Making Under Stress (TADMUS)** to improve decisions
whether to engage a radar screen target - is it civilian, friendly,
hostile, hostile but merely on patrol or harassing, or does the hostile
intend to attack the ship? The program was built around known human
cognitive decision making processes, including known cognitive biases
such as the **confirmation bias**, **availability bias,
representativeness heuristic, contrast bias and framing.** The program
employs de-biasing decision support systems. The program combines
decision making training with a **Decision Support System (DSS)** which
is not "command" based (what to do), but rather "information" and
"status" based, with **trigger point** reminders and a "**Quick Check"
de-biasing technique** when time to decide is short. Unlike command
based systems, the system provides critical information in graphic and
other forms compatible with human cognitive processes. Short and long
term memory is relieved, leaving more capacity for cognition.
Situational awareness is enhanced. The system fits the "**story model**"
of human decision making, in which the most coherent story - rather than
specific pieces of evidence - becomes the decision and action taken. The
DSS rejects the "checklist mentality", replacing it with an
"intelligent" assistant, promoting the development and analysis of
alternatives within the human mind. The program acknowledges the role of
RPD, and uses Critical Thinking to fill in the gap. The results of
critical thinking training greatly reduced "**coherence shifts**" and
the effects of biases, increasing the number of correct decisions and
resulting actions. E.g., "Integrated Critical Thinking Training and
Decision Support for Tactical Anti-Air Warfare", Marvin S. Cohen, Ph.D.,
Jared T. Freeman, Ph.D., and Bryan B. Thompson; "Decisionmaking in
Complex Military Environments", Gary Klein; "Principles for Intelligent
Decision Aiding", Susan G. Hutchins, 1996.

Many scholars have noted the unfortunate use of the term "bias" in
describing what has turned out to be normal human cognitive processes,
such as the "**Hindsight Bias**", but the nomenclature is firmly
established in the literature and therefore unavoidable. Human emotions
also plays a part in higher cognition - they are not completely separate
processes, and emotion can play a strong role in decision making under
stress and uncertainty. E.g., "Deep Survival: Who Lives, Who Dies, and
Why", Laurence Gonzales.

The **Outcome Bias** contaminates our ability to judge the quality of a
decision and the character of the person who made it. There is an entire
field of cognitive science on **how to judge the quality of another's
decision**, and at least three models for doing so. E.g. "On the
Assessment of Decision Quality: Considerations Regarding Utility,
Conflict and Accountability", Gideon Keren and Wandi Bruine de Bruin,
from *Thinking: Psychological Perspectives on Reasoning, Judgment and
Decision Making*, Eds. Harding, D and Macchi, L., Wiley 2003.

The research also helps explain such things as inter-group conflict, why
it is so hard to change someone's mind, why history keeps repeating
itself, and many other social and political phenomena. An interesting
study on **Belief Perseverance"** used a model in which people formed
beliefs about which kind of personality made for a **better
firefighter**. Groups were intentionally mislead - lied to - and told
afterward that they had been intentionally mislead. Nonetheless, they
continued to hold onto their misinformed beliefs against every effort to
correct it, including explaining the lies and trickery. "Perseverance of
Social Theories: The Role of Explanation in the Persistence of
Discredited Information", Craig A. Anderson, Mark R. Lepper, and Lee
Ross, Journal of Personality and Social Psychology, Vol 39, No.6,
1037-1049 (1980).

**Cultural cognition** sheds light on why social-political issues can
seem intractable -- resistant to change based merely on empirical
evidence, and provides methods for progress in resolving them.

The new **Foundational Doctrine** for Wildland Firefighters continues to
rely on quality, professional decision making on the fire ground. Region
One\'s recent Doctrinal policy explains, for example, that risk
management is evaluated on the decision-making process, not on the
outcome; training will teach employees how to think and make appropriate
decisions; safe practices are to be embedded in all workforce decisions
and actions; training will teach how to think, make good decisions, and
act decisively - not what to think, and so on. "Foundational Doctrine
and Guiding Principles for The Northern Region Fire, Aviation and Air
Program", USDA Forest Service, October 2006.

Understanding and training in decision processes, including **Critical
Thinking**, seems appropriate at every level from basic training to
advanced leadership training. The current Leadership Training courses
teach RPD but expressly do not teach critical thinking. Improvement of
our existing **Decision Support Systems** could also facilitate better
decisions under stress and uncertainty.

Building all the **appropriate knowledge structures and schemas** is
also important at every level of training. Our knowledge of fire
behavior, including conditions conducive to blow ups, has exploded over
the past few decades.

The requirements for creating a "**Just Culture**" and "**High
Reliability Organization**" is beyond the scope here, but this paper
provides useful knowledge in understanding those requirements as well.
Current accident investigation processes are inconsistent with the
requirements. Number range CHAPTERDr. Mary Omodei and others at the
Complex Decision Research Group, LaTrobe University, Melbourne,
Australia, developed a protocol for wildland fire accident
investigations to get to the bottom of human factors and decisionmaking.
They are collecting the data. They note how the hindsight bias impedes
collecting the information needed to understand the role of human
factors. They also note how the self-affirming biases or
"self-protective justifications" are a normal part of human cognition
and should be expected to occur and need to be dealt with appropriately
in an accident investigation. See "**Identifying Why Even Well-Trained
Firefighters Make Unsafe Decisions: A Human Factors Interview
Protocol**", In Butler, B.W. and Alexander, M.E. Eds. 2005. Eighth
International Wildland Firefighter Safety Summit-Human Factors-10 Years
Later. Dr. Omodei has commented to me that "The threat of litigation
and/or being called before an enquiry, either judicial or agency
mandated, is in my view the single biggest impediment to accurate
investigation and/or research into the human factors underlying
"problematic" decision making in incident management (at all levels)."
They recommend separate, priviledged investigations to get at the human
factors. They recognize the value of Safe-Net type reporting.

In a similar vein, the following is from the medical side of the same
issue:

"The importance of near-misses and no harm events stems from the
documented observation of their frequency: they occur 300 to 400 times
more often than actual adverse events and thus enable quantitative
analysis and modeling."

\* \* \*

"One study documented that intensive care entails 178 activities per
patient per day and reported an average of 1.2 errors per patient per
day.\[fn.29\] This works out to safety ratio of 0.955 compared with
civilian airline ratio of 0.98." "Nature of Human Error, Implications
for Surgical Practice", Alfred Cuschieri, MD, FRCS, FACS (Hon), FRSE

AMERICAN SURGICAL ASSOCIATION FORUM, Annals of Surgery • Volume 244,
Number 5, November 2006.

For further analysis of how the hindsight bias and the outcome bias
interfere with current accident analysis and prevention efforts, see
also "**Perspectives on Human Error. Hindsight Biases and Local
Rationality"**, Woods, D.D. and Cook, R.I., In F. Durso (Ed.), Handbook
of applied cognitive psychology (pp. 141-191). NY Wiley.

The intent of this paper, then, is to encourage better decision making,
better actions, better judgments about the decisions and actions of
others, and to encourage development of better decision support systems
and remedial measures.

**[II. Cognitive Biases.]{.underline}**

**A. Hindsight Bias**

Research on the human mind has demonstrated that hindsight bias is
robust and virtually impossible to eliminate. It wreaks havoc on people
caught second-guessing their own actions, as well as others with outcome
knowledge who judge those actions:

"Consider a decision maker who has been caught unprepared by some turn
of events and who tries to see where he went wrong by recreating his
preoutcome knowledge state of mind. If, in retrospect, the event appears
to have seemed relatively likely, he can do little more than berate
himself for not taking the action which his knowledge seems to have
dictated. He might be said to add the insult of regret to the injury
inflicted by the event itself. When second guessed by a hindsightful
observer, his misfortune appears to have been incompetence, folly, or
worse." "Hindsight = Foresight: The effect of outcome knowledge on
judgment under uncertainty", B. Fischoff 1975.

\- Hindsight bias naturally results from knowing the outcome of a
situation. The mind uses outcome knowledge to judge or learn from the
past - to make sense of it. People are generally unaware of the effect
outcome knowledge has on their conclusions about predictability. Even if
they are made aware of hindsight bias and attempt to reduce it, it
cannot be eliminated, because the mind cannot ignore the truth of a
known outcome when trying to judge an act, omission or decision in real
time. Hindsight bias creates the illusion that the outcome was
predictable. Worse, it creates in the post-event judge the illusion that
he surely would have predicted it. See e.g., "Hindsight = Foresight: The
effect of outcome knowledge on judgment under uncertainty", B. Fischoff
1975 (Links to Abstract and full article):

[[http://qhc.bmjjournals.com/cgi/content/full/12/4/304]{.underline}](http://qhc.bmjjournals.com/cgi/content/full/12/4/304)

[[http://qhc.bmjjournals.com/cgi/reprint/12/4/304]{.underline}](http://qhc.bmjjournals.com/cgi/reprint/12/4/304)

-Hindsight bias has significant implications in determining liability,
fault, or blame, bordering on creation of strict liability for an actor
implicated in a bad outcome, judged by a person with knowledge of that
outcome. Knowledge of subsequent remedial measures can also increase
hindsight bias. Hindsight bias has been found to increase in some group
settings. Most strategies to de-bias people, including judges and
juries, are ineffective, but suggestions are made in this article
concerning litigation techniques. Fully informing the decision makers of
the cause and effects of hindsight bias may help reduce its effects, but
tests continue to demonstrate that hindsight bias cannot be eliminated
even by those who understand it. See e.g., "Hindsight Bias and the
Subsequent Remedial Measures Rule: Fixing the Feasibility Exception", K.
Eberwine 2005:

http://law.case.edu/student_life/journals/law_review/55-3/eberwine.pdf

-While Eberwine's article suggests that the use of **"counterfactuals"**
(if only X, there would have been a different outcome) may help reduce
hindsight bias, studies indicate that counterfactuals actually increase
hindsight bias. Counterfactuals ask the mind to disregard knowledge of
what actually happened, the truth, and to put in place a falsehood,
which simply cannot be done for the same reason that hindsight bias
exists in the first place. However, the use of "**semifactuals**", (even
if X, the outcome would have been the same) does not increase hindsight
bias, because it does not ask the mind to replace the true outcome with
a false outcome - the known outcome stays in place. The issue is
particularly important in efforts to find the cause(s) of a bad outcome.
Use of semifactuals does not eliminate hindsight bias, it just does not
make it worse. See e.g., "Counterfactuals, Causal Attributions, and the
Hindsight Bias: A Conceptual Integration", N. Roese and J. Olson, 1996:

[[http://www.psych.uiuc.edu/\~roese/Roese%20&%20Olson%20(1996).pdf]{.underline}](http://www.psych.uiuc.edu/~roese/Roese & Olson (1996).pdf)

-Attempts to debias hindsight with counterfactuals - forcing people to
think about alternative outcomes, can backfire. One study found that
people can be asked to consider one or two simple alternative outcomes
(counterfactuals) without increasing hindsight bias, but when people
were asked to consider ten alternatives which could have changed the
outcome, hindsight bias was increased. In other words, the harder you
try to debias hindsight the worse you can make it. "When Debiasing
backfires: Accessible Content and Accessibility Experiences in Debiasing
Hindsight", L. Sanna, et al. 2002:

http://www2.gsb.columbia.edu/divisions/management/seminars/papers/schwarz.pdf

-One strategy when asking people to judge past events, is to use the
semifactual "even if" rather than the counterfactual "if only", since
the semifactual method allows the "judge" to consider the antecedent
act, omission or decision without trying to force the mind to accept an
outcome other than the known true outcome. These concepts have the same
impact on self-blame as they do on determinations of fault or liability.
See e.g., "Semifactual "even if" Thinking", R. McCloy and R. Byrne,
2002:

http://www.tcd.ie/Psychology/Ruth_Byrne/pdf/mccloyandbyrne2002.pdf

-Similarly, research found that when people are shown computer animated
re-enactments, hindsight bias is increased rather than reduced.
"Computer animations used in court colored by bias, researchers say", J.
Barlow, 2006:

[[http://www.news.uiuc.edu/NEWS/06/0410animation.html]{.underline}](http://www.news.uiuc.edu/NEWS/06/0410animation.html)

-Testing on 167 U.S. Magistrate Judges confirmed the profound impact
that hindsight bias (and several other biases) has on the quality of
decision making. "Inside the Judicial Mind", Chris Guthrie, Jeffrey J
Rachlinski, Andrew J. Wistrich, 86 Cornell L.Rev. 777 (2001).

-Another law review article discusses the unavoidable inaccuracies of
decision making, fairness issues, and other aspects of hindsight bias.
"A Positive Psychological Theory of Judging in Hindsight", J.
Rachlinski, 65 U.Chi.L.Rev. 571(1998).

Page 575:

"This Article develops the thesis that the law reflects an understanding
of the hindsight bias. In Part I, it describes the psychological
research on judging in hindsight, with special attention to the sources
and explanations for the hindsight bias. Part II discusses the
application of the hindsight bias to judgments of liability and fault in
the legal system and concludes that the bias has a big impact on
judgments of liability. Part III analyzes the consequences of this
phenomenon for the legal system. Part IV \*576 documents the adaptations
that the legal system has developed to accommodate the hindsight bias,
and Part V states conclusions."

-A search of all Federal and state cases for "hindsight bias" yielded
only one opinion in which the Court allowed an expert in psychology to
explain hindsight bias to the jury in an effort to de-bias the trial. It
was a medical malpractice case - failure to see a small mass on a lung
x-ray. The radiologist still lost. No case was found in which such
testimony was disallowed. *Gehlen v. Snohomish County Public Hospital
District No. 1,* 2001 WL 815005 (Wash.App.Div. 1)(unpublished opinion).
In the *Gehlen* case there was also expert testimony about the
likelihood of negligence as the cause of not seeing the small mass:

> "Dr. Paul Berger, a radiologist who practices in Coeur d\'Alene,
> testified about \'perceptual error.\' He described it as looking at an
> object and not seeing it or not seeing a characteristic or aspect of
> it. In his opinion, all radiologists make perceptual errors, and it
> would be impossible for a reasonably prudent radiologist to interpret
> x-rays on a daily basis without making perceptual errors. He stated
> that in 80 percent of cases in which a radiologist makes a significant
> diagnostic error, the reason for the diagnostic error is a perceptual
> error. Dr. Charles White, a chest radiologist from Baltimore, also
> testified about perceptual errors in radiology. In his opinion, no
> radiologist is 100 percent correct 100 percent of the time, and
> missing something on an x-ray is not necessarily negligence or
> carelessness."

The statistical evidence would favor a conclusion that missing the small
mass was, more likely than not, non-negligent. But Plaintiff's expert
testified that in his (hindsight) opinion, the radiologist should have
seen it.

An article published after the *Gehlen* case demonstrates the existence
of "**visual hindsight bias**" using radiology data as an example in
which 82% of missed tumors were "visual in hindsight" by radiologists
with outcome knowledge:

"THE "SAW-IT-ALL-ALONG" EFFECT: DEMONSTRATIONS OF VISUAL HINDSIGHT
BIAS", Erin M. Harley, University of California, Los Angeles

Keri A. Carlsen and Geoffrey R. Loftus, University of Washington,

Journal of Experimental Psychology: Learning, Memory, and Cognition

2004, Vol. 30, No. 5, 960--968

There are many other cognitive biases which affect our judgment as well,
which are helpful to understand in order to improve the quality of our
decisions and choices we make based on those decisions. "Inside the
Judicial Mind" supra. addresses **Anchoring, Framing, Hindsight Bias,
Representativeness Heuristic and Egocentric Biases**:

**[Anchoring]{.underline}** is a bias in which the mind in search of a
value seizes upon whatever random value is available to use as a
reference point. As explained in "Inside the Judicial Mind":

> "Anchors induce people to consider seriously the possibility that the
> real value is similar to the anchor, thereby leading them to envision
> circumstances under which the anchor would be correct. Even when
> people conclude that an anchor provides no useful information,
> mentally testing the validity of the anchor causes people to adjust
> their estimates upward or downward toward that anchor. As a
> consequence, even extreme, wholly absurd anchors can affect judgment.
> . . . people evaluating hypothetical settlement offers were more
> likely to accept a \$12,000 final settlement offer when it followed a
> \$2,000 opening offer than when it followed a \$10,000 opening offer.
> . . .In criminal cases, the influence on judges of biased or
> misleading anchors, such as prosecutor or defense attorney sentencing
> recommendations, can produce biased criminal sentences."

(footnote omitted)

More on [anchoring]{.underline}, from "Inside the judicial Mind":

> "In five separate studies, researchers have found that plaintiffs\'
> lawyers\' damage requests influenced mock jurors\' assessments of the
> appropriate amount of damages to award in civil suits. \[FN61\] In one
> study, for instance, mock jurors awarded slightly more than \$90,000
> when the plaintiff\'s lawyer requested \$100,000 in damages; but when
> the plaintiff\'s lawyer requested \$500,000 in damages in the very
> same case, mock jurors awarded nearly \$300,000. \[FN62\] Even silly
> and outrageous damage requests can influence juror decision making.
> For example, mock jurors in another study awarded the plaintiff
> substantially more in damages when the plaintiff\'s lawyer requested
> an outlandish \$1 billion than when the plaintiff\'s lawyer requested
> a more plausible amount. \[FN63\] The moral of these anchoring studies
> seems to be, \"Ask and ye shall receive.\" In each, \"when more money
> was requested for damages by the plaintiff\'s attorney, the jurors
> awarded more.\" \[FN64\]"

**[Framing]{.underline}**, also from "Inside the Judicial Mind"
(footnotes omitted):

"When people confront risky decisions\--such as deciding whether to
settle a case or to proceed to trial\--they categorize their decision
options as potential gains or losses from a salient reference point such
as the status quo. This categorization, or \"framing,\" of decision
options influences the way people evaluate options and affects their
willingness to incur risk. People tend to make risk-averse decisions
when choosing between options that appear to represent gains and
risk-seeking decisions when choosing between options that appear to
represent losses. For example, most people prefer a certain \$100 gain
to a 50% chance of winning \$200 but prefer a 50% chance of losing \$200
to a certain \$100 loss. From a rational-choice perspective, people\'s
risk preferences should depend upon their wealth relative to the size of
the stakes involved. In practice, however, people tend not to make such
normatively appropriate calculations. Instead, people make choices
designed to maintain or slightly improve the status quo, which
translates into risk-averse choices for most gains and risk-seeking
choices for most losses. "

The **Representativeness Heuristic** (a heuristic is a rule of thumb or
cognitive shortcut). From "Inside the Judicial Mind":

"When people make categorical judgments (e.g., assessing the likelihood
that a criminal defendant is guilty), they tend to base their judgments
on the extent to which the evidence being analyzed (e.g., the
defendant\'s demeanor) is representative of the category. \[FN130\] When
the evidence appears representative of, or similar to, the category
(e.g., defendant is nervous and shifty), people judge the likelihood
that the evidence is a product of that category as high (i.e., evidence
of guilt). When the evidence being analyzed does not resemble the
category (e.g., defendant appears at ease), people judge the likelihood
that the evidence is a product of that category as low (i.e., evidence
of innocence). Psychologists refer to this phenomenon as the
\"representativeness heuristic.\"

"Although the representativeness heuristic is useful, it can lead people
to discount relevant statistical information. In particular, people
undervalue the importance of the frequency with which the underlying
category occurs\--this is known as the \"**base-rate\" statistic."**

This heuristic impacts decision making by over-reliance on
representativeness evidence and under-reliance on hard evidence such as
statistical evidence. This can lead to the "**inverse fallacy**", which
". . . refers to the tendency to treat the probability of a hypothesis
given the evidence (for example, the probability that a defendant was
negligent given that the plaintiff was injured) as the same as, or close
to, the probability of the evidence given the hypothesis (for example,
the probability that the plaintiff would be injured if the defendant
were negligent)." (Quoted from "Inside the Judicial Mind").

**Test Results** from "Inside the Judicial Mind":

"To test whether judges would commit the inverse fallacy, we gave the
judges in our study a *res ipsa loquitur* problem \[under this
discredited legal doctrine a jury can infer negligence from an adverse
event that is of the kind which ordinarily does not result in injury in
the absence of negligence\]. In an item labeled \"Evaluation of
Probative Value of Evidence in a Tort Case,\" we presented all of the
judges with a paragraph-long description of a case based loosely on the
classic English case, Byrne v. Boadle: \[FN145\]"

-For this paper I have converted the warehouse accident test to a
wildland fire burn-over:

The firefighter was working near an escape route when the fire blew up
and he was cut off and burned over, resulting in his death. The Incident
Commander is not sure how the burn-over happened, but he agrees that
either the escape route was negligently selected or the fire run was
extraordinarily fast-moving. Government safety inspectors conducted an
investigation and determined that in firefighting (1) when escape routes
are negligently selected, there is a 90% chance that they will fail to
provide adequate escape time (2) when escape routes are safely selected,
they fail to provide a safe route only 1% of the time; (3) firefighters
negligently select escape routes only 1 in 1,000
times.![](media/image1.wmf){width="3.9756944444444446in"
height="2.795138888888889in"}

Given these facts, how likely is it that the burn-over was due to the
negligent selection of the escape route?

Chose from the following (circle your answer): 0-25%, 26-50%, 51-75%,
76-100%.

When presented with a problem like this one, most people commit the
**inverse fallacy** and assume the likelihood that the escape route was
negligently selected is 90%, or at least a high percentage. \[FN146\] .
. . In fact, however, the actual probability that the escape route was
negligently selected is only 8.3%.[^2]

Explanation from "Inside the Judicial Mind":

"Of the 159 judges who responded to the question, \[FN148\] 40.9%
selected the right answer by choosing 0-25%; 8.8% indicated 26-50%;
10.1% indicated 51- 75%; and 40.3% indicated 76-100%. Overall, the
judges did well; more than 40% of them got the correct answer to a
difficult question in a short period of time. Those judges who did not
get the correct answer, however, exhibited a significant tendency to
choose the highest range. \[FN149\] Although we did not inquire into the
reasoning process that led these judges to their answers, the number of
judges who chose the highest range suggests that many committed the
inverse fallacy. In fact, roughly as many judges gave the right answer
as gave the answer suggested by the inverse fallacy."

At the suggestion of Jeffrey Rachlinsky, Professor of Law, I ran the
same test using civil and criminal lawyers at the U.S. Attorney's
Office, and came up with the same results.

The Representativeness Heuristic - Inverse Fallacy also creates
erroneous legal doctrine. The **Res Ipsa Loquiter** doctrine provides
that a jury can infer negligence from an adverse event that is of the
kind which ordinarily does not result in injury in the absence of
negligence. The doctrine is erroneous and has been rejected in recent
Restatements of the law.

The **Point Fire fatality litigation** may provide some lessons
concerning the biases and thought processes addressed in this paper.
This flat-ground sage brush fire had been fully contained, and other
engines were moved to the northern perimeter to stop escape from the
expected winds. Pumper 620 broke down as they were leaving to re-fill
and stage on the road.

**EXCERPT OF DISTRICT COURT FINDINGS OF LIABILITY FOR THE POINT FIRE:**

**8.** The fire was caused by dry-lightning, and the National Weather
Service in Boise

had forecast\--both the day before and the morning of July 28,
1995\--that \"gusty

erratic winds to 55 MPH\" could be \"near any thunderstorms.\"

**46.** The procession of these four vehicles began working the northern
perimeter of the

fire.

**47.** 622 and 620 applied water through the front bumper nozzles to
the flames, and had

an immediate effect− the flames diminished in the area where the trucks
were

working, and the smoke changed color indicating that the fire was being
suppressed.

**50.** At about 8:15 p.m., the procession had nearly circumnavigated
the fire\'s entire

perimeter, and had reached the fence at the southeast corner of the
fire.

**58. When they reached the western perimeter, 622 ran out of the water.
Black radioed**

**IC Kerby on the BLM channel and asked for direction. IC Kerby told him
to refill**

**and stand by \[on the road\] because a wind storm was to pass
through.**

**60.** 622 then proceeded east through the black burned-out area until
it turned north just

before the fence in order to get to the fence break that was located at
the northeast

corner of the fire perimeter.

**61.** Just before 622 reached the fence break, 620 radioed 622 and
stated that they were

overheating and requested assistance. At this time, 620 did not appear
to be in any

distress. 622 responded that 620 should remove a screen on the front of
the vehicle.

620 responded to the effect that they heard the message and would check
the screen.

That is the last radio contact 622 had with 620.

**62.** 622 proceeded through the fence and onto Swan Falls Road,
stopping by the tender

625 to refill. Tender 625 was located at this time on Swan Falls Road at
the

southeast corner of the fire perimeter.

**63.** Black told Captain McPherson and Captain Stear that 620 was
right behind them

and coming in to refill.

**64.** When 620 did not arrive at the refilling site, captain McPherson
started walking

north on Swan Falls Road to look for the truck. When Captain McPherson
had

reached the middle of the fire's eastern perimeter, he saw 620 in the
black burned out

area heading towards him, retracing the route taken by 622. When
McPherson

first saw 620, it was about 600 feet away from McPherson, heading for
the northeast

perimeter of the fire with the apparent intent to follow 622's path
through the break

in the fence.

**65.** In a wildfire, any black burned-out area might be a safety zone
depending on how

much unburned fuel remains there; a lack of unburned fuel ensures that
the area will

not be burned over a second time. But the black's function as a safety
zone is also

dependant on two other variables: the wind, and the movement of the
firefighters. A

high wind will kick up dust and ash in the black area, obscuring
visibility. This

poses little danger if the black contains no unburned fuel and the
firefighters are

stationary. However, when the firefighters are moving in the black
towards the

perimeter of the fire, beyond which lies unburned fuel in the path of
the high winds,

the black is much less of a safety zone. The complete lack of visibility
that

accompanies high winds in the black makes the firefighters prey to
disorientation,

not a problem while they are sitting still, but a major concern when
they are moving

toward unburned fuels in the path of the winds.

**66.** McPherson felt that 620 was just minutes from reaching the fence
break, and the

truck was close enough that McPherson could see passenger Oliver "real
well."

**67.** McPherson started walking toward the fence break with the idea
that he would meet

up with 620 there.

**68.** At this point the winds increased dramatically. Visibility in
the black burnt-out area

was reduced to almost nothing as the winds kicked up the dust and ash.

**69.** 620, now close to the fence, turned north attempting to find the
fence break.

McPherson could hear 620 bouncing and moving north at a high rate of
speed.

**70.** Due to a combination of obscured visibility, disorientation, and
panic, 620 overshot

the fence break and drove into unburned cheat grass and sagebrush due
north of the

fire's northern perimeter.

**71.** The winds revived the fire, pushing it northward at a furious
rate.

**72.** 620 was still moving northward at a fast rate, but it was now
trying to outrun the fire

that was close behind it. After driving about 1,750 feet from the fire's
northern

perimeter, 620 stalled in the middle of unburned cheat grass and
sagebrush.

**73.** Bill Buttram got on the radio and relayed a frantic message to
Captain McPherson

over the Kuna channel: "We're on the north line, Doyle, we got fire
coming hard,

this thing has died."

**74.** Captain McPherson responded inaudibly, and Buttram said "it's
not going to let us

out of here."

**75.** Chief Cromwell then came on the radio and asked 620 to identify
its problem.

**76.** Buttram responded that "We're surrounded by fire." When asked to
repeat his

message, Buttram stated "The truck has been overtaken by fire."

**77.** That was the last radio communication anyone received from 620.

**78.** Shortly thereafter, 620 was overtaken by the fire.

![](media/image2.wmf){width="5.979861111111111in"
height="7.604861111111111in"}

**DISTRICT COURT'S DESCRIPTION OF FORESEEABILITY AND WARNING TO AVOID
HINDSIGHT BIAS:**

**91. A danger is reasonable \[sic.\] foreseeable if it \"is apparent,
or should be apparent, to one in the position of the actor. The actor\'s
conduct must be judged in the light of the**

**possibilities apparent to him at the time, and not by looking backward
\'with the**

**wisdom born of the event.\'\" W. Page Keeton et al., *Prosser and
Keeton on the Law***

***of Torts* § 31, at 170 (5th ed. 1984).**

**DISTRICT COURT'S CONCLUSIONS ON FORESEEABILITY, WITH KNOWLEDGE OF THE
OUTCOME:**

**113**. The BLM IC breached his duty to warn Buttram and Oliver after
the red flag\
warning to stay away from the fire\'s northern perimeter because high
winds would\
drive the fire in that direction. Instead of so warning 620, the BLM IC
instructed\
them to refill. He knew that to refill, 620 would most likely drive
toward the fence\
break at the northeast corner of the fire. In other words, the BLM IC
directed 620\
toward the northern perimeter of the fire at a time when high winds were
forecast to\
drive the fire in that very direction. As the Court discussed previously
in paragraph\
65, the black\'s function as a safety zone is partly dependent on
whether the\
firefighters are moving through the black towards the unburned fuels
that are in the\
path of oncoming winds. The BLM contends that IC Kerby gave 620 \"the
safe\
assignment of going to the road and staying there. \" See BLM\'s
Post-Trial Brief at\
11. The BLM asserts that \"it does not make sense that \[Buttram\] would
drive wildly\
around if he could not see at all.\" Id. From this, the BLM concludes
there is no\
proof that the \"BLM proximately caused Buttram to drive out of the safe
zone and\
into harm\'s way.\" Id. The Court disagrees. At the time IC Kerby gave
his refill\
order, it was foreseeable that the high winds could kick up the dust and
ash in the\
black and completely obscure Buttram\'s and Oliver\'s visibility. It was
also\
foreseeable that Buttram and Oliver would be very near the fire\'s
northern perimeter\
\-- and moving toward that perimeter\--at the time the high winds were
due to come\
through the area. Finally, it was foreseeable that the lack of
visibility could cause\
panic and disorientation that would expose the firefighters to great
risk because of\
their close proximity to the dangerous northern perimeter of the fire.
By instructing\
620 to refill, the BLM IC placed Buttram and Oliver in a foreseeably
dangerous\
position, and thereby breached his duty to provide for their safety.

**133.** The BLM knew a great deal about the fire but nothing about the
experience and

knowledge of Buttram and Oliver.

**134.** The Kuna RFD knew a great deal about Buttram and Oliver but
nothing about the

fire.

**135.** If either had shared its knowledge, this tragedy could have
been avoided.

The Court's conclusions about foreseeability, causation, responsibility
and blame do not attribute causation to mechanical failure or inadequate
maintenance - 620 had earlier break downs that season. The Court's
conclusions seem like a lot to foresee from an order to re-fill and
stage on the road because high winds were forecast to arrive.

**Egocentric Biases**

Number range CHAPTERDr. Mary Omodei and others at the Complex Decision
Research Group, LaTrobe University, Melbourne, Australia, developed a
protocol for **wildland fire accident investigations** to get to the
bottom of **human factors and decision making**. They are collecting the
data. They note how the hindsight bias impedes collecting the
information needed to understand the role of human factors. They also
note how the self-affirming biases or "**self-protective
justifications**" are a normal part of human cognition and should be
expected to influence witness descriptions of what happened. She
concludes that current accident investigation processes do not uncover
human factors and are, in fact, detrimental to getting at that critical
information. She suggests strategies, including separate types of
investigations which include privileged conversations. See
"**Identifying Why Even Well-Trained Firefighters Make Unsafe Decisions:
A Human Factors Interview Protocol**", In Butler, B.W. and Alexander,
M.E. Eds. 2005. Eighth International Wildland Firefighter Safety
Summit-Human Factors-10 Years Later. Dr. Omodei has commented to me that
"The threat of litigation and/or being called before an enquiry, either
judicial or agency mandated, is in my view the single biggest impediment
to accurate investigation and/or research into the human factors
underlying "problematic" decision making in incident management (at all
levels)."

The following article is particularly useful reading for **prosecutors
and investigators** because it explains how the mind takes an initial
impression and skews the subsequent processing of information during the
investigation and development of the case. The article includes some
de-biasing strategies such as ensuring that all evidence collected by
investigators is turned over to prosecutors regardless of its perceived
value, fresh-look or "devil's advocacy" exercises, neutral committees to
assist with decision-making, etc.

"IMPROVING PROSECUTORIAL DECISION MAKING: SOME LESSONS OF COGNITIVE
SCIENCE", 47 Wm. & Mary L. Rev. 1587, A. Burke, Associate Professor,
Hofstra University School of Law, (2006). Excerpt:

"This Article explores four related but separate aspects of cognitive
bias that can contribute to imperfect theory formation and maintenance:
**confirmation bias, selective information processing, belief
perseverance, and the avoidance of cognitive dissonance**. Confirmation
bias is the tendency to seek to confirm, rather than disconfirm, any
hypothesis under study. \[FN34\] Selective information processing causes
people to overvalue information that is consistent with their
preexisting theories and to undervalue information that challenges those
theories. \[FN35\] Belief perseverance refers to the human tendency to
continue to adhere to a theory, even after the evidence underlying the
theory is disproved. \[FN36\] Finally, the desire to avoid cognitive
dissonance can cause people to adjust their beliefs to maintain existing
self-perceptions. \[FN37\] This Part summarizes the empirical literature
regarding each of these cognitive phenomena."

Page 1602:

"No reason exists to believe that lawyers are immune from the documented
bounds of rationality, and yet the literature on prosecutorial decision
making continues to describe prosecutors as rational, wealth-maximizing
actors who would make better, more just decisions if they only had
better, more just values. [\[FN95\]]{.underline} Through the lens of the
cognitive phenomena summarized in Part I, a more \*1603 complicated
story is evident. That prosecutors should be motivated by justice, not
conviction rates, should go without saying. The harder question to
answer is whether good motives, both individually and institutionally,
are enough. The implications of the cognitive literature suggest not."

Page 1616:

"Another possible method of improving prosecutorial decision making is
to train prosecutors and future prosecutors about the sources of
cognitive bias and the potential effects of cognitive bias upon rational
decision making."

In connection with civil case analysis, it should be recognized that
Number range CHAPTERthe "**self-serving bias**" causes both civil
litigants and their attorneys to overestimate the fair value of their
claim, and to over-predict the likelihood of a favorable verdict, on
both liability and damages, compared to a control group which does not
know which side they are on when they analyze the same claim.

**List of Cognitive Biases**

Here is a link to a long list of cognitive biases which is interesting
reading:

[[http://en.wikipedia.org/wiki/List_of_cognitive_biases]{.underline}](http://en.wikipedia.org/wiki/List_of_cognitive_biases)

The list includes the **Planning Fallacy** in which we underestimate
task-completion times, **Neglect of Probability** in which we disregard
probability when making a decision under uncertainty and **Zero-Risk
Bias** -- the preference for reducing small risk to zero over a greater
reduction in a larger risk, and the **Valence Effect,** commonly known
as wishful thinking, in which we tend to overestimate the likelihood
that good things will happen to us rather than bad things.

One can posit that the **Planning Fallacy** had a causal role in the
Cramer Fire in which time to complete construction of the helispot was
repeatedly under-estimated. Greater use of objective criteria rather
than subjective estimates might help. For example, in the Dude Fire when
Hotshot Superintendent Paul Gleason placed himself between the rescuers
and the fire so he could push them to move out if needed, he did not
just estimate the rate of spread of the blowup below them, he used his
watch, measured and calculated the rate of spread, giving him a more
objective window of opportunity for decision-making. Several fatal down
hill line construction fires come to mind - how long will it really take
to connect to that safer anchor point? On the other hand, if initial
attack fails due to not doing it, will the consequential risks created
by a larger fire increase? Here is an interesting note on deciding to
accept risk by taking action, versus accepting risk by deciding not to
act:

"Most real-life decisions are vague and ill-defined (Fischhoff, 1996).
Decision makers

may, for example, hold multiple and ambiguous goals, making it difficult
to judge what

would be the best decision. Yet, process-oriented students of decision
making would argue that half of the solution involves appropriate
structuring of the decision problem.

Baron (1994) notes that people have difficulty in following this
process-oriented

approach. Normatively, the decision structure should incorporate
potential outcomes, because only these affect the fulfillment of the
decision maker's goals---a requirement known as "consequentialism". Even
if it does not affect the consequences of a decision (in terms of their
goals), people are sensitive to the manner in which an outcome has been
obtained. Baron and his colleagues (e.g., Spranca, Minsk & Baron, 1991;
Ritov & Baron, 1992) have shown that decisions with identical outcomes
are judged as worse when they result from acts of commission than acts
of omission. For example, most people are reluctant to vaccinate
children against a potentially lethal flu when side effects of the
vaccine can cause death. Faced with a flu epidemic that is expected to
kill 10 out of

10,000 children, most people are not willing to accept a 9 out of 10,000
risk of death from vaccination, and the median acceptable maximum risk
is 5 in 10,000. Apparently, killing a child with a vaccination (that is,
an act of commission) is perceived as worse than causing its death by
failing to vaccinate (that is, an act of omission) (Ritov & Baron,
1990). This pattern of results poses a violation of consequentialism,
unless feelings of guilt (resulting from commission but not from
omission) may be included in the definition of "consequence". Indeed,
some argue that strong emotional responses should be considered as part
of the decision outcome (see commentaries included in Baron, 1994)." "On
the Assessment of Decision Quality: Considerations Regarding Utility,
Conflict and Accountability", Gideon Keren and Wandi Bruine de Bruin,
from *Thinking: Psychological Perspectives on Reasoning, Judgment and
Decision Making*, Eds. Harding, D. and Macchi, L., Wiley 2003.

**[III. Outcome Knowledge and Judging Quality of
Decisions]{.underline}**

After the hindsight bias contaminates our opinion that the outcome was
foreseeable, and we would surely have foreseen it, the **Outcome Bias**
contaminates our ability to judge the quality of the decision and the
character of the person who made it. The outcome bias influences people
to judge a decision more harshly if they are aware of a bad outcome,
than they judge the same decision if they are unaware of the bad
outcome. The outcome is irrelevant to judging the quality of a decision
under uncertainty. Nonetheless, even in tests segregating out subjects
who persisted in the belief that outcome is always relevant to the
quality of the decision and the character of the person who made it,
knowledge of bad outcomes significantly increased condemnation of the
decision and the decision maker. See e.g., "Outcome bias in decision
evaluation.", Jonathan Baron and John c. Hershey, University of
Pennsylvania. <http://www.sas.upenn.edu/~baron/judg.html>

The hindsight bias and the outcome bias interfere with identifying the
human factors contributing to an accident, cognitive and otherwise, in
accident investigation and prevention. See e.g. Number range
CHAPTER"Perspectives on Human Error. Hindsight Biases and Local
Rationality", Woods, D.D. and Cook, R.I., In F. Durso (Ed.), Handbook of
applied cognitive psychology (pp. 141-191). NY Wiley

The outcome bias is also affected by group identity: in-group judges can
be expected to judge decision quality and the quality of the decision
maker less harshly than out-group judges. See "The One-of-Us Effect in
Decision Evaluation", Raanan Lipshitz, Ziv Giland, and Ramzi Suleiman,
University of Hofstra, 2000. Further comment about outcome bias in this
paper:

"Judgment by outcome has received particular attention because of its
important normative implications:

Judging single decisions on the basis of their outcomes is inappropriate
for three reasons. The first reason is that the outcome of a decision,
by itself, cannot be used to improve a decision unless the decision
maker is clairvoyant (Baron & Hershey, 1988). A second, similar argument
can be made from the standpoint of fairness: it is [unfair]{.underline}
to use information not available to the decision maker to judge the
quality of his decision... \[Finally\], under risk or uncertainty...
judgment-by-outcomes is [logically unwarranted]{.underline} because it
involves backward reasoning equivalent to affirming the consequent [(a
®→ b, therefore b® → a)]{.underline}, (Lipshitz, 1995, p. 123)."

\* \* \*

"The one-of-us effect was demonstrated in a classical study by Hastorf
and Cantril (1954). Following a Dartmouth-Princeton football game, two
samples of students from the two colleges were shown the same film of
the game. Results from a questionnaire administered after the film
revealed basic disagreements between the two samples over what actually
took place during the game. In particular, each sample claimed that the
team representing its own school played in a more sportsmanlike fashion
than the opposing team. The hypothesis that judgment-by-outcomes is
moderated by the one-of-us effect is also supported by Tajfel and
Turner's (1985) social identity theory, Pettigrew's (1979) theory of the
ultimate attribution error, and findings from research on procedural
justice."

\* \* \*

"The influence of outcome information attracted the attention of
researchers partly owing to its normative implications. Its attenuating
effects reflect violations of Decision Theory (i.e., "outcome bias,"
Baron & Hershey, 1988), hinder drawing of valid lessons from experience
(Fischhoff, 1975), handicap just distribution of rewards and
punishments, and weaken the authority of valid professional doctrines
and standard operating procedures (Lipshitz & Barak, 1995).
Unfortunately, the moderating influence of the one-of-us effect on
outcome bias probably worsens rather than helps its dysfunctional
consequences."

\* \* \*

"Our findings show that cognitive models cannot be applied effectively
in order to predict, for example, the effects of specific outcome
information on decision evaluation in concrete real-world situations,
without an understanding of the socio-cultural context to which the
models are applied."

Another problem with making decisions about decisions is that there are
at least three distinctly different models available for judging
decisions. From "On the Assessment of Decision Quality: Considerations
Regarding Utility, Conflict and Accountability", Gideon Keren and Wandi
Bruine de Bruin, from *Thinking: Psychological Perspectives on
Reasoning, Judgment and Decision Making*, Eds. Harding, D and Macchi,
L., Wiley 2003:

"Referring to an operation as successful *after* the patient has died
remains unsatisfactory for most laypersons."

\* \* \*

". . . the majority of researchers emphasize that the process, rather
than the outcome, should be the object of evaluation. Their theories are
considered "normative", prescribing

how people should make, and judge, decisions. "Descriptive" research,
which describes how decisions are actually made, shows that people focus
on decision outcomes. Normative and descriptive considerations are
deeply interrelated, and there is an ongoing interplay between the two
perspectives (Coombs, Dawes & Tversky, 1970; Keren, 1996). To assess
decision quality, it is important to examine both normative and
descriptive facets of the relevant issues. \* \* \* An alternative
option, and the one that we recommend, is what Keeney and Raiffa (1976)
termed the "prescriptive" approach. It offers guidelines to decision
makers who search for optimization, yet takes into account their limited
capacity for memory and information processing that makes them
vulnerable to human error, reasoning biases and swaying emotions."

\* \* \*

"Which specific considerations prevail in the process of assessing
decision quality depends, to a large extent, on the decision-making
model that is adopted by the judge."

\* \* \*

"The first is the so-called gambling paradigm, which has been the
dominating metatheory in the field of decision making since its
inception. Its main underlying assumption is that every decision problem
can be translated into a choice between gambles, relying heavily on
utility theory (e.g., Luce & Raiffa, 1957; Keeney & Raiffa, 1976;
Schoemaker, 1982). The second approach is the conflict model, which
considers a decision to be the resolution of an emotional conflict, in
which to choose one option means to forego another (e.g., Janis & Mann,
1977; Coombs, 1987). The third and last approach is the accountability
model (Tetlock, 1991; Lerner&Tetlock, 1999), which asserts that the
decision maker's major goal is to defend decisions convincingly---if
held accountable. Although the latter two models were not intended to be
normative, each offers implicit guidelines to judge decision quality,
assigning different (relative) weights to the outcome- and
process-oriented approach."

"This chapter emphasizes that the judge of the decision and the decision
maker may not use the same framework to evaluate the decision -- and,
hence, may disagree. Evaluations may even vary between judges who
endorse different perspectives and different models. For example,
patients and colleagues may take a different approach to judging the
quality of a physician's decision. While the colleagues may emphasize
the decision process, the patients will probably focus on the outcome.
As suggested before, the latter will be more likely to consider a
decision to operate wrongly if it leads to a death. Whatever the
perspective taken, a perfectly neutral judge rarely, if ever, exists."

\* \* \*

"It should be emphasized that the three frameworks discussed in this
chapter do not

exhaust all possible views, and other paradigms or subtypes of paradigms
are conceivable. For instance, the decision maker or the judge may adopt
a "rule-based" view, according to which a decision is good when it
conforms to certain "acceptable" rules of conduct. Another example
refers to, the distinction between adapting a promotion or a prevention
outlook (Higgins, 1998) may be a major source of discrepancies in
judging decision quality."

"The question of which model the judge (or the decision maker) should
adopt is a question that obviously cannot be answered unequivocally. The
major problem underlying the judgment of decision quality is largely a
question of compatibility between the decision maker and the judge (who
performs a second-order decision). There are two major facets for which
the compatibility between the decision maker and the judge can be
assessed. First, there is the question whether both employ similar
frameworks in structuring the decision problem. For instance,
discrepancies between the decision maker and the judge may arise if the
former was motivated by emotional considerations associated with
conflict, whereas the latter, in making her judgment, was solely guided
by utility theory. Even a judge who uses the same model as the decision
maker may not entirely understand the decision maker's position at the
time the decision was made. It is difficult, if not impossible, to
ignore outcome knowledge when attempting to assess in retrospect the
decision maker's situation. Similarly, differences between the decision
maker and the judge may come about from different interpretations of the
decision problem at hand. Indeed, framing effects probably constitute
the most ubiquitous phenomenon of decision making. Thus, gaps may simply
result from different framing of the decision problem by the decision
maker and the judge. Besides such "perceptual" discrepancies that can
lead to diverging interpretations, differences may also stem from
differences in the tacit assumptions underlying the conduct of
communication, as has been convincingly shown by Schwarz (1998)."

\* \* \*

"The above two issues lead to a fundamental question: what exactly is
meant by a good

decision? Students of decision making would correctly claim that
examining decision quality should be restricted to the first facet, the
way the problem has been structured and the extent to which this
structuring lends itself to acceptable solutions given the decision
maker's goals. In practice, however, decisions and judgments are often
strongly influenced by the underlying basic value system. Hence,
judgments along the two facets mentioned above may often be confounded.
More specifically, the perspective adopted by judges to assess the first
facet may be strongly biased by their stand on the second facet."

\* \* \*

"Our inquiry leads us to conclude that there are no unequivocal
standards or guidelines for judging decision quality."

\* \* \*

"At the end of the day, it is probably the case that, at least in
practice, the justification of a decision or its judgment is mainly
driven by the strength of the supporting arguments (Shafir et al.,
1993). Such a conclusion is probably most compatible with the
accountability model of decision making. The fact that those who assess
and judge decision makers (for instance, committees assessing decisions
made by societal decision makers) frequently adopt one or the other
version of an accountability model should not be taken as normative
evidence for the superiority of the model."

**[IV. Coherence Based Reasoning.]{.underline}**

Unfortunately, the cognitive process can lead to "**coherence shifts**"
which can result in decisions which are skewed to seem "coherent" and
which are made with overconfidence that the decision is right. Evidence
is given more weight than it deserves as it coheres with the emerging
decision. The article below demonstrates for example that when the mind
rejects weak evidence, the mind actually increases the strength of
belief in evidence previously offered, which had not previously been
believed with any strength. In other words, a defendant offering a weak
alibi may think he is defending himself, while in fact he is
contributing to his own conviction - when the weak alibi is rejected,
the mind increases its belief in the strength of opposing evidence. The
article includes actions to mitigate against coherence shifts including
asking the decision maker to "**consider the opposite**". This
mitigation also works on the self when making decisions. This has
significant implications for making firefighting decisions and for
judging those decisions later. Accepting an assignment which appears in
hindsight to have been too dangerous, might be an example of a "can do"
coherence shift. The Navy's **TADMUS** program (articles cited in
Summary) greatly reduced coherence shifts, consistent with the
scientific literature.

"A THIRD VIEW OF THE BLACK BOX: COGNITIVE COHERENCE IN LEGAL DECISION
MAKING", 71 U.Chi.L.Rev. 511, Dan Simon (2004). Excerpts:

Page 511:

Based on a connectionist cognitive architecture, coherence-based
reasoning shows that the decision-making process progresses
**bidirectionally**: premises and facts both determine conclusions and
are affected by them in return. A natural result of this cognitive
process is a skewing of the premises and facts toward inflated support
for the chosen decision. The Article applies this research to four
important aspects of the trial. It argues that the current doctrine in
these areas is based on misconceptions about human cognition, which lead
to systematic legal errors. By identifying the cognitive phenomena that
lie at the root of these failings, the research makes it possible to
devise interventions and introduce procedures that reduce the risk of
trial error.

Page 513

**Coherence-based reasoning** posits that the mind shuns cognitively
complex and difficult decision tasks by reconstructing them into easy
ones, yielding strong, confident conclusions. The research reveals an
unconscious transformation of the way decisions are mentally
represented, ultimately leading to a seemingly straightforward choice
between a compelling alternative and a weak one. To date,
experimentation has revealed eight cognitive features of coherence-based
reasoning, which harbor interesting\--and potentially
troubling\--implications for decision making in the legal domain and
beyond.

Page 516:

**Coherence-based reasoning applies to mental tasks in which the person
must make a discrete decision or judgment in the face of complexity**.
Tasks are said to be complex when their constitutive considerations are
numerous, contradictory, ambiguous, and incommensurate.

\* \* \*

In a nutshell, coherence-based reasoning suggests that decisions are
made effectively and comfortably when based on coherent mental models.
Loosely defined, mental models capture the decision-maker\'s perception
of the task at hand\--that is, the way the considerations of the
decision are represented in her mind. \[FN18\] A mental model of a
decision task is deemed \"coherent\" when the decision-maker perceives
the chosen alternative to be supported by strong considerations while
the considerations that support the rejected alternative are weak.

\*517 The central finding of coherence-based reasoning research is that
the cognitive system imposes coherence on complex decision tasks.
Throughout the decision-making process, the mental representation of the
considerations undergoes gradual change and ultimately shifts toward a
state of coherence with either one of the decision alternatives. Due to
these coherence shifts, at the culmination of the process, the
decision-maker\'s mental model is skewed toward conformity with the
emerging decision. As the hard case morphs into an easy one, the
decision follows easily and confidently. The fact that decisions are
ultimately based on skewed mental models and backed by high levels of
confidence facilitates the making of the decision, but at the same time
it can also harbor problematic implications.

Page 518:

The **first application** follows from the finding that coherence shifts
occur prior to the making of a decision.

\* \* \*

**Second**, coherence-based reasoning speaks to an ongoing debate as to
whether fact-finders evaluate evidence in a holistic or an atomistic
manner. This debate has particular significance with respect to the
admissibility of potentially prejudicial evidence. While the empirical
findings support the holistic account, I argue that its corollary
prescription is wrong. I also caution against a Supreme Court decision
that tends toward a relaxation of the restraints on the admissibility of
evidence. [\[FN26\]]{.underline}

\* \* \*

The **third** application concerns verdict determinations in criminal
trials. The primary finding of the research is that coherence shifts
polarize perceptions of the evidence. Jurors with a slight initial
inclination to acquit or convict are likely to amplify their perception
of the case, so that evidence that is weakly probative of guilt can be
transformed to create mental models that strongly support either
innocence or guilt. Normatively, this matters for jurors who vote to
convict, because the evidence is bolstered from overall ambiguity to a
belief beyond a reasonable doubt. Thus, coherence shifts can undermine
the role of the heightened standard of proof.

\* \* \*

The **final** application pertains to the doctrine of harmless error in
criminal trials, which has been torn between two modes of analysis: one
that focuses on assessing the impact of the error on the trial outcome,
and the other that focuses on assessing guilt as it appears from the
remaining, presumably untainted, evidence. \* \* \* In cases that
concern the erroneous admission of impermissible evidence, the appellate
judge\'s perception of the remainder of the evidence is likely to have
been affected by the impermissible evidence due to coherence effects.
Since that evidence might well be tainted, judges ought to shy away from
guilt-focused analysis.

Page 520:

**Connectionist Representations**. All mental processing starts with a
mental representation of the task. \[FN27\] Decisions are made within
certain problem spaces that contain and delimit the representation of
the task variables, that is, all the factors that are to be included in
the decision at hand. The representations of the variables are called
mental models of the decision task.

Coherence-based reasoning presumes a connectionist architecture of
mental representations, which can be likened to an intricate electrical
network. \[FN28\] Each variable is represented in the mind as a unit,
\*521 which is then connected through the network to all the other
units. \[FN29\] The initial representation of a mental task is
determined foremost by the rich and detailed store of background
knowledge about one\'s physical, social, and conceptual worlds. \[FN30\]
The mental model of a complex decision task contains a myriad of
variables that point in more than one direction and thus do not all fit
into a coherent mental model. One subset of variables (a1, a» »,.» .
.an) supports conclusion A, and the other subset (b1, b2. . .b%l» ,. .
.b»ts the opposite conclusion B.

\* \* \*

**Constraint Satisfaction Mechanisms**. In all but easy cases, neither
subset of variables dominates the other. Since each variable has some
bearing on the task, it can be said to impose a constraint on the
network. \[FN31\] Connectionist systems process mental tasks through a
nonconscious process in which the mentally represented variables
interact with one another like an electrical network. Activation spreads
throughout the mental model. \[FN32\] Each and every constraint
influences, and is influenced by, the entire network, so that every
processing cycle results in a slightly modified mental model. Over time,
unsupported variables or those suppressed by other variables degrade and
even die out, while those that are mutually supported gain strength.

\*522 The central feature of constraint satisfaction mechanisms is that
the mental model will reconfigure itself until the constraints settle at
a point of maximal coherence. Coherence, the state at which similarly
linked variables are similarly activated, occurs when the variables that
support the emerging decision are strongly endorsed and those supporting
the alternative are dismissed, rejected, or ignored. In complex
decisions, the initial mental representation of the task is naturally
incoherent. Constraint satisfaction processes force the task variables
to change toward a better fit with the gradually emerging state of
coherence. \[FN33\] This reversed induction gives coherence-based
reasoning its bidirectional character: while the strength of supporting
variables determines the conclusion, the variables themselves are
transformed by the cognitive process so as to provide considerably
stronger support for the conclusion. \[FN34\]

In sum, the ultimate state of coherence is essentially a byproduct of
the cognitive system\'s drifting toward either one of two skewed mental
models. Within each of these models, the initially complex and
incoherent mental model has been spread into two subsets, one of which
dominates the other, thereby enabling a relatively easy and confident
\*523 choice. This skewed representation reflects an artificial
polarization between the inflated representation of the variables that
support the chosen conclusion and the deflated ones that support the
rejected conclusion; it differs considerably from the way the task
variables were perceived before the decision-making process got
underway, and it differs also from the way they will be perceived some
time after the completion of the task.

Page 524:

The first finding concerns the fundamental property of coherence-based
reasoning: that in the course of making a complex decision, the mental
model of the task shifts toward coherence with the emerging decision.

Page 533:

The second feature of coherence-based reasoning concerns its
metacognitive dimension\--the degree to which people are aware of the
process, particularly of their shifting perceptions of the task
variables. We hypothesized that **coherence shifts transpire without
awareness**\--that they are mostly an automatic, rather than a
controlled and conscious, form of cognitive processing. \[FN63\]

\* \* \*

Page 534:

The next series of experiments examined the important theoretical
question of the role that coherence shifts play in the decision-making
process. It could be argued that the shifts play no active role in the
process itself, but serve only as post hoc rationalizations for
decisions driven by other factors or different mechanisms. This is the
view offered by **cognitive dissonance theory**, which posits that
attitudes and preferences change exclusively due to post-decision
regret. Only after a person commits herself to a course of action does
regret cause the arousal of dissonance, which is then reduced by
rationalizing the decision. \[FN66\] We sought to test this aspect of
coherence-based reasoning.

In a series of studies based on the Quest case, we induced participants
to delay their decisions and measured their preliminary rating of the
arguments during the period of delay. \[FN67\] **Contrary to dissonance
\*535 theory, significant coherence shifts were observed at this early
stage**. \[FN68\] This finding suggests that coherence shifts play a
functional role in the decision-making process, in that the spreading
apart of the variables creates dominance of one alternative over its
rival, thereby enabling a confident decision. Similar predecisional
shifts were observed in a study based on the Jason Wells case. \[FN69\]
In addition to the shifts that preceded the decision, coherence shifts
were also observed following the point of decision, though the
post-decisional shifts were typically marginal. \[FN70\] **Coherence
shifts, the data suggest, precede decisions**. \[FN71\]

Page 536:

**All mental processing draws closely from one\'s background
knowledge**. \[FN73\] A decision to cross a street, for example, is
contingent on one\'s experience-born knowledge about vehicles, motion,
and driver behavior. A choice to form a friendship is influenced by
one\'s knowledge of cues for trustworthiness, love, selfishness, and the
like.

Page 538:

Another feature of coherence-based reasoning that warrants attention
concerns the effect of changes in one task variable on other variables.
Connectionist theories posit that any variable can potentially influence
the entire network, including variables to which it is not directly
linked. In a study based on the Quest case, one-half of the participants
received an account in which the defendant Smith was described as a
benevolent person, and the other half received a description of a
malevolent defendant. We expected that this manipulation \*538 would
affect the verdicts so that more verdicts favoring the defendant would
be rendered in the former case than in the latter. The crucial question,
however, was whether and how this manipulation would influence the
ratings of the other variables involved in the case, none of which had
any plausible relation to Smith\'s character.

As expected, the information about Smith\'s previous conduct had a
strong influence on the distribution of the verdicts. Of the
participants who received positive information about Smith, 72 percent
rendered verdicts in his favor, whereas only 22 percent of those who
received negative information decided for him. More importantly, we
analyzed the ratings of all of the other variables in the case to see
whether they were influenced by the new information. All the variables
shifted to cohere with the chosen decision. \[FN78\] **In other words,
the manipulated piece of information had a substantial effect on
variables with which it had no plausible relationship**.

Page 539:

Another experiment based on the Jason Wells case was designed to get a
closer look at the dynamics of coherence-based reasoning by examining
**what happens when people change their minds** during the
decision-making process. In light of the finding that coherence effects
occur before decisions are made, \[FN80\] one might believe that people
should rarely change their minds in response to new information and that
when they do, they are left in a state of strong incoherence with their
previous mental models. Yet, experience suggests that, under some
conditions, people respond to new evidence that contradicts \*539 prior
leanings by changing their minds, \[FN81\] and that changing one\'s mind
does not seem to have notable influences on the resulting decisions.

\* \* \*

The results showed that a majority of participants followed their
initial leanings, but some did switch their verdicts. \[FN84\] Not
surprisingly, participants who did not switch their verdicts displayed
coherence shifts that resembled those observed in the previous delay
studies\--that is, a significant shift from the pretest to the interim
judgments, and then a further, more moderate shift at the decision
phase. \[FN85\] Of particular interest, though, were those participants
who switched their \*540 verdicts. Switchers displayed a unique pattern;
their ratings dovetailed with their new verdict preferences. Their
ratings first shifted from pretest levels to a state of coherence with
their initial leanings, but as their preferred verdicts changed, so did
their ratings of the evidence. \[FN86\] The ratings of the evidence,
then, shifted twice\--in opposite directions\--from an initial state of
complexity toward coherence with whichever verdict seemed more appealing
at that time. Interestingly, switchers were no less confident than those
who did not switch.

The findings of this tracking experiment also support the conclusion
that coherence shifts occur before participants commit themselves to
verdicts. \[FN87\] Participants showed strong coherence shifts at the
interim measurement, despite the unavailability of supposedly important
evidence, and despite being urged to withhold the decision.

In all, the experiments showing the effects of indirect influences
provide the additional methodological benefit of strengthening the basic
finding of coherence shifts. In these experiments, we showed not only
that people drift spontaneously toward polarized mental models, but that
they can be driven to do so by experimental manipulation. \[FN88\]

\* \* \*

People tend not to appreciate the incompatibility between their initial
and eventual mental models of the task. This finding supports the view
that **coherence shifts are governed by automatic cognitive mechanisms
that operate under the level of conscious awareness**. Coherence shifts
are part of the many important cognitive processes that take place in
the background of our conscious arena, without which making sense of the
world would be difficult, if at all possible. \[FN106\] This finding is
consistent with a robust body of literature demonstrating that people
have a general tendency to perceive objectivity in their reasoning
processes, \[FN107\] and to view the world through a perspective of
**\"naive realism.\"** \[FN108\] The lack of awareness gives the
decision-maker a false sense of constancy in his own perception of the
case. The ensuing decision is thus experienced as rationally warranted
by the inherent values of the variables, rather than by an inflated
perception imposed by the cognitive system. The lack of awareness then
helps the decision-maker maintain the phenomenological experience of
rationality and objectivity.

Coherence shifts, consequently, do not represent conscious, strategic,
or deceitful conduct on the part of a decision-maker; rather, they \*546
are the natural consequence of the normal mechanisms of cognitive
processing.

\* \* \*

Coherence effects are not isolated from noncognitive characteristics of
the task. Recently obtained findings strongly support the view that
**coherence-based reasoning interacts with motivations. Not only do
people\'s choices comport with their desired goals, but the mental
models of the entire task tend strongly to cohere with the respective
outcome.** In a separate set of studies we found that coherence effects
interact with preexisting attitudes to generate mental models that
cohere with those attitudes.

\* \* \*

In a number of experiments, we found that changing one aspect of the
case triggered changes throughout the mental models: not only did it
alter the verdict distribution, but it also influenced the ratings of
variables that were unrelated to the manipulated variable. Such
circuitous and indirect influences inhere in the connectionist nature of
cognition. This finding becomes particularly acute in combination with
the finding that people have low awareness of the shifts.
**Decision-makers are likely to perceive sufficient reason to base a
decision on a particular set of variables, unaware that their perception
of those variables is influenced by an extraneous\--possibly
illegitimate\--variable**.

\* \* \*

The experimental evidence illustrates that coherence can be transitory.
**Coherence shifts were found to decay one week after the task was
completed**. At least in some conditions, then, coherence can be
understood as an ad hoc state that is constructed only to solve the task
at hand. Dissipation of coherence enables decision-makers to approach
new tasks unencumbered by previous coherence shifts. In subsequent
tasks, different, perhaps opposite, pressures may activate the same
variables, causing them to take on different values.

\* \* \*

From a prescriptive point of view, an important experimental finding is
that coherence shifts can be reduced by means of a straightforward
moderating technique. \[FN111\] A moderating instruction to \"**consider
the opposite**\" **reduced coherence shifts by about one-half**, so that
participants\' reasons for deciding as they did were closer to their
initial, pre-coherence perception of the case. The fact that the
technique is simple and can be self-administered makes it a potentially
powerful and expedient means of tempering the effects of coherence-based
reasoning.

Undoubtedly, coherence effects have their limits. While the observed
coherence shifts are substantial and highly significant from a
statistical standpoint, they do not reach extreme values. \[FN112\]
**Coherence shifts are mediated by task-specific factors, most notably,
by the degree of ambiguity inherent in the task. Ambiguous variables are
more amenable to change; \[FN113\] unambiguous variables, such as
indisputable facts and paramount principles are less likely to shift.
\[FN114\] It is also likely that coherence shifts are mediated by the
idiosyncratic characteristics of the decision-maker.** \[FN115\]

\* \* \*

Coherence shifts skew the mental models and generate \*549 an inflated
sense of confidence even in close decisions, but it cannot be said that
the shifts necessarily result in objectively wrong decisions, nor that
they cause cardinal changes in the decision-maker\'s own perceptions of
the case. \[FN116\] They do, however, cause a substantial increase in
the risk of error in certain circumstances.

\* \* \*

According to **the story model**, people make sense of complicated
bodies of evidence by constructing narratives, rather than by
aggregating measures of the strength of the raw evidence. Stories are
structured around episodes held together by certain types of causal and
physical relationships that explain the actors\' conduct and mental
states. A pervasive type of narrative structure is based on schemas of
human action: initiating events cause characters to respond in a variety
of ways and to form goals that then motivate subsequent actions. The
acceptance of a story is determined by its global strength based on
**four certainty principles**\--coverage, coherence, uniqueness, and
goodness-of-fit. Of the stories constructed from the evidence presented
at trial, a juror adopts as the best explanation for the events the one
that best meets these certainty principles. \[FN167\]

\* \* \*

**Coherence research fits with the story model**. Both approaches posit
that evidentiary conclusions are not derived from mathematical
computations of the independent values of raw evidence. Inferences,
rather, are based on constructed representations of coherence, and it is
these constructed representations that ultimately determine the
verdicts. Coherence research overcomes an important limitation of the
story model. As indicated by its name, the story model proposes that the
representation of evidence bears a distinct narrative structure.
Pennington and Hastie\'s results showed that participants\' stories were
centered on a narrative that captured the intentional and causal scheme
of the defendant\'s behavior\--intentions, psychological states, \*564
goals, and motivations. \[FN176\] But, in a range of evidentiary
situations, the material facts \"may concern a situation or state of
affairs rather than a sequence of events.\" \[FN177\] A narrative of
human intentionality is of little relevance to negligence cases where
the contested issue is a failure in appreciating a risk, to
identification cases, or to cases in which the material facts concern
the physical conduct of the defendant, the quality of a product, the
extent of damages, and the like. \[FN178\]

\* \* \*

Complex decisions are solved rather by nuanced cognitive processes that
progress bidirectionally between premises and facts on the one hand, and
conclusions on the other. Ultimately, people make decisions through what
appears to be a \*584 rational-like choice in which a strong alternative
is straightforwardly preferred over its rival. However, this dominance
is the product of an unconscious cognitive process that reconstructs and
transforms difficult and complex decisions into easy ones by amplifying
one alternative and deflating the other. This transformation of the
mental model of the decision lies at the heart of the decision-making
process.

**[V. Knowledge Structures and Schemas.]{.underline}**

This is an excerpt of an article on how the mind subconsciously uses
knowledge structures and schemas when confronted with making a decision
or judgment. Clearly the vast majority of Americans are not consciously
prejudiced, and are offended when they encounter prejudice. At a
subconscious level the problem is far more complex and intractable.
Efforts to de-bias can also backfire -- for example, studies have shown
that instructing a jury to avoid racial prejudice actually increases
biased results. We commonly see knowledge structures and schemas at work
among firefighters debating issues of skill and competence based on
which category they are in (FS, BLM, NPS, CDF, etc.) or (Hotshot Sup.,
Line Officer, IC, militia, etc,). Knowledge structures and schemas will
also be at work in judging decisions in hindsight.

CATEGORICALLY BIASED: THE INFLUENCE OF KNOWLEDGE STRUCTURES ON LAW AND
LEGAL THEORY, 77 S.Cal.L.R. 1103 (2004), Ronald Chen and Jon Hanson.
Excerpts:

Page 1110:

This Article is the beginning of our attempt to understand better not
only those who respond to the laws and those who make the laws, but also
those who devise the theories on which those laws are based.
**Knowledge** \*1111 **structures**, as we shall see, influence
virtually every feature of our cognitive processes and every thought we
have\--from the trivial and humorous to the profound and horrific.
\[FN26\]

Page 1128:

**Schemas** do much more than simply constrain or bias our thinking in
humorous or harmful ways. The human system of processing information is,
in many cases, an efficient means of understanding our worlds and
ourselves. Classification of people, objects, and other stimuli is often
both indispensable and ineluctable. Like Polinsky\'s economic schema,
our day-to-day schemas can be very useful in providing us a workable
procedure for solving complex problems.

Still, as social psychologists have demonstrated, \"virtually any of the
properties of schematic functioning that are useful under some
circumstances will be liabilities under others.\" \[FN103\] The
categories and schemas that operate, usually automatically, influence
all aspects of information processing\--from what information we focus
on, to how we encode that information, to which features of that
information we later retrieve and remember, and to how we draw
inferences and solve problems based on that information. Given the
unconscious and biasing influence of our schemas, combined with the fact
that our schemas themselves will often reflect our unconscious motives,
we should be mindful, even distrustful, of our schemas and the
conclusions that they generate.

These effects, the processes that drive them, and the biases they
engender are the primary subject of this Article. A central goal is to
offer a broad understanding of how individuals utilize **categories,
schemas, and scripts** to help make sense of their worlds. In doing so,
we serve another main objective: to provide a comprehensive (yet
manageable) synthesis of a vast body of social psychology literature.
\[FN104\] This overview should transform how we make sense of our laws
and legal-theoretic world.

Page 1131:

**Categories and schemas** are critical building blocks of the human
cognitive process. \[FN109\] They allow humans to process or at least
cope with the infinite amount of information in their environs.
Categories and schemas influence every feature of human cognition,
affecting not only what information receives attention, but also how
that information is categorized, what inferences are drawn from it, and
what is or is not remembered. \[FN110\]

Page 1132:

Here, categorization and the use of categories will refer only to the
classification of elements, experiences, instances, or arguments into
groups. Once an element is categorized, an individual can apply a schema
to it **in order to draw inferences and derive predictions.** \[FN113\]
Where categorization focuses on the classification of instances, schemas
represent knowledge about the categories \[FN114\] and focus on the
\"application of organized generic prior knowledge to the understanding
of new information.\" \[FN115\] Put slightly differently, where
categories are the \*1133 classifications into which we place the
information we encounter, schemas refer to the embodiment of \"our
knowledge about the category and its members.\" \[FN116\]

Page 1139:

As we have been asserting, schemas have powerful effects. They help us
organize, find meaning in, and make predictions about our environs
\*1140 including the actions and behaviors of others. \[FN150\] In doing
so, schemas \"influence the encoding of new information, memory for old
information, and inferences where information is missing.\" \[FN151\]
Indeed, as Hazel Markus and Robert Zajonc explain, the influence of
schemas on our systems for processing information is ubiquitous:

In general, information processing may be seen as consisting of schema
formation or activation, of the integration of input with these schemas,
and of the updating or revision of these schemas to accommodate new
input. Accordingly, it should be \[and is\] possible to observe the
influence of schemas at every stage of information processing (e.g.,
**encoding, storage, retrieval, inference**), at all levels of
processing (conscious, preconscious), and on all parameters of the
response (speed, confidence, accuracy, etc.). \[FN152\] In short,
schemas interact with our information processing at every step, at all
levels, and on every parameter.

Page 1159:

In light of the significant role that schemas play in attention and
categorization, **their dangers become more evident**. William von
Hippel described the good and the bad of schemas this way:

Schema\[s\] clearly serve an important conceptual and organizational
role in information processing, and they provide insight into what is
likely to occur and what has probably occurred. For these reasons, they
seem to allow perceivers to process more information with less effort.
Yet herein lies their fault. . . . \[Very often\] schema\[s\] do not
really allow more information processing with less effort. Rather,
schema\[s\] simply allow the perceiver to quickly encode the gist of an
event, thereby causing the perceiver not to attend to the available
perceptual information. So the gist is stored, but the original
perceptual event is lost. In such a fashion, the momentary facilitation
in understanding that schema\[s\] bring about can \*1160 be outweighed
by the long-term loss to memory of the complex, perceptual world.
\[FN224\] More tersely, Kunda explains that \"the seemingly irrelevant
details that \[schemas\] lead us to ignore may actually be important.\"
\[FN225\]

Such dangers\--created by the fact that our categories and schemas may
bias our understanding of the world in numerous ways\--are one of the
key concerns of this Article.

**[VI. Dispositionist Thinking and Judging.]{.underline}**

Dispositionist thinking coupled with other cognitive processes discussed
in these articles appears to play a strong role in social and political
processes as well as ascribing **causation, responsibility and blame**.
There is more slavery today than ever before, just not openly sponsored
by governments. Our dispositionism lets us believe these are just bad
people doing bad things, comfortable with the fact that we are good
people doing good things, while ignoring the root causes, and leaving
the problem to law enforcement, which will never be adequate. To the
extent we can categorize the victims, the **just world heuristic** helps
us sleep at night, knowing that people get what they deserve. Here is an
excerpt from an article on the subject.

THE SITUATION: AN INTRODUCTION TO THE SITUATIONAL CHARACTER, CRITICAL
REALISM, POWER ECONOMICS, AND DEEP CAPTURE,152 U.Pa.L.Rev. 129 (2003),
Jon Hanson;David Yosifon

Page 312:

We all see dispositionally, and this dispositionism had long made people
blind to the situation of slavery, and to the situational influence on
slaves.

\* \* \*

Page 313:

Yet this common sense dispositionist presumption about Africans had not
always been the common sense. In fact, the dispositionism emerged in
America precisely because it was needed, as indicated above, \[FN654\]
to reconcile principles with practices. \[FN655\] And few people, we
suspect, felt the dissonance and the need to reconcile more intensely
than did Thomas Jefferson \[FN656\]\--civil rights theorist, father of
the American Revolution, author of the Declaration of Independence,
President of the United States, and Virginian slaveholder. \[FN657\]
Indeed, \*313 Jefferson presumed to analyze the inherent nature of
slaves\' inferiority through a neutral and scientific approach. In an
extended argument in his Notes on the State of Virginia, Jefferson
enumerated in remarkably dispositionist terms the evidence that he found
to justify the maintenance of slavery. \[FN658\]

\* \* \*

It is important to note that Jefferson does not entirely disregard the
influence of the situational effects of slavery. In fact, he purports to
examine and control for such evidence.

\* \* \*

Jefferson continues his proof by comparing American slaves to Roman
slaves,

\* \* \*

Thomas Jefferson, who announced to the world that \"all men are created
equal,\" \[FN672\] who founded the University of Virginia and carried
the Enlightenment\'s torch through his lifelong emphasis on the
importance of education, who believed he was an objective natural
historian and scientist, still could not see the situation in the case
of slavery. Dispositionism runs deep, and we can know with fair
confidence that, were we living in Virginia at the time that Jefferson
wrote, our position likely would have been even less situationally
sensitive than his was. \[FN673\]

Excerpts from another article which outlines the importance of taking
into account **internal (in the mind) and external situation** rather
than merely **disposition - motives, preferences, choices and will**:

THE SITUATIONAL CHARACTER: A CRITICAL REALIST PERSPECTIVE ON THE HUMAN
ANIMAL, 93 Geo.L.J. 1 (2004), Jon Hanson and David Yosifon

Page 6:

In The Situation, we highlighted several foundational studies
illustrating both the strength of dispositionism and the extent to which
our dispositionism is \*7 wrong. \[FN7\] Our exposition centered on the
path-breaking work of Yale psychologist Stanley Milgram, who cracked the
dispositionist nut wide open in a dramatic series of experiments in the
1960s.

Milgram arranged an experimental situation in which
subjects\--compensated volunteers\--were led to believe that they were
participating in a study on memory. \[FN8\] In the basic design of the
experiment, the subject first met another \"subject\"\--who was actually
one of Milgram\'s confederates\--and the two drew straws to determine
what part in the experiment they would take. The confederate was
inevitably assigned the role of the \"student,\" and promptly strapped
into a chair with electrodes affixed to his body. The true subject was
(seemingly randomly) assigned the role of the \"teacher,\" and was
instructed to administer an electric shock\--by flipping a switch on a
shock box\--each time the \"student\" incorrectly answered a question
posed by the experimenter. The \"teacher\" was led to believe that the
shocks would be painful, and that their intensity would increase in
fifteen-volt increments with each wrong answer\-- from 15 volts all the
way up to 450 volts, which was labeled \"Danger! XXX!\" on the shock
box. \[FN9\]

Before the experiment was undertaken, Milgram described the protocol to
lay people and psychologists and then asked both groups to estimate how
far most \"teachers\" would go with the shocking before refusing to
continue. Those surveyed believed, as might the reader, that most would
refuse early on. College students predicted that just 1 in 100 subjects
would shock all the way to 450 volts, and professional psychologists
predicted that only 1 in 1000 \-- \"the sadists\"\--would go that far.
\[FN10\]

But we humans do not\--and this is a central theme of critical
realism\-- understand ourselves well. \[FN11\] In the basic design of
the experiment, 100% of the subjects continued with the shocking at
least to 350 volts, and 65% went all the way to 450 volts (\"Danger!
XXX!\"). \[FN12\]

\* \* \*

In our dispositionism we \*8 fail to appreciate the powerful, but
unseen, situational influences over the subjects\' behavior in
Milgram\'s lab. Milgram performed his study in numerous settings on
hundreds of subjects who were, in all respects, typical people. They
were not sadists; they were simply, like all of us, situational
characters who were subject to unappreciated but profound influences in
the situation. Indeed, Milgram was able to alter his subjects\' behavior
by altering the situational influences. By varying the proximity of the
\"teacher\" to the \"student,\" or the \"teacher\" to the
\"experimenter,\" or by altering the prestige of the experimental
setting (by moving the location of the experiment from Yale to
Bridgeport, Connecticut), Milgram discovered he could increase or
decrease the level of shocking that subjects would be willing to
administer. \[FN13\]

Experiments like Milgram\'s, and there are literally hundreds of others,
\[FN14\] have demonstrated that we place far too much emphasis on
disposition\--on an individual\'s perceived motivations, preferences,
choices, and will\--in accounting for her conduct. In so doing we fail
to appreciate the very potent, though often unnoticed, influences of
situation.

Page 8:

At almost every turn, dispositionism defines or biases what we see and
how we construe what we see: behavior is strongly presumed to reflect
freely willed, preference-satisfying individual choice. But as
dispositionists, we are both consistent and consistently wrong.

Page 20:

Our purpose in reviewing a parade of dispositionism is to raise the
stakes of what probably will come as a surprise: the dominant
attributional schema informing our self-conceptions, our lay and social
theories, and our laws, is, in important ways and to significant
degrees, \*21 wrong.

Page 21:

The stakes of dispositionism are huge\--and they are, in our view,
stakes we are losing and will continue to lose if we persist in ignoring
the extent of our self-deception. Our sense of ourselves is wrong not
just in the details or on average, or in some bounded way. \[FN64\]
Rather, it is monumentally wrong, or so suggests the best available
social science.

Page 32:

5\. **The Situation of Our Interiors**

\"The more we examine the mechanism of thought, the more we shall see
that the automatic, unconscious action of the mind enters largely into
all its processes. Our definite ideas are stepping-stones; how we get
from one to the other, we do not know: something carries us; we do not
take the step.\" \~ Oliver Wendell Holmes, Sr. \[FN103\]

Our concern here is less with what so many of us, laypeople and
theorists alike, find self-evident and more with what we do not. As was
true exteriorly, it is the unavailable or less salient features of our
interiors that often wield the most influence over us. It is, in the
words of the elder Holmes, something \"we do not know\" that \"carries
us.\" Even Descartes\' terse proposition about the unity of thinking and
being is flawed in an illustrative way. The Cartesian insight imagines
thinking as that cognitive process that we are aware of and that is
independent from all else, including even the body in which that
thinking occurs. That is the flaw that neuroscientist Antonio Damasio
has dubbed \"Descartes\' Error\" in his book of the same title.
\[FN104\] Human cognitive processing is not simply the stuff of
conscious thinking. Our thinking is situational, and is influenced
outside of our awareness and control by everything from our bodies to
our social environments: \"Consciousness, the feature at the center of
what makes humans unique, is the culprit \[of our dispositionism\], for
it permits a view of who we are and what we are capable of that is
independent of the knowledge and feelings that may drive beliefs,
attitudes, and behavior.\" \[FN105\] But, \*33 as Damasio states:

\[T\]he comprehensive understanding of the human mind requires an
organismic perspective; that not only must the mind move from a
nonphysical cogitum to the realm of biological tissue, but it must also
be related to a whole organism possessed of integrated body proper and
brain and fully interactive with a physical and social environment.
\[FN106\]

Thinking may imply being, but our thinking is not what we experience it
to be. The problem is, again, one of perceiving disposition and
overlooking situation. That partial vision is, as we\'ve argued, a key
feature of being human and may be part of what Augustine was suggesting
when he wrote: \"Fallor ergo sum\" (I am deceived, therefore I am)
\[FN107\] or, in any event, it is what we mean when we write \"I think
dispositionally, therefore I am deceived.\"

Our point in this discussion has been that there is more to the
\"**situation**\" than what occurs outside of the human actor. Just as
there is an unseen exterior situation that gives rise to the exterior
fundamental attribution error, there is an interior
situation\--undetected but incredibly powerful\--that gives rise to the
interior fundamental attribution error. Our experiences are wrapped in
two layers of situational influences. To better understand what moves us
requires understanding them both; and to better understand the power of
either requires understanding its relationship with the other.

Page 33:

**Situation**, as we mean it, includes anything that influences our
attitudes, memories, cognitions, emotions, behaviors, and the like in
ways that we tend not fully to appreciate or control. The situation,
then, is part of the human predicament: it is in and around us, it is
influencing us, and it is doing so in ways that we do not appreciate, do
not understand, do not have a place for in our theories, and do not \*34
individually or dispositionally control.

Page 39:

Most of the best-known choice biases occur within a precise time period
and largely without relation to temporal considerations. In this
subsection, we briefly consider three of them: **heuristics, endowment
effects, and framing effects**.

i\. **Heuristics**

People, for good reason, are cognitively frugal. Some say, as we noted
earlier, that humans are cognitive misers. \[FN127\] Because cognitive
capacity is scarce, corner cutting is not just useful, it is necessary.
People engaging in inferential tasks \"virtually always\" rely on
judgmental strategies\--termed heuristics\--that help them reduce
complex problems into manageable ones. \[FN128\] Such strategies
\"probably produce vastly more correct or partially correct inferences
than erroneous ones, and they do so with great speed and little
effort.\" \[FN129\] Still, there are significant problems with such
mental rules of thumb. First, \"\[a\]lthough these heuristics often lead
to effective reasoning, they also \*40 lead to systematic biases and
errors.\" \[FN130\] Second, we normally do not realize we have these
biases, leaving us undefended against their harmful effects. \[FN131\]
And, third, for the same reasons, our cognitive shortcuts leave us
susceptible to exterior situational manipulation. \[FN132\]

**Availability**, for example, \"is a heuristic that is used to evaluate
the frequency or likelihood of an event on the basis of how quickly
instances or associations come to mind.\" \[FN133\] **This shortcut may
not mislead us, but it often does.** \"There are many factors
uncorrelated with frequency \... \[that\] can influence an event\'s
immediate perceptual salience, the vividness or completeness with which
it is recalled, or the ease with which it is imagined.\" \[FN134\]

Page 42:

iii\. **Framing Effects**

And more generally, the way in which an issue is presented to us
significantly influences how we perceive it. Psychologists have dubbed
this the framing effect. Even minor alterations in the presentation of
options that are substantively identical seem to influence our
perceptions and attitudes regarding the options. \[FN147\] Kahneman and
Tversky, the cognitive psychologists who identified and named the
phenomenon, describe it as \"both pervasive and robust.\" It is \"as
common among sophisticated respondents as among naïve ones \.... In
their stubborn appeal, framing effects resemble perceptual illusions
more than computational \*43 errors.\" \[FN148\] As another decision
theorist has explained, the power of the phenomenon results from our
\"tendency to accept problem formulations as they are given \... \[to\]
remain, so to speak, mental prisoners of the frame provided to us by the
experimentalist, or by the \'expert,\' or by a certain situation.\"
\[FN149\] More succinctly, \"**framing\" is one identified piece of the
manipulable situation**.

Page 43:

iv\. **Summary**

There is something familiar about the source of these well-documented
heuristics. They are different ways of talking about the same basic
phenomenon. They are all just manifestations in different contexts of
what we have been describing throughout this Article and its companion:
**we see the vivid and we miss the pallid. Small pieces of the picture
tend to dominate our assessment of the whole image.** **We readily see
what is available, anchored, and presently normal, all according to how
it had been framed, and we find it difficult to see much else**. As Ziva
Kunda puts it, these choice biases \"may be viewed as a kind of mental
contamination \.... Even though we do not want our judgments to be
contaminated in this manner, it is very difficult to eliminate the
contamination.\" \[FN150\]

That difficulty, we believe, is largely the consequence of our interior
situations. We do not see these biases at work. We do not see, in other
words, that we do not see. This inability to see our interior situation
is the source of the **interior fundamental attribution error**. And our
interior myopia helps give rise to the **exterior fundamental
attribution error**. People\'s behavior, like news of a homicide, is
available. Their situation, like statistics on stomach cancer, generally
is not. We are dispositionists because of what comes to mind most
easily\--and, once in our minds, anchors our attributions. \[FN151\]
**Interiorly and exteriorly, we humans miss the situational forest for
the dispositional trees**.

In part because our exterior dispositionism is causally related to our
interior dispositionism, our interior situation can be easily exploited
through the manipulation of our exterior situation. Each study
demonstrating a choice bias is itself indirect proof of that fact.
**Scientists were able to manipulate cognitions by manipulating the
exterior situation. And the interior situation leaves open and unguarded
the gates through which the Trojan horse of exterior situation freely
enters, not as a trophy of our dispositional triumph, but as a hidden
means of influencing our behavior.** \[FN152\]

Page 51 re **schemas:**

Thus, the benefit of such knowledge structures is that they provide us,
often automatically, with a way of understanding our world so that we
can operate reasonably well within it, at the same time that they free
up cognitive capacity to cope with other pressing issues. \[FN198\]
Similarly, the concepts, insofar as they are shared, allow us to
communicate efficiently with those around us. \[FN199\] In short,
**without the knowledge structures \"\[w\]e would be unable to extract
meaning from the huge amount of information that surrounds us, unable to
generalize from one experience to another, and unable to communicate
effectively with each other.\"** \[FN200\]

But those benefits are not without costs: \"A price is paid for this
mental economy.\" \[FN201\]

Page 52 re schemas:

**Some beliefs, theories, and schemas are relatively poor and inaccurate
representations of the external world.** Furthermore, objects and events
are not always labeled accurately and sometimes are processed through
**entirely inappropriate knowledge structures.** \[FN203\] \"Without
these structures stored in memory, life would be a buzzing confusion,
**but the clarity they offer is helpful only in proportion to their
validity and to the accuracy with which they are applied to the data at
hand.**\" \[FN204\] And when that is not the case, they can be
misleading and harmful. Indeed, that is the main point that this Article
is making about the dominant self-schema: dispositionism. We are not who
our knowledge structures tell us we are.

Indeed, the best known\--indeed, for many, the only known\--example of
the operation of interior schemas is the **group-based stereotype**.
\[FN206\] \"\[S\]tereotypes \*53 are typically viewed as cognitive
structures that contain our knowledge, beliefs, and expectations about a
social group \....\" \[FN207\] They are described as \"culturally
shared, indeed hackneyed, notions\" \[FN208\] about those groups. Here
is where the human tendency to rely on knowledge structures has come to
be understood as an inherently pernicious process, instead of as a
necessary and often helpful one. \[FN209\] Stereotypes, in other words,
are the stereotype of what we are calling \"knowledge structures.\"

Page 54:

The dynamics of our stereotypical thinking are driven by the interaction
of our interior and exterior situation. **Stereotypes that are prominent
in our culture meet with a cognitive situation within us that is poised
to confirm them. We tend to test a hypothesis by asking questions about
whether there is evidence to confirm it and forgetting to ask whether
there is evidence that would disconfirm it.** That unbalanced
positive-test strategy and the resultant confirmatory bias \[FN216\]
occurs, not only for hypotheses that we generate ourselves, but also for
any hypotheses or schemas that occupy our interiors. Thus our minds
automatically search for, and disproportionately emphasize, evidence in
the world that will tend to confirm our racial or sexual stereotypes (be
they negative or positive, conscious or implicit). \[FN217\]

The tendency is heightened by the fact that as a consequence of this
bias in our mental processing, social psychologists have suggested that
encouraging reflection about stereotypes, without more, may perversely
result in the search and location of further confirmatory evidence for
the stereotype under review. \[FN218\] **More attention, in other words,
may not counteract the confirmation bias; it may simply give it more to
work with.** \[FN219\]

Page 63:

i\. **Causation, Responsibility, and Blame**

There is a second \"fundamental process underlying much of social
perception and action\" that has long been understood by social
psychologists, but not by the rest of us: namely, the process of
**making causal attributions**. \[FN266\] People are eager to understand
the causes of salient outcomes and actions in their environs and have
fairly deeply ingrained schemas (using the term broadly \[FN267\]) for
identifying those causes. Fritz Heider introduced the concept of
attributions in the middle of the last century, and since then probably
no feature of cognitive process has received more attention from social
and cognitive psychologists. \[FN268\]

First, social psychologists went on to discover a number of systematic
biases in people\'s attributional processes\--including the
**actor-observer bias**, \[FN273\] the **false consensus effect**,
\[FN274\] the **self-centered bias**, \[FN275\] **attributional
schemas**, \[FN276\] \*64 and, most important, the **fundamental
attribution error**. \[FN277\] There is little need, in light of our
discussion above (and below), to further describe those sorts of biases
or their sources. They are all manifestations of the more general
phenomena that we are highlighting in this Article: our perceptions and
construals are skewed by what we tend to see, which tends to be only a
small piece of the whole picture. For example, instead of relying on
covariation principles of the sort that Kelley imagined, people too
readily rely more on temporal and spatial contiguity and salience in
making causal attributions. \[FN278\]

For those sorts of reasons, Kelley\'s preliminary hypothesis is now
understood to have significant shortcomings, at least as a descriptive
model. This leads to the second significant development in attribution
theory. Numerous scholars have worked to develop more successful
descriptive models of people\'s **attributional processes**. \[FN279\]
For example, Bernard Weiner, with some of his colleagues, has shown that
people tend to focus on three (or four) **causal dimensions**: locus,
stability, control, (and, in some models, intent). \[FN280\] In his
early work, which focused on how people made attributions in terms of
these categories with respect to achievement efforts, he focused on
three dimensions: whether the cause was stable or temporary, whether the
locus of the cause was internal or external to the individual, and
whether the person had control over the cause. \[FN281\]

A little introspection\--or a careful read through the newspaper
headlines\-- should confirm that we humans are indeed focused on those
dimensions when examining causation for all sorts of surprising outcomes
that we encounter. Furthermore, as this line of research shows, our
reactions (affective and behavioral) vary significantly depending on how
we perceive a cause along those dimensions. Thus, the destruction caused
by a **forest fire** seems different to us when it was caused by a
person, rather than a bolt of lightening (locus). Likewise, it matters
if the person had control over the outcome (controllability), whether
the person has created several such fires in the past (stability), and
whether the person was a careless camper or a profiteering arsonist
(intent). Weiner\'s basic attributional model has enjoyed considerable
empirical support and has been expanded to apply in numerous settings.
\[FN282\]

And that leads to the third major development in attribution theory
since Kelley\'s initial effort. Social psychologists, including Weiner,
have constructed more refined theories to capture not just how people
make causal attributions, but also how they assign **responsibility and
blame** based on those causal \*65 attributions. Kelly Shaver has argued
that people tend to assign **responsibility** for harmful outcomes when,
roughly, the harm was foreseeable and when the person acted volitionally
and without justification. \[FN283\] And people assign **blame**, a more
punitive designation, when the harmful outcome was intended. \[FN284\]
In Shaver\'s words, \"An assignment of blame is \... a particular sort
of social explanation. It is the outcome of a process that begins with
an event having negative consequences, involves judgments about
causality, personal responsibility, and possible mitigation.\" \[FN285\]
Again, there is considerable evidence to support those attributional
theories. \[FN286\]

Page 65:

That leads to the most recent development in attribution theory. There
is significant evidence that **many of our legal institutions reflect
people\'s attributional impulses (and not, for example, a means of
promoting efficiency or wealth maximization). Put most simply, just as
people want to identify causation, they likewise want to, when
attributionally appropriate, assign responsibility and blame.** Moving
from causal attributions to attributions of responsibility or blame, the
lay scientist turns in her lab coat for a judicial robe.

\* \* \*

Just as the habits of the professional scientist reflect the same urges
and biases as the habits of the lay scientist, \[FN288\] so do the
judgments and sentences of the courtroom judge reflect those of the lay
judge. And **our legal system appears to reflect and, to a significant
degree, satisfy our impulses to establish causation, assign
responsibility, and lay blame**. \[FN289\]

Page 66:

But, if they are not pointed out to us, our attributional processes
begin and remain largely automatic and unconscious \[FN291\]; they are
experienced, if at all, as obvious and natural. In that way, our
attributions manifest themselves more as conclusions than as a process
or analysis that yields a conclusion. **Like thinking, preferring and
choosing, our attributional ascriptions appear to us as self-evident
and, in a sense, dispositional. We miss much of the situation out of
which they emerge.** \[FN292\]

Page 73:

Social psychologists have given a variety of names to this process\--for
example, **confirmatory bias, perseverance bias, hypothesis-based
filtering, elastic justification, and, more generally, motivated
reasoning**. \[FN329\] Perhaps unsurprisingly (given the topic),
researchers have found a great deal of evidence suggesting that such
devices are extremely powerful. Richard Nisbett and Lee Ross summarized
the literature in 1980 as follows:

**When people already have a theory, before encountering any genuinely
probative evidence, exposure to such evidence (whether it supports the
theory, \*74 opposes the theory, or is mixed), will tend to result in
more belief in the correctness of the original theory than normative
dictates allow.**

**\... When people approach a set of evidence without a theory and then
form a theory based on initial evidence, the theory will be resistant to
subsequent evidence\.... \[and\]**

**\... When people formulate a theory based on some putatively probative
evidence and later discover that the evidence is false, the theory often
survives such total discrediting.** \[FN330\]

The influence of those biases, and others, \[FN331\] makes clear one of
the great problems with our schemas: we create them too quickly and
maintain them too loyally. And, again, this problem is not limited to
the processes of just lay scientists: \"The tendency of professional
scientists to persist in adhering to theories well past the point at
which such adherence can be justified by the evidence has been observed
by many.\" \[FN332\] We will return to that point briefly below,
\[FN333\] and more thoroughly in subsequent articles. \[FN334\] For now,
**the crux of our point is that all of us are subject to the same
biasing process that we just do not see.**

**To be sure, we often see others as biased, prejudiced, vested,
incoherent, inconsistent, or closed-minded. \[FN335\] But those
experiences are schematized as dispositional quirks, not as a reflection
of their deeper interior situations. \[FN336\] In ourselves, we see what
every person is presumed capable of\--clarity, objectivity, and
open-mindedness. \[FN337\] And we can maintain that self-affirming
view\--as do those who we feel should not\-- in significant part because
we do not see the interior situation.** \[FN338\]

As with the other biases, there\'s a critical kicker to this analysis:
exterior \*75 situation, which we began by emphasizing our failure to
appreciate, can wield an immense influence over which schemas we adopt,
which we reject, and how and when we apply them. And that process is
itself guided in part by the operation of interior schemas functioning
below the level of our conscious awareness. Furthermore, once our
schemas are in place, exterior situation can provide us the evidence and
ambiguity we need to sustain them.

Page 75:

No human inference process is without schematic structuring, and none is
insulated from schematic distortions. \[FN342\]

Page 84:

As this discussion helps to demonstrate, professional scientists and lay
scientists are bedeviled by the same situation. **Knowledge structures
and schemas are all around us, guiding our every, or most every,
thought, and simultaneously assisting and distorting what we \"know.\"
The coloring concepts and theories we employ can lead us to focus on
irrelevant details in our environment, to overlook the relevant details,
and to misunderstand our world. \[FN387\] As we hope the reader has
already recalled (with the aid of the knowledge structure that we are
attempting to create), that is precisely the mechanism behind the
exterior and interior fundamental attribution errors.**

Page 84-85:

Where lay people and economists see \"thinking,\" they vastly overstate
its significance and vastly understate the interior situation of our
thinking\-- that is, our unseen cognitions. Social psychology and
related fields make clear that all of our cognitive processes are more
or less influenced by **unseen and distorting influences, from
heuristics and framing effects on one hand to schematic and
attributional processes on the other**. And all of those unseen
cognitions and cognitive processes render us more or less **vulnerable
to outside manipulation**\--indeed, the experiments revealing the
cognitive phenomena simultaneously \*85 reveal the extent to which they
can be tapped through exterior situation.

In short, **this section has shown some of the ways in which, although
it is true that we experience ourselves thinking, we do not think the
way we think we think. There is more to the situation.** And, as the
next section illustrates, we have barely scratched the surface of our
interior situations.

\*91 It is not just that our minds have a mind of their own (as the
previous analysis has indicated), it is also that those inner minds have
a motivation\--actually, a whole set of motivations\--of their own.
\[FN421\]

\* \* \*

four general types of motivations stand out as particularly significant
aspects of the situational character\'s inner life: **(a) the motive to
understand; (b) the motive to self-affirm; (c) the motive to simplify;
and (d) the motive to cohere**. \[FN422\] As our review of more specific
motives will reveal, motivations can be, and often are, in tension with
one another. The motive to understand, for example, is often in tension
with the motives to self-affirm and simplify. The motive to cohere,
then, pushes us to reconcile our conflicting motivations by altering
exteriorly and interiorly our cognitions, attitudes, or behavior.

\* \* \*

Sometimes, though, our self-affirming motives can conflict with our
dispositionism\--that is, there are instances, such as following a
failed or disappointing performance, that we look to situation for
causal attributions in order to avoid the disheartening conclusion that
that failure reflected our own dispositional shortcomings. \[FN461\] One
experimenter interviewed politicians several months after an election.
The winners attributed their performance largely to dispositional
factors such as hard work, perseverance, skill, planning, and strategy.
The losers, on the other hand, looked to situation, and attributed their
performance to the politics of the district, their opponents\' name
recognition, to their lack of money, and so on. \[FN462\] **The groups
thus revealed the two-sided nature of the self-serving attributions: \"a
\'self-enhancing bias\' (attributing success to internal relative to
external causes) and a \'self-protecting bias\' (attributing failure to
external relative to internal causes).**\" \[FN463\]

Page 101:

Our self-affirming motivations extend beyond our self and group
identifications. Just as each of us is motivated to believe that \"I am
good,\" and, with respect to our groups, \"we are good,\" so too are we
motivated to believe that \"our world is good.\"

In a pioneering project within social psychology, Melvin Lerner
demonstrated that people seek to confirm a \"**just world
hypothesis**.\" Through a series of experiments, Lerner demonstrated
that, \"we do not believe that things just happen in our world; there is
a pattern to events which conveys not only a sense of orderliness or
predictability, but also the compelling experience of appropriateness
expressed in the typically implicit judgment, \'**Yes, that is the way
it should be.\"\' \[FN479\] And, yes, \"people get what they deserve.\"
\[FN480\]**

\* \* \*

**In our effort to maintain our belief in a \"just world\" hypothesis,
we tend to attribute bad outcomes to individual dispositions, because it
is generally more comforting to presume that it is the person who was
bad, rather than the situation.**

Page 107:

We humans seek explanations that are coherent, that we can make sense
of, and that can be supported by reasons. \[FN508\] This **coherence
motive** animates the relationship and tradeoff among motives. Because
we value coherence, the desire to see it in ourselves dovetails with our
motive for self-affirmation. That powerful driving force in our
self-conception has figured prominently in social psychological
research. Inquiry into \"**cognitive dissonance**,\" for example, has
been a mainstay of the field for decades. \[FN509\] Often it is the
case, as we have already suggested, that our motivations are in
conflict. Though motivated to view ourselves positively, our behavior
can pose problems for that self-conception. There can be many reasons
for such dissonance\--not least of which is the fact that we are
dispositionist situational characters.

While social psychologists debate some of the details of the cognitive
dissonance dynamic, \[FN510\] it is the basic pattern, about which most
agree, that best illustrates the dispositional illusion. The
dispositionist sees behavior as reflecting little more than thinking,
preferring, and willing. From that conception, the dissonance can be
eliminated by bringing behavior into line with those interior elements.
But it can also be, and commonly is, eliminated, by bringing the latter
into line with the former. \[FN511\] That basic challenge to
dispositionism\--that our behavior influences our beliefs and
attitudes\--has been a recurring theme in the cognitive dissonance
literature: \"Regardless of the exact motivational underpinnings of
dissonance, the evidence clearly indicates that attitudinally discrepant
actions can result in a reanalysis of the reasons why a person engaged
in a certain behavior (or made a certain choice), and cause a person to
rethink the merits of an attitude object.\" \[FN512\] That subconscious
task can be accomplished in a myriad of ways, from changing our opinions
outright to more subtly trivializing \*108 a belief that is incongruent
with our behavior. \[FN513\]

The important critical realist lesson here is not just that behavior may
not reflect a preference or belief. That much we have said before when
describing how situation can powerfully influence behavior\--disposition
notwithstanding. The lesson is also that our preferences, such as they
are, are themselves malleable, constructed, and contingent\--subject to
changes in our behavior and in our situation. The malleability of our
attitudes, combined with our sense that attitudes are stable, assists us
in our motive for coherence.

Page 109:

This desire to see ourselves in a positive light is an important motive
behind what Lee Ross and his co-authors have dubbed \"**naïve
realism**\"\--the name \*110 given to \"three related convictions about
the relation between \[one\'s\] subjective experience and the nature of
the phenomena that give rise to that subjective experience.\" \[FN523\]
First, we naively believe that we see the world as it really
is\--through objective, unfiltered lenses. Most of us think that we
\"get it\" for the same reason that the vast majority of us believe that
we are above average drivers\--it is self-affirming. Second, it almost
goes without saying that anyone else who is similarly neutral and
intelligent will see the world as we do\--that is, accurately. At times,
though, we are confronted with views that conflict with our own, an
experience that creates a kind of dissonance. That suggests the third
tenet of naïve realism. When our interpretation of a particular
situation apparently conflicts with someone else\'s, something has to
give. Because we presume that we see things as they are, something must
be distorting the perceptions of those who see things otherwise. Social
psychological research shows that an extremely common means of relieving
that dissonance is to attribute the gap between our outlook and theirs
to a lack of objectivity on their part. We assume that there is some
dispositional source of their bias\--lack of intelligence, or laziness,
or corruption. To be sure, we ourselves will admit to having a
particular vantage point and set of experiences that inform our judgment
and perspective\--but as it turns out, our particular background was the
path to authentic insight. \[FN524\]

**This is a key source of our biases: we don\'t believe that we are
subject to them (allowing us to trust our own clear vision) and we are
extremely quick to see them in others (allowing us to distrust others\'
obscured vision). \[FN525\] And so it is that we are quick to see
ideological or political bias on the part of our adversaries and
gullibility or vanity on the part of even our friends and family when
they fail to share our worldview.** \[FN526\]

Page 110:

In sum, we see bias there, but not here\--and, in either case,
dispositionism. \[FN530\]

**Regarding groups:**

page 119:

Social scientists have discovered numerous manifestations of our
**motive for group coherence**. That motive, combined with the
self-affirming faith we place in our own knowledge structures,
contributes, for instance, to a phenomenon that social psychologists
have dubbed the \"**false consensus\" effect**. \[FN536\]

\* \* \*

This false consensus effect, a tendency to regard one\'s own views to be
commonly held by others, has been demonstrated by more than a hundred
empirical studies, over a wide range of topics from particular food
preferences to broad political and social policy views. \[FN539\]

\* \* \*

The group-coherence motive combined with dispositionism can yield some
troubling and otherwise perplexing phenomena. Because we are
dispositionists, our perception that certain behaviors are common (or
uncommon) leads us to perceive that the attitudes, preferences and
beliefs of others correspond to that common (or uncommon) behavior.
Because we do not ourselves subscribe to those attitudes, we infer from
others\' behavior that our attitudes are exceptional. That dynamic
contributes to the tendencies known in social psychology as
\"**pluralistic ignorance**\" and \"**false uniqueness**.\" Both
illustrate the power of the group coherence motive.

\* \* \*

**The bad news was that some subjects began to alter their own behavior
and beliefs** to more closely correspond with their ignorant
perceptions. \[FN543\] Prentice and Miller attributed that troubling
example of self-fulfilling group perceptions to a basic motive to avoid
dissonance with one\'s key constituencies. Once again, it is crucial to
recognize that the subjects were not faking their new attitudes. These
are not rational actors operating within stable preferences, altering
their conduct through clever performance while remaining true to their
core beliefs. These are situational characters in whom stable
preferences and core beliefs are largely an illusion. Our behavior and
attitudes and the behavior and perceived attitudes of our groups are all
mutually constructed and reconstructed as the situation requires. As
each of us looks out at others seeing disposition and missing situation,
we infer attitudes that do not exist. But, as these studies reveal, our
false perceptions can be tragically powerful as we each seek to bring
our own view and behavior into sync with shared misperceptions. Prentice
and Miller \"believe that group identification is the root cause for
many cases of pluralistic ignorance\--\*115 that individuals often act
of a desire to be good group members but interpret others\' similarly
motivated behavior as reflecting personal beliefs and opinions.\"
\[FN544\]

The problem of pluralistic ignorance and the motive for group coherence
distorts many social norms and would seem to have significant
implications for policy and law. The dynamic seems to be at work in
creating and reinforcing gender and sex roles. \[FN545\] It is also
behind the pervasive, dysfunctional classroom dynamic in which students
do not ask questions because they assume that others\' silence suggests
they are themselves alone in their ignorance, thus contributing to the
silence that encourages others to do the same. \[FN546\] And so it is
that even in most \"learning\" environments, ignorance begets ignorance.

**Regarding free will:**

Page 124:

And so we come at last to the conscious will, that inarticulable, yet
unmistakable, inner experience of \"oomph\" that is, in many ways, the
crown jewel of our \*125 dispositionist self-conception. \[FN596\] We
have been emphasizing throughout this Article that there is a vast
interior situation that invisibly influences our thoughts, preferences,
and actions, and leaves us vulnerable to exterior situational forces
that do the same. What, then, of the conscious will?

Page 125:

\* \* \* None of the researchers in this field of social science have
concluded, nor do we, that the \"conscious will\" is purely and totally
an illusion. What is asserted\--and what researchers have
demonstrated\--is that the experience of will is far more widespread
than the reality of will. Wegner calls the latter the empirical will
\[FN599\] and argues that our perceived will is often an unreliable and
misleading basis for understanding our behavior. The experience of will
occurs often without empirical will, and thus creates the illusion of
will. Moreover, it contributes to the illusions of choice, preference,
and, more generally, dispositionism.

Page 133:

What is now fairly clear is that the dominant lay and legal theories of
the person (or \"personology\") are wrong\--not just \"too simple,\" but
fundamentally wrong. Social science has clearly demonstrated that we are
not who we think we are. It is true that we experience ourselves
thinking, preferring, acting, and willing, but those comforting
perceptions are often illusory, and they obscure the far more
significant influence of our unseen interior situation.

Page 177:

We hold onto our dispositionist views tenaciously, even well beyond the
point where our claims about the truth of the matter have been
abandoned. We are dispositionists, not because humans are dispositional,
but because there is too much that has already been built upon the
dispositionist foundation to begin building elsewhere, and because
dispositionism helps us to sleep.

As Nobel Laureate Isaac Bashevis Singer elegantly expressed the dilemma:
\"Of course I believe in free will. I have no choice.\" \[FN820\] And so
it is that the illusion of our freedom, our dispositionism, our wills,
are forced upon us by fears and forces in our situation that we do not
see, and would prefer not to. We have no choice but to pretend that we
have a choice.

Page 178:

Taken together, the social psychological findings we have reviewed here
should shake our self-conception at its foundation. What has been
revealed in the studies we reviewed cannot be considered marginal or
anomalous. Situation, it seems, moves us far more than we suspect in our
slumbering, blissful dispositionism. That conclusion is hard to take,
hard even to get our mind around, because of this very dispositionism,
which sees us as responsible for our situations and not the other way
around.

Page 179:

There is a tragic irony in our predicament. By blinding ourselves to the
very forces that impinge upon our **freedom**, we are surrendering to
them. To be serious about **liberty** requires that we not unwittingly
turn over the situational strings to whoever has the means and ends to
manipulate them. Indeed, our greatest dispositional act may be to
acknowledge that we are situational characters and to choose to
understand and gain some voice and control over the situation that
largely controls us. In that very important sense, **we do have a
choice**.

**[VII. Interpersonal and Intergroup Conflict]{.underline}**

The following article sheds more light on interpersonal and intergroup
conflict grounded in cognitive biases and dispositionist thinking. Among
the contributing causes are **Self Serving Attributions, Self Interest
bias, Reactive Devaluation of Proposals from Counterparts, Fundamental
Attribution Error, Positive Halo Effect, Perception of Hostile Media
Effect against Group or Cause, Biased Assimilation of New Information to
Preexisting Beliefs, Cognitive Dissonance, Belief Perseverance, Attitude
Polarization, False Polarization Effect, False Consensus Effect,
Illusion of Personal Objectivity, Overconfidence in Social Prediction,
Rationalization, the Planning fallacy, Nonconsious Priming and
Correspondence Bias.** The extent to which we see bias in other groups
and not in our own is startling, as the testing among groups in this
article demonstrates. I have not set forth the statistics in this paper.
I discussed with Professor Pronin the extent to which this kind of
knowledge was being implemented in society, government and politics, and
did my own research on our major Departments, International
Organizations and Non-Government Organizations. My conclusion is that we
are not putting such knowledge to good use. Rather, it seems we are
employing the same old agendas and strategies as we have been for
thousands of years.

THEORETICAL NOTE

Objectivity in the Eye of the Beholder: Divergent Perceptions of Bias in
Self Versus Others,

Emily Pronin, Princeton University; Thomas Gilovich, Cornell University;
Lee Ross, Stanford University; Psychological Review, Copyright 2004 by
the American Psychological Association, 2004, Vol. 111, No. 3, 781--799.
The Summary:

"Important asymmetries between self-perception and social perception
arise from the simple fact that other people's actions, judgments, and
priorities sometimes differ from one's own. This leads people not only
to make more dispositional inferences about others than about themselves
(E. E. Jones & R. E. Nisbett, 1972) but also to see others as more
susceptible to a host of cognitive and motivational biases. Although
this **blind spot** regarding one's own biases may serve familiar
self-enhancement motives, it is also a product of the phenomenological
stance of **naive realism**. It is exacerbated, furthermore, by people's
tendency to attach greater credence to their own introspections about
potential influences on judgment and behavior than they attach to
similar introspections by others. The authors review evidence, new and
old, of this asymmetry and its underlying causes and discuss its
relation to other psychological phenomena and to **interpersonal and
intergroup conflict**."

Page 794:

"Our account of the asymmetry in perceived bias is best appreciated when
considered within the broader framework of naive realism. The essential
component of naive realism---namely, the (false) sense that one sees the
world as it is and not as it is filtered through one's expectations,
needs, or motives, or "constructed" by one's sensory and cognitive
apparatus---is presented in Figure 6. Also presented there are the two
components of naive realism that follow immediately from this core
conviction: the expectation that "objective and reasonable others" will
share one's perceptions and judgments, and the inference that those who
do not share one's perceptions and judgments are therefore either
uninformed, biased, or under the influence of idiosyncratic
dispositions. Finally, Figure 6 presents a number of more specific
attributional and

information-processing phenomena that stem from these three basic
components of naive realism, some of which we shall enlarge upon briefly
here. When others see things differently, when confronted by an
individual whose responses differ from one's own, the **naive realist
faces an attributional dilemma**: Why does this person see the world so
differently? Sometimes this dilemma is resolved in the fashion
emphasized by Jones and Nisbett (1972): by concluding that the other
person possesses some telling disposition (or set of dispositions) that
is dictating his or her reactions. This inference is particularly likely
when the discrepancy involves not a differing viewpoint or
interpretation but a behavior that differs from the way one would behave
(or anticipates one would behave). The tendency to resolve this
attributional dilemma by drawing a dispositional inference is, in turn,
related to the **correspondence bias** (Jones, 1990) or **fundamental
attribution error** (Ross, 1977). Often, this error results from the
incorrect and insufficiently examined assumption that the person who
behaves differently is in fact responding to the same situation one has
in mind oneself. By assuming that the other person is responding
differently to the same "**object of judgment**" as oneself (Asch,
1952), one fails to appreciate the true situational constraints
governing the actor's behavior and thus runs the risk of drawing an
unwarranted dispositional inference. As Figure 6 makes clear, the
attributional dilemma set in motion by responses discrepant from one's
own is often resolved in other ways. Most charitably, perhaps, people
may assume that those others are misinformed or have not had sufficient
contact with the pertinent information. This motivates attempts to
"**set the other person straight**," accompanied by the expectation that
such efforts at persuasion will prove easy and should proceed smoothly
to a successful conclusion. When that expectation is not borne out, less
benign attributions result, including the attribution of bias that we
have emphasized throughout this article."

Page 796:

"The convictions of naive realism can make parties feel that the other
side is irrational or too biased to be reasoned with (rather than merely
subject to the same cognitive and motivational biases that afflict all
human beings---including oneself and one's ideological and political
allies). Moreover, when the parties do air their grievances, they may
conclude that the other side is being "**strategic**" and doesn't really
believe what it is saying. Or perhaps worse, the conclusion reached may
be that the other side really does believe what it is saying and that a
rational, interest-based negotiation will thus prove fruitless, and that
therefore only the use of force can win one's own side the just outcome
to which it is entitled. Of course, when force is applied, the cycle of
aggression and retaliation is apt to be self-perpetuating, as both sides
see the other as the instigator and their own side as acting in
self-defense or trying to teach the other side a lesson."

Page 797:

"The evidence is all too plain that human inferential
shortcomings---including **overconfident prediction** and **biased
assimilation** of the lessons of history, compounded by people's
unwillingness to consider the possibility that they are just as
susceptible to those biases as those they revile---are continually and
ubiquitously making their influence felt. At the very least, these
shortcomings in judgment and insight serve to exacerbate and perpetuate
the historical and economic roots of the conflicts that are
all-too-present elements of the human condition."

**[VIII. Cultural Cognition.]{.underline}**

Some scholars have used information described above about the decision
making process, to construct algorithmic models testing the ability of
populations or groups to change their minds in light of their own
cultural world views and situations. They discuss seemingly intractable
issues such as the debates over gun control, abortion and the death
penalty. Others have applied this body of knowledge to address
terrorism. In one study, opposing sides on the Middle East crisis were
tricked into accepting the settlement proposed by the other side, as
being the best solution, simply by being told (erroneously) that the
proposal had been made by their own leadership.

The following article tests whether a population can change strongly
held beliefs from a "false" belief to a "true" belief simply based on
empirical evidence, concluding it is very difficult. The article applies
algorithmic models based on social and psychological science, and
concludes that a population [can]{.underline} change its mind if what
they call their "**Breakthrough Politics Model**" is employed.

That agency action is supported by a compelling factual basis is often
insufficient to achieve consensus when cultural values among groups are
at stake -- grazing, mining, environmental, Native American, etc. This
article provides some lessons in solving this problem.

MODELING FACTS, CULTURE, AND COGNITION IN THE GUN DEBATE,

Donald Braman (Yale Law School), Dan M. Kahan (Yale Law School) and
James Grimmelmann (U.S. Court of Appeals for the Third Circuit); Social
Justice Research, Vol. 18, No. 3, September 2005. A few Excerpts:

Page 285:

"Our argument turns on a particular account of how culture and empirical
information interact in the formation and transmission of belief. We
will fill out the details of that account---and the extensive research
in social psychology on which it rests---by developing a series of
**models that simulate the formation and transmission of belief**.
Section 2 will present the "**Factual Enlightenment Model**," which
shows how persuasive empirical proof can indeed generate societal
consensus on a disputed issue. Section 3 will present the "**Cultural
Cognition Model**," which shows how various social and psychological
mechanisms can generate beliefs that are uniform within and polarized
across distinct cultural orientations. Section 4 develops a
model---"**Truth vs. Culture**"---that shows that cultural cognition
constrains factual enlightenment when these two dynamics of
belief-formation and transmission are pitted against one another. And
finally, in section 4, we develop a "**Breakthrough Politics Model**,"
which shows how persuasive empirical proof can dispel culturally
influenced states of false belief once policy options are invested with
social meanings that make them compatible with diverse

cultural orientations.

Page 292 (footnote omitted):

"The basic idea behind the Truth vs. Culture Model is that the same
psychological and social processes that induce individuals to form
factual beliefs consistent with their cultural orientation will also
prevent them from changing their beliefs in the face of contrary
empirical data. **Cognitive-dissonance avoidance** will steel
individuals to resist empirical data that either threatens practices
they revere or bolsters ones they despise, particularly when accepting
such data would force them to disagree with individuals they respect.
The cultural judgments embedded in affect will speak more
authoritatively than contrary data as individuals gauge what practices
are dangerous and which not. And the culturally partisan foundation of
trust will make them dismiss contrary data as unreliable if they
perceive that it originates from persons who don't harbor their own
cultural

commitments.

"This picture is borne out by additional well-established psychological
and social mechanisms. One constraint on the disposition of individuals
to accept empirical evidence that contradicts their culturally
conditioned beliefs is the phenomenon of **biased assimilation** (Lord
et al., 1979). Unlike rational **Bayesian** information-processors,
individuals don't update their prior beliefs based on new evidence;
instead they evaluate new evidence based on its conformity to their
priors, dismissing as unpersuasive evidence that contradicts their
existing beliefs. This feature of human decision-making is
extraordinarily pervasive and robust; it affects not only ordinary
citizens---who presumably are not in a position to evaluate complicated
forms of empirical data on their own---but also trained social
scientists who clearly are (Kohler, 1993)."

Page 293:

"A second mechanism that inhibits revision of culturally grounded
factual belief is **coherence-based reasoning** (Holyoak and Simon,
1999; Simon, 2003). If after assessing a conflicting body of evidence a
decision-maker finds one conclusion even slightly more persuasive than
another, she will then reevaluate the body of evidence in a biased
fashion, revising upward her perception of the persuasiveness of
evidence that supports the favored conclusion and downgrading the
persuasiveness of evidence that refutes it. After reevaluating the
evidence in this way, the favored conclusion will appear all the more
correct, inducing the decision-maker to revise her assessment of the
supporting and conflicting evidence all the more dogmatically, and so
forth and so on---until she terminates the process without the slightest
doubt as to either the correct outcome or the quality of the evidence
that supports it. This process, moreover, continues over time and across
contexts: any initial leaning toward a particular view will generate a
persistent evidence re-evaluation and filtering effect (Simon et al.,
2001). As a result of coherence-based reasoning, new pieces of
disconfirming evidence will not only fail to shake culturally grounded
factual beliefs but will fail even to induce in individuals the
discomfiting experience of lingering doubt that might trigger
reappraisal.

"Two additional mechanisms interfere with receptivity to empirical
evidence that originates from individuals outside of one's cultural
group. The first is **naive realism**. This phenomenon refers to the
disposition of individuals to view the factual beliefs that predominate
in their own cultural group as the product of "objective" assessment and
to attribute the contrary factual beliefs of their cultural and
ideological adversaries to the biasing influence of their worldviews.
Under these conditions, evidence of the truth will never travel across
the boundary line that separates a factually enlightened cultural group
from a factually benighted one. Indeed, far from being admitted entry,
the truth will be held up at the border precisely because it originates
from an alien cultural destination. The second mechanism that constrains
societal transmission of truth---**reactive devaluation**---is the
tendency of individuals who belong to a group to dismiss the
persuasiveness of evidence proffered by their adversaries in settings of
intergroup conflict (Ross, 1995)."

Page 297:

"The **Truth vs. Culture Model** showed empirical evidence and social
meaning at war. We now describe a state of affairs in which the two
might peacefully coexist. We will call it the **Breakthrough Politics
Model**. TheModel involves not just a set of mechanisms but also a
process consisting of three steps. The first is the devising of policies
that satisfy what we call the criterion of "**expressive
overdetermination**." A policy can be said to be expressively
overdetermined when it is sufficiently rich in social meanings that
individuals of otherwise opposing **cultural orientations ("hierarchist"
or "egalitarian," "individualist" or "solidarist,"** to use the types
that figure in the cultural theory of risk) can see their way of life
affirmed by it. Such affirmation helps to counteract the constraining
pressure that **cognitive-dissonance avoidance** exerts when individuals
contemplate revising a position or belief affiliated with their cultural
identity. Experimental research shows that where individuals feel
self-affirmed they are indeed more open to reconsidering their beliefs
on culturally contested issues, including the death penalty and abortion
(Cohen et al., 2000; Sherman and Cohen, 2002).

"The second step involves what we call "**identity vouching**." Public
figures who

are associated with competing cultural orientations must be recruited to
advocate

expressively overdetermined policies.16 Their participation exploits the
culturally partisan nature of trust; positions that individuals might
otherwise have rejected out of hand will be acceptable to them if
sponsored \[by those\] who possess high esteem within their cultural or
ideological group (Lorge, 1936). It also reinforces the selfaffirming
effect of expressive-overdetermination insofar as individuals determine
what it means to support a policy in part based on the identity of those
who are sponsoring it (Cohen, 2003).

"The third step we call "**discourse sequencing**." The adoption of
expressively overdetermined policies by identity vouchers can be
expected to change the common perception that the outcome of the
gun-control debate is a measure of the social status of competing social
groups. The dissipation of that perception in turn neutralizes the
tendency of individuals to dismiss as biased and disingenuous evidence
originating from persons of opposing orientations (Robinson et al.,
1995). The effects of na¨ive realism and reactive devaluation having
been neutralized, the truth will indeed be empowered to cross cultural
boundary lines and colonize those who previously held false beliefs on
the basis of their immersion in their cultural norms. Empirical data
thus does play a critical role in policy deliberation. But it comes into
play only after the formation of a new expressively pluralistic regime
of gun politics."

Page 301

"Of course, it is silly to think that individuals never change their
minds on culturally contentious issues in response to new empirical
evidence. But it is just as absurd to believe that they ever do so at
the expense of their cultural commitments. We accept J.S. Mill's
observation that truth, "even if extinguished once, twice or many times"
is likely "in the course of the ages" to be "rediscovered" and finally
to take hold at the point "when from favorable circumstances . . . it
has made such head as to withstand all subsequent attempts to suppress
it" (Mill, 1975, p. 29). But what we have shown is that culture is one
of the forces that "suppress\[es\]" truth."

**[IX. CONCLUSION]{.underline}**

We should incorporate this body of knowledge in firefighter training
from basic to advanced, improve our decision support systems, our
accident investigations and our safety programs - including creation of
a "just culture" in a "high reliability organization". With the adoption
of the new Foundational Doctrine, the time is appropriate to use this
knowledge to fully implement that Doctrine. All the stakeholders need to
be involved in the process.

[^1]: The views expressed in this paper are my own and not the position
    of the Department of Justice or the U.S. Attorney for the District
    of Arizona.

[^2]: Event Total

    Injury No Injury

    Actual Negligent 0.090% 0.010% 0.10%

    Condition Not Negligent 0.999% 98.901% 99.90%

    Total 1.089% 98.911% 100%

    Because the defendant is negligent .1% of the time and is 90% likely
    to cause an injury under these circumstances, the probability that a
    victim would be injured by the defendant\'s negligence is .09% (and
    the probability that the defendant is negligent but causes no injury
    is .01%). Because the defendant is not negligent 99.9% of the time
    and is 1% likely to cause an injury under these circumstances, the
    probability that on any given occasion a victim would be injured
    even though the defendant took reasonable care is 0.999% (and the
    probability that the defendant is not negligent and causes no injury
    is 98.901%). As a result, the conditional probability that the
    defendant is negligent given that the plaintiff is injured equals
    .090% divided by 1.089%, or 8.3%.