**Informed Consent in the Elicitation of Environmental Values**

Baruch Fischhoff

Department of Social and Decision Sciences

Department of Engineering and Public Policy

Carnegie Mellon University

Pittsburgh, PA 15213-3890

baruch@cmu.edu

412+268-3246 (ph) 6938 (fax)

August 1998

prepared for the "Workshop on Alternatives to Traditional CVM in
Environmental Valuation," Vanderbilt University, October 15-16, 1998

**Abstract**

Agreeing to participate in a study eliciting environmental values means
agreeing to abide by the commitment implied by any proposal that one
accepts or rejects in that study. By soliciting participation,
investigators promise to provide the information (and the
information-processing opportunities) needed to evaluate the proposals
that they present. This article (a) describes the relationship between
investigators and potential participants, in terms of securing informed
consent; (b) proposes an analytical standard for determining the
information needs of participants (and, hence, the adequacy of any
attempt to inform); and (c) offers two general approaches for increasing
the capacity of the communication channel. It concludes by discussing
the organization of research programs sensitive to these issues.

**Informed Consent in the Elicitation of Environmental Values**

Researchers are often required to secure informed consent from the
participants in their studies. When individuals sign the associated
consent forms, they agree that the benefits of participation outweigh
the risks, and that they have made that determination of their own free
will and with the needed information. In order to ensure that
investigators pose this decision fairly, Institutional Review Boards
evaluate proposed studies, including their informed consent procedures.
These IRBs must decide whether the risks and benefits of a study are
described accurately and intelligibly. If the IRB has doubts, then it
can request revisions in a study's design or in its informed consent
procedures. It can require guardians to make these choices for
individuals judged incapable of deciding on their own. It can serve as a
guardian itself, banning tasks that it determines to be fundamentally
incomprehensible or to offer unacceptable bundles of risks and benefits.

Studies eliciting environmental values presumably breeze right through
IRB review. After all, investigators are just presenting some (often
interesting) information and asking some (often thought-provoking)
questions. Even if that information is often incomplete, it seldom is
false. Although participants are forced to do some intellectual work, it
hardly seems like the type that could do any serious damage, except
perhaps to depress them over the state of the environment. The risks of
these labors should be very small in absolute terms, or relative to the
intrinsic interest of the task and any extrinsic rewards provided in
return for participation. Indeed, such studies seem so benign that
investigators might not even think to consult their IRBs, unless funding
agencies make that demand.

This article analyzes the conditions for informed consent in the
elicitation of environmental values, as a way of characterizing
investigators' communication obligation. It begins by discussing the
information needs implied by the relationship between respondent and
investigator. It then offers an analytical formulation of those needs.
For many tasks eliciting environmental values, the demands of providing
the required information are considerable. To help meet these needs, the
article draws on cognitive psychology to offer two general strategies
for communicating efficiently. It concludes with a discussion of the
research philosophies for coping with these communication challenges.

**Bounding Study Impacts**

When evaluating research proposals, IRBs typically assume that the
impacts of participation end once the direct interaction with
participants is over. At most, a study might have a small "tail," as
participants reflect on the experience. The investigators will, of
course, reflect on it for a much longer time. However, their conclusions
are assumed to have no effect on participants. There are, of course,
exceptions, some quite familiar to environmental researchers. For
example, finding a high level of pollution can undermine a community's
economic viability \-- or get its plight taken seriously. Assessing
individuals' medical status can reduce their insurability -- or set them
on the road to treatment.

Given the limited benefits of participating in research, significant
intrusions on individuals' real lives need to be taken seriously, even
if their probability is very small. Recognizing how difficult it is for
potential participants to anticipate such possibilities, IRBs take
special care to insure that investigators circumscribe the full set of
possible impacts. Whether the conclusions drawn from a study of
environmental values matter depends on how seriously investigators treat
their results. At one extreme, lie [gist]{.underline} studies:
Investigators claim no more than to have received a general answer to a
general question. As a result, their conclusions depict no more than
respondents' overall orientation toward a topic. That should have little
effect on respondents, beyond some slight change in their reputation.

At the other extreme, lie [contract]{.underline} studies: Investigators
seek respondents' consent to a specific transaction, in order to make
claims like "people are willing to pay 7% for 'green' products" or
"people are willing to incur an 0.2% rise in unemployment to meet our
Kyoto obligations." If respondents are "held to their word," then their
participation in a study will encourage the adoption of policies that
fit investigators' interpretation of their responses. Participants will
benefit, if these are policies that they actually desire. Participants
will lose, if they mean something different than what the investigators
claim. Participants will also lose if investigators extract a firmer
commitment than they intended. Interpreting a gist response as a
contractual one is akin to a politician's invoking general election
results as a mandate for a specific piece of legislation -- unless that
was, indeed, the focus of the campaign.

These possibilities can be seen in the familiar context of contingent
valuation studies. Participation provides citizens with a unique
opportunity to express their views on specific environmental issues.
That chance presumably encourages many people to accept, and stick with,
CV tasks despite their intellectual demands. If their responses are
interpreted as intended, then their efforts will be rewarded. If not,
then they will have signed an IOU that they were not really willing to
pay (or an invoice that they did not really intend to deliver, when
someone else will pay the contingent value).[^1]

Thus, informed consent for participating in value-elicitation studies
requires understanding how one's responses will be used. With gist
studies, participants can err by expecting too much, from investigators
who never intended to take their responses seriously. With contract
studies, participants can err by unwittingly making stronger commitments
than they intend. In either context, they can err by making commitments
that are interpreted differently than they had intended.

Investigators' main tool for reducing these risks is communication,
making it clear what they will do with the responses that they collect.
The first part of that task should be simple: placing a study
appropriately on the gist-contract continuum.[^2] The main obstacle here
might be overcoming respondents' preconceptions, regarding how seriously
to take their task.

Communicating the commitment that investigations will impute to a pure
gist response should also be relatively easy. Anyone in the same
social-linguistic culture should agree that a given response expresses
how much respondents care, in a general sense about, say, the
environment, their personal health, or campaign finance reform. No one
would legitimately interpret strong general concern as showing equally
strong support for any specific proposal.

Conveying the content of a contract question is also easy \-- if the
issue is familiar. As the vote approaches on a widely debated
referendum, many citizens should know just what commitment their vote
implies. Political processes succeed when they achieve this sort of
informed consent of the governed.

Communicating a complex novel issue can be another matter entirely. We
have proposed a framework for fully specifying such contracts (Fischhoff
& Furby, 1988). It views value elicitation as a request to accept or
reject a proposed *transaction*. An informed response means
understanding the proposed transaction's *good*, *payment*, and
associated *social context*. Informed consent for participating in a
contract study means expecting such understanding to be made possible.
If that expectation is violated, then respondents face a difficult
choice. They can rescind their consent; however, that is often difficult
in the socially coercive conditions of an interview or experiment. They
can press further for understanding; however, the interviewer or
experiment often cannot or will not meet their needs. Or, they can
reduce their expectations, treating the task as somewhere between gist
and contract \-- and hoping that investigators similarly scale back
their own expectations.

Unfortunately, some transactions require investigators to specify \--
and respondents to absorb \-- many details. In those cases, it is not
hard to imagine respondents deciding that it is not worth fighting for a
better understanding. They provide the most contract-like responses that
they can, then get on with their lives. When respondents don't make a
fuss, investigators might feel comfortable treating their answers as
contractual. Given the transaction costs of producing complete answers,
individual respondents will have acted reasonably. However, the group
that they represent may suffer, if investigators treat its members as
having made commitments that they never intended.

We like our framework, and have used it in various contexts (Fischhoff,
1993, 1997; Fischhoff & Furby, 1986; Fischhoff et al., 1993; Fischhoff &
Welch, in press; Furby & Fischhoff, 1988). However, analogous problems
arise with whatever framework one uses. They reflect the complexity of
the underlying choices, not our particular approach to organizing them.
Elements appear in the framework because their specification could
affect respondents' values. Unless investigators make them clear,
respondents are free to make them up. A feature might "go without
saying," if respondents make the same assumption [and]{.underline} agree
with the investigators' implicit specification. If not, then different
respondents may be answering different questions than one another, and
than the one attributed to them. Investigators will, then, violate the
conditions of the informed consent agreement. Respondents who chanced
across research reports could justifiably argue, "How dare you interpret
my responses that way? I meant nothing of the kind."

Investigators often work hard to make their tasks realistic and
engaging. However, even fully engaged respondents may not understand
some critical information. The next two sections present, in turn,
analytical and practical approaches to making this task more tractable.
The first offers a way to prioritize information, so that investigators
can focus on the facts that matter most, as well as evaluate their own
success. The second offers two strategies for communicating those
critical features most efficiently, by building on respondents'
cognitive strengths.

**A Materiality Standard for Informed Consent**

As mentioned, IRBs often require investigators to revise informed
consent procedures. From the IRB perspective, this happens so frequently
because investigators have difficulty maintaining an independent
perspective on their own activities. As a result, they must be
constrained by individuals better able to empathize with participants
(i.e., the IRB members). From the investigators' perspective, however,
revisions are requested so often because they lack clear guidance on
what to say and do. For example, many have had the experience of being
forced to use consent forms that seemed too complicated for subjects to
understand. As a result, approval often feels like a trial-and-error
process, with an unsatisfactory resolution.

The model for research informed consent is provided by medical informed
consent. Physicians, like investigators, sometimes do things that are
ill-advised. They also sometimes receive unclear guidance about what
they should do. There are two general standards for medical informed
consent. About half the United States have a professional standard; in
them, physicians must tell their patients what their colleagues tell
theirs. The other half have a materiality standard; physicians must tell
their patients whatever is "material" to their decisions. Materiality
might be operationalized in terms of value-of-information analysis
(Fischhoff, 1985). Namely, information is material, to the extent that
its provision affects the expected utility of patients' choices.

Decision analysis shows how to calculate such informativeness for many
classes of decisions (Clemen, 1993; Raiffa, 1968). We have adapted it in
an approach to medical informed consent decisions (Merz et al., 1993).
Our approach postulates a population of patients with varying physical
conditions and values. It represents the physical conditions by
probability distributions over possible consequences. It represents the
values by distributions of tradeoffs across those consequences. Monte
Carlo sampling produces a set of hypothetical patients, each
characterized by a combination of physical states and values. The
expected utility of the decision is calculated for each such patient,
with and without knowledge of the probability of each possible
consequence. Informed consent procedures should focus on the facts whose
provision makes the greatest difference in expected utility.

One worked example looks at carotid endartarechtomy. Scraping out the
artery leading to the brain can reduce the risk of stroke for patients
with arteriosclerosis there. However, having a sharp object in one's
neck can also cause problems. So can major surgery itself. The analysis
creates a population of hypothetical patients, for whom the surgery
would be a rational choice, if there were no side effects. It finds that
about 15% of them should decline the surgery, upon learning about the
probability of death in surgery. Another 5% should decline if told the
risk of stroke caused by the surgery. An additional 3% should be
dissuaded by hearing the risk of facial paralysis. Many other side
effects are possible. However, few would affect many choices. Thus,
although the set of potentially relevant facts is large, the set of
critically relevant ones is quite small.[^3]

This example assumes patients who know nothing without being told. That
seems reasonable for an unfamiliar surgery. In other cases, people will
know some things already. Then, the analysis starts with their prior
probabilities over the possible consequences. It examines the impact of
improving those assessments to the extent possible, by conveying current
scientific results. In effect, the analysis produces a supply curve for
information. Other things being equal, people should learn first the
things that will make the biggest difference to their choices. The
adequacy of an informed consent procedure reflects how far the
individual has gone from existing to perfect knowledge. Where perfection
is impossible, one must choose an acceptable level of misunderstanding.

Our procedure assumes that people are \-- or would like to be \--
rational decision makers, who make perfect use of what they hear
(Viscusi & O'Connor, 1984). That makes analysis easier. However, one
could also create more behaviorally realistic models. One
straightforward approach is to assume that people understand messages
incompletely. Doing so might shift communication priorities, toward more
readily grasped facts. It would also lower the asymptotic level of
understanding, possibly below the acceptable level. Incorporating
uncertainty in people's assessment of their own relevant values is
another way to make the modeling more realistic. Adding noise (or error)
to the integration process is yet another.

As with our framework for specifying transactions, this analytical
approach is not the only one possible. However, the issues that it
addresses need to be accommodated in any approach to facing informed
consent needs explicitly. In the medical context, generic analyses are
an inferior substitute for personalized ones (McNeil et al., 1978); if
at all possible, people should weigh their own values and prospects. In
the research context, generic analyses are essential. Researchers cannot
wait until a choice has been fully presented, before asking research
participants whether they want to learn about it. Rather, they must
promise that "people like yourself \[typically, always, often\] feel
comfortable answering these questions in a \[gist, contractual\] sense."
The novelty of many research questions ensures that perfect prior
information is impossible. As a result, informed consent forms routinely
assert subjects' right to quit at any time, as the task is revealed.
Unfortunately, the social dynamics of research settings often make it
hard to exercise this right. Thus, when subjects are "informed,"
uncoercerced "consent" may no longer be that possible.

In order to make such promises, researchers must understand the
cognitive demands of their tasks and the cognitive capacities of their
audiences. Experience and intuition provide some rough guidance.
However, they cannot be reviewed, nor focus design efforts very
specifically, nor protect against unrealistic expectations. A "curse of
cleverness" can lead us to prize novel questions, capturing nuances that
have eluded our colleagues. However, those questions are even more novel
for respondents, unfamiliar as they are with the research context and
syntax.

This section has described a general strategy for analyzing the
tractability of evaluation tasks. It has three steps. The first is
formally specifying the evaluation task. The second is creating a formal
model of the decision facing research participants. The third is
assessing their chance of identifying the response that best represents
their values. That chance will depend on the knowledge that they bring
with them, or manage to absorb from the task. Its asymptotic value
depends on how much certainty is possible, even with unlimited study.
Its acceptable value is a personal matter. People might, for example,
seek greater confidence when the decision is important and when others
know more than they do (hence might take advantage of them).

Analyzing tasks in these terms allows investigators to assess and
fulfill their duty to inform. They can tell potential respondents how
far a good-faith effort should get them, in understanding the task.
Given that assessment, and knowledge of how their responses will be
used, respondents can decide whether participation is worth the gamble.
Such disclosure might also make it easier, for those who accept the
research challenge, to drop out when things aren't working. As
mentioned, although always offered, the exit option is difficult to
exercise.[^4] Fuller disclosure might also encourage participants to try
harder, by enlisting them as collaborators.

**Two Strategies for Communicating Efficiently**

How far respondents get into their evaluation task depends, obviously,
on how well investigators communicate the information critical to it.
That process can be approached in a piecemeal or holistic way. The
former strategy involves conveying the most important individual facts
in isolation. The latter involves aggregating facts into chunks that can
be understood as meaningful wholes. These are the standard options for
increasing the capacity of any communication channel. This section
describes ways to pursue them in this context.

[Piecemeal strategies: Identifying a supply curve for
information]{.underline}

If people can understand anything they are told, then communications
should focus on the facts that they most need to know, but currently do
not. With completely novel tasks, one might reasonably assume total
ignorance. In that case, the formal analysis of priorities dictates the
order of communication -- as with the carotid endartarechtomy example.

One possible complication is that some things will prove
incomprehensible, despite the best design efforts. Those facts can be
cut from the communication list. Not wasting time on them will improve
the efficiency of communication, while acknowledging a lower level of
maximum understanding. One possible simplification is that some things
are redundant; knowing about one allows people to predict others. If so,
then only one needs to be communicated. With artificial tasks, any two
features can appear together (e.g., the sizes and probabilities of gains
in laboratory gambles), allowing no efficiencies. However, in the
real-world, features may be correlated; and, more important for more
communication purposes, they may be seen as correlated (e.g., the sizes
and probabilities of gains in real-life gambles).

The question, then, is what do people know already, or spontaneously
infer. A standard method for determining intuitive understanding is the
think-aloud protocol (Ericsson & Simon, 1994; Schriver, 1989).
Individuals like the intended respondents describe what comes into their
minds, as they read a minimal version of the task. The interviewer asks
them to elaborate on each topic that they raise, as a check on whether
they really understand it. Especially if the topic is in the news,
people will often have a sort of "false fluency," using the words, but
with a limited grasp of their meaning (e.g., "safe sex"; McIntyre &
West, 1992). If they routinely and confidently make a particular
inference, then that fact goes without saying \-- hence can be omitted
in the actual study. Given fixed channel capacity, omitting an item
allows one to go one item further down a prioritized list of facts.

If the actual study encourages reflection and clarification (Schkade &
Payne, 1994), then the think-aloud protocol may evoke cognitive
processes approximating of those actual respondents. One might, then, be
reasonably confident that the study will communicate effectively if the
pretest did. "Manipulation checks" provide the conventional way to
measure such success. Placed at the end of the study, they ask
respondents to recall and interpret critical elements. If they reveal
that a respondent answered a different question them the one that the
researchers asked, that answer would have to be deleted or reinterpreted
(Fischhoff & Welch, in press).

If the actual study involves a structured survey, then it will, by
design, lack the probing interactive character of a think-aloud
protocol. As a result, there is more chance of communication failure,
even when pretests of the final form went well. Manipulation checks,
again, provide a way to measure communication success. A possible
intermediate design stage is an experimental test of the structured
survey, determining whether people are sensitive in it to the features
that attracted attention in the interactive format (Fischhoff, 1994,
1996; Quadrel, 1990). If think-aloud subjects claim that a feature is
very important, then survey subjects should respond differently to
versions with different values for that feature. This is, of course, the
logic of embedding tests. However, it applies equally well to any
feature of the specification. Such tests could include both between- and
within-subject designs, considering their respective strengths and
weaknesses (Keren, 1993).

[Holistic strategies: Completing mental models]{.underline}

People can absorb information more quickly if they can organize it into
"chunks," clusters of facts that can be processed as units. Mnemonists
have mastered this skill to a high art. They can create chunks from
virtually any set of facts, by integrating its members into highly
flexible, standard templates. Ordinary people tend to organize
information into domain-specific "mental models." If they are accurate
and appropriate, then these mental models provide a ready-made platform
for integrating the features of a task. Invoking them should exploit
what people know already, allowing task features to go without saying,
while making it easier to absorb new features. On the other hand,
existing mental models can interfere with the processing of features
that do not fit so well and invoke misconceptions that the investigator
never stated.

Psychology has developed many approaches to studying mental models
(Gentner & Stevens, 1983; Hirschfeld & Gelman, 1994; Johnson-Laird,
1983; Newell & Simon, 1972; Rouse & Morris, 1986), each adapted to
particular kinds of problem. We have developed a method that seems
useful for complex, loosely bounded health, safety, and environmental
problems. It attempts to capture the potential messiness of lay
cognitions, but in a way that makes contact with scientific analyses of
the same problems. We have applied its basic logic to a variety of
topics. These include radon in homes, climate change, cryptosporidium in
water supplies, HIV/AIDS, nuclear energy sources in space, and
electromagnetic fields from 60 Hz fields. Each application raises new
modeling issues, and requires collaboration with substantive experts in
a new domain (Bostrom et al., 1992; Fischhoff, Bostrom & Quadrel, 1997;
Fischhoff, Riley, Kovacs & Small, 1998; Morgan et al., in preparation).

An application begins by creating an "expert model" of the problem,
capturing its formal structure as it relates to the focal policy
question. To the extent possible, we make these models computationally
tractable, even if we lack many of the requisite parameter estimates.
Doing so increases the opportunities for formal analyses of information
needs, as well as sharpening our own thinking. We then conduct
structured, open-ended interviews, eliciting respondents' beliefs about
the domain, trying to get as close as possible to their natural way of
thinking and talking. The interviews begin with a request to "tell me
whatever you know or have heard about\..." Follow-up questions ask
respondents to elaborate on everything they say, and then on their
elaborations. Once they lose momentum, respondents are asked more
specific questions about the general topics in the expert model. For
example, if they have not raised the possibility of remediation efforts,
then they are asked "what can be done about\...?"

All this design work is costly, but so is making major, but avoidable
policy mistakes. We pursue it in the belief that pursuing design in an
explicit, deliberate way will hold overall costs down. It should create
a cumulative record of empirical studies to guide future design, perhaps
even provide partial solutions to some new applications. Those studies
would, naturally, draw on the basic research literatures on related
topics (e.g., survey design, risk communication, environmental
perception), further substantiating results, and perhaps attracting
additional researchers.

**Eliciting Environmental Values When Communication Is Difficult**

The approaches advocated here should not be entirely foreign. Rather,
they attempt to formalize what investigators are trying to do anyway.
Any conscientious design process asks: What do respondents most need to
know? What do they know already? How can we go furthest in bridging the
gap? Is that good enough? How do we let potential respondents know what
to expect, and what is at stake?

Justifying the effort would require having confidence that it would pay
the benefits of (a) improving communication, (b) improving peer (and
IRB) review of the informed consent process, (c) creating a science of
communication, of interest in its own right, and (d) integrating value
elicitation research with other environmental and social science.

Facing these issues directly means being willing to take "no answer" for
an answer, when respondents cannot master our questions. However, it
need not increase our chances of having to do so, if better design helps
respondents to provide informed answers and informed consent for their
use.

Mounting such a research program would require a research philosophy
with the following elements:

\(a\) Viewing validity as a continuous, rather than a dichotomous
variable. Arguing matters of degree is the norm in conventional science.
However, it runs a foul of the short-term needs of agencies and
litigants hoping to make categorical claims. In principle, having more
realistic expectations from research should promote effective policy
making, as should having a more stable, consensual scientific base. In
practice, though, a long-term perspective may be impossible with
decision-driven science, inhibiting research progress on those topics.

\(b\) Accepting the relevance of a broad range of basic sciences. Like
other problem-driven research, environmental value elicitation strongly
reflects the perspectives of the subdisciplines in at the beginning.
However, conscientiously they pursue the work, they are likely to
provide too narrow a base for solving such complex problems.
Accommodating additional perspectives requires deliberate institutional
design (e.g., genuinely collaborative projects, and not just
interdisciplinary review). The present proposals require collaboration
among disciplines.

\(c\) Recognizing the full context of our tasks for respondents. If we
want to take their responses seriously, then we need to expect (and
encourage) them to do the same for our tasks. They should care about
what we are going to do with their responses, and whether we have
leveled with them. Involved respondents should be willing to read
between the lines for missing details and engage the task socially and
emotionally, as well as cognitively. Protest responses are often taken
as a measurement failure, reflecting respondents' inability or
unwillingness to answer our questions. However, they may also be taken
as a staging success, reflecting respondents who care enough to express
their frustration, rather than "just saying something."

In a recent study, we offered respondents several opportunities to
withdraw from the study, as they learned more about the tasks and
potential uses of their responses (Welch, Fischhoff & Frederick, 1998).
As their involvement increased, their willingness to have their
responses used decreased (from 78% to 59%). However, there was
relatively little change in the legitimacy that they attached to the
elicitation exercise -- just in that of their own responses. On-line
experiments often make it easy (mechanically, if not psychologically)
not to submit one's responses at the end. Eliciting more about people's
comfort levels with different uses of their responses might improve our
ability to interpret them appreciatively. It also provides a way of
fulfilling our informed consent obligations, where it is hard to explain
fully in advance what people are getting themselves into.

The hope of the present perspective is that a broader perspective will,
over time, simplify, rather than complicate matters. Fragmented,
incomplete tasks place unnatural demands on respondents. That should
reduce their ability to produce meaningful answers, as well as their
willingness to try. Often, however, scientific and policy concerns force
us to ask unfamiliar questions. It stands to reason that it will take
multiple disciplines to create and integrate the pieces of that
environment. Respondents should appreciate the efforts of investigators
who present many details. They should be happier still if those are also
the most relevant details. Providing more context might mean providing
more cues on how to respond strategically. However, it might also create
the trusting relations that encourage honest responses. Ideally (and
idealistically), respondents and investigators will work together to
estimate these critical values.

**References**

Bostrom, A., Fischhoff, B., & Morgan, M.G. (1992). Characterizing mental
models of hazardous processes: A methodology and an application to
radon. [Journal of Social Issues]{.underline}, [48]{.underline}(4),
85-100.

Clemen, R. (1993). [Decision analysis]{.underline}.

Ericsson, K.A. & Simon, H.A. (1994). [Protocol analysis: Verbal reports
as data (2^nd^ edition)]{.underline}. Cambridge, MA: MIT Press.

Fischhoff, B. (1985). Cognitive and institutional barriers to \"informed
consent.\" In M. Gibson (Ed.), [To breathe freely: Risk, consent, and
air]{.underline} (pp. 169-185). Totowa, NJ: Rowman & Allanheld.

Fischhoff, B. (1991). Value elicitation: Is there anything in there?
[American Psychologist]{.underline}, [46]{.underline}, 835-847.

Fischhoff, B. (1993). Transaction analysis: A framework and an
application to insurance decisions. [Journal of Risk and
Uncertainty]{.underline}, [7]{.underline}, 53-69.

Fischhoff, B. (1994). What forecasts (seem to) mean. [International
Journal of Forecasting]{.underline}, [10]{.underline}, 387-403.

Fischhoff, B . (1996). The real world: What good is it? [Organizational
Behavior and Human Decision Processes]{.underline}, [65]{.underline},
232-248.

Fischhoff, B. (1997). What do psychologists want? Contingent valuation
as a special case of asking questions. In R. J. Kopp, W.W. Pommerehne,
and N. Schwarz (Eds.), [Determining the value of nonmarketed
goods]{.underline} (pp. 189-217). New York: Plenum.

Fischhoff, B., Bostrom, A., & Quadrel, M.J. (1997). Risk perception and
communication. In R. Detels, J. McEwen & G. Omenn (Eds.), [Oxford
textbook of public health]{.underline} (pp. 987-1002) London: Oxford
University Press.

Fischhoff, B. & Furby, L. (1986). [A review and critique of Tolley,
Randall et al. \"Establishing and valuing the effects of improved
visibility in the Eastern United States\"]{.underline}. ERI Technical
Report 86-8. Eugene, OR: Eugene Research Institute.

Fischhoff, B. & Furby, L. (1988). Measuring values: A conceptual
framework for interpreting transactions. [Journal of Risk and
Uncertainty, 1]{.underline} 147-184.

Fischhoff, B., Quadrel, M.J., Kamlet, M., Loewenstein, G., Dawes, R.,
Fischbeck, P., Klepper, S., Leland, J., & Stroh, P. (1993). Embedding
effects: Stimulus representation and response modes. [Journal of Risk
and Uncertainty]{.underline}, [6]{.underline}, 211-234.

Fischhoff, B., Riley, D., Kovacs, D., & Small, M. (1998). What
information belongs in a warning? A mental models approach. [Psychology
and Marketing]{.underline}.

Fischhoff, B., & Welch, N. (in press). Construal processes in preference
elicitation. [Journal of Risk & Uncertainty]{.underline}.

Furby, L. & Fischhoff, B. (1988). [Specifying subjective evaluations. A
critique of Dickie et al.\'s interpretation of their contingent
valuation results for reduced minor health symptoms]{.underline}. USEPA
Cooperative Agreement #CR814655-01-0. Eugene, OR: Eugene Research
Institute.

Gentner, D. & Stevens, A.L. (Eds.) (1983). [Mental models]{.underline}.
Hillsdale, NJ: Erlbaum.

Hirschfeld, L.A. & Gelman, S.A. (Eds.) (1994). [Mapping the
mind]{.underline}. New York: Cambridge University Press.

Johnson-Laird, P.N. (1983). [Mental models]{.underline}. New York:
Cambridge University Press.

Keren, G. (1993). Between- or within-subjects design: A methodological
dilemma. In G. Keren & C. Lewis (Eds.), [A handbook for data analysis in
the behavioral sciences]{.underline} (pp. 257-272). Hillsdale, NJ:
Lawrence Erlbaum Press.

McIntyre, S. & West, P. (1992). What does the phrase \"safer sex\" mean
to you? Understanding among Glaswegian 18 year olds in 1990.
[AIDS]{.underline}, [7]{.underline}, 121-126.

McNeil, B.J., Weichselbaum, M.D. & Pauker, S.G. (1978). Fallacy of the
five-year survival rate in lung cancer. [New England Journal of
Medicine]{.underline}, [289]{.underline}, 1397-1401.

Merz, J., Fischhoff, B., Mazur, D.J., & Fischbeck, P.S. (1993).
Decision-analytic approach to developing standards of disclosure for
medical informed consent. [Journal of Toxics and Liability]{.underline},
[15]{.underline}, 191-215.

Morgan, M.G., Fischhoff, B., Bostrom, A., & Atman, C. [Risk
communication: The mental models approaches]{.underline}. Book in
preparation.

Newell, A. & Simon, H.A. (1972). [Human problem solving]{.underline}.
Englewood Cliffs, NJ: Prentice Hall.

Quadrel, M.J. (1990). [Elicitation of adolescents\' risk perceptions:
Qualitative and quantitative dimensions]{.underline}. Ph.D.
dissertation. Carnegie Mellon University.

Raiffa, H. (1968). [Decision analysis: Introductory lectures on choices
under uncertainty]{.underline}. Reading, MA: Addison- Wesley.

Rouse, W.B. & Morris N.M. (1986). On looking into the black box:
Prospects and limits in the search for mental models. [Psychological
Bulletin]{.underline}, [100]{.underline}, 349-363.

Schkade, D. & Payne, J.W. (1994). How people respond to contingent
valuation questions: A verbal protocol analysis of willingness to pay
for an environmental regulation. [Journal of Environmental Economics and
Management,]{.underline} 26, 88-109.

Schriver, K.A. (1989). Evaluating text quality: The continuum from
text - focused to reader - focused methods. [IEEE Transactions of
Professional Communication]{.underline}, [32]{.underline}, 238-255.

Viscusi, K. & O'Connor, C. (1984). Adaptive responses to chemical
labeling. [American Economic Review]{.underline}, [74]{.underline},
942-956.

Welch, N., Fischhoff, B., & Frederick, S. (1998). [The social context of
contingent valuation transactions]{.underline}. Under editorial review.

**Endnotes**

[^1]: This article assumes that people mean what they say, in order to
    focus on the possibility that they do not know what they are talking
    about.

[^2]: This continuum parallels that of basic values-articulated values,
    which I have advanced as a way to think about investigators'
    expectations regarding what respondents have to offer (Fischhoff,
    1991). I chose alternative terms here as a way of emphasizing how
    these issues look to potential study participants. Their primary
    concern is how their responses will be used. That usage depends, in
    part, on where investigators place their responses on the basic
    values-articulated values dimension.

[^3]: In this case, the communication itself might be fairly easy. The
    probabilities of the three focal side effects are large enough to be
    readily comprehended (and not millionths of a per cent). The events
    of death and stroke should be familiar, especially to candidates for
    carotid endartarechtomy, who face this prospect in their everyday
    lives. As a result, informed consent briefings could focus on facial
    paralysis, helping patients understand just how it would affect
    their lives and might be treated.

[^4]: Speculatively, one reason for the vehemence (or deviance) of many
    protest responses is participants' anger at having to fight their
    way out of their task.