**A Standard Multi-Attribute Representation of Policy Options**

In 1994, the Office of Science and Technology Policy asked several
research groups to create a standard procedure for evaluating
risk-related policy options, in a way that could be comparable across
decisions, programs, and agencies. The Carnegie Mellon proposal took
advantage of research in the "psychometric paradigm" of risk perception
research (Fischhoff et al., 1978; Slovic, 1987, 2001). That research
found that policy attributes tend to be correlated (e.g., involuntarily
imposed risks tend to have inequitable distributions of outcomes,
technologies with catastrophic potential tend to be uncertain). As a
result, one can capture much of the variance in the evaluation of
options by characterizing each by a vector of attributes that includes a
representative of each cluster (or dimension) of attributes. We proposed
characterizing policies by a standard vector of such representative
attributes. Because the values evoked by correlated attributes may
differ, it would be good to have more than one representative (e.g.,
both voluntariness and equity, both catastrophic potential and
uncertainty).

Such a representation could be used in various contexts. We proposed one
based on EPA's extensive experience in risk-ranking, reflected in the
many state-level (and other) exercises that it conducted during the late
1980s and early 1990s (Environmental Protection Agency, 1993). Each
exercise developed its own multi-attribute characterization of risks.
That increased the face validity of the proceedings. However, making
each exercise very responsive to participants came at a price. It
severely limited the comparability of the rankings done by different
groups, as well as the opportunities for rigorously developing the
attributes used by each group. Fischhoff (1995) and Morgan et al. (1996)
describe the rationale for our proposal for a standard multiattribute
representation, focused on how it could be used for risk ranking.

With primary funding from NSF and EPA, our research group developed a
detailed procedure for applying this approach. It includes how to create
such representations and communicate them comprehensibly. It recognizes
that such valuations are constructive, in the sense that individuals
must articulate specific preferences from their general values
(Fischhoff, 1991, 2005). People cannot be expected to "read off" answers
to these novel questions. The obvious risk in constructive valuation
processes is producing biased results. We addressed this concern by
having participants triangulate between group and individual
perspectives, as well as between decomposed and holistic judgments.

We developed and evaluated the method in an experimental test bed,
focused on the diverse risks arising in a hypothetical middle school
(e.g., accidents, infectious disease, possible carcinogens,
interpersonal violence). Our empirical tests involved diverse citizens,
including participants in workshops held by the Harvard Center for Risk
Analysis. The results showed good construct validity (DeKay et al.,
2001; Florig et al., 2001; Morgan et al., 2001). On the strength of
these results, we extended the method to characterize ecological risks.
As points of departure, we took many of the sets of attributes that
previous researchers had used to characterize ecological health and
changes (Willis et al., 2004). In addition to developing a canonical
presentation of such risks, we used this new test bed to understand
further the properties of this evaluation procedure, namely the
relationship between group and individual analyses (Willis et al.,
2005).

In parallel, the British government has been engaged in a sustained
effort to manage risks in a systematic way. This effort has produced
guidance on risk analysis, cost-benefit analysis, and public
consultation (e.g., Health and Safety Executive, 2001; HM Treasury,
2003, 2004). This year, it issued guidance on how to consider public
concerns that are not readily monetized, complementing economic
evaluations, in a way that is compatible with its commitment to informed
participation (HM Treasury, 2005). To this end, it adopted a variant of
our procedure, in the form of a Concern Assessment Tool adapted in
conjunction with staff from several government agencies and in response
to input from a period of public comment. The proposal is meant to fit a
variety of consultative settings and budgets, so that even a simple
application will show good faith and structure the conversation, by
getting the broad set of issues on the table in a standard way.

We believe that it could be adapted to many of EPA's needs, consistent
with its comparable guidance.

**References**

DeKay, M., Florig, K., Fischbeck, P.S., Morgan, M.G., Morgan, K.M.,
Fischhoff, B., and Jenni, K.E. (2001). The use of public risk ranking in
regulatory policy. In P.S. Fischbeck & S. Farrow (Eds.), [Improving
regulation: Cases in environment, health, and safety]{.underline} (pp.
208---232). Washington, DC: Resources for the Future.

Environmental Protection Agency, (1993). [Guidebook to comparing risks
and setting environmental priorities]{.underline}. Washington: Office of
Policy Planning and Evaluation.

Fischhoff, B. (1991). Value elicitation: Is there anything in there?
[American Psychologist]{.underline}, [46]{.underline}, 835-847.

Fischhoff, B. (1995). Ranking risks. In [Risk: Health Safety &
Environment]{.underline}, [6]{.underline}, 189-200.

Fischhoff, B. (2005). Cognitive issues in stated preference methods. In
K-G. Mäler & J. Vincent (Eds.), [Handbook of Environmental Economics,
Vol.2]{.underline} (pp. 937-968). Amsterdam: Elsevier.

Fischhoff, B., Slovic, P., Lichtenstein, S., Read, S. & Combs, B.
(1978). How safe is safe enough? A psychometric study of attitudes
towards technological risks and benefits. [Policy Sciences]{.underline},
[8]{.underline}, 127-152.

Florig, H.K., Morgan, M.G., Morgan, K.M., Jenni, K.E., Fischhoff, B.,
Fischbeck, P.S., & DeKay, M. (2001). A deliberative method for ranking
risks (1): Overview and test bed development. [Risk
Analysis]{.underline}, [21]{.underline}, 913-922.

Health and Safety Executive. (2001). [Reducing risks protecting
people]{.underline}. London: HMSO.

HM Treasury. (2003). [The Green Book: Appraisal and evaluation in
central governance]{.underline}. London: HMSO.

HM Treasury. (2004). [Management of risks: Principles and concepts (the
Orange Book)]{.underline}. London: HMSO.

HM Treasury. (2005). [Managing risks to the public: Appraisal
guidance]{.underline}. London: HMSO.

Morgan, M.G., Fischhoff, B., Lave, L. & Fischbeck, P. (1996). A proposal
for ranking risks within federal agencies. In C. Davies (Ed.),
[Comparing environmental risks]{.underline} (pp. 111-147). Washington,
DC: Resources for the Future.

Morgan, K.M., DeKay, M.L., Fischbeck, P.S., Morgan, M.G., Fischhoff, B.,
& Florig, H.K. (2001). A deliberative method for ranking risks (2):
Evaluation of validity and agreement among risk managers. [Risk
Analysis]{.underline}, [21]{.underline}, 923-938.

Slovic, P. (1987). The perception of risk. [Science]{.underline},
[236]{.underline}, 280-285.

Slovic, P. (Ed.), (2001). [The perception of risk]{.underline}. London:
Earthscan.

Willis, H.H., DeKay, M.L., Fischhoff, B., & Morgan, M.G. (2005).
Aggregate and disaggregate analyses of ecological risk perceptions.
[Risk Analysis]{.underline}, [25]{.underline}, 405-428.

Willis, H.H., DeKay, M.L., Morgan, M.G., Florig, H.K., & Fischbeck, P.S.
(2004). Ecological risk ranking: Development and evaluation of a method
for improving public participation in environmental decision making.
[Risk Analysis]{.underline}, [24]{.underline}, 363-378.

Baruch Fischhoff, <baruch@cmu.edu>, December 19, 2005