Document ID: EPA-HQ-ORD-2009-0645-0003
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2009-08-18T04:00Z

Using Probabilistic Methods to Enhance the Role of 

Risk Analysis in Decision Making 

Managers' Summary

Prepared by the EPA Risk Assessment Forum Working Group

Co-leads:

Bob Hetes, Marian Olsen, Kathryn Gallagher, Cynthia Stahl, Rita Schoeny 

And the RAF PRA Technical Panel

This document is a draft for peer review. It has not been formally
released by the U.S. Environmental Protection Agency and should not at
this stage be construed to represent Agency policy. It is being
circulated for comments on its technical merit and policy implications.
Mention of trade names or commercial products does not constitute
endorsement or recommendation for use.

This document was produced by a Technical Panel of the EPA Risk
Assessment Forum.  The authors drew on their experience in doing
probabilistic assessments and interpreting them to improve risk
management of environmental and health hazards.  Interviews,
presentations, and dialogues with risk managers conducted by the
Technical Panel have contributed to the insights and recommendations in
this summary and the associated White Papers.Using Probabilistic
Methods to Enhance the Role of 

Risk Analysis in Decision Making 

Manager’s Summary

EPA has been called upon by numerous advisory bodies such as the Science
Advisory Board and U.S. National Academy of Sciences to incorporate
probabilistic risk information into the Agency decision making process. 
Probabilistic risk assessment (PRA) is a group of techniques that
provide a range and likelihood estimate for one or more steps of hazard,
exposure or risk, rather than a single point estimate.  A Risk
Assessment Forum PRA technical panel has been formed from
representatives of Offices and Regions who conduct PRA.  The panel has
developed several products to promote enhanced use of PRA, including a
white paper describing PRA and its utility and application in Agency
decisions, and a compendium of case studies.  The panel is developing
Agency resources such as a clearinghouse of case studies, best
practices, and resources.  Seminars are being developed to raise general
knowledge of how these tools can be used, and act as a precursor for
future training.  The purpose of this paper is to present general
concepts and principles of PRA, to describe how PRA can improve the
bases of Agency decisions, and to provide illustrations of how PRA has
been used in risk estimation and to describe the uncertainty in risk
decision making.  

Why should I care about PRA?  Why is it important to risk managers?

The use of PRA is often a major recommendation in reviews of EPA
products and procedures (e.g., Science Advisory Board review of EPA
practices in 2006, NAS review of the Dioxin Reassessment, OMB’s
Circular A-4 and Updated Principles for Risk Analysis).  The Agency has
some basic guidance, as well as some program-specific procedures and
applications for PRA.  The enhanced use of PRA and characterization of
uncertainty would respond to outside recommendations and potentially
enhance the overall transparency and quality of EPA assessments.  These
approaches would provide additional tools to address specific challenges
faced by managers and improve confidence in Agency decisions. 
Specifically, PRA can inform decision makers about specific segments of
the population at risk, not just the mean (average) or extreme values. 
A PRA can also confirm or support the conclusions of a deterministic
risk estimate.  Having this information can be important to risk
managers if a different decision might be made when the upper or lower
ends of the range of estimated exposures, doses, or risks are used.

What is PRA?  How does it compare with current practice?

A basic characteristic of PRA is that it does not generate a single
point estimate but rather produces a range and likelihood that a
particular exposure, dose, or effect will occur.  Probabilistic risk
assessment, in its simplest form, is a group of statistical techniques
that allow analysis of variability and uncertainty to be incorporated
into exposure and/or risk assessments.  As mentioned above, this kind of
information can be critical in risk management decisions - especially if
one were interested in a specific portion of the population and the
likelihood of exposure or risk is known.

What are common challenges facing EPA decision makers?

  SEQ CHAPTER \h \r 1 EPA Offices and Regions are faced with similar
mandates; basic attributes critical to decisions are (1) understanding
whom or what we are protecting, and (2) what is the appropriate degree
of confidence in the estimated protection provided by a particular
decision.  A further complication is the fact that decisions are often
time-sensitive and need to be made based on the current state of
knowledge.  Health and environmental impacts of environmental exposures
cannot be isolated and directly measured.  Therefore, risk assessment
methods have been developed to estimate health and ecological risks
based on available data and information.  The risk decision making may
also consider other assessments conducted to address other factors, such
as economic impacts of risk management actions.  Uncertainty can be
introduced into any assessment at any step in the process, even when
using the most accurate data with the most sophisticated models.  As a
result, EPA must always make decisions in the presence of uncertainty.  

How can PRA and enhanced characterization of uncertainty and variability
help?

There are several ways in which various types of PRA can enhance risk
management decision making.  First, a sensitivity analysis (either
deterministic or probabilistic) can determine if more refined
information about the distribution and range of data can have a
substantial effect on the choice of decision options.  If so, there are
two ways that PRA can improve decisions.  First, using PRA one can
explicitly address the elements of a risk-based decision – whom or
what we are protecting and with what degree of confidence.  Secondly,
the use of PRA can characterize the inherent uncertainties and the
impact of those uncertainties on the decision.  When uncertainty is
present, PRA can better inform a decision, increase the transparency of
the inputs to the decision, and assist in selecting among various
management options.  Below we describe how PRA can:  

1) enhance EPA decisions by providing more information about the
possible impacts of alternative regulatory decisions;  

2) provide clarity on whom we are protecting and the confidence in the
estimates of protection provided by a given regulatory decision;

3) allow for more detailed comparison of alternative risk management
options in terms of estimated impacts on both protection and costs; and 

4) improve the overall confidence in specific decisions.  

As a manager what do I need to know about PRA?  

	We all have common baseline experiences with probability, uncertainty
and variability such as weather forecasting, political polls, or climate
change predictions.   PRA can be used on many levels or degrees of
sophistication to support or improve decisions.  The following sections
describe the basics of PRA in more detail, how it can be used to support
decisions, and what to consider in pursuing a PRA or using PRA results. 

How does EPA typically address variability and scientific uncertainty?

EPA cannot perform a time- and resource-intensive risk assessment for
every situation and EPA decision, and therefore, must be strategic in
determining whether more intensive assessments are needed.  When EPA
does not explicitly quantify the degree of confidence in a risk
estimate, the Agency attempts to increase the confidence that risk is
not underestimated by using default options to deal with uncertainty and
variability.  As depicted below, EPA most often uses risk assessment
methods that rely on default assumptions using a combination of point
values -- some conservative (high parameter values that are more likely
to overestimate risk) and some typical or average.  These values are put
into a model or single model structure and the risks are calculated as
shown below in Equation 1.

Equation 1:

    

This approach typically produces a single estimate of risks (e.g., 10-6
cancer risk; that is increased risk of 1 in a million additional
cancers); these may be referred to as deterministic assessments or point
estimates of risk.  These point estimates are useful, particularly in
screening assessments, but the inherent uncertainties are not fully
quantified.  These inherent limitations can affect EPA decisions in the
following ways:

Inability to explicitly characterize the basic elements of EPA decisions
-- whom or what are we protecting or with what degree of confidence

Inability to more realistically or accurately compare across alternative
risk management choices (e.g., cleanup levels, permit levels,
regulations, actions) or risks due to different levels of conservatism

Decreased ability to make tradeoffs or appropriate balance between
benefits and costs

Liability to criticism and debate of being overly conservative and
unrealistic, or of providing inadequate protection; these criticisms
frequently cannot be directly answered, which reduces the credibility of
EPA decisions.     

What are variability and uncertainty and their relevance in risk
assessment? 

	The 2004 Staff Paper Risk Assessment Principles and Practices ( 
HYPERLINK "http://www.epa.gov/osa"  www.epa.gov/osa ) and the PRA White
Paper #1 describe uncertainty and variability in some detail and can be
referred to for more information.   

	Variability refers to the inherent natural variation, diversity, and
heterogeneity across time and/or space, or individuals within a
population.  While we can better describe and understand variability in
the world, or a particular system, it is unavoidable and cannot be
reduced.  Variability is present in all aspects of the source to effect
continuum (Figure 1 below): 

Figure 1. Source-to-Effect Continuum

in how pollutants are released (e.g., effectiveness of emission
controls), 

influenced by environmental conditions once released (e.g., meteorology
– wind and precipitation), 

exposure to receptors  (e.g., inhalation or ingestion rates), 

effect (e.g., endpoint, health status, genetic susceptibility).  

An example of variability is the amount of water consumed by a
population.  For example, if we conducted a survey of 1,000 people and
asked them how much water they consumed, we might have the following
distribution (i.e., plot of the data in Figure 2) which shows that 5% of
the population consumes 2.41 liters of water/day or higher while the
average individual consumes 1.4 liters of water/day , based on studies
identified in the Exposure Factors Handbook (EPA, 1997).

Figure 2.  Probability Distribution of Drinking Water Intake

 

	Uncertainty refers to imperfect knowledge or lack of precise knowledge
of the real world, either for specific values of interest or in the
description of the system.  While numerous schemes for classifying
uncertainty have been proposed, most focus on two broad categories.     

Parameter uncertainty refers to uncertainties in specific estimates or
values used in a model.  

Model uncertainty refers to the gaps in scientific knowledge or theory
that is required to make accurate predictions.  

From a risk manager’s perspective, both are important in that
variability is related to our understanding of whom or what we are
protecting and uncertainty relates to our confidence in the estimate and
the level of protection. 

How does PRA address variability and uncertainty?

By contrast, as depicted below PRA uses distributions of values that
reflect variability and/or uncertainty in parameters and/or models; the
result is an overall probability statement of the risk (e.g., what are
the risks to the average or mean individual and high end individual such
as the 95th percentile as illustrated in Equation 2 below).  This
hypothetical example illustrates the approach for estimating individual
risk.  In some cases exposure and health risk are estimated for the
entire population or particular subgroups of the population.  

Equation 2:

  

 

What is the impact of uncertainty on decisions?

When uncertainty is present -- where data and information are incomplete
or are inadequate -- making informed decisions is more difficult and
there is greater potential for decision errors.  In the case of
environmental regulations, specific decisions may either lead to over-
or under-regulation compared to decisions that could be made with
perfect information.  Setting an environmental standard that is too lax
may threaten public health, while a standard that is unnecessarily
stringent may impose a significant economic cost for a marginal gain in
public health and environmental protection. 

What key questions can be asked or considered by decision makers? 

The PRA Technical Panel conducted several dialogues with EPA decision
makers, asking them what questions arise when they are faced with the
task of making decisions in the presence of uncertainty.  The following
questions represent typical concerns.

How representative or conservative is the estimate, (e.g., what is the
variability around an estimate)?

What are the major gaps in knowledge, and what are the major assumptions
used in the assessment? How reasonable are the assumptions?

Would my decision be different if the data were different?  Would
additional data collection and research likely lead to a different
decision? How long will it take to collect the information, how much
would it cost, and would the resulting decision be significantly
different?

Will the use of additional resources, such as a probabilistic approach,
impact the decision making in a timely manner (i.e., better characterize
uncertainties, better identify variability, impact timelines, etc.)?

What are the liabilities/consequences of making a decision under the
current level of knowledge and uncertainty?  

What is the percentile of the population to be protected?

How do the different alternative decision choices and the interpretation
of uncertainty and variability impact the target population?

How can PRA help inform decisions? 

PRA can provide information to decision makers on specific questions
related to uncertainty and variability.  For questions of uncertainty
and to minimize the likelihood of unintended consequences, PRA can help
provide the following types of information:

 

Characterize uncertainty in estimates (what is the degree of confidence
in the estimate?).  That is, could the prediction be off by a factor of
2, a factor of 10, or a factor of 1,000?

Identify the critical parameters and assumptions which most impact or
influence a decision and the risk assessment;

Identify the “tipping points” where the decision option chosen would
be different if the risk estimates were different, or a different
assumption were valid;

Estimate the likelihood that critical data values exist or the validity
of assumptions;

Estimate the degree of confidence in a particular decision and/or the
likelihood of specific decision errors 

Estimate, (in conjunction with other techniques, such as sensitivity
analysis and value of information) the possibility of alternative
outcomes with additional information, or estimate trade-offs related to
different risks or risk management decisions;

Identify impact of additional information on decision making considering
the cost and time to obtain the information and resulting change in
decision (that is, value of information).

For consideration of variability, PRA for example can provide the
following types of information for exposures:

Explicitly define the exposures for various sectors of the population
(whom are we trying to protect?) That is, will the regulatory action
keep 50% of the population, 90% of the population 99.9%  or some other
fraction of the population below a specified exposure, dose, or risk
target? 

Provide information including the variability in the exposures among the
population, and information on the percentile of the population that is
being evaluated in the risk assessment (i.e., people who consume a glass
of water/ day or people who consume a gallon of water/day).  This
information is helpful in addressing comments: 

from the regulatory community on conservatism of EPA’s risk
assessments;

from the community regarding concerns whether their particular exposures
were assessed in the risk assessment, 

about whom or what is being protected by a risk management action, and 

whether and what additional research may be needed to reduce
uncertainty.

PRA helps inform decisions by identifying the alternatives available to
the   HYPERLINK "http://en.wikipedia.org/wiki/Decision_maker" \o
"Decision maker"  decision maker , the   HYPERLINK
"http://en.wikipedia.org/wiki/Uncertainty" \o "Uncertainty"  uncertainty
 they face, and by providing evaluation measures of outcomes (often
referred to as decision analysis). Uncertainties are often represented
as   HYPERLINK "http://en.wikipedia.org/wiki/Probabilities" \o
"Probabilities"  probabilities  or   HYPERLINK
"http://en.wikipedia.org/wiki/Probability_distributions" \o "Probability
distributions"  probability distributions , in graphs or numerically.

A few hypothetical examples of the types of risk management questions
which can explicitly be addressed through PRA are illustrated in Figure
3 below. 

 

Figure 3.  What Questions Can PRA Address?

What are some of the limitations of PRA?

	PRA relates to the application of probabilistic techniques to one or
more phases of the risk assessment paradigm, including hazard
characterization, exposure, toxicity, and/or risk assessment.  Data may
not be available to support probabilistic techniques at all of these
stages in the same assessment, requiring the risk assessor to continue
to apply some deterministic science-policy assumptions and conversions. 
If science-policy assumptions, or default values for parameters are
applied to a PRA, they should to be clearly articulated in the
dissemination of results. PRA typically requires more time to develop
than a deterministic assessment, but these techniques fit into a
graduated, or tiered approach, to risk analysis.  Additional limitations
are that:

PRA is generally more data intensive, requiring additional financial,
time and analytic resources to obtain the necessary statistical
distribution input data for each aspect of the risk assessment.  It is
anticipated that more routine incorporation of probabilistic designs in
risk assessment and its supporting research could reduce this cost
differential.

PRA techniques have been most successful on the exposure aspect of human
health risk assessment. 

	The dissemination of a statistical distribution or probability output
number should be carefully related to the quality and coverage of the
input statistical distribution data, otherwise the PRA results could
lead to a false sense of accuracy.

PRA can be used to characterize the uncertainty and variability in
situations with limited data.   As yet, there is not extensive
experience using PRA to characterize the range of effects or the
dose-response for populations, including sensitive populations and
life-stages.

What is EPA’s experience in PRA?

	In the past, EPA has usually, but not always, relied on deterministic
or point estimates to evaluate risk; (e.g., 10-6 or one in a million
risk of cancer).  However, the use of PRA to evaluate uncertainty and
variability in risk assessments is increasing.  These efforts are varied
across Programs and Regions, as well as in complexity and applications. 
Many PRA applications focused on specific elements of a risk assessment
(e.g., exposure), variability, or uncertainty.  The document Case Study
Examples of the Application of Probabilistic Risk Assessment in EPA
Regulatory Decision Making contains summary examples of PRAs that have
been conducted to support regulatory decisions and/or regulatory impact
analyses.  A few examples of PRA use in EPA include: 

EMAP program:  The Office of Research and Development (ORD) developed
and Office of Water (OW) adopted applied probabilistic sampling
techniques to evaluate nation’s aquatic resources under CWA Section
305(b)

Hudson River PCB-Contaminated Sediment Site: Region 2 evaluated the
variability in risks to anglers who consume recreationally caught fish
contaminated with PCBs from sediment contamination in the Hudson River.

Chromated Copper Arsenate Risk Assessment: ORD and the Office of
Pesticide Programs (OPP) conducted a probabilistic exposure assessment
of children’s exposure (addressing both variability and uncertainty)
to arsenic and chromium from contact with CCA-treated wood playsets and
decks. 

Evaluating Ecological Effects of Pesticide Uses: OPP developed a
probabilistic model which evaluates acute mortality levels in generic
and specific ecological species for user-defined pesticide uses and
exposures.  

PM2.5 Health Impacts: The Office of Air and Radiation (OAR) used expert
elicitation to more completely characterize, both qualitatively and
quantitatively, the uncertainties associated with the relationship
between reduction in PM2.5 and benefits of reduced PM2.5-related
mortality. 

EPA’s experience with PRA includes not only individual assessments or
applications but also the development of general guidance and policies
such as these:

Policy for Use of Probabilistic Analysis in Risk Assessment (1997),

Guiding Principles for Monte Carlo Analysis (1997), and

Risk Assessment Guidance for Superfund, Volume III – Part A. Process
for Conducting Probabilistic Risk Assessment.

How should a risk manager approach a PRA?  What should a manager
consider? 

There is a range of probabilistic risk analysis techniques that may be
useful to support environmental decisions.  Communication between the
risk managers and risk assessors is critical for clear definition of the
specific needs of the decision maker and the questions to be addressed
by the PRA. The risk assessor and risk manager should evaluate the types
of techniques appropriate to meet the goals of the assessment and
establish a process for completing and reviewing the PRA in a
cost-effective and timely manner. The dialogue should continue until the
PRA is completed to everyone’s satisfaction.  

When should we consider doing PRA?

	Conducting a sensitivity analysis within the context of the decision
can help managers determine whether having such information is critical
and that the time and resources spent to perform PRA are warranted in
specific cases.  PRA may not be needed when the decision is routine,
legislatively mandated, or a standard methodology is prescribed. 
Furthermore, PRA may not be needed when there is high confidence in the
data and models used to support the decision.  On the other hand
planning and scoping discussions or a preliminary analysis may indicate
that information from a PRA may be critical or influence the risk
management decision.  Some examples include :

A specified target level of protection in a population is identified
(e.g., 95th percentile), and it is necessary to demonstrate that this
goal is met;

Significant equity issues are raised by variation in risks among the
exposed population of concern;

Screening level point estimates of risk are higher than the level of
concern;

Uncertainty is high, and decisions are contentious or have large
resource implications;

Specific critical risk estimates and assumptions point to different risk
management alternatives;

Scientific rigor and quality of the assessment is critical to
credibility of the EPA decision. 

	

What is the right level of analysis?  

As is the case for risk assessment in general, approaches to PRA and
specific analytical methods may vary dramatically in terms of complexity
and resource implications.  The concept of iterative or tiered analyses
to address this continuum is widely accepted in risk assessment, and the
same principle applies to PRA as well.  There is a wide range of methods
and approaches to PRA, of varying complexity and rigor, which can be
applied for different purposes ranging from sensitivity analysis to
integrated analysis of uncertainty and variability.  The goal is to
choose a level of detail and refinement for an analysis appropriate to
the overall objectives of the decision and the types of available data
and analyses needed to support decisions.  Early and continued dialogue
between risk manager and risk assessor is critical to developing a clear
understanding of overall project objectives, needs of the decision
maker, timing, and how PRA may play a role.  These discussions should
focus on deciding the following: 

(1) whether or not the risk assessment, in its current state, is
sufficient to support risk management decisions (a clear path to exiting
the process is available); and 

(2) if the assessment is determined to be insufficient, whether or not
progression to a higher level of complexity would provide a sufficient
benefit to warrant the additional effort of performing a PRA. 

If I am going to use PRA what are things to consider?

	If one decides that the use of PRA would provide valuable information
in support of a decision, some other things to consider in moving
forward include:  

Resources needed to develop the PRA and review the document, 

Expertise of EPA staff to develop a PRA or review a PRA submitted by a
contractor or member of the regulated community;

Data availability and format (e.g., electronic or paper copy) to develop
distributions to include in the PRA, 

Time needed for the development and review of the analyses, 

Funding, either intramural or extramural that may be necessary for
development and review of the document, 

Peer-review including either internal and/or external review which has
time and cost implications, and

Communicating results to the scientific community, Agency executives,
stakeholders and the general public.

What are the resources and costs needed to conduct a PRA?

PRA can be expected to require more time, effort and resources than
standard default-based deterministic assessments.  The costs and
resources will vary depending on the tool or approach that is selected.
That is, there is a continuum of PRA approaches to choose from, ranging
from simple approaches such as sensitivity analyses to complex
approaches such as two-dimensional Monte Carlo analyses   In some cases,
simple sensitivity analyses, which may require limited time and risk
assessor resources, can be conducted in-house.  More sophisticated
analyses may require specific expertise or use of specific tools or
models.  Proper application of probabilistic methods requires not only
software and data, but also guidance and training for both analysts
using the tools as well as for managers and decision makers tasked with
interpreting and communicating the results.  While increases in
resources needed to conduct a probabilistic assessment can be expected,
the development of standardized approaches and/or methods can lead to
the routine incorporation of PRA in Agency approaches and greatly
reduced costs in future applications.

Does PRA require more data than conventional approaches? 

	In general, PRA requires more data than conventional approaches because
distributions of values rather than single values are used.  How much
more data is required is often the topic of debate in the technical
community.  Minimum data needs vary depending on the analytical approach
used; empirical-based (observational or frequentist) methods have
significant data requirements compared to so called subjective methods. 
However, some of the data that would be applied in a frequentist
approach may already be available as part of the underlying data set
used in standard deterministic analyses.  As a result, PRA can be
applied in most cases, as long as methods used are appropriate for the
available body of evidence and data.  

Communication of PRA Results to the Manager and Community.  Does
presentation of results matter?  

	The lack of familiarity with PRA presents a challenge in effectively
presenting results to managers, stakeholders, and the public.  Many view
PRA as a highly technical discipline utilizing sophisticated mathematics
and requiring extensive training to understand.  Single point estimates
are easy to grasp for most people, based in part on familiarity with the
approach over the history of EPA. While some people initially have
difficulty interpreting probability distributions of values, we all have
common baseline experiences with probability, uncertainty and
variability (e.g., weather forecasting); these could be used to frame
discussion of results.  It is not necessary to understand the underlying
mathematics or even to include results as full distributions.  Results
can be distilled to the critical essence or decision-meaningful value of
interest.  

The audience and its range of knowledge and expertise must be considered
in developing materials for effective communication.  It is helpful when
a decision is made to conduct a PRA to consider early explanation or
training of the community, managers, and others in the basic principles
before the final decision is presented.  Alternatively, it may be
helpful to present the results of the PRA with the point estimate to
provide context for the results.

How can I get more information on PRA?

	This document provides a general overview and basic concepts to
establish some familiarity and a foundation for further education on
PRA.  The white paper entitled "Using Probabilistic Methods to Enhance
the Role of Risk Analysis in Decision making – Uses and Case Studies
in EPA" provides more of a detailed discussion of PRA and EPA's
experience with it. There are numerous additional resources for more
detail on PRA.  Additionally, the RAF PRA technical panel has been
tasked with developing resources to facilitate the understanding and
implementation of PRA.  It is developing an electronic clearinghouse of
resources (policies, guidance, tools, case studies) as well as specific
training seminars which will soon be available.  More information on the
PRA technical panel and the clearinghouse can be found at   HYPERLINK
"http://www.epa.gov/raf"  www.epa.gov/raf  or on the Environmental
Science Connector.  See also the EPA source for links to risk assessment
methods and policies:   HYPERLINK "http://www.epa.gov/risk" 
www.epa.gov/risk . 

GLOSSARY

Analysis. Examination of anything complex to understand its nature or to
determine its essential features (WHO IPCS Risk Assessment Terminology)

Assessment. The analysis and transformation of data into policy-relevant
information that can assist decision making and action.

Assessment end point. 1. Quantitative or qualitative expression of a
specific factor or metric with which a risk may be associated, as
determined through an appropriate risk assessment. 2. An explicit
expression of the environmental value that is to be protected,
operationally defined by an ecological entity and its attributes. For
example, salmon are valued ecological entities; reproduction and age
class structure are some of their important attributes. Together, salmon
“reproduction and age class structure” form an assessment end point.

Ecological risk assessment. An ecological risk assessment evaluates the
potential adverse effects that human activities have on the plants and
animals that make up ecosystems. The risk assessment process provides a
way to develop, organize, and present scientific information, so that it
is relevant to environmental decisions. When conducted for a particular
place, such as a watershed, the ecological risk assessment process can
be used to identify vulnerable and valued resources, prioritize data
collection activity, and link human activities with their potential
effects.

Ecosystem. The interacting system of a biological community (plants and
animals) and its nonliving environment.

Environment. The sum of all external conditions affecting the life,
development, and survival of an organism.

Expert elicitation.  Expert elicitation (EE) is a systematic process of
formalizing and quantifying, typically in probabilistic terms, expert
judgments about uncertain quantities.  

Frequentist (or frequency) probability. A view of probability that
concerns itself with the frequency of events in a long series of trials,
or is based upon a data set.

Inputs. Quantities that are input to a model.

Model. 1. A set of constraints restricting the possible joint values of
several quantities. 2. A hypothesis or system of belief regarding how a
system works or responds to changes in its inputs. 3. A mathematical
function with parameters that can be adjusted so the function closely
describes a set of empirical data. A mechanistic model usually reflects
observed or hypothesized biological or physical mechanisms and has model
parameters with real-world interpretation. In contrast, statistical or
empirical models selected for particular numerical properties are best
fits to data; model parameters may or may not have real-world
interpretation. When data quality is otherwise equivalent, extrapolation
from mechanistic models (e.g., biologically based dose-response models)
often carries higher confidence than extrapolation using empirical
models (e.g., logistic models).

Modeling. 1. Development of a mathematical or physical representation of
a system or theory that accounts for all or some of its known
properties. Models often are used to test the effect of changes of
components on the overall performance of the system. 2. Use of
mathematical equations to simulate and predict real events and
processes. 3. Development or application of conceptual or graphical
methods to depict the structure and organization among major elements of
the system to be modeled.

Parameter. 1. A variable, measurable property whose value is a
determinant of the characteristics of a system (e.g., Temperature,
pressure, and density are parameters of the atmosphere.). 2. A constant
or variable term in a function that determines the specific form of the
function but not its general nature, as “a” in f(x) = ax, where
“a” determines only the slope of the line described by f(x). 3. A
variable entering into the mathematical form of any probability
distribution model such that the possible values of the variable
correspond to different distributions.

Probability. 1. Frequentist approach/ The frequency with which samples
are obtained within a specified range or for a specified category (e.g.,
the probability that an average individual with a particular mean dose
will develop an illness). 2. Bayesian approach. The degree of belief
regarding the different possible values of a quantity or event.

Probabilistic risk analysis. Application of a computational method,
based on a randomized sampling of available data or information or
probabilities obtained from experts, to produce a probability
distribution to more fully describe the data than selecting a single
point in the distribution, e.g., the mean.

Risk. 1. Risk includes consideration of exposure to the possibility of
an adverse outcome, the frequency with which one or more types of
adverse outcomes may occur, and the severity or consequences of the
adverse outcomes if such occur. 2. The potential for realization of
unwanted, adverse consequences to human life, health, property, or the
environment. 3. The probability of adverse effects resulting from
exposure to an environmental agent or mixture of agents. 4. The combined
answers to (1) What can go wrong? (2) How likely is it? and (3) What are
the consequences?

Risk analysis. 1. A process for identifying, characterizing,
controlling, and communicating risks in situations where an organism,
system, subpopulation, or population could be exposed to a hazard. Risk
analysis is a process that includes risk assessment, risk management,
and risk communication (WHO). 2. A detailed examination, including risk
assessment, risk evaluation, and risk management alternatives, performed
to understand the nature of unwanted, negative consequences to human
life, health, property, or the environment; an analytical process to
provide information regarding undesirable events; the process of
quantification of the probabilities and expected consequences for
identified risks.

Risk assessment. 1. A process intended to calculate or estimate the risk
to a given target organism, system, subpopulation, or population,
including the identification of attendant uncertainties following
exposure to a particular agent, taking into account the inherent
characteristics of the agent of concern, as well as the characteristics
of the specific target system (WHO). 2. The evaluation of scientific
information on the hazardous properties of environmental agents (hazard
characterization), the dose-response relationship (dose-response
assessment), and the extent of human exposure to those agents (exposure
assessment) (NRC, 1983). The product of the risk assessment is a
statement regarding the probability that populations or individuals so
exposed will be harmed and to what degree (risk characterization)
(USEPA, 2000). 3. Qualitative and quantitative evaluation of the risk
posed to human health or the environment by the actual or potential
presence or use of specific pollutants.

Risk-informed decision making. An approach to decision making in which
insights from probabilistic risk analyses are considered with other
insights and factors.

Risk management. A decision-making process that takes into account
environmental laws, regulations, political, social, economic,
engineering, and scientific information, including a risk assessment, to
weigh policy alternatives associated with a hazard.

Scenario. 1. An outline or model of an expected or supposed sequence of
events. 2. A set of facts, assumptions, and inferences about how
exposure takes place and regarding how exposures translate into adverse
effects that aides the analyst in evaluating, estimating, or quantifying
exposures and risks. Scenarios might include identification of
pollutants, pathways, exposure routes, and modes of action, among
others.

Sensitivity analysis. A study of how the variation in data inputs
(including inputs to models) affect the outputs of a model or choice
among potential decision options.

Levels. Refers to various hierarchical levels of complexity and
refinement for different types of modeling approaches that can be used
in risk assessment. A deterministic risk assessment with conservative
assumptions is an example of a lower level type of analysis that can be
used to determine whether exposures and risks are below levels of
concern. Examples of progressively higher levels include the use of
deterministic risk assessment coupled with sensitivity analysis, the use
of probabilistic techniques to characterize either variability or
uncertainty only, and the use of two-dimensional probabilistic
techniques to distinguish between but simultaneously characterize both
variability and uncertainty.

Two-dimensional probabilistic analysis. A modeling approach in which
inter-individual variability in exposure and risk is characterized using
frequency distributions, and in which uncertainty in the estimates of
statistics of the frequency distributions (e.g., the mean, median,
standard deviation, percentiles) are characterized using probability
distributions.

Uncertainty. Occurs because of a lack of knowledge. It is not the same
as variability. For example, a risk assessor may be very certain that
different people drink different amounts of water but may be uncertain
about how much variability there is in water intakes within the
population. Uncertainty often can be reduced by collecting more and
better data, whereas variability is an inherent property of the
population being evaluated. Variability can be better characterized with
more data but it cannot be reduced or eliminated. Efforts to clearly
distinguish between variability and uncertainty are important for both
risk assessment and risk characterization, although they both may be
incorporated into an assessment.

Uncertainty analysis. A detailed examination of the systematic and
random errors of a measurement or estimate; an analytical process to
provide information regarding uncertainty.

Value of information. A quantitative measure of the value of knowing the
outcome of an uncertain variable prior to making a decision. Decision
theory provides a means for calculating the value of both perfect and
imperfect information. The former value, informally known as the value
of clairvoyance, is an upper bound for the latter. Obtaining meaningful
value-of-information measurements requires an awareness of important
restrictions (concerning the nature of free will) on the validity of
this kind of information.

Variability. Refers to true heterogeneity or diversity, as exemplified
in natural variation . For example, among a population that drinks water
from the same source and with the same contaminant concentration, the
risks from consuming the water may vary. This may result from
differences in exposure (i.e., different people drinking different
amounts of water and having different body weights, different exposure
frequencies, and different exposure durations), as well as differences
in response (e.g., genetic differences in resistance to a chemical
dose). Those inherent differences are referred to as variability.
Differences among individuals in a population are referred to as
inter-individual variability, differences for one individual over time
is referred to as intra-individual variability.

EXTERNAL PEER REVIEW DRAFT – DO NOT CITE OR QUOTE

 PAGE   1