# Risk Assessment Guidance for Superfund: Volume III - Part A, Process for Conducting Probabilistic Risk AssessmentEPA 540-R-02-002 OSWER 9285.7-45 PB2002 963302 www.epa.gov/superfund/RAGS3A/index.htm December 2001
Superfund



# Risk Assessment Guidance for Superfund: Volume III - Part A, Process for Conducting Probabilistic Risk Assessment

Office of Emergency and Remedial Response U.S. Environmental Protection Agency Washington, DC 20460

Printed on Recycled Paper

## D ISCLAIMER 

### This document provides guidance to EPA Regions concerning how the Agency intends to exercise its discretion in implementing one aspect of the CERCLA remedy selection process. The guidance is designed to implement national policy on these issues. Some of the statutory provisions described in this document contain legally binding requirements. However, this document does not substitute for those provisions or regulations, nor is it a regulation itself. Thus, it cannot impose legally-binding requirements on EPA, States, or the regulated community, and may not apply to a particular situation based upon the circumstances. Any decisions regarding a particular remedy selection decision will be made based on the statute and regulations, and EPA decision makers retain the discretion to adopt approaches on a case-by-case basis that differ from this guidance where appropriate. Interested parties are free to raise questions and objection about the substance of this guidance and the appropriateness of the application of this guidance to a particular situation, and the Agency welcomes public input on this document at any time. EPA may change this guidance in the future.

i

## A BOUT THE 

## R EVISION


### W HAT IT IS 

### EPA’s Process for Conducting Probabilistic Risk Assessment is anupdate of the 1989 Risk Assessment Guidance for Superfund (RAGS) . It is Volume III, anupdate to the existing two-volu me set of RAGS. Volume III: Part A provides policy and guidance on conducting probabilistic risk assessment for both human and ecological receptors.
W HO IT 

### ’ S FOR 

### RAGS Volume III: Part A is written primarily for risk asse ssors. Ris k assessment reviewers, remedial project managers , and risk managers involved inSuper fund site cleanup activities will also benefit from this addition to RAGS.
W HAT 

### ’ S NEW 

### RAGS Volume III: Part A provides gu idance on applying probabilistic analysis to both h uman heal th and ecological risk assessment. New informationand techniques are prese nte d that reflect the views of EPA Superfund prog ram. A t iered approach is described for determining the extentand s cope of th e modeling effort that is consistent with the risk assessment o bjectives, the data available, and the information that may be used to support remedial action decisions at Superfund hazardous waste sites. RAGS Volume III: Part A contains the following information:

• For the risk assessor— updated policies and guidance; discussion and examples of Monte Carlo modeling techniques for estimating exposure and risk.

• For the risk manager and the remedial project manager—an introductionto PRA, a chapter on communicating methods and resultsof PRA wit h the public, and a chapter on the role of PRA in decision making

.

ii

## T ABLE OF 

## C ONTENTS
Table of Contents

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Acronyms and Abbreviations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi
Preface ............................................................................. i
1.0 What is the Purpose of RAGS Volume 3 Part A?

........... . . . . . . . . . . . . . . . . . . . ii 2.0 What is Probabilistic Risk Assessment and how is it used in Risk Characterization?

.. ii 3.0 What are the Advantages and Disadvantages of PRA for Remedial Decisions?

...... iii 4.0 How is RAGS Volume 3, Part A Organized?

........... . . . . . . . . . . . . . . . . . . . . . . . iii 5.0 What are the Key Guiding Concepts in RAGS Volume 3: Part A ?

........... ...... iii References for Preface

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv Chapter 1 Overview of Probabilistic Approach to Risk Assessment .................... 1-1
1.0 Introduction

........... ............. ............ ............ .......... 1-1 1.1 The Role of Risk Assessment in Superfund

........... . . . . . . . . . . . . . . . . . . . . . . 1-4 1.1.1 Risk Assessment in the United States

........... . . . . . . . . . . . . . . . . . . . . . 1-4 1.1.2 Risk Assessment at EPA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5 1.1.3 Risk Assessment in Superfund

........... . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5 1.1.4 Probabilistic Risk Assessment and Its Role in Superfund

........... ..... 1-7 1.2 Basic Concepts of Probabilistic Risk Assessment

........... . . . . . . . . . . . . . . . . . . 1-9 1.2.1 What is PRA?

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10 1.2.2 What is a Monte Carlo Simulation?

........... . . . . . . . . . . . . . . . . . . . . . 1-13 1.2.3 Why is Variability Important in Risk Assessment? How is it Addressed by the Point Estimate and Probabilistic Approaches?

........... ....... 1-15 1.2.4 Why is Uncertainty Important in Risk Assessment? How is Uncertainty Addressed by the Point Estimate and Probabilistic Approaches?

......... 1-17 1.2.5 Reasonable Maximum Exposure at the High-end

........... .......... 1-21 1.3 Advantages and Disadvantages of Point Estimate and Probabilistic Approaches

... 1-21 1.4 Conducting an Acceptable PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24 1.4.1 Key Policies for Applying PRA at Superfund Sites

........... ........ 1-26 1.5 Organization of the Guidance

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-27 1.6 Next Steps for PRA Implementation

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30 References for Chapter 1

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-31 Exhibit 1-1 Definitions for Chapter 1

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2 Exhibit 1-2 Nine Criteria for Evaluation of Cleanup Alternatives

. . . . . . . . . . . . . . . . . . . . . . . . . . 1-6 Exhibit 1-3 Cancer and Noncancer Risk Models

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11 Exhibit 1-4 Use a PDF and CDF To Display:

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12 Exhibit 1-5 Quantifying Variability and Uncertainty

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-20 Exhibit 1-6 Advantages and Disadvantages of Point Estimate Approach

. . . . . . . . . . . . . . . . . . . 1-22 Exhibit 1-7 Advantages and Disadvantages of Probabilistic Risk Assessment

. . . . . . . . . . . . . . . 1-23 Figure 1-1 Example of a normal distribution that characterizes variability in adult body weight 1-12 Figure 1-2 Conceptual model of Monte Carlo analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14 Figure 1-3 Example of a probability distribution for risk illustrating the 95 th percentile and two different risk levels of concern (A and B)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-16Figure 1-4 Illustration of “Vertical” and “Horizontal” Confidence Intervals (or limits) on a risk estimate

............................................................ 1-19 Chapter 2 Workplan and The Tiered Approach ........... . . . . . . . . . . . . . . . . . . . . . . . . . 2-1
2.0 Introduction

.......................................................... 2-1 2.1 Workplan

............................................................ 2-1 2.2 Special Administrative Considerations in PRA

............................... 2-4 2.2.1 Scoping of PRA

................................................ 2-4 2.2.1.1 PRA Scope of Work for Fund-lead Sites

....................... 2-5 2.2.1.2 PRP Scope of Work for PRP-Lead Sites

....................... 2-5 2.2.2 Development of Probability Distributions

............................ 2-6 2.2.3 EPA Review of PRA Documents

................................... 2-6 2.2.4 Peer-Review

................................................... 2-6 2.2.5 Response to Comments on PRA

.................................... 2-6 2.2.6 Administrative Record

........................................... 2-7 2.2.7 Communication with Stakeholders

................................. 2-7 2.2.8 Communication with EPA Management

............................. 2-7 2.3 Overview of the Tiered Approach

......................................... 2-7 2.3.1 Getting Started

................................................ 2-11 2.3.2 Tier 1

........................................................ 2-14 2.3.3 Tier 2

........................................................ 2-15 2.3.4 Tier 3

........................................................ 2-17 2.3.5 Flexibility in Defining Tiers

...................................... 2-18 References for Chapter 2

............................................................ 2-19 Exhibit 2-1 Definitions for Chapter 2

................................................ 2-2 Exhibit 2-2 Examples of Important Contents of A PRA Workplan

......................... 2-4 Exhibit 2-3 Stakeholders Potentially Involved in EPA’s Decision-Making Process for PRA

..... 2-8 Exhibit 2-4 Typical Elements of Tier 1 Risk Assessment

............................... 2-11 Exhibit 2-5 Typical Elements of Tier 2 Risk Assessment

............................... 2-15 Exhibit 2-6 Typical Elements of Tier 3 Risk Assessment

............................... 2-17 Figure 2-1 Schematic Diagram of Tiered Approach

.................................... 2-9 Figure 2-2 Schematic diagram of deliberation/decision cycle in the tiered process for PRA

.... 2-10 Chapter 3 Using Probabilistic Analysis in Human Health Assessment ........... ....... 3-1
3.0 Introduction

.......................................................... 3-1 3.1 Characterizing Variability In Exposure Variables

............................. 3-1 3.1.1 Developing Distributions For Exposure Variables

...................... 3-5 3.1.2 Characterizing Risk Using PRA

................................... 3-6 3.2 Role of the Sensitivity Analysis

.......................................... 3-9 3.3 Exposure Point Concentration Term

...................................... 3-10 3.4 Characterizing Uncertainty in Exposure Variables

........................... 3-11 3.4.1 Parameter Uncertainty

.......................................... 3-11 3.4.2 Scenario and Model Uncertainty

.................................. 3-17 3.5 Example of PRA for Human Health

...................................... 3-17

ivReferences for Chapter 3

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-27 Exhibit 3-1 General Equation for Exposure

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1 Exhibit 3-2 Definitions for Chapter 3

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2 Exhibit 3-3 Equation for Cancer Risk

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Exhibit 3-4 Equation for Noncancer Hazard Quotient

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7 Exhibit 3-5 Using the Tiered Process for PRA Hypothetical Case Study for Human Health Risk Assessment

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18 Exhibit 3-6 Risk Equations

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23 Figure 3-1 Example of a frequency distribution for adult drinking water ingestion rates

....... 3-4 Figure 3-2 Hypothetical PRA results showing a PDF and CDF

. . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8 CDFs of risk based on Monte Carlo simulations described in Table 3-2.

.......... 3-16 Figure 3-3 CDFs of risk based on Monte Carlo simulations described in Table 3-2.

.......... 3-16 Figure 3-4 Figure 3-5 Site map for future wildlife refuge

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22 Figure 3-6 Results of sensitivity analysis for preliminary 1-D MCA (Tier 2)

. . . . . . . . . . . . . . . 3-26 Table 3-1 Methods for characterizing parameter uncertainty with Monte Carlo simulations

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-12 Table 3-2 Example of 1-D MCA and 2-D MCA

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14 Table 3-3 Concentrations in Surface Soil (mg/kg)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22 Table 3-4 Exposure Parameters used in Point Estimate Analysis.

. . . . . . . . . . . . . . . . . . . . . . . . 3-24 Table 3-5 Point Estimate Risks and Exposure Pathway Contributions

. . . . . . . . . . . . . . . . . . . . 3-24 Table 3-6 Input Distributions for Exposure Variables used in 1-D MCA for Variability

...... 3-25 Table 3-7 1-D MCA Risk Estimates using Preliminary Inputs

. . . . . . . . . . . . . . . . . . . . . . . . . . 3-25 Table 3-8 Exposure Duration Survey Results.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26 Table 3-9 Refined Point Estimate and 1-D MCA Risk Estimates

. . . . . . . . . . . . . . . . . . . . . . . . 3-26 Chapter 4 Probabilistic Analysis in Ecological Risk Assessment ........... ............ 4-1
4.1 Introduction

........... ............. ............ ............ .......... 4-1 4.1.1 Basic Approach for Performing Ecological Risk Assessments

............ 4-1 4.1.2 Predictive vs Observational Approaches

........... . . . . . . . . . . . . . . . . . . 4-6 4.1.3 Potential Advantages and Limitations of Probabilistic Methods in ERA

.... 4-7 4.1.4 Focus of This Chapter

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8 4.2 Deciding If and When to Use PRA in Ecological Risk Assessment

........... .... 4-8 4.2.1 Technical Considerations

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-9 4.2.2 Cost and Schedule Considerations

........... . . . . . . . . . . . . . . . . . . . . . . 4-11 4.3 Problem Formulation

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11 4.4 Modeling Variability in Exposure

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12 4.4.1 Characterizing Variability in Dose

........... . . . . . . . . . . . . . . . . . . . . . . 4-12 4.4.2 Characterizing Variability in Exposure Concentration

........... .... 4-15 4.5 Modeling Variability in Toxicity

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-15 4.5.1 Variability in Response Among Members of a Population

........... ... 4-15 4.5.2 Variability in Response Among Species

........... . . . . . . . . . . . . . . . . . 4-20 4.6 Modeling Variability in Risk

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-22 4.6.1 Variability in Hazard Quotient

........... . . . . . . . . . . . . . . . . . . . . . . . . . 4-22 4.6.2 Variability in Response

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-26 4.6.3 Joint Probability Curves

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-30

v4.7 Modeling Uncertainty in Ecological Risk Assessments

........... ............ 4-31 4.7.1 Uncertainty in Exposure

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-31 4.7.2 Uncertainty in Toxicity

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-32 4.7.4 Uncertainty in Response

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-34 4.7.3 Uncertainty in Hazard Quotient

........... . . . . . . . . . . . . . . . . . . . . . . . . 4-35 4.8 Interpreting Results of an Ecological PRA

........... . . . . . . . . . . . . . . . . . 4-37 4.9 Guidelines For Planning And Performing a Probabilistic ERA

........... ...... 4-39 4.9.1 Planning an Ecological PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-39 4.9.2 Evaluating an Ecological PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . 4-41 4.10 Example of the Tiered Process in ERA

........... . . . . . . . . . . . . . . . . . . . . . . . . . 4-41 References for Chapter 4

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-49 Exhibit 4-1 Definitions for Chapter 4

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3 Exhibit 4-2 Ecological Risk Assessment Guidance and Policy Directives

. . . . . . . . . . . . . . . . . . . 4-4 Exhibit 4-3 Modeling Variability in Response for a Dichotomous Endpoint

. . . . . . . . . . . . . . . . 4-17 Exhibit 4-4 Modeling Variability in Response for a Continuous Endpoint

. . . . . . . . . . . . . . . . . . 4-19 Exhibit 4-5 Hypothetical Species Sensitivity Distribution

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-21 Exhibit 4-6 Modeling Variability in a Dichotomous Response

. . . . . . . . . . . . . . . . . . . . . . . . . . . 4-27 Exhibit 4-7 Modeling Variability in a Continuous Response

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-29 Exhibit 4-8 Example Elements of a Workplan for Ecological PRA

. . . . . . . . . . . . . . . . . . . . . . . . 4-40 Exhibit 4-9 Checklist for Including a PRA as Part of the ERA

. . . . . . . . . . . . . . . . . . . . . . . . . . . 4-41 Exhibit 4-10 Refined Screening Point Estimate Inputs and Results

. . . . . . . . . . . . . . . . . . . . . . . . . 4-43 Exhibit 4-11 Screening Level PRA Calculations of HQ Distribution

. . . . . . . . . . . . . . . . . . . . . . . 4-45 Exhibit 4-12 Simulated Distribution of Responses

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-47 Figure 4-1 Ecological Risk Assessment Framework (U.S. EPA, 1992a)

........... ......... 4-1 Figure 4-2 Eight-step Ecological Risk Assessment Process for Superfund

. . . . . . . . . . . . . . . . . . 4-5 Figure 4-3 Example of cases where use of PRA may be helpful

. . . . . . . . . . . . . . . . . . . . . . . . . 4-10 Figure 4-4 Example Graphical Presentations of Dose Distributions.

. . . . . . . . . . . . . . . . . . . . . . 4-14 Figure 4-5 Example Comparison of Exposure Distribution to TRV.

. . . . . . . . . . . . . . . . . . . . . . 4-22 Figure 4-6 Example Distribution of HQ Values.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-23 Figure 4-7 Example Presentation of Species Sensitivity Distribution.

. . . . . . . . . . . . . . . . . . . . . 4-25 Figure 4-8 Example Joint Probability Curve.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-30 Figure 4-9 Example Presentation of Uncertainty in Exposure.

. . . . . . . . . . . . . . . . . . . . . . . . . . . 4-31 Figure 4-10 Example Presentation of Uncertainty in Response.

. . . . . . . . . . . . . . . . . . . . . . . . . . . 4-35 Figure 4-11 Example Presentation of Uncertainty in Exposure and TRV.

. . . . . . . . . . . . . . . . . . . 4-36 Figure 4-12 Example Presentation of Uncertainty in HQ Estimates

. . . . . . . . . . . . . . . . . . . . . . . . 4-37 Chapter 5 Probabilistic Risk Assessment and Preliminary Remediation Goals ........... 5-1
5.0 Introduction

........... ............. ............ ............ .......... 5-1 5.1 General Concepts Regarding EPCs and PRGs

........... . . . . . . . . . . . . . . . . . . . . 5-4 5.1.1 Sources of Uncertainty in the EPC

........... . . . . . . . . . . . . . . . . . . . . . . . 5-5 5.1.2 Pre- and Post-Remediation Exposure Point Concentrations

........... ... 5-6 5.1.3 Remediation Action Levels and 95% UCL Calculation Methods

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-7 5.1.4 Consideration of Risk from Acute Toxicity

........... . . . . . . . . . . . . . . . . 5-8

vi5.1.5 Characterization of Uncertainty in the EPC: Point Estimates and Distributions

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8 5.1.6 Multiple Chemicals

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9 5.2 When to Use PRA for Developing PRGs

........... . . . . . . . . . . . . . . . . . . . . . . . . 5-9 5.3 Methods for Developing PRGs

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10 5.4 Backcalculation

........... ............. ............ ............ ...... 5-10 5.4.1 Difficulties with Backcalculation

........... . . . . . . . . . . . . . . . . . . . . . . . 5-11 5.5 Iterative Methods

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-12 5.5.1 Iterative Reduction

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13 5.5.2 Iterative Truncation

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13 5.5.3 Example of Iterative Methods

........... . . . . . . . . . . . . . . . . . . . . . . . . . . 5-15 5.5.4 Multiple Exposure units and Iterative Methods

........... ............ 5-17 5.6 PRGs for Groundwater

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-18 5.7 PRGs for Other Contaminated Media

........... . . . . . . . . . . . . . . . . . . . . . . . . . . 5-19 5.8 Measurement of Attainment

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-21 5.9 Summary of Recommended Methods

........... . . . . . . . . . . . . . . . . . . . . . . . . . . 5-23 References for Chapter 5

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24 Exhibit 5-1 Summaries of Some Key Terms

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1 Exhibit 5-2 Definitions for Chapter 5

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2 Exhibit 5-3 Criteria for Iterative Truncation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14 Exhibit 5-4 Example of Iterative Methods

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16 Exhibit 5-5 Evaluation of Alternative RALs Using Iterative Truncation

. . . . . . . . . . . . . . . . . . . . 5-20 Figure 5-1 A hypothetical example of the use of iterative methods

........... ............ 5-12 Figure 5-2 Lognormal probability plot of soil concentrations, including 4 nondetects

........ 5-16 Figure 5-3 Hypothetical example of a mixed, bimodal distribution.

. . . . . . . . . . . . . . . . . . . . . . 5-18 Table 5-1 Soil sample

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-16 Table 5-2 Pre- and Post-Remediation EPCs (95% UCLs) for Chemical X in Surface Soil Samples

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-17 Table 5-3 Summary of Potential Methods for PRG Development by Environmental Medium

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-23 Chapter 6 Communicating Risks and Uncertainties in Probabilistic Risk Assessments .... 6-1
6.0 Introduction

........... ............. ............ ............ .......... 6-1 6.1 Stakeholder Involvement

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4 6.2 Communication and Presentation

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5 6.2.1 Communication of PRA With Concerned Citizens, Other Stakeholders, and Managers: An Overview

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-6 6.2.2 Steps for Communication of the Results of the PRA

........... ......... 6-7 6.3 Communicating Differences Between Point Estimate and PRA

........... ...... 6-10 6.4 Graphical Presentation of PRA Results to Various Audiences

........... ....... 6-11 6.4.1 Public Meeting

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11 6.4.2 EPA Senior Staff

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17 6.4.3 Press Releases

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19 6.5 Perception of Risk And Uncertainty

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19

vii6.6 Trust and Credibility

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21 6.7 Communication Issues for RPMs

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-21 References for Chapter 6

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23 Supplemental References

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-24 Exhibit 6-1 Definitions for Chapter 6

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2 Exhibit 6-2 Stakeholders Potentially Involved in the Decision-Making Process for PRA

........ 6-4 Exhibit 6-3 Important Steps for Communicating PRA Results

. . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7 Exhibit 6-4 Key Considerations in Developing Understandable Material

. . . . . . . . . . . . . . . . . . . . 6-8 Figure 6-1 Hypothetical PRA results showing a PDF and CDF

........... . . . . . . . . . . . . . . . 6-12 Figure 6-2 Results of a sensitivity analysis shown as a pie chart and tornado plot. .

.......... 6-16 Figure 6-3 The results of a 2-D MCA

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17 Table 6-1 Examples of Graphics for Communicating PRA Concepts in this Guidance Document

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-14 Chapter 7 Role of the PRA in Decision Making ........... . . . . . . . . . . . . . . . . . . . . . . . . . . 7-1
7.0 Introduction

........... ............. ............ ............ .......... 7-1 7.1 General Principles of Risk-Based Decision Making In Superfund

........... ..... 7-1 7.2 Interpreting A Risk Distribution

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-3 7.2.1 What Is A Distribution Of Risk And What Does It Look Like?

........... 7-3 7.2.2 What Is the RME Range?

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4 7.2.3 Relating the Risk Distribution to the Risk Management Goal for Human Health

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4 7.2.4 Relating the Risk Distribution to the Risk Management Goal for Ecological Risk Assessment

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-6 7.3 Factors to Consider in Choosing the Percentile for the RME

........... ......... 7-6 7.4 Uncertainty Associated with the Use of the 99.9 th Percentile

........... ........ 7-11 7.5 Moving From A PRG To A Remedial Goal

........... . . . . . . . . . . . . . . . . . . . . . 7-11 References for Chapter 7

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-15 Exhibit 7-1 Definitions for Chapter 7

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-2 Exhibit 7-2 Examples of Demographic, Cultural, and Behavioral Factors that Can Affect Exposure

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-7 Exhibit 7-3 Examples of Physical or Geographical Factors that Can Affect Exposure

.......... 7-7 Exhibit 7-4 Examples of Toxicity Considerations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7-9 Figure 7-1 Hypothetical PRA results showing a CDF for lifetime excess cancer risk.

......... 7-3 Figure 7-2 Example of a probability distribution for risk illustrating the 95 th percentile

........ 7-5 Figure 7-3 Box and whisker plots characterizing uncertainty in the RME

. . . . . . . . . . . . . . . . . . 7-10 Figure 7-4 Example of graphic showing variability in risk (i.e., RME range, or 90 th to 99.9 th percentiles) associated with different choices of PRG for plutonium in soil (pCi/g). 7-14 Figure 7-5 Example of graphic showing uncertainty in a 95 th percentile of the risk distribution associated with the same choices of PRG as Figure 7-4.

........... ............ 7-14

viiiAppendix A Sensitivity Analysis : How Do We Know What’s Important? ........... ..... A-1
A.0 Introduction

........... ............. ............ ............ ............ .... A-1 A.1.0 Utility of Sensitivity Analysis

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 A.2.0 Common Methods of Sensitivity Analysis

........... . . . . . . . . . . . . . . . . . . . . . A-10 A.2.1 Tier 1 Approaches

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-11 A.2.1.1 Percentage Contribution of Exposure Pathways to Total Risk

....... A-12 A.2.1.2 Inspection of Risk Equation

........... . . . . . . . . . . . . . . . . . . . . . . . A-13 A.2.1.3 Sensitivity Ratio (SR)

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . A-13 A.2.1.4 Sensitivity Score

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 A.2.2 Tier 2 Approaches

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 A.2.2.1 Graphical Techniques

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 A.2.2.2 Correlation Coefficients

........... . . . . . . . . . . . . . . . . . . . . . . . . . . A-21 A.2.2.3 Focusing on the RME Range of the Risk Distribution

........... .. A-27 A.2.2.4 Inspection

........... ............. ............ ............ A-27 A.3.0 Advanced Concepts in Sensitivity Analysis

........... . . . . . . . . . . . . . . . . . . . . A-28 A.3.1 Relating the Change in Risk to the Change in Input Variable X

........... ..... A-28 A.3.2 Normalized Partial Derivative

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-31 A.3.3 Regression Analysis: R 2 , Pearson r, and Partial Correlation Coefficients

........ A-32 A.3.3.1 Calculations of R 2 and Adjusted R 2

........... . . . . . . . . . . . . . . . . . . . . A-33 A.3.3.2 Relative Partial Sum of Squares (RPSS)

........... . . . . . . . . . . . . . . . . A-35 A.3.3.3 Spearman’s Rank Correlation Coefficient (Rho)

........... .......... A-36 References for Appendix A

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-37 Exhibit A-1 Definitions for Appendix A

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2 Exhibit A-2 Utility of Sensitivity Analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-3 Exhibit A-3 Some Key Indices of Sensitivity Analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-10 Exhibit A-4 Categories of Solutions for Sensitivity Ratios of Multipicative or Additive Equations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-17 Exhibit A-5 Simplifying Assumptions in Regression Analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . A-32 Figure A-1 Results of 2-D MCA in which parameters of input distributions describing variability are assumed to be random values.

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-9 Figure A-2 Scatterplots of simulated random values from a 1-D MCA of variability. The output from the model is a contaminant concentration in soil (C) that corresponds with a Figure A-3 Scatterplots of simulated random values from a 1-D MCA of variability for example in Figure A-4 Top panel - bar graph showing the r 2 values (square of Spearman rank correlation coefficient), a metric for the dependence of HI on exposure factors based on 1-D MCA for variability. Bottom panel - bar graph, sometimes referred to as “tornado plot”,
Figure A-5b Hypothetical 3-D response surface for Y given two input Figure A-5c Hypothetical 3-D response surface when Y is a linear function of two input variables: prescribed (fixed) level of risk for a hypothetical population

. . . . . . . . . . . . . . . . . . A-23 Section A.2.0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-24

showing rank correlation coefficient.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-25 Figure A-5a Hypothetical 2-D response surface for Y given one input variable: Y= F(X ).

...... A-29 variables: Y = f(X 1, X 2 )

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-30
Y= f

(X 1, X 2 )

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-30

ixTable A-1 Overview of Sensitivity Analysis Methods Applicable in Tiers 1, 2, and 3 of a PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-4 Table A-2 Point estimates and probability distributions for input variables used in the hypothetical example of HI associated with occupational exposure via water and soil ingestion. A-11 Table A-3 Percent contribution of exposure pathways to HI for the example in Section A.2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-12 Table A-4 Results of the Sensitivity Ratio (SR) approach applied to the hypothetical example of RME HI given in Section A.2.0. Includes both soil ingestion and tap water ingestion pathways

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-14 Table A-5 Results of the Sensitivity Ratio (SR) approach applied to the hypothetical example of RME HI given in Section A.2.0. Includes only tap water ingestion pathway

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-15 Table A-6 Examples of algebraic solutions to Sensitivity Ratio calculations for additive and multiplicative forms of risk equations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-17 Table A-7 Calculation of coefficient of variation (CV = SD / Mean) for the hypothetical example of RME HI given in Section A.2.0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-19 Table A-8 Results of the Sensitivity Score (Score) approach applied to the hypothetical example of RME HI given in Section A.2.0

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-20 Table A-9 Results of Tier 2 sensitivity analyses applied to hypothetical example in Section A.2.0: Pearson product moment correlations and Spearman rank correlations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-22 Appendix B Selection and Fitting of Distributions ........... . . . . . . . . . . . . . . . . . . . . . . . . B-1
B.0 Introduction

........... ............. ............ ............ ............ .... B-1 B.1.0 Conceptual Approach for Incorporating a Probability Distribution in a PRA

...... B-3 B.2.0 Preliminary Sensitivity Analysis

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-4 B.3.0 What Does The Distribution Represent?

........... . . . . . . . . . . . . . . . . . . . . . . . . B-5 B.3.1 Concepts of Population and Sampling

........... . . . . . . . . . . . . . . . . . . . . . . . . . . B-6 B.3.2 Considering Variability and Uncertainty in Selecting and Fitting Distributions

... B-12 B.4.0 Do Data Exist To Select Distributions?

........... . . . . . . . . . . . . . . . . . . . . . . . . B-13 B.4.1 What are Representative Data?

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-14 B.4.2 The Role of Expert Judgment

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-15 B.5.0 Fitting Distributions to Data

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-16 B.5.1 Considering the Underlying Mechanism

........... . . . . . . . . . . . . . . . . . . . . . . . B-17 B.5.2 Empirical Distribution Functions (EDFs)

........... . . . . . . . . . . . . . . . . . . . . . . B-22 B.5.3 Graphical Methods for Selecting Probability Distributions

........... ........ B-22 B.5.4 Parameter Estimation Methods

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-24 B.5.5 Dealing with Correlations among Variables or Parameters

........... ........ B-26 B.5.6 Censored Data

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-28 B.5.7 Truncation

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-30 B.6.0 Assessing Quality of the Fit

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-31 B.6.1 What is a Goodness-of-Fit Test?

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-31 B.6.2 What are some common Goodness-of-Fit Techniques?

........... ........... B-33 B.6.3 Cautions Regarding Goodness-of-Fit Tests

........... . . . . . . . . . . . . . . . . . . . . . B-34 B.6.4 Accuracy of the Tails of the Distribution

........... . . . . . . . . . . . . . . . . . . . . . . B-34 B.7.0 Selecting Probability Distributions Based on State of Knowledge

........... ... B-35 References for Appendix B

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-49

xExhibit B-1 Definitions for Appendix B

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-2 Exhibit B-2 General Strategy for Selecting and Fitting Distributions

. . . . . . . . . . . . . . . . . . . . . . B-3 Exhibit B-3 Factors to Consider in Selecting a Probability Distribution

. . . . . . . . . . . . . . . . . . . B-16 Exhibit B-4 Variations of the EDF

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-22 Exhibit B-5 Estimating the area of a hypothetical exposure unit

. . . . . . . . . . . . . . . . . . . . . . . . . B-24 Exhibit B-6 Criteria for Evaluating Parameter Estimation Methods

. . . . . . . . . . . . . . . . . . . . . . . B-25 Exhibit B-7 Parameter Estimation Methods

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-25 Exhibit B-8 Correlation of Input Variables for 1-D MCA of Variability

. . . . . . . . . . . . . . . . . . . B-27 Exhibit B-9 Steps for Simulating Uncertainty in Linear Regression Equation Using a Bivariate Normal Distribution to Correlate Parameters ( $ 0 , $ 1 )

........... ............. B-47 Figure B-1 (page 1 of 2). Conceptual approach for incorporating probability distributions for variability in PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-7 Figure B-1 (page 2 of 2). Conceptual approach for incorporating probability distributions for variability in PRA

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-8 Figure B-2a (page 1 of 3). Conceptual approach for quantifying model and parameter uncertainty in PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-9 Figure B-2a (page 2 of 3). Conceptual approach for quantifying model and parameter uncertainty in PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-10 Figure B-2a (page 3 of 3). Conceptual approach for quantifying model and parameter uncertainty in PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-11 Figure B-3 Comparison of step-wise EDF and linearized EDF for ingestion rate

........... B-38 Figure B-4 Graphical assessment of beta and lognormal distributions fit to the cumulative distribution reported in the literature (circles)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-39 Figure B-5 Histograms of lead concentrations in quail breast muscle

........... ......... B-41 Figure B-6 Lognormal probability plots of lead in mourning dove breast tissue

............ B-43 Figure B-7 Histograms of meal size

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-44 Figure B-8 Probability plot of meal size data

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-45 Figure B-9 Simple linear regression of zinc concentrations in soil and dust

. . . . . . . . . . . . . . . . B-48 Figure B-10 Results of Monte Carlo simulation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-49 Table B-1 Examples of Preliminary Distributions Based on Information Available

.......... B-5

..................... B-18

............. B-30

................ B-36 B-38

.............. B-41

.... B-42

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-44 Table B-2 Examples of Selected Probability Distributions for PRA Table B-3 
 Theoretical bounds and parameter values for selected distributions Table B-4 
 Strategies for conducting PRA based on available information Table B-5 Selected statistics for reported and fitted distributions for ingestion rate (mg/day). Table B-6 Sample values of lead concentration (ppm) in quail breast muscle Table B-7 Parameter estimates for lognormal distribution of lead concentrations (ppm). Table B-8 Meal size (g/meal) Table B-9 Zinc concentrations in paired (i.e., co-located) soil and dust samples (ppm) for n=21 locations

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-48 Appendix C Characterizing Variability and Uncertainty in the Concentration Term ...... C -1
C.0 The Concentration Term and the Exposure Unit

........... . . . . . . . . . . . . . . . . . . . . . . . . C-1 C.1.0 Variability in PRA

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-1 C.1.1 Temporal Variability

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-2 C.1.2 Spatial Variability

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-3 C.1.3 Example of Temporal and Spatial Variability

........... . . . . . . . . . . . . . . . . . . . . C-4 C.1.4 Spatial and Temporal Variability for Different Exposure Media

........... ..... C-5 C.1.4.1 Variability of Concentrations in Soil

........... . . . . . . . . . . . . . . . . . . . . C-5

xiC.1.4.2 Variability of Concentrations in Groundwater

........... ............. C-5 C.1.4.3 Variability of Concentrations in Surface Water

........... ............ C-5 C.1.4.4 Variability of Concentrations in Sediment

........... . . . . . . . . . . . . . . . C-5 C.1.4.5 Variability of Concentrations in Fish

........... . . . . . . . . . . . . . . . . . . . . C-5 C.1.4.6 Examples of Temporal and Spatial Variability in the Concentration Term for Selected Exposure Media

........... . . . . . . . . . . . . . . . . . . . . . . . . . . C-6 C.2.0 Nonrandom Exposures

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-7 C.3.0 Sources of Uncertainty in the Concentration Term

........... . . . . . . . . . . . . . . . . C-8 C.3.1 Quantification of Uncertainty Based on the Size of the Exposure Unit

........... C-8 C.3.1.1 When the Exposure Unit Is Smaller than the Site

........... .......... C-8 C.3.1.2 When the Exposure Unit is the Same Size as the Site

........... ....... C-9 C.3.1.3 When the Exposure Unit is Larger than the Site

........... ........... C-9 C.4.0 Summary of Recommendations for the Concentration Term

........... ....... C-10 C.5.0 Methods for Estimating Uncertainty in the Mean Concentration

........... .... C-10 C.5.1 Quantifying Uncertainty without Information About Locations of Samples and Receptors

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-12 C.5.2 Quantifying Uncertainty with Information About Locations of Samples and Receptors

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-12 References for Appendix C

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C-14 Figure C-1 Spatial and temporal variability in contaminant concentrations in groundwater

.... C-7 Table C-1 Examples of temporal and spatial variability in selected media for the concentration term in common exposure scenarios

........... . . . . . . . . . . . . . . . . C-6 Table C-2 Summary of factors that may be considered in developing an EPC

. . . . . . . . . . . . . C-10 Appendix D Advanced Modeling Approaches for Characterizing Variability and Uncertainty D-1
D.0 Introduction

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-1 D.1.0 Expressing Variability and Uncertainty Simultaneously

........... ............ D-1 D.2.0 Two-Dimensional Monte Carlo Analysis (2-D MCA)

........... ............. D-3 D.3.0 Microexposure Event Analysis

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-6 D.4.0 Geospatial Statistics

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-10 D.4.1 Correlation and Spatial Autocorrelation

........... . . . . . . . . . . . . . . . . . . . . . . . D-11 D.4.2 Effective Sample Size (n*) and Degrees of Freedom

........... ............. D-12 D.4.3 Assessment of Additional Site Sampling

........... . . . . . . . . . . . . . . . . . . . . . . . D-13 D.4.4 Map Generalization

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-15 D.4.5 Implementation Issues Related to Georeferenced Data

........... ............ D-16 D.5.0 Expert Judgment and Bayesian Analysis

........... . . . . . . . . . . . . . . . . . . . . . . . D-16 References for Appendix D

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-25 Exhibit D-1 Definitions for Appendix D

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-3 Exhibit D-2 Positive Spatial Autocorrelation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-10 Exhibit D-3 Examples of Risk Assessment Issues Linked to Geospatial Statistics

........... D-10 Exhibit D-4 Effect of Spatial Autocorrelation (r) on Effective Sample Size ( n* )

............ D-13 Exhibit D-5 Components of Bayes Theorem in PRA

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-17

xiiFigure D-1 Panel A shows a family of 20 CDFs for a hypothetical random variable. Panel B shows the “90% credible interval” for the CDF based on 2500 simulations

........... .. D-1 Figure D-3 Output from a 2-D MCA showing the estimated mean Hazard Quotient (HQ) and the

Figure D-6 Hypothetical example showing the effect of model time step on the probability

Figure D-9 Expected Loss associated with various types of information incorporated into a generic Figure D-10 Conceptual model for evaluating the expected value of including uncertainty in a Figure D-2 Diagram showing of a 2-D Monte Carlo model

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-4 90% confidence interval

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-5 Figure D-4 Time Step for MEE

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-7 Figure D-5 Flowchart showing general approach for Microexposure Event (MEE) analysis.

... D-8 distribution for soil and dust ingestion rate in children over a 1-year period

....... D-9 Figure D-7 Effect of an outlier on measured correlation

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-11 Figure D-8 Conceptual model of Bayesian Monte Carlo analysis

. . . . . . . . . . . . . . . . . . . . . . . . D-18 uncertainty analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-21 Bayesian Monte Carlo analysis

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D-23 Appendix E Definitions of Terms Relevant to PRA and References for Further Reading ... E-1
E.0 Definitions of Terms

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-1 E.1 Additional Information

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-14 References for Appendix E

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-15 References for Further Reading

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E-16 Appendix F Workplan and Checklist for PRA ........... . . . . . . . . . . . . . . . . . . . . . . . . . . . F-1
F.0 Introduction

........... ............. ............ ............ ............ ..... F-1 F.1.0 Workplan

........... ............. ............ ............ ............ F-1 F.2.0 Focal Points for PRA Review

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .F-2 F.3.0 Checklist for Reviewers

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-2 F.4.0 Internal and External Review

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .F-3 References for Appendix F

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-6 Exhibit F-1 Examples of Elements of the Workplan for PRA

........... . . . . . . . . . . . . . . . . . . F-1 Exhibit F-2 Key Focal Points for PRA Review

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F-2 Table F-1 Example of a Generic Checklist for Reviewers

........... . . . . . . . . . . . . . . . . . . . . F-4 Appendix G Frequently Asked Questions for PRA ........... . . . . . . . . . . . . . . . . . . . . . . . . G-1 References for Appendix G

........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . G-6 Appendix H Index ........... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . H-1

xiii

## A CRONYMS AND 

## A BBREVIATIONS

ED EC 1-D MCA One-dimensional Monte Carlo analysis 2-D MCA Two-dimensional Monte Carlo analysis 95% UCL 95% upper confidence limit AM Arithmetic mean ARARs Applicable or relevant and appropriate requirements AT Averaging time AWQC Ambient water quality criterion 

### BCa Bias correction acceleration method BMD Benchmark dose BMDS Benchmark dose software BMR Benchmark Response BTAG Biological Technical Assistance Group BW Body weight C Concentration CAG Community advisory group CDF Cumulative distribution function CI Confidence interval CIC Community involvement coordinator CIP Community involvement plan CLT Central limit theorem COC Chemical of concern CQR Continuous quadratic regression CSF Cancer slope factor CTE Central tendency exposure CV Coefficient of variation DI Daily intake DQO Data quality objectives EC 0 Exposure concentration that produces zero effect

20 Concentration that causes a 20% effect ECDF Empirical cumulative distribution function ED Exposure duration

10 Dose that causes a 10% effect EDF Empirical distribution function EF Exposure frequency EPA U.S. Environmental Protection Agency EPC Exposure point concentration ERA Ecological risk assessment ERAF Risk Assessment Forum ERAGS Ecological Risk Assessment Guidance for Superfund EU Exposure unit EVIU Expected value of including uncertainty EVOI Expected value of information EVPI Expected value of perfect information EVSI Expected value of sample information GIS Geographical Information Systems GM Geometric mean GoF Goodness-of-Fit GSD Geometric standard deviation HEAST Health effects assessment summary table HHEM Human Health Evaluation ManualHI HQ IR Irsd IRIS LADD LCL LED 10 LHS LOAEL LOD LOEC MCA MCL MDC MEE MLE MoMM NCP NOAEL NOEC OLS PBPK PCBs pCi/g PDF PDFu PDFv PMF PPT PRA PRG PRP QAPP RAGS RAL RBC RCRA RfC RfD RG RI/FS RME RMSE ROD ROS RPSS RPM RSS Hazard Index Hazard Quotient Iterative reduction Soil and dust ingestion rate Integrated Risk Information System Life-time average daily intake Lower confidence limit Lowest effect dose - lower confidence bound for dose that causes a 10% effect Latin hypercube sampling Lowest-observed-adverse-effect level Limit of detection Lowest-observed-effect-concentration Monte Carlo analysis Maximum contaminant levels Maximum detected concentration Microexposure Event Analysis Maximum Likelihood Estimation Method of Matching Moments National Oil and Hazardous Substances Pollution Contingency Plan No-observed-adverse-effect level No-observed-effect-concentration Ordinary least squares Physiologically-based pharmacokinetic Polychlorinated biphenyls Picocuries/gram Probability density function Probability distribution for variability Probability distribution for uncertainty Probability mass function Parts per trillion Probabilistic risk assessment Preliminary remediation goal Potentially responsible party Quality Assurance Project Plan Risk Assessment guidance for Superfund Remedial action level Risk based concentration Resource Conservation and Recovery Act Reference concentration Reference dose Remediation goal Remedial Investigation/Feasibility Study Reasonable maximum exposure Root mean squared error Record of decision Rank order statistic Relative partial sum of squares Remedial project manager Regression sum of squares

xvSCM SD SE SMDP SOW SR SSD SSE SSR SST TAB TAG TOSC TRV TSS UCL VOI Site conceptual model Standard deviation Standard error Scientific/Management Decision Point Statement of Work Sensitivity ratio Species sensitivity distribution Sum of squares due to error Sum of squares due to regression Sum of squares for total (regression plus error) Technical Assistance to Brownfields Community Technical assistance grant Technical outreach services for communities Toxicity reference value Total sum of squares Upper confidence limit Value of information

xvi

## A UTHORS 

## , C ONTRIBUTORS 

## , AND 

## R EVIEWERS


### This manual was developed by EPA’s Office of Emergency and Remedial Response. A number of individuals have reviewed and/or have been contributing authors of this document. Members of the EPA RAGS Volume III Workgroup, which was responsible for developing this document, included the following EPA headquarters and regional office staff. 

## RAGS VOLUME III W ORKGROUP 

## P ARTICIPANTS 

## EPA HEADQUARTERS Office of Emergency and Remedial Response David A. Bennett S. Steven Chang David E. Cooper Janine Dinan Elizabeth Lee Hofmann Office of Policy Economics and Innovation Timothy M. Barry 

## EPA REGIONAL OFFICES Region 1 Ann-Marie Burke Region 5 Amy Mucha James Chapman Region 2 Audrey Galizia Region 6 Maria L. Martinez Marian Olsen Region 8 Susan Griffin Region 3 Nancy Rios Jafolla Gerry Henningsen Dale Hoff Region 4 Ted W. Simon Region 10 Joe Goulet Sharon R. Thoms

Technical assistance and production support was provided to EPA in the development of this guidance under Contract Numbers GS-10F-0137K and GS-35F-0555K. An earlier draft of this document was peer reviewed by a panel of experts at a peer-review workshop held in November 2000. In addition, individuals in EPA and from the public provided valuable comments on earlier drafts of this guidance during the peer review process.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Preface ~ December 31, 2001 

## P REFACE Risk Assessment Guidance for Superfund (RAGS) Volume III: Part A (hereafter referred to as RAGS Volume 3: Part A) provides technical guidance on the application of probabilistic risk assessment (PRA) methods to human health and ecological risk assessment in the U.S. Environmental Protection Agency (EPA) Superfund program. RAGS Volume 3: Part A supplements existing human health and ecological assessment guidance provided in the RAGS series. This guidance focuses on Monte Carlo analysis (MCA) as a method of quantifying variability and uncertainty in risk. Primarily geared toward the risk assessor, it is intended, both in content and format, to be most accessible to those readers who are familiar with risk assessment and basic statistical concepts. Chapters 1, 2, 6, and 7 are also directed towards risk managers. The term risk manager is used in this guidance to refer to individuals or entities that serve as the decision makers at hazardous waste sites. The term is used to emphasize the separation between risk assessment and risk management activities. Risk managers may include individual remedial project mangers (RPMs), site partnering teams, senior EPA managers (Section Chiefs, Branch Chiefs, or Division Directors), or other decision makers. An attempt has been made in this document to define all relevant technical terms using plain language and to illustrate concepts with examples. An exhibit at the beginning of each chapter provides definitions of terms used in that chapter. In addition, a comprehensive definition of terms is provided in Appendix E. Other useful information has been presented in exhibits placed throughout each chapter. Bullets are used throughout the text to emphasize important concepts and policy statements related to the use of PRA. References are listed at the end of each chapter. RAGS Volume 3: Part A was developed by the Superfund Probabilistic Risk Assessment Workgroup and the Ecological Risk Assessment Forum (ERAF); both are intra-Agency workgroups that have focused on improving the Risk Assessment Guidance for Superfund and implementing Superfund Reform activities. The guidance has undergone extensive review by Superfund and other programs within the Agency. In February 2000, a draft of the guidance was announced in the Federal Register to provide an opportunity for public comment (U.S. EPA, 2000a). In August 2000, a notice of peer review was announced in the Federal Register (U.S. EPA, 2000b), and in November 2000, RAGS Volume 3: Part A received a formal peer review from panelists outside the Agency. The Agency may incorporate PRA under fund-lead and Potentially Responsible Party (PRP)-lead risk assessments. Implementation of successful PRAs requires careful planning. EPA strongly recommends that PRPs involve the Agency in all decisions regarding the planning, submittal, and technical details of any PRA. Coordinating with EPA early in the process will help ensure that PRAs conform to the recommended guidelines as part of the Superfund risk assessment process for protecting human and ecological health. PRPs should submit workplans for Agency review before initiating any PRA. Similarly, when EPA chooses to use PRA for an EPA-lead risk assessment, a PRA workplan will assist in directing site investigation and risk assessment activities, whether conducted by EPA or an EPA contractor. A workplan specifies contractor activities in the risk assessment and provides risk assessors and risk managers with an opportunity to obtain internal feedback from knowledgeable EPA staff, prior to initiating work on the assessment. A tiered approach to PRA is advocated, which begins with a point estimate risk assessment. Important considerations include the time required to perform the PRA, the additional resources involved in developing the PRA, the quality and extent of data on exposure that will be used in the assessment,

iRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Preface ~ December 31, 2001 and the value added by conducting the PRA. Project scoping is an essential component of all risk assessments and is especially important in PRA. Implementation of a PRA usually requires special computer software that may be commercially available or that may need to be custom-designed for a specific application. Although commercial software packages are noted in this guidance, any mention or use of a particular product in RAGS Volume 3: Part A does not constitute an endorsement of that product by the Agency. 

### 1.0 W HAT IS THE 

### P URPOSE OF 

### RAGS V OLUME 

### 3 P ART 

### A?
RAGS Volume 3: Part A addresses the technical and policy issues associated with the use of PRA in EPA Superfund program. This guidance builds upon basic concepts of risk assessment outlined in RAGS Volume I (U.S. EPA, 1989a; 2001), recent guidance for ecological risk assessment (U.S. EPA, 1992, 1994, 1997a, 1998a; 1999), and the Agency Probabilistic Analysis Policy document (U.S. EPA, 1997b). RAGS Volume 3: Part A addresses the use of PRA for both human health and ecological risk assessments. RAGS Volume 3: Part A was developed to provide risk assessors and risk managers with basic guidelines for incorporating PRA into Superfund site-specific risk assessments. It is not intended to be a detailed technical reference on PRA methods, however, it does direct the reader to appropriate literature on important technical subjects. A primary purpose of RAGS Volume 3: Part A is to help prevent misuse and misinterpretation of PRA. 

### 2.0 W HAT IS 

### P ROBABILISTIC 

### R ISK 

### A SSESSMENT AND HOW IS IT USED IN 

### R ISK 

### C HARACTERIZATION 

### ? PRA is a risk assessment that uses probability distributions to characterize variability or uncertainty in risk estimates. In a PRA, one or more variables in the risk equation is defined as a probability distribution rather than a single number. Similarly, the output of a PRA is a range or probability distribution of risks experienced by the receptors. The evaluation of variability and uncertainty is an important component of the risk characterization of all risk assessments. As stated in the 1995 Risk Characterization memorandum from Administrator Carol Browner (U.S. EPA, 1995), ... we must fully, openly, and clearly characterize risks. In doing so, we will disclose the scientific analyses, uncertainties, assumptions, and science policies which underlie our decisions... There is value in sharing with others the complexities and challenges we face in making decisions in the face of uncertainty. In addition, the 1997 EPA Policy for Use of Probabilistic Analysis in Risk Assessment (U.S. EPA, 1997b) states:



### L It is the policy of the U.S. Environmental Protection Agency that such probabilistic analysis techniques as Monte Carlo analysis, given adequate supporting data and credible assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk assessments. A more extensive general discussion of PRA can be found in Chapter 1 of the guidance. The use of PRA in Superfund remedial decision making is presented in Chapter 7 of the guidance.

iiRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Preface ~ December 31, 2001 

### 3.0 W HAT ARE THE 

### A DVANTAGES AND 

### D ISADVANTAGES OF 

### PRA FOR 

### R EMEDIAL 

### D ECISIONS 

### ? The primary advantage of PRA within the Superfund program is that it can provide a quantitative description of the degree of variability or uncertainty (or both) in risk estimates for both cancer and non-cancer health effects and ecological hazards. The quantitative analysis of uncertainty and variability can provide a more comprehensive characterization of risk than is possible in the point estimate approach. Another significant advantage of PRA is the additional information and potential flexibility it affords the risk manager. Superfund remedy decisions are often based on an evaluation of the risk to the individual at the reasonable maximum exposure (RME) level (U.S. EPA, 1990). The RME represents the highest exposure reasonably likely to occur (U.S. EPA, 1989a). When using PRA, the risk manager can select the RME from the high-end range of percentiles of risk, generally between the 90 thand 99.9 thpercentiles, referred to in this guidance as the RME range

. However, PRA may not be appropriate for every site. Disadvantages of PRA are that it generally requires more time, resources, and expertise on the part of the assessor, reviewer, and risk manager than a point estimate approach. 

### 4.0 H OW IS 

### RAGS V OLUME 

### 3, P ART 

### A O RGANIZED 

### ? Although the primary audience of this guidance is the risk assessor, Chapter 1 provides a basic overview of PRA for risk assessors and risk managers. The centerpiece of RAGS Volume 3: Part A is the tiered approach described in Chapter 2. The tiered approach is a framework that enables the risk manager to decide if and when to undertake a PRA and to determine the appropriate level of complexity for the PRA. Chapter 3 provides a description of using PRA for human health risk assessment. Chapter 4 discusses the issues of using PRA for ecological risk assessment. Chapter 5 presents a discussion of using PRA to determine preliminary remediation goals. Chapter 6 details issues associated with communicating risk estimates developed with PRA. Chapter 7 provides information for risk managers choosing to base remedial decisions on the results of a PRA. Eight appendices to this guidance expand on technical aspects of topics important to PRA, such as sensitivity analysis and selecting and fitting probability distributions. 

### 5.0 W HAT ARE THE 

### K EY 

### G UIDING 

### C ONCEPTS IN 

### RAGS V OLUME 

### 3: P ART 

### A ? ( 1) Use a tiered approach to incorporating PRA into site risk assessments. (2) Submit a workplan for Agency review prior to initiating work on a PRA. (3) Perform a point estimate assessment prior to considering a PRA. (4) While PRA can provide a useful tool to characterize and quantify variability and uncertainty in risk assessments, it is not appropriate for every site.

iiiRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Preface ~ December 31, 2001 (5) PRA generally requires more time, resources, and expertise on the part of the assessor, reviewer, and risk manager than a point estimate risk assessment. (6) The decision to use PRA is site-specific and is based on the complexity of the problems at the site, the quality and extent of site-specific data, and the likely utility of the result. (7) If the additional information provided from a PRA is unlikely to affect the risk management decision, then it may not be prudent to proceed with a PRA. However, if there is a clear value added from performing a PRA, then the use of PRA as a risk assessment tool generally should be considered despite the additional resources that may be needed. (8) Communicating the results of a PRA will be more challenging than communicating the results of a point estimate risk assessment because PRA and its perspective will be new to most participants. (9) If the decision is made to conduct a PRA, it is important to include community in the planning process. Communication on PRA may involve: providing the community with a basic understanding of the principles of PRA, discussing the proposed workplan and inviting comments on the proposed approach, discussing site-specific data, and communicating the final results and how they impact decisions for the site.

ivRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Preface ~ December 31, 2001 R EFERENCES FOR P REFACE U.S. EPA. 1989a. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment) . Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR 300: 55 Federal Register , 8666-8865, March 8.
U.S. EPA. 1992. Final Guidelines for Exposure Assessment . EPA/600/Z-92/001. Federal Register, 22888-22938. May 29.
U.S. EPA. 1994. Role of Ecological Risk Assessment in the Baseline Risk Assessment. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-17.
U.S. EPA. 1995. Memorandum from Carol Browner on Risk Characterization . Office of the Administrator, Washington, DC. February 22.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments . Interim Final. Environmental Response Team, Edison, NJ. EPA/540/R-97/006, OSWER Directive No. 9285.7-25. June.
U.S. EPA. 1997b. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte Carlo Analysis . Office of Research and Development, Washington, DC. EPA/630/R-97/001. May 15.
U.S. EPA. 1998a. Guidelines for Ecological Risk Assessment. Risk Assessment Forum. Environmental Protection Agency, Washington DC. EPA/630/R-95/002F. April. Federal Register 63(93): 26846-26924. May 14.
U.S. EPA. 1999. Ecological Risk Assessment and Risk Management Principles for Superfund Sites . Final. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-28P.
U.S. EPA. 2000a. Superfund Probabilistic Risk Assessment to Characterize Uncertainty and Variability

. Washington, DC. Federal Register [FR Doc. 06-3492] 65(31): 7550-7552. February 15.
U.S. EPA. 2000b. Peer Review for Superfund Probabilistic Risk Guidance . Washington, DC. Federal Register [FR Doc. 00-21197] 65(162): 50694. August 21.
U.S. EPA. 2001. Risk Assessment Guidance for Superfund : Volume I. Human Health Evaluation Manual (Part D, Standardized Planning, Reporting, and Review of Superfund Risk Assessments). Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-47. December.

vRAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001


## C HAPTER 

## 1
O VERVIEW OF 

## P ROBABILISTIC 

## A PPROACH TO 

## R ISK 

## A SSESSMENT 

### 1.0 I NTRODUCTION This chapter is intended for risk managers and risk assessors as an overview of the probabilistic approach to risk assessment in the context of the Superfund program at the U.S. Environmental Protection Agency (EPA). The goals of this chapter are to provide the reader with information about (1) the role of risk assessment in the Superfund program; (2) the basic concepts of probabilistic risk assessment (PRA); (3) important policies and guiding principles for PRA, as outlined throughout this guidance; and (4) the next steps that will be undertaken in the Superfund program to provide guidance on PRA. Section 1.1 (1.1.1–1.1.3) describes the role of risk assessment from three perspectives, including the role of risk assessment in areas external to EPA, Agency-wide, and within Superfund. Section 1.1 (1.1.4) also introduces PRA and identifies its place in the Superfund program. Section 1.2 introduces the basic concepts of PRA, including the key terms of variability, uncertainty, Monte Carlo analysis (MCA), and reasonable maximum exposure (RME). PRA concepts are presented using a comparison between PRA and the traditional point estimate approach. Sections 1.2.4 and 1.3 summarize the advantages and disadvantages of PRA and point estimate risk assessment. Section 1.4 provides a summary of policies and guiding principles for using PRA in the Superfund program. EPA’s policies on conducting PRA are highlighted throughout the guidance using pointers and are linked to more detailed policy discussions in other chapters in the guidance. Section 1.5 outlines the organization of this document and provides a brief summary of the content of each subsequent chapter and appendix. Section 1.6 presents EPA’s next steps for PRA implementation in the Superfund program. Key terms used throughout this guidance include: Probabilistic Risk Assessment (PRA), Monte Carlo Analysis (MCA), Probability Density Function (PDF), Cumulative Distribution Function (CDF), Reasonable Maximum Exposure (RME), Sensitivity Analysis, Tiered Approach, Variability, Uncertainty, and Preliminary Remediation Goal (PRG). Terms and their definitions are identified in an exhibit at the beginning of each chapter. Terms and definitions relevant to Chapter 1 are presented in Exhibit 1-1. In addition, a glossary of terms used throughout the guidance is given in Appendix E.

Page 1-1RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

E 1-1
D C 1

Confidence Interval Confidence intervals may th

counted with integers (e.g., one, two, three) and that has no upper limit. Examples include the number of tosses needed.

th risk). approaches.

CTE Risk - The estimated risk corresponding to the central tendency exposure. occurrence for a random independent variable. Each value c x c .

Frequency Distribution or Histogram It conveys the range of values and the count (or proportion of the sample) that was observed across that range.
using these inputs to calculate a range of risk values.

Parameter
[bounds]. XHIBIT

EFINITIONS FOR HAPTER Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population, usually considered to be the mean or median of the distribution. - A range of values that are likely to include a population parameter. describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95 percentile risk). When used to characterize uncertainty in a risk estimate, it is assumed that methods used to quantify uncertainty in the model inputs are based on statistical principles such as sampling distributions or Bayesian approaches. For example, given a randomly sampled data set, a 95% confidence interval for the mean can be estimated by deriving a sampling distribution from a Student's t distribution. Confidence Limit - The upper or lower value of a confidence interval. Countably Infinite - Used to describe some discrete random variables, this term refers to a set of numbers that can be required for a coin to show a head—we can count each toss, but it is possible that at least one more toss is The number of dust particles in a volume of air is another example. Countably finite implies there is an upper limit (e.g., days of work per year). Credible Interval - A range of values that represent plausible bounds on a population parameter. Credible intervals may describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95 percentile The term is introduced as an alternative to the term confidence interval when the methods used to quantify uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian For example, multiple estimates of an arithmetic mean may be available from different studies reported in the literature - using professional judgment, these estimates may support a decision to describe a range of possible values for the arithmetic mean. Cumulative Distribution Function (CDF) - Obtained by integrating the PDF, gives the cumulative probability of of the function is the probability that a random observation will be less than or equal to Expected Value of Information (EVOI) - The expected increase in the value (or decrease in the loss) associated with obtaining more information about quantities relevant to the decision process. EVOI is a measure of the importance of uncertainty in risk and the potential for changing a risk management decision if uncertainty is reduced (see Appendix D). - A graphic (plot) summarizing the frequency of the values observed or measured from a population. Monte Carlo Analysis (MCA) or Monte Carlo Simulation - A technique for characterizing the uncertainty and variability in risk estimates by repeatedly sampling the probability distributions of the risk equation inputs and Numeric Stability - Stochastic variability, or "wobble" associated with random sampling, calculated as the average percent change in the model output after rerunning Monte Carlo simulations with the same set of input assumptions. Used as a metric for evaluating the adequacy of the number of iterations in a MCA. - A value that characterizes the distribution of a random variable. Parameters commonly characterize the location, scale, shape, or bounds of the distribution. For example, a truncated normal probability distribution may be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean ingestion rate). Point Estimate - In statistical theory, a quantity calculated from values in a sample to estimate a fixed but unknown population parameter. Point estimates typically represent a central tendency or upper bound estimate of variability.

Page 1-2RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

E 1-1
D C 1— Continued

bounding risk estimate depending on the choice of inputs.

Also called a probability model

.

RME Risk in the values of the model’s input(s). Common metrics of sensitivity include:
< - A statistic r that measures the strength and direction of linear ( r

2 ) variable.
< - Ratio of the change in model output per unit change in an input variable; elasticity

. < r r 2

. Stochastic Dominance

Examples
Variability

population. n XHIBIT

EFINITIONS FOR HAPTER Point Estimate Risk Assessment - A risk assessment in which a point estimate of risk is calculated from a set of point estimates for exposure and toxicity. Such point estimates of risk can reflect the CTE, RME, or Probabilistic Risk Assessment (PRA) - A risk assessment that yields a probability distribution for risk, generally by assigning a probability distribution to represent variability or uncertainty in one or more inputs to the risk equation. Probability Density Function (PDF) - A function representing the probability distribution of a continuous random variable. The density at a point refers to the probability that the variable will have a value in a narrow range about that point. Probability Distribution - A mathematical representation of the function that relates probabilities with specified intervals of values for a random variable. Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random variable. The mass at a point refers to the probability that the variable will have a value at that point. Random Variable - A variable that may assume any value from a set of values according to chance. Discrete random variables can assume only a finite or countably infinite number of values (e.g., number of rainfall events per year). A random value is continuous if its set of possible values is an entire interval of numbers (e.g., quantity of rain in a year). Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S. EPA, 1989a). The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case) that is still within the range of possible exposures. Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and extent of contamination, to evaluate potential risk, and to develop alternatives for cleanup. - The estimated risk corresponding to the reasonable maximum exposure. Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes Sensitivity analysis can provide a quantitative ranking of the model inputs based on their relative contributions to model output variability and uncertainty. Pearson Correlation Coefficient association between the values of two quantitative variables. The square of the coefficient is the fraction of the variance of one variable that is explained by the variance of the second Sensitivity Ratio also called Spearman Rank Order Correlation Coefficient - A “distribution free” or nonparametric statistic that measures the strength and direction of association between the ranks of the values (not the values themselves) of two quantitative variables. See Pearson (above) for - Implies no intersection between two or more CDFs. For example, if the CDF for A and B do not overlap and the CDF for A is greater than the CDF for B, then at every cumulative percentile, the value of A is greater than that of B. Therefore, it can be stated that distribution A stochastically dominates distribution B. It should be noted that even when the CDFs for A and B do not overlap, the PDFs for A and B can overlap. Uncertainty -Lack of knowledge about specific variables, parameters, models, or other factors. include limited data regarding the concentration of a contaminant in an environmental medium and lack of information on local fish consumption practices. Uncertainty may be reduced through further study. - True heterogeneity or diversity that characterizes an exposure variable or response in a Further study (e.g., increasing sample size, ) will not reduce variability, but it can provide greater confidence (e.g., lower uncertainty) in quantitative characterizations of variability).

Page 1-3RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001



### 1.1 T HE 

### R OLE OF 

### RISK 

### A SSESSMENT IN 

### S UPERFUND The role of risk assessment in the Superfund program today is built upon a foundation of scientific and management principles, policies, and laws that have been established over the past two decades. Since the enactment of the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) in 1980 the risk assessment policies and guidance documents have evolved to reflect advances in science and changes in federal regulations.
1.1.1 RISK A SSESSMENT IN THE U NITED S TATES Risk assessment has a long history beginning in 1940. In 1983, the National Research Council published Risk Assessment in the Federal Government: Managing the Process (NRC, 1983) which outlines the four steps of risk assessment (hazard identification, dose-response, exposure assessment, and risk characterization) that are used today. The NRC addressed three main objectives in risk assessment: (1) assessment of the benefits of separating the analytical process of risk assessment from the regulatory process of risk management; (2) consideration of the feasibility of creating a single regulatory agency for the purpose of conducting all government risk assessments; and (3) consideration of the feasibility of creating uniform guidelines for risk assessment (NRC, 1983). The Committee concluded that regulatory agencies should maintain a conceptual distinction between risk assessment and risk management, and develop uniform inference guidelines in risk assessment for use by all federal regulatory agencies. The Committee also recommended that Congress establish a Board on Risk Assessment Methods in order to ensure that risk assessment procedures be continuously reviewed and modified as the science advances. The Committee rejected the proposal for a single federal risk assessment agency based on inadequate evidence to show that one administrative structure would be more advantageous (NRC, 1983). Since 1983, there have been ongoing advancements in the field of risk assessment. These include: (1) a continued increasing role for risk assessment in the decision-making process of many regulatory agencies, as exemplified by several bills introduced by the 103 rd and 104 th Congresses in 1994-1995; (2) an increased awareness of the need for uncertainty analysis and for quantifying and communicating uncertainties in risk estimates (Science and Judgement in Risk Assessment, NRC, 1994); (3) guidance about more inclusive approaches to risk assessment, as exemplified by environmental health legislation such as the Food Quality Protection Act (FQPA) of 1996 and the Presidential/Congressional Commission on Risk Assessment and Risk Management (1997); and (4) setting the stage for a more open decision-making process through stakeholder involvement in the risk management process, as outlined in Improving Risk Communication (NRC, 1989).

Page 1-4RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

1.1.2 RISK A SSESSMENT AT EPA EPA has refined the risk paradigm through deliberations of the Risk Assessment Forum, Science Policy Council, and other Agency-wide bodies. Such deliberations have led to consensus in guidance, policies, and memoranda that respond to the requirements set out by various environmental statutes. Individual offices have also developed regulations, guidance, and other supporting documents to aid in the implementation of particular environmental statutes. In 1986, EPA issued final guidelines relating to risk assessment for cancer, mutagenic effects, developmental effects, exposure assessment, and chemical mixtures. Since 1986, EPA has updated or issued revised final guidelines for developmental toxicity, exposure assessment, reproductive toxicity, neurotoxicity, and ecological risk assessment; and is now revising carcinogen risk assessment guidelines. (See http://www.epa.gov/ncea/raf/rafguid.htm for details on guidelines .) Other notable documents that guide risk assessment at EPA include:

• Framework for Ecological Risk Assessment (U.S. EPA, 1992b) • Guidelines for Ecological Risk Assessment (U.S. EPA, 1998) • Guidance for Risk Characterization (U.S. EPA, 1995a) • Policy for Risk Characterization (U.S. EPA, 1995c) • Policy on Evaluating Health Risks to Children(U.S. EPA, 1995d) • Policy for Use of Probabilistic Analysis in Risk Assessment (U.S. EPA, 1997g) • Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment (U.S. EPA, 1997g) • Guidance on Cumulative Risk Assessment. Part 1. Planning and Scoping (U.S. EPA, 1997e) • Risk Characterization Handbook (U.S. EPA, 2000)
1.1.3 RISK A SSESSMENT IN S UPERFUND The activities and publications described above have provided a strong foundation for the development of risk assessment guidance on conducting human health—and ecological risk assessments in the Superfund program. EPA uses risk assessment (NRC, 1983, 1994) to carry out CERCLA, as amended by the Superfund Amendments and Reauthorization Act of 1986 (SARA). Under CERCLA/SARA, EPA’s Superfund program is authorized to protect human health and the environment from current and potential threats posed by releases of hazardous substances, pollutants, or contaminants. The blueprint for the Superfund program is the National Oil and Hazardous Substances Pollution Contingency Plan (NCP) (U.S. EPA, 1990). Among other things, the NCP calls for the identification and mitigation of environmental impacts at hazardous waste sites, and for the selection of remedial actions to protect human health and the environment. An important part of the NCP is the implementation of a Remedial Investigation and Feasibility Study (RI/FS), which is designed to support risk management decisions within the Superfund program. A risk assessment is an integral part of the RI/FS, and is

Page 1-5RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
generally conducted at a site to determine the need for action and to ensure that a selected remedy will be protective. The NCP also establishes some benchmarks for protectiveness and lays out nine criteria (some risk-based) against which each cleanup option should be evaluated (see Exhibit 1-2). Guidance for risk assessment in the Superfund program has been developed to facilitate consistent site-specific responses. Early major guidance documents developed by EPA included : Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM)(Part A, Baseline Risk Assessment) (U.S. EPA, 1989a) and Risk Assessment Guidance for Superfund. (RAGS) : Volume II. Environmental Evaluation Manual (U.S. EPA, 1989b). RAGS Volume I: Part A provides an approach for conducting site-specific baseline (i.e., without remediation or institutional controls) human health risk assessments. RAGS Volume II , aimed at site E 1-2
N INE C E C LEANUP A )

1. environment 2.

3. 4.
5. Short-term effectiveness 6. 7. Cost

8. 9. XHIBIT

RITERIA FOR VALUATION OF

LTERNA TIVES (U.S. EPA, 1990
Thre sho ld Cr iteria Overall protection of human health and the Compliance with ARARs
Ba lanc ing C riteria Long-term effectiveness and permanence Reduction in toxicity, mobility, or volume through treatment Imp leme ntability

Mo difying Criteria State acceptance Community acceptance
managers, provides a framework for considering environmental effects at sites. More recently, EPA developed guidance for conducting ecological risk assessments within the Superfund program. This guidance, Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments (U.S. EPA, 1997a), discusses scientific methods and stakeholder input. Over the years, the Superfund program has expanded RAGS to include the following documents relating to human health:

• RAGS Volume I, Part B: Development of Risk-based Preliminary Remediation Goals (Risk Equations and Parameters) (U.S. EPA, 1991b) • RAGS Volume I, Part C: Risk Evaluation of Remedial Alternatives (U.S. EPA, 1991c) • RAGS Volume I, Part D: Standardized Planning, Reporting, and Review of Superfund Risk Assessments (U.S. EPA, 2001a) • RAGS Volume I, Part E: Supplemental Guidance for Dermal Risk Assessment ( U.S. EPA, 2001b ) Additional ecological guidance documents include:

• Role of the Ecological Risk Assessment in the Baseline Risk Assessment . OSWER Directive No. 9285.7-17 (U.S. EPA, 1994a) • Issuance of Final Guidance: Ecological Risk Assessment and Risk Management Principles for Superfund Sites . OSWER Directive 9285.7-28 P (U.S. EPA, 1999) • The Role of Screening-Level Risk Assessments and Refining Contaminants of Concern in Baseline Risk Assessments . 12 th Intermittent Bulletin, ECO Update Series. (U.S. EPA, 2001d)
Page 1-6RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
This document ( RAGS Volume 3: Part A ) provides guidance for probabilistic approaches for both human health and ecological risk assessment. The Superfund program has also issued supplementary documents, including:

• Human Health Evaluation Manual, Supplemental Guidance: “Standard Default Exposure Factors” (U.S. EPA, 1991a) • Supplemental Guidance to RAGS: Calculating the Concentration Term (U.S. EPA, 1992d) • Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions (U.S. EPA, 1991d) • Use of IRIS ( Integrated Risk Information System) Values in Superfund Risk Assessment (U.S. EPA, 1993) • Final Soil Screening Guidance, May 17, 1996. Soil Screening User’s Guide (U.S. EPA, 1996) • Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites (U.S. EPA, 2001c). EPA will continue to develop Superfund guidance and tools to improve the practice of risk assessment. Superfund guidance documents are available from EPA’s Superfund publications web site ( http://www.epa.gov/superfund/pubs.htm ). The role of risk assessment in Superfund, described above, can be summarized by a number of principles that are followed and developed in RAGS Volume 3: Part A , including:

• The Superfund risk assessment process should rely on early problem formulation, planning, and scoping for improved remedial investigations and feasibility studies, risk assessments, and risk management decisions.

• The use of a tiered process in Superfund risk assessment and management is beneficial in that it promotes an efficient allocation of resources and improved decision-making.

• Early and continuing involvement of stakeholders throughout the Superfund risk assessment process provides an opportunity to build stakeholder trust and meet stakeholder needs, which can result in improved risk assessments and faster, more-informed risk management decisions.
1.1.4 P ROBABILISTIC RISK A SSESSMENT AND I TS R OLE IN S UPERFUND RAGS Volume I (U.S. EPA, 1989a ) and supporting guidance describe a point estimate approach to risk assessments in the Superfund program. Point estimate risk assessments use single values (point estimates) to represent variables in a risk equation. The output of the risk equation in a point estimate risk assessment is, therefore, a point estimate of risk, which can be a central tendency exposure (CTE) estimate of risk (e.g., the average expected risk) or reasonable maximum exposure (RME) estimate of risk (e.g., the risk expected if the RME was to occur), depending on the input values used in the risk equation. RAGS Volume 3: Part A describes a probabilistic approach to risk assessment. Probabilistic risk assessment uses probability distributions for one or more variables in a risk equation in order to quantitatively characterize variability and/or uncertainty. The output of a PRA is a probability
Page 1-7RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
distribution of risks that reflects the combination of the input probability distributions. If the input distributions represent variability, then the output risk distribution can provide information on variability in risk in the population of concern. If the input distributions reflect uncertainty, then the output risk distribution can provide information about uncertainty in the risk estimate. Information from a PRA can be used to make statements about the likelihood of exceeding a risk level of concern, given the estimated variability in elements of the risk equation. Since the results of point estimate methods generally do not lend themselves to this level of risk characterization (e.g., quantitative uncertainty assessment), PRA can provide unique and important supplemental information that can be used in making Superfund risk management decisions at Superfund sites. Monte Carlo Analysis (MCA) is perhaps the most widely used probabilistic method in PRA. MCA is a specific probabilistic method that uses computer simulation to combine multiple probability distributions in a risk equation (see Section 1.2.2 for further discussion of Monte Carlo simulation). Monte Carlo methods have been in used in modeling since 1946 when Stanislaw Ulam used MCA to conduct uncertainty analysis at Los Alamos during the conceptual stage of the hydrogen bomb project. The history of the use of MCA (from the 1940s to the present) can be found in Rugen and Callahan, 1996. The application of probabilistic analysis to human health and ecological risk assessment is a relatively recent development that was facilitated by development of statistical sampling techniques to obtain a probabilistic approximation to the solution of a mathematical equation and/or model, and increased speed and capacity of modern computers which can support the intensive computational requirements of MCA. Desktop computers and commercial software are currently available which enable risk assessors to make, in minutes, PRA calculations that only a few years ago would have required days. The potential value of PRA to support risk-based decisions has become increasingly apparent over the last several years. This has prompted the need for appropriate policy and guidance documents that define the role of PRA in the Superfund program and that promote and facilitate the highest quality and consistent application of PRA in the Program where appropriate. EPA previously issued guidance that addresses the use of quantitative uncertainty analysis in risk assessment. RAGS Volume I (U.S. EPA, 1989a) and the Final Guidelines for Exposure Assessment Guidelines (U.S. EPA, 1992a) emphasize the importance of assessing variability and uncertainty in risk estimates conducted in the Superfund program. Guidance is also available for characterizing the 95% upper confidence limit (UCL) for the mean exposure concentration (U.S. EPA, 1992d, 1997f). At the regional level, EPA Regions 3 and 8 issued guidance on the appropriate use of probabilistic methods in risk assessment (U.S. EPA, 1994b, 1995e). The importance of adequately characterizing variability and uncertainty is addressed in the 1995 memorandum on Risk Characterization Policy and Guidance (U.S. EPA, 1995b). In the spring of 1997, EPA released the memorandum, Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment (U.S. EPA, 1997g). According to the Policy Statement of the memorandum, probabilistic analysis techniques, “given adequate supporting data and credible assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk assessments.” As such, a PRA, “will be evaluated and utilized in a manner that is consistent with other risk assessments submitted to the Agency.” Along with this Policy Statement, the Agency released a set of guiding principles for use and review of probabilistic analyses (U.S. EPA, 1997g). Hence, both RAGS and Agency-wide guidance emphasize the importance of review of the scientific and technical merit of a probabilistic analysis to determine whether or not the assessment is of sufficient quality to support a remedial decision.

Page 1-8RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
Currently, EPA’s Office of Emergency and Remedial Response (OERR) is implementing PRA as part of its Superfund reform activities. This guidance, RAGS Volume 3: Part A , provides risk assessors with comprehensive guidance on when and how it may be appropriate to conduct PRAs using Monte Carlo analysis within the Superfund program. It describes basic concepts in PRA, an approach for conducting MCA, and EPA’s policy for implementing PRA in the Superfund program. The Agency also intends to supplement this guidance with additional examples and case studies in PRA (see Section 1.6). 

### 1.2 B ASIC 

### C ONCE PTS OF 

### P ROBABILISTIC 

### RISK 

### A SSESSMENT This section describes what a PRA is and compares and contrasts it to the more familiar point estimate methods for human health risk assessment (U.S. EPA, 1989a) and ecological risk assessment (U.S. EPA, 1997a). A risk assessment performed using probabilistic methods is very similar in concept and approach to the point estimate method, with the main difference being the methods used to incorporate variability and uncertainty into the risk estimate. A variety of modeling techniques can be used to characterize variability and uncertainty in risk. This guidance focuses on MCA, perhaps the most common probabilistic method that risk assessors will encounter. Basic concepts on how to use MCA to propagate variability and uncertainty in exposure through a risk model are presented. Many of the concepts presented in this guidance are applicable to other probabilistic approaches to risk assessment. At some sites, probabilistic analysis can provide a more complete and transparent characterization of the risks and uncertainties in risk estimates than would otherwise be possible with a point estimate approach. However, a PRA is not necessary or desirable for every site. The tiered approach presented in Chapter 2 highlights important scientific and management decisions for determining if PRA is appropriate at a specific site. The decision to perform PRA is appropriate only after the risk assessor and the remedial project manager (RPM) at the site determine whether a PRA will enhance decision making at the site. If a PRA is conducted, the assumptions and inputs to the probabilistic model should be sufficiently documented so that the results can be independently reproduced. An essential concept in PRA that will be important throughout this section and the rest of the guidance is the distinction between “variability” and “uncertainty”. Variability refers to true heterogeneity or diversity. For example, among a population that drinks water from the same source and with the same contaminant concentration, the risks from consuming the water may vary. This may be due to differences in exposure (i.e., different people drinking different amounts of water, having different body weights, exposure frequencies, and exposure durations) as well as differences in response (e.g., genetic differences in resistance to a chemical dose). Differences among individuals in a population are referred to as inter-individual variability, while differences for one individual over time are referred to as intra-individual variability. Uncertainty occurs because of a lack of knowledge. For example, we can be very certain that different people drink different amounts of water, but we may be uncertain about how much variability there is in water intakes among the population. Uncertainty can often be reduced by collecting more and better data, while variability is an inherent property of the population being evaluated. Variability can be better characterized with more data, but it cannot be reduced or eliminated. Sometimes there can be confusion about whether data are representative of variability or uncertainty, especially when the distinction depends on how the problem is framed. For example, one of Page 1-9RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
the exposure variables that may be considered in a risk assessment of workers exposed via inhalation to an indoor air contaminant is the fraction of time spent indoors on site. Assume that time-activity information is available from surveys of a representative population of workers. This data set may be used to define a probability distribution (e.g., empirical, normal) that characterizes inter-individual variability in exposure times among workers. Sources of uncertainty would include the choice of the probability distribution used to characterize variability, as well as the parameter estimates that are based on a finite data set. Using the same data set, uncertainty in a parameter, such as the arithmetic mean exposure time, may also be defined by a probability distribution. Efforts to clearly distinguish between variability and uncertainty are important for both risk assessment and risk communication. Section 1.2.4 and Chapter 3, Section 3.4 present an overview of the different sources of uncertainty. Guidance on selecting and fitting probability distributions is given in Appendices B and C, and advanced methods for characterizing both variability and uncertainty are discussed in Appendix D.
1.2.1 W HAT IS PRA? Probabilistic risk assessment is a general term for risk assessments that use probability models to represent the likelihood of different risk levels in a population (i.e., variability) or to characterize uncertainty in risk estimates. A risk assessment performed using probabilistic methods would rely on the same fundamental exposure and risk equations as do point estimate approaches. U.S. EPA guidance, including RAGS Volume I: Part A (U.S. EPA, 1989a), the Standard Default Exposure Factors Guidance (U.S. EPA, 1991a), Supplemental Guidance for Developing Soil Screening Levels (U.S. EPA, 2001c), and Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments (U.S. EPA, 1997a) present methods for estimating risk using standardized exposure and risk models. Examples of typical exposure and risk equations that would be used in risk calculations, in this case, for a drinking water exposure scenario, are provided in Exhibit 1-3:

Page 1-10RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

E 1-3
C ANCER AND N ONCANCER R ISK M ODELS

CDI = ) C = ) IR = EF = ) ED = BW = body weight (kg) HQ = hazard quotient AT =
CSF = ) -1 RfD = ) XHIBIT

Exp osure M ode l:
Can cer R isk M ode l:
No ncan cer R isk M ode l:

chronic daily intake of the chemical (mg/kg-day con centra tion of the chem ical in an e xpo sure m edium (e.g., mg /L ingestion rate (e.g., L/day for wa ter, mg/d ay for so il, etc.) exposure frequency (days/year expo sure duration (yea rs)

averaging time (equal to ED x 3 65 days/year for noncarcinogens and 70 years x 365 days/year for carcino gens) cancer slope factor (linear low-dose cancer potency factor) for the chemical (mg/kg-day reference dose for the chemical for assessing noncancer health effects (mg/kg-day

In the point estimate approach, a single numerical value (i.e., point estimate) is chosen for each variable shown in Exhibit 1-3. For example, point estimates may include a drinking water ingestion rate of 2 L/day and a body weight of 70 kg for an adult. Based on the choices that are made for each individual variable, a single estimate of risk is calculated. In the probabilistic approach, inputs to the risk equation are described as random variables (i.e., variables that can assume different values for different receptors in the population) that can be defined mathematically by a probability distribution. For continuous random variables, such as those in Figure 1-1 (body weight), the distribution may be described by a PDF, whereas for discrete random variables (e.g., number of fish meals per month), the distribution may be described by a probability mass function (PMF). The key feature of PDFs and PMFs is that they describe the range of values that a variable may assume, and indicate the relative likelihood (i.e., probability) of each value occurring within that range for the exposed population. For example, the distribution of tap water ingestion (mL/day) among the general U.S. population might be characterized by a lognormal distribution with a log-mean of 6.86 and a log-standard deviation of 0.575 (Table 3-11 of U.S. EPA 1997b). One might use a PDF to show how approximately half the population drinks more than 1 L/day of tap water, but only 10% of the population drinks more than 2 L/day. After determining appropriate PDF types and parameter values for selected variables, the set of PDFs is combined with the toxicity value in the exposure and risk equations given in Exhibit 1-3 to estimate a distribution of risks. Guidance on selecting and fitting distributions for variables in risk equations is provided in Appendix B. In human health risk assessments, probability distributions for risk should reflect variability or uncertainty in exposure. In ecological risk assessments, risk distributions may reflect variability or uncertainty in exposure and/or toxicity (see Sections 1.4 and 1.4.1, Item 3).
Page 1-11RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
A continuous probability distribution can be displayed in a graph in the form of either a PDF or corresponding CDF; however, for clarity, it is recommended that both representations be presented in adjacent (rather than overlaid) plots. Figure 1-1 illustrates a PDF and CDF for a normal probability distribution for adult body weight. Both displays represent the same distribution, but are useful for conveying different information. Note that it is helpful to include a text box with summary statistics relevant to the distribution (e.g., mean, standard deviation). The types of information that PDFs and CDFs are most useful for displaying are presented in Exhibit 1-4. E 1-4
U SE A PDF AND O D :
PDF

C

C

C

C
CDF

C

C th to 99 th percentiles)

C

C XHIBIT
CDF T ISPLAY
The relative probability of values The most likely values (e.g., modes) The shape of the distribution (e.g., skewness, kurtosis, multimodality) Small changes in probability density

Percentiles, including the median High-end risk range (e.g., 90 Confidence intervals for selected percentiles Stochastic dominance (i.e., for any percentile, the value for one variable exceeds that of any other variable) Source: U.S. EPA, 1997g

Figure 1-1. Example of a normal distribution that characterizes variability in adult body weight (males and females combined). Arithmetic mean=71.7 kg, standard deviation=15.9 kg (Finley and Paustenbach, 1994). Body weight may be considered a continuous random variable. The left panel shows a bell-shaped curve and represents the P DF, while the right panel shows an S-shaped curve and represents the CDF. Both displays represent the same distribution (including summary statistics), but are useful for conveying different information.

Page 1-12RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
The CDF for risk can be especially informative for illustrating the percentile corresponding to a particular risk level of concern (e.g., 95 th percentile=1E-06). A text box may also be included on the graph to highlight important summary statistics, such as the parameters of the input distribution, or selected percentiles of the output distribution for risk. For example, a clear description of the parameters for the probability distribution should be given, as well as an indication of whether the distribution represents variability or uncertainty.
1.2.2 W HAT IS A M ONTE C ARLO S IMULATION ? Perhaps the most common numerical technique for PRA is Monte Carlo simulation. Monte Carlo simulation has been widely used to explore problems in many disciplines of science as well as engineering, finance, and insurance (Rugen and Callahan, 1996). The process for a Monte Carlo simulation is illustrated in Figure 1-2. In its general form, the risk equation can be expressed as a function of multiple exposure variables (V i ) and a toxicity term: Risk=f(V 1 , V 2 , ...V n ) x Toxicity. Solutions for equations with PDFs are typically too complex for even an expert mathematician to calculate the risk distribution analytically. However, numerical techniques applied with the aid of computers can provide very close approximations of the solution. This is illustrated here for the simplified case in which the assessment variables are statistically independent, that is, the value of one variable has no relationship to the value of any other variable. In this case, the computer selects a value for each variable (V i ) at random from a specified PDF and calculates the corresponding risk. This process is repeated many times (e.g., 10,000), each time saving the set of input values and corresponding estimate of risk. For example, the first risk estimate might represent a hypothetical individual who drinks 2 L/day of water and weighs 65 kg, the second estimate might represent someone who drinks 1 L/day and weighs 72 kg, and so forth. Each calculation is referred to as an iteration, and a set of iterations is called a simulation.

L A convenient aid to understanding the Monte Carlo approach for quantifying variability is to visualize each iteration as representing a single individual and the collection of all iterations as representing a population. Each iteration of a Monte Carlo simulation should represent a plausible combination of input values (i.e., exposure and toxicity variables), which may require using bounded or truncated probability distributions (see Appendix B). However, risk estimates are not intended to correspond to any one person. The “individuals” represented by Monte Carlo iterations are virtual and the risk distributions derived from a PRA allow for inferences to be made about the likelihood or probability of risks occurring within a specified range for an exposed human or ecological population. A simulation yields a set of risk estimates that can be summarized with selected statistics (e.g., arithmetic mean, percentiles) and displayed graphically using the PDF and CDF for the estimated risk distribution. Often the input distributions are assumed to be independent, as shown in Figure 1-2. More complex Monte Carlo simulations can be developed that quantify a dependence between one or more input distributions by using conditional distributions or correlation coefficients (see Appendix B, Section B.5.5 for a discussion of correlated input distributions).

Page 1-13RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

Figure 1-2. Conceptual model of Monte Carlo analysis. Random variables (V 1 , V 2 , ...V n ) refer to exposure variables (e.g., body weight, exposure frequency, ingestion rage) that are characterized by probability distributions. A unique risk estimate is calculated for each set of random values. Repeatedly sampling (V i ) results in a frequency distribution of risk, which can be described by a PDF. In human health risk assessments, the toxicity term should be expressed as a point estimate. In ecological risk assessment (see Sections 1.4 and 1.4.1) the toxicity term may be expressed as a point estimate or as a probability distribution.

Page 1-14RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
The rapid evolution in computing power has greatly reduced concerns among regulators regarding the number iterations needed in MCA.

L While this guidance does not prescribe specific criteria or set an arbitrary “minimum” number of iterations needed for PRA, a general rule of thumb is that a sufficient number of iterations should be run to obtain numerical stability in percentiles of the output (e.g., risk distribution) that are important for decision making. Numerical stability refers to the stochastic variability, or “wobble” associated with random sampling, and can be evaluated by running multiple simulations with the same set of input assumptions and calculating the average percent change in a specified percentile of the output (e.g., Maddalena et al., 2001). For example, it may be determined that 5,000 iterations are sufficient to achieve numerical stability in the 50 th percentile, but insufficient for the 95 th percentile risk estimate when a criteria of ± 1% is applied for multiple simulations. As discussed in Section 1.4, one of the eight conditions specified by EPA for the acceptance of PRA is that the numerical stability of the output be presented and discussed, since it will vary depending on what percentile of the risk distribution is evaluated. While some commercial software now have a feature to automatically stop simulations after a specified criterion for numerical stability is achieved (Burmaster and Udell, 1990), care should be taken to understand how this criterion is implemented across the entire range of the output distribution.
1.2.3 W HY IS V ARIABILITY I MPORTANT IN RISK A SSESSMENT ? H OW IS IT A DDRESSED BY THE P OINT E STIMATE AND P ROBABILISTIC A PPROACHES ? As noted previously, variability refers to true heterogeneity or diversity that occurs within a population or sample. Factors that lead to variability in exposure and risk include variability in contaminant concentrations in a medium (air, water, soil, etc.), differences in ingestion rates or exposure frequencies, or in the case of ecological assessments, inter- and intra-species variability in dose-response relationships. Risk Assessment Guidance for Superfund Volume I (Section 6.1.2 of U.S. EPA, 1989a) and the NCP Preamble (U.S. EPA, 1990) state that human health risk management decisions at Superfund sites will generally be based on an individual that has RME. Likewise, RME estimates of risk are the most appropriate basis for decision making using an ecological risk assessment. Use of the RME and CTE risk descriptors in ecological risk assessment are discussed in Chapter 4. The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case) that is still within the range of possible exposures based on both quantitative information and professional judgment (Sections 6.1.2 and 6.4.1 of U.S. EPA, 1989a). In addition, the Agency released guidance in 1992 (U.S. EPA, 1992c) recommending the inclusion of a “central tendency” exposure estimate to an individual, as well as a high-end exposure estimate, in the risk assessment. Generally, the CTE is considered to be a measure of the mean or median exposure. The difference between the CTE and the RME gives an initial impression of the degree of variability in exposure or risk between individuals in an exposed population. Depending on assessment needs at a site, a range of point estimates of risk can be developed to represent variability in exposures. To support the evaluation of RME risk estimates using the point estimate approach described in Section 1.3, the Superfund program developed guidance with recommended default values for exposure variables as inputs to the risk equations (U.S. EPA, 1992a, 1996, 1997a, 2001d). These standardized values are a combination of average (e.g., body weight, skin surface area) and high-end exposure assumptions (e.g., drinking water intake, exposure duration). A
Page 1-15RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 CTE risk estimate is based on central estimates (e.g., mean, 50 th percentile) for each of the exposure variables. Available site-specific data on plausible mean and upper range values for exposure variables should be used to support CTE and RME risk estimates. The point estimate approach to risk assessment does not determine where the CTE or RME risk estimates lie within the risk distribution. For example, the RME risk estimated with the point estimate approach could be the 90 th percentile, the 99.9 th percentile, or some other percentile of the risk distribution. Without knowing what percentile is represented by the RME risk estimate, the risk manager might be unsure about the likelihood of the RME risk occurring or being exceeded in the receptor population and about what level of remedial action is justified or necessary to achieve the protective objectives of CERCLA. In a PRA, distributions used as inputs to the risk equations can characterize the inter-individual variability inherent in each of the exposure assumptions. By characterizing variability with one or more input distributions, the output from the Monte Carlo simulation is a distribution of risks that could occur in that population (Figure 1-3). The central tendency of the risk distribution (e.g., arithmetic mean, geometric mean, 50 th percentile) may be characterized as the CTE risk estimate. Similarly, the high-end of the risk distribution (e.g., 90 th to 99.9 th percentiles) is representative of exposures to the RME individual. In addition to providing a better understanding of where the CTE and RME risks occur in the distribution, a PRA can also provide an estimate of the probability of occurrence associated with a particular risk level of concern (e.g., cancer risk of 1E-05). A PRA that quantifies variability can be used to address the question, “What is the likelihood (i.e., probability) that risks to an exposed individual will exceed 1E-05?” Based on the best available information regarding exposure and toxicity, a risk assessor might conclude, “The estimated distribution for variability in risk across the target population indicates that 10% of the individuals exposed under these circumstances have a risk exceeding 1E-05.” This type of evaluation can be achieved using a technique known as one-dimensional Monte Carlo Analysis (1-D MCA). Guidelines for interpreting the high-end of the risk distribution in terms of the RME risk estimate are discussed further in Section 1.4.1 and Chapter 7.

Figure 1-3. Example of a probability distribution for risk illustrating the 95 th percentile and two different risk levels of concern (A and B). Assuming the 95 th percentile corresponds to the RME, the need for remedial action depends on how the RME risk compares with the risk level of concern. For Case A (RME > level of concern), remedial action may be warranted. For Case B (RM E < level o f conc ern), re med ial action may b e unne cessa ry. Page 1-16RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 The agreement (or lack of agreement) between the results of the point estimate calculations and the PRA calculations is expected to vary as a function of the form of the exposure or risk model and the attributes of the input variables. In general, if the terms in the denominator of the exposure or risk equation have low variability and do not approach zero, then the CTE point estimate is likely to agree quite well with the arithmetic mean from the PRA simulation, and the RME point estimate is likely to correspond to the high-end of the risk distribution (see discussion of RME range in Section 1.2.5). However, if the exposure or risk model has terms in the denominator that are a significant source of variability, or if the terms approach zero, then the agreement between the point estimate values and the PRA values may be more substantial. In addition, since the RME point estimate of risk reflects a combination of central tendency and high-end input values, it is difficult to anticipate what percentile of a distribution of variability it represents.

L If results of PRA calculations differ substantially from point estimate calculations, a risk manager may benefit from understanding the reasons for the differences and the relative strengths of the different approaches. Since point estimate and PRA approaches may yield different estimates of CTE and RME risks, the two approaches also may support different risk management decisions. This does not imply that either approach is invalid. Likewise, a correspondence between the point estimate and PRA results does not imply a greater accuracy or certainty in the modeling assumptions and inputs. Simply stated, PRA, based on the same risk equations and data as the point estimate approach, provides a different means of characterizing variability and uncertainty. Potential sources of variability and uncertainty in risk estimates should be identified, discussed, and to the extent practicable, quantified. Advantages and disadvantages of PRA and point estimate risk assessment are discussed in Section 1.2.4 and 1.3.
1.2.4 W HY IS U NCERTAINTY I MPORTANT IN RISK A SSESSMENT ? H OW IS U NCERTAINTY A DDRESSED BY THE P OINT E STIMATE AND P ROBABILISTIC A PPROACHES ? Uncertainty derives from a lack of knowledge. Various taxonomies of uncertainty relevant to risk assessment have been presented (Finkel, 1990; Morgan and Henrion, 1990; Cullen and Frey, 1999). U.S. EPA guidance, including the Final Guidelines Exposure Assessment Guidelines (U.S. EPA, 1992a), Exposure Factors Handbook (U.S. EPA, 1997b,c,d), and Guiding Principles for Monte Carlo Analysis (U.S. EPA, 1997g) describe a variety of different types of uncertainty in risk assessment as well as modeling strategies for quantifying uncertainties. Potential sources of uncertainty in risk assessment can be divided into one of three broad categories: (1) Parameter uncertainty - uncertainty in an estimate of an input variable in a model. In PRA, this may refer specifically to a statistical concept of uncertainty in estimates of population parameters (e.g., arithmetic mean, standard deviation) from random samples, due to the quality, quantity, and representativeness of available data as well as the statistical estimation method. (2) Model uncertainty- uncertainty about a model structure (e.g., exposure equation) or intended use, including the relevance of simplifying assumptions to the endpoint of the risk assessment, the choice of probability distribution to characterize variability, and interpolation or extrapolation beyond the scale used to calibrate a model from empirical data.

Page 1-17RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 (3) Scenario uncertainty - uncertainty regarding missing or incomplete information to fully define exposure. This may include descriptive errors regarding the magnitude and extent of chemical exposure or toxicity, temporal and spatial aggregation errors, incomplete analysis (i.e., missing exposure pathways), and potential mis-specification of the exposed population or exposure unit. Sources of uncertainty described by these categories are important because they can influence risk management decisions in both point estimate and probabilistic risk assessment. As additional sources of uncertainty are quantified and included in the risk assessment, uncertainty in risk estimates may appear to increase, suggesting there may be little confidence in a risk management decision. This situation may appear to be counterintuitive for those managers who expect confidence to increase as uncertainty is quantified. However, as discussed below and in Chapter 6 (see Section 6.4.2), uncovering and quantifying these sources of uncertainty may help to provide perspective, and make the decisions using the tiered process more transparent. In PRA, there are a variety of methods that can be used to effectively quantify uncertainty as well as communicate confidence in risk estimates (see Chapter 3, Section 3.4; Chapter 6, Section 6.4, and Section 6.5). Parameter uncertainty may be the most readily recognized source of uncertainty that is quantified in site-specific risk assessments at hazardous waste sites. Parameter uncertainty can occur in each step of the risk assessment process from data collection and evaluation, to the assessment of exposure and toxicity. Sources of parameter uncertainty may include systematic errors or bias in the data collection process, imprecision in the analytical measurements, inferences made from a limited database when that database may or may not be representative of the variable under study, and extrapolation or the use of surrogate measures to represent the parameter of interest. In the point estimate approach, parameter uncertainty is addressed in a qualitative manner for most variables. For example, the uncertainty section of a point estimate risk assessment document might note that a soil sampling plan yielded a small sample size that may not be representative of overall contaminant concentrations and, as a result, the risk estimate may over- or under-estimate actual risk. Uncertainty in the concentration term is addressed quantitatively to a limited extent in a point estimate approach by using the 95% UCL for the arithmetic mean concentration in both CTE and RME risk estimates; this accounts for uncertainty associated with environmental sampling and site characterization (U.S. EPA, 1992d, 1997f). The 95% UCL is combined in the same risk calculation with various central tendency and high-end point estimates for other exposure factors. Some examples of the models that EPA uses in the risk assessment process are the equations used to calculate exposure and risk, the linearized multistage model used to estimate cancer dose-response relationships, and media-specific models to estimate contaminant concentrations. All models are simplified, idealized representations of complicated physical or biological processes. Models can be very useful from a regulatory standpoint, as it is generally not possible to adequately monitor long term exposure for populations at contaminated sites. However, models that are too simplified may not adequately represent all aspects of the phenomena they were intended to approximate or may not capture important relationships among input variables. Other sources of model uncertainty can occur when important variables are excluded, interactions between inputs are ignored, or surrogate variables that are different from the variable under study are used.

Page 1-18RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 In most probabilistic assessments, the first step of analysis is usually an analysis of variability in exposure or risk. However, PRA methods may also be used to characterize uncertainty around the best estimate of the exposure or risk distribution. This is done using "2-dimensional" MCA (2-D MCA) (see Appendix D). One convention that has been used to distinguish between probability distribution functions for variability and uncertainty is to use subscripts “v” and “u” to indicate PDFs that characterize variability (PDFv) or uncertainty (PDFu). Figure 1-4 shows an example of the results of this type of 2-D MCA. This analysis can provide a quantitative measure of the confidence in the fraction of the population with a risk exceeding a particular level; which is sometimes referred to as a vertical confidence interval (Figure 1-4). For example, a conclusion Figure 1-4. Illustration of “Vertical” and “Horizontal” Confidence Intervals (or based on this type of limits) on a risk estim ate. T his type o f outpu t can b e pro duc ed fro m a 2 -D M CA in which pro bab ility distributio ns of un certain ty are intro duc ed into the risk eq uation . output might be, “While See Chapter 3 and A ppendix D for further discussion of 2-D M CA in quantitative the best estimate for the uncertainty analysis. variability distribution for risk across the target population indicates that 10% of the individuals exposed under these circumstances have a risk exceeding 1E-06, the uncertainty is such that we can only be reasonably certain (e.g., 95% sure) that no more than 20% of the exposed population has a risk that exceeds 1E-06.” Additionally, the output from a 2-D MCA can provide a quantitative measure of the confidence in the risk estimate for a particular fraction of the population; which is sometimes referred to as a horizontal confidence interval . This type of output might support the following type of conclusion, “While the best estimate for the variability distribution for risk across the target population indicates that 10% of the individuals exposed under these circumstances have a risk exceeding 1E-06, the uncertainty is such that we can only be reasonably certain (e.g., 95% sure) that the risk for this group of individuals does not exceed 2E-06.” The term “confidence interval” is used loosely in this context to convey information about uncertainty; however, it is not the same as a statistical confidence interval that one might obtain by estimating a population parameter from a sample. The vertical and horizontal bars shown in Figure 1-4 represent a range of possible estimates for the percentile given one or more sources of uncertainty that were included in the simulation. If the target audience for this graphic has a greater understanding of statistics, it may be less confusing if alternative phrases are used to describe the results, such as “credible interval” or “probability band”. In general, one should avoid developing input distributions to a PRA model that yield a single risk distribution that intermingles, or represents both variability and uncertainty. By separately characterizing variability and uncertainty, the output from a PRA will be easier to understand and communicate. A number of tools can aid in evaluating the uncertainty in estimated distributions for
Page 1-19RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 variability. Both simple and very complex approaches have been applied to this problem. Two basic methods for quantifying variability and parameter uncertainty simultaneously are described in Exhibit 1-5. PRAs that use these approaches can provide quantitative estimates of uncertainty in percentiles of the risk distribution based on confidence intervals or credible intervals for one or more parameter estimates. Techniques for characterizing both variability and uncertainty in PRA are discussed in more detail in Chapters 3, 4, 5, and 7, and Appendices A, C, and D. A common apprehension concerning the utility of PRA is that it may require more information and data than are available to generate credible PDFs. Risk assessors may feel that they can’t specify a PDF because they don’t have enough information to choose a distribution type, estimate parameters, or evaluate the representativeness to the site population of concern. However, if sufficient information exists to support a meaningful point estimate evaluation (i.e., if some sort of central tendency and upper bound values are available for each input variable), then it is usually possible to perform a screening level, or preliminary 1-D MCA that may provide additional useful information regarding variability. Likewise, an initial two-dimensional analysis may be performed that does not require collection of any new data, but simply characterizes uncertainty in the existing data. The results of such a 2-D MCA can help to identify the main sources of uncertainty in the risk results, and can support decisions to collect more data and/or proceed with additional tiers of analysis in order to improve the assessment. As with a preliminary 1-D MCA, the decision to conduct a more advanced probabilistic analysis does not always result in added data requirements. E 1-5 Q V U NCERTAINTY 1. Single source of uncertainty

For example,

A comparison of the

2. XHIBIT

UANTIFYING ARIABILITY AND

Run multiple one-d imens ional M onte C arlo simulations (1-D MC A) in which each simulation uses a different point estimate for a parameter selected from an uncertainty distribution, combined with PDFv’s for one or more variables. separate simulations can be run in which the mean of the exp osure conc entratio n varia bility distribu tion is represented by either the 95% lower or upper confidence limit on the mean. outp ut of these simulatio ns wo uld p rovid e a pa rtial characterization of the quantitative impact of uncertainty in the mean exposure concentration on the risk estimate (provided that certain conditions hold; i.e., risk increases with increasing exposure concentration) (see Chapter 3, Section 3.3.1). Multiple sources of uncertainty

Run a single tw o-dim ensio nal M onte C arlo simulatio n (2-D M CA ), in which sepa rate p rob ability distributions are specified for variability and param eter uncertainty and va lues from these distributions are randomly selected and used in each iteration of the Monte Carlo simulation (see Ap pen dix D ).

Use of probabilistic methods (e.g., MCA) to propagate variability and uncertainty through risk models offers five key advantages over point estimate approaches in addressing uncertainty in risk estimates: (1) Probabilistic methods may often provide a more complete and informative characterization of variability in exposure or risk than is usually achievable using point estimate techniques. (2) Probabilistic methods can provide a more quantitative expression of the confidence in risk estimates than the point estimate approach.
Page 1-20RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 (3) Sensitivity analysis methods using PRA may help risk assessors to better identify influential exposure factors. (4) Probabilistic methods can account for dependencies between input variables (e.g., body weight and skin surface area). (5) Probabilistic methods provide quantitative estimates of the expected value of additional information that might be obtained from data collection efforts (Morgan and Henrion, 1990). The importance of quantifying uncertainty in an expected value of information (EVOI) framework is discussed in Appendix D. Since both point estimate and probabilistic approaches in risk assessment are applied to the same conceptual models (i.e., the same exposure and risk models), uncertainties in the conceptual model are generally addressed in the same manner. If other models are available to explain or characterize a given phenomenon, the risk estimates associated with each of those conceptual models could be compared to determine the sensitivity of the risk to the uncertainty in the choice of a model (see Chapter 2 and Appendix A). For example, when deciding on a contaminant concentration term for tetrachloroethylene in groundwater for a residential exposure assessment 10 years in the future, it would be appropriate to compare and contrast several fate and transport models and their results before deciding on a concentration term.
1.2.5 REASONABLE M AXIMUM E XPOSURE AT THE H IGH - END Risk management decisions at Superfund sites should be based on an estimate of the risk to a reasonably maximum exposed receptor, considering both current and future land-use conditions. The RME is defined as the highest exposure that is reasonably expected to occur at a site. In general, risks corresponding to the 90 th to 99.9 th percentiles of the risk distribution estimated from a PRA are considered plausible high-end risks, and the RME risk should be selected within this range (see Section 1.2.4, Section 1.4.1, and Chapter 7 for further discussion). In comparison with point estimate risk assessments, PRA can provide the entire range of estimated risks as well as the likelihood of values within the range (i.e., the frequency distribution) As noted in Chapter 7, estimates of risk become more uncertain at very high percentiles (e.g., the 99.9 th ), so results of PRA calculations at these extreme values should be used with caution. Risk frequency distributions toward the 99.9 th percentile may be numerically unstable due to the uncertainties embedded in the input exposure assumptions. This guidance recommends that a risk manager select the RME in consultation with a risk assessor. One item for discussion should be the numerical stability of the high-end RME risk value (i.e., a stable value on the frequency distribution within the high-end range that could be reproduced in successive Monte Carlo simulations.) 

### 1.3 A DVANTAGES AND 

### D ISADVA NTAGES OF 

### P OINT 

### E STIMATE AND 

### P ROBABILISTIC 

### A PPROACHES As discussed in Chapter 2, a PRA should not be conducted until adequate point estimate calculations have been completed. Once this has been done, the potential benefits of proceeding to a PRA evaluation should be based on an understanding of the potential advantages and limitations in each approach. Potential advantages and disadvantages of point estimate calculations are summarized in Exhibit 1-6 and potential advantages and disadvantages of PRA are listed in Exhibit 1-7. Page 1-21RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 In general, compared to a point estimate risk assessment, a PRA based on the same state of knowledge may offer a more complete characterization of variability in risk, can provide a quantitative evaluation of uncertainty, and may provide a number of advantages in assessing if and how to proceed to higher levels of analysis. However, there are also some real and perceived disadvantages regarding additional effort on the part of both the risk assessor and the risk manager, and the potential to cause confusion if the effort is not clearly presented. In general, the key question to consider in deciding whether a PRA should be performed is whether or not the PRA analysis is likely to provide information that will help in the risk management decision making. For some sites, the additional information provided by a PRA will not affect the decision that would have been made with a point estimate approach alone, and a PRA will not be useful. However, when the decision whether or not to take action is not completely clear, PRA may be a valuable tool. The tiered process for PRA (Chapter 2) introduces the concept of scientific management decision points (SMDPs) to guide the complexity of analysis that may be needed for decision making. An SMDP marks a point in the process in which the potential that another analysis may influence the risk management decision is evaluated based on the problem formulation, the information available to define input variables, the results of previous analyses, and the feasibility of a subsequent analysis.

L A point estimate approach is conducted for every risk assessment; a probabilistic analysis may not always be needed.

E 1-6
A D P OINT E STIMATE A PPROACH Advantages • •

• additional work.
•

• • Disadvantages • • lost.
•

•

• XHIBIT

DVANTAGES AND ISADVANTAGES OF
Calculations are simple and do not require any advanced software. EP A ha s estab lished d efault inp uts and metho ds to h elp stan dard ize po int estima te calculations betwe en sites. Useful as a screening method—may allow risk management decisions with no Central tendency and RME estimates of risk provide a semi-quantitative measure of variab ility. Method is easily described and communicated. Requires less time to complete; not as resource intensive.

Com putational simplifications may resu lt in deviations from target values. Results are often viewed as “the answer”; importance of uncertainty is sometimes Information from sensitivity analysis is generally limited to dominant exposure pathways and chemicals of concern; may not highlight the key exposure variables and uncertain para meters. Does not provide a measure of the probability that risk exceeds a regulatory level of concern, or the level of confidence in a risk estimate. Provides fewer incentives for collecting better or more complete information.

Page 1-22RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

E 1-7
A D P R ISK A Advantages • • •

•

• information.
•
Disadvantages • ;

•

•

• •

• XHIBIT

DVANTAGES AND ISADVANTAGES OF ROBABILISTIC SSESSMENT
Can make more complete use of available data when defining inputs to the risk equation. Can provide a more comprehensive characterization of variability in risk estimates. Can provide a more comprehensive characterization of uncertainty in inputs, which may support statements regarding confidence in risk estimates. Communication of uncertainty in the risk assessment can help to build trust among stakeholders. Sensitivity analysis can identify the exposure variables, probability models, and model parameters that influence the estimates of risk. Puts the risk assessment in a Value-of-Information framework (see Appendix D ). Can identify data gaps for further evaluation/data collection and can use wider variety of site-specific Allows available site-specific information to inform the choice of high-end percentile from the risk distribution that corresponds with RME risk.

Concepts and approaches may be unfamiliar there is often apprehension regarding added costs and potential for inadvertent error and/or intentional misrepresentation. Places more burden on risk assessors to ensure the PRA is done correctly and on managers to understand and make decisions within a range of alternatives. May require more time and resources to select and fit probability distributions, and may require greater effort to communicate methodology and results. May convey false sense of accuracy when data are sparse. Complexities of the PRA approaches may obscure important assumptions or errors in basic exposure or risk models. If communication of the more complex PRA is unsuccessful, then it may generate mistrust of the assessment and risk management decisions.

Page 1-23RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001


### 1.4 C ONDUCTING AN 

### A CCEPTABLE 

### PRA In 1997, EPA issued a memorandum which contained its policy statement on PRA (U.S. EPA, 1997g). The 1997 EPA Policy Statement is as follows:
Itis the policy of the U.S. Environmental Protection Agency that such probabilistic analysis techniques as Mont e Carlo analysis, given adequate supporting data and credible assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk assessments. As such, and provided that the con ditions d escribed below are met, risk assessments using MonteCarlo analysis or other probabilistic techniqueswill be evaluated andutil ized in a manner that is consistent with other risk assessments submitted to th e Agency for review or consideration. I t is not the intent of this policy to recommend that probabilistic analysis be conducted for all risk assessments supporting risk management deci sions. Such a nalysis should be a part of a tiered approach to risk assessment that progresses from simpler (e.g., deterministic) to more complex (e.g., probabilistic) analyses as the risk management situationrequires. Use of Monte Carlo or other such techniques in risk assessments shall not be cause, per se , for rejection of the risk assessment by the Agency. For human health risk assessments, the applic ation of Monte Carlo and other probabilistic techniques has been limited to exposure assessments in the majority of cases. The current policy, Conditions for Acceptanc e and associated guiding principles are not intended to apply to dose response evaluations for human health risk assessment untilthis application of probabilistic analysis has been studied further. In the case of ecological risk assessment, however, this policy applies to all aspects including stressor and dose-response assessment. In support of this policy statement, EPA has outlined eight conditions for acceptance (in italics below), and good scientific practice of PRA. A PRA that is submitted to the Agency for review and evaluation should generally comply with each condition in order to ensure that adequate supporting data and credible assumptions are used in the assessment. These conditions are as follows: (1) The purpose and scope of the assessment should be clearly articulated in a "problem formulation" section that includes a full discussion of any highly exposed or highly susceptible subpopulations evaluated (e.g., children, the elderly). The questions the assessment attempts to answer are to be discussed and the assessment endpoints are to be well defined. (2) The methods used for the analysis (including all models used, all data upon which the assessment is based, and all assumptions that have a significant impact upon the results) are to be documented and easily located in the report. This documentation is to include a discussion of the degree to which the data used are representative of the population under study. Also, this documentation is to include the names of the models and software used to generate the analysis. Sufficient information is to be provided to allow the results of the analysis to be independently reproduced. Possible sources of bias inherent in the input distributions should be discussed along with the expected impacts on the resulting risk estimates. For example, if a site-specific study of fish consumption indicated consumption rates are five to ten times higher than other studies from similar
Page 1-24RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 populations, this possible bias or inaccuracy should be discussed in the document. Computer programs should generally be described in sufficient detail to allow the reviewer to understand all aspects of the analysis. Computer code/spreadsheets should provide adequate documentation and annotation. (3) The results of sensitivity analyses are to be presented and discussed in the report. Probabilistic techniques should be applied to the compounds, pathways, and factors of importance to the assessment, as determined by sensitivity analyses or other basic requirements of the assessment. Sensitivity analysis is a valuable tool in any tier of a PRA. (4) The presence or absence of moderate to strong correlations or dependencies between the input variables is to be discussed and accounted for in the analysis, along with the effects these have on the output distribution. (5) Information for each input and output distribution is to be provided in the report. 	This includes tabular and graphical representations of the distributions (e.g., probability density function and cumulative distribution function plots) that indicate the location of any point estimates of interest (e.g., mean, median, 95 th percentile). The selection of distributions is to be explained and justified. For both the input and output distributions, variability and uncertainty are to be differentiated where possible. (6) The numerical stability of the central tendency and the higher end (i.e., tail) of the output distributions are to be presented and discussed. As discussed in Section 1.2.5, numerical stability refers to the observed numerical changes in parameters of the output distribution (e.g., median, 95 th percentile) from a Monte Carlo simulation as the number of iterations increases. Because most risk equations are linear and multiplicative, distributions of risk will generally be right-skewed, and approximate a lognormal distribution. Values in the tails of the distribution typically are less stable than the central tendency, and the rate of convergence for the tails will depend on the form of the risk model, the skewness of the probability distributions selected for input variables and the numerical methods used to simulate probability distributions. Provided that appropriate numerical methods are employed, numerical stability is generally not a concern for most 1-D MCA models, which can be run with a sufficient number iterations in minutes with modern high speed computers; however, it can be an important consideration for more complex simulations, such as with 2-D MCA models. (7) Calculations of exposures and risks using deterministic (e.g., point estimate) methods are to be reported if possible. Providing these values will allow comparisons between the probabilistic analysis and past or screening level risk assessments. Further, deterministic estimates may be used to answer scenario specific questions and to facilitate risk communication. When comparisons are made, it is important to explain the similarities and differences in the underlying data, assumptions, and models. If results of PRA calculations differ substantially from point estimate calculations, a risk manager may benefit from understanding the reasons for the differences and the relative strengths of the different approaches. Sometimes, a closer look at uncertainties in the underlying data, assumptions, and
Page 1-25RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001 models will lead a risk assessor to revisit parts of the assessment in order to provide a more consistent basis for comparison. (8) Since fixed exposure assumptions (e.g., exposure duration, body weight) are sometimes embedded in the toxicity metrics (e.g., Reference Doses, Reference Concentrations, Cancer risk factors), the exposure estimates from the probabilistic output distribution are to be aligned with the toxicity metric.
1.4.1 K EY P OLICIES FOR A PPLYING PRA AT S UPERFUND S ITES EPA’s recommended process for conducting an acceptable PRA generally follows the policy and guiding principles presented above. In addition, this section highlights four key policies for conducting acceptable PRAs at hazardous waste sites. (1) Follow the Tiered Approach to PRA In accordance with the 1997 EPA Policy Statement (U.S. EPA, 1997g), this guidance recommends using a tiered approach when considering PRA to help with risk management decisions. A tiered approach begins with a relatively simple analysis and progresses stepwise to more complex analyses. The level of complexity should match the site-specific risk assessment objectives and the risk management goals. The tiered approach, with helpful suggestions on risk communication, is presented in Chapter 2. A brief introduction is given below. The premise for recommending a tiered approach is that there is a balance between the benefits of conducting a more complex analysis, and the cost in terms of additional time, resources, and challenges for risk communication. PRA may require additional resources compared with the point estimate approach, and may not be used routinely for screening level assessment. At more complex hazardous waste sites, PRA may not be warranted if the investment of time and resources is unlikely to provide information on variability and uncertainty in risk that will affect the risk management decision. This guidance recommends that a point estimate risk assessment be conducted in the first tier after completing the remedial investigation (RI) planning, site scoping, problem formulation, data collection, and the development of a site conceptual model. In general, when site decision making would benefit from additional analysis beyond the point estimate risk assessment, and when the risk manager needs more information to complete the RI/FS process, the risk manager would proceed to higher tiers. Sensitivity analysis should be conducted in each tier to guide decisions regarding data collection and the complexity of the analysis needed to characterize variability and/or uncertainty in risk. Sensitivity analysis can also play an important role in risk communication by supporting decisions to continue characterizing less influential variables with point estimates in higher tiers. (2) Select the RME Risk from the RME Risk Range (90 th to 99.9th percentile ) The RME is defined as the highest exposure that is reasonably expected to occur at a site. Final Guidelines for Exposure Assessment (EPA, 1992a) states that the “high-end” of exposure for a population occurs between the 90th and 99.9 th percentiles, with the 99.9 th percentile considered a bounding estimate. Using a point estimate approach, the calculation of the RME risk would be based on high-end input values in combination with average input values. For example, for estimation of risks from the ingestion
Page 1-26RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
of groundwater, default exposure is based on a high-end water intake rate (2 L/day), a high-end exposure frequency and duration (350 days/year for 30 years), and an average body weight (70 kg). With the probabilistic approach, the calculation of the RME risk would be based on a range of input values, or frequency distributions, including low, average, and high-end values for each of the input exposure factors. For example, for estimation of risks from ingestion of groundwater, exposure would be based on the combination of lognormal distributions for water intake rate, body weight, and exposure duration (each using a specified mean and standard deviation) and a triangular distribution for exposure frequency (using a specified minimum, most likely value, and maximum). As a result, the RME risk would become a probability distribution ranging from low- to high-end values based on varying a combination of input values. In PRA, a recommended starting point for risk management decisions regarding the RME is the 95 th percentile of the risk distribution (see Chapter 7). (3) Use PRA for Dose-Response in Ecological Assessment, not in Human Health Assessment Approaches to characterizing variability and uncertainty in toxicological information should reflect both the latest developments in the science of hazard and dose-response evaluation and consistent application of EPA science policy. This statement is consistent with the 1997 EPA Policy Statement presented in Section 1.4 above (U.S. EPA, 1997g). Probabilistic approaches to ecological dose-response assessment may be explored, as discussed and demonstrated in Chapter 4. This guidance does not develop or evaluate probabilistic approaches for dose-response in human health assessment and, further, discourages undertaking such activities on a site-by-site basis . Such activities require contaminantspecific national consensus development and national policy development. Parties wishing to undertake such activities should contact the OERR to explore ways in which they might contribute to a national process for the contaminant of interest to them. (4) Prepare a Workplan for EPA Review and Approval A workplan should be developed and submitted for review before commencement of a PRA. The workplan should document the combined decisions of the RPM and risk assessor involved in the risk assessment, and positions of the stakeholders. The workplan should address conditions and policies presented in this section of RAGS Volume 3: Part A , the software to be used, the exposure routes and models, and the input probability distributions and their basis, including appropriate literature references. The workplan is discussed in more detail in Chapter 2. A checklist of some of the key considerations to assist in the review of a PRA is provided in Appendix F. 

### 1.5 O RGANIZATION OF THE 

### G UIDANCE Subsequent chapters of RAGS Volume 3: Part A focus on the following topics: Chapter 2 - The Tiered Approach to PRA Chapter 2 includes information regarding organizational issues that may need to be considered by the RPM in developing a PRA. Examples, include: workplans, involvement of the Community Involvement Coordinator (CIC), additional meetings with communities, and review of PRA documents.
Page 1-27RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
Chapter 2 also presents the tiered approach in full detail. The approach begins with RI planning, scoping, problem formulation, and data collection. Tier 1 entails a point estimate risk assessment and sensitivity analysis. Tier 2 proceeds with additional data collection, a MCA to characterize variability and/or uncertainty, and a more in-depth sensitivity analysis. More advanced techniques are used in Tier 3 to simultaneously characterize variability and uncertainty. The endpoint of the tiered approach is to provide information that helps risk managers complete the RI/FS process. Chapter 3 - Probabilistic Human Health Risk Assessment Chapter 3 provides a discussion of how PRA approaches may be utilized in human health risk assessment. Probabilistic approaches focus on the exposure assessment, and an example is included to illustrate the application of the tiered approach to a human health risk assessment. Chapter 4 - Probabilistic Ecological Risk Assessment Chapter 4 provides a discussion of how PRA approaches may be utilized in ecological risk assessment. This includes a discussion of basic tactics, such as how to decide if, and when, a PRA is needed, along with technical discussions and examples of how to model variability and/or uncertainty in exposure, toxicity, and risk (characterized both as hazard quotients and responses) for different types of ecological receptors, both within and between species. The chapter also provides a discussion of how the results of an ecological PRA can be used in risk management decision making, and provides guidelines for planning and performing an ecological PRA. Chapter 5 - PRA and Preliminary Remediation Goals (PRGs) This chapter provides a discussion about issues associated with deriving PRGs from both point estimate risk assessment and PRA. Issues and limitations associated with back calculation are highlighted, along with an explanation and recommendation regarding the iterative forward calculations. Chapter 6 - Communicating Risks and Uncertainties in PRA Chapter 6 provides a basic overview of the current Superfund guidance on communicating with the public. With this as a basis, the chapter provides specific information regarding continuous involvement of stakeholders in the PRA process, various tools that may be useful in communicating the principles of PRA, organizational issues regarding planning of communication strategies, and examples of procedures that may be helpful at individual sites. This chapter also provides references to various documents on current approaches for communicating risk to the public. Chapter 7 - Role of PRA in Decision Making This chapter provides guidance on how to interpret the results of a PRA to determine if an unacceptable risk is present, and criteria to consider when moving from a risk-based PRG to a remedial goal.

Page 1-28RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
Appendix A - Sensitivity Analysis Important information from PRA includes the results of sensitivity analysis. This appendix outlines the methodology and interpretation of statistical methods used to conduct sensitivity analysis with point estimate and probabilistic models. Appendix B - Selecting and Fitting Distributions One of the more challenging aspects of PRA is choosing appropriate probability distributions to represent variability and uncertainty in the input variables. This appendix presents a process for selecting and fitting distributions to data, including hypothesizing families of distributions, parameter estimation techniques, and goodness-of-fit tests. Appendix C - Exposure Point Concentration (EPC) An important variable in most risk assessments is the concentration term. This appendix presents the basic principles of the EPC, and different methods for quantifying both variability and parameter uncertainty in the EPC. Appendix D - Advanced PRA Models Sometimes a more complex modeling approach can be used to improve the representativeness of the probabilistic risk estimates. These approaches are generally anticipated to be applied in Tier 3 of the tiered approach. Examples include the use of Microexposure Event modeling, geostatisics, and Bayesian Monte Carlo analysis. Appendix E - Definitions A list of definitions is provided at the beginning of each chapter. This appendix provides a compilation of all definitions presented in the guidance. Appendix F - Generic Checklist After a PRA has been submitted to the Agency, an efficient process is needed to evaluate the accuracy and clarity of the results. This appendix suggests a series of elements of the review process that can be adopted to structure the review of PRAs for both human health and ecological risk assessment. Appendix G - Frequently Asked Questions (FAQ) about PRA Risk assessors and risk managers who read RAGS Volume 3: Part A will find that probabilistic risk assessment covers a wide variety of topics ranging from statistical theory to practical applications and policy decisions. U.S. EPA OERR plans to maintain and periodically update a list of frequently asked questions and responses on an EPA Superfund web page at http://www.epa.gov/superfund/index.htm

. This appendix provides a preliminary list of anticipated questions.

Page 1-29RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001
Appendix H - Index This index includes keywords and concepts used throughout this guidance document. They are listed alphabetically with numbers indicating the appropriate chapter and page number(s) within each chapter. Commas separate page numbers within a chapter or appendix, while semi-colons separate chapters and appendices. For example: probability density function, 1-5, 6-8; 4-3, 10-12; C-1, 8-10. This would indicate Chapter 1, page 5, and pages 6-8; Chapter 4, page 3, and pages 10-12; Appendix C, page 1 and pages 8-10. 

### 1.6 N EXT 

### S TEPS FOR 

### PRA I MPLEMENTATION This guidance has presented the current principles, including the tiered approach, and examples to aid in conducting acceptable PRAs at Superfund sites. Policies and practices will change over time as scientific advances continue in the future. The PRA Workgroup intends to keep current and provide new information on EPA Superfund web page at http://www.epa.gov/superfund/index.htm . EPA expects to make the following PRA support items available on-line in the near future:

• RAGS Volume 3: Part B : A workbook that serves as a companion to RAGS Volume 3: Part A ; it will include case studies and examples in PRA. • Guidance on Probability Distributions : Documents and/or spreadsheets to aid in selecting and fitting probability distributions for input variables. • Guidance on Data Representativeness : A ranking methodology to evaluate data representativeness for various exposure scenarios. • Hands-On Training : Basic MCA training materials, and limited computer hands-on training sessions available to Regional EPA and State staff. • Access to PRA Workgroup : A workgroup to provide support on PRA to EPA regional risk assessors. • FAQs : A list of Frequently Asked Questions (FAQs) about PRA and responses from the PRA Workgroup, maintained and periodically updated on-line.

Page 1-30RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

REFERENCES FOR C HAPTER 1 Burmaster, D.E. and E.C. Udell. 1990. A Review of Crystal Ball®. Software Review 10: 343–345. Cullen, A.C. and H.C. Frey. 1999. Probabilistic Techniques in Exposure Assessment: A Handbook for Dealing with Variability and Uncertainty in Models and Inputs . Plenum Press, NY. Finley, B.L. and D.J. Paustenbach. 1994. The Benefits of Probabilistic Exposure Assessment: Three Case Studies Involving Contaminated Air, Water and Soil. Risk Anal . 14(1):53–73. Finkel, A.M. 1990. Confronting Uncertainty in Risk Management: A Guide for Decision Makers. Center for Risk Management, Resources for the Future. Washington, DC. Maddalena, R.L., T.E. McKone, D.P.H. Hsieh, and S. Geng. 2001. Influential Input Classification in Probabilistic Multimedia Models. Stochastic Environmental Research and Risk Assessment 15(1):1–17. Morgan, G.M. and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis . Cambridge University Press, NY. National Research Council (NRC). 1983. Risk Assessment in the Federal Government: Managing the Process . National Academy Press. Washington, DC. National Research Council (NRC). 1989. Improving Risk Communication. National Academy Press. Washington, DC. National Research Council (NRC). 1994. Science and Judgement in Risk Assessment. National Academy Press. Washington, DC. Presidential/Congressional Commission on Risk Assessment and Risk Management. 1997. Risk Assessment and Risk Management in Regulatory Decision Making . Final Report, Volume 2. Rugen, P. and B. Callahan. 1996. An Overview of Monte Carlo, A Fifty Year Perspective. Hum Ecol Risk Assess. 2(4):671–680.
U.S. EPA. 1989a. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM)(Part A, Baseline Risk Assessment). Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1–89/002. NTIS PB90-155581.
U.S. EPA. 1989b. Risk Assessment Guidance for Superfund. (RAGS) : Volume II. Environmental Evaluation Manual . Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/001.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR 300: 55 Federal Register, 8666-8865, March 8.

Page 1-31RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS): Volume I–Human Health Evaluation Manual Supplemental Guidance: “Standard Default Exposure Factors.” Interim Final. Office of Solid and Emergency Response, Washington, DC. OSWER Directive No. 9285.6-03.
U.S. EPA. 1991b. RAGS Volume I, Human Health Evaluation Manual (Part B: Development of Risk-based Preliminary Remediation Goals). Office of Emergency and Remedial Response. Washington, DC. EPA/540/R-92/003. December.
U.S. EPA. 1991c. RAGS Volume I, Human Health Evaluation Manual (Part C: Risk Evaluation of Remedial Alternatives). Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-01C. October.
U.S. EPA. 1991d. Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions

. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9355.0-30.
U.S. EPA. 1992a. Final Guidelines for Exposure Assessment . EPA/600/Z-92/001. 57 Federal Register, 22888-22938, May 29.
U.S. EPA. 1992b. Framework for Ecological Risk Assessment. EPA 630/R-92/001. February.
U.S. EPA. 1992c. Guidance on Risk Characterization for Risk Managers and Risk Assessors. Memorandum from F. Henry Habicht II, Deputy Administrator. Office of Solid Waste and Emergency Response, Washington, DC.
U.S. EPA. 1992d. Supplemental Guidance to RAGS: Calculating the Concentration Term . Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-081.
U.S. EPA. 1993. Use of IRIS ( Integrated Risk Information System) Values in Superfund Risk Assessment . Memorandum from William H. Farland and Henry L. Longest II. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.16, December 21.
U.S. EPA. 1994a. Role of Ecological Risk Assessment in the Baseline Risk Assessment. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-17.
U.S. EPA. 1994b. Use of Monte Carlo Simulation in Risk Assessments . Region 3, Hazardous Waste Management Division. Office of Superfund programs, Philadelphia, PA. EPA/903/F-94/001.
U.S. EPA. 1995a. Guidance for Risk Characterization . Office of Research and Development. Washington, DC. http://www.epa.gov/ORD/spc/rcpolicy.htm

. U.S. EPA. 1995b. Memorandum from Carol Browner on Risk Characterization . Office of the Administrator, Washington, DC. February 22.

Page 1-32RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

U.S. EPA. 1995c. Policy for Risk Characterization. Office of Research and Development. Washington, DC. http://www.epa.gov/ORD/spc/rcpolicy.htm

. U.S. EPA. 1995d. Policy on Evaluating Health Risks to Children . Office of Children’s Health Protection. Washington, DC. http://www.epa.gov/children/whatwe/rrguide.pdf

. U.S. EPA. 1995e. Use of Monte Carlo Simulation in Performing Risk Assessments (Technical Section). Region 8, Hazardous Waste Management Division, Superfund Management Branch Technical Guidance, Denver, CO, RA-10.
U.S. EPA. 1996. Final Soil Screening Guidance, May 17, 1996. Soil Screening User’s Guide . Office of Solid Waste and Emergency Response, Washington, DC. EPA 540/R-96/018.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments . Interim Final. Environmental Response Team, Edison, NJ. EPA/540/R-97/006, OSWER Directive No. 9285.7-25, June.
U.S. EPA. 1997b. Exposure Factors Handbook, Volume 1. Office of Research and Development,Washington, DC. EPA/600/P-95/002Fa.
U.S. EPA. 1997c. Exposure Factors Handbook, Volume 2. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fb.
U.S. EPA. 1997d. Exposure Factors Handbook, Volume 3. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fc.
U.S. EPA. 1997e. Guidance on Cumulative Risk Assessment. Phase 1. Planning and Scoping

. Washington, DC.
U.S. EPA. 1997f. Lognormal Distribution in Environmental Applications . Office of Research and Development, and Office of Solid Waste and Emergency Response, Washington, DC. EPA/600/R-97/006. December.
U.S. EPA. 1997g. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte Carlo Analysis . Office of Research and Development, Washington, DC. EPA/630/R-97/001. May.
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment . Final. National Center for Environmental Assessment, Washington, DC. EPA/630/R-95/002F.
U.S. EPA. 1999. Ecological Risk Assessment and Risk Management Principles for Superfund Sites . Final. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7-28P.
U.S. EPA. 2000. Risk Characterization Handbook . Office of Science Policy. Office of Research and Development. EPA 100-B-00-002. December.
Page 1-33RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 1 ~ December 31, 2001

U.S. EPA. 2001a. Risk Assessment Guidance for Superfund : Volume I. Human Health Evaluation Manual (Part D, Standardized Planning, Reporting, and Review of Superfund Risk Assessments). Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-47. December.
U.S. EPA. 2001b. Risk Assessment Guidance for Superfund: Volume 1, Human Health Evaluation Manual (Part E, Supplemental Guidance for Dermal Risk Assessment) . Interim. Review Draft–For Public Comment. Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285.7-02E-P. September.
U.S. EPA. 2001c. Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites

. Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9355.4-24. December.
U.S. EPA. 2001d. The Role of Screening-Level Risk Assessments and Refining Contaminants of Concern Baseline Risk Assessments. Office of Solid Waste and Emergency Response. 12 th Intermittent Bulletin, ECO Update Series. EPA 540/F-01/014. June.

Page 1-34RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 

## C HAPTER 

## 2
W ORKPLAN AND 

## T HE 

## T IERED 

## A PPROACH
2.0 I NTRODUCTION While probabilistic risk assessment (PRA) can provide useful information for risk management, not all site decisions will benefit from probabilistic approaches. Similarly, not all PRAs need involve complex models and quantitative uncertainty analysis methods; often, very useful information can be obtained by taking the point estimate approach one step further to explore variability in selected input variables. The level of effort and complexity of the risk assessment should match site-specific needs. The use of a tiered approach for moving from a point estimate risk assessment to PRAs of varying levels of complexity is recommended (Figure 2-1 and 2-2). This chapter outlines the basic steps of a tiered approach for including PRA in a site risk assessment. The major feature of the tiered approach is an iterative evaluation of the risk estimates developed at each tier to determine if they are sufficient for risk management decisions. Built into the tiered approach are opportunities for communication with stakeholders with a view to saving time and costs, and facilitating a successful remedial process.
2.1 W ORKPLAN In practice, the potential value of PRA may be considered at various planning stages of a risk assessment. For some sites, PRA and point estimate risk assessment approaches may be discussed in the initial scoping of the risk assessment. For other sites, PRA may become a viable option only after the point estimate risk assessment results are available. Ideally, PRA should be considered as early as possible in the planning of risk assessment activities at a site so that sampling plans and data collection efforts may be appropriately directed. Initial PRA discussions should be included as part of the risk assessment workplan. If a PRA is being considered following completion of a point estimate risk assessment, the original workplan for the point estimate assessment should be expanded to include needs that are unique to PRA. The methods and procedures used to prepare a workplan to gather additional information for a baseline point estimate risk assessment are documented in RAGS Volume I: Part A (U.S. EPA, 1989). This chapter of RAGS Volume 3: Part A describes the procedures that would be used to prepare a workplan to gather additional information to conduct a PRA. Separate workplans may be warranted for human health and ecological risk assessments. Like the quality assurance project plan (QAPP), the workplan for a PRA should document the combined decisions of the remedial project manager (RPM) and the risk assessor. Meaningful involvement of stakeholders early in the decision-making process also will save time and effort.

Page 2-1RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001

E 2-1
D C 2

counted with integers (e.g., one, two, three) and that has no upper limit. Examples include the number of tosses Countably finite implies there is an upper CTE Risk - The estimated risk corresponding to the central tendency exposure.
using these inputs to calculate a range of risk values. Parameter For example, a truncated normal probability distribution may

estimates for exposure and toxicity. estimate depending on the choice of inputs. payment of Superfund cleanup costs. Preliminary Remediation Goal

equation quantity. density comes

Also called a probability model

.

EPA, 1989). that is still within the range of possible exposures. XHIBIT

EFINITIONS FOR HAPTER Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in a population, usually considered to be the mean or median of the distribution. Countably Infinite - Used to describe some discrete random variables, this term refers to a set of numbers that can be required for a coin to show a head—we can count each toss, but it is possible that at least one more toss is needed. The number of dust particles in a volume of air is another example. limit (e.g., days of work per year). Monte Carlo Analysis (MCA) or Monte Carlo Simulation - A technique for characterizing the uncertainty and variability in risk estimates by repeatedly sampling the probability distributions of the risk equation inputs and - A value that characterizes the distribution of a random variable. Parameters commonly characterize the location, scale, shape, or bounds of the distribution. be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max [bounds]. It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean ingestion rate). Point Estimate - In statistical theory, a quantity calculated from values in a sample to estimate a fixed but unknown population parameter. Point estimates typically represent a central tendency or upper bound estimate of variability. Point Estimate Risk Assessment - A risk assessment in which a point estimate of risk is calculated from a set of point Such point estimates of risk can reflect the CTE, RME, or bounding risk Potentially Responsible Party (PRP) - PRPs are individuals, companies, or any other party that are potentially liable for (PRG) - Initially developed chemical concentration for an environmental medium that is expected to be protective of human health and ecosystems. PRGs may be developed based on applicable or relevant and appropriate requirements (ARARs), or exposure scenarios evaluated prior to or as a result of the baseline risk assessment. (U.S. EPA, 1991a, 1991b). Probabilistic Risk Assessment (PRA) - A risk assessment that yields a probability distribution for risk, generally by assigning a probability distribution to represent variability or uncertainty in one or more inputs to the risk Probability Density Function (PDF) - A graph that shows the probability of occurrence of an unknown or variable A PDF is used to characterize a continuous random variable, X. PDFs can be used to display the shape of the distribution for an input variable or output variable of a Monte Carlo simulation. The term from the concept that a probability at a point, x, for a continuous distribution is equal to the area under the curve of the PDF associated with a narrow range of values around x. Probability Distribution - A mathematical representation of the function that relates probabilities with specified intervals of values for a random variable. Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random variable. The mass at a point refers to the probability that the variable will have a value at that point. Random Variable - A variable that may assume any value from a set of values according to chance. Discrete random variables can assume only a finite or countably infinite number of values (e.g., number of rainfall events per year). A random value is continuous if its set of possible values is an entire interval of numbers (e.g., quantity of rain in a year). Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S. The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case) Remedial Investigation/Feasibility Study (RI/FS) - Studies undertaken by EPA to delineate the nature and extent of contamination, to evaluate potential risk, and to develop alternatives for cleanup.

Page 2-2RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001

E 2-1
D C 2 RME Risk
SMDPs provide a

relative contributions to model output variability and uncertainty.
< - A statistic r ( r 2 ) is the fraction of the variance

< elasticity

. < - A “distribution free” or nonparametric statistic r that measures

r 2

. Examples include limited XHIBIT

EFINITIONS FOR HAPTER —C ontinued - The estimated risk corresponding to the reasonable maximum exposure. Scientific/Management Decision Point (SMDP) - A point during the tiered process in PRA when the risk assessor communicates results of the assessment to the risk manager. At this point, the risk manager determines whether the information is sufficient to arrive at a decision or if additional data collection or analysis is needed. tool for transitioning to a subsequent tier or for exiting the tiered process. Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes in the values of the model’s input(s). Sensitivity analysis can provide a quantitative ranking of the model inputs based on their Common metrics of sensitivity include: Pearson Correlation Coefficient that measures the strength and direction of linear association between the values of two quantitative variables. The square of the coefficient of one variable that is explained by the variance of the second variable. Sensitivity Ratio - Ratio of the change in model output per unit change in an input variable; also called Spearman Rank Order Correlation Coefficient the strength and direction of association between the ranks of the values (not the values themselves) of two quantitative variables. See Pearson (above) for Uncertainty - Lack of knowledge about specific variables, parameters, models, or other factors. data regarding the concentration of a contaminant in an environmental medium and lack of information on local fish consumption practices. Uncertainty may be reduced through further study.

A PRA workplan should be developed early in the risk assessment planning process for the site, regardless of who will actually develop the PRA (e.g., Environmental Protection Agency (EPA), EPA contractor, or potentially responsible party (PRP)). If a PRP performs the PRA, the workplan should be submitted to EPA for review and approval prior to commencing the PRA. It should describe the intended PRA in sufficient detail so that EPA can determine if the work products will adequately address risk assessment and management needs (see Exhibit 2-2 for contents of a typical workplan). It is important that the risk assessor and RPM discuss the scope of the probabilistic analysis and the potential impact it may have on the remedial investigation/feasibility study (RI/FS).

L Given the time and effort that can be expected to be invested in conducting a PRA, it is important that a workplan undergo review and approval by EPA, prior to proceeding with the assessment. In general, regions should not accept probabilistic analysis when a workplan for the analysis has not been submitted to the Agency, and approved by the regional risk assessor and RPM. The tiered process for PRA, described in Section 2.3, is an iterative process. As new information becomes available, it should be used to evaluate the need to move to a higher tier. The decision to move an assessment to a higher tier of complexity should result in a revised workplan reflecting the greater complexity and demands of the higher tier. The proposed probabilistic sensitivity analysis developed at the lower tier should be included in the revised workplan, along with a point estimate risk assessment based on any data collected as part of a lower tier. The probabilistic methods used in a PRA can often be restricted to the chemicals and pathways of concern that contribute the greatest risk. The less sensitive chemicals and exposure pathways should still remain in the PRA using point estimates, unless there is a compelling reason to exclude them from the assessment altogether. As stated in Appendix A (Section A.1, Risk Communication ), the decision to represent an input variable with a point estimate, rather than a
Page 2-3RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 probability distribution, will generally be made on a case-by-case basis. The decision will reflect an attempt to balance the benefits of simplifying the analysis (e.g., easier to communicate; focuses discussion on more critical areas) with the potential for arbitrarily reducing the variance in the output distribution (e.g., discounting variability in multiple variables with negligible contributions to risk may end up having a non-negligible effect on the RME percentile). Throughout the process of developing the PRA, EPA risk assessor and other contributors to the assessment should have a continuing dialogue to discuss the elements of the workplan and their potential impacts on the assessment. This dialogue, along with interim deliverables, will help to ensure that the risk assessment report will meet the needs of the Agency and that any problems are identified and corrected early in the process.
2.2 S PECIAL A DMINISTRATIVE C ONSIDERATIONS IN PRA Inclusion of a PRA in the RI/FS will generate certain administrative activities for the RPM. The scope of these activities will depend on whether the PRA is conducted by EPA and its contractors or by the PRP. The following sections provide practical advice for the RPM who is considering applications of PRA at a site. E 2-2
E XAMPLES OF I MPORTANT C ONTENTS OF ORKPLAN
1. endpoints and/or human risk 2. assessment 3. and proceeding to the subsequent tiers 4.

5.

6.

7. term 8. analysis 9. XHIBIT

A PRA W Statement of the ecological assessment Summ ary of the point estimate risk Potential value added by conducting a PRA Discussion of adequacy of environmental sampling for PRA or moving to a successive tier (e.g., data qua lity issues) De scriptio n of the m ethod s and mod els to be used (e.g., model and parameter selection criteria) Propo sal for obtaining and basis for using exposure factor distributions or ecological toxicity distributions Methods for deriving the concentration Pro pos al for p rob abilistic se nsitivity Softw are (e .g., date a nd ve rsion o f prod uct, random numb er generator) 10. Bibliography of relevant literature 11. Propo sed schedule, discussion points, and expertise needed

2.2.1 S CO PING O F PRA The RPM will generally be involved in the discussions among EPA project team, as well as PRPs and other stakeholders, regarding the level of PRA that is appropriate for the site. As outlined in the tiered approach (see Section 2.3), the scope and complexity of the PRA should satisfy the risk assessment and management decision making needs of the site. Team members should meet to discuss the scope of the PRA, the anticipated community outreach, and the required level of review. These discussions can be useful for ascertaining the level of contractor involvement, specific requirements for deliverables from PRPs, and the anticipated number of responses to comments. These meetings should include consideration of funding, resources, and availability of personnel to work on the PRA.

Page 2-4RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 2.2.1.1 PRA S CO PE OF W ORK FOR F UND - LEAD S ITES A Statement of Work (SOW) should be developed before any work is started on a PRA, regardless of whether the PRA is to be submitted to the Agency or developed by the Agency. The SOW should outline the general approach that EPA and its contractor will use in developing the PRA. The SOW should include the general approaches for the following PRA items: selection of input probability distributions, documentation of methods and results, selection of computer programs, submission of computer codes and outputs, comparison of the results from the point estimate and probabilistic assessments, and the format for presenting the final PRA in the RI/FS document. The SOW should be sufficiently detailed to support a milestone schedule, which should be submitted as part of the SOW. Based on the complexity of the PRA, and consistent with the RAGS Volume I: Part D principles of involving the risk assessor early and often in the risk assessment process (U.S. EPA, 2001), it may be appropriate to obtain submission of interim deliverables to allow the risk assessor the opportunity to identify potential problems early in the process. Within the RI/FS workplan, additional resources may be required to hold additional meetings, to respond to comments specific to the PRA, and to develop handouts describing PRA in terms accessible to a wider audience than risk assessors. Where appropriate, these additional resource requirements should be included in the SOW along with interim and final deliverable dates. Chapter 6 provides guidance on communicating concepts and results of PRA to various audiences.
2.2.1.2 PRP S CO PE OF W ORK FOR PRP-L EAD S ITES The SOW for PRP-lead sites should follow the same general outline as the SOW for fund-lead sites (Section 2.2.1.1). Legal documents such as Unilateral Orders, Administrative Orders of Consent, and Consent Decrees should contain language requiring the PRP to submit a workplan before any work on the PRA is started. It is also important that interim deliverables, including computer code or spreadsheet models, be submitted so that EPA can review and verify the results of the PRA. A comparison of the results of the PRA and the point estimate assessment should be included in the final RI/FS. Depending on the complexity of the site and the anticipated PRA, the RPM may be involved in more extensive negotiations with the PRPs. These negotiations may involve both EPA staff and contractor support. These activities may need to be included in the appropriate SOWs. If warranted by the complexity of the PRA, the RPM may consider the need to expand oversight contracts to include additional resources for the contractor to review and comment on the interim deliverables and finalize the PRA. This may require a specialized level of expertise that will need to be discussed with the contractor. Further, the contract section regarding community involvement may also need to be expanded to include additional resources for developing handouts describing PRA in terms accessible to a wider audience than risk assessors and for holding additional community meetings.

Page 2-5RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 2.2.2 D EVEL OPM ENT O F P ROBABILITY D ISTRIBUTIONS A key component of any PRA is the selection of representative probability distributions. The information available to support the characterization of variability or uncertainty with probability distributions may be an important factor in the decision to conduct a PRA. In some cases, this may require resources to conduct exploratory data analysis or to collect site-specific information. As part of this process, a PRA using preliminary distributions based on the available information may be considered to identify the variables and exposure pathways that may have the strongest effect on the risk estimates. Appendix B (Section B.2.0) provides a more detailed description of preliminary distributions and their potential role in the tiered process. All of these activities may require extensive discussions with the PRPs and the community. In addition, for PRP-lead sites, they may require additional resources to critically review the proposed distributions. The RPM should consider these potential activities in developing the SOW and legal documents to assure adequate resources are available to address them.
2.2.3 EPA REVIEW OF PRA D OCUMENTS The review of PRA documents may require more time than is usually allocated for point estimate risk assessments. In part, the additional time is needed for reviewing and discussing input distributions, for developing and running computer simulations, and for discussing outcomes of the assessment with the PRP or EPA contractor. The early involvement of an EPA risk assessor may reduce the time needed for review of the final risk assessment documents, although additional review time may still be required, depending on the complexity of the PRA conducted. In addition to EPA’s review, it may also be important to include external reviewers with specialized expertise in PRA to aid in the review. This additional support may involve resources and time to review documents and verify simulation results, as well as additional contractual arrangements. As stated in Chapter 1, Section 1.4 (Conducting an Acceptable PRA), it is important that negotiations with the PRP address the assurance that adequate details will be included in the submission so that the methods can be evaluated, and the results independently reproduced.
2.2.4 P EER -R EVIEW Depending on the level of complexity of the PRA, and whether new science is being used, it may be necessary to conduct a peer review of the document. The Agency’s guidance on peer review (U.S. EPA, 2000b) should be consulted for information regarding the criteria for determining whether or not a peer review is appropriate and, if it is, the process that should be followed.
2.2.5 RESPONSE TO C OMMENTS ON PRA The time and resources needed to respond to comments on a PRA may vary depending on the complexity of the PRA. In developing the SOW, workplan, and schedule for the RI/FS, it is important that the RPM include adequate resources and time for the thorough evaluation of the PRA. In developing the response to comments, it may be necessary to consider alternative PRAs submitted by reviewers. The RPM should plan for sufficient time and resources needed for such activities.

Page 2-6RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 2.2.6 A DMINISTRATIVE RECORD Criteria should be established for documentation to be included in the administrative record. Examples may include documentation regarding the basis for selection of input distributions, a description of the design of the PRA conducted, the computer codes used in simulations, how tiering decisions are made, and the results of the PRA. The RPM should consider using technologies such as a CD-ROM to document the appropriate information for the record.
2.2.7 C OMMUNICATION WITH S TAKEHOLDERS Chapter 6 provides details regarding the goal of early involvement of the public in the PRA process. For example, Section 6.1 of Chapter 6 provides additional topics for consideration in development of community involvement plans (CIPs) where PRA is considered. In general, early involvement of the community in the RI/FS process is important, but such involvement should meet the site-specific needs. Important considerations include resources, funding, and the level of effort appropriate for the site.
2.2.8 C OMMUNICATION WITH EPA M ANAGEMENT Communication with EPA managers regarding PRA is discussed in Chapter 6. The RPM may need to consider allocating additional resources for prebriefings of appropriate management levels, development of handouts, and follow-up to the management meetings. Coordination with appropriate EPA staff and contractors may be necessary to assure the communication is effective. 

### 2.3 O VERVIEW OF THE 

### T IERED 

### A PPROACH The tiered approach presented in this guidance is a process for a systematic, informed progression to increasingly more complex risk assessment methods including PRA. A schematic presentation of the tiered approach is shown in Figure 2-1 and Figure 2-2. Higher tiers reflect increasing complexity and, in many cases, will require more time and resources. Higher tiers also reflect increasing characterization of variability and/or uncertainty in the risk estimate, which may be important for making risk management decisions. Central to the concept of a systematic, informed progression is an iterative process of evaluation, deliberation, data collection, work planning, and communication (see Figure 2-2). All of these steps should focus on deciding (1) whether or not the risk assessment, in its current state, is sufficient to support risk management decisions (a clear path to exiting the tiered process is available at each tier); and (2) if the assessment is determined to be insufficient, whether or not progression to a higher tier of complexity (or refinement of the current tier) would provide a sufficient benefit to warrant the additional effort. The deliberation cycle provides an opportunity to evaluate the direction and goals of the assessment as new information becomes available. It may include evaluations of both scientific and policy information. The risk manager, in the decision-making process, is encouraged to seek input on a regular basis from EPA staff and other stakeholders. Exhibit 2-3 lists some of the potential stakeholders that may contribute to the deliberation process.

Page 2-7RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 Although PRA may involve technical dialogue between EPA and outside “experts”, input from members of the general public who may have an interest in the outcome of the remedial process should also be sought at appropriate stages of the process. Frequent and productive communication between EPA and stakeholders throughout the risk assessment process is important for enhancing the success of a PRA.

E 2-3
S P OTENTIALLY I EPA’ S D -M AKING P PRA

C

C

C

C etc.)

C

C

C etc.) XHIBIT

TAKEHO LDERS NVOLVED IN ECISION ROCESS FOR EPA risk assessors and manage rs Memb ers of the public Rep resentatives from state o r county environm ental or health agenc ies Other federal agencies (e.g., health agencies, Natural R esource D amage Assessment trustees, Tribal government representatives Po tentially resp onsib le parties and their representatives Rep resentatives from fed eral facilities (e.g., De partm ent of D efense, Dep artme nt of Energy,

Page 2-8RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001

Figure 2-1 . Schematic Diagram of Tiered Approach.

1 Examples of advanced methods for quantifying temporal variability, spatial variability, and uncertainty (see Appendix D)

Page 2-9RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001

Fig ure 2-2 . Schematic diagram of deliberation/decision cycle in the tiered process for PRA. SMDP refers to a scientific/management decision point, which implies that the decision involves consideration of not only the risk assessment, but also Agency policy, stakeholder concerns, cost, schedule, feasibility and other factors.

Page 2-10RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 2.3.1 G ETTING S TARTED All risk assessments should begin with problem formulation, scoping, preparation of a workplan (Section 2.1), and data collection. Problem formulation generally is an iterative process where reevaluation may occur as new information and data become available. The RPM should convene a scoping meeting prior to any risk assessment activities. Depending on the site-specific factors, discussion of performing a PRA may be appropriate at this initial scoping meeting. Alternatively, this discussion may be more productive at a later stage of the tiered process. The risk manager should initiate discussions with EPA staff and other stakeholders early in the process, well before planning a risk assessment. Early communication with risk assessors or other EPA staff can help the risk manager evaluate the adequacy of the current information and plan additional data-gathering activities. Early communication with communities and other stakeholders should establish trust and facilitate a successful remedial process (see Chapter 6 on risk communication). Generally, once the appropriate steps have been taken to adequately formulate and identify the problem and complete a workplan (Section 2.1), data collection efforts towards the point estimate risk assessment may begin. The process for conducting a point estimate risk assessment (Tier 1) is documented elsewhere in various RAGS volumes and related Superfund risk assessment guidance documents (e.g., U.S. EPA, 1989, 2001).
2.3.2 T IER 1 Tier 1 consists of the well-established process for planning and conducting human health and ecological point estimate risk assessments. Typical elements of a Tier 1 risk assessment, as they relate to higher tiers, are presented in Exhibit 2-4. A more detailed discussion of these elements can be found in Chapters 3 and 4 and Appendix A (Sensitivity Analysis). A more detailed schematic presentation of the tiered process, showing the various elements of the deliberation/decision cycle and their linkage to higher tiers is shown in Figure 2-2. The two main factors to consider when determining whether the results of a risk assessment are sufficient for decision making are: (1) the results of a comparison of the risk estimate with the risk level of concern; and (2) the level of confidence in the risk estimate. E 2-4
T YPICAL E LEMENTS OF T ISK A

- point XHIBIT

IER 1 R SSESSMENT Analysis Tool - point estimate risk assessment Variability Modeling - semi-quantitative, using centra l tende ncy exp osure (CT E) an d rea sona ble maximum expo sure (RME ) estimates as input variables Uncertainty Modeling - semi-quantitative using con fidenc e limits on certain poin t estimate s (e.g., concentration term) Sensitivity Analysis - point estimate calculation of percentage contribution of exposure pathways, for both CT E and RM E risk. Systematically vary one input variable at a time across a plausible range and rank inp uts bas ed o n sensitivity ra tios or se nsitivity score s. Risk-Based Decision-Making Output estimate of risk— Does the point estimate exceed the risk level of concern?

In Tier 1, comparison of risk estimates with risk levels of concern is relatively straightforward, since the outcome of a point estimate risk assessment is a single estimate of risk that either will exceed or not exceed the risk level of concern. Evaluating confidence in the Tier 1 risk estimates is more difficult because quantitative measures of
Page 2-11RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 uncertainty often are not easily obtained from a point estimate analysis. Uncertainty arises from two main sources: (1) uncertainty in the inputs to the risk equations that stems from lack of knowledge (data gaps), and (2) uncertainty in the accuracy of the point estimate that stems from the mathematical simplifications that are inherent in point estimate computations. There are usually many sources of uncertainty in the values used to calculate risk. One of the most familiar (but not always the most significant) is uncertainty in environmental concentration values of contaminants. This source of uncertainty is usually accounted for by calculating a 95% upper confidence limit (95% UCL) for the mean concentration in the exposure equation (U.S. EPA, 1992b). Chapter 5, Appendix C, and Appendix D provide more complete discussions of policies and methods for quantifying uncertainty in the exposure point concentration. Uncertainties in other variables in the risk equations (intake rates, exposure frequency and duration, toxicity factors, etc.) may also be significant, and are often addressed by choosing inputs that are more likely to yield an overestimate than an underestimate of risk. These sources of uncertainty are usually addressed qualitatively, by providing a discussion of the likely direction and magnitude of the error that may be associated with the use of the specific inputs (U.S. EPA, 1989). Stakeholders can provide useful information about uncertain variables and sources for site-specific data. This is an important reason to ensure that stakeholders are given the opportunity to review the risk assessment and be involved in the process.
Decision Alternatives The evaluation of the point estimate risk assessment will yield one of two outcomes: (1) sufficient for risk management decisions; or (2) insufficient for risk management decisions. If the risk manager views the results of the point estimate risk assessment as sufficient for risk management decision making, the risk manager can exit the tiered approach and complete the RI/FS process (Figure 2-2). Depending on site-specific information, the results may support a decision for “no further action” or for a “remedial action.” A “no further action” decision may result when the risk estimate is clearly below the level of concern (e.g., the National Oil and Hazardous Substances Pollution Contingency Plan (NCP) risk range of 1E-04 to 1E-06) and confidence in the risk estimate is high. A decision for remedial action may result when a national standard (e.g., maximum contaminant levels (MCLs) applied to groundwater) may be exceeded , or when the risk is clearly above the level of concern (e.g., the NCP risk range of 1E-04 to 1E-06) and confidence in the risk estimate is high. The decision for a specific remedial action involves consideration of the NCP’s nine criteria for remedial decisions (U.S. EPA, 1990) and other site-specific factors. An alternative conclusion would be that the results of the point estimate risk assessment are not sufficient for risk management decision making. For example, results may not be sufficient when the risk estimate is within the NCP risk range of 1E-04 to 1E-06 and confidence in the risk estimate is low. In this case, the risk manager should not exit the tiered approach. Instead, appropriate steps should be taken to increase the confidence that a management decision is protective. These steps may include discussing the point estimate sensitivity analysis, identifying data gaps, communicating with stakeholders (e.g., to obtain site-specific information), discussing the potential value of conducting a PRA (or a more advanced probabilistic analysis), work planning, and additional data collection (see Figure 2-2). A sensitivity analysis can be a valuable component of the evaluation of a risk assessment. Sensitivity analysis can identify important variables and pathways that may be targets for further analysis and data collection. The type of information provided by a sensitivity analysis will vary with each tier of a PRA. Several methods are available at each tier, and the results of the analysis can vary greatly
Page 2-12RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 depending on the methods used. A comprehensive discussion of these methods is presented in Appendix A and briefly summarized here. Sensitivity analysis in Tier 1 will usually involve relatively simple methods and will not involve Monte Carlo simulation. A typical approach would be to calculate the relative contributions of individual exposure pathways to the point estimate of risk. A more complex approach involves selecting values from a plausible range for a specific input variable to the exposure or risk equation and to use these values (i.e., low-end estimate and high-end estimate) to calculate corresponding point estimates of risk. The sensitivity of the risk estimate to each variable is then evaluated by calculating a sensitivity ratio, which is simply the percentage change in the risk estimate divided by the percentage change in the input variable value (see Appendix A, Section A.2.1.3, Sensitivity Ratios). The sensitivity ratio (SR) approach is typically applied to one variable at a time because jointly varying point estimates for multiple variables can be cumbersome (see Chapter 3, Table 3-2 for an example of two jointly varied inputs). Information provided by the SR approach is generally limited to bounding estimates of risk based on small deviations and/or plausible ranges of point estimates for inputs. Because the point estimate approach does not generate a distribution of risk, SRs cannot provide quantitative information about the relative contributions of input variables to the variance in risk or the uncertainty in selected percentile of the risk distribution. This limitation of the SR approach may be particularly important if the ranking of input variables may change depending on the percentile range that is evaluated. For example, in a probabilistic analysis, the soil ingestion rate variable may contribute most to the variability in risk across the entire risk distribution, but the exposure duration may be the driver in the high-end (> 90 th percentile) of the risk distribution, where the RME risk is defined. In addition, for standard product-quotient risk equations, the SR approach also has difficulty distinguishing the relative importance of exposure variables in the risk equation. Appendix A presents a hypothetical example to illustrate why this happens for the common risk equations. An improvement over the SR approach, called Sensitivity Score, involves weighting each ratio by the variance or coefficient of variation of the input variable when this information is available. In general, the most informative sensitivity analysis will involve Monte Carlo techniques (see Appendix A, Table A-1). Potential strengths and weaknesses of sensitivity analysis methods may be an important factor in deciding whether or not to conduct a probabilistic analysis in Tier 2. Once data gaps have been identified, steps may be taken to gather additional data and revise the point estimates of risk based on these data. As with any data collection effort, the data quality objectives (DQO) process should be followed to obtain samples appropriate for the risk assessment and sufficient to support the remedial decision (U.S. EPA, 1992a, 1993, 1994, 2000a). The deliberation and decision cycle (Figure 2-2) should then be reiterated to determine if the refined risk assessment is sufficient to support risk management decisions. The collection of additional data may also provide a compelling reason to consider moving to Tier 2 and conducting a PRA. If, during the PRA discussions, it is determined that information from a PRA may influence the risk management decisions, PRA may be warranted. This iterative process of collecting data, recalculating point estimates, and reconsidering the potential value of PRA may continue until sufficient data are available to support risk management decisions, or data collection efforts are not possible due to resource constraints. For example, soil ingestion rate data may be limited to a few studies with small sample sizes, but a new soil ingestion study may be prohibitively expensive, time consuming, or difficult to conduct in a manner that will reduce the uncertainty in the risk estimate. Uncertainty due to data quantity is not necessarily a reason to exit the tiered process at Tier 1.

Page 2-13RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 In cases where there is uncertainty in selecting a probability distribution because of small sample sizes, it may be informative to develop a preliminary probability distribution such as a triangular or uniform (see Appendix B, Section B.2.0). These preliminary distributions will contribute to the variability in the risk estimate, and can therefore be included in the probabilistic sensitivity analysis. Results of Monte Carlo simulations that include one or more preliminary distributions may lead to several alternative decisions. If the sensitivity analysis suggests that the risk estimate is relatively insensitive to the variable described with the distribution, then the uncertainty associated with the choice of a distribution should not affect the risk management decision process using the tiered approach (e.g., choice of RME percentile, derivation of a PRG). In other words, the choice would be to continue with the tiered process. If, however, the variables described by preliminary distribution are important sources of variability or uncertainty in the risk estimate, then this information should be presented in the scientific management decision point (see Figure 2-2). The uncertainty may be sufficiently important in the risk management decision to warrant additional data collection efforts. Conversely, it may be necessary to exit the tiered process if the uncertainty cannot be reduced. Although the tiered process may be stopped at this point, it can still be informative to present the results from the PRA. For example, information about uncertainty may affect the choice of the percentile used to characterize the RME risk. In addition, it may be appropriate to weight the results of the point estimate analysis more heavily in the risk management decision when uncertainty in the PRA is high. Further guidance on appropriate choices for distributions based on the information available to characterize variability is given in Appendix B. PRA also may be warranted if it would be beneficial to know where on the risk distribution the point estimate lies. An example of this would be a risk estimate that is within the NCP risk range of 1E-04 to 1E-06. The assessment may be sufficient to support risk management decisions if it could be shown that the point estimate of risk lies sufficiently high in the risk distribution. For example, a “no further action” decision may be strengthened if the point estimate is at the 99 th percentile of the risk distribution, if risks in lower percentiles of the RME risk range are below the NCP risk range, and if there is high confidence in the risk result. This type of evaluation can be conducted using PRA techniques. Even if the RME point estimate of risk exceeds the risk level of concern, and PRA is not needed to confirm this result, information from a PRA can be helpful in determining a strategy for achieving a protective preliminary remediation goal (PRG). A detailed discussion of the use of PRA in setting remediation action levels is given in Chapter 5. The advantages and disadvantages of the point estimate approach and PRA are presented in Chapter 1 (Exhibits 1-5 and 1-6).
2.3.3 T IER 2 Tier 2 of the tiered approach to risk assessment will generally consist of a simple probabilistic approach such as one-dimensional Monte Carlo analysis (1-D MCA). A 1-D MCA is a statistical technique that may combine point estimates and probability distributions to yield a probability distribution that characterizes variability or uncertainty in risks within a population (see Chapter 1). Guidance for selecting and fitting distributions is presented in Appendix B. Typical elements of a Tier 2 risk assessment, as they relate to higher and lower tiers are presented in Exhibit 2-5. A more detailed discussion of these elements can be found in Chapters 3 and 4, and Appendix A (Sensitivity Analysis).

Page 2-14RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 While most of the Tier 2 assessments are expected to use 1-D MCA to characterize variability in risk, sometimes a 1-D MCA of uncertainty may be of interest. For example, as suggested in Exhibit 2-5, a probability distribution for uncertainty in the arithmetic mean or median (i.e., 50 th percentile) for selected input variables may be specified in a 1-D MCA to yield a probability distribution for uncertainty for the central tendency risk estimate. However, as most Tier 2 assessments are expected to combine input distributions for variability, this guidance focuses on 1-D MCA for characterizing variability in the risk estimate.
Decision Alternatives Generally, the three main questions to consider when determining whether the results of a 1-D MCA are sufficient for risk management decisions are: (1) What is the RME risk range and how does it compare to the level of concern?; (2) Where does the point estimate risk lie on the risk distribution?; and (3) What is the level of confidence in the risk estimate? In Tier 2, similar to the point estimate approach, the level of confidence in a single 1-D MCA risk distribution is generally addressed in a qualitative or semi-quantitative way. As discussed in Chapter 1 (Section 1.2.4) and Chapter 3 (Section 3.4.1), one should avoid developing input distributions to a PRA model that yield a single risk distribution that intermingles, or represents both variability and uncertainty. In Tier 2, the preferred approach for characterizing uncertainty in the risk estimate is to perform multiple 1-D MCA simulations (of variability), which uses a different point estimate for uncertainty for one or more parameters, combined with probability distributions for variability for one or more variables. Chapter 3 (see Table 3-2 and Figures 3-3 and 3-4) presents an example of iterative 1-D MCA simulations using combinations of point estimates characterizing uncertainty for two variables. More advanced PRA techniques such as two-dimensional Monte Carlo analysis (2-D MCA), in which distributions for variability and uncertainty are propagated separately through an exposure model, can be undertaken in Tier 3 (Appendix D). E 2-5
T YPICAL E LEMENTS OF T ISK A

- risk

Also, risk XHIBIT

IER 2 R SSESSMENT Analysis Tool - 1-D MCA Variability Modeling - full characterization of variability in risk using PDF s or PM Fs for input variables Uncertainty Modeling - semi-qu antitative e stimate of uncertainty using iterative 1-D M CA simulations of varia bility, or a sin gle 1-D M CA of unc ertainty in the CT E risk Sensitivity Analysis - varying multiple variables with probability distributions gives a quantitative ranking (e.g., correlation coefficient) of the relative con tribution s of exp osure pathw ays and variab les to CT E or R M E risk Risk-Based Decision-Making Output distribu tion for v ariab ility: Does the risk level of concern fall within an acceptable range on the risk distribution (i.e., RME range)? distribu tion for u ncerta inty: What is the 90% confidence interval for the CTE risk?

In order to use a PRA to determine if risks are unacceptable and to develop preliminary remediation goals (PRGs) that are protective of the RME individual (see Chapter 5), a single point from the RME risk range should be selected (e.g., 95 th percentile). In general, this can be accomplished by selecting an estimate within the RME risk range based on the level of confidence in the output of the 1-D MCA. Uncertainty in risk estimates may be quantified or reduced by considering site-specific factors, biological data, and toxicity data. Stakeholders can provide useful information about uncertain variables and sources for site-specific data. More detailed guidance for choosing a percentile value within the RME range is provided in Chapter 7.

Page 2-15RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 The evaluation of the risk assessment in a 1-D MCA in Tier 2 will yield one of two outcomes: (1) sufficient for risk management decisions; or (2) insufficient for risk management decisions. If determined to be sufficient, the risk manager can exit the tiered approach and complete the RI/FS process. The results of a 1-D MCA may support a decision for “no further action” or for a “remedial action.” A “no further action” decision may result when the RME risk range (or a specified point in the RME risk range) is clearly below the level of concern (e.g., Hazard Index=1) and confidence in the risk distribution is high. A decision for remedial action may result when a national standard (e.g., MCLs applied to groundwater) may be exceeded, or when the RME risk range (or a specified point in the RME risk range) is clearly above the level of concern and confidence in the risk distribution is high. The decision for a specific remedial action involves consideration of the NCP’s nine evaluation criteria for remedial decisions (U.S. EPA, 1990; see Chapter 1) and other site-specific factors. An alternative conclusion at the end of a Tier 2 analysis would be that the results of the 1-D MCA are not sufficient for risk management decisions. There are several factors that might support this conclusion: (1) The RME risk range is close to the NCP risk range and confidence in the risk distribution is low. In this case, the risk manager might decide to not exit the tiered approach, and instead continue taking appropriate steps to increase the confidence in the risk estimate. (2) Uncertainty is high and it is believed that more than one variable is a major contributor to the uncertainty in the risk estimate. It can be difficult to explore uncertainty in more than one variable using 1-D MCA simulations of variability, even using iterative approaches discussed in Chapter 3 (Section 3.4.1). (3) Results of the point estimate risk assessment differ significantly from the results of the 1-D MCA. While the RME risk estimates are not expected to be identical, typically the RME point estimate will correspond with a percentile value within the RME range (i.e, 90 th to 99.9 th percentile) of the risk distribution. If the RME point estimates fall outside this range, further steps may be warranted to evaluate the choices for input variables—both the RME point estimates, and the probability distributions and parameters (including truncation limits) for the 1-D MCA. The deliberation/decision cycle (Figure 2-2) between Tier 2 and Tier 3 is similar to the cycle between Tier 1 and 2 and includes discussing the Tier 2 probabilistic sensitivity analysis, identifying data gaps, communicating with stakeholders (e.g., to obtain site-specific information), discussing the potential value of further analysis with probabilistic methods, work planning, and additional data collection. As with the Tier 1 assessment, additional data collection should follow the DQO process (U.S. EPA, 1992a, 1993, 1994, 2000a) and point estimates of risk should be revisited with the new data. The deliberation/decision cycle is an iterative process in which the level and complexity of the analysis increases until the scope of the analysis satisfies decision-making needs. This iterative process should continue until sufficient data are available to support risk management decisions. As in all tiers, stakeholder involvement should be encouraged. Once a 1-D MCA for variability or uncertainty is completed and is available for review and interpretation, a stakeholder meeting should be convened. Interested stakeholders should be given the opportunity to review the 1-D MCA and provide comments. Communication issues specific to PRA are discussed in Chapter 6 (Risk Communication).

Page 2-16RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 In addition to identifying data gaps, consideration for a refined 1-D MCA or more advanced PRA techniques may begin as a means of determining what benefits they may confer to the decision-making process. If, during further discussions of PRA, it is determined that information from a more advanced PRA may influence the risk management decision, the use of an advanced PRA may be warranted. If additional data have been collected, the point estimate and 1-D MCA should be refined. Specifically, an advanced PRA may be warranted if it would be beneficial to characterize uncertainty in more than one variable at a time. A 2-D MCA can simultaneously characterize variability and uncertainty in multiple variables and parameter estimates. The decision to employ such advanced methods should be balanced with considerations of resource constraints and the feasibility of reducing uncertainty in a given variable. A detailed discussion of advanced PRA methods, including 2-D MCA, is provided in Appendix D.
2.3.4 T IER 3 Tier 3 of the tiered approach to risk assessment consists of advanced PRA methods, such as 2-D MCA, Microexposure Event Analysis (MEE), geostatistical analysis of concentration data, and Bayesian statistics. Typical elements of a Tier 3 risk assessment are presented in Exhibit 2-6. A more detailed discussion of these elements is given in Appendix D. As in other tiers, Tier 3 includes an iterative process of deliberation and decision making in which the level and complexity of the analysis increases until the scope of the analysis satisfies decision-making needs. As in all tiers, stakeholder involvement is encouraged. Generally, the various elements of the deliberation/decision cycle for Tier 3 are the same as those for Tier 1 and 2 (Figure 2-2). An advanced PRA would be conducted and made available for review to the risk manager and stakeholders. The risk manager must determine if the results of the advanced PRA are sufficient for risk management decision making. Issues to consider when making this determination are similar to those identified for evaluating point estimate risk results and 1-D MCA results, and focus on evaluating the sources and magnitude of uncertainty in relation to the established risk level of concern. If the results are sufficient for risk management decisions, the risk manager may exit the tiered approach and complete the RI/FS process. If the results are not found to be sufficient for risk management decisions, data gaps should be identified and if additional data are collected, all stages of the risk assessment, including the advanced PRA, the 1-D MCA, and the point estimate risk assessment, should be refined. Alternatively, additional advanced PRA methods may be explored. Refer to Appendix D for a discussion of more advanced PRA techniques. Overall, analysis should continue within Tier 3 until sufficiently informed risk management decisions can be made. E 2-6
T YPICAL E LEMENTS OF T ISK A

- risk
limits— ( XHIBIT

IER 3 R SSESSMENT Analysis Tool - 2-D MC A, ME E, geostatistics, and Bayesian statistics Variability Modeling - full characterization using PDFs or PM Fs for input variables Uncertainty Modeling - quantitative, segregating unce rtainty from variab ility, and asso ciated with multiple variab les simulta neo usly Sensitivity Analysis - varying parameters of probability distributions to identify and rank order para mete r unce rtainty with the same sensitivity analysis m ethod s used for T ier 2 (se e Ap pen dix A ). Also , explo re altern ative ch oices of pro bab ility distribu tions an d sou rces o f mod el unce rtainty. Risk-based Decision-M aking Criteria distribution for variability with confidence Does the risk level of concern fall within an acceptable range on the risk distribution i.e., RME range), and with an acceptable level of uncertainty?

Page 2-17RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 2.3.5 F LEXIBILITY IN D EFINING T IERS The assignment of specific analytical tools to Tiers 1, 2, and 3 (Figure 2-1 and Exhibits 2-4 through 2-6) results in generalizations that may not be applicable to all site assessments. Upon completion of the deliberation phase between Tier 1 and Tier 2, the conclusion may be that analytical tools in Tier 3 would be applicable and beneficial for addressing decision making issues. For example, geospatial modeling may be beneficial for improving estimates of uncertainty in the exposure point concentration or in designing field sampling plans to further reduce uncertainty. An improved estimate of the 95% UCL from geospatial analysis (shown in Exhibit 2-6 as a Tier 3 analytical tool) would then be integrated into a Tier 2 assessment, or the complete distribution for uncertainty in the mean concentration could be incorporated into a 2-D MCA in Tier 3. Flexibility in defining the level of complexity of the analysis used in a given tier is essential to accommodating the wide range of risk assessment issues likely to be encountered. An important benefit gained from use of the tiered approach is to ensure a deliberative process in the advancement of the assessment to higher levels of complexity. It is far more important that a deliberative process take place and be documented, than it is to constrain a set of analytical tools to a specific tier.

Page 2-18RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 2 ~ December 31, 2001 REFERENCES FOR C HAPTER 2
U.S. EPA. 1989. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM)(Part A, Baseline Risk Assessment) . Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR 300: 55 Federal Register, 8666-8865, March 8.
U.S. EPA. 1991a. Risk Assessment Guidance for Superfund (RAGS) , Volume 1: Human Health Evaluation Manual (HHEM), Part B, Development of Risk-Based Preliminary Remediation Goals. Office of Emergency and Remedial Response, Washington, DC. EPA/540/R-92/003. NTIS PB92-963333
U.S. EPA. 1991b. Role of the Baseline Risk Assessment in Superfund Remedy Selection Decisions

. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9355.0-30.
U.S. EPA. 1992a. Guidance on Data Usability in Risk Assessment. Part A. Final. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.09A. NTIS PB92-96336.
U.S. EPA. 1992b. Supplemental Guidance to RAGS: Calculating the Concentration Term . Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive 9285.7-081.
U.S. EPA. 1993. Data Quality Objectives Process for Superfund: Interim Final Guidance . Office of Research and Development, Washington, DC. EPA/540/R-93/071.
U.S. EPA. 1994. Guidance for the Data Quality Objectives Process (EPA QA/G-4). Office of Research and Development, Washington, DC. EPA/600/R-96/055. September.
U.S. EPA. 2000a. Data Quality Objectives Process for Hazardous Waste Site Investigations . Office of Environmental Information, Washington, DC. EPA/600/R-00/007. January.
U.S. EPA. 2000b. Peer Review Handbook: 2 nd Edition. Science Policy Council. Washington, DC. EPA/100/B-00/001. December.
U.S. EPA. 2001. Risk Assessment Guidance for Superfund : Volume I. Human Health Evaluation Manual, Part D: Standardized Planning, Reporting, and Review of Superfund Risk Assessments. Office of Emergency and Remedial Response. Washington, DC. OSWER Directive No. 9285-47. December.

Page 2-19RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 

## C HAPTER 

## 3
U SING 

## P ROBABILISTIC 

## A NALYSIS IN 

## H UMAN 

## H EALTH 

## A SSESSMENT


### 3.0 I NTRODUCTION This chapter outlines how probabilistic analysis may be applied to human health risk assessments in the Environmental Protection Agency’s (EPA) Superfund program. The paradigm for human health risk assessment as described in EPA’s Risk Assessment Guidance for Superfund (U.S. EPA, 1989), includes data collection/evaluation in addition to exposure and toxicity assessment and risk characterization. Although the strategies and methods used in collecting and analyzing data can significantly impact the uncertainty in a risk estimate, they are issues relevant to risk assessment in general, and are addressed in other guidance documents, such as EPA’s Guidance for Data Useability in Risk Assessment (U.S. EPA, 1992b). RAGS Volume 3: Part A focuses on a tiered approach for incorporating quantitative information on variability and uncertainty into risk management decisions. 

### 3.1 C HARACTERIZING 

### V ARIABILITY 

### I N 

### E XPOSURE 

### V ARIABLES Exhibit 3-1 gives the general equation used for calculating exposure, often expressed as an average daily intake. In a point estimate approach, single values (typically a mixture of average and high-end values) are input into the equation. In probabilistic risk assessment (PRA), the only difference is that a probability distribution, rather than single value, is specified for one or more variables. A Monte Carlo simulation is executed by repeatedly selecting random values from each of these distributions and calculating the corresponding exposure and risk. For the majority of PRAs, it is expected that probability distributions will be used to characterize inter-individual variability, which refers to true heterogeneity or diversity in a population. Thus, variability in daily intake, for example, can be characterized by combining multiple sources of variability in exposure, such as ingestion rate, exposure frequency, exposure duration, and body weight. Variability in chemical concentrations (Chapter 5 and Appendix C) and the toxicity term in ecological risk assessment (Chapter 4) may also be considered in risk calculations. E 3-1 G ENERAL E E

where, I = C = CR =
EF = ED = BW = body weight AT = XHIBIT QUATION FOR XPOSURE

Eq. 3 -1

daily intake contaminant concentration contact rate (ingestion, inhalation, derm al con tact) exposure frequency exposure duration averaging time

Page 3-1RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment

E 3-2
D C 3 95% UCL for mean n ) was time. th percentile (see below), which is a

th 95 th - The number in a distribution that is greater than 95% of the other values of the distribution, and less interpolated from among two values. Arithmetic Mean
It can be

Credible Interval Credible intervals may th

CTE Risk - The estimated risk corresponding to the central tendency exposure. of occurrence for a random independent variable. Each value c x c .
Frequency Distribution/Histogram from a population. across that range. High-end Risk th percentile. Low-end Risk - A risk descriptor representing the low-end, or lower tail of the risk distribution, such as the 5 th or 25 th percentile. .Parameter For example, a truncated normal probability distribution may

that point.

1989). still within the range of possible exposures. Chapter 3 ~ December 31, 2001

XHIBIT

EFINITIONS FOR HAPTER - The one-sided 95% upper confidence limit for a population mean; if a sample of size ( repeatedly drawn from the population, the 95% UCL will equal or exceed the true population mean 95% of the It is a measure of uncertainty in the mean, not to be confused with the 95 measure of variability. As sample size increases, the difference between the UCL for the mean and the true mean decreases, while the 95 percentile of the distribution remains relatively unchanged. percentile

than 5% of the values. When estimated from a sample, this quantity may be equal to an observed value, or (AM) - A number equal to the average value of a population or sample. Usually obtained by summing all the values in the sample and dividing by the number of values (i.e., sample size). Assessment Endpoint - The specific expression of the population or ecosystem that is to be protected. characterized both qualitatively and quantitatively in the risk assessment. Central Tendency Exposure (CTE) - A risk descriptor representing the average or typical individual in the population, usually considered to be the arithmetic mean or median of the risk distribution. - A range of values that represent plausible bounds on a population parameter. describe a parameter of an input variable (e.g., mean ingestion rate) or output variable (e.g., 95 percentile risk). The term is introduced as an alternative to the term confidence interval when the methods used to quantify uncertainty are not based entirely on statistical principles such as sampling distributions or Bayesian approaches. For example, multiple estimates of an arithmetic mean may be available from different studies reported in the literature—using professional judgment, these estimates may support a decision to describe a range of possible values for the arithmetic mean. Cumulative Distribution Function (CDF) - Obtained by integrating the PDF or PMF, gives the cumulative probability of the function is the probability that a random observation will be less than or equal to Exposure Point Concentration (EPC) - The average chemical concentration to which receptors are exposed within an exposure unit. Estimates of the EPC represent the concentration term used in exposure assessment. - A graphic (plot) summarizing the frequency of the values observed or measured It conveys the range of values and the count (or proportion of the sample) that was observed - A risk descriptor representing the high-end, or upper tail of the risk distribution, usually considered to be equal to or greater than the 90

- A value that characterizes the distribution of a random variable. Parameters commonly characterize the location, scale, shape, or bounds of the distribution. be defined by four parameters: arithmetic mean [location], standard deviation [scale], and min and max [bounds]. It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithmetic mean ingestion rate). Probability Density Function (PDF) - A function representing the probability distribution of a continuous random variable. The density at a point refers to the probability that the variable will have a value in a narrow range about Probability Mass Function (PMF) - A function representing the probability distribution for a discrete random variable. The mass at a point refers to the probability that the variable will have a value at that point. Reasonable Maximum Exposure (RME) - The highest exposure that is reasonably expected to occur at a site (U.S. EPA, The intent of the RME is to estimate a conservative exposure case (i.e., well above the average case) that is

Page 3-2RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

E 3-2
D C 3 — Continued

their relative contributions to model output variability and uncertainty. Common metrics of sensitivity include:

< - A statistic r that measures the strength and direction of linear ( r

2 ) is the

< - Ratio of the change in model output per unit change in an input variable; also called elasticity

. < - A “distribution free” or nonparametric statistic r that measures the strength and direction of association between the ranks of the values (not the values r 2

. concern”. XHIBIT

EFINITIONS FOR HAPTER
Sensitivity Analysis - Sensitivity generally refers to the variation in output of a model with respect to changes in the values of the model’s input(s). Sensitivity analysis can provide a quantitative ranking of the model inputs based on
Pearson Correlation Coefficient association between the values of two quantitative variables. The square of the coefficient fraction of the variance of one variable that is explained by the variance of the second variable. Sensitivity Ratio Spearman Rank Order Correlation Coefficient themselves) of two quantitative variables. See Pearson (above) for Target Population - The set of all receptors that are potentially at risk. Sometimes referred to as the “population of A sample population is selected for statistical sampling in order to make inferences regarding the target population (see Appendix B, Section B.3.1, Concepts of Populations and Sampling).

Page 3-3RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 Figure 3-1 shows a hypothetical example of an input distribution for drinking water ingestion rate. Assume that survey data for drinking water ingestion rates were compiled in order to select and fit a probability distribution. One of the first steps in exploring the data set may be to plot a frequency distribution. In the graph, the height of the bars (the y-axis) represents the relative frequency of ingestion rates in the population and the spread of the bars (the x-axis) is the varying amounts of water ingested (L/day). Since ingestion rate is a continuous random variable, the probability distribution can also be represented graphically with a probability density function (PDF). Assume that the following parameters are estimated from the sample: arithmetic mean=1.36, standard deviation=0.36, geometric mean=1.31, and geometric standard deviation=1.30. These parameter estimates may be used to define a variety of probability distributions, including a 2-parameter lognormal distribution. The fit of the lognormal distribution can be evaluated by visual inspection using the PDF given by Figure 3-1, or by a lognormal probability plot (see Appendix B). The y-axis for a PDF is referred to as the probability density , where the density at a point on the x-axis represents the probability that a variable will have a value within a narrow range about the point. This type of graph shows, for example, that there is a greater area under the curve (greater probability density) in the 1-2 L/day range than 0-1 L/day or 2-3 L/day. That is, most people reported consuming 1-2 L/day of drinking water. By selecting a lognormal distribution to characterize inter-individual variability, we can state more precisely that 1 L/day corresponds to the 15 th percentile and 2 L/day corresponds to the 95 th percentile, so approximately 80% (i.e., 0.95–0.15=0.80) of the population is likely to consume between 1 and 2 L/day of drinking water.

Figure 3-1. Example of a frequency distribution for adult drinking water ingestion rates, overlaid by a grap h of the p rob ability de nsity functio n (PD F) for a logno rmal d istribution define d by the samp le statistics. The distribution represents inter-individual variability in water intakes and is characterized by two parameters. Typically, the geometric mean (GM ) and geometric standard deviation (GSD), or the arithmetic mean (AM) and arithmetic standard deviation (SD) are presented to characterize a lognormal distribution.
Page 3-4RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 3.1.1 D EVELOPING D ISTRIBUTIONS F OR E XPOSURE V ARIABLES When site-specific data or representative surrogate data are available, a probability distribution can be fit to that data to characterize variability. Appendix B describes how to fit distributions to data, how to assess the quality of the fit and discusses topics such as the sensitivity of the tails of the distribution to various PDFs, and correlations among variables. Many of the issues discussed below regarding the use of site-specific data or surrogate data are relevant to both point estimate risk assessment and PRA. For the majority of the exposure variables, such as exposure duration, water intake rates, and body weight, site-specific data will not be available. The risk assessor will have to either select a distribution from existing sources, or develop a distribution from published data sets and data summaries. Examples of sources for these distributions and data sets are EPA’s Exposure Factors Handbook (U.S. EPA, 1997a,b,c), Oregon Department of Environmental Quality’s Guidance for Use of Probabilistic Analysis in Human Health Risk Assessment (Oregon DEQ, 1998), and the scientific literature. An appropriate PDF should be determined in collaboration with the regional risk assessor. The process by which PDFs are to be selected and evaluated should be described in the workplan. EPA’s Superfund program is in the process of developing a ranking methodology to evaluate data representativeness relevant to various exposures scenarios. Following peer review and project completion, the results will be posted on EPA Superfund web page.

L At this time, EPA does not recommend generic or default probability distributions for exposure variables. Regardless of whether a PDF is derived from site-specific measurements or obtained from the open literature, the risk assessor should carefully evaluate the applicability of the distribution to the target population at the site. The distribution selected should be derived from the target population or from a surrogate population that is representative of the target population at the site. For example, a distribution based on homegrown vegetable consumption in an urban population would not be representative for a farming population in the Midwest. If such a distribution were to be used, (and no other data were available), the uncertainty and bias that this PDF would impart to the risk estimate should be communicated to the risk decision makers. For purposes of risk management decision making, the significance of not having site-specific data should be evaluated in the context of representativeness and sensitivity analysis. If published data are representative of the potentially exposed population, then site-specific data may be unnecessary. For example, body weights of children and adults have been well studied from national surveys and can generally be considered reasonable surrogates for use in site risk assessments. Furthermore, even if a variable is likely to vary among different exposed populations, it may not contribute greatly to the variance or uncertainty in risk estimates. In this case, surrogate data may also be used with confidence in the risk estimate. In addition, the PRA may be simplified by using point estimates instead of probability distributions for the “less sensitive” exposure variables. In part, the decision to use a point estimate in lieu of a probability distribution must balance the benefit of simplifying the analysis and the communication process (see Chapter 6), against the reduction (however small) in the variance of the risk distribution. The utility of sensitivity analysis in identifying the important factors in a risk estimate is discussed further below and in Appendix A.

Page 3-5RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 It is also important to evaluate the sample design and sample size when deciding to apply a distribution to a specific site. Depending on the situation, a very large data set derived from a national population may be more useful than a site-specific data set derived from a small, incomplete, or poorly designed study. Appendix B provides additional discussion on how to evaluate the data and studies that form the basis for a distribution. Often, the question arises regarding the appropriateness of combining data sets to derive a PDF. Before combining data sets, a careful evaluation should be made of the representativeness of the study populations, and the similarity in study designs and quality. In addition, statistical tests may be used to determine whether or not data sets are compatible with a common probability distribution (Hedges and Olkin, 1985; Stiteler et al., 1993). In general, risk assessors should be reluctant to combine data sets for the purpose of developing a PDF that characterizes variability. Due to the number of potential differences inherent in the study design, alternative data sets may provide a better measure of uncertainty in the probability distribution and parameter estimates, rather than a means of increasing the overall sample size for defining a single probability distribution. For example, if multiple data sets are available, a more informative approach may be to incorporate each data set into the PRA in a separate analysis, as a form of sensitivity analysis on the choice of alternative data sets. Each probability distribution used in a Monte Carlo Analysis (MCA) should be presented with sufficient detail that the analysis can be reproduced (see Chapter 1, Section 1.4, Condition #2). This information may be presented in tabular and/or graphical summaries. Important information for a summary table would include a description of the distribution type (e.g., lognormal, gamma, etc.), the parameters that define the distribution (e.g., mean and standard deviation, and possibly upper and lower truncation limits for a normal distribution), units, and appropriate references (see Table 3-6, for example). The table should also indicate whether the distribution describes variability or uncertainty. The report should discuss the representativeness of the data and why a particular data set was selected if alternatives were available. Graphical summaries of the distributions may include both PDFs and cumulative distribution functions (CDFs), and should generally be used to document distributions that characterize site-specific data.
3.1.2 C HARACTERIZING RISK U SING PRA Quantitative risk characterization involves evaluating exposure (or intake) estimates against a benchmark of toxicity, such as a cancer slope factor or a noncancer hazard quotient. The general equation used for quantifying cancer risk from ingestion of contaminated soil is shown in Exhibit 3-3, and the equation for noncarcinogenic hazard is shown in Exhibit 3-4. A Hazard Index is equal to the sum of chemical-specific Hazard Quotients. At this time, this guidance does not propose probabilistic approaches for dose-response in human health assessment and, further, discourages undertaking such activities on a site-by-site basis . Such activities require contaminant-specific national consensus development and national policy development (see Chapter 1, Section 1.4.1). Chapter 4 discusses methods for applying probabilistic approaches to ecological dose-response assessment. The probabilistic calculation of risk involves random sampling from each of the exposure variable distributions. The output of this process is a distribution of risk estimates. When the calculation of risk (or any other model endpoint) is repeated many times using Monte Carlo techniques to sample the variables at random, the resulting distribution of risk estimates can be displayed in a similar fashion. The type of summary graph used to convey the results of a MCA depends on the risk management needs. For example, Chapter 1, Figure 1-3 shows how a PDF for risk might be used to compare the probabilistic
Page 3-6RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 estimate of the RME risk (e.g., 95 th percentile) with a risk level of concern. This type of summary can also be used to effectively illustrate the relationship between the RME risk determined from point estimate and probabilistic approaches.

E 3-3
E C ANCER R ISK

where, C = ED = IR = BW = body weight (kg) CF = conversion factor (1E-06 kg/mg) AT = EF = exposure frequency (days/year) CSF = -1

E 3-4
E N ONCANCER H AZARD Q

= RfC = g/m 3 ) XHIBIT
QUATION FOR

Example for Soil Ingestion

concentration in soil (mg/kg) exposure duration (yea rs) soil ingestion rate (mg/d ay) averaging time (d ays) oral cance r slope facto r (mg/k g-day)

XHIBIT
QUATION FOR UOTIENT

where , RfD reference d ose, o ral or d erma lly adjusted (m g/kg-day) reference concentration, inhalation (

In addition, the CDF can be especially informative for illustrating the percentile corresponding to a particular risk level of concern (e.g., cancer risk of 1E-04 or Hazard Index of 1 ). Figure 3-2 illustrates both the PDF and CDF for risk for a hypothetical scenario. Factors to consider when applying the PDF or CDF are discussed in Chapter 1, Exhibit 1-3. When in doubt about the appropriate type of summary to use, both the PDF and CDF should be provided for all risk distributions. At a minimum, each summary output for risk should highlight the risk descriptors of concern (e.g., 50 th , 90 th , 95 th , and 99.9 th percentiles). It can also be informative to include the results of the point estimate analysis—the risks corresponding to the central tendency exposure (CTE) and the reasonable maximum exposure (RME).

Page 3-7RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Figure 3-2. Hypothetical PRA results showing a PDF (top panel) and CDF (bottom panel) for cancer risk with selected summary statistics. The CDF rises to a maximum cumulative probability of 1.0. The CDF clearly shows that the level of regulatory concern chosen for this example (1E-06) falls between the 90 th and 95 th percentiles of the risk distribution.

Page 3-8RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 

### 3.2 R OLE OF THE 

### S ENSITIVITY 

### A NALY SIS Prior to conducting a PRA, it is worthwhile to review several points pertaining to the sensitivity analysis. As shown in Chapter 2 (Figures 2-1 and 2-2), sensitivity analysis can play an important role in decision making at each tier of the tiered process. Beginning with Tier 1, a point estimate for risk should be calculated prior to conducting a PRA. Based on the results of the point estimate, the risk assessor and risk decision makers should determine whether a probabilistic analysis will offer additional benefit. One factor in this decision may be the results of a sensitivity analysis. A primary objective of the sensitivity analysis is to determine which variables and pathways most strongly influence the risk estimate. At many Superfund sites, an estimate of cumulative risk considers contamination in multiple media, moving through multiple pathways and interacting with a number of receptors. Depending on the complexity of the site, and the modeling approaches, a risk assessment may involve one exposure pathway and few variables, or multiple pathways with many variables (e.g., multimedia fate and transport models). However, resources and time are often limited. The sensitivity analysis is invaluable in focusing these limited resources on the most influential variables and pathways. Several methods for conducting sensitivity analysis are described in Appendix A. It is important to note that when a sensitivity analysis is performed and the major variables are identified, this does not mean that the less influential pathways and variables should be eliminated from the risk assessment. It means that because they are not major contributors to the variability or uncertainty in risk, they can be described with point estimates without affecting the risk management decision. If distributions are readily available for these less influential variables, one may use distributions. The key goal is to provide a comprehensive risk characterization that is scientifically credible and sufficient for risk decision making. The time and effort required to achieve various levels of complexity should be weighed against the value of the information provided to the risk managers. Additionally, if a variable is specified as influential in the sensitivity analysis, this does not automatically mean that a distribution has to be developed for this variable. If the risk assessor feels that data are simply not sufficient from which to develop a distribution, then a plausible point estimate can be used. The risk assessor should be aware of a possible problem arising from using point estimates in the absence of data adequate to support a distribution. If a variable has the potential to significantly impact the risk outcome, and a very high-end or low-end point estimate is used in the PRA, this has the potential to right-shift or left-shift the final distribution of risk. Even though there might not be enough data to develop a distribution of variability for an influential variable, it would be prudent to communicate the importance of this data gap to the risk decision makers, and perhaps run multiple simulations with several plausible input distributions for that variable. Communication of this uncertainty may persuade the risk decision makers to collect additional data to better define the variable.

Page 3-9RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 

### 3.3 E XPOSURE 

### P OINT 

### C ONCENTRATION 

### T ERM A brief discussion of the concentration term is provided below. A more complete discussion of the concentration term in PRA is provided in Appendix C. The reader is also referred to Chapter 5 on development of PRGs. The major source of uncertainty in Superfund risk assessments is often incomplete knowledge of the concentration of one or more chemicals in various exposure media. In any risk assessment, the derivation of the concentration term will reflect assumptions about: (1) properties of the contaminant, (2) the spatial and temporal variability in contamination, (3) the behavior of the receptor, and (4) the time scale of the toxicity of the chemical(s). Contaminant concentrations contacted by a receptor are likely to vary depending on the spatial variability of contamination and the movements of the receptor. Different individuals may be exposed to different concentrations based on inter-individual variability in activity patterns. If information regarding activity patterns is unavailable, receptors are typically assumed to exhibit random movement such that there is an equal probability of contacting any area within the exposure unit (EU). An EU is defined as the geographical area in which a receptor moves and contacts contaminated medium during the period of the exposure duration. In addition, in Superfund risk assessments, the toxicity criteria are often based on health effects associated with chronic exposure (e.g., lifetime risk of cancer following chronic daily intake over a period of 30 years). Hence, the most appropriate expression for the concentration term, for the majority of risk assessments, is one that characterizes the long-term average exposure point concentration within the EU.

L The most appropriate expression of the exposure point concentration term for chronic exposure will characterize the long-term average concentration experienced by a receptor within the exposure unit. In point estimate risk assessments, the exposure point concentration term is usually calculated as the 95% upper confidence limit (95% UCL) of the arithmetic mean because of the uncertainty associated with estimating the true (i.e., population) mean concentration at a site. If the sampling density is sparse relative to the size of the EU, the uncertainty may be high due to the relatively small number of measurements available to estimate the mean concentration within the EU. The decision to use the upper confidence limit to define the concentration term introduces a measure of protectiveness by reducing the chance of underestimating the mean. Although there will be situations in which modeling variability in concentration will be the appropriate choice (e.g., non-random movement within an EU, acute exposure events, migration of groundwater contaminant plume, migration of fish, etc.), in most cases, characterization of the concentration term will focus on uncertainty. Appendix C provides a more complete discussion on characterizing both variability and uncertainty in the concentration term. Table 3-1 summarizes a number of appropriate methods for characterizing uncertainty in the parameter of an exposure variable, such as the arithmetic mean of the concentration term.

Page 3-10RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 

### 3.4 C HARACTERIZING 

### U NCERTAINTY IN 

### E XPOSURE 

### V ARIABLES Uncertainty is described as a lack of knowledge about factors affecting exposure or risk. To evaluate regulatory options, risk assessors are expected to translate the available evidence, however tentative, into a probability of occurrence of an adverse health effect. Data from a sample or surrogate population are used to develop estimates of exposure and risk in a specific target population (see Section 3.1.4 and Appendix B, Section B.3.1). This extrapolation requires assumptions and inferences that have inherent strengths and limitations, and may bias the outcome of the risk estimate. For example, a common assumption in risk assessments for carcinogens is that a contaminant concentration within the boundaries of a hazardous waste site represents the concentration that a receptor is exposed to throughout the period of exposure, with the corresponding dose averaged over the course of a lifetime. This assumption may be conservative (i.e., result in overestimation of exposure) if it is unlikely that receptors will be exposed at the hazardous waste site for the entire exposure duration. It is incumbent on the risk assessor to clearly present the rationale for the assumptions used in a risk assessment, as well as their implications and limitations.
U.S. EPA guidance, including the Exposure Assessment Guidelines (U.S. EPA, 1992a), Exposure Factors Handbook (U.S. EPA, 1997a,b,c), and Guiding Principles for Monte Carlo Analysis (U.S. EPA, 1997d) have classified uncertainty in exposure assessment into three broad categories: (1) Parameter uncertainty - uncertainty in values used to estimate variables of a model; (2) Model uncertainty- uncertainty about a model structure (e.g., exposure equation) or intended use; and (3) Scenario uncertainty - uncertainty regarding missing or incomplete information to fully define exposure. Each source of uncertainty is described in detail below, along with strategies for addressing them in PRA.
3.4.1 P ARAMETER U NCERTAINTY Parameter uncertainty may be the most readily recognized source of uncertainty that is quantified in site-specific risk assessments at hazardous waste sites. Parameter uncertainty can occur in each step of the risk assessment process from data collection and evaluation, to the assessment of exposure and toxicity. Sources of parameter uncertainty may include systematic errors or bias in the data collection process, imprecision in the analytical measurements, and extrapolation from surrogate measures to represent the parameter of interest. For example, soil data collected only from the areas of highest contamination, rather than the entire area that a receptor is expected to come into contact, will result in a biased estimate of exposure. In general, parameter uncertainty can be quantified at any stage of the tiered process, includingpoint estimate analysis (Tier 1), one-dimensional Monte Carlo analysis (1-D MCA) (Tier 2), and twodimensional Monte Carlo analysis (2-D MCA) (Tier 3). In the point estimate approach, parameter uncertainty may be addressed in a qualitative manner for most variables. For example, the uncertainty section of a point estimate risk assessment document might state that an absorption fraction of 100% was used to represent the amount of contaminant in soil absorbed from the gastrointestinal (GI) tract, and as a result, the risk estimate may overestimate actual risk. In addition, a sensitivity analysis may be performed, wherein one input variable at a time is changed, while leaving the others constant, to examine Page 3-11RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 the effect on the outcome. In the case of absorption from the GI tract, different plausible estimates of the high-end, or RME absorption fraction might be used as inputs to the risk equation. The differences in the risk estimates would reflect uncertainty in the RME absorption fraction. Quantitative approaches for characterizing parameter uncertainty in exposure variables in a Monte Carlo simulation are summarized in Table 3-1. If uncertainty in only a few parameter values is of interest, multiple 1-D MCA simulations can yield the same results as a 2-D MCA simulation, but without the time and effort of a 2-D MCA. An example illustrating this concept is given in Table 3-2. With multiple 1-D MCA simulations, variability is characterized in one or more variables using probability distributions for variability (PDFv’s), and uncertainty in a parameter is characterized with a series of different point estimates from a probability distribution for uncertainty (PDFu) (e.g., 95% lower confidence limit LCL [95% LCL], sample mean, and 95% UCL). In a 2-D MCA simulation, variability is characterized in one or more variables using PDFv’s, and uncertainty in one or more parameters is characterized with PDFu’s. With both approaches, the influence of the parameter uncertainty can be presented as a credible interval or confidence interval (CI) around the risk distribution, depending on how the PDFu’s are defined. When only a few sources of parameter uncertainty are quantified, multiple 1-D MCA simulations are preferred over a 2-D MCA because the approach is easier to use and communicate. However, if the goal is to explore the effect that many sources of parameter uncertainty may have on the risk estimates simultaneously, a 2-D MCA is preferred. Iterative 1-D MCA simulations with different combinations of confidence limits may be impractical.

Table 3-1. M ethod s for Charac terizing P aram eter U ncerta inty with M onte C arlo S imulatio ns. Approach Example of M odel Input M ethod Example of M odel Output Single Point Estimate • 95% UCL 1-D MCA PDFv 1 for risk, calculated using the 95% UCL for one parameter. Multiple Point Estimates • 95% LCL • sample mean • 95% UCL 1-D MCA Three PD Fv’s for risk, representing the 90% CI for each percentile of the risk distribution. 2 The 90 % C I only ac counts for uncertainty in a single parameter (not multiple parameters). Par ame tric PDFu 1 PDF u for the mean based on the sampling distribution, derived from a Student’s t -distribution. 2-D MCA One PDF v for risk with confidence intervals at each percentile of the risk distribu tion. T he CI reflects uncerta inty in one or m ore parameters. No n-parame tric PDFu PDFu for the mean based on boo tstrap resamp ling methods. 2-D MCA Same as parametric probability distribution for unc ertainty.

1 Probability distribution for uncertainty (PDFu) and probability distribution for variability (PDFv). 2 The 95% UCL for the concentration term represents a 1-sided confidence interval (CI), meaning there is a 95% probability that the value is greater than or equal to the mean. Similarly, the 95% LCL would represent the 1-sided CI in which there is a 95% probability that the value is less than or equal to the mean. Both values are percentiles on the probability distribution for uncertainty (PDFu), also called the sampling distribution for the mean. Together, the 95% LCL and 95% UCL are equal to the 2-sided 90% confidence interval only for cases in which the PDFu is symmetric. For example, the sampling distribution for the arithmetic mean of a sample from a normal distribution with an unknown variance is described with the symmetric Student’s t- distribution, whereas the PDFu for the mean of a lognormal distribution is asymmetric. In order to compare the results of multiple 1-D MCA simulations and a 2-D MCA simulation, the same methodology should be employed to define the PDFu and the corresponding confidence limits.

Page 3-12RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 It is generally incorrect to combine a PDFu for one parameter (e.g., mean of the concentration term) with one or more PDFv’s in other exposure factors when conducting a 1-D MCA for variability. However, distributions for uncertainty and variability may be appropriately combined in a 2-D MCA. As discussed in Appendix D, with 2-D MCA, a clear distinction should be made between probability distributions that characterize variability (PDFv) and parameter uncertainty (PDFu). A 2-D MCA propagates the uncertainty and variability distributions separately through an exposure model, thereby making it possible to evaluate the effect of each on the risk estimates. Example: Comparison of Multiple Point Estimates of Uncertainty in 1-D MCA, and Distributions of Uncertainty in 2-D MCA Table 3-2 illustrates an application of the approaches presented in Table 3-1 for quantifying variability and parameter uncertainty. This is a hypothetical example, and no attempt was made to usestandard default assumptions for exposure variables. Two sources of variability are quantified: (1) interindividual variability in exposure frequency (EF), characterized by a triangular distribution, and (2) interindividual variability in exposure duration (ED), characterized by a truncated lognormal distribution. In addition, two sources of uncertainty are presented: (1) a point estimate for soil and dust ingestion rate, intended to characterize the RME; and (2) an upper truncation limit of the lognormal distribution for ED, intended to represent a plausible upper bound for the exposed population. Methods for quantifying these sources of uncertainty are discussed below. Additional sources of uncertainty may also have been explored. For example, the choice of a triangular distribution for a PDFv may be provocative for some risk assessors, since there are few cases in which empirical data suggest a random sample is from a triangular distribution. Nevertheless, triangular distributions may be considered rough, or “preliminary” distributions (see Chapter 2 and Appendix B, Section B.2) for cases when the available information supports a plausible range and central tendency. The choice of distributions is a potential source of uncertainty that can be explored by rerunning simulations with each alternative, plausible choice, and examining the effect on the RME risk. Simulations with preliminary simulations may yield at least three different outcomes. First, this type of sensitivity analysis can help guide efforts to improve characterizations of variability for selected variables that have the greatest affect on the risk estimates. Second, results may provide justification to exit the tiered process without continuing with additional Monte Carlo simulations since further effort would be unlikely to change the risk management decision. Finally, if the major sources of uncertainty can be clearly identified, a subset of the less sensitive variables may be defined by point estimates without significantly reducing the uncertainty in the risk estimates. Parameter uncertainty can be quantified for both point estimates and PDFv’s. In this example, both types of inputs (i.e., point estimates and PDFv’s) are presented as sources of parameter uncertainty: the RME point estimate for soil and dust ingestion rate (IRsd), and the upper truncation limit on a PDFv for ED. For IRsd, assume that three different studies provide equally plausible values for the RME: 50, 100, and 200 mg/day. A uniform PDFu is specified to characterize this range of plausible values. For ED, assume that the maximum value reported from a site-specific survey was 26 years, but surrogate data for other populations suggest the maximum may be as long as 40 years. A uniform PDFu is specified to characterize this range of plausible values as well. In Cases 1-3, the impact of uncertainty in IRsd and ED was evaluated using a series 1-D MCA simulations. Inputs for uncertain parameters associated with IRsd and ED in Case 1, 2, and 3 represent the minimum, central tendency, and maximum values, respectively. Each simulation yields a different
Page 3-13-- -- --

-- -- -- RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 risk distribution based on different combinations of point estimates for parameters. Although a PDFu was specified for IRsd, it would have been incorrect to combine the PDFu with the PDFv’s for EF and ED in a 1-D MCA because the result would have been a single distribution of risk that co-mingled uncertainty and variability. In Case 4, a single 2-D MCA simulation was run using the PDFu’s for uncertainty and the PDFv’s for variability. By propagating variability and uncertainty separately, the 2-D MCA yields a series of distributions of risk, from which credible intervals can be calculated for each percentile of the CDF.

Table 3-2. Example of 1-D MCA and 2-D MCA.

Va riab le Type of Input 1-D MCA 2-D MCA Case 1 Case 2 Case 3 Case 4 C (mg/kg) pt estimate 500 500 500 500 IRsd (mg/d ay) pt estimate 50 100 200 see below PDF u for pt estimate uniform (50, 200) a
CF (kg/mg) pt estimate 1E-06 1E-06 1E-06 1E-06
EF (days/year) PDF v triangular min = 200 mode = 250 max = 350 triangular min = 200 mode = 250 max = 350 triangular min = 200 mode = 250 max = 350 triangular min = 200 mode = 250 max = 350 ED (years) PDFv T-lognormal mean = 9 stdv = 10 max = 26 T-lognormal mean = 9 stdv = 10 max = 33 T-lognormal mean = 9 stdv = 10 max = 40 T-lognormal mean = 9 stdev = 10 max = PDFu (see below) PDFu for parameter of PDFv max ~ uniform (26, 40) b

BW (kg) pt estimate 70 70 70 70 AT (days) pt estimate 25550 25550 25550 25550 CSF (mg/kg-day) -1 pt estimate 1E-01 1E-01 1E-01 1E-01

a Uncertainty in the RME point estimate, defined by a uniform distribution with parameters (minimum, maximum). b Uncertainty in the upper truncation limit of the lognormal distribution, defined by a PDFv with parameters (mean, standard deviation, maximum) and a PDFu for the maximum defined by a uniform distribution with parameters (minimum, maximum).

Page 3-14RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001
Monte Carlo Simulation Results Figures 3-3 and 3-4 illustrate CDFs for risk produced from Monte Carlo simulations using Crystal Ball® 2000. The 1-D MCA simulations (Figure 3-3) were run with 10,000 iterations and Latin Hypercube sampling. The 2-D MCA simulation (Figure 3-4) was run with 250 iterations of the outer loop (uncertainty) and 2,000 iterations of the inner loop (variability). Details regarding 2-D MCA simulation are given in Appendix D. Figure 3-3 shows CDFs for risk based on three simulations of a 1-D MCA simulation. Each simulation used a different combination of plausible estimates of the RME value for IRsd and the upper truncation limit for ED, as discussed above. The results provide a bounding estimate on the risk distribution given these two sources of uncertainty. The 95 th percentile risk, highlighted as an example of the RME risk estimate, may range from approximately 7E-06 to 3.5E-05.

5 Figure 3-4 shows a single CDF for risk, representing the central tendency risk distribution. This CDF was derived by simulating uncertainty in the risk distribution using 2-D MCA. For this example, the 2-D MCA yields 250 simulations of the risk distributions for variability, so that there are 250 plausible estimates of each percentile of the risk distribution. In practice, more than 250 simulations may be needed to adequately quantify uncertainty in the risk distribution. Results of a 2-D MCA can be presented as probability distributions of uncertainty, or box-and-whisker plots of uncertainty at selected percentiles of the risk distributions. Figure 3-4 shows the central tendency (50 th percentile) estimate of uncertainty for the entire CDF of risk. In addition, a box-and-whisker plot is shown at the 95 th percentile of the CDF. Selected statistics for the box-and-whisker plot are included in a text box on the graphic (i.e., minimum; 5 th , 50 th , and 95 th percentiles, and maximum). The 90% credible interval is given by the

th and 95 th percentiles. For this example, the 90% credible interval for the 95 th percentile of the risk distribution is: [9.1E-06, 3.1E-05]. Figures 3-3 and 3-4 demonstrate that the two approaches (i.e., multiple 1-D MCA and 2-D MCA) can yield the same results. However, when there are numerous sources of uncertainty, 2-D MCA offers at least two advantages over multiple 1-D MCA simulations: (1) 2-D MCA allows the multiple sources of uncertainty to be included simultaneously so the approach is more efficient than a series of 1-D MCA simulations; and (2) multiple 1-D MCA simulations yield multiple estimates of the RME risk, but it is not possible to characterize the uncertainty in the RME risk in quantitative terms; a 2-D MCA yields a PDFu for RME risk, which allows for statements regarding the level of certainty that the RME risk is above or below a risk level of concern. The 95 th percentile is a focus of this example because it is a recommended starting point for determining the risk corresponding to the RME. Chapter 7 provides guidance to the risk decision makers on choosing an appropriate percentile (on a distribution of variability) within the RME risk range (90 th to 99.9 th percentiles). The chapter also includes a qualitative consideration of the uncertainty or confidence surrounding a risk estimate in the decision-making process.

Page 3-15RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 Figure 3-3

Figure 3-4

Page 3-16RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001
3.4.2 S CENARIO AND M ODEL U NCERTAINTY All models are simplified representations of complex biological and physical processes. As such, they, and the scenarios to which they are applied, may introduce a significant source of uncertainty into an exposure and risk estimate. Models may exclude important variables or important pathways of exposure, ignore interactions between inputs, use surrogate variables that are different from the target variables, or they may be designed for specific scenarios and not others. As a result, a model may not adequately represent all aspects of the phenomena it was intended to approximate or it may not be appropriate to predict outcomes for a different type of scenario. For example, a model intended to estimate risk from continuous, steady state exposures to a contaminant may not be appropriate or applicable for estimating risk from acute or subchronic exposure events. In any risk assessment, it is important to understand the original intent of a model, the assumptions being made in a model, what the parameters represent, and how they interact. Based on this knowledge, one can begin to understand how representative and applicable (or inapplicable) a model may be to a given scenario. If multiple models exist that can be applied to a given scenario, it may be useful to compare and contrast results in order to understand the potential implications of the differences. The use of multiple models, or models with varying levels of sophistication, may provide valuable information on the uncertainty introduced into a risk estimate as the result of model or scenario uncertainty. The collection of measured data as a reality check against a given parameter or the predicted model outcome (such as the collection of vegetable and fruit contaminant data to compare against modeled uptake into plants) is also useful in attempting to reduce or at least gain a better understanding of model and scenario uncertainty. 

### 3.5 E XAM PLE OF 

### PRA FOR 

### H UMAN 

### H EALTH The following hypothetical example provides a conceptual walk-through of the tiered approach for PRA in Superfund risk assessment. The example begins with a baseline human health point estimate risk assessment (Tier 1) and moves to Tier 2, in which multiple iterations of a 1-D MCA are run using default and site-specific assumptions for input distributions. The general concepts associated with the tiered approach are discussed in Chapter 2, and a similar example for ecological risk assessment is given in Chapter 4. The 1-D MCA results are based on simulations with Crystal Ball® 2000 using 10,000 iterations and Latin Hypercube sampling. These settings were sufficient to obtain stability (i.e., <1% difference) in the 95% percentile risk estimate. The example is presented in Exhibit 3-5. Tables and figures supporting the example are given immediately following the exhibit.

Page 3-17RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

E XHIBIT 3-5 U SING THE T IERED P ROCESS FOR PRA H YPOTHETICAL C ASE S TUDY FOR H UMAN H EALTH R ISK A SSESSMENT

Page 3-18RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Page 3-19RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Page 3-20RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Page 3-21RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Fig ure 3-5 . Site map for future wildlife refuge showing boundaries for the exposure unit and potential hotspot, as well as sampling locations (n=35). Sample numbers correspond with concentration data given in Table 3-3.

Table 3-3. Concentrations in Surface Soil (mg/kg).

1 The 95% UCL was estimated using the Land method (see Appendix C).

Page 3-22RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

E 3-6
R ISK E

Risk = x

Risk = x

Risk = x

Cs = Concentration of ChemX in soil (mg/kg) IRs = FI = CF = Conversion factor (1E-06 kg/mg) SA = 2 /event) AF = 2 ) ABS = IRa = 3 /hr) PEF = 3 ) ET = EF = Exposure frequency for receptor (days/year) ED = BW = AT = CSF = 

### ) -1 XHIBIT
QUATIONS

Soil Ingestion Cs x CF x IRs x FI x EF X ED Oral CSF BW x AT Dermal Absorption Cs x CF x SA x AF x ABS x EF X ED Dermal-Adjusted CSF BW x AT Inhala tion of F ugitive D ust Cs x 1/PEF x IRa x ET x EF X ED Inhalation CSF BW x AT
Tota l Risk = Sum of risks from each exposure pathway (soil + dermal + inhalation)
Wh ere: Soil ingestion rate for receptor (mg/day) Fraction ingested from contaminated source (unitless) Skin surface area available for exposure (cm Soil to skin adherence factor for ChemX (mg/cm Absorption factor for ChemX (unitless) Inhalation rate for receptor (m Soil-to-air particulate emission factor (kg/m Exposure time for receptor (hours/day) Exposure duration for receptor (years) Body weight of receptor (kg) Averaging time (years) Cancer slope factor (oral, dermal, inhalation) (mg/kg-day

Page 3-23RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Table 3-4. Exposure Parameters used in Point Estimate Analysis. Exposure Variable CTE Value RME Value Units Reference

IRs 50 100 mg/day CTE: U.S. EPA, 1997a, p. 4–25 RME: U.S. EPA, 2001 FI 0.5 1 unitless Site-specific CF 1E-06 1E-06 kg/mg Constant SA 3300 3300 cm 2 /event U.S. EPA, 2001, 50 th percentile value for all adult workers—exposure to face, forearms, and hands AF 0.1 0.2 mg/cm 2 CTE: U.S. EPA, 1998; Table 3.3, value for gardeners RME: U.S. EPA, 2001 ABS 0.1 0.1 unitless U.S. EPA, 1998, default for semi-volatile organic compounds (SVOCs) IRa 1.3 3.3 m 3 /hr U.S. EPA, 1997a, p. 5–24, outdoor worker hourly average: mean and upper percentile PEF 1.36E+09 1.36E+09 kg/m 3 U.S. EPA, 2001 ET 8 8 hours/day Site-specific EF 200 225 days/year CTE: Site-specific assumption RME: U.S. EPA, 2001 ED 5 25 years CTE: U.S. EP A, 1993, p. 6 RME: U.S. EPA, 2001 BW 70 70 kg U.S. EPA, 1993, p. 7 AT 25550 25550 days constant CTE = central tendency exposure; RME = reasonable maximum exposure.

Table 3-5. Point Estimate Risks and Exposure Pathway Contributions
. Risk Estimate by Exposure Pathway Inside Spill Area (n = 15) Outside Spill Area (n = 20) CTE RME CTE RME Soil Ingestion 6.5E-06 (43 %) 1.5E-04 (60 %) 1.7E-06 (43 %) 4.0E-05 (60 %) Dermal Absorption 8.6E-06 (57 %) 9.6E-05 (40 %) 2.3E-06 (57 %) 2.6E-05 (40 %) Inhalation 9.9E-10 (< 1 %) 1.4E-08 (< 1 %) 2.7E-10 (< 1 %) 3.8E-09 (< 1 %) Total Risk 1.5E-05 2.4E-04 4.1E-06 6.6E-05 Example of % contribution: % Soil for RME risk inside spill area = (Soil risk / Total risk) x 100% = (1.46E-04 / 2.42E-04) x 100% = 60%

Page 3-24RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001
Table 3-6. Input D istributions for Exposure Variables used in 1-D MCA for Variability.

Exposure Va riab le 1 Distribution Type Parameters 2 Units Reference IR_soil Triangular 0, 50, 100 mg/day U.S. EPA, 1993, 2001 SA_skin 3 Lognormal 18150, 37.4 cm 2 U.S. EPA, 1997a, Table 6-4 (Total male/female body surface area) Absorption Fraction Uniform 0.1, 0.2 mg/cm 2 U.S. EPA, 2001; minimum truncation limit is professional judgment IR_air Lognormal 1.68, 0.72 m 3 /hour U.S. EPA, 1996, p.5–10 EF Triangular 200, 225, 250 days U.S. EPA, 2001; truncation limits are professional judgment ED Lognormal 4 11.7, 7.0 years U.S. EPA, 1997b, Table 15-161 and U.S. EPA, 2001 (Mean value is based on average of total median tenure for professional specialty and farming, forestry, and fishing) Truncated Lognormal 5 14.0, 9.4, 44.0 years Site-specific survey data, used in refined 1-D MCA BW Lognormal 71.75, 14.2 kg U.S. EPA, 1997a, Tables 7-4 and 7-5; (Combined male/female body weight distributions)

1 All other exposure parameters are inputted as point estimates (see Table 3-4). 2 Parameters for lognormal PDF are X ~ Lognormal (arithmetic mean, arithmetic standard deviation) unless otherwise stated. Parameters for triangular PDF are X ~ Triangular (minimum, mode, maximum). Parameters for uniform PDF are X ~ Uniform (minimum, maximum). 3 A point estimate of 0.189 was used to adjust the surface area skin (SA_skin) distribution, which is based on total body surface area, to account for skin exposures limited to face, forearms, and hands (U.S. EPA, 1997a, Vol. I). 4 Parameters for preliminary lognormal PDF for ED were converted from a geometric mean of 10 and a 95 th percentile of 25. 5 Parameters for site-specific lognormal PDF for ED are arithmetic mean, standard deviation, and upper truncation limit.

Table 3-7. 1-D M CA Risk Estimates using Preliminary Inputs. Cumulative Percentile Spill Area Risk

50th 5.7E-05 90th 1.3E-04 95th 1.6E-04 99th 2.4E-04 99.9th 3.9E-04

Page 3-25RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001

Figure 3-6. Results of sensitivity analysis for preliminary 1-D MCA (Tier 2) showing the Spearman Rank correlations (see Appendix A and B) between input variables and risk estimates.

Table 3-8. Exposure Duration Survey Results. Survey Results (years) Summary Statistics 24.9 20.3 17.2 n 20

8.4 11.7 6.5 min 3.0 3.0 4.7 16.5 max 44.2 6.8 20.9 6.0 arithmetic mean 14.0 18.5 10.6 18.8 standard dev 9.4

9.1 12.7 11.7 median/GM 11.7 7.2 44.2 GSD 1.8
Table 3-9. Refined Point Estimate and 1-D M CA Risk Estimates.

Cumulative Percentile Spill Area Risk Refined RME 3.1E-04 Point Estimate 50 th 6.7E-05 90th 1.6E-04 95th 2.1E-04 99th 3.2E-04 99.9th 5.3E-04

Page 3-26RAGS Volume 3 Part A ~ Process for Conducting Probabilistic Risk Assessment Chapter 3 ~ December 31, 2001 REFERENCES FOR C HAPTER 3 Hedges, L.V. and I. Olkin. 1985. Statistical Methods for Meta-Analysis . Academic Press, Inc. Orlando. Oregon DEQ. 1998. Guidance for the Use of Probabilistic Analysis in Human Health Exposure Assessments . Waste Management and Cleanup Division. Interim Final. November. Stiteler, W.M., L.A. Knauf, R.C. Hertzberg, and R.S. Schoeny. 1993. A Statistical Test of Compatibility of Data Sets to a Common Dose-Response Model. Regulatory Tox. Pharm . 18: 392–402.
U.S. EPA. 1989. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM)(Part A, Baseline Risk Assessment). Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581.
U.S. EPA. 1992a. Final Guidelines for Exposure Assessment . EPA/600/Z-92/001. 57 Federal Register , 22888-22938. May 29.
U.S. EPA. 1992b. Guidance on Data Usability in Risk Assessment . Part A. Final. Office of Solid Waste and Emergency Response, Washington, DC. OSWER Directive No. 9285.7.09A. NTIS PB92-96336.
U.S. EPA. 1993. Data Quality Objectives Process for Superfund . Office of Solid Waste and Emergency Response. Washington, DC.
U.S. EPA. 1996. Final Soil Screening Guidance, May 17, 1996. Soil Screening User’s Guide . Office of Solid Waste and Emergency Response, Washington, DC. EPA 540/R-96/018.
U.S. EPA. 1997a. Exposure Factors Handbook, Volume 1. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fa.
U.S. EPA. 1997b. Exposure Factors Handbook, Volume 2. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fb.
U.S. EPA. 1997c. Exposure Factors Handbook, Volume 3. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fc.
U.S. EPA. 1997d. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte Carlo Analysis . Office of Research and Development, Washington, DC. EPA/630/R-97/001. May.
U.S. EPA. 1998. Guidelines for Ecological Risk Assessment . Final. National Center for Environmental Assessment, Washington, DC. EPA/630/R-95/002F.
U.S. EPA. 2001. Supplemental Guidance for Developing Soil Screening Levels for Superfund Sites

. Office of Solid Waste and Emergency Response. Washington, DC. OSWER Directive No. 9355.4-24. December.

Page 3-27RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-1 Figure 4-1 . Ecologi cal Ris k Assess ment Framework (U.S. EPA, 1992a) 

## C HAPTER 

## 4
P ROBABILISTIC 

## A NALYSIS IN 

## E COLOGICAL 

## R ISK 

## A SSESSMENT 

### 4.1 I NTRODUCTI ON 4.1.1 B ASIC A PPROACH FOR P ERF ORMI NG E COLOGICAL RIS K A SSESS MENTS Ecological risk

assessm ent (ER A) is de fined by

the 199 7 Env

ironmental Pr

otection Agen cy’s (EPA) Ecological Risk Assessm ent Guidance for Sup erfund: Process for Des igning and Conducting Ecolog ical Ris k Asses sments (ERAGS) (U.S. EPA, 1 997a) as an evalu ation of the “likelihood

that adverse ecological effects are occ urring or m

ay occur as a resu

lt of exposure to one o

r more stressors”. The ERAGS document is generally similar to , and consistent with the earlier framework guidance and approach (U.S. EP

A, 1992a) which was expanded upon and superceded by the Guidelines for Ecological Risk Assessment

(U.S. EPA, 1998). The EPA has developed exten sive technical and p olicy guidance on how ER As shou ld be pl anned a

nd perf ormed (see Ex hibit 4-2). I

n general, th

is proce ss has t

hree m ain elements, as shown in Figure 4 -1:RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-2 Problem Formulation provides a found ation for the entire risk assessment. This elem ent includes the specification of risk managem

ent goals and asses sment endpoints, the developm ent of a site conceptual m

odel wi th expo sure pa

thway s and re ceptors , and t he development of a sampling an

d analysis plan to collect data on exposures and measures of effects that are needed to support the ERA. In g eneral, problem fo rmulation serve s as the foundation of an ERA and often is an iterative pro cess, whereby s ubstantial re-ev

aluation may

occur as new inform

ation and data are collect ed durin g the site inv

estiga tions. Collection of data in subsequ ent itera tions is often triggered b

y identification of m

ajor data gaps and unc ertainties in the risk cha racterization that prevent confident decision mak ing by risk

managers. Analys is include s two principal meas ureme nt steps that are based u pon the problem form ulation : Assessment of ex

posures and assessm

ent of ecological effects. Assessment of exposures includes the identification of stres sors at the site that m ay affect ecological receptors, a characterization of the sp atial and/or temporal pattern of the stressors in the environment at the site, and an analy sis of the level of c ontact or co-o ccurrence between the stress ors and the ecological receptors. Assessment of ec ological effects includes identification of the type s of effects which different stressors m ay have on

ecological receptors, along with a cha racterization of the relationship between the level of exposu re to the stressor and the expected biological or ecological response. This is referred to as the stressor- response relationship.
Risk Characteriz ation combines the e xposure characterization and the effects characteriz ation in orde r to pro vide a qua ntita tive li keliho od or qua litat ive desc ripti on of th e natur e, freq uency, and severity o f ecological risks attributable to exposure to stres sors at a site, as well as an evaluation of

the eco logica

l relevance of

the effe cts. G ood risk

charact erizat ions ex press re sults clear ly, arti culat e major assumpt ions an d uncer taint ies, id entif y reason able al terna tive interpretations, and separate sc ientific conclusions from policy judgm

ents (U.S. EPA,1995, 1998).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-3 E XHIBI T 4-1
D EFI NIT IONS FOR C HAPTE R 4
Asses sment E

ndpoint - An ex plicit ex pression of

an environm

ental va lue (ecolog

ical res ource) that

is to be protected, operationally defined by risk managers a nd risk assessors as valuable attributes of an ecological ent ity. Benchmark Dose

(BMD) - T

he dose w hich cause s a spec ified le vel of r esponse. T

he lower confidenc e limit

on the BMD is u sually referred to as the BM DL. Comm unity - An assemblage of populations of different species specified by locales in space and time. Conceptual Model - A site conceptual model (SCM) in the problem formulation for an ecological risk assessment is a w

ritten description and visual representation of predicted relationship s between ecological entities and the stressors to which they may be exposed, including sources and pathways of stressors. Ecologica

l Risk A ssessm ent (ERA) - The proce ss that ev

aluates t he likeli hood that adve

rse ecolog

ical ef fects may occur or are occurring as a result of exposure to one or more stressors. Lines of Evidence- Information derived from diff erent sources or techniques that can be used to characterize the level of risk posed to exposed receptors; weight-of-evidence ge nerally refers to the quantity of science, while strength of evidence generally refers to the quality of science. Lowest-Observed-Adverse-Ef fect Level (LOAEL) - T

he lowest level of a stressor ev aluated in a test that caused a statistically s ignific ant eff ect on one or more m

easurem ent endpoints linked to undes irable (adverse) biological changes. Measurem

ent Endpoint (Mea

sure of E ffec

t) - A measurable ec ological property that is related to the valued characte ristic chos en as the assess ment endpoi

nt. Measure

ment endpoi

nts (also c alled m

easures of ef fect) often are express ed as the statistic al or nume ric sum marie s of the obs ervations that ma ke up the measur ement. No-Observed-Adverse-Effec t Level (NOAEL ) - The highest le

vel of a stressor adminis tered in a test that did not cause a statistically significant ef fect in any m

easurement endpoint linked to an undesirable (adverse) biological change. Population

- An aggregate of

individuals of a species within a specified location in space and time. Receptor - The ec ological e

ntity (w ith various

levels of orga nization) e

xposed to the

stressor

. Risk Char

acterization (ecological)

- The thir

d and last pha

se of ER A that int egrate s the anal

yses of expos

ure to stressors with as sociated ecological effects to ev

aluate lik elihoods of a

dverse ecological

effects. The ecologica l releva nce of the adverse effects is dis cussed, inc luding conside ration of t he type s, seve

rity, a nd magn itu des of th e effe cts , th eir spa tia l an d te mpor al p att ern s, an d th e li keli hoo d of reco very. Scientif ic/Management D ecision Point (SMDP) - A time during the ERA when a risk a ssessor communicates results or plans of the

assess ment at that stag e to a ris k mana ger. T he risk m anager decides if

inform ation is suff icient to proc eed wit

h risk m anagem ent strategies or w

hether m ore inform

ation is needed to characteri ze risk. Species - A gr

oup of orga

nisms tha t actually or potentiall

y inter breed and a

re reproduct

ively isolated f rom similar groups; also, a taxonomic grouping of morphologically sim ilar individuals. Stressor - Any chem

ical, physical or biological entity that can induce an adverse response in an ecological receptor; Superfund considers all stressors, but focuses on chemical (toxicant) stressors. Toxicity Refere

nce Value (TRV) - A

dose or conc entration used to approxim ate the e xposure thre shold for a spe cifi ed ef

fect in a spe cifi ed r ecep tor . A TRV is o ften bas ed o n a NO AEL o r LOAE L fro m a lab ora tor ybased test in a relevant receptor species.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-4 E XHIBI T 4-2
E COLOGICAL R ISK A SSESSMENT G UIDA NCE AND P OLICY D IRECTIVES
EP A has de veloped extensive guidanc e and p olicies on metho ds and a ppro aches for pe rforming E RAs, including t

he following: (1) Ecologi cal Ri

sk Ass essment

Guidance f

or Super fund: Process for

Designing and Conducti ng Ecologi cal Ri sk Ass essment

s ( “ERAGS”), Int

eri m Final (

U.S. EPA, 1997a)

. This document i ncludes processes and steps specif ical ly s elect ed for us e in ERAs at Superf und si tes . Thi s docume nt s upers edes t he 1989 EPA RAG S, Vol ume II, Envir onmental Eva luatio n M anu al, Interim Final (U.S. EPA , 1989). Supplements to ERA GS include the EcoUpd ates (U.S. EPA, 1991-presen t, I ntermi tten t Bull etin Series , 1991 to present), which provid e brief recom mend ations on co mmo n issues for Superfun d ER As. (2) Gu ideline s for E colo gica l Risk A ssessm ent ("G uide lines" ) (U.S. E PA , 199 8). This document updates ge neral (nonprogram speci fic) guidance that expands upon and re places t he earl ier Fra me wo rk for E colo gica l Risk A ssessm ent (U.S. EPA , 1992a). The ap proaches and methods out lin ed in t

he Gu idelines and in ERAGS are generally consistent with each other. (3) Risk Asses sment Gui dance f or Super fund ( RAGS): Volume 1–Human Healt h Evaluat ion Ma nua l (Part D, Sta nda rdized P lannin g, Rep orting, an d Rev iew of Su perfun d Risk Assessm ents) , (U.S . EP A, 20 01) . This g uidan ce sp ecifies for mats tha t are req uired to present data and re sult s in bas elin e ris k asses sments (both human and ecologi cal) at Superfund sites. (4) Poli cy Memor andu m: Guid ance on R isk Cha racterization for R isk Ma nag ers and Risk Assessors , F. He nry H abich t, Dep uty Ad ministrato r, Feb . 26, 1 992 (U.S . EP A, 19 92b ). Th is policy requir es basel ine ri sk ass essment s to pres ent ranges of ris ks based on “ centra l tendency” and “reasonable maximum” (RM E) or “high-end” exposures with corresponding risk estimates. (5) Po licy M emo rand um: Role o f the Eco logical R isk Assessm ent in the B aseline R isk Asses sment , Elliot

t Laws, Assis tant Administr ator, August 12, 1994 (U.S. EP A, 1994). Th is policy requir es the same hig h levelof ef fort and quali ty f

or ERAs as commonly perf ormed for human h ealth risk assessments at Sup erfund sites. (6) Po licy M emo rand um: EPA Risk Charac ter izat ion Progr

am , Carol Browner, Adm inist rator, March 2 1, 1995 (U .S. EPA, 199 5). This policy clari fies t he presentation of hazards and uncertainty

in human and ERA s, call ing for clar ity, transparency, r easonableness, and con sistency. (7) Issuan ce of Fin al Guidance: Ecological Risk Ass essment and Risk Management Princi ples for Supe rf und S it es. Ste phenD. Lu ft ig for Larry D. Ree d, Oct ober7, 1 999 (U. S. EPA, 1999). T his document presents si x key principles in ecological ri sk management and decision m aking at Supe rfund sites.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-5 SM DP= Scientif

ic/Management De cision Point Fig ure 4-2 . Eight-st ep Ecological Risk Assessment Process for Superfund (U.S. EPA , 1997a). ERA is a key

component of the remedial investigation process that E PA uses at Superfund

sites. ERAGS is a program-specific gu idance for Superfund that focuses on chem ical stressors released into the environment from hazardo us waste sites. This g

uidance refers to ERA as a “qualitative and/or quantitative appraisal of the actual or potential im pacts of contamina

nts from a hazardous waste site on plants and animals other than human s and domesticated species. An excess

risk does not ex

ist unless: (1) the stressor has the ability to cause one or m

ore adverse effects, a

nd (2) the stressor co- occurs with or contacts an ecological component long enough and a

t a sufficient intensity to elicit the identified adverse effect.” The ERAGS

document provides guidance on usi ng an eight-step process for complet ing an ERA for the Superfund Prog

ram, as show

n in Figure 4-2.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-6 4.1.2 P REDICTI VE VS O BSERVATIONAL A PPROACHES In general, conclusions about ec ological hazards from environ mental contam

ination may be based on informa

tion derived from

two different techniques: the predictive approach (a comparison o f calculated exposures with a set of toxicity reference

values), and the

observational approa

ch (direct evaluation of the ran

ge of potential expo sures, coupled with site- specific toxicity testing and population demo graph

ic estim ates). Predictive Approach : The core of all Superfund ERAs is the predictiv e approach, including exposu re asses sment , toxic ity as sessment, an d risk charact erizat ion. Th e predictive a

pproach is based on a com

parison of calc ulated e stima

tes of chemic al expo sure of a recept or to one or m

ore Toxicity Reference Values (TRVs) app ropriate for that chem

ical and that receptor. The ra tio of exposu re at th e site to the T RV is re ferre d to as t he Hazar d Quoti ent (HQ ). The p redic tive approach has alway s been used at Superfu nd sites because it is relativ ely easy to implement, and because it can be used to evaluate not only current risks, but also risks that m ight exist in the future if any im

portant changes were to occur in the lev el of contamination (e.g., due to on-going fate and transport proce sses), o

r to cha nges i n land u se (a change i

n land u se mi ght al

ter a number of habitat factors that influence the num ber and identify of ecological receptors). The predicti ve app roach, however, has

the inherent un

certaint ies of the assum

ptions i n the ex posure and toxicity m

odels which are seldom

site-specific and thus can lead to either ov er-protective or under-protective estimates of risk

. Direct Observation : If there is a need to reduce uncertainties in the predictive appro ach, direct observations of expo sure and effects can be c ollected at Superfund hazardous waste sites. The predictive approach u sed in ERA does not negate the use of descriptive toxicolog ical approaches or the use of site- specific exposure data, suc h as toxicity testing

or bioaccumulation measu remen ts. Sit e-sp ecific o

bserv ations, such as

toxicit y test ing of inver tebrates over a gradie nt of si te conta minan t exposure levels, m

ay be used to

devel op site- specific

and ch emica

lspecific toxicological relations hips. Site-sp ecific measures of ex posure or ecosystem characteristics can be used to reduce uncertainty in the exposure assessm ent and aid in the developmen

t of cleanup goals in the Baseline ERA . The direct observation of the exposure and effects on ecol ogica

l recept ors doe s not ho

wever constit ute a complet

e risk assessm ent. I f field or laboratory studies a

re NOT designed ap

propriately to elicit stressor- response relationships, direct im pacts sh ould not be use d as the

sole m easure o f risk because

of the d ifficulty in interpre ting a nd usin g thes e result s to dev

elop cle anup g oals in the ERA . Furthermor e, poor ly designed toxicolo gical evaluations of environmental media from the site may not allo w a definitive identification of the cause of adverse response. For exam ple, receptor abundance and diversity as dem

ographic data reflect m

any factors (habitat suitab ility, availability of food, predator-prey

relationships amon

g others). I f these factors are not prop erly controlled in the experimental desig

n of the study c ollecting the observ ational data, conclusions regarding chemical stressors can

be confounded. In addition, direct observation prov ides information abo

ut current risks only

and not potential risk

s should land use or exposure change in the future. Hence, direct observations may be used as a line of e vidence in an ER

A, but should not be the sole evidence used to characterize the presen ce or absence of the risk

s of an adverse e ffect in the future.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-7 4.1.3 P OTENTIAL A DVANTAGES AND L IMITA TION S OF P ROBABILIS TIC M ETHODS IN ERA Probabilistic risk assessm ent (PRA) is a com

putational tool that m

ay help increase the strength of the predictive evaluation of ecolo gical risks, as well as sometim es helping to better evaluate distributions of obse rvational data

for an ERA. The pote

ntial ad vanta ges of PRA c ompa

red to, or poss ible benefits in augmentation of, the convention al point estimate app

roach for characterizing

variability in ex

posure or risk ar e discu ssed in Chapte r 1 and Ex hibit s 1-6 and 1-7. In br ief, po int es timate calcul ation s util ize simplifications and assu

mptions in order to deal with the com plex mathem

atics of combining

inputs that are inherently variable. Probabilistic m odels, in contrast, are d

esigned to com

bine sets of inform

ation on inputs that are expressed as probability distributions . Therefore, PRA generally can y ield risk estimate s that allo w for a more comp lete cha racterization of

varia bility and uncertaint

y, and a pote

ntially more useful sensitivity a

nalysis as com

pared to estimating

sensitivities of inputs from point estim ates (see Append ix A). For ex ample , sensi tivity analy sis can help determin e major contrib utors to exposu re factors and sources of un certainty that could he lp to design better s ampling an d analysis plans in later iterations to help fill data g aps and reduce uncertainties for risk characterizatio n. Because of the inherent differences in the computational a pproach, as in the case

with any addi tio nal risk asse ssmen t inf ormat ion, PRA may s ometi mes le ad to a dif fer ent risk asse ssmen t out come and risk m

anagement decision than would be derived from

the use of point estim

ate calculations alone. The differences in the decisions stemming from the two approaches will vary from case to ca se, depending m

ainly on the form

of the exposure or risk

model, the a

ttributes of the distributions of the input values, and the quality, quantity , and representativeness of the data on w hich the input distributions are derived. Sometimes the differences between the tw o approaches will be q

uite large, and the infor mation gai ned fro m a PRA can pl ay an impor tant r ole as we ight-of-evid ence in communica ting ri sks to stakeholders and risk managers. Even thoug

h PRA ma

y have som

e advantages, it also has limitation s and potential for m isuse. PRA can not fill basic

data gaps and can not eliminate a ll of the potential concerns associated with those data gaps. That is, if one or more of the input distributions are not well characterized and the dis tri buti on(s ) must be es tima ted or as sumed , the n the resu lts of th e PRA a ppro ach w ill shar e the same uncertainty as the po int estimate va lues. However, given eq

ual states of know

ledge, the PRA

approach may yield a more com

plete characteri zation

of the e xposure or risk distribution than the p oint est imate approach. Of course, any

predict ion of exposure

or risk is base d on th e use of

mathemati cal models to represent very complex env ironmental, biolog

ical, and ecological systems. No matter how sophisticated the computations, questions will alway s exist as to whether the calculated values are a good approximation of the truth. Therefore, even when PRA is used as a supplem

ental tool to point estimations (determ

inistic) of risks in the E

RA process, a weight-of- evidence approach that combines the predictive approach w

ith direct observations w

ill still provide the m

ost appropriate basis for decis ion making. A second application of PRA in ERA, besides the characteriza tion and incorporation of distributions of data for ER A, is the characteriz ation of uncertainty in calculated estimates o f exposure or risk. In t his app lication , whate

ver un certaint y may exi st in one or m

ore of t he inpu t distrib utions i s characterized, and q

uantitative estimates

of the confidence lim

its around the mean, upper bound, or any other percentile of the outpu t distribution are calculated. This use of PRA is often especially im

portant in risk mana gement dec ision m

aking , since

the ran ge of uncerta

inty a round c entral te

ndency exposu reRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-8 (CTE) and reasonable m

aximum exposure (RME) or other upper bound estim ates of exposure or risk

can sometimes be quite large. A s stated before, the point estimate approach c an also provide estim ates of uncer taint y, but the PRA appr oach of ten pro vides a mor e complet e chara cteri zation of the un certa inty.
4.1.4 F OC US O F T HIS C HAPTER This chapter focuses on the a pplication of PRA as a tool for predicting ecological risks a t Superfund sites. Som

e of the methods and approaches describe d in this chapter are sim ilar to those that have been dev

eloped by U.S . EPA's Office of Pesticide Prog rams Com

mittee on 

### Federal I

nsecticide, Fungic ide and Ro denticid e Act (FIFRA) R

isk Assessm

ent Methods (ECOFR

AM, 1999a, 1999b ) for use in a ssessin

g env ironmental ha

zards of pesti cide pro ducts. Howev er, the metho ds described in this chapter are specifically d esigned to be app licable at Superfund sites an d to be consistent with other Superfund guidanc e. This chapter does

not see k to provide

guida nce on the many b asic issues that must be face d in planning and performing any ERA. Prior to considering the use of PRA in an ER A, fundamen

tal concept s will a

lready have been de

velop ed, suc

h as a problem

form ulation with a concept

ual site mode l, selection of representative receptors, definition of expo sed populations, definition of risk man agement objectives and g

oals, s election

of asse ssmen

t endpo ints, ca

lculation of TRVs

and development of site sampling pla

ns, etc. Likew

ise, this chapter does not repeat the presentation of b asic statistical and mathe matic al methods u sed in P RA, si nce the se are d

escribed in othe r chapte rs and a ppendic es of th

is document . In summary:

L This chapter focuses on application of PRA techniques to ERA at Superfund sites.

L The reader is assume d to be familiar with the basic methods used in ERA at Superfund sites, and th is chapter does not ad dress basic tactical and technical issues in ERA.

L The reader is assume d to be familiar with the basic mathematical principles and techniques of PRA as

described in other cha

pters and appendices o

f this document

.


### 4.2 D ECIDI NG 

### I F AND 

### W HEN TO 

### U SE 

### PRA IN 

### E COLOGICAL 

### RISK 

### A SSE SSMENT As shown in Fig

ure 4-2, the

ERA process for S uperfund includes a num ber of scientific/ managem ent decision points (SMDP s) (U.S. EPA, 1997a). The SMDP is a point of consultation between the risk man

ager, EPA R

egional Biological Technical Assistance Group (BTAG) coordinator, EPA regional ecotoxicolog

ist, and other stakeh

olders, and is intended to provide an opportunity for reevaluation of direction and goals of the assessment at critical po ints in the process. I t is during the SMDP discussions that it is important to decid

e whether or not a P RA is likely to be useful in decision making . If so, the pursuit of distributed data is justified. Within the 8- step process of developing the ERA, PRA c ould provide insig ht at several steps. A decision to m ove forward with distributional analyses should be considered within the B TAG context during the documentation of the outcome of the SMDPs after Step 3 w

ithin the process. As a reminder, PR A is NOT intended to be a replacement for point estimate

analy ses; rather PRA

supplements

the req uired pr esentati on of p oint est

imate s of risk . It

is also em phasiz ed that the use of PRA should never be vi ewed as

or used in an a ttempt to sim

ply g enerateRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-9 an alternati ve risk estimate or PRG, compared to that which was derived by a point esti mate ERA; instead , PRA

should be used to prov ide insi ghtful

inform ation on distributions o f vario

us factors (exposu re, toxicity, and ha

zards)which can provide weight- of-e vidence eva

luation s of pot

ential ri sks in conjunction with a point estim ate ERA. There are a number of fac tors to consider in m

aking these decisions, as di scussed below.
4.2.1 T ECHNICAL C ONSIDERATI ONS The fundamental reason

for performing

any predictive

risk assessm

ent (point estimate or probabilistic) is to provid e information to risk

managers in order to help support the risk man agement decision-m

aking process. As noted abov e, a properly perfo

rmed PRA may help to yield mo re description of var iability in expo sure and risk

than ca n be ach ieved using the poi nt estim ate approach. Therefor

e, if any o f a site’ s data m

ay be better d

escribed and ev aluated by di stributi ons, then a PR A can b e applie d to any part of an E

RA or even to the entire ERA for expressing risk characterization in probab ilistic terms; again, alway s in conjunction with the requ ired point estimate E

RA. Howev

er, when risk estimates derived from the point estimate ap proach are either far below or far above a level of risk m anagement concern, any su

ch potential improv

ements in risk

characterization are not likely to influenc e risk managem ent decision mak

ing. I n these cases, PRA

is not likely to be as useful in decision making . Even so, PRA m

ay help in these s ituations by prov iding informa

tion that may

be useful in better decidin g where the gradient of

excess risks are red uced to acceptab le levels. Rather, it is more common for a PRA to be useful when po int estimates of risk s are close to the decision threshold (such that PR

A-based refinem ents in the risk

estimates m

ight b e imp ortant i

n mak ing ri sk m

anage ment decisions). I

t is for this reason that PRA m

ay be useful to a

pply either during

the developm

ent of the ERA after the screen (Steps 3 to 6, U. S. EPA, 1997 a), or after point estim ate results from the baseline ERA hav e been comp leted (S teps 1 t

o 7, U .S. EPA, 199 7a). The results of a point estim ate risk assessm

ent will normally

present the range of risks based on central tendency expo sure and reasonable m aximum exposure input assum ptions and on the no- observedadverse-effect-level (N OAEL)- and lowest-ob served-adv erse-effect- level (LOAEL)- based TRVs (U.S. EPA, 1 992b, 1997b) . The bo unds for the hi ghest HQ are derived from

the rati o of the

RME compare d to the NOAEL-

based TRV, and the bounds for the lowest HQ are based on the ratio o f the CTE compa red to the LOAEL-

based TRV. These two boun ded extreme estim

ates of risk can be used to screen out cases where PRA is not likely to be a s useful. That is, if the risk to the RME receptor is clearly below a lev el of concern using the NOAEL- based TRV, then risks to the exposed population

are likely to be low and PRA analysis is likely not need ed. Likewise, if risks to the CTE receptor are clearly above a level o f concern using the LO AEL-

based TR V, then risks to the exposed popula tion are likely to be o f defini

te concern , and a

PRA m ay no t prov

ide as m uch add itional useful i

nformation to

the risk mana ger, except in the case where uncertainties remain high and the derivation o f an appropriate and realistic clean-up goal may be difficult. If the risks are intermediate between these two bounds (e.g. , risks to the C TE receptor are below a level of concern based on the LOAEL- based TRV but are above a level of conce rn based on the NOAEL-

based TRV), then PRA might be helpful in further characterizing the site risks in balance with the point estim

ates of risks and in supporting decision m aking or in deciding if additional itera tions of analyses would

be needed. This conc ept is illustrated graphically in Figure 4- 3.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-10 Figure 4-3 . Exam ple o f cases w here u se of P RA may b e help ful. In case s A an d E , the rang e of risks ( CT E to RM E) es timated by the p oint estim ate me thod are eithe r well be low (C ase A ) or we ll abo ve (C ase E ) the likely level of concern based on the NOA EL-LOA EL range, and PR A is not li kely to alt er risk management decisions regar ding the pote ntial nee d for re med iation. In c ases B , C, and D, the poin t estimate s of risk o verlap or fall within the range of po tential concern, suggesting that P RA-b ased risk estimates might b e helpful in suppo rting risk manage ment dec isions. The second main tec

hnical reason to consider c onducting PRA is that the PRA m

ethodology c an help characterize and quantify the deg ree of variability and uncertainty aroun d any particular estim

ate of exposure or risk (e. g., the CTE or RME). The purpose of the analysis would be to estimate the uncerta inty a round a n expos ure or t oxicity or risk estimate, g enerally

with li ttle or n o additi onal da

ta acquisition. The only additional information needed to perform the analysis is an estimate of the uncertainty in the true parameter valu es of the key

variables in the v

ariability model. In som

e cases, these estimates of u

ncertainty around para

meter values may be developed from statistical analysis of the available data. A

lternatively, profes sional judgment m ay be used to e stablish credible bounds on the parameters, especially

when relevant d

ata are sparse.

L Even in the presence o f data gaps, uncertain ty analysis using PRA can provide useful information. Ind

eed, it is when data are limiting or absent that a qu antitative probab ilistic a nalysis

of unce rtainty may be

most h elpful.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-11 4.2.2 C OST AND S CHEDULE C ONSIDERATI ONS

Performing a

PRA can som

etimes add tim

e and cost to an ER

A. As discussed in Chapter 2, in part, the decision to progress from a point estimate assessment to a P RA reflects a belief that the potential value of the PRA for risk m anagement decision mak

ing outweigh s the additional time and costs. The tiere d proce ss enco

urage s a sy stema

tic appr oach for both t

he poin t estim ate and probab ilistic assessm ents, w hereby the lea st com

plex m ethods are app lied first. For exam ple, th e initial

Tier 2 assessm ent may be conduc ted with a set of prelim inary probabi lity d

istribut ions for varia bility (PDFv ), developed with m

uch the same information and assu mptions that were

applied to develop

point estimates in Tier 1. Paramete

r values can be e stimated by setting the arithm etic mean equal to

the CTE point estimate, and the 95 th percentile equal to the R ME point estimate. The choice of distributions m ay differ depending on the

state of knowled

ge for a particular v

ariable (see Appendix B ). For exam ple, unboun ded v ariables

mig ht be characteri zed with log norma l distrib utions w hile bou nded di stributi ons are characterized by beta or Johnson Sb distribution s. Certain variable s may co ntinue to be characteriz ed by point estimate s, espe cially if the s ensitiv

ity analysis

sugg ests tha

t the chemica

l, path way, and/or exposu

re variables are relative ly minor co ntributors to total exposure an d risk. The decision to collect additional data or explore alternative methods for d eveloping pro

bability distributions can be reexamined in an iterative fashion by

evaluating the

expected benefits of the added information to the risk m anagement deci sio n-makin g proc ess . The se co ncep ts ar e pre sent ed in great er de tai l in C hapt er 2 ( see F igur es 2-1 and 2- 2). 

### 4.3 P ROB LE M 

### F ORMULATIO N Once a decision has b een made to in clude PRA in an ERA, the first step sh ould be to re-visit the problem form

ulation step and carefully determine the sco

pe and objectives of the PRA. Typically , a considerable amoun

t of knowledg e will have been gained during the screening lev

el and baseline point estimate evaluatio

ns, and this kn

owledge should be used to help focus and narrow the scope of the PRA. That is, the PRA

will g enerally

utiliz e the sa

me ba sic expo sure and risk mode ls used in the p oint est imate approach, but the PR

A will typically

evaluate only

a sub-set o f the scenarios considered. For example, chemicals, pathways, and/or receptors that are found to contrib ute a negligible

level of exposure or risk may usually

be omitted from

the PRA, while those factors that contribute sig nificantly to an excess

level of risk concern in the point estimate ap proach should gen

erally be retained. As noted previou sly, when a chemical or pathway is omitted from a PRA

analysis, this does

not mean that it is eliminated from the overall risk assessment; rather, it m ay be kept in the assessment as a point estimate. The next step in problem formulation for a PRA should be to define whether the goal of the analysis is to characterize variability alone , or to characterize both variability a nd uncertainty. I

n either case, sensiti vity analysis (as summarized in the prece ding paragraph, or for more deta ils see Appendix A) should be used to he lp identify which o f the input variables contribute the most to the variability in the outputs (expos ure, to xic effe cts, or risk), and the initial PRA sh

ould fo cus on definin

g the probabi lity density functions (PD Fs) for those input v ariables. An analy sis of uncertainty, if thought to prov ide additional useful inform

ation, may

also be included at the initial level, or m ay be delayed until the initial analysis of varia

bility is comp leted. As alwa ys, problem

form ulation should be vi ewed as

an itera tive p rocess, and it i s reason

able and appropriate that decisions regarding the

scope and direction of the PRA should b e reassessed (at SMDPs) as inform

ation becomes av

ailable from the initial evaluations. As stressed above, the fundam ental cr iterion which should b e used i s whether or not furthe r PRA evaluations a

re like ly to provide additional information to a poin t estimate ERA that will help strengthe n and support the risk managem ent decision-m

aking process.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-12 

### 4.4 M ODELI NG 

### V ARIABI LIT Y IN 

### E XPOSU RE There are two main types of descriptors o f exposure that m ay be used in E

RA: dose and concent ration. For te

rrestrial receptors such as ma

mmals or b irds, ex

posure is mo st often

describ ed in terms of ing ested dose (mg /kg- day). I

n most cases, this will be based on chemical inge sted from drink ing water and/or the diet, inc luding incidental soil in gestion, but could

also include am

ounts of chemical taken up acro ss the skin or

throug h inhala tion as additional rout

es of ex posure. The exp osure le

vels a re most often ex

pressed as dose s, sinc e that te

rm ten ds to no rmalize the confound

ing fa ctors of varia ble daily intake

rates an d body weig hts that occur i f/when one onl

y evaluates concent

rations. For aq

uatic receptors, the main route of exposu re is usually by dir

ect con tact and less of ten by ingestion, s o expos

ure is usually characteriz ed in terms of c oncentration of contam inants in surface water, p ore water and/or sediment. L ikewi se, exp osure o f terrest rial plan ts and t errestria l inve rtebrate s, such

as earth worm s, is usually described in terms of concentration of contaminants in soil. In some cases, ex posure of terrestrial receptors is characterized in terms of specific tissue or whole- body concentrations of contaminants. Exam ples of calculat

ing an d prese nting dose-

based a nd conc entration-ba sed dist ribution s of exp osure a

re presented bel ow. 4.4.1 C HARA CTE RI ZI NG V ARIABIL ITY I N D OSE The general equation use d for calculating the dose of a contaminan t of concern in a specified environmental medium (e.g., water, soil, air, diet, etc.) by a particular mem

ber of a population of exposed receptors is:
DI i,j = C i x IR i,j / BW j where: DI i,j = Average daily intake of chem

ical due to ingestion of medium "I" by a population member "j " of the expose d popul ation (mg/kg-day) C i = Conc ent rat ion o f che mical in en viro nment al med ium "I" (mg/un it med ium) IRi,j = Intake rate of medium

"I" at the site by population m ember "j" (units of m

edium per day) BW j = Body weigh t of pop ulati on member "j " (kg) Total exposure of a population membe r "j" is then the sum of the exposures across the different media:
DI total,j = 3 DI i,j In thi s bas ic eq uat ion, IR i,j and BW

j are random v

ariables (i.e., they have different m easurable values for different mem

bers of the exposed po

pulation) that are often correlated . For exam ple, a receptor with a relatively low in

take rate can also b e expected to have a low body weight. Som e studies utilize paired measu remen ts of

IR and B W by in divi dual , and pres ent a dis tri buti on of the r ati o (IR i,j /BW j ), refer red to as a body we ight-nor malized i ntake ra te (mg/kg-day). This ex press ion pro vides an alter native to usin g a correlation coefficient to relate two input variables (see Appendix B), and can be entered into the dose equation as follows:RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-13 where t he ratio is char acteriz ed by

a single proba

bility distribution. Becaus e the variabili ty in t his ratio is likely t o be dif

feren t than t he varia bilit y in the ra tio of the IR and BW variabl es tre ated i ndepen dentl y, accounting for the c orrelation can affect the distribution of dose and risk . If em

pirical data for quantifying the ratio are limited but a relationship is expected, plausible ranges of c orrelations may be explored as a source of uncertainty in the risk estimates. The conc entration term

(C i ) may be char acteriz ed by

a point estimate or a

probab ility distribution, depending

on the relationship between the geog raphic scales of the m

easurement data an

d receptor home rang e (see A ppendix C, Section C. 3.1). If the home rang e of the receptor is sm

all compared to the spatial distribution of sam pling locations, C i may

be char acteriz ed by

the pro bability distr ibuti on for va riabi lity in measure d conce ntrat ions. Al terna tively, if the h ome range i s large compared with the exposure area evaluated, then a point estim ate (e.g., m

ean or uncertainty in

the mean) may be more appropriate. In the PR A approach, PDF s should be defined for as many of the input variables as reasonable, especially for those v

ariables that are judged (v ia sensitivity analysis) to contribute the m ost to the variab ility i

n total exposu

re. The basic pr

inciples for sele cting the ke

y va riables to mo del as P DFs are presented in Appendix A, and the basic m ethods used for selecting

and fitting distributions are described in detail in Appendix B

. Figure 4- 4 shows several examples of g raphical formats wh

ich may b e used to present the estimated distribution o f ingested doses in an exposed population. If a sing le distribution is plotted (top panel), the PDF form

at is usually the m

ost familiar and use ful for risk assessors an d manage rs, but the cumu lative distribution fu

nction ( CDF) f

ormat tends t o be les s clutte

red whe n mul tiple di stributi ons are shown (e.g. , com

pare the midd le gra ph to the bottom

graph ). I n additi on, per

centiles can be read dir

ectly from a CDFformat, but not from a PDF format grap

h. In all cases, it is very

useful to superim

pose the CTE and RME point estim ate ranges of expo sure directly on the same graph as is used to show

the distribution of exposures estim ated by PRA. This provides a convenient way to compare the results of the two alterna tive c ompu

tational methods, an

d interpret addi

tional i nformation that the P RA can add to the point estimate E

RA.

L A conve

ntional point e stimate , range of expo sure ( CTE toRME) or toxic ity (NOAEL to LOAEL) and

corresponding risk ran

ges should be calcula ted and presented for com

parison with the PRA results.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-14 Figure 4- 4. Exam ple Gra phical Pre sentations of Do se Distributions.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-15 4.4.2 C HARA CTE RI ZI NG V ARIABIL ITY I N E XPOSURE C ONCE NTR ATI ON As noted above , in some cas es the most app ropriate descriptor of exposure is concentration (either in an abiotic m

edium such as water, soil, or sedim ent, or in the tissues of the receptor), rather than ingested dose. Assuming that the concentration values in the med ium of concern are measured rather than m odeled, PRA is

not req uired to gener ate the distribu

tion of concent

rations. Rathe

r, the a vailab le data may be used to define an appropriate theoretical or em pirical distribution function (ED F), as described in Appendix B. If co ncentrations in the m

edium are m

odeled (calculated by PRA) rather than measured, then the exposure distribution m ay be estim

ated by using

distribution functions (PDF s or CDFs, rather t

han usi ng po int estim

ates as inputs t

o the fa te and t ranspor t mod el(s) and/or up take m

odels that predict the concentration levels in the m edium of concern . The resulting distribution(s) of concentration may

be displayed g

raphically using the same form ats as illustrated in Figu

re 4-4, exc ept that the x-axis has units of concentration rather than dose. As above, the po int estimate rang es of concentration used in the CTE and RME calculations should be plotted on the same gra phs to provide a convenient basis for comparing the results of the two ap proaches and to help in

terpret the additional information that the PRA can add to the point estimate o utputs. 

### 4.5 M ODELI NG 

### V ARIABI LIT Y IN 

### T OXICITY 4.5.1 V ARIABIL ITY I N RESPONSE A MONG M EMBERS OF A P OPU LAT ION Data on the toxicity

of a chemical u sually comes from laboratory

studies whereby groups of organ isms (laborat

ory m amm als, fish, bent

hic org anism s, plan

ts, eart hworms, etc. ) are ex

posed t o differing levels of chemical, and o ne or more resp onses (endpoints) are m easured (survival, growth, repro ducti on, etc .). The se toxi cologi cal obs ervati ons def ine the exposu re-base d stre ssor-re sponse curve that is characteristic for that specific receptor, chemical, and response.
In the point estimate approach, inform ation fr om the dose/stressor-

respons e curve is ge

nerally converted to one or more TRVs, each representing a specific p oint on the dose-based or concentrationbased stressor-respo

nse curve. Fo

r example, the highest dose or co ncentration that did not cau se a statistically significant change in a toxicolog ically significant en

dpoint is defined as either the NOAEL dose or the no- observed-e ffect concentra tion (N OEC), while t

he lowe st dose or conc entration that d id cause a statistically sig

nificant effect on a relevant endpoint is the LOAEL dose or the lo west-observedeffect c oncentr

ation (L OEC). Gener

ally, exposu

res below NOA

EL-or NOE C-b ased TRVs are interpre ted to p ose acc eptable

risk, while exposure s above LOAE L- or LOEC -bas ed expo sures ar e judged to pose

potenti ally u

naccept able risk. I

t is esse ntial to note the need f

or hig h qualit y toxi city d

ata to deriv e reliab le and c onfiden

t TRVs. S trong sampl ing an d study desig ns, tha t gene rate dat a for sit e exposure factors and toxico logical stressor-response relationships, are o f critical importance for producing hig

h quality ERA s by either point e stimate or PRA approaches. Shortcomings in e ither area could be major data gaps or uncertainties that detract from the confidenc e in the risk characterization of the ERA , and m

ay be a basis for purs

uing a dditional iterat

ions of sampl ing or

studies that are more stron gly design ed to fi ll tho se cri tical data gap s and re duce un certa inty. Use of the TRV approach, however, does have some potential limitations. Most important is that the ability of a study

to detect an adverse effect depends on both the range of d oses tested and the statistical power of the study (i.e., the ability to detect an effect if it occurs). Thus, studies with low power (e.g., those with only a few test animals per dose group) te nd to yield NOAEL or NOEC values that are higher than studies with good power (those with m any animals per dose group). In addition, theRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-16 choice of the TRV is restricted to d oses or concentrations that w ere tested, which m

ay or may not be close to the true threshold for adverse effects, an d this uncertainty increa ses as the interval be tween doses inc rea ses . Fin all y, it i s not alwa ys eas y to in ter pret the s igni fic ance of an expo sur e tha t exc eeds some particular TRV, since the severity and incidence o f response depends on the shape and slope o f the exposu re response cur

ve (informa

tion tha t is not capture d in a p oint est imate TRV).
One w ay to resolve some of the se stres sor- respons e limi tations is to ap

ply un certaint y fact ors to the NO AEL or

NOEC and LOAEL or

LOEC , which

calcula tes an a djusted TR

V that red

uces the study ’s exposure level of co

ncern to account for thos e uncertainties, so that there is a lesser chance of ov erlooking possible adverse exp osures (i.e., avo iding a false neg ative conclusion). Another way to resolve som

e of the stre ssor-respons e limi tations is to fit

a ma thema tical eq uation t o the av ailable exposu

re-re sponse data and describe the entire exp osure-response curve. This m ay be done usin

g any con venient data fitting softwar e, but EPA ha s deve loped a softwa re pack age sp ecifical ly des igned for this type of effor

t. This software is referred to as the Benchmark Dose Software (BMD S), and is availab le along with detailed docum entation and ex planatio n of the methodolog

y at www.epa.gov/n cea/bmd s.htm

. The most appropriate m

athematical form

of the exposure-

response model d

epends on whether the endpoin t mea sured is

discret e and d ichotom ous (e. g., s

urviv al) or co ntinuou s (e.g ., gr

owth ra te). Fo r a dichotomous end

point, the result of the fitting exercise is a ma

thematical exposure-

response model P

that yields the probability

of a response in an individual exposed at any specified lev el of exposure (expressed either as dose or concentratio n). Exhibit 4- 3 shows an exam ple of this process using

hypothetical data. Thus, for an ind

ividua l with a n expos ure level of "x" , the probabil ity of a response in t

hat indi vidua l is simply P(x). In a po

pulation of individuals with exposures x1, x 2, x3, ...xi, the expected num ber of respons es (e.g ., deat

hs) in t he expo sed pop

ulation is the s um of the pro babiliti es acros s all ind

ividua ls in the population. Stated another way, the average fraction

of the population that w

ill experience the respons e is gi

ven by the expecte d valu e of P ( i.e., t

he ave rage v alue of P(x)).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-17 E XHIBI T 4-3
M ODELING V ARIA BI LI TY I N R ESPO NSE F OR A D ICHOTOM OUS E NDPOI NTRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-18 For a continuous endp oint, the BMDS software yields equations th at give the expected mean response m(x) at a specified exposure level, along with the standard deviation s(x) that characterizes how variab le the re sponse is among d ifferent indiv iduals exposed

at that same exposu

re lev el. The standar

d devia tion m ay be mode led either as a c onstant (hom ogene ous v ariance)or a fu nction o f the ex

posure level (heterogeneo

us variance), with the choice depending on which approach yields the best ag reement with the observed variances. I n most cases the

re will not be sufficient data to allow a m

eaningful analysis of the true shape of the underly ing distribution of resp onses at a given exposure, so the choice of the distributional form of the variability in response will require an assumption. In the absenc e of any clear evidence to the contrary, it is consid ered likely that the distribution of responses will not be strongly skewed, and that the di stribution may be reasonably well modeled using a normal PDF (truncated as necessary to prohibit selection of biolog ically impossib le or implausible v alues). Thus, variability in respo

nse at dose "x" m

ay generally

be modeled a s: Response(x) ~ NOR

MAL[m(x), s(x), min, m

ax]
Howev er, if a vailab le data sugg est som e other distribu

tional f orm i s mor

e appro priate, that form should be used and justified. Exhibit 4-4 shows an exam ple of this process using

hypothetical data. In this case, the mean response was found to

be well mod

eled by the Hill e quation, and the standa rd deviation was found to be best characterized as a constant (rho=0). Thus, g iven an exposure level "x", the m ean response m(x

) may be calculated from the model, and this value along

with the standard dev

iation may then be used as parameters for an a

ppropriate type of P DF (e.g., n ormal) to describe the expected distribution of responses in a population of different individuals exposed at level "x". Section 4.7.2 desc ribes methods that may b e used to characterize and quantify the u ncertainty associated w

ith this approach.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-19 E XHIBI T 4-4
M ODELING V ARIA BI LI TY I N R ESPO NSE F OR A C ONTINUOU S E NDPOI NTRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-20 4.5.2 V ARIABIL ITY I N RESPONSE A MONG S PECIE S In som e cases, risk m

anagement decisions may also consider com

munity- level effects as well as population-lev

el or sub-pop

ulations effects. That is, a stres sor might b e considered to be below a level of concern for the sustainability of a comm

unity if only a small fraction of the total number o f exposed species are affected. I

n this case, toxicological responses may be best characterized by the distribution o f toxicity values a

cross species. This is referred to as a Species Sensitiv

ity Distribution (SSD ). This type of approach is gen

erally used for com

munities of aquatic receptors, since all of the d ifferent species that mak e up the comm unity (e.g. , all fish, bent

hic invertebrat

es, aqu atic pla nts, an d amp hibians that res ide in a stream) will be exposed to approxim ately the sam

e concentration of contam inant in the water. The process for generating

an SSD consists o f the following steps : (1) Select an appropriate ty pe of endpoint (lethality , growth, repro duction, etc.), and se lect an appropriate type of p oint estimate from the exposure-response curve for each species. For example, the TRV m

ight be the LC5 0 for lethality or the EC20 for grow th. The key requirement is that the SSD be com posed of TRV endpoints that a re all of the same type, not a mixture. (2) Collect all reliable values for that type of TRV from the literature for as m

any relevant species as possible. When more than one value is available for a particular species, either select the value that is judged to be of hig hest quality and/or h

ighest relevance, or combine the values across stud

ies to derive a sing

le composite TRV for eac h species. I

t is important to have only

one value per s

pecies to maintain equal weighting across species. (3) Characterize the distribution of TRVs across species with a n appropriate CDF. Note that there is no a priori

reason to expe ct that a

n SSD will be well characteri zed by a param etric distribution, so both param etric and empirical d

istributions should be consid ered. Once an SSD ha s been developed,the fraction of species in the exposed com munity that may be affected at some sp ecified concentration m ay be determ

ined either from the

empirical distribution or from the fitted distribution. Exhibit 4- 5 shows exam ples of this approach. In this hy pothetical case, the TRV selected for use was the LC low (in this case, the LC low is defined as all LC values <=LC10). A total of 13 such value s were located. The first g raphical presentation is the em pirical distribution function, where the Rank

Order Statistic (ROS) of each value is p lotted as a function of the log of the corresp onding

value . This m

ay be used di

rectly to estim ate the fraction

of the s pecies i n a com munit y that will be affected by

any particular env

ironmental concentration . For exam ple, in this case, it m

ay be seen that a concentration o f 10 ug/L wo uld be expected to exc eed the LC low for abo ut 33% o f the aq

uatic species for which toxicity data are available. The second graph shows how the data may be characterized by fitting to a continuous distribution. I n this case, a log

normal distribution was selected as a m atter of convenience, but other distributions ma y also yield acceptable fits. Based o n the best fit lognormal distribution for the SSD data, it is calculated that a concentration of 10 ug /L would be expected

to impact about 31% of the exposed species. Howev

er, as noted above, there is no special reas on to expect that an SSD will be well c

haracterized by a continuous param etric distribution, so som e caution should be us ed in the use of a continu

ous distribution to fit an S SD, especially w hen the SSD is bas ed on a limited number of species

and when the purp ose of the SSD is to estimate percentiles a nd exposures outside the observed range. The r

isk as sessor should a

lways presen t an evaluation

of the r obustne ss of an

SSD to aid in the decision process.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-21 E XHIBI T 4-5
H YPOTHETICAL S PECIES S ENSITI VITY D IST RIBUTI ONRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-22 Fig ure 4-5 . Exa mpl e Co mpa ri so n o f Exp os ur e Di st ri butio n t o TRV. 

### 4.6 M ODELI NG 

### V ARIABI LIT Y IN 

### RISK 4.6.1 V ARIABIL ITY I N H AZARD Q UOTIENT As noted above , the most com mon descriptor of risk used in predictiv e risk assessm

ents is the Hazard Quotient (H Q). The HQ is the ratio of the exposure for s ome gene ralized or typical hypothetical member of the receptor population at a site, compa red to an appropriate TRV v alue that equates to an accepta ble level of risk for that rec eptor an d chem ical. U sually

the HQ approa ch is not based on a sin

gle value , but on a range of v alues in which differen

t leve ls of ex

posure (CTE and

RME) ar e com pared to both the NOAEL

to LOAEL ben

chmarks. In g eneral, HQ valu es below 1 are interpreted as indicating acceptable risk, wh

ile HQ values ab ove 1 are interpreted as indicating the pote ntial for adverse effects. Because exposure varies among different members of an expos ed population of receptors, HQ values also vary among

membe rs of the exposed pop

ulation. Several a

lternative approaches for characterizing this variability by PRA meth

ods are presented below

. Variability Within a P

opulation Figure 4- 5 illustrates the simplest approac

h for summarizing variability in HQ value s among the

mem bers of an expo sed pop

ulation . In

this format, the TRV values appropriate for a particular exposu re are si mply superimposed on the graph illustrating the distribution o f exposures. This m

ay be done either for a dose-

based (as shown in the f

igure)or for a concent ration- based exposure parameter. This fo rmat allows an easy eva

luation of the fract ion of t he popu latio n above (HQ > 1) and below

(HQ < 1) each TRV, especially when presented in CDF form

at. However, this format does not allow fo r a quan titative estim

ate of the fraction of the pop ulation with HQ value s above any value other than 1,although a similar calculation and presentation could be m

ade for any multiple of the TRVs, which w ould di rectly equate toRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-23 Figure 4- 6. Exam ple Distribution o f HQ Values. that multiple of the HQ (e.g., depicting the results for a value equal to 10-times the TRV would show the fraction of the p opulati on with

an HQgreater than 10). More directly, the distributio n of HQ valu

es may b e calculated by div iding each exposu

re value by one or all of the TRVs based on the NOAEL, LOAEL , BMDL, etc., a s shown in Fig ure 4-6. Note that dividing a d

istribution by a constant does not chang e the shape of the distribu tion (only its scale), so the shap e of t he HQ d ist rib uti on wi ll ap pear iden tic al to that of th e exp osur e dis tri buti on. Fi gure 4-6 illustrates two HQ distribution s; one calculated using the NOAEL-

based TRV, the other using the LOAEL- based TRV. In a case such as this w here there are two or m

ore HQ distributions, a

CDF forma

t is generally easier to evaluate than a PDF format, since overlap betwe en the curves is m inimized. The CDF format a

llows an easy qu

antitative evaluation of the fraction of the pop ulation above and below any particular HQ level. For example , in the case shown in Figure 4- 6, it may

be seen that 83% of the populat ion is expected

to have HQ v

alues be low 1 b ased on

the NOAEL-

based TR V, while 4% are expected to have HQ values above 1 based on the LOAEL- based TRV. This type of description (percentage of the po

pulation with HQ v

alues within a specified ran ge) is very

helpful in predicting proportions of a population exposed to specified do ses of concern.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-24 Variability Between Species A similar approach

may be used for characterizing

the variability in

risks amon

g different species in a comm

unity. Fig

ure 4-7 is an

example that compares the distribution of concentration va

lues in a water body (the v

ariability mig

ht represent either time

or space) to an appro

priate SSD of TRVs for different species of aquatic receptors that migh t reside in that water bod y. Three different g

raphical formats are illustrated. In the upp er panel, the PDF of concentration is com pared to the CDF of the SSD. This format is easy

to understand and m

ay be interpreted v

isually, but is difficult to interpret quantitativel

y. The midd le pane l shows

that sa me informa

tion, b ut with both d istribut ions pre sented i

n CDF format . This allows for a qua ntita tive eval uatio n of the fract ion of t he spec ies th at will be above their re spectiv

e TRVs at any sp ecified part of the exp osure d istribut ion. F or exam

ple, us ing a simpl e graphical interpolation pro cess (shown by the dashed lines), it m ay be seen that the 90 th percentile of concentrati on (21 ug/L) will impact appr oximately 24% of the expos ed species. The bo ttom panel shows the results when this sam e process is repeated (m

athematically) for each of the concentration percentiles. As seen, hazards to the comm unity of receptor spe cies is quite low until con centration values reach the 80 th to 85 th percentile, but then rise ra pidly. For exa

mple, a concen

tration value equal to the 95 th percentile (about 28 ug /L, which will occur approximately 5% of the time) is expected to impact approximately 68% of the exposed spec ies, and the 99 th percentile (which will occ ur about 1% of the time) is expected to

impact nearly

all of the exposed spe cies.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-25 Figure 4-7 . Example Presentation of Species Sensitivity Dist ribution.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-26 4.6.2 V ARIABIL ITY I N RESPONSE As note d above, HQ and Ha zard I ndex (HI) (w

here ap propria te) va lues are a conv enient way to characteri ze risk to ecological rece ptors, but inte rpreting the biol ogical significanc e of the ranges of HQ values greater than 1 is not alway s easy. One of the main adv antages to the PR

A approach is that distribu tions of exposu re ma y be c ombin

ed with exposu re-re sponse distribu

tions in order t o generate distributions that characterize the frequency and magnitude (severity) of respo nses in an exposed population. Two examples of this approa ch are presente d below. Example 1: Dichotomous Respon se In this hy pothetical example, a toxic chemical is being transported by surface water run- off from a Super fund sit e into a nearby stream . Beca

use of short- term a

nd seas onal variabili

ty in r ainfall l evels (which influences both ru n-off rate and stream flow), the concentration of the chem ical in the stream h

as been observed to vary as a fu nction of time. The risk manager at the site wants to k now two thing s: (1) How often will the

concentration enter a range that can cause acute lethality in fish?; and (2) When that happens, what p ercent of the fish population is likely to d

ie? Exhibit 4- 6 summ

arizes the hypot hetical concent

ration d ata and illustra tes the basic ap

proach. In this cas e, the c oncentr ation da ta are most conve

niently mode led as a

n emp irical P DF. N ext, as sume that the acute c oncentr

ation- lethalit

y curve is availab le for the chemical of interest in a relevan t indicator species of fish. For convenience, assume the re sponse function is the same as that shown in Exh ibit 4- 3. Then , the P DF for acute m ortality may be generated by repeated sampling from the conce

ntration distribution and calculating the probability of response

(acute mortality

) for each concentration v alue selected. Because this is a case wher e the enti re po pula tio n of f ish at th e exp osur e loc ati on may b e ass umed t o be e xpos ed to the s ame concentration in water, the probability of m

ortality in a single fish is equivalent to the average fraction of the population that is expe cted to die as a result of the exposure. The resulting PDF is shown in the graph in Exhibit 4 -6. As seen, lethality is exp ected to be low or ab sent about 95% of the tim e, but about 5% of the time the concentration may enter a range w

here acute lethality m

ay occur. The exte

nt of mortality within

the exposed population

is expected to rang e from about 2 0% at the 97 th percentile of exposure (i.e., this is expected to occur about 3% of the time), u p to about 70% at the 9 9 th percentile of exposu re (i.e. , this i s expected to o ccur ab out 1% o f the tim e).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-27 E XHIBI T 4-6
M ODELING V ARIA BI LI TY I N A D ICHOTOM OUS R ESPONSERAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-28 Example 2: Continuous Response Exhibit 4-7 provides a hy pothetical example o f modeling variability in respo nse for a continuous endpoint. I

n this example, assume that a tox ic chemical has b een released by a Superfund site and has been tra nsporte d in low

level s by a

ir to a nearby

mead ow. A mong the rec eptors o f potent

ial conc ern in the meadow are a nu

mber of diffe

rent ty pes of small m

amm al, and the fiel d mou se has b een sele cted to serve as an indicator species for this group . The goal of the PRA is to characteriz e the effects of the chemical on the growth of field m ice in the meado

w. Exposure occu

rs mainly by ingestion o f seeds that have been contam

inated by uptak

e of the chemical from soil, and it has been determ ined that the variability in av

erage daily intak

e (DI) of chemical from

the diet can be m

odeled as a logno

rmal distribution with me an of 104 mg /kg- day, and a stan

dard deviation of 1 27 mg/k g-day . Assume for conve nie nce t hat the e xpos ure-r espo nse c urve for gr owth inhi bit ion i n mice by the chemi cal is th e same as that presented prev

iously in Exh ibit 4-4. Giv

en these inputs, the exp ected d

istribut ion of r esponse s is derived as follows:
Step 1: Draw a random

value for the D I of a random mem ber of the population Step 2: Calcula

te the m ean resp onse m

(d) and the sta ndard d eviati on of the response s(d)for a group of indiv

iduals exposed at that do se (d) Step 3: Define the dist

ribution of resp onses a

t that d ose as NORMAL [m(d) , s(d)] Step 4: Draw a

respon se from

that di stributi on, and

track this asthe out

put va riable An example o f the output for this exam ple is shown in the tw

o graphs at the b ottom of Exhibit 4-7. As seen, m

ice that are not exposed to the chemical d isplay a range

of growth rates ranging from about + 10% to +

40%. Ma ny of the m

ice (abo ut 90%) r esiding in the c ontam

inated f ield are experiencing a range of growth rates that are only slig htly decreased from

rates expected for unexp osed animals. However, about 10% o f the animals hav e weight g ains that are mark edly less than for unexposed anim

als, ranging from about +5% to -30% (i.e., a net weight loss of 30% compared to the starting weig ht). It should be noted that the response d istribution calculated in this wa y is what wou

ld be expected for a large population

of exposed receptors. If the actual exposed population is sm all, then the actual res pons e dis tri buti on may va ry some what compa red t o the typi cal resp onse show n in E xhib it 4-7 . In cases w here it i s imp ortant t

o eval uate thi s vari ability

about t he expe cted average pattern of respo nse, th is may be done by r

unning repeate d Monte Carlo s imula tions us ing a numb er of tri als (iter ations) within each simulation that is equal to the expected size of the exposed population. Each simulation will thus represent a possible response distribution in the exposed population, and the range of responses across different populations m

ay be evalu

ated by com

paring the m

ultiple simulations. As noted above , the magnitude of the variability between populations is expected to be sm all if the population siz e (number of trials) is large, althou

gh this depends on the characteristics of the exposure and response fu nctions.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-29 E XHIBI T 4-7
M ODELING V ARIA BI LI TY I N A C ONTINUOU S R ESPONSERAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-30 Figure 4-8 . Example Joi nt Probabil ity Curve. 4.6.3 J OINT P ROBABILITY C URVES In th is appro ach, if data are

avail able to characte

rize the proba

bility of a pa rticular exposu re occurring, and an

exposure-respo

nse curve is av

ailable, these may

be combined to yield a curve (referred to as a Joint Probability C urve) that shows the probability that a response greater than some specified magnitude will occur. An exa mple is shown

in Figure 4- 8. The upper panel sho ws a hypothetical cumulative ex

posure probability distribu tion (plotted on the prim

ary y- axis) along with the exposure-respon se curve (plotted on the secondary y

-axis). The steps needed to generate the Joint Probability Curve are as follows: Step 1: Select

an exp osure le

vel "x " and re cord the

probab ility (P x ) of exceeding that exposure. For exa mple, in Fig

ure 4- 8, at an

exposu re of 12units, the cum

ulativ e proba

bility of expo sure is 84%. Thus, the probability of exceeding that exposure is 16%. Step 2: Find t

he expe cted res

ponse a t that sa me ex posure (R

x ). In this case, the response at an exposure of 12 is 2. 2. Step 3: Plot a

data po int at R x on the x-axis and P

x on the y- axis. Step 4: Repeat this p

rocess for many differen t exposure levels, be ing su re to draw samples that adequately cov er all parts of the probability scale. The lower panel of Fig

ure 4-8 show s the results obtaine d using

the hy potheti cal data

in the upper panel. The ad

vantage of th

is format is that i t gives a cle ar visual disp lay of both the pro babil ity and magn itude (severi ty, extent) of response. Further, the area to the left of the curve is a

relative index of the population-lev

el or comm

unity-lev el risk, and comparison of this area across different scenarios is helpful in com paring different risk scenarios (both in

risk characterization and risk mana gement). H owev er, this approac h is bas ed on the mea

n response at a dose, a nd does not acc ount for varia

bility in respons e betwe en mu

ltiple in dividu als all expose d at tha t dose. Employing a two- dimen sional M onte Ca rlo anal ysis (2-D MCA) procedure could help to display this variability in response between the individuals at a given dose.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-31 Figure 4- 9. Example Present ation of Uncertainty in Exposure. Note that unless 2-D MCA is used, th is approach does not req

uire Monte Carlo m

odeling. Rather, the calculations can usually be perform

ed in a spreadsheet form

at using built- in spreadsheet functions. 

### 4.7 M ODELI NG 

### U NCERTAI NTY IN 

### E COLOGICAL 

### RISK 

### A SSESSMENTS As em phasiz ed abov

e, one of the g

reatest potenti al bene fits of t he PRA approa ch is the ability

to comb ine esti

mates of unce rtainty associa ted with differe nt com ponents of the e xposure and risk models in order to describe the overall uncertainty in final exposure or risk estimates. Some basic options for characteri zing and presenting uncert ainty in exposure, to xicity, HQ, and response are presented bel ow. 4.7.1 U NCERTAINTY I N E XPOSURE Most estimates of dose -based exposu re for terrestrial receptors (birds, m ammals) are

based on calculated estimates of chemical intak e using sim

ple or complex food web mo dels, sometim es coupled with environm

ental fate and transport m odels that can link risk to a receptor w ith a source of contam ination . In

cases w here receptors are exposed m

ainly by direct contact rather than ingest ion (e. g., f

ish, so il invertebrates, etc.), c oncentrationbased (as opposed to dose-based) descriptors of exposures m ay be derive d using

mathemati cal fate and tran sport m

odels. The basi c principles for modelin

g uncerta inty i n ecolo gical exposu

re models (either dose-

based or conc ent rat ion-b ased ) are the s ame as di scus sed i n App endi x D. In brief, probability distribution functions of uncertainty

(PDFu's) are used to characterize

the uncertainty in the para

meters of the probability distribution functions of variability

(PDFv' s) for some or all v

ariables in the exposure model. Then, a 2-D MCA is us ed to de rive quantitative estim

ates of the uncertainty around ea

ch percent ile of the vari ability distribution of exposure. Figure 4- 9 illustrates the type o f tabular and graphic outputs that this approach generate s. If exposure is based on measu red rather than calculated v alues by PRA

(e.g., m easured concentrations in an abiotic medium, measured conce ntrations in receptor tissues), u ncertainty in theRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-32 empirical or best- fit continuous distribution throu gh the data can be quantified using the statistical methods detailed in Appendix B. As discussed in Chapter 1, it is important to understand that there are many sources of uncertainty and that th

is approach to uncertainty analysis focuses m

ainly on param

eter uncertainty and uncertainty in the true shape of input va riable distributions. I

t does not capture other sources of uncertainty relating to the fundamenta l adequacy of the exposure and risk models used to describe the behavior of com

plex biological sy stems or of sam pling and analy

tical errors and uncertainties, so the uncertainty estim

ates should always be interpreted in this lig ht as being som

ewhat incomplete.
4.7.2 U NCERTAINTY I N T OXICITY Toxicity information used for ERAs is o ften a source of uncertainty in the risk assessm

ent process . This u

ncertain ty m ay ari se from

mult iple are as and m

ay inc lude bo th quan

titative uncer

tainty in the dose-respon se data (involv ing toxicokinetics

and study desig

ns) and qualitative un

certainty in the releva nce of t he data (involvi ng toxic odynamics ). Meth ods for charac terizi ng the qua ntita tive uncertainty in both p oint estimates of toxicity (TRVs) and in full expo sure-response cu rves are outlined below.
Unce rta int

y in T RVs TRVs for a

chem ical are point e stima

tes of exposure

level s that d

o not ca use an u naccept able effect in an exposed receptor population. Ideally , all TRVs would be based on NOAEL a

nd LOAEL value s derived from

studies in whi ch the r

eceptor , endpo int, ex

posure route an

d durati on were

all matched to the a ssessment endpoin ts defined for the site. H owever, such e xact matches are s eldom available. Therefore, it is often necessary to ex trapolate available toxicity data across route, duration, endpoint and/or species, leading to uncertainty in the most ap

propriate value to use

as the NOAEL or LOAEL. There are no default methods for developing TRVs on a site. However, som e options include the use of allome

tric dose scaling m

odels, physiolog

ically-based biokinetic mo dels, benchmark dose estimates or other ap

proaches based m

ainly on policy

and/or professional judgm ent. Guidelines for dealing with the unce rtainty in components of the TRV derivati on by uses of PRA are provided below. Uncertainty in NOAE

Ls and LOAELs
Uncerta inty i n the N OAEL or LOA

EL for a chem

ical has two components: (1 ) uncert ainty within a study; and (2) u

ncertainty between stu

dies, under exact specified conditions of exposure.
Assum ing that a sin

gle st udy has been

selecte d to provide t

he NOA EL and /or LOA EL va lues to be used in deri ving a TRV for

a chem ical, it is cust omary to defi ne the NOAEL as the h

ighest exposu re that did not cause a statistically significant effect, and

the LOAEL is the lowest exposure that did cause a statistically significant effect. As noted earlier (see Section 4.5.1), this approach has a num ber of limitations, and there

may be substantial uncertainty as to whether the observ ed NOAEL and

LOAEL value s actual ly bra

cket t he true thresho ld effect leve l. One way to quan

tify u ncertain ty in t he expo sure levels that cause som

e specified level of ad

verse effect is throug

h the use of exposure-

response curvefitting software such a s EPA's BMDS packag e. In this approach, the risk assessor selects some level of effect that is judged to be below a level of concern, and another le vel of effect that wo

uld be of concern. The choice of these response levels is a m atter of judgment, and depends on the nature and severity of the endpoin t being evalu ated. A specifi ed level of eff

ect is re ferred t

o as a B enchm

ark R esponse (BMR),RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-33 and the exposure that ca

uses that response is referred to as the Benchm ark Dose (BMD). Given inform ation on the num ber of test org anism s in each test g roup an d on th e vari ability

of the r esponse in those organism s, the BMD software uses maxim um likelihoo d methods to d erive the 5% lower confide nce bou nd on t he expo sure tha

t causes the BMR . This is

referred to as the BMDL . This uncerta inty b ound m

ay be used to

quantif y the uncerta

inty i n the B MD, and

hence t o characteriz e this source of uncertainty in the TRV. The simplest method for approximating the uncertainty distribution around the BMD is to assume the distribution is approximately normal, with mean equal to the BMD and stand ard devi ation (stan dard er ror) gi ven by: Stdev=(BMD -

BMDL) / 1.645 For adv anced a nalyse s, a m

ore accu rate cha racterization of

the unc ertainty

distribution ar

ound the BMD may be derived by Mo

nte Carlo simulation . In this approach, each m odel parameter is as sumed to be norma lly di stribute d, with

mean and sta ndard e rror v alues pr

ovide d by t

he BMD S outpu

t. Mont e Carlo simulation is then u sed to select alternative model param eter sets, being sure to account for the covariance between parameters (the cov ariance matrix is also provided by the BMDS output). For each parameter data set, the BMD is calculated, a nd the distribution of BMD values across m

any iterations is a better approx imation of the uncert ainty in the BMD.
Uncerta inty i n the ef fect lev el (NOA EL or L OAEL)for a chemica l may also ar ise beca use ther e is more than one study availab le for the chemical, and the studies do n ot yield equal estim

ates of the effect level. I

t is important to n

ote that the process of rev iewing availab le toxicity studies, ch oosing the most relevant endpoint for use in deriv ing a TRV, and identify

ing the m ost relevant study

is a process requiring basic toxicolog ical expertise (not probability or statistics), and this proce ss must be com pleted both for point estim

ate and probabilistic risk

assessments. In general, studies based on d ifferent receptors, endpoints, exp osure routes and/or duration s are not equally re levant for evalu ating a particular assessment endpoin

t in a particular indicator specie s. However, in some cases, multiple studies of the same endpoin

t in the same species will be avail able. I

n such a case, assum

ing that all the studie s are judged to be eq

ually reliable , the b est estim ate of the LC50

may be deri ved by calc

ulating the g eometric mean of the av

ailable alternative va

lues (after adjustment to con stant hardness). Uncertain ty around the best estimate m

ay then be based

on the observed

inter-study

variability, usin

g the basic principles for choosing PDFu's as described i n Appendix B. Uncertainty in Extrapolation of TRVs In general, extrapolation of TRVs across species or endpoints is not desirable, since the magnitude and direction of any potential error is generally not kno wn. Sometimes, extrapolations between species are attem pted based on allom

etric scaling m

odels that seek to adjust toxicity values account ing for differe

nces in body weigh

t. Alte rnativ ely, p

hysio logica

lly- based pharma

cokinetic (PBPK) models tha

t seek to account for differences in a num ber of other physio logical variables (meta bolism rate, organ s ize, b lood flow, etc. ) can be used. Howev

er, the validi

ty of t hese m odels is often not well established. In those ca ses where these m odels are used, and w

here the uncertainty in

the model is judged to warrant quantitative evaluation, the prim ary source of the model should b e consulted in order to deri ve an estimate of the uncer tainty in the q uality of the e xtrapol ation an

d in the param eters of the model. As noted earlier, PR A may capture uncertainty asso ciated with mode l input parameters, but doe s not us

ually capture

all sou rces of uncerta

inty i n the m odel. In p articula

r, mo st mo dels of this sort are desig ned to extrapol ate only

the average respons e as a fu nction o f dose, and are

not int ended t o extrapo late v ariabili

ty bet ween in dividu als at a specifie d dose. When no mat hema tical m odel is availa

bleRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-34 to support quantitative extrapolation across species, exposure duration or end point, professional judgment and/or policy may be used to select extrapo lation factors to account for the uncertainty. The risk assessor should

ensure that th e risk

mana ger un derstands the u

ncertain ty ass ociated

with any m odel sel ected an

d applie d, and that the

results of the c alculati ons (po int estim ate or P RA) are conditional upon the m

odel selected.
Uncert ainty in Paramete

rs of th e Dose-

Respon se Model

s When tox icological exp osure-

respons e data a

re fit to mathemati cal equations, the fitt ing so ftware will usually prov

ide quantitative inform

ation on the uncertainty

in the best estim

ates for each of the model param

eters. For exam

ple, in the dichotom

ous model illustrated in Exhibit 4-3 , the output from the BMDS s oftware

include d the fo llowing

inform ation on the unc ertainty

in the p arame

ters of t he best- fit logistic equation:

Parameter Best Est Std Error (SE)

a -4.80 0.83 b 0.101 0.019
Becaus e the un certa inty in t he best estimat e of eac h model pa rameter is asympto tical ly normal ly, uncertainty in the para

meters may

be modeled a s: PDFu ( parameter i)=NORMAL (best estimate of para meter i, SE of param

eter i) Note that the param

eters of the mode l are generally not independent, and generally shou ld not be treated as such . Thus, when m

odeling the unc ertainty

in the p arame

ters of t he best- fit expo sure-respons e mod el, the PDFv' s for the parameters should be correlated accord ing to the correlation matrix or the variancecovariance m

atrix, as provided by the m odeling software.
4.7.4 U NCERTAINTY I N RESPONSE If the risk characterization phase of the risk assessm ent focuses on an estim

ation of the distribution of responses rather than the distribution of HQ values, the unc ertainty in the distribution of responses can be ev

aluated using two- dimensional Monte C arlo techniques using PDFu's for the para meter s of t he ex posu re an d exp osur e-res pons e mode ls de rive d as d escr ibe d abo ve. Th e same graph ical out put may be used for this presentat ion as was illus trated in

Figu re 4- 9, exce

pt that the x- axis is response rather than H Q. This format is illustrated in Figure 4- 10 for a dichotom ous endpoint (e.g ., acute lethality). I

n this example, the average p robability of response among the membe rs of the exposed population (shown in the graph by the black diam

ond symb ols) is 8.2%, with a co nfidence bound around the mean of 4. 9 to 12.8%. This is equ ivalent to concluding

that about 8.2% of the population is expected to suffe r acute lethalit

y, bu t the tru e fractio n dyin g cou ld rang e from

4.9 to 12.8%.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-35 Figure 4- 10. Example Present ation of Uncertainty in Response.

4.7.3 U NCERTAINTY I N H AZARD Q UOTIENT Once the uncertainty

in exposure and/or toxicity distributions has be en characterized as desc rib ed ab ove, t here are a numbe r of o pti ons f or pr esen tin g the r esul tan t unc ert ain ty in the H Q (or HI, if appropriate and applicable for summ ing HQs) distribution s. Figure 4- 11 shows one sim ple graphical forma t, wher e the po int estim

ate of the TRV is s

uperim posed on the un

certaint y bou nds of t he expo sure distribution (upper panel), or the uncertainty bou nds of the TRV are superim posed on the best estim

ate of exposu re (low er pane l). One could also su perim pose the rang e of TRVsover the range of exp osures, to capture most of the uncertainty in the HQ. Furtherm ore, such distributional outp uts should always show the poin t estim ate ranges of C TE and R

ME expo sures in

respect to the r anges of TRVs, for use

in weigh t-of- evide nce to help inte

rpret the PRA and poi

nt estim ate resu lts. The advan tage o f this format i s that no additional Monte C arlo modeling is needed to derive initial descriptors of uncertainty in risk. For example, in the upper panel it m ay be seen that the best estimate of the fraction of the population exposed at a lev el below the TRV i s about

83%, bu t that th is is uncertain due to u

ncertain ty in t he expo sure estimates, and the true percent below the TRV m ight range from

74 to 90%. Sim

ilarly, in the bottom panel, the best

estimate of the fractio

n of the popula tion bel ow the TRV is als o about 83%, bu t due to uncertainty in the TRV the actual value could ra nge from 64 to 91%. Uncertainty could also be presented by show

ing a com bined graph with both ranges of ex posure and TRVs, such as described below.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-36 Fig ure 4-1 1. Exa mpl e Pr es en ta ti on of Unc er ta in ty

in Exp os ur e a nd TRV.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-37 Figure 4- 12. Exam ple Prese ntation of Unc ertainty in HQ E stimates. A more com

plete characteri zation

of unce rtainty in HQ may

be achi eved b

y usin g PRA to combine the un

certainty in both the exposure and the TRV terms, resulting in the uncertainty bounds on the HQ distribution itself (see Figure 4- 12). In this example, it m ay be seen that 6 3% of the exposed population is estimate d to have an H Q below 1.0, b ut that this is uncertain due to uncertainty in both the exposu re distri bution and the

TRV, and

that th e true fr

action o f the po

pulation below

a lev el of concern (HQ < 1) could range from 45 to 81%.



### 4.8 I NTERP RETI NG 

### RESULT S OF AN 

### E COLOGICAL 

### PRA
In s ome c ases, the inform ation contribute d by a

PRA m ay provide a

more comp lete characte rizatio n of ris ks to a popul ation of receptors than can be o

btained by us ing po int estim

ate methods. H owever, whethe r by PRA o

r by point estim

ate or a combina

tion, the results of the risk assessment m

ust be interpreted to reach a risk man agement de cision.
In contrast to the case for hum an health risk assessm

ents (where default risk

-based decision rules are wel l establ ished), there a re no establishe d defau

lt decis ion rule s for id

entify ing w hen risk s to ecological receptors are and are not of concern. In the p oint estimate approac h, EPA gu

idance (U.S. EPA 1992b, 1995) recom

mend s consid

eration of both

the RME and CTE exposu re/dose estimates along wi

th TRVs based on both LOA ELs and NOAELs (U.S. EPA 1997a) to reach a risk

managem

ent decision. The same principle app lies to probabilistic ERAs.
In som e cases, interpretation of an ecological PRA is relatively sim ple. For exam

ple, if the distribu tion of HQ v alues ca

lculated using an appropriate

NOAEL-ba sed TRV a

re less t han 1.0 for nea rly all membe

rs of the population, the

n it is likely that risks are within an acceptable range for the populat ion. C onversely, if the d

istribut ion of H Q val

ues calc ulated u sing a

LOAEL-ba sed TRV a

reRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-38 signif icantly greater than 1.0 for

most mem bers of an expo sed pop

ulation , then it is lik

ely that risks are not acceptable for the pop ulation. However, for cases which fall between these bounding conditions (and for cases where one ne eds to clearly define the boundaries of potential excess risks for a g radient of contamination and exposures), the leve l of risk or response that is considered acceptab le must be define d by the risk assessor and the risk manager on a site-specific an

d receptor-specific basis. This evaluation should take t

he following factors into acc ount: (1) The Risk Management G oal The risk mana gement objective fo

r mos t Super

fund ER As is d efined a s popul ation su stainab ility (U.S. EPA, 1 999). In th

is case, harm

to som e mem bers of the exp osed po

pulation may

be acce ptable, if that har m doe s not le ad to an

overa ll reduction in populat

ion vi ability . This s ituation (protection of a population rather than protectio n of individuals) is sometimes equated with use of the CTE (average) receptor as the

basis f or risk mana gement dec ision m

aking . That is , if the

HQ for the CTE receptor is below a level of concern, it is som etimes assum

ed that population risk

s are acceptable. However, the choice of the CTE receptor as the basis for risk managem ent decision mak

ing may not be sufficiently protective in all cases. For the vast m ajority of wild populations, the proportion of the population that mu st be protected to ensure population stability will be unknown. At a small number of sites, a population biolog ist may be able to provide som e information. Moreover, the percentile of the CTE receptor in the exposu re or risk distribution m

ay vary depending on the shape of the distribution

. The prop ortion o f the po

pulation exper

iencing exposu re gre ater tha

n that o f the CTE

receptor could range from less than 10% up to 50% or even higher. Also, the ecolo gical significance of an adverse effect on some m

embers of a po

pulation depends on the

nature of the stressors and on the life history and population biology

of the receptor species. Because of these com plexities, use of the CTE as a decision threshold for nonthreatened or endangered spec ies may b e appropriate in a sm all number of cas es, but risk assessors and risk

managers should realize that the choice of the CTE receptor requires a speciesand endpoint- specific justification and the CTE shou ld not be used as the default basis for a risk managem ent decision. Rather, for the majority of ERAs, the risk managem

ent decision should be based on the RME receptor or an upper percentile of the distribution of variability in risk/exposure. (2) The Toxicological Basis of the TRV The biolo gical signif

icance o f a distr

ibution of va riability in HQ cannot be inter

preted w ithout a proper understanding of the nature of the TRV being used to evaluate the distribution. This includes the nature of the toxicological endpoint underlyin g the TRV, its relevance to the assessm ent endpoint, and the shape (steepness) of the dose-response

curve. For example, an H Q of 2 based on an EC20 for reduction in reproductive success would lik ely be interpreted as more significan t toxicologically than an HQ of 2 based on the EC20 for an increase in liver weig ht. Likewise, an HQ of 2 based on an LC low for acute le thality would be more sig

nificant if the d ose- respons e curve for let hality were st eep tha n if it were shallow, since i

t would be easi er to ca

use gr eater re sponse with sm

aller inc reases i n expos ure to contaminants.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-39 (3) The Characteristics of the Receptor Ultimately, the que stion w hich m ust be a ssessed

is whether an effect o f degr ee "x" o ccurring

in "y" percent of the po

pulation is biologically

and ecologically

significant. This, in turn depends on the attributes of the receptor being evaluated. Fo

r example, a re duction of 10% in the repro ductive success of a fec und and comm on spec ies (e.g ., the field m ouse) m ight n ot lead to a sig

nificant reducti on in population numb er, while the sam e effect could be of con

cern in a species with lo

wer fecundity and/o

r lower p opulati on dens ity (e. g., t he mo ose). Th

us, the interpr etation of an an

alysis of va riability in exposure and/or effect often requires the input of a traine d population biologist with expertise in the receptor of concern.
Becaus e of the se issue s, there is no d efault r ule for what le

vel of effect i s and is

not acc eptable for an exposed

ecolog ical pop ulation ; excep t for the

case of no potential excess ri sks w here the RME exposures do not excee d the TRV based on a NOAEL, assum ing there is reasonab le confidence in those exposure and toxicity

values. I n some cases, mathematical models m ay be availab le for predicting the populat ion- level consequ

ences o f a giv

en patte rn of ef fects (e .g., see EC OFRAM 1999a for som

e aquat ic population models), but in general the extrapolation from a distribution of individ ual responses to an estimation of popu

lation-level e ffects is difficult. For this rea son, close consultation b etween the risk manager an

d the ecological risk

assessor is necessary for translating results of a n ERA into an appropriate and successful risk managem

ent decision. 

### 4.9 G UIDELINES 

### F OR 

### P LANNING 

### A ND 

### P ERFORMI NG A 

### P ROBABI LIS TIC 

### ERA
4.9.1 P LANNING AN E COLOGICAL PRA Chapte r 2 provi des a gene ral di scuss ion of t he key step s that should be foll owed whe n plann ing a PRA. These g

uidelines are equally a

pplicable to ecological PRA as to hum an health PRA. Of the key steps in the pr ocess , most impor tant a re the follo wing: Dialogue Among S

takeholders As discussed in Sec tion 4.2, the decis ion if and when to perf orm an ecological PRA is an SMDP shared by risk

assessors, risk m

anagers, and stak

eholders, including members of the public, representatives from

state or county en

vironmental a

gencies, tribal g overnment representatives, natural resource trustees, private contractors, and potentially responsible parties (PRPs ) and their representatives

. A scoping m

eeting should be held after the comp letion of the baseline risk assessment in order to discuss the potential purpose and objectives of a PRA, and to identify the potential value of the analysis to the risk management process. If it is decided to perform at least an initial PRA evaluation, subsequent meeti ngs o f a sim

ilar ty pe shou ld occu r iterati vely in orde r to ass ess whe

ther any further effort i s warranted. Preparation of a W

orkplan Any PRA beyon d the si mplest s creen ing level evaluat ion sho uld alw ays be acc ompanie d by a workplan. The pu

rpose of the work

plan is to ensure that all p arties agree on the p urpose and scope of the effort, and on the spec ific methods, data, a nd proc edures that wil l be use d in the PRA. Workp lans sho

uld be developed acco

rding to availab le guidance for workplans for nonpro babilistic ERA (U.S. EPA, 1992b,RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-40 E XHIBI T 4-8
E XAMPLE E LEMENTS OF A W ORKPLAN FOR E COLOGICAL PRA
1. Int roducti on/Overvie w Conceptual s ite model Asse ssmen t endp oints Indicat or species Measures of ex posure and ef fect
2. D escrip tion of E xpo sure an d R isk M ode ls Ba sic exp osure mod els (fate an d trans por t, uptake , food web , intake, etc .) Basic ri sk models ( HQ, dichotomous res ponse, cont inuous response)
3. Results f

rom a Po int Esti mate Assessment CTE and RME ris k estimat es f rom base li ne eva luation 4. Rationale why a PRA will be helpful

Goals of the assessment (vari abilit y, uncertai nty, both) Expected benefi t to r isk mana ger
5. De scr ipt ionof t

he Pr opose d PRA Exposure scenar ios t o be evaluat ed Output var iables to be modeled i n vari abili ty an d/or uncer tain ty s pace
6. Pr opo sed P DF s, and th eir ba sis Method for perfor

ming sen sit ivi ty an alys is and f or sele ctin g key var iables Data source f

or charact erizi ng key v ariabl es Approach for sel ecti ng and paramet erizi ng key v ariabl es Proposed lis t of PDFs for exposure var iables (optiona l but desirabl e) Method for dealing wit h the concentrati on term Method for dealing wit h correlations 7. Proposed Software and Simul ation Approach Commerci al or

cust om Monte Carl o or Lat in Hy percube Numb er of Iterat ions M ethod (s) for se nsitivity analysis
8. Pr elimina ry Res ults (op tional, b ut help ful) Resul ts of a s creen ing lev el e valu ati on Identifi cation of variables where more effort is needed to improve the dis tri buti on functi on 1997a) and sho

uld con sider th

ree elem ents: (1 ) the 16

guidi ng pri nciples of MCA (U.S. EPA, 1997b) ; (2) the eight guide lines for PRA report s ubmis sion (U .S. EPA, 199 7b); and (3) the tiered a pproach to ERA (U.S. E

PA, 1997a). D evelopment of a workplan fo r PRA is discussed in greater detail in Chapter 2, and Exh ibit 4- 8 sum

marizes the k

ey ele ments of a pr oper wo rkplan . The workpla

n must be submitted

to the BTAG coordinator an d/or regional ecotoxico logist for review and for approval by the risk manager. The EPA strongly

recommends that PRPs wh o wish to perform

PRAs of ecolog

ical risk involv

e the Agency in the dev

elopment of a

work plan in order to

mini miz e chanc es of si gnific ant disa greem ent, as is requi red by EPA p olicy.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-41 4.9.2 E VALUATING AN E COLOGICAL PRA When an ecolog ical PR A is su bmitted to EPA for conside ration, it will be rev

iewed i n order to determine if it has b een performed in accord with sound princ iples of ERA (U.S . EPA, 1997a, 1998), and with so und pri nciples of PRA (U.S. EPA, 1997b) . A g eneral checkl ist that may be help ful to re

viewe rs is provid ed in A ppendix F, and key feature s of thi s check

list are summ

arized in Exh ibit 4- 9. Eig

ht speci fic conditi ons for accepta

nce of a PRA s ubmit ted to EPA are provid

ed in U .S. EPA (1997

b). At the discretion of EPA risk assessor or risk manager, the PRA report may be submitted for additional EPA internal rev iew and/or an external rev iew process in accord w

ith Agency guidelines for conducting peer rev

iews (U.S. EPA , 2001). The external p eer review m ay be used in c ases where the issues are complex

or contentious and the opinions of outside expert peer reviewers can improve the PRA.



### 4.10 E XAMPLE OF THE 

### T IERE D 

### P ROCESS IN 

### ERA
As disc ussed in detail i n Chap ter 2, one of the key elements in the risk

assessm ent proc ess is deciding if and w

hen further analysis is warranted. This includes de cisions regarding whether to em

ploy PRA ca lculations to su

pplement poin

t estim ate calc ulation , and i f so, w

hat level of eff

ort to in vest i n those P RA calc

ulation s. The f

ollowin g sect ion pres ents a r

elativ ely si

mple hypot

hetical exam

ple illustrating how the tiered approach mig

ht operate at a site where ecological risk is an important conc ern. E XHIBI T 4-9
C HECKLI ST FO R I NCLUDI NG A PRA AS P ART OF THE ERA (S EE A PPENDIX F )

• All risk assessments shoul d include point esti mates prepared according to current Superfund national and regional guidance.

• A workplan must

be submitted for

revie w and approval by the appropri ate EPA regionalproject mana ger (RPM) an d/or

BTAG coordi nat or pr ior to s ubmi ss ionof t

he PRA.

• A tiered approach should be used to determine the level of complexity appropriate for the ERA. The decision to asc end to a highe r level of comp lexity should be m ade with the risk man ager, regional risk assessor and other stakeho lders.

• The eig ht c ondit ions for accept ance pr esen ted i n th e EPA poli cy on PRA (U.S. EPA, 1997b) sh ould be clearly a ddr essed by eac h PR A sub mitted to the A genc y.

• Information in t

he PRA should possess suff icient detai l that a reviewer can recreate both the input distri butions and all f

acets of the anal ysis. This includes copies of publis hed papers, electronic versi ons of n eces sar

y da ta and ot

her mat eri als deeme d appr opri ate by EPA.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-42 Problem Formulation PestCorp is a form

er chemical m

anufacturing facility that produced ma inly chlorinated pesticides 10 to 20 years ago. Data co llected on the PestCorp property indicate that a number of sp ills or releases of chlor inated p esticide s took

place w hen the facility was in operati on, and

that sit e soils are broa

dly conta minated , espec ially wi th pest icide X. Thi s conta minated soil ha s lead t o impact s on a nea rby lake of abou t 300 ac res th at rec eives su rface water r unoff from the P estCor p site . Samples from the l ake reveal low but de

tectable levels of pesticide X in water, with relatively high v alues in sediment and in the tissues of a variety

of aquatic organism

s (crayfish, snails, b enthic macroinvertebrates and fish). The concentration values in

all media (water, sediment, aqua tic organisms) tend to be highest in the part of the lake receivin

g runoff from

the PestCorp property , with a gradien t of diminishing

values at locations further away from

the area where runoff enters the lake.
A BTAG comm ittee for med by EPA

to iden tify p otential

ecolog ical con cerns at the site recognized that many differen t species could be expo sed to the contam inants in the lake, including aquatic receptors residing in the lake (fish, in vertebrates, aquatic plan

ts), as well as m

ammals an

d birds that frequent the lake for food or water. B ecause pesticide X is lipophilic an d tends to biom

agnify in the food web, the BTAG

decided that the hig

hest risks would lik

ely occur in hig

her-level p

redators such as mam malia n omn ivores , and s elected the raco on as a good indicato r specie s to represent t

his trophic group. Pathways of exposure that were identified as warran ting quantitative evaluation included (a) ing estion o f water, (b) ingestion o f aquati c food i tems, and (c)inciden

tal ing estion o f sedim

ent whi le feeding or drink

ing at the lak

e. The BTAG determ

ined that the assessm

ent endpoint was protectio n of mamm alian omnivo

re populations. Point Estimate Risk Ev

aluation A series of iterative sc

reening-lev

el point estimate calcu

lations (Steps 1 to 2 of the 8-step ERAGS process) were performed to in vestigate whether or not there was a basis for concern at the site. Initial calculations u sing simplified and conservative inputs (i.e., exposu re based on the m

aximum measu red concentrati on in ea

ch me dium, an area

use factor of 1 , and t

he mo st conse

rvativ e avai lable TRVs) indicated that the HQ v alue for pesticide X could be quite large. Therefore, a refined screening level evaluation was performed in which point estim

ates of CTE and RME risk were derived usin

g the best information cu rrently available . Key elements of the approac h are summarized below:

• The CTE receptor was assum ed to be exposed at a location where concen tration values were the average for th e whole

lake, and the

RME rec eptor was assum ed to be expose d at a lo cation w

here concentrations were equal to the 95 th percentile of values from the lake.

• Becaus e only

limit ed data were av ailable for measured

concen trations of pest icide X in aquatic prey items, the concentration value s in aquatic prey were estimated using a linear bioaccu mulat ion m odel: C (prey) =C(sed)x BAF . The B AF was estimated from the

existin g data by finding the best fit correlation between the concentration value s in sediment an d crayfish at 7 locati ons in t

he lak e: C(cr ayfish )=5.04x C(se d) (R 2 =0.792 ).

• The TRV values were based on a study in mink in w

hich the toxicity end

point was the percent inhibition of reproductive success.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-43 E XHIBI T 4-10
R EFI NED S CREENING P OINT E STIMATE I NPUTS AND R ESULTS These inpu ts and the resu lting HQ v alue s are show n in Exhi bit 4 -1 0. A s see

n, estimated

risk s to t he C TE receptor appro ach or s lightl y exce ed a lev

el of concern (H Q=4.7E -01 to 1.4E +00), a nd risk s to an RME receptor are well above a level of conce rn (9.1E+00 to 2.7 E+01). The chief pathway contributing to the dose an d risk is ing estion o f contam

inant in aquatic food w eb items (crayfish, fi sh, am phibian s, etc. ).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-44 SMDP 1 at Step 2 of ERAGS The BTAG considered these results to indicate that inhibition of reproduction was po ssible in at least some m

embers of the ex

posed population, but that the fraction of the pop ulation that was affected and the degree of impact on the p opulation was difficult to judg e from the point estimate calculations. Based on this, a de cision was mad e to conduct a screening

level PRA in order to provide some additional information on the magnitude and probability of risk

. Workplan 1 The contractor performing the risk assessm

ent developed a b

rief workplan that p

roposed an approac h for a screenin

g level PRA . The pl an calle d for a Monte C arlo- based e valua tion of variab ility i

n exposure and risk among d ifferent memb ers of the exposed m ammalian omnivore (rac oon) population. In brief, all ex

posure inputs that were treated as constants in the p oint estimate approac h (i.e., were the same for CTE and

RME exposure) were also treated as constants in the PRA evalu ation. Because water contributed so little to dose or HQ, this pathway was not ev aluated in the PRA, but was accounted fo r by adding in the p oint est

imate value s to the

PRA r esults. All variables

that are fractions (i.e, may

only assume valu

es betw een ze

ro and o ne) wer e mod eled as beta distributions, and all oth er var

iables were modeled as log

normal. For screening purposes, the parameters for all distributions were selected so that the mean and 95 th percentile values of the PDF's matched the corresp onding CTE and RME point estimates. The BTAG

reviewed this prop osed approach and au thorized PRA w

ork to begin

. Screeni ng Leve l PRA R esults The scree ning l evel P RA inp

uts and the resulting

estimates of the va

riability in HQ are shown in Exhibit 4-11. The CTE and RME po int estimates are also shown for compariso n. As seen, the PRA distr ibuti on of HQ val ues ind icate s that about 6 8% of th e indivi duals in the po pulat ion are likely to have HQ values below 1E+00, while 3 2% have HQ v alues above 1E+0 0. Comp arison o f the CTE

point e stima

tes of H Q to th e mea

n HQ v alues de rived by PR

A rev eals the values are very close. This is ex pected because both dep end on the mean values of the input variables, and the same me an values were u sed in both sets of calcu lations. With regard to

upper-boun

d estimates, the RME

point estimate v

alues are at the 98 th percentile of the PRA HQ distribution, w

ithin the target range (90th to 99 th ) usually considered appropriate. Note, h owever, that the 98 th percentile is about 5-fold hig her than the 95 th percentile, emphasizing the hig h sensitivity of the RME HQ values to the precise percentile of the RME

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-45 E XHIBI T 4-11
S CREENING L EVEL PRA C ALCULATIO NS OF HQ D IST RIBUTI ONRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-46 SMDP 2 The BTAG considered these results, and decided that it was very probable that pesticide X was causing an effect in some members of the ex posed population, but decided that a final risk managem

ent decision would be facilitated by characterizing the distribution of response s (rather than the distribution of HQ values). The BTAG asked the contractor perform

ing the work

to develop a p

roposed approach for characterizing the distribution of responses. Workplan 2 The contractor obtained a copy of the toxicity repo

rt upon which the TRVs were based, and determined that the study did include sufficient dose-respo nse data to support reliable dose-response mode ling. The con

tractor recom

mend ed that this be done us ing EPA' s BMDS . The B TAG approved this proposed approach and authorized work to proceed.
PRA Re finemen t 1 The contractor fit the raw dose- response data (inhibition of reproduction in m ink) to a num

ber of alternat ive m odels av ailable in BMDS , and f

ound that the d

ose- respons e curve could be wel l characterized by the Hill Equation with nonconstant variance, as follows: Mean Response at dose d (% decrease in reproduction)= (100 x d 2.5 )/(0.9 2.5 + d2.5 ) Std. Dev. in

Response at dose d

(%)=SQRT[1.6 @ (mean response a t dose d) 1.3 ] Based on this m

odel, the point estim

ate LOAEL value (0.6 mg /kg- day) corresponds to an effect level of about 2 7%, and

the NOAEL of

0.2 m g/kg -day corresp onds to

an effe ct lev el of ab out 2%. Using this exposure-resp onse model in place of the point-estimate TRV values , the refined PRA predicted a distribution of resp onses in the exposed p opulation as shown in Exhibit 4-12. As seen, approximately 81% of the population was predicted to experience a n effect on reproduction sm aller than 10%, wh ile 9% w ere exp ected to

have a reduc tion of 10 to 3 0%, 4% a reducti on of 3 0 to 50 %, and 6 % a reduction of more than 50%. On av erage across all m

embers of the ex

posed population, the predicted reductio n in reproducti

ve suc cess wa s about

9%.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-47 E XHIBI T 4-12
S IMULAT ED D IST RIBUTI ON OF R ESPONSESRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-48 SMDP3 The BTAG debated the lik ely population- level consequences of this predicted distribution of responses in mem

bers of the exposed po

pulation. After consulting with a field biologist with experience in the population dy

namics of m

ammals su

ch as racoons, the B TAG decided that the distribu tion of respons es in the expos ed popu lation would ca

use a continued

stress o n the m amm alian om nivore commun ity and that reduction s in population num ber were likely

over time. Based on this, the risk manager an

d the BTAG agre ed that remedial actio n was desirable and tha

t a range of alternativ

e clean-up strategies should be in

vestigated. This was performed u sing the m ethods described in Cha

pter 5 (see Exhibit 5-5 ).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 4 ~ De cem

ber 31, 2001

Page 4-49 REFERENCES FOR C HAPTER 4
ECOFRAM. 199 9a. ECOFRAM Aquatic Report (Draft). Ecological Com mittee on FI

FRA Risk Assessment Methods . Draft report availab le online at http://www. epa.gov/opp efed1/ecorisk. Report dated May 4

. ECOFRAM. 199 9b. ECOFRA M Terrestrial Draft Report. Ecolog ical Comm

ittee on FIFR

A Risk Assess ment Met hods. Dr aft re port ava ilabl e onlin e at htt p://w ww.epa. gov/oppe fed1/ ecori sk. Report dated May 1 0.
U.S. EPA. 1 989. Risk Assessment Guida nce for Superfund, V olume II: Environmenta l Evaluation Manual. Interim Final

. Office of Emergency and Remedial Respon se. Washington, D.C. EPA/540/1- 89/001. March.
U.S. EPA. 1 991-present. Eco U pdate . Interm

ittent Bulletin Series. Office of Emergen cy and Reme dial Re sponse. 1991 t o prese nt. U.S. EPA. 1 992a. Framework for Eco logical Risk Assessment. EPA Risk Assessm ent Forum. EPA/63 0/R-92/0 01. Febr uary. U.S. EPA. 1 992b . Pol icy M emora ndum: Guidance on Risk Characterization for Risk Managers and Risk Assessors

from F. H enry Habicht, D eputy Adm inistrator, February 26

. U.S. EPA. 1 994. Memorandu m : Role of the Ecologica l Risk Assessment in the Baseline Risk Assessment . Elliott Laws, Assistant Administrator, Office of Solid Waste and Emergency Response. OSWER

Directive No. 9285.7-17. August 12.
U.S. EPA. 1 995. EPA Risk Character ization Program . Memorandum from the Adm

inistrator. March 21.
U.S. EPA. 1 997a. Ecological Risk Assessm ent Guidance for Sup erfund: Process for Des igning and Conducting Eco logical Risk Assessments. Interim Final. Solid Waste and Emergency Response. OSWER

Directive No. 9285.7-25. June 5.
U.S. EPA. 1 997b. Guiding Principles for Mon te Carlo Analyses . Ris k Asse ssmen t For um. EPA/630/R- 97-001.
U.S. EPA. 1 998. Guideli

nes for Ecolog

ical Ris k Asses sment. Risk Assessm

ent Forum. U .S. Envir onme ntal Protection Agency, Wa shing ton DC . EPA/630/R- 95/002 F. Apr il. Published May 14. Federal Register 63(93):26846- 26924.
U.S. E PA. 19 99. Mem

orandum : Issuance of Final Guid ance: Ecological Risk A ssessment and Risk Management Principles for Superfund Sites . P. Stephen D. Luftig for Larry D. Reed, Office of Emergency

and Remedial Response. OSWER Directive No. 9285.7-28. October 7.
U.S. EPA. 2001. Risk Assessment Guida nce for Superfund : Volume I. Human Health Evaluatio n Manual (Part D, Standardized Planning, Reporting, and Re view of Superfund Risk

Assessments). Offic e of Emerge ncy and Re medial Re sponse . Washi ngton, DC . OSWER Di recti ve No. 92 85.7-47. D ecemb er.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-1 

## C HAPTER 

## 5
P ROBABILISTIC 

## R ISK 

## A SSESSMENT AND 

## P RELIMINARY 

## R EMEDIATION 

## G OALS 

### 5.0 I NTRODUCTI ON According to the National Contingen cy Plan (NCP) (U.S. E

PA, 1990a, 40CFR §300.430(d)(4)), risk asses sment and risk manage ment deci sion makin g go hand- in-hand: da

ta from the rem edial in vestigation ar

e used to characterize risk

, and results of the ba seline risk assessm ent help to esta blish ac ceptabl e expos

ure levels for use in developin

g remedial a

lternatives. I

n practice, risk managers may id

entify two m

ajor objectives of risk assessm ent: (1)to dete

rmine if rem ediation is nece ssary (i.e., Is there unacceptable risk at the site? ); and ( 2) if remed iation i s neces sary, to dete rmine a prelim inary remediation g oal (PRG) (i.e., What chemical concent rations would result i

n a risk estima te that will be adequately protective of human health and the environ ment? ). The answer to the first question (is there unacceptable risk? ) depends upon a n umber of factors, includi ng the meas ured or estimated concentra

tion lev els of conta minan ts in sit e med ia, and takes uncerta inty i n the measurem

ents into account. I

n contrast, the answer to the second question ( what is the PR G need

ed to achieve a specified level of protection? ) does not necessarily depend o

n any kno wledge of the actu

al level or pattern of site- specific concentration data, an d does not nec essarily

depend on the uncerta

inty i n site concentration data. Thus, while exposure point concent rations (EPCs)and PR

Gs are c losely

related to each other, they have important differences (see Section 5.1 for furthe

r elabor ation on EPCs a nd PRG s).
Once a risk m

anage r has se lected a

PRG a t a site, de termin ing w hether a particu lar area meet s or wil l meet the PRG

require s carefu

l com parison of site data with the PRG, inc luding a consideration of the uncertainty in the site data. For a further disc ussion on vari abi lit y and u ncer tai nty i n the conc ent rat ion t erm, readers are urg ed to consult A ppendix C in th is guidance. E XHIBI T 5-1
S UMM ARIES OF S OME K EY T E R M S Preliminary Rem

ediation Goal (PRG

) - initially developed chemical concentration for an environm ental m edium tha t is expe cted to be prot ectiv e of hu man hea lth a nd eco systems. PRGs may be developed based on a pplicable or

releva nt and appropriat

e requirem

ents, or ex

posure sce

narios evaluated prior to or as a result of the baseline risk assess ment. (U .S. EPA, 1991a).
Generic PRG

- a chemical conc entratio n protec tive of human health developed prior to the baseline risk assess ment tha t uses def ault expos

ure assum

ptions represent ing com mon expos

ure scena rios, e.g. , Region 3 ris

k-based c oncentrations

(RBCs) or Region 9 PRGs. Site-specific PRG

- site-specific chemical concentrat

ion, protectiv

e of hum an health a

nd ecosystems , based on ex

posure sce

narios in the baseline r isk ass essme

nt. Gene rally calculat

ed for the various

exposure scenarios

considered in the baseline r isk ass essme

nt.
Rem ediati

on Go als (RG) - site-specific chemical concentrat

ion, protectiv

e of hum an health a

nd ecosystems, chosen by the risk manager as appropriate for a lik

ely la nd use sce

nario.
Remedia tion Action Level (RAL) - the “not-to-exceed” level; a concentration such that remedi ation of a

ll concentr

ations abov

e this lev el in an exposure

unit lower

s the EPC suf

ficientl y to achieve a targe t risk le vel. T he RAL will depe nd on the mea n, varianc e, and sam

ple size of the concentrations within an exposure unit as well as considerat

ions of short

-term effe cts of the

chemic als of concer n. Cleanup Leve

l (Final R

eme diatio n Leve

l) - chemical concentration chosen by the risk manager after c onsidering bot

h RGs and the nine rem

edy selection criteria

of the NC

P (U.S. EPA, 1990a)

. Also re ferred to as Final Remedi ation Lev els (U.S. EPA, 1991a), c

hemica l-specif ic cleanup levels are documented in the Rec ord of Decision (ROD) . A cleanup level may dif

fer from a PRG because risk managers ma

y conside r details of the sit

e-specif ic exposure, various uncertainties in the risk estimate, and implementation issues (e.g., the technical feasibility of

achievi ng the PRG)

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-2 E XHIBI T 5-2
D EFI NIT IONS FOR C HAPTE R 5 95% UCL for mean

- The one-sided 95% upper confidence limit for a population mean; if a sample of size ( n ) was r epeatedly drawn f rom the population, the 95% UCL w ill equal or excee d the true popula tion mea n 95% of the

time. It is a m

easure of uncerta

inty in the

mean, not to be conf

used with the 95 th perce ntil e (see b elow) , whic h is a mea sure o f variab ilit y. As sample size inc reases , the diff

erence be tween t he UCL f or the mean and the tr

ue mea n decreas es, while the 95 th percentil e of the dist ribution rem

ains rela tively unchanged. 95 th Percentile

-The number in a distribution that is greater than 95% of the other values of the distribution, and less than 5%of the v

alues. W hen estim ated from a sam ple, this quant

ity m ay be equal to an obse

rved va lue, or inter

polated fr om among two values. Applicabl e or Rele

vant and A

ppropriate Requireme

nts (ARARs) - Federal or state environm ental standards; the NCP states that ARA Rs should be c onsidered in de termini ng rem ediation goa

ls. ARA Rs may be selec ted as sit e-specif ic cleanup levels. Backca lculation - A m ethod of ca

lculating a PRG that

involves algebra ic rearr angem ent of the risk equa tion to solve

for con cen tra tio n as a fun cti on o f ris k, ex pos ure , an d to xici ty. Bootstrap Me

thods - Parame

tric and non-pa

ramet ric me thods for e stimat

ing confi dence inter

vals f or a stati stic by

resam pling directly from the data s et with r

eplacem ent. Coverage - Co nfi den ce in ter vals are e xpe cted to e ncl ose a tru e bu t un kno wn pa ramet er ac cor din g to a spe cifi ed p rob abi lit y, such as 90%

or 95%. Thi s is the e

xpected c overage

of the c onfidence

interval , given a

specif ied signif icance le vel (alpha ). The difference betwe en the expected coverage and the actual coverage is one metric for evaluating statistical m ethods that yield different confidence intervals. Exposure Point C oncentration (EPC) - The

averag e chem ical conce

ntration to w

hich rece ptors are e

xposed w ithin an expos

ure unit. Estim

ates of the EPC repre sent the c oncentration term us ed in exposur e asses sment. Exposure U

nit (EU) - For Superfund risk assessment, the geographical area about which a rece ptor moves and contacts a contamina

ted medi um during the period of

the exposur

e duration. Forward C alculation

- A m ethod of ca

lculating a risk e stimat

e that invol

ves the s tandard arr

angem ent of the risk equa tion to sol ve fo r ri sk as a fun cti on o f con cen tra tio n, e xpo sur e, an d to xici ty. Iterativ e Reduction

(IR) - A method of

calculat ing a PRG that involv

es succe ssively

lowering the concent

ration term

until the calcu lated risk is accept able. This met hod can be appl ied t o any med ium. Iterativ e Trunca tion(IT) - A

method of

calculat ing a PRG that involv

es developing an ex

pression f or the conce

ntration ter

m in which hig

her value s of conc entration a re rem oved or “tr

uncated” to reduce the

maxi mum c oncentration,and re-calculating risks associated with the reduced concentration. The method may be repeated with consecutively lower truncation lim

its until ris

k is acc eptable. Land Method

- The conv

entional m

ethod for c alculating

uncertaint

y in the mean c oncentration (e.g., 95% UCL) w hen the sample data are obtained f

rom a log normal dis

tribution (U.S. EPA, 1992). Maxim um Detected Conce

ntration (MDC) - The maxim

um concentration detected in a sample. True Mea n Concentrat

ion- The ac

tual ave rage conc entration in a

n exposure unit. Even w

ith extens ive sam

pling, the tr

ue mean cannot be known. Only an estim ate of the true mean is possible. A g reater number of representative sam ples increase s confide

nce that the

estim ate of t he mea n more c losely

represe nts the true

mean.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-3 Two Office of Solid Waste an d Emerge ncy Response (O SWER) guidance documents in pre paration: (1) Draft Guidance on C

alculation of Upper C

onfidence Limits for Exp osure Point Concentratio ns at Superfund Sites (U.S. EPA, 2001a), and (2) Draft Guidance on S urface Soil Cleanup at Superfund Sites: Applying Cleanup Levels (U.S. EPA, 2001b), also addr ess topics rel ated to the calcu lation of EPCs and compari son of those EPCs to a PRG. In practice, calcula

tions of risks , given conc entration data, are com monly referred to as “f orward calcu latio ns”, whi le cal culat ions of PRGs, bas ed on cho sen tar get ris k levels, are ref erred to as “ba ckcalculat ions”. This term

inolog y refle cts the algebraic rea rrange ment of the s tandard

risk e quation

needed to solve for the concen

tration term when point estimates are u sed to characterize exp osure and toxicity in put variables. For pro

babilistic risk assessm

ent (PRA), the proce ss for developing a PRG can be more com plex. This chapter presents me thods and recom

mendations for de

veloping site- specific PRGs within the framework of PRA. Are there different types of PRGs? Generic PRGs hav

e been developed

for some chem

icals and exposure m

edia using point estimates based on standard default expos ure assumptions (e. g., U.S. EPA, 1991b) and toxicity criteria available in the In tegrated Risk Information Sy stem (IR IS) or Health

Effects Assessme

nt Summ ary Table(s) (HEAST) or from Environmen

tal Protection Agenc

y’s (EPA’s) National Center for Env ironmental Assessm

ent. Soil Screening Guidance levels, Region 9' s PRG table and R egion 3' s Risk Based C oncentrations (RBCs) tab le are examples o f gener ic point estimate PRG s. Gen

eric PR Gs are o ften use d for screening chem icals of potenti al conc ern in D ata Evaluation and H azard Id entification steps of the risk assessment proces s.

L There is a clear distinction between generic PR Gs, site-specific PRGs, remediation goals (

RGs), and clea nup levels. The focu s of this chapter is on site-specific PRGs. At this time, E

PA does not recom

mend the use of PRA to develop generic PRG s. Until the science a nd policy decisions asso ciated w

ith the use of d

efault a ssumptions in

PRA have ev

olved , generic PR

Gs shou ld only be developed from

point estimate m

ethods, as was done

in the examples

listed above. As indicated in Exhibit 5-1, site- specific PRGs ge nerally are developed after the baseline risk assessment. Howev er, dur ing the feasib ility study o r even later in the Su perfund proces s, the m

ethods describ ed in th

is chapter may

be used to m

odify cleanup lev

els at the discretion of the risk manager. However, it is generally not appropriate to use PRA

for modify

ing cleanup levels during the feasibility study if PRA

was not used in the baseline risk

assessm ent.

L Risk-based PRGs are initial guidelines and do not represent final cleanup levels. Only after appropriate analysis in the rem edial investigation/fea sibility study (R I/FS), consid

eration of public comm

ents, and issuance of the record of decision (RO D) does a RG b ecome a final clean up level. A cleanup level m

ay differ from

a RG because risk

managers may consid

er various uncertainties in the risk estimate. While the two main criteria for determining a cleanup level are: (1) protection of hum an health and the environment, and (2) comp liance with applicable or relev ant and appropriate requirem ents (ARARs), a cleanup level may differ f rom t he RG because

of mo difyin

g crite ria, suc h as fea sibility , perm

anence, state an

d com munit yRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-4 acceptance, and cost effectiveness. These and other factors are reflected in the nine evaluation criteria outlined in the NCP (U.S . EPA, 1990a; 40CFR §300.43 0(e)(9)(iii)) (see Chapter 1, E xhibit 1-2). This chapter and Append ix C pr ovide a com prehens ive de scription of the issues associa

ted with developing site- specific PRGs with b oth point estimate a nd probabilistic approaches, including the use of geosta tistics. Becau se methods fo r calcul ating a 95% up

per confidence

limit for the mean (95% UC L) are discussed fully in the Draft Guidance on C alculation of Upper C

onfidence Limits for Exp osure Point Concentrations at Super

fund Sites (U.S. EPA, 2001a) and Draft Guidance on S urface Soil Cleanup at Superfund Sites: A pplying Cleanu p Level s (U.S. EPA, 2001b) , they are cov ered on ly bri efly in

this g uidance . In

gener al, this cha pter, A ppendix C, and the Su perfund guida nce und er development should be c onsulte d by r isk as sessors when developing

site-specific PR

Gs. 

### 5.1 G ENERAL 

### C ONCE PTS 

### REGARDING 

### EPC S AND 

### PRG S PRGs d evelo ped fro m poi

nt estim ate risk assessm ents and PRAs will be discuss ed in th is sectio n to compare and co

ntrast the two approaches. The PRG is a spec ial case of the concentration term (or EPC) in the risk equation. The in

tent of the EPC is to

represent the average chemical concen tration in an environm

ental medium in an exposure unit (EU ) (i.e., the area throug

hout which a receptor moves for the duration of exposure). The EPC should be d

etermined for indiv

idual EUs within a site. Because an EPC is calculated from a sample, there is uncertainty that the sample m ean equals the true m

ean concentration within the EU; therefore, to acc ount for associated uncertainty,the 95% upper confidence limit for the m

ean (95% UCL) is generally used for Superfu nd risk assessm ents (U .S. EPA, 199 2). For both p oint est imate and pro babilist ic appro aches, the PRG

is an assu med v alue of the EPC

that y ields a risk e stima

te that i s at or b elow an

accepta ble risk level

.

L The EPC usually re

presents the average con centration within the EU estimated from a sample; the P

RG usually represents th e average concentration within the EU that corresponds to an acceptable level of risk. The PRG may be though t of as a goal for the pos t-remedia tion EP C (see Se ction 5.1.2). Specif icall y, after rem ediation is com pleted, the average concent ration ( or the 9 5% UCL

used as a me asure o f uncert

ainty in the average) for the EU should be sufficiently low to be protective of hum

an health and the ecosystem. While the methods used to

calculate the pre-

and post-rem

ediation EPC m

ay differ, the interpretation of the EPC rem

ains constant. For exam

ple, if the 95% UCL is used to represent the E PC before reme diation, then the EPC following remediation (e.g ., the PRG) sho uld also represent a 95% UCL (Bowers et al., 1996). Risk assessors m

ay consider both variability and u ncertainty in the dev

elopment of an EPC. The calculat ion of a 95% UC L generally

require s knowledg e of not only chem

ical con centrati on me asurem

ents wi thin the EU but also the receptor’s behavior. Relevant inform ation may include the variability in concentrations in the give n sam ple, th e sam

pling location s, and variab ility i n the m ovem ent and activity pa

tterns o f recept

ors with in the EU. A discu ssion of spatial and tem poral variability ass ociated with characterizin g contamin

ation in different exposu re me dia is presented

in App endix C , and i

mport ant sou rces of uncerta

inty i n the EPC are discuss ed in Section 5.1.1. For all risk a ssessm

ents, chemic al conc entration mea surements sho uld be c ollected

in a m anner that is consistent wit h an understandi ng of both the source of con tamination and the def inition of the exposure unit. An invest ment of time and re sources should be made in p lanning , scopi ng, a nd prob lem fo

rmula tion. P art of th

is investment is to follow the Data Q uality Objectives (DQ O) process to obtain sam ples appropriate for the riskRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-5 assessment and su

fficient to support the rem edial decision (U.S. E

PA, 1993, 199 4, 2000). Using new methods of sample collection an

d analysis such as dynamic w orkplans and real-

time analysis may enab le risk mana gers to get the most “bang

for the buck” from the resources a vailable for site characterization. Inform ation about these methods and the DQO process is available from EPA's Office of Emergency and Remedial Respo nse (U.S. EPA, 2001c) and Tech

nolog y I nnova tion Of fice (U. S. EPA , 2001 d, 200 1e). The world wide w eb addr

ess is http://clu-in.org

/char1_edu.cfm#syst_plan.
5.1.1 S OU RCES OF U NCERTAINTY I N THE EPC The 95% UCL is generally used as the EP C to rep

resent u ncertain ty in t he me an conc entration in bo th the central tendency exposure (CTE) and reasona ble maxim

um exposure (R

ME) risk estimate s for Superfund (U.S. EPA, 1 992). Similarly , in PRA, a p robability distribution for u ncertainty may

be used in a two- dimensional Monte Carlo analysis (2 -D MCA) simulation (see Ap pendix D) to represent a

source of uncertainty

in the EPC. There are numerous p otential sources of uncertainty in the estimate o f the true mean concentration within the E U. The sources of uncertainty when the EPC is expressed as either a sing le number or a distribution are the sam e and can be grouped into the following fo ur broad categories: (1) Uncertainty in the sample data. A limited number of measure ments in the sampl e are us ed to make inferences about the EPC and the spatial distribution of concentrations at a site. Uncertainties may arise from m

any factors, including

both sampling

variability and measureme nt error. As the nu

mber of sam ples inc reases, the unc ertainty

gener ally d ecrease s (e.g ., more informatio n will b e avai lable to characterize the spatial distribu tion and variation in concentration). I n point estimate risk assessments, the 9 5% UCL is generally used as the E PC to account for the uncertainty in estim ating the average co

ncentration within an EU

. (2) Uncertainty about the location of the EU. When the size o f a receptor’s EU is less than the size of the site, the placem

ent of the EU m

ay be a source of uncertainty, espec ially when the contamination is distributed unevenly

across the site and the PRA includes exposu re scenarios for future land uses. (3) Uncertainty in the beh avior of the receptor. Even in the case of extremely well characterized sites, it remains uncertain w

hether the receptor will con tact the environm

ental medium

in a temporal and/or spatial distribution that ca n be adequately re

presented by the e

nvironmental samples collected. (4) Uncertainty in chemical concen trations over time. The concentration in a given mediu m may undergo tem

poral changes, w

hich may introduce uncertainty in estimates of a lon

g-term average. Examples include the movem ent or attenuation of a so lvent plume in groundwater; aero bic or anaerob ic deg

radation ; the change i

n the av erage concent

ration i n a fish popula tion due to cha

nges i n population dynam

ics; and the mixin

g of surface and subsurface soil over time. A lack of knowledge in all four cat egorie s may

be con sidered

when s electing

approa ches to quantif

y uncertainty in the con

centration term. O

ne of the first steps in qu antifying uncertain

ty is to define the EU, or the geographical are a in which indiv idual receptors are random ly exposed for a relevant exposure du ration. Depending on the receptor’s mov ement and activ

ities, an EU m

ay be as sm

all as a child’s play area (e.g., sandbox) or as larg e as the foraging area of an upper troph

ic level anim

al predator (e.g., an entire military base). The relationship between the size of the EU, the movem ents of the target receptor, and health endpoin t of concern (i.e., acute or chronic) may dictate the appropriate use of sample data in developing an EPC. One of the assum ptions generally

made for the concen tration term i

n Super fund risk asse ssmen

t is that receptors conta

ct allRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-6 parts of an EU at ran

dom, and tha

t measurem

ents are obtained from

a simple (or stratified) random sam ple. If an individual is random

ly exposed within the same EU over a long period of time, the most appropriate metric for the EPC would b e the true (but unk nown) population m

ean of the concentrations w ithin the EU (e.g ., 95% UCL). Often, the scale of the EU will be different (sm aller or larger) than the scale of the samp le data. For example, an ec ological receptor popu lation may have a sm

all home rang e relative to the size of the entire site, or the endpoint of concern may be acute toxicity, requiring an evaluation of a short-term exposure scenario. If the receptors are not expected to contact all parts of the s ite with equal probability , then the EU m ay be redefined so that only a subset of the data collected for site characterization are used to estimate the EP C. In addition, the location of the E U ma y be u nspecif ied with in the s ite beca

use ther e may be m ultiple areas that prov

ide suit able habitat for the receptor popu lation. Departing from the assum ption of random exposure within one u nique geog raphic a rea pres ents an additional cha llenge to estim ating an EPC . In

some cases, it ma

y be i nformative to develop m

ultiple estimates of the EPC in a PRA . By treating the EPC as a ran

dom variab le, risk assessors can explore the effe ct of un

certaint y in the locati on of the EU. A va riety o f mod

eling approac

hes are availa

ble to calculate an EPC (e.g ., arithmetic m ean, or 95% UCL) based on the spatial v ariability in chem

ical concentrations measu red over an ar

ea larg er than the EU. Method s such a s geostatistics (see Se ction 5. 5.2 and

Appendix D), Microexposure Event Mo deling (MEE) (see A ppendix D), and ran

dom walk scenarios (Hope, 200 0, 2001) may be used to quantify

both the spatial and tem

poral variability in exposure to vary ing concentrations. Using these methods, risk

assessors may

redefine the EU to b e more representativ e of the random

movem ent of the receptor during the period o f exposure. Because these modeling approaches may

be considered m

ore advanced m

ethods for qua ntifyi ng the EPC, they a

re generally

consid ered in Tier 3 of the PRA proces s (see Chapter 2 ).
5.1.2 P RE - AND P OST -R EMEDIATION E XPOSURE P OINT C ONCENTRATIONS The differences between pre- and post-rem

ediation EPCs are discu ssed below. I n general, both estimates of the EPC are based

on the same concepts regarding the exposed population

and the definition of the

EU. Howev er, the post-remed

iation E PC wil l tend t

o yiel d lower

estimates of (post-

remed iation) risk a

nd can require more advan

ced methods for calc

ulating uncerta inty ( e.g., 95% UC L). The pre-rem

ediation EPC is determ

ined based on existing

site sampling

at the time of the remedial investigation, p

rior to remediation. By contrast, the post-rem ediation EPC ge

nerally is determ

ined based on a prediction of site conditions after remediation. F or example, in surface soil, the post- remediation EPC

can be deter mined by sub stitu ting the nondet ect le vel (gene rally, half th e labor atory re porti ng limit) for some o f the hi gh concent rations in the s ample

and rec alculati

ng the EPC. The und

erlyin g assu mption in cal culating

a postremediation EPC

is that remediation will have sufficiently reduced the chem

ical concentrations at the site, and the risk existing after remediation is complete will be eq ual to or less than the target risk level of concern. The preceding discussion is most applicable to surface soil PRGs. In g eneral, compared with other exposure media (e.g., gro undwater, air), surface so il is stationary with relativ ely constant chem

ical concentrations within an EU. F or other environm ental media, m

ore complex app

roaches may

be needed to estim

ate the post-rem ediation EPC. Modelin

g of the rem

ediation process ma

y introduce additional uncertainty not encountered in risk estimates based on the pre-rem ediation EPC.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-7 5.1.3 REMEDIATION A CTION L EVELS (RAL S ) AND 95% UCL C ALCULATION M ETHODS The EPC should incorpo

rate knowledg e about the spatial distribution of contamination, the behavior of the receptor, the location o f the EU, land use,and other factors. These factors affect both the num erical value of an EPC and unce

rtainty associated with this estimate. I n many cases, it is presum ed factors associated with lan d use will not chang e after remediation. The remediation action le vel (RAL) is the

maximu m concentration tha

t may be left in place at any location within an EU such that the average concentration (or 95% UC L as a measure of the averag e) will not present a risk abov

e levels of concern. This RAL m ay be considered a “not-to- exceed” threshold or action level for th e purpo ses of s ite re mediati on. Usin g surfac e soil as an exa mple, are as with in the E U that h ave concent rations greate

r than t he RAL may be exca vated and rep laced w

ith clea n fill (e .g., nondet

ect surr ogate values). To obtain a post-rem ediation EPC, the 9 5% UCL is calculated after substituting the surrogate nondetect value for all m

easurements located within the EU that are greater than the RA L. When ap propria te, the same statistic al method of uncerta inty should b e used t

o estim ate UCL s for both the pre- and post-rem

ediation EPCs. H owever, in som e instances, the m

ethod used for calculating the pre-reme diation EPC will be inappropriate for calculating the post-rem

ediation EPC, becaus e the distribution of contaminant concen

tration will have chan

ged. For exa

mple, pre-

remediation site sam

pling may

suggest that variab ility i

n conce ntration s can be

reasonably character

ized by a lo gnorm

al distri bution, which would s

upport the use of the Land method for estim ating the 95% UC L. The post-remediation site cond itions, however, may reflect a mixture of clean fill and contam ination, resulting in a

poor fit to a log

normal distribution (see Figur e 5- 3, Sect

ion 5.5 .3). I

n this c ase, the Land metho d would not be appropr

iate. B ecause of the d

ifferenc e in the stat istical d istribut ion of c oncentr

ation m easurem ents use d to est

imate the pre -rem

ediation EPC a nd post

remediation EPC , a non-p arametric (i.e., distribution free) method should be considered for calculating uncertainty in the av

erage concentrations in both pre- and post-rem ediation scenarios. I

n general, when

the method used to calculate the 95% UCL for a post-rem ediation scenario is different thanthat of the preremed iation s cenario , the 9 5% UCL for the pre-remed

iation s cenario

should be reca lculated with the postremed iation m ethod. Resul

ts of this change in methodol ogy can be presented as pa

rt of a q uantitat ive un certaint y analysis. Specifically , this recalculation will allow for an evaluation of the effect that a RAL has on the confidence interval for the mean. The d iscordance between pre-

and post-rem

ediation distributions can be expected to increase as the degree of rem ediation needed to achiev e a target risk level of concern incre ases. In general, risk assessors sh ould be aware of the practical and statistical issues associa ted with the variou s met

hods of calcula ting t he 95% UCL, an

d the ap plication of the se methods to both the pre-

and po stremediation concentratio n distribution. Different m ethods can yield v

ery different confidence intervals, som e of which are expected to yield more accurate coverag e (i.e., likelihood that the confidence interval includes the parameter) depending

on characteristics of the un derlying distribution

of concentrations, such as distribution shape, sample size, and variance (Gilbert, 1987; H all, 1988). I nformation about a variety of p arametric and nonparametric m

ethods, such as bootstrap resampling, can be found in The Lognormal D istribution in Environmenta l Applications (U.S. EPA, 1997), Estimating EPCs W hen the Distribution is N

either Normal nor L ognormal (Schulz and Griffin, 1999) and a Supe rfund guidance do cument currently under developm

ent, Draft Guidance on Calculation of Upper Confidence Limits for E xposure Point Concen trations at Superfund Sites (U.S. EPA, 2001a).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-8 5.1.4 C ON SIDER ATIO N OF RIS K FR OM A CUTE T OXICITY Sometimes

a risk assessm

ent will need to address more than one health endpoint of concern (e.g., cancer and noncancer). The R

AL should be sufficiently low so that it is sim

ultaneously protective of each endpoint of concern. Generally , when acute toxicity is a concern, the lon

g-term average con

centration across the entire EU may not be th e appro priat e metric for ass essin g risks. Fo r exampl e, a sin gle epis ode of a child i ngesti ng a handful of soil containing malathion m

ay result in an acu

te toxic effect to that child. Therefore, the RAL m ust not only be low

enough to redu

ce the post-rem

ediation EPC to acceptable long- term average levels, but also low enoug h that a cute toxicity w

ill not b e an iss

ue. This consid eration applies

to both point e stima

te and p robabil istic estimates of PRG s.

L For consideration of acute toxicity, the risk assessor should consult, as appropriate, with a to

xicologist in the developme nt of RALs.
For a sm all num ber of chemic als, tox

icity v alues ha ve bee n determ ined ba sed on acute ef

fects (e .g., nitrate in drinki ng wa ter). H owev er, at present, EPA d oes not have acute to xicity

criteria or gu idance on acut

e toxici ty applied to the R AL. H ence, c onsulta

tion wi th a toxicologist is v

ital.
5.1.5 C HA RAC TERIZ ATIO N OF U NCERTAINTY I N THE EPC: P OINT E STIMATES AND D ISTRIBUTIONS In point estim

ate risk assessm

ents, the 95% UCL is typically used to characterize uncertain ty in the EPC (U.S. EPA, 1 992). In P RA, ei ther a p oint est

imate (e.g. , 95% U CL) or a proba bility distribution m

ay be used to characterize uncertainty

in the concentration term . The probability distribu tion may characterize either variability or unc ertainty. The term s probability distribution for variability (PD Fv) and probability distribution for uncertainty (PDF

u) can be used to distin

guish between pro

bability distributions for v ariability and uncertainty, respectiv

ely. The decis ion to u se a poi nt estim ate, PD Fv, or PDFu , as the input f or the c oncentr

ation te rm in a Monte Carlo model w

ill depend on the goals of the Monte Carlo sim ulation, as determin

ed by the tiered p

rocess (see Chapte r 2). If the g oal is to

charact erize v ariabili ty in r isk, i n general, a one- dimen sional M onte Ca rlo anal ysis (1-D MCA) will be used and the appropriate input for the concentration term w ill be a point estim

ate that characte rizes u ncertain ty in t he me an conc entration withi n the E U. As explained in S ection 5 .1.1, risk a ssessor s will need to consider the relationship between the size of the EU, the movem ents of the target receptor, and health endpoint of concern (i.e., acute or chron

ic) to determine ho

w to use the available sample data to define the EPC. A PDFu is ty

pically not an appropr

iate cho ice for t

he conc entration term

in a 1-D MCA when the goal is to characte rize v ariabili ty in r isk. Mixing

of a PD Fu for t

he conc entration term

with P DFv’s

for other expo sure variab les in 1 -D M CA would yi eld a si ngle r isk di stributi on from

which the rela

tive c ontribu

tions of varia bility and unc ertainty

could n ot be ev aluated . Use of a PD Fu for t he conc entration term

may be con sidered

in 2-DMCA si

mulat ions (se e Appe

ndix D) , where

the g oal may be to chara cteriz e both v

ariabili ty and uncerta inty in ris k. When the sample s ize is small a nd the variance is large, the 95% UC L may exceed the maxim um detected concentration (MDC). In such a case, the MDC is generally used to estim ate the EPC, althoug

h the true mean m ay still be higher than this maxim um value (U.S. EPA, 1 992). For poorly characterized sites, the

re may be cons idera ble unc ertai nty that site r emediat ion wil l be suf ficie nt to re duce th e 95% UCL to a heal th-prot ective level. Poor sit e chara cteri zation may provide a n impetu s for th e risk mana ger to op t for a mor e healt h-prote ctive remedial alternative

or to collect additional data.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-9 To ensure that actual cleanup b ased on a RAL is protective gen erally requires post-

remediation confirmation sam

pling. This step in the risk man agement pro cess is emphasized further in Section 5.8 on measu remen t of atta inmen t. 5.1.6 M ULTIP LE C HEMICALS Developing PRGs for m

ultiple chemicals in o

ne or more env

ironmental m

edia is particularly challeng ing. When multiple chem

icals are present, the total risk level should be considered for regulatory purposes with each chemical contributing

a portion of the total risk . This issue is quite com plex and usually w

ill affect both the calculation of the risk and developm ent of site-specific PRGs. Chem icals may ex

hibit different spatial and temporal variab ility within the EU . Fate and transport characteristics may vary between chemicals as well as between different areas of the site. Co- located sampling , or geostatistical techn iques (e.g., co -kriging) may provide insights regarding relationsh ips in spatial patterns for differen t chemicals (see A

ppendices C and D ) and the corresponding ex

posures for receptors. 

### 5.2 W HEN TO 

### U SE 

### PRA FOR 

### D EVELOP ING 

### PRG S Becaus e point estimate risk assessm ents and PRA e mploy differe nt appro aches to

charact erize v ariabili ty and uncertainty, the

resulting RME risk

estimates and calcu

lations of PRGs are o ften different. The ma gnitude of the diff erence c an depe

nd on m any fa ctors, i ncludin g the numb er of in

put va riables describ ed with

probab ility distributions in the PRA , the choice of distribution s used to characterize variability or unc ertainty (especially fo r those variables that are highly rank ed in a sensitivity

analysis), the perce

ntile of the probability distribution that corresp onds with RME

point e stima

te for ea ch inpu t varia ble, an d the ch oice of percent

ile from the PR A used to represent the RME risk

(e.g., 95 th percentile). Since the re sults of a point estim ate approach and PR

A can be expected to differ, but the magnitude

of the difference is not k nown a priori , this c an pres ent a challeng e in decidin g whe ther or not to c onduct a PRA to develop a P RG. The poten tial adv antag es and d

isadva ntage s of bot

h the poin t estim ate approach a

nd the P RA can

be fact ored int o the de cision ( see Cha

pter 1, Exhibi

ts 1- 6 and 1 -7). In general, PRA m

ay be appropriate fo

r developing site-specific PRG s in cases where PR A has also been used to estimate site- specific risks. As indicated by the tiered approach (see Chapter 2), if the risk m anager determines that quan

tifying va riability and uncertainty

may en hance risk m

anagement decision mak

ing, PRA m

ay be warranted. I f a PRA is feasible, the risk man ager should proceed

to Tier 2 and em

ploy PRA to complete the RI/F S proce ss. Us

ually, embe dded in

a site- specific

PRG a re all of the exp osure a ssumptions an d toxici ty metrics used in the

risk assessm

ent. Hence, introdu

cing the use of P RA for PRGs in the feasibility study (or any time after the rem

edial investigation and baseline risk asse ssment are com plete) would, in effect, undermine the tiered approach.

L If only point estimates w ere used in the risk asse

ssment, probabilistic meth ods should not be used fo r PRG developmen t. If additional data have been collected to conduct PRA, the point estimate risk assessment shou

ld be revisi ted with the new data as well. As dis cussed in Chapter 2, a point e stima

te risk assessm ent (Tier 1) shou

ld always accom

pany a PRA . PRA is intende d to enhance risk managem ent decision mak

ing, and should

not be viewed as a sub stitute for point estimate approaches. Using the tiered approach, a risk assessor can determ ine the appropriate level of complexity that is supported by the available inform ation to conduct the risk

assessment and to calculate a PRG.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-10 Eq uation 5-1

Eq uation 5-2 

### 5.3 M ETHODS FOR 

### D EVELOP ING 

### PRG S Risk assessors may use PRA to quantif y sources of uncerta inty and variabilit y in the calculatio n of PRGs as well as risks. Two of the com mon methods for calculating PRGs in PRA include: (1) backcalculatio n (see Section 5.4), which i s equiv

alent in concept to the

point e stima

te calcu lation o f a PRG ; and (2 ) iterati ve for ward calculation methods, including iterative reduction and iterative

truncation (see Section 5. 5). Backcalculation can be used in PRA when the target risk and concentration terms are expressed as point estimates. I terative methods can be more involved, but unlike backcalc ulation, ther e are no constrai nts on their appl ication to PRA. The t wo approac hes yi eld the same result w hen the same assum ptions a re used in the r isk as sessment. 

### 5.4 B ACKCALCULAT ION Traditionally, risk is c alculated as a function of multiple exposure v ariables, including the

concentration term, and tox

icity v alue (E quation

5-1 ). I f one or

more of the e xposure varia bles is describ ed by

a PDF , a Mont

e Carlo simulation w

ill yield a distribution for risk (see Chapter 1). Backcalculation m

ethods can be env

isioned as setting a target risk level (e.g., RME risk equal to 10 -6 or Hazar d Index equ

al to 1)and the

n alge braicall y reversing

the risk equation to sol

ve for the con centrati on term (Equation 5-2 ). A Monte Carlo sim ulation using Equa

tion 5-2 will yield a distribution of concentrations that reflects the combination

of distributions from

all other exposure variables.

where,
Toxicit y = toxicity term re presenting either the ca

ncer slope factor (CSF) or reference dose (1/RfD) for the chem

ical in the exposure m

edium C = concent

ration term V = algebraic com bination of the

toxicit y term with al l exposure va

riables except C IR = ingest

ion or i nhalatio n rate AT = aver agin g time BW = body weig ht ED = exposure duration EF = exposure frequency This calculation produces a distribu tion of PRGs that rep

resents the same so

urces of variability

as a forward calculation of risk . Each percentile of the PRG distribution (i.e. , the " percentile) corresponds to the 1- " percentile from the distribution of risk estim ates. For exam

ple, if the 95 th percentile of the distribution of risk estimates was cho sen to represent the RME individual, the 5 th percentile (1-0. 95=0.05) would be the corresponding conce

ntration value from

the distribution of PRG s (Bowers, 1999). The correspondence between the risk distribution and

the PRG distribution is intuitive—just as selecting a higher percentile on the risk distr ibuti on is more protec tive, a l ower pe rcent ile on t he PRG di strib ution is more pr otect ive. The RME rangeRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-11 for the risk distribution 90 th to 99.9 th percentile is analogou s to an RME rang e for the PRG distribu tion of 0.1 st to 10 th percentile. Backc alc ula tio n has been a fami lia r meth od of devel opin g PRGs and ma y be ap prop ria te in some situations for the sake of clarity and transparen cy due to the general understanding of this method a mong risk assessment practitioners. Once a back calculation has been perform ed to determine a

PRG, the PR

G should be used as the concentration term in a forward calculation to ensure that the risk at the PRG is acceptable. 

### 5.4.1 D IFF ICULT IES WITH 

### B ACKCALCULAT ION There are limitations in the use of backcalculation in PRA (Ferson, 1996). Sim ple rearrangem

ent of Equation 5- 1 does not suff ice whe n the variable (i.e., the concentrati on or ri sk ter m) that is backcalc

ulated i s represented by a p

robability distribution (Bu rmaster et al., 19 95; Ferson, 1996). The difficulty for PRA arises because each risk e stimate from an MCA that uses the familiar “forward- facing” risk equ ation represents a combination of ran

dom value s selected from the input distributions. Therefore, the output can be consid ered conditional on all of the in puts. Rearrangin

g the risk eq

uation does not m

aintain the same co

nditional probabi lities; t

herefore , the d

istribut ion for risk e stima

ted as a function of the

distribution for conce

ntration in Equation 5- 1 does not retu rn the s ame d

istribut ion for concent

ration w hen app lied in Equation 5- 2. Whil e there are techniques that can m aintain the dependencies an d correlations between exp osure factors when the risk equation is rearranged (e.g., deconv olution), they are c omplex and be yond the scope of this guidance. Backcalculation m

ethods may

also be difficult to im

plement in situations in which comp lex fate-and-transport considerations are prese nt. Leaching o f soil contamination to groundwater, bioconcentration of chemicals at hig

her trophic levels, an

d other multim

edia processes that result in exposure via sev eral enviro nmen tal me dia are situations in w hich ba ckcalc ulation may not be useful. Note t

hat thes e diffic

ulties a re not unique to backcal culation. Uncert ainty in fate-and-transport consideratio ns makes any type of PRG deter minatio n chall enging. Further, the backc alculation approach only provides inform

ation on the EPC that corresponds to a risk level of conc ern; it d

oes not specify an RA L that w ould achieve

this EP C. For exam

ple, when a ris k equation is algebraically solv

ed for concentration (see E quation 5-2), a PRG is dev eloped without a correspo nding RAL. Thus, there is no inform

ation associated with the P RG value to in dicate the highest con

centration in the EU tha

t must be rem

oved so that the average conce ntration (or 95% UCL) w ithin the EU is at or b elow the PRG. Hence, additional efforts are needed. In addition, post-rem ediation concentrations m ay need to satisfy

more than one regulatory constra

int. For exam

ple, the averag e (or 95% UCL) conc entration within an EU may need to be less than a concentration associated with chronic toxicity

or cancer and sim

ultaneously, the R

AL concentration m

ay need to be less than a concentration that m ight cause acute toxicity. In spite of the se caveats, back calculation methods may be appropriate for some sites. For exam

ple, when the target risk

is spec ified by

a single num

erical v alue and the ri sk m

anage r has chosen a p ercentil e of v ariabili ty to represent t he RME individual, the n a backcalculated PRG can be derived from a PRA. Although back

calculation methods may be appropriate for some sites, risk assessors should be familiar with their limitations. Because of these lim itations, this guidance recommen ds iterative forward calc

ulations as the primary method for calculatin g PRGs when performing a PRA. I terative methods avoid difficulties associa ted with apply ing MCA to a b ackca

lculation, and can provide m ore informatio

n for the risk

mana ger.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-12 Fig ure 5-1 . A hyp othetic al exam ple o f the use o f iterative m ethod s to determine the EPC that corresponds with a target RME H azard Index (HI) of 1.0. Assume that t

he EPC is represented by t he 95% UCL and the RME H I is the 95 th

percentile of t

he output distri

bution. In t

his case, f

our separate Monte Carlo simulations were run wit h iterat ively decreasi ng values for the E PC . The least-squ ares, b est-fit line to these four d ata poin ts sugge st that a rea sona ble P RG would be ap pro ximate ly 660 ppm
. 

### 5.5 I TERATIVE 

### M ETHODS Iterati ve methods simply invo lve calcu latin g risk wit h the “f orward -facing” equation (see Equation 5 -1) a num ber of times (iteratively

) using prog

ressively lower values for the conce

ntration term until the risk is suffi cient ly prote ctive. This i terat ive method has also been called the “repeated runs” method. Note that iterative m ethods for calc ulating a PRG are not unique ly applicable to PRA. Iterative m ethods also may be used to develop PRGs in point estimate risk as sessments.

L EPA recommends iterative simulations as a g eneral approach for calculating PRGs fro m probabilistic risk assessments. Most ofte n, itera tive fo rward calcu latio ns are p erfor med using a systematic trial- and-error m ethod until the percent ile of v ariabili ty in r isk chosen to represent the RME indiv

idual is at or below accepta ble risk level s. Som

etime s, a short cut can be used t o reduc e the nu mber of si mulatio ns need ed with the tri al-and-er ror meth od. If succ essive “guesses” of the E

PC are plotted with the corresponding risk estimate, the exa

ct solution can be determ ined from the best-fit line, thereby significantly reducing the effort required to implem ent this method. An example is given in Figure 5- 1. For man

y risk equation s, the relationship between the EPC and the RME risk will be approximately linear. Nevertheless, the final estimate of the EPC should be checked by running another simulation for risk

with this estimate. A possible and sig

nificant advantag e of iterative forward calculations over back -calculations is that the method is intuitiv

e and yields a d

istribution of risks rather than a distribution of PR Gs (as with a back

-calculation method). The distribu

tion of risks will be

more fam

iliar to the public and othe r stakeholders, and thus, both the method and the

resulting output m

ay be easier to communicate to senior level m anagers and stak

eholders (see Chapte r 6). Two general typ es of iterative m ethods are described in m ore detail in Sections 5.5 .1 and 5.5.2. The main difference between the methods is in the interpretation of the concentration term that is being reduced. With i terat ive redu ction , the co ncent ratio n is ass umed to be the pos t-remedia tion EP C, where as with itera tive truncation, i t represents the RAL needed to achieve a post -remediation EPC.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-13 5.5.1 I TERATIVE REDU CTI ON Iterative redu

ction can be applied to any mediu m. Generally , a point estimate representing the EP C (e.g., 95% UCL) is successiv

ely lowered, each

time repeating

the Monte Carlo sim

ulation of variability

in risk. When the EPC is reduced until the endpoint of con cern (e.g., R

ME risk corresponding

to the 95 th percentile) is at or below an acceptable lev

el of risk, the PR

G is set at the correspon

ding EPC. The goal is to identify the point estimate that correspon

ds to a target risk

level. Note that the PRG is not the same as the RAL. The RAL is the maximum concentration that may be left in place within an EU to ach ieve the PRG. The conc ent rat ion a t whi ch th e ris k is ac cept abl e def ine s the PRG. T here for e, th e PRG b ear s the same uncertainties as the EPC. For example , assume that a risk assessor exam

ined the carcinogenic effects from chronic consum ption o f a chem

ical in g roundwater, then the exposure unit m ay be determ

ined by the lon g- term average conce

ntration at any well that potentially draws drinking wate r from the contam

inated groundwater. Uncertainty in the lo

ng-term average con

centration can reflect a num ber of factors that contribute to spatial and temporal va

riability , inclu

ding t he direction of groun

dwater flow, n

atural a ttenuati on, and

other f ate and transport variables. Rem

ediation by a pum

p-and-treat system fo

r a prolonged perio d of time m ay be used to lo

wer the concentrations at the wells. Even thoug h the remediation strategy m ay be com

plicated by spatial and

temporal variability, iterative reduction can be used to establish a PRG. A remediation strategy m

ay be considered a potential candidate if it can ac hieve the PRG

by reducing

the average concentration at each of the well locations. The concept of “hot- spot” removal, or truncation of the h ighest concentrations first, would not be an option under this sce nario (s ee Sect

ion 5.5 .2).
5.5.2 I TERATIVE T RUN CAT ION Iterative trunc ation is a method of calculating a P RG that involv es developing

an expression for the concentration term in

which higher v

alues of concentration are rem oved or “truncated” to reduce the ma ximum concentration. These hig her values are replaced

by the surrog ate nondetect value. The risk is recalculated for each successive redu

ction in the highest value. The m ethod is repeated with co nsecutively lower truncation limits until risk is acceptable.
Itera tive tr uncation is m

ost app licable to surfa

ce soil cleanup

as the s patial v ariabili ty over time is minimal compared to other media (e.g ., surface water). With each iteration of the risk equation (e.g., Equation 5-1 ), the highest con centration value is trunc ated corresponding to a different RAL. In this way a“not-to- exceed” level is spec ified and the PRG is recalculated the same way in each iteration . The process continues until the risk

distribution yields risk

estimates at or be low the level of c oncern. Iterative trunc ation can be applied to either the empirical distrib ution function (EDF) for the concentration term, or a fitted distribution for v ariability in concentrations within the EU. A pplied to the EDF, the maximu

m det ected co ncentra tion wi thin the EU is r eplaced with a surrog ate va

lue for a nonde

tect (e. g., half the reporting lim

it or the backg

round value for some chem icals), and the EPC (e.g., 95% UC L) is recalculated for this alte red data set. If this new EPC

yield s unacc eptable

risk, then the two h

ighest detecte d conce

ntration s are replaced by the no

ndetect value and the EPC is recalculated. In the th ird iteration, the three hig hest detections are replaced, and so on, until the target risk level is achieved. Alternatively , the sample d ata may b e fit to a probability distribution for variability, and the process would be repeated with decreasing values in the hig

h-end tail of the continuous distribu tion.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-14 When the concentration term is a distribution representing uncertainty in the m

ean concentration, then, similar to the recalculation of the point estim

ate 95% UCL described above, this distribution of uncertainty in the mean concentration should be determ ined anew each tim

e a datum is rep laced with the nondetect value. When a distribution of variability in concentration is used for the EPC,for example, in an ecological risk assessment where sampling m ay be sparse relativ

e to the foraging

area of a small home rang e receptor (see Append ix C), then the

distribution d

evelo ped in a n identi cal way

with the high

value s replac ed by

the sur rogate nondetect value sho uld be used in the it erative determinat ion of a PRG. The decision to apply iterative truncation should be made after considering a variety of characteristics of the sam ple data and po st-re

media tion sce nario (s ee Exhi

bit 5- 3). For exam

ple, sm all sam ple siz e may

result i n high uncertainty

in the 95% UCL, thereby limiting the use of iterative truncation . Quantitative criteria regarding these factors are not prov ided in this guidanc e given that the level of ce

rtainty require d for de cision m

aking will vary on a case-by- case basis. Use of geostatistical methods (Appen

dices C and D) m

ay aid in interpreting site dat a or im provin g sam pling design . Geostatistics is capable of describing

the spa tial dist ribution of a contaminant in a quantitative fashion. These methods establish a correspondence b etween the actual sam pling locations and the locations a receptor would be ex pected to frequent. Additionally , it enables the estim ation of concentrations in unsam

pled locations. Hence,for determ ination of conc entratio

ns at sp ecific lo cations

at a site or within EUs of various sizes an d shapes, geostatistics may

provide an inv

aluable tool. Geostatistics has applications b oth to developing the EPC and P RG and has b een r ecomme nded and u sed a t some sites for characterization of soil and groundwater contamination (U. S. EPA, 1990 b, 1991c). Although the con sideration and use of geostatistics is encouraged, a full consideration o f geostatistics is bey

ond the scope of this guidance. Those interested in greater detail than provided in A ppendices C and D are urged

to consult the Superfun

d guidance docum ent curr ently under d

evelo pmen t, Draft Guidance on Surface Soil Cleanu

p at Superfund Sites: A

pplying Cleanup Levels (U.S. EPA, 2 001b) , for additional discuss ion of how ge ostatist ics can be used

to quan tify the concentrat ion term or the PRG. General ly, it

erativ e trunca

tion m ethods fail to produce adequa te clean up strat egies when si

te E XHIBI T 5-3
C RIT ERI A FOR I TERATIVE T RUNCATION
1. Sample size ( n ) is sufficie nt. Sma ll sampl e si zes l ead t o lar ge es ti mates of unce rtainty in the c onc entratio n term

. Small samp le size may cause the risk asse ssor to ove rlook

some s ources of unce rtainty.
2. Co nce ntra tion d istribu tion is n ot hig hly skew ed. A hig hly

skew ed dis tri

buti on may y iel d unre li able est imat es of unce rtainty, esp ecially for small sam ple sizes.
3. Sampli ng desi gn yiel ds a rep resen tati ve dis tri buti on of meas urement s withi n the exposure unit. Simple random sampling may fai l to r epresent

a patchy spati al distribu tion of c ontam inants. Sim ilarly, hotsp ot (e.g., c luster) sam pling m ay fail to repr esent ra ndo m mo vem ent of re cep tors. To evaluate potential biases i n sampling, analy ses wi th both standar d stat ist ical methods and geos tat istical meth ods maybe requi red.
4. Assumpti ons about the pos t-re medial distri bution of concentrati on are reasonable. If thes e assumptions are shown to be incorrect by subsequent sampl ing event s, the proces s f or developing a PRG may need to be repeate d and addi ti onal remedi al a cti vit ies may be requi red.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-15 characterization is incom plete. This problem, however, is n ot specific to PRA. Both point estim ate and probabilistic methods are sensitive to poor site characterization. Risk ass essor s shoul d real ize that applic ation of iter ative t runca tion may re sult i n areas on-site t hat have concentrations higher th

an the PRG. This is because the PRG w ill reflect an averag e concentration (or 95% UCL) from

a distribution of concentration s in which the m aximum is truncated at the RAL . For exam ple, Figur e 5- 3 (see S ection 5 .5.3) shows how the concent ration d istribut ion can be trun cated at an RA L, whi le still leaving behind concent rations greater than the PRG.
5.5.3 E XAM PLE O F I TERATIVE M ETHODS The iterative truncation m ethod is easiest to think

about with regard

to soil cleanup when contaminated soil is removed and replaced with c lean fill soil. This replacem ent would reduce both

the mean and

95% UCL. In m ost cases, risk assess ors may a ssume that the co ncentrations of chem

icals in clean fill soil can be represented by the surrog ate nondetect value (e.g., half the detection limit). A lternatively, the fill may be sampled so that the measured concentratio

ns in the fill dirt m

ay be used to c alculate the post-re mediation concentration

term. Generally, m

etals and other inorga

nic chemicals will be present in clean fill, alb eit at lower concentrations than on site. A sim

ple exa mple using

the 95% UCL a s a poin

t estim ate for t he EPC is giv en in Exhibit 5 -4. In t

his example, back

ground concentrations

of chemical X we

re very low

and hence, the fill was assumed to h ave a concentration of half the de tection limit. The risk

managem

ent objective is to identify a PRG in which

the 95 th percent ile risk estimate is b elow 1E -04 and to d eterm ine the RAL ne cessary

to achi eve th

is PRG . This examp le illustrates how iterative trun cation is applied to the em pirical distribution function, rather than fitting the concentrations to a param etric distribution. Assume that iterativ

e reduction of the 95% UCL demonstrated that a post-rem

ediation EPC of no greater than 33 mg

/kg is needed

to achieve a R

ME risk of 1E- 04. What is the RA L that yields this E

PC? The risk assessor recogniz es that the post-remediation concentratio n distribution is very

often a mixed distribution, consist ing of a gro up of nondetect values and a truncat ed para metric distribution. Becaus e of the comp lex natu

re of mixed dis tributions (Roe der, 19 94), non-pa ramet ric me thods fo r calcul ating the 95% UCL of the arit hmeti c mean (e.g ., bootstrap resam pling) were determ

ined to be appropriate (U .S. EPA, 199 7; Section 5.1.3).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-16 Figure 5- 2. Log norm al pro bab ility plot of so il concen trations, including 4 nond etects. Table 5- 1. Soil sample (n=70) (mg/kg). E XHIBI T 5-4
E XAMPLE OF I TERATIVE M ETHODS Scoping an d Problem For mulati on Chromium contami nati on was prese nt at a 12-acre industr ial faci lit y. I n scoping and problem formulation, all stakeholde rs agreed that the facility would ma intain itself and the curre nt land use would continue int o the f oreseeabl e fut

ure. Most of

the f acil ity area was maintained as gree n space and as a buffer wit

h the surrounding community. Surrounding the facil ity t o the fence li ne were lawns and orn ament al s hrubs ten ded by l andsc ape work ers. These l andsc ape work ers were c onsi dered t o be the hig h ri sk gr oup as t hey w ould mov e fr eely and randomly

over t he entire area of t

he f acil ity out side the buildings. Hence, the landscape workers would be exposed to an average concentration over the entire area of t

he facili ty outsi de the buildings. The managem ent of the f acilit y was very cooperative and concerned ab out their workers. Nonetheless, the f acilit y management did not wish to bear more cost tha n nece ssary.
Site Characteri

zati on - Soi l Sampl e (n=70) Seventy surface s oil samples were obtained using a sampling grid placed over all 12 acres. Five or six sampling l ocations were placed in each acre. None of the samples was composited. The grid-based sampling permits a r ough estimation of t he percentage of the si te that would need acti ve remediation. The detection li mit for t he chromium was 1 mg/kg. Four of the samples were nond etects. S amp ling results a re sho wn in T able 5 -1. Altho ugh the samp les from the site ap pea red to occur ina lognormaldist ribut ion (Fi gure 5-2) , th e presumed post -remedi ation dist ribut ion would be a mixed distributi on, consisti ng of a truncated l ognormal distributi on and a group of data at the surrogat e nondetect

value.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-17 In this exam

ple, a series of iterative

truncations showed that removal of all sample results g reater than 100 mg/k g (n=11) and repla

cement of these w

ith the nondetect surrog ate of 0.5 mg /kg yield ed a 95% UCL of 33 mg/k g and RME risk

below 1E- 04. Table 5-2 summa rizes the results of the calculations for the three conditi ons: (1)pre-remed iation c oncentr

ations; (2) post-rem

ediation concen trations using iterativ e trunca tion to achieve an RA

L of 10 0 mg

/kg; and (3)post-

remed iation c oncentr

ations a ssumi

ng the 95% UC L calcu

lated is used as the RAL. Note that if the PR G of 33 mg

/kg was ap

plied as a “not-to-exceed” lev el (i.e., RAL), the resultin g rem ediation effort w ould increase f

rom 1 5 to 40 % of the site, y ielding a 95% U CL of 1 4 mg

/kg. While this wo uld be a protect ive de cision, other i

nformation w as used

to supp ort the selection of the second scenari o inste ad. A toxi cologi st was c onsul ted, who indica ted tha t acut e expos ure to t he worker s at leve ls of 10 0 mg/kg would not present a health risk. To build a dditional protectiveness in to the remedy , the management also indicated scheduling fo

r the landscape work

ers would be perform

ed so the areas tended would be rotated am ong all the workers.

Table 5- 2. Pre- and P ost-Reme diation EP Cs (95 % U CLs) for C hemical X in Surface So il Samples.
Rem ediatio n Sce nario RAL (mg/kg) EPC (mg/kg) 95% UCL Percent of Si te t o be Remediated 1. Pre -re mediation NA 93 NA 2. Post- remediat ion usi ng the PRG as the 95% UCL 100 33 15%

3. Post- remediat ion usi ng the PRG as the RAL (i. e., “ not-t o-exceed”) 33 14 40%

NA=not a pplicable f

or a pre-re

mediation scenario. Figur e 5- 3 show s a conc eptual f ramew ork for consid

ering the pos t-rem

ediation distribution as

a mix ture between a group

of nondetects and a distribution of contam ination truncated at the RA L. Prior to rem

ediation, the EPC exceed s a level that w

ould be protect ive of huma n health and eco system

s. I f the hi gh- end soi l concent rations are rem

oved and the

soil is replaced

with cl ean fill,the resulting distrib ution w ill be b imoda

l, with one pea k occu rring at the nondetect conce ntration , and t

he second occu

rring near the

mean of the p ostremediation distribution.
5.5.4 M ULTIP LE E XPOSURE UNI TS AND I TERATIVE M ETHODS When multiple EU s are present at the site, there may b e a small num ber of samples w

ithin a given EU and the uncertainty in the concentration term generally w

ill be large. I

t may be possible to use kn owledge of the mechanism

of how the contam

ination occurred along

with spatial patterns in the sampling resu lts in other nearby EUs to quantify uncertainty. Geo statistical techniques for estim ating the mean concentration m ay provide u seful insights into the im

portance of accounting

for spatial relationships am ong the sam

ple data. Appendix C also provides a discussion

of the situation of m

ultiple EUs within a larger site.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-18 Figure 5- 3. Hypotheti cal exampl e of a mix ed, bimodal di stri bution t hat re present s a combinat ion of the preremediation distri

bution truncated at the remediation acti on level (RAL) and a uniform distri bution representing clean fil l at t

he surrogate nondetect concentrati on. Shaded po rtions represent equal areas. In t

his example, the PRG is defined by t

he post-remediation EPC (95% UCL ).


### 5.6 PRG S FOR 

### G ROUNDW ATER For some chem

icals encountered at hazardous waste sites, chem ical-specific ARA

Rs may exist, and may be considered as PRG s. ARARs m ay be selected as site-specific cleanup levels. The m aximum contaminant levels of the Safe D

rinking Water Act are examples o f ARARs.

L For groundwater con tamination, ARARs should be applied as RALs if they are protective. Of course, for cases in which an ARAR is less protective than

a remediation goal determined from a risk assessment, then a risk-bas ed PRG m ay be deve loped in accordance with the NCP (U.S . EPA, 1990a). As an exposure m

edium, g roundwater is the oppos ite of soil in that gro undwater is not static, an d receptors are usually ex

posed at one location (i. e., the well head). Often, a single well can be considered the EU when assessing risk

s associated with either the residential or industrial/occupational scenarios. The EPC m ayRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-19 still ref lect the concept of averagin

g over a lon

g tim e period (e.g. , yea rs) due to poten

tial cha nges i n concentrations in well wa ter over time. For example , chemical fate an d transport modelin

g may suggest that concentrations are decreasing over time. Similarly, there may be temporal and

spatial variability depending on the seasonal fluctuations of the water table. Ideally, the risk assessment wo

uld focus on indiv

iduals who ma

y be exposed at locations nearest to the center of the contam inant plume, w

here concentrations are lik ely to be hig

hest (Freeze and Cherry , 1979; Sposito, et al., 1986). Because of the uncertainty in the mov

ement of g

roundwater and the nec essity of sam pling the m

edium at fixed locati ons, identifying a meaningful RAL need ed to achieve a given PRG is diffic ult. In most cases, ARARs will be applicable as RA

Ls or “not-to -exceed” lev els.


### 5.7 PRG S FOR 

### O THER 

### C ONTAMINATE D 

### M EDIA
Itera tive tr uncation techn

iques ar e gene rally a pplied t o a stat ic me dium, such as

soil, r ather than dy

namic or fluid medi a such a s water

and air . This is

simp ly bec ause it is diffic ult to d esign a met

hod tha t will se lectiv ely remove high concen trati ons fro m a fluid med ium. Iterat ive redu ction may be more rel evant th an ite rative truncati on whe

n an RA L cannot be dev

eloped. These is sues are

discuss ed below with respect to sedim ent, surface water, and fish.
Sediment Sediment m

ay be transported over time m ore readily than soils. If it can be assumed that the sediment remains in place, then iterative truncation tec hniques may

be applied. However, at som e sites, sediment may be considered a fluid m

edium. For example, sedim ent may b e resuspended by the movem ent of water craft, wav

es, changing tides,or erosion. Sim ilarly, the depth of the contaminated se diment m ay change over time as

new layers of sedim

ent are deposited abov

e more contam

inated sediment. Exhibit 5-5 gives an exa

mple of the use of iterative truncation to evaluate alternative RALs for sedim ent of a lake contam

inated by pesticide ru

noff. In th

is example, the RAL is related to both the ecological end point of concern (i.e., reduction in reproductive succes s of mam malian om

nivores at the lak

e) and the fraction of areal extent of the lake that would require rem ediation at that RAL.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-20 E XHIBI T 5-5
E VALUATI ON OF A LTERNATIVE RAL S U SING I TERATIVE T RUNCATIONRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-21 Biota ( Fish, Aquati

c Inver tebrate s, Plan ts) Biota, such as fish, aquatic invertebrates, an d plants can serve as bioindicators or indirect estim ators of contam ination in othe r expos ure media tha

t would be targets for r

emed iation. The con

centrati on of chemica

ls fish may refle ct a combination of exposures via se diment, the water column, and food source (e.g ., prey). Therefore , the u se of bi oindica tors to devel op PRG s in other me dia intr oduces a sourc es of un

certaint y. I f there is a high correlation between concentrations in fish and sediment, then sediment c oncentrations may be considered when dev

eloping PRGs to protect the receptor popu lation. The EU, in th

is case, is the area where the angler population, or ecological predator population, harvests fish. Howeve r, in risk assessm

ents that include a fish ingestion exposu

re pathway, there may be high uncertainty about the true concentration term. Concentrations ma

y be affected by

many factors, including chan ges in the fish po

pulation and chang es in fish prefe rence s, whic h may be diff icult to addr ess in r isk asse ssments . The ch oice of fish sp ecies consumed by a given indi vidu al may a lso affe ct th e con cent rat ion t erm. Fish population studies an d fate and transport consid erations of the contam

inants may indicate if and when a fish po pulation will r

each a c alculate d clean

up level. For man

y site s, it m

ay be difficul

t to obt ain this level of site- specific data due to resource and time constra

ints. Although rem

ediation may

not imm ediately reduce contam

inant concentrations in biota, the determ ination of a cle anup le

vel ca n serve as a ta

rget fo r any future d

ecline i n conce ntration s. I

n general, iterative reduction m

ethods are applicable for de veloping PR

Gs to protect aquatic eco systems; however, under some conditi ons iter

ative truncati on ma

y also be used . For examp le, if c ontam ination is corre lated to

relatively static sediment, and

the home-

range of the fish is relatively sm all (e.g., nonm

igratory) then iterativ

e truncation may be applicable. Surface Water The development of P RGs for surfac e water is alsodifficu

lt with iterativ

e trunca tion. F or fluid

medi a (e.g., gro undwater or surface wate r), iterative reduction can be performed u sing a range of EPCs to determ ine a PRG with acceptable risk at the target RME percentile.


### 5.8 M EASUR EMEN T OF 

### A TTAI NMENT The NCP (U.S. EPA, 1990a) provid es for c ontinue d mon

itoring for g roundwater cle anups t

o ensur e attainm ent of t he rem edial ac tion ob jectives . In

addition, it is

comm on practice am

ong re media l project mana

gers to conduct confirm

ation sampling

after completing

a remedy for soil contamina

tion. However, completion o f the remedial action accord

ing to this strateg

y does not neces sarily mean that risks within EU s at the site have been reduced to levels spec ified in

the RO D. The degre e of unc ertainty about w hether t he rem edial action at the site has ach ieved the cle anup le

vel should determin

e whether confi

rmatio n sam pling is warra

nted. In g eneral, confirm ation sa mplin g foll owing

cleanu p activ ities is recom

mend ed. Sa mplin g afte r the rem edial in vestigation is complete m

ay show addition

al areas needing rem

ediation (i.e., where a

dditional contamination

exists). If additional sam

pling is conducted after the remedial inv estigation, the concen

tration term and corresponding estim

ates of risk should be recalculated. The PRG developed in the

remedial investigation may not be health-p

rotective in light of the additional contam ination. The same concepts that relate the conc entration term to the PR

G should be applied in this situation.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-22 Confirmation sam

pling activities are included in remed ial design/remed

ial action plans to ensure the remedy is successful. In

addition, the five-

year review presents a second opportunity to ensure that any contamination left on

site does not pose an

unacceptable risk.

L If confirmation sampling indicates an insufficient reductio n in risk, a more extensive remediation effort m ay be needed. Po ssible reasons for not ac

hieving remedial action objectives can include inadequate s ite characterization or the discovery of unknown contamination. For post-rem

ediation sampling , the DQO proce ss should generally be followed. I f the post-rem ediation risk associated with the confirmation sam ple indicates risk excee ds a level of con cern, then additional remediation m

ay be warranted.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-23 

### 5.9 S UMM ARY O F 

### RECOMMENDED 

### M ETHODS Tab le 5-3 summar izes the p ossi ble metho ds fo r deve lopi ng PRGs for va rio us en viro nment al med ia. It should be noted that iterativ e reduction (IR

) can be used in all ca ses, whereas iterative truncation (IT) is lim ited to situa tions w here the highe st conce

ntration s can be

identif ied and removed. Back

calculat ion may be applica

ble in all ca ses, bu

t becau se of ca

veats noted in

Section 5.4.1 , iterat

ive ap proache s are g enerally

recom mend ed in th is docum ent.

Ta ble 5 -3. Summary of Potential Methods for PR G D evelopment by Environmental Med ium

. Potenti al Exposure Med ium Backcalculati on Iterati ve

Reduct ion (IR) Iterati ve

Truncation (I T) Explanations for IT

Soil X X X Appli cable if soi l i s re lat ively f

ixed. Sediment X X X Appli cable if sedi ment is rel ati vely

fi xed. In some si tuat ions, sediment trans port may be a better assumption due to current veloci ty, tides , res uspensi on, etc

. Bio ta (Fish, A qua tic Invertebrates, Plants) - bioi ndicators of con tamina tion in sediment X X SA Depends on home- rang e of f ish rel ati ve t o the scale of t

he sampling design. I f con tamina tion is co rrelated to relative ly stati c sediment, and the home-range of t he fish is relatively sm all (e.g., no n-migra tory) then I T may be applicabl e. Surface W ater X X NA No t app licable as surfac e wate r is a fluid medi um. Gro un dwa te r ( GW) X X NA Not ap pli cabl e as GW is a f lu id me diu m. General ly , ARARs mus t al so be s ati sfi ed. Home-grown produce, milk, liv estock, other foodit ems X X SA De pen ds on relative c ontrib utions o f soil uptak e (appl icabl e) vs . f oli ar depos it ion ( not applicable).
X=applicable NA=not a pplicable SA=som etimes applicableRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-24 

### REFERE NCES F OR 

### C HAPTE R 

### 5 Bowers, T.S., N. S. Shifrin, and B. L. Murphy. 19 96. Statistical Approach to Meeting Soil Clean up Goals. Environ. Sci. Techn ol. 30:1437–1444. Bowers, T.S. 1999 . The Concentration Term and Derivation o f Cleanup Goals Us ing Probabilistic Risk Assessment. Hum. Ecol. Risk Assess . 5(4):809–821.
Burmaster, D .E., K. J. Lloy d, and K.M. Thom

pson. 1 995. The

Need f or New Methods to Backcalcu

late Soi l Cleanup Targets in I

nterval and Probab ilistic Cancer Risk A ssessments. Hum. Ecol. Risk Assess. 1(1):89–100. Ferson, S. 1996. What Monte Carlo Methods C annot Do. Hum. Ecol. Risk Assess. 2:990–1007.
Free ze, R.A . and

J.A. Cherry . 19 79. Grou ndwa ter. Pren tice Hall , I

nc., NJ. Gilbert, R.O. 19 87. Statistical Methods for Environm ental Pollution Monitoring . Van Nostrand Rein hold, NY. Hall, P. 1988. Theoretical Comparison of Bootstrap Confidence Intervals. Ann. S tatist. 16:927–953. Hope, B.K. 20 00. Ge

nerating Probab ilistic S patially

-Exp licit I

ndivid ual and Popula tion Ex posure Estimates for Ecological Risk

Assessments. Risk An al. 20(5):573–589. Hope, B.K. 2001. A Case Study Comparing

Static and Spatially Explicit Ecological E xposure Analysis Methods. Risk Anal . 21(6):1001–1010. Roeder, Kathryn. 1 994. A Graphical Techn ique for Determinin

g the Num

ber of Compo

nents in a Mixture of Normals. J. Amer. Stat. A ssoc. 89(426):487–495. Schulz , T.W. a

nd S. G riffin. 1999. Estimating Risk Assessm ent Exp osure P oint Concentra tions When the D ata are not Normal or Lognorm al. Risk An al. 19: 577–584. Sposito, G., W.A . Jury, and V.K. Gupta. 1986. Fundam ental Problems in the Stochastic Convection-Dispersion Model of Solute Transport in Aquifers and Field Soils. Water Res. 22(1):77–88.
U.S. EPA. 1 990a. National Oil and H azardous Substances P

ollution Contingency Plan . Final Rule. 40 CFR

300: 55 Federal Register, 8666-88 65, Thursday, March 8.
U.S. EPA.19 90b. Geostatistics for Wa ste Management . A Users Manual for the GEOPACK G eostatistical Softwa re. EPA/ 600/8-90 /004, J anuar y. U.S. EPA. 1 991a. Risk Assessment Guida nce for Superfund (RAGS) , Volume I: Hum an Health Evaluation Manual (HHEM), Part B, D evelopment of Risk- Based Preliminary Remediation Goals. Office of Emergency

and Remedial Response, Washing ton, DC. EPA

/540/R-92/003 . NTIS PB 92-963333.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 5 ~ De cem

ber 31, 2001

Page 5-25 U.S. EPA. 1 991b. Risk Assessment Guida nce for Superfund (RAGS) , Volume I: Hum an Health Evaluation Manual (HHEM), Supplem ental Guidance: Standard Default Exposure Factors,Interim Final. Office of Emergency

and Remedial Response, Washing ton, DC. OSWE

R Directive No. 9285.6-0 3. June.
U.S. E PA. 19 91c. G

EO- EAS 1. 2.1 Us ers Gui de. EPA/600/ 8-91 /008. A pril.
U.S. EPA. 1 992. Supplemental Guidance to RAGS: Calcula ting the Concentration T erm . Offic e of So lid Waste and Emerg ency Response, Washington, DC . OSWER Directiv e No. 9285.7- 081.
U.S. EPA. 1 993. Data Quality Objectives Process for Superfund: Interim Final Guidance . Office of Research and Developm

ent, Washington, D C. EPA/540/R- 93/071.
U.S. EPA. 1 994. Guidance for the Dat a Quality Objecti ves Process (EPA QA/G-4). Office of Research and Devel opme

nt, Wash ington , DC. EPA/6 00/R- 96/055 . Septe mber.
U.S. EPA. 1 997. The Lognormal D istribution in Environmenta l Applications . Office of Research a nd Development, and Office of So lid Waste and Eme rgency Resp onse, Washington, DC. EPA/60 0/R- 97/006 . December.
U.S. EPA. 2 000. Data Quality Objectives Process for Hazardou s Waste Site Investig

ations . Office of Environ mental Inf

ormatio n, Wash ington, DC. EPA/6 00/R-00/ 007. J anuar y. U.S. EPA. 2 001a. Draft Guidance on C alculation of Upper C

onfidence Limits for Exp osure Point Concentratio ns at Superfund Sites . Office of Em ergency and Remedial Respo nse, Washington, DC.
U.S. EPA. 2 001b. Draft Guidance on S urface Soil Cleanup at Superfund Sites: Applying Cleanup Levels. Office of Emergency and Remedial Respon se, Washington, DC.
U.S. EPA. 2 001c. Integrating Dynamic F ield Activities into the Superfun d Response Process: A Guide For Project Managers. Final Draft. Office of E mergency and Remedial Response, Washing ton, DC. OSWER Directi ve No . 9200 .1-4 0. Dec embe r. U.S. EPA. 2 001d. Improving Sampling, Analysis, and Data Management for Site Inve stigation and Cleanup. Technolo gy I

nnova tion Of fice. EPA/542/F-0 1/030a. April.
U.S. EPA. 2 001e. Resources for Strategic Site Investigation and Monitoring. Technology I nnovation Office, Washing ton, D C. EPA

/542/F- 01/030

b. Sep tembe

r.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-1 

## C HAPTER 

## 6
C OMMUNICATING 

## R ISKS AND 

## U NCERTAINTIES IN 

## P ROBABILISTIC 

## R ISK 

## A SSESSMENTS 

### 6.0 I NTRODUCTI ON The Environme

ntal Protection Agen

cy (EPA) has developed a g uidance documen

t, Risk Assessment Guidance for Superfund: Volume I–Human Health Evaluation Manual, Supp lement to Part A: Community Involv

ement in Superfund R isk Assessments (U.S. EPA, 1 999a) and two v ideotapes,

“ Superfund Risk Ass essment and How You Can Hel p, An Overview” (10 minutes) (U .S. EPA, 199 9b) and “Superfund Risk Assessment and How You Can Help” (40 minut es) (U. S. EPA, 20 00b), t o improve comm unity involv

emen t in the Superf und risk assessm ent proc ess. The

videotapes (a

vailab le in both English and Spa

nish) show exam

ples of how regio

ns have involved com

munities in the risk

assessment process at several Su

perfund sites. The gu

idance document and videotapes, a long with the Superfund Commu nity Inv

olvemen t Handb ook and

Toolki t (U.S. EPA, 1 998), s hould s erve a s a prim

ary comm unity involvem ent resource for risk a ssessors and remed ial project managers (RPMs). The Handbook and Toolkit offers the following specific guidance:

• Provides sug gestions for how Superfund staff and com munity m embers can wo

rk together during the early

stages of Superfun

d remedial investigation and feasibility study (RI

/FS) and later cleanup

• Identif

ies whe re, wit hin the framewor k of the hu man healt h risk as sessmen t method ology, community i nput ca n augment a nd improve EPA’s es timate s of exp osure and ris k.

• Recomm

ends questions the site team (risk assessor, RP M, and comm unity involv

ement coordi nator [CIC]) sho uld ask t he communit y.

• Illustrates why

commu nity involvement is va

luable during the h

uman health risk

assessment at Superfund sites. This chapter provides g

uidance and suggestions on how to d eal with risk com

munication issues that arise during a p

robabilistic risk assessm

ent (PRA). Specifica

lly, the concepts o f uncertainty and variability m

ay present additional commun ication challenges for P RA. For exam ple, whereas discussions of uncertainty for point estimate risk assessmen ts are often qualitative in nature, PRA ope

ns the floor for dis cussion

and pre sentation of quantitati ve uncertaint

y anal ysis. Conce

pts asso ciated w

ith quantitative characte

rizatio ns of un certaint y may be more difficul

t to com munic ate and

may not be well receiv ed due to stak

eholder desires for cert ainty (Slovic et al. , 1979 ). As s uch, th is chapt er hig hlight s appropriate stakeholder involvem ent and principal risk commun ication skills that are effectiv e for communicating PRA concepts and ris k information. Key factors for successful communication of PRA include early and co

ntinuous involvement of stak eholders, a well- developed com munication plan, good graphics, a work

ing knowle dge of the factors that may influ ence perceptions of risk

and uncertainty, and a foundation of trust and credibility.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-2 E XHIBI T 6-1
D EFI NIT IONS FOR C HAPTE R 6 Central Tendency

Exposure

(CTE) - A risk

descriptor

represe nting the av

erage or typica l individual i

n a population, usually considered to be the m

ean or m edian of t he distribution. Comm unity A dvisory Group (CAG

) - A g roup form

ed to provide

a public f

orum for comm unity m embers to present and discuss their needs and concerns related to the Superfund decision-making process. A CAG serves as the focal point for the exchange of information among the local comm unity, EPA, State regulatory agency, a nd other pertinent Federal agencies involved in the cleanup of a Superfund site. Comm unity Inv olveme nt Coordinator

(CIC) - As a m

ember of the CAG

and site team, the CIC coordinates communication plans (i.e., the CIP) and addresses site-specific CAG organizational issues. Comm unity Inv olveme nt Plan (CIP) - A plan that ident ifies c ommunit

y conce rns and the pr

efere nces of the comm

unity for the communication of site-related issues. Confidence Interval - A range of va lues that are likely to include a population parameter. Confidence intervals may describe a param eter of an input var

iable (e.g ., mea

n ingestion r

ate) or output variable

(e.g., 95 th percentil e risk). When used t

o charact erize unc ertainty

in a risk estim ate, it is assum ed that m ethods used to quantify

uncertaint

y in the model inputs are based on sta

tistical principles s

uch as sa mpling dis

tributions or Ba

yesia n approaches . For exam ple, give n a random

ly sam pled data s et, a 95% c onfidence interval for the m ean can be

estim ated by deriving a sam pling distribut

ion from a Student's

t distribution. Credible Int

erval - A range of values that represe nt plausible bounds on a population parameter. Credible intervals may describe a param eter of an input var

iable (e.g ., mea

n ingestion r

ate) or output variable

(e.g., 95 th percentil e risk). The term is intr oduced as an altern ative to th e term confiden ce interval whe n the metho ds used to quanti fy uncertainty

are not bas ed entirel

y on sta tistical principles s

uch as sa mpling dis

tributions or Ba

yesia n approaches

. For exam ple, mult iple estim ates of an arithm

etic m ean ma y be av ailable f rom dif ferent s tudies repor

ted in the literature —using prof

essional j udgment,these es timate s may

support a decision to desc ribe a ra nge of pos sible values f or the arithm

etic m ean. Cumulati ve Distr ibution Function

(CDF) - O

btained by

integrat ing the PDF, g

ives the cumulat ive probabil

ity of occurrence for a random independent variable. Each value c of the f unction is the

probability

that a ra ndom observation xwill be less than or equal to c . Hazard Q uotient (HQ) - The ratio of estimated site-specific exposure to a single chemical f rom a site over a specif

ied period to the e stimat

ed daily exposure level, a

t which no a

dverse health eff

ects are likely to occur. Hazardous

Substance R

esearc h Centers (HSRC) - Research centers providing free technical assistance to communities with enviro nmental con tamination programs thr ough two distinct outreach programs: Technical Outreach Services for Com munities (TOSC) a

nd Technic al Ass istance t o Brownf ields Com

munity (TAB ). Histo gram - A gr

aphing tec

hnique whic

h groups the da

ta into inter

vals and di splays

the count of

the observ

ations within each interval. It conveys the range of values and the relative frequency (or proportion of the sample) that was observed across that range. Monte Carlo A

nalysi s (MCA)

or Monte Car

lo Simulat

ion- A tec

hnique for characte

rizing the uncertaint

y and variabili ty in ris k estim ates by repeate dly sam

pling the proba

bility dis tributions of

the risk equation inputs and using these

inputs to calculate a distribution of risk va lues. A set of i terations or

calculat ions from Monte Carlo sampling is a sim ulation. For e

xample , a single

iteration f

or risk f rom inge stion of w

ater m ay re present a hypothetical individual who drinks 2 L/day and weighs 65 kg; another iteration may re present a hypothetical individ ual who dr inks 1 L/day and weighs 72 kg. Parameter

- A va lue that cha

racteriz es the dist ribution of a

random v

ariable. Paramet

ers com monly charact erize the location, scale, shape, or bounds of the distribu tion. For example, a truncated normal probability distribution may be define d by four

param eters: a rithmet ic mea n [location], sta

ndard devia

tion [scale], a

nd min and m

ax [bounds]. It is important to distinguish between a variable (e.g., ingestion rate) and a parameter (e.g., arithm etic mean ingestion r

ate). Percentile - A num

ber in a dis

tribution such t

hat X % of the value s are le ss than the number a nd 1-X % are greate

r. For exam ple, the 95 th percentil e is a num ber in a dis

tribution such t

hat 95% of the value s are le ss than the number a nd 5% are g reater.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-3 E XHIBI T 6-1
D EFI NIT IONS FOR C HAPTE R 6— Continued Point Estima

te Risk Asses sment

- A ris k asse ssment

in which a

point estim

ate of r isk is c alculate d from a set of point estima tes for exposure a

nd toxicity . Such point es timate s of risk

can ref lect the C TE or RME, de

pending on the choice of inputs. Potentially

Responsible

Party (PRP) - Individual s, compa nies, or any

other party

that is pote

ntially liable f or Superfund cleanup costs. Prel iminar y Remedia tion Goal (P RG) - Initially developed chemical concentration for an environmental medium that is expec ted to be protective of

human hea

lth and ecos

ystem s. PRGs m

ay be de veloped bas ed on applica ble or releva nt and appropria

te require ments ( ARAR s), or ex posure sce

narios ev aluated pri or to or as a

result of the baseline r isk ass essme

nt. (U.S. EPA,

1991a, 1991b). Probabilistic R

isk As sessm

ent (PRA) - A

risk as sessm

ent that y ields a proba

bility dis tribution for risk, ge

nerally by assigning a probability distribution to represent variability or uncertainty in one or more inputs to the risk equation. Probability D ensity

Function (PDF)- A f

unction or gra

ph represe

nting the proba

bility dis tribution of a

continuous random v ariable. The densi

ty at a point refe

rs to the proba

bility tha t the var iable w ill have a value in a

narrow range about

that point. Rank Corre

lation (Spearman Rank Order Correlation Coefficient) - A “distribution free” or nonparametric statistic r that measures the strength and direction of association between the ranks of the values (not the values themselves) of tw

o quantitative variables. Reasonable

Maxim um Exposur

e (RME) - T

he highest

exposure that is re asonably expected to occur at

a site ( U.S. EPA, 19 89). The in tent of the RME is to est imate a conservat ive exposur e case (i.e., well above the average case) that is still within the range of possible exposures. Remedi al Inves

tigation/Fe asibility

Study (RI/FS) - St

udies underta

ken by EPA to deline ate the na ture and ex

tent of contamina

tion, to eva

luate potenti al risk, and to deve lop alterna tives f or cleanup. Sensitivit y Ana lysis - Sensitiv

ity ge nerally refer s to the va

riation in output of

a model with res pect to cha

nges in the values of the model’s input(s). Sensitivity analysis c an provide a quantitative ranking of the model inputs based on their rela

tive contr ibutions to m

odel output var

iability and uncerta

inty. Com

mon me trics of sensitiv ity include:
< Pearson Correlation Coef ficient - A statistic r

that me asures the

strength a

nd direction of

linear as sociation betwee n the value s of tw

o quantitativ

e varia bles. The square of the coef ficient ( r

2 ) is the f raction of the variance of one variable that is explained by the va riance of the second variable.
< Sensitivit

y Rati o - Ratio of the change in model output per unit change in an input variable; also called elasticity

. < Spearma

n Rank Orde

r Correlat ion Coeff icient - A “distribution free” or nonparametric statistic r that measures the strength and direction of association between the ranks of the values (not the values themselves) of

two quantit

ative v ariables . See Pears on (above) f or r 2

. Stakeholder- Any

individual or

group who ha

s an intere st in or m

ay be a ffec ted by EPA ’s site decision-m aking process. Technica l Assis tance G rant (TAG ) A federal gra

nt that is int

ended to provide a com

munity with the opportunity to hire independe

nt experts to help eva

luate and e

xplain the r

esults of a risk a ssessm ent. Technical Outreach Services for Comm unities (TOSC) -

A serv ice of t he HSRC w

ith the aim

to provide inde

pendent technical information and assistance to help communities with hazardous substance pollution problems. Uncertai nty - Lack of know ledge about

specif ic varia bles, para meter s, model s, or other f

actors. E xample s include limited data regarding the concentrat ion of a c ontaminant

in an envir

onmental medium

and lack of infor mation on l oca l fis h co nsu mpti on p ract ices . Un cert ain ty may b e red uce d th rou gh fu rth er st udy. Variable - A quantity that can assume m any values.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-4 E XHIBI T 6-2
S TAKEHO LDERS P OTENTIALLY I NVOLVED I N THE D ECIS ION -M AKING P ROCES S FOR PRA

C EP A risk assessors and manage rs

C Memb ers of

the publ ic

C Rep resen tatives fro m state o r cou nty environm ental or health agenc ies

C Other fed eral agencies (e.g., health agen cies, Natural Resources Dama ge Assessment (NR DA ), trustees, e tc.)

C Tribal gover

nment repr esentativ es

C Po tentially resp onsib le par ties (PR Ps) a nd the ir represent ativ es

C Rep resen tatives fro m fed eral fac ilities (e.g., De partm ent of D efense , Dep artme nt of E nergy, etc.) Section 6.1 discusses the need for early an d continuing stak

eholder involve

ment. Section 6.2 recommen

ds a seven- step process for com municating P RA results to stak eholders, and Sections 6.3 and 6.4 provide

guidance on sp ecific techniques for com municating information. The succ ess of risk commun ication efforts will depend o n the extent to which the comm unication strategy addresses the needs of a diverse audience, with different pe rceptions of risk and uncertainty (Section 6. 5), and the degree of trust and credibility that is established from the outset of the process (Section 6. 6). Section 6.7 prov

ides a discussion of risk

commu nication issues that are uniqu ely relevant to RPMs. 

### 6.1 S TAKEHOL DER 

### I NVOLVEM ENT Many stakeholde

rs may be interested in a risk a ssessm

ent (see Exhibi t 6-2 ). I t is ge nerally impor tant toinvolve and engage interested stakeho lders ea rly and

continu ously

throughout the decision-m

aking process (U.S. EPA, 2 001). Public involve

ment activities should be tailored to the needs of the commu nity and described in the site com

munications strateg

y. The CIC shou

ld coordinate these first steps throug h the de velopment of a Comm

unity Involv ement Plan (CI

P). Coordination between the RPM , risk

assessor, and CIC

is need ed to determine the appro

priate points in the RI

/FS process to comm

unicate with the com

munity, a nd plan for the appropriate lev el of comm

unication. The CIP shou

ld identify com

munity con

cerns and the preferences of the com munity for the commun ica tio n of s ite -rela ted issu es. T he CIP may be updated during the RI/FS as n eeded. Examples of ou

treach activities include giving oral presentations and poster sessions at public m eetings, coordinating

group m eetings or focused workshops, con

ducting interview s with comm unity mem

bers on specific issues, an d distributing fact sheets. Idea

lly, t he publ ic and other interested stakeh olders w ould be invol ved ea rly in the site -spe cific decision-m

aking proces s. I f the comm

unity has not

been previou sly in

volved, eff

orts sho uld be m ade, in coordination with the C IC, to iden tify and com

municate with the

appropriate individuals in the comm unity prior to

the Agency’ s receipt of the PRA workpla

n. The p ublic an d other stakeh

olders should be given the

opportunity to provide input to the workplan for a P RA (see Chapter 2 , Section 2.1). The initia l com munit y meeting can ser ve to establish a rapp ort between EPA and the comm unity and facilitate the exchang e of information n eeded to support a PR

A. This information

may inclu

de policy decisions assoc iated with both point estim ate and probabilistic approaches, as well as technical details regarding the conceptual exposure m odel and the selection of distributions. A discussion of these topics may increase certainty about the assumptions m ade in the risk asse ssment. For example, the comm unity may

be able to offe r insig hts reg arding

site- specific

activities and sourc es of ex posure dataRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-5 not readily ava

ilable to the risk assess or. This type of d iscussion should allow for the free exchange of inform ation with the public a

nd sets the stage for fu

ture dis cussion s. I

t is im portant that an appropriate level of detai l be pre sented a t the first me eting. Instead of overlo ading the aud ience w ith informatio

n, it is generally better to coordinate several m eetings so that com

plex policy and tec

hnical concepts can be broken down in

to smaller discussion topics. Following the approval of the PR A work plan, t he publ ic and other interested stakeh olders should be involved in various stag es of the PRA dev elopment, includin

g providing

and/or reviewing

data, reviewing the selected distributions (e.g ., selected creel survey) and com menting on

PRA docum

ents as appropr iate dur ing pu blic com ment periods . On- going

comm unity involv

emen t may require consideration of EPA’s resources including the availability of p ersonnel and contractor sup port. Other considerations include EPA ’s compliance w ith provision in the National Contingen cy Plan (NCP ) for involving the comm

unity. The appropriate level of com munity inv

olvement in the PRA should b e based on a number of factors including the nature and extent of c ontamination at the site, the expressed interest s of the

comm unity mem

bers, the com

plexity of the P RA, an

d the ro le of PR

A in sit e-sp ecific remediation or cleanu

p decisions. 

### 6.2 C OMMUNICATIO N AND 

### P RESENT ATION Commu nication is a two-

way process that should involve the transfer of inform ation between the Agency an

d the stakeholders, as well as active listen ing by the Agency to the stakeholder’s ideas and concerns. The goals of risk com munication are to present risk inform ation in an understandable manner through an open,honest, frank, and transparent presentation an d discussion of risks , including uncertainties. In

meeting these goals, it is im portant that the RPMs and risk assessors be sin

cere and direct in their presentat

ion of t he resu lts of the PRA, accept the pub lic and other in terested stake holders as valuable contributors

to the process, and listen to the concerns and ideas that are raised. One goal of communication should be to respect the stakeholder’s conc erns. The public and other interested stakehold ers should have the opportunity to u nderstand the PRA a

nd its effects on the decision-m

aking process. Technical Assistance Grants (TAGs) may be one way to advance this goal by providing the commun

ity the opportunity

to hire independent exp erts to help evaluate and explain the results of the PRA. Alternatively, the RPM and risk assessor may use the tools outlined in

Sections 6.3 to 6.6 to prese

nt PRA concepts and t

he resu lts of the PRA to the c omm unity in a m

anner that is eas ily understood. This m

ay require sign

ificant up-front pla

nning, testing , and post- evaluation to identify the appropriate messages to comm unicate and to determ

ine how well this inform

ation was comm

unicated. The site-specific PRA

commu nication plan should be consistent with the NC P’s provisions on comm unity involv

emen t. I t is im portant to reco gniz e that c omm

unity involv

emen t is part of a reg ulatory process and that EPA

generally w

ill consider all timely

public input, but m

ay not im plement all of it. Ultimately, EPA must m

eet the legal requirem

ents of the Superfund la

w in mak ing decisions reg arding remedial actions. A vast body

of literature exists regardin g risk com

munication. S ince the early 1980 's, a num ber of resea rchers have de

velop ed mo dels for

comm unicatin g risk to the p ublic. These m

odels ar e avai lable in the s cientifi c literat

ure, an d a list of supp lemen

tal refer ences is provi ded at t

he end of this chapter

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-6 6.2.1 C OM MU NICAT ION O F PRA W IT H C ONCERNED C ITIZ ENS , O THER S TAKEHOLDERS , AND M ANAGERS : A N O VERVIEW Before the decision to co nduct a PRA is m

ade, a CI P should be in place. Generally, w hen a decision is made to conduct a PRA, an important step should be to work

with citizens to de

velop a commun ication strategy for PRA and its applica tion within the Superfund process (see Chapter 1). The initial introduction of the com munity to the RI/FS process should include a discussion of the principles of risk a ssessm

ent. Thi s discus sion m

ay be best pre sented i

n an inf ormal setting such as a publi c availa bility session . Beca

use of t he pote ntially comp lex natu

re of PR A and q uantitat ive un certaint y analysis, a sm

all group m

eeting may

be an appropriate forum

in which to discuss issues and facilitate an exchange of ideas. If there is interest among a large grou

p of stakeholders, multiple sm all group sessions may be scheduled. Such m eetings may

provide the foun

dation for building tru st and credibility (see Section 6.6). In general, it is important to identify whether a Comm unity Adviso

ry Group (CAG) should be formed. The purp ose of a CAG is to provide a pub lic forum for com munity m embers to present and discuss their needs and co ncerns related to the Supe rfund decision-m

aking process. The CIC is an important m

ember of the team

and may coordinate comm

unication plans, hand- out materials, and address site-specific org anizational issues. A number of resources may be available to the

commu nity to aid in und

erstanding technical material in a PR

A. In addition to the TAG prog ram, which p

rovides funds for q

ualified citizens’ gro

ups affected by a Sup erfund site to hire independ ent technical advisors, another program is the Technical Outreach Services fo

r Comm unities (TOSC), which u ses university e ducational and technical reso urces to help comm

unities underst

and the technic al issue s invo lved i n haza rdous waste site s in the ir commun ities. This is a no- cost, non-ad vocate, technical assistan ce program supported by the Hazardous Substance Research Ce

nters. The tiered approach for PRA presented in Chapter 2

(Figures 2- 1 and 2-2) encourages risk assessors and RPMs to p articipate in discussions with stakeholders early in the process of dev

eloping point estimate and probabilistic approaches. If a decision is m ade to perform a PRA, a continuing dialog ue shou ld be us eful to evaluate inte rim re sults of

the PR A and d eterm ine if ad ditiona l activ ities are warranted (e.g., data collection, further m odeling). These on- going discussio ns should help assure that RPMs are aware of the details of the PRA an alysis and are com

fortable with the m

aterial that will be shared with the com

munity, other interested stakeholders , and senior m anagers.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-7 6.2.2 S TEPS F OR C OMMUNICATION OF THE RESULTS OF THE PRA The complexity of a PRA will v ary depending

on the site-sp ecific nature of the assessm ent perf ormed . For examp le, P RAs may incl ude a n ana lysi s of va ria bil ity, unce rta int y, or b oth. Some analyses ma

y involve simulations to ev aluate temporal v

ariability (e.g., Microexposure Even t analysis) and spatial variability

(e.g., g eostatistics). The challenge for presenters is to identify the critical information and level of detail to be presented to various audiences that m ay be involved in the Superfund decision-making process (e.g. , senior risk mana gers, concer

ned citizens, cong

ressional staff, and PR Ps). The 7-step process, described below (and summarized in Exhibit 6- 3), may be repeated many times du ring the perfor mance of a PRA. For co mmunicati on purp oses, a PRA normal ly will i nvolve more interaction w

ith stakeholders than a point estimate ri sk assessment because PRA concept s and re sults ar

e often more difficul

t to comm unicate. (1) Identify the Audience The first step sho

uld be t o identi fy the audience of potentially interested stakeholders. Str ategies for prese nting PRA information norm

ally will be tailored to

the audience. Participants in the audience may change during

the tiered process depend ing on the complexity of the PRA (see Chapter 2) and the specific site-managem ent decisions being mad

e. (2) Identify the Needs of th e Audience The second step should be to identify the needs of the audience. The relevant informa tion and the appropriate level o f detail will vary depending on the audience. For example, som e participants may be well informed about PRA concepts and will not need m

uch introductory PR

A information. For other audiences, PRA co

ncepts may

be new, so it m

ay be beneficial to hold an informal meeting to discuss the general objectives and

methods used to conduct a PRA. Once introductory PRA concepts hav

e been discussed and are understo od by the audien ce, more adv

anced discussions m

ay be warranted on topics such as the sources of data used in the PRA , the most critical variables in the PR A (identified during the sensitivity analy sis), the selection of distribution s, and the level of characterization of un certainty (see also Section 6.5). The risk assessor should select the key information for each topic and discuss the significance of this inform ation based on the intend ed audience. E XHIBI T 6-3

I MPORTA NT S TEPS FOR C OMM UNICATI NG PRA R ESULTS (1) Identi fy t

he audienc e (2) Identi fy t

he needs of

the audi ence (3) Develop a communicat ion plan (4) Pract ice to as sure cla rit y of pres entati on (5) Presen t i nfor

mati on (6) Post-meeting review of present ation and community

feedback (7) Upd ate informati on as needed for future assessments and presentat ionsRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-8 (3) Develop a Communication Plan The third step should be to develop a plan to comm unicate significant inform ation to the public in an easily understandable form at (Exhibit 6-4). Adequate planning in the presentation of PRA information is essential. A thoroug h understandin

g of the desig

n and re sults of the P RA wil l help t

o place the inf ormat ion in proper context and und

erstandable format (U .S. EPA, 1994). Even mo

re importantly, the risk assessors and RPMs shou ld clearly identify the main me ssages to be presen ted. Section 6.4 prov

ides examples of graphics that m ay be useful in p

resentations of PRA. Handouts, glossa

ries, and other m

aterials may c omplemen

t a presentation and prov

ide information for discussion following the meetings. In addition , examples des igned to help dem onstrate concepts unique to PRA (e.g. , using

one pro bability distribution to

describ e vari ability

and a s econd d

istribut ion to describe parameter u

ncertainty) may

help facilitate the flow of commun ication and increase the lev el of underst anding . One useful t

echniqu e in pub lic me etings

is to in volve mem

bers of the aud

ience to illustra te a con cept. F or exam

ple, th e topic of discu ssion m ay be the method u sed to s elect an d fit a probability distribution us ed to characterize v ariability in a PRA . To demonstrate this concept, a risk assessor can draw a be ll-shaped curv e on a flip chart and la

bel the x-axis , “number of liters of water consumed per d

ay”, and the y-axis , “number of people who consum e a specific amou

nt of water in a day”. Next, each meeting participant can be ask

ed to identify their own consumption pattern, perhaps by holding up a 0. 5 liter b ottle an

d aski ng how ma

ny su ch bottl es are c onsum

ed on a n aver age da y. Thi s commun ity-specific information can then be plotted on a new

graph in the form

of a histogram

and the bars can be connected to form a curv e or distribution sim

ilar to the one first drawn. The resulting distribu tion (for an exa

mple, see Fig

ure 6- 1) can t

hen be used to

discuss the fol lowing

PRA c oncepts

in more detail:

• Variability (between indiv iduals)

• Shape of the distribution a nd plausible range of values

• Central tendency ex

posure (CTE) and reasonable maximu m exposure (R

ME) estimation

• Uncerta

inty i n the di stributi on (sam

ple siz e, pote

ntial res ponse b ias, dif

ference s in act

ivity patterns)

• Uncerta

inty i n a para meter estimate (dif

ference betwee n the 95 % upper confide

nce lim it (UCL) for a m

ean and the 95 th percentile) Using this inf ormat ion as a basis, the risk

assessor can compare

the results from the

comm unity analysis with data from various geographic area s in the U.S. w here water consum

ption patterns may differ. The risk assessor can then lead a discussion with the com

munity regarding the va

rious sources of uncer taint y in sele cting an d fitt ing expo sure di strib ution s, incl uding: (a) Extent of Representa tion - Are the a

vailable data representativ e of the target pop

ulation? For example, would the data on water consumption collected during the m eeting be representative for various population gro ups? E XHIBI T 6-4
K EY C ONSI DERATI ONS I N D EVELOPI NG U NDERSTANDABLE M ATERI AL

• Identi fy mai n messages

• Place information in appropriate context

• Use clear fo rmats

• Use examp les and graphs

• Provide handout s and gl ossari es

• Presen t i nfor

mati on wi th mi nimu m jarg onRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-9 (b) Data Quantity - Wha t sam ple siz e is nee ded to d

evelo p a dist ribution ? This d

iscussio n will introduce the concept that uncertainty in both p oint estimates and probability distributions may be redu ced by inc reasi ng the sa mple size (c) Data Qu ality

- Are the data collected using a cceptable study proto cols? Is the information available from the peer-rev iewed literature? An exa mple can be m

ade of the data collected during the m

eeting to high

light issues associated w

ith survey de sign, and m

ethods for controlling for potential b ias or error. For exam ple, if the survey

data were to be used

in a risk assessmen

t for a drinking

water scenario, the data

quality ma

y be imp roved by re

peat sampl ing ov er ti me Other exposure va

riables that can be used in this distribution exam

ple include: fish consum

ption rates, c hemic al conc entrations in so il, and fraction of tim

e spent indoors . In

gener al, exa mples should focus on vari ables that ma y be o f intere st, are easily illustra ted, an d are un likely to ma ke par ticipant s uncomfortable div

ulging person

al information such as age. (4) Practice to Assure Cla rity of Presentation The fourth step should be to practice the presentation to assure that the inform ation is presented clearly to the i ntended audience. Staff from

comm unicatio n grou ps or pu blic inf ormat ion offi ces with

in EPA regional o ffices may help to determine w hether or not the presentation addresses the needs of various audiences. Also, practicing the presentation with co- workers who are

unfamiliar with the site can help assure that the approp riate messages are being conveyed, and will help the team pre pare for poten tial q uesti ons tha t will arise during t he meeti ng. (5) Present Information A number of factors should be considered when developin

g a plan to prese

nt the PRA in a meeti ng. A lthoug h the si ze of the pub lic me eting can som etime s be un

predicta ble, ty pically indiv iduals will feel more com

fortable asking questions and expressing opinions in sm

all, informal setting s. For any audience, it is usually

helpful to have general fact sheets on PR A available for distribu

tion. The fact sheets may contain information that describes the PRA process, how inform ation from the P RA will be used at the site , and how the comm unity may

comm ent on t he PRA report. The meeting

team should usually include the C IC, RPM, Risk Assessor, and additional support a s necessary. Audio-v isual materials and eq

uipment should be checked prior to the start of the m

eeting. For exam ple, ov erheads should be vi ewed fr

om the audie

nce sea ting t o assure that in forma

tion is a ccessib le and readable. Presentation s using portable computers can be effective for showing how the results of the PRA may differ with changes

in modeling

assumptions. (6) Post-meeting Review of Presenta tion and Community Feedback At the end of a m

eeting, it can be helpful to encourage participants to provide feedback regard ing effec tive and ineff

ective communica tion t echni ques. No t only ca n this i nformat ion be u sed to i mprove present ations o ffered t

o sim ilar aud iences i n the fu ture, it also pr ovide s a sens e for how well the main messages an

d specific technical issues were comm unicated.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-10 (7) Update Information as Neede d for Future Assessments an d Presentations Shortly after the meeting or briefin g, m odificat ions should be

made to the m aterials for futu re presentations where approp riate. In ad

dition, if information is obtained that is relevant to the risk assessment, this info

rmation m

ay be included in

a subsequent analy sis, and the process would be repeated. 

### 6.3 C OMMUNICATING 

### D IFFERENCES 

### B ETWEEN 

### P OINT 

### E STI MATE AND 

### PRA
One method for effectively exp laining the PRA approach to quantify

ing variability

and uncertainty is to em

ploy comp arisons to the mo re easily understood point estimate m ethodology. These comparisons can focus on either the inputs or the outputs associated w ith the two approaches. The commun icator may fo cus on a specific input v ariable, such as drink

ing water intak

e, and explain that with the point estim

ate methodolog

y, a single average or h

igh-end value (e.g.,2 liters per day fo r adults) norma lly is used to

quantif y exp osure, whereas

with P RA, a probabi lity d istribut ion (e. g., l ognor

mal) i s used to characterize v

ariability in exposure am ong a population. In addition , the outcomes (e.g., cancer risk estimates) ca

n be compared by showing where the point estim

ate(s) of risk fall within the distributi on of risks generated wit h PRA. When comm

unicating results from

point estimate and

PRA mod

els, an important concept to keep in mi nd is that both metho ds yie ld risk estimates wit

h vary ing de grees of unce

rtainty . Cont

inuing with the above exa

mple, concepts assoc iated with unce rtainty (e.g. , repre sentativ

eness, data quantity , and d ata quality ) can be introdu ced by askin g the audienc e if thei

r estim ate of water con sump tion on a speci fic day would be

equal t o their average daily

consum ption ra te ove r a 1-year p eriod. This exa mple highli ghts a

comm on sour ce of un certaint y in exposure

data (i. e., usi

ng short-t erm survey

data to estimate long-term behavior). Section 6.5 discusses different perceptions of uncertainty

. It is com mon to accept output from quan titative models without fully understa nding or appreciating the correspo nding uncertainties and

underlying assumptions. O ne challenge in pres enting PRA results is to determ

ine the most effectiv

e way to com

municate sources o f uncertainty without undermi ning the credi bilit y of the as sessmen t (see Sectio n 6.6). Fo r exampl e, it may be c ounte rintu itive that the more source s of unc ertainty

that are accoun ted for in a PR

A, the wider the confid

ence in terval s tend to be in the risk

estimates (see Sectio

n 6.4.2). The audien

ce may q uestion the utility of a

method that appears to introduce m ore complexity

in a risk m

anagement decision. It may be useful to point out that many sources of uncertainty are present, and m ethods available to ac

knowledge and quantify them may differ in

point estimate and

probabilistic risk assessm ents. The basic concepts of PRA s described in Chapter 1 may be used in develop ing presentations. Exhibits 1-5 and 1-6 in Chapter 1 sum marize som

e of the advantages and disadvantages of point estimates and proba

bilistic approaches that should be considered when e valuating difference s in the risk estimates of the two approaches. For example, point estimates of risk do not specify the proportion of the population that m

ay experience unacc eptable risks. I n contrast, PRA m

ethods allow statem

ents to be made regard

ing both the prob ability of exceeding a target risk, and the level of confidence in the risk estimate. When summ

arizing results of PRA, graphs and tables generally should include the results of the point estimates of risk

(e.g., CTE an

d RME). I

t may be informative to note where on the risk distribution each of the point estim

ates lies. By u

nderstanding the assu

mptions rega

rding the inputs an

d modelingRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-11 approaches used to deriv

e point estimates an

d probabilistic estimates of risk, a com municator will be better prepared

to expl ain the signif

icant di fference s in risk

estimates tha

t may occur. Specia

l emphasis should be given to the mod el and parameter a ssumptions that ha ve the mo st influence on the risk estimates, as determ

ined from the sen sitivity analy sis (see Appendix A). 

### 6.4 G RAPHICAL 

### P RESENTA TION OF 

### PRA RESULT S TO 

### V ARIOU S 

### A UDIENCES Graphics can be an e ffective tool for com municating c oncepts in PRA. As the old adag e goes, “A picture is worth a thousand words.” A graphic usually

can be most easily understood by a diverse audience when it conv

eys a single message. It is g enerally a goo d idea to keep the graphics sim ple so that the messag e is clear. I n general, each g

raphic should be dev

eloped and mo dified depending on the type of presentation a

nd the intended audience

.

L The key

to pres enting g

raphics in PRA effectiv

ely is to select a relativ ely sma ll number of appropriate messages, and to find a balance between meaningful information and overw

helming detail. Points to cons ider when developing graph ics for p ublic m eeting s, seni or staff

, and t he pres s are presented below. Ce

rtainly , recom

mend ations f or prese nting clear an

d inform ative graph

ics are applicable to all three forums. Practical recom mendations for g

raphical analysis techniqu es and tips for successful visual displays of quantitative inform ation are given

by Tufte (1983) and

Helsel and Hirsch (1993).
6.4.1 P UBLIC M EETING For a p ublic av ailabili ty ses sion (or

meet ing), care should be

taken to assu re that t he gra phics ar e of appropriate size an

d the lettering is easy

to read. For exam

ple, a graphic on

an 8 1⁄2 x 11 inch

sheet of paper, or a font siz e smaller than 18 pt in a comp uter presentation, m

ay not be easily

seen from the back of a large auditorium . It m ay be appropriate to

present information using large p osters, spaced so that the audienc e may

move among the

m and discuss the posted resu

lts with the risk assessor or RP M. Han

douts and a glossary

of terms m

ay also be used. Using slides w ith too much tex

t should be avoid

ed, since the inform ation m ay be difficul

t to read and understand. Pre -pla nning and pil ot testin

g the graph

ics before the pre senta tion may be helpfu l in ass uring th at the message is accura tely por trayed to the c ommunity. Consistent with EPA ’s guidance on risk characterization, the C TE and RME cancer risk s and noncancer hazards, and EPA’s decision p oint should be hig hlighted on g

raphics. The discussions accompany

ing the graph

should empha size that these values represent risks to the average an d high- end individuals, respectiv

ely, and serve as a point of reference to EPA’s decision po int. The distribution of risks should be char

acteriz ed as re

present ing variabili

ty am ong the popul ation ba sed on differen ces in exposure. Sim

ilarly, graphics that show uncertainty in risk can be described using term s such as “confid ence in terval ”, “cre dible in

terval ”, or p lausible

range . The g

raphics need not hig

hlight all percentiles. In stead, selected percentiles that may inform risk mana

gement dec isions (such as the 5 th ,50 th , 90 th , 95 th , and 99 th percen tiles) sh ould be the foc us. Fi gure 6 -1 shows an

exam ple of a PDF fo r varia bility in risk with an as sociated text box for identify ing key risk percentiles.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-12 Figure 6-1 . Hypot

heti cal PRA resul ts showi ng a pr obabil ity dens ity f unct ion ( PDF) (top panel) for cancer ris k with sel ected summary stati stics f or central tendency and high- end percen ti les . Thi s vi ew of a dis tri buti on is usef

ul f or il lus tra ti ng the sh ape of the dis tri

buti on (e.g., slight ly ri ght-skewed) and explaini ng the concept of probability as the area under a curve (e.g., most of the area is below 1E-06, but there is a small chance of 2E-06). Although percen ti les can al so be ove rlayed on thi s gr aphic , a c umul ati ve di str ibut ion f unct ion ( CDF) (bottom panel ) may be pref erable f or explai ning t

he concept of

a percent ile.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-13 Figur e 6- 2 gives two exam

ples of graph

ics that can be used to

display results of a se nsitivity analysis from a Monte Ca

rlo Analysis (MCA). While both graphic s are likely to be understood by non-technical audienc es, the

pie cha rt may be more fam

iliar. The pie chart (Fig

ure 6- 2A) sug

gests that the

results should sum t

o 1.0, which may

not be true if t

here are correla tions am ong one or m

ore va riables, or if on

ly a subset of the va

riables are displayed (e. g., those that co ntribute at least 1%). The av ailable data can be normalized so

that the squared correlation coefficients do sum to 100%, and this appro ach has been adopted by som

e commercial software available to run Monte Carlo sim ulations (e.g., Cryst al Ball ® by Decisioneering, w

ww.decisioneering. com). The ben efit of showing the

squared correlation coefficient (r 2 or r-square , also c alled th e coeffi cient of

determ ination ), rathe r than t he corre lation c oefficie

nt (r) is that r-squareis proportional to the total variation in risk associated with specified inpu t variable. Therefore, one can use the r-squareto describe, in quantitativ e terms, the con

tribution of the input variable to the total v

ariance in the risk distribution. In this example, ex posure duration (ED) co

ntributes approx imatel y two-thir ds (64% ) to the total varianc e in ris k.
A more tec hnica l graphi c is the tornad o plot ( Figure 6 -2B). In addit ion to s howing t he rel ative magnitude of the correlations ( r-square ), it illustrates the direction of influence a specific variab le has on the final risk estim

ate. Bars that extend to the right indicate a positive correlation (e.g ., high risk estimates correspond with high values for the variable), whereas bars that extend to the left indicate a negat ive correlation (e.g ., hig

h risk estimates cor

respond with low val

ues for the va

riable. ) In

this example, the ex

posure duration (ED) has the largest positiv e correlation with risk, while body weight (BW) has the largest n

egative correlation with risk. The gra phics sh own in this cha

pter are a sma ll fracti on of the graphics tha t mig ht be us ed to comm unicate concepts relate d to PR

A. Nu merou s additi onal ex ample s are given t hroug hout th is guida nce doc umen t. Table 6-1 provi des a su

mmary of cross r eferenc es to ot

her fig ures tha t were develop ed for t his guid ance do cument t o convey sp ecifi c conce pts re gardin g variabi lity an d uncer taint y.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-14 Ta ble 6 -1. Exa mple s of G raph ics for C omm unicatin g PR A C onc epts in this G uidan ce D ocu men t.

General PRA Topic Area Location Variability Uncerta inty Conc eptual Diagra ms for Fund amental C oncepts
M onte C arlo A nalysis Figure 1-2 X Tier ed pr oces s f

or PRA Figure 2-1, 2 -2 X X PDFs a nd CDFs Input variable(s) Figure 1- 1, 4- 4, 4- 5, 4-6 X

Risk dis tri bution wi th sel ected percen til es highl ight ed Figure 6-1 X
Comparin g RME risk (e. g., 95 th perc entile) with risk levelof concern Figure 1- 3, 4- 3, 7- 2, X

Selec ting and Fitt ing Probabi li ty Dis tri buti ons Fitti ng di str ibut ions - f requen cy di str ibut ion ove rl ai d by

a PDF Figure 3-1 X
Lognorma l proba

bil ity pl ot Figure 5-2 X Sen sitivity An alysis Sensit ivi ty an alys is - tornado plot of Spearman rank correlations Figure 3- 6, 6- 2b X

Sensitivi ty analy sis - pie chart Figure 6- 2a X Joint probability

curve Figure 4-8 X Variability in toxicity Specie s se nsi ti vit

y di str ibut ion Figure 4-7 X Iter ative Simul ations CD Fs from multiple 1-D M CA simulatio ns to convey uncer tai nty in t he ri sk di str ibut ion Figure 3-3 X

PRG Selecti on Estimation fr om best-fit line f or RM E risk and EPC Figure 5-1 X

RM E risk ranges corresponding to alt ernative ch oi ce s o f PRG Figure 7-4 X

90% credibl e in terval for RME risk ( 95 th percen ti le) corre spondi ng t o alt ernative choic es of PRG Figure 7-5 XRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-15 Bi-model distributi on for concentration showing pre- reme dia ti on EPC, post -r emedi ati on EPC, remedi ati on act ion l evel , an d uni form di str ibut ion for clea n fill Figure 5-3 X X

2-D M CA Results Illustration of tabu lar and grap hic ou tputs o f a 2-D MCA Figure 4-9 X
Confidence intervals ( or credible interval s) on a ris k dis tri

buti on Figure 1- 4, 4- 10, 4-11, 4-12 X

Box-and- whi ske r pl ot f or r esu lt s of

2-D MCA Figure 3-4, 7 -3 X Ho rizon tal bo x-and -whisker plots w ith multiple CDFs Figure 6-3 X XRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-16 

### A. Pie Chart

B. Tornado Plot

Figure 6- 2. Resu lts of a sens itivity analysis sho wn as a pie ch art (A) and to rnad o plo t (B) . Bo th grap hics illustrate the concept

of th e relative cont

ribut ion to v arianc e for expos ure vari ables t hat cont ribut e at l east 1% to t

he vari ance in risk. The pie chart suggests t hat the sum of t

he squared rank correlations equals 1.0, which is t rue only if

the resu lt s ar e norma li zed t o 100%. The tor nado pl ot gi ves bot

h th e magni tude and di rect ion (positive or neg ati ve) of the correlati on. ED= exposure duration, IR_soil=soil i ngestion rate, AF=ab sorption fracti on, EF=exp osure frequ ency, S A_ skin= surface are o f skin, and BW =b ody w eight.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-17 Figure 6- 3. The results of a 2-D MC A. The graphic shows a method of presenting variabili ty as a cumulat ive dist ributi on functi on and u ncert aint y as box plot s at

the 25 th , 50 th , an d 95 th

percenti les of varia

bil ity. The CDF of the 50 th percenti le i s repres ented by t he soli d line and the CDFs gi ven by t he dotted l ines r epresent the 5 th and 95 th perc entiles o f uncer tainty for ea ch pe rcentile o f variab ility. 6.4.2 EPA S ENIOR S TAFF For comm

unicating PRA with EPA’s senior risk managers (e.g., EPA

Section Chiefs, EPA Branch Chiefs, or E

PA Division D irectors), an executive summary or executive briefing

package m

ay be appropr iate. Th is prese ntation should highl ight m

ajor findi ngs, comp are poin

t estim ate and probab ilistic results, provide sensitiv

ity analysis results, and state uncertainties addres sed in the PRA.

EPA senior level risk managers would generally be most interested

in the risk estim

ates at the 50 th , 90 th , 95 th , and 99.9 th percentiles (i.e., a C TE risk estimate a nd the RME risk range). EPA senior mana gers m ay als o wish to kno

w the u ncertain ty sur roundin g each of the p ercentil es of risk. Thi s uncertainty can be d

escribed in a table (e.g ., confidence intervals around the 95 th percen tile risk ) or a graphic (e.g. , box-and-

whisker plots). I

t is advisable for the risk assessor to hav e this information o

n hand du ring t he brief ing to respond to questions . Pres enting distributions o f uncert

ainty along

with distributions of variability can create a very

busy figure or table—it is best to k eep things sim

ple.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-18 Figur e 6- 3 show s cum

ulativ e distrib

ution fu nctions (CDFs ) for the

Haza rd Quot ient (HQ ) for a single chemic al, representing v ariability in HQ. One method of displaying uncertainty is to use box-and- whisker plots. I

n this example, the horizontal box and whiskers repres ent uncertainty around selected percen tile esti mates of va riability . Spec ifically , the th ree box -and

-wh isker plots correspon

d to the 25 th , 50 th , and 95 th percentiles of the distribution for variability in H Q. The box shows the 25 th and 75 th percentiles (i.e., interquartile range) of uncertainty , whereas the whisk ers show the 5 th and 95 th percentiles of uncertainty . In this example, uncertain ty in the 95 th percentile HQ is quan tified by the box-and- whiskers plot in w

hich the 5 th percentile of uncertainty is 1.1, the 50 th percentile is 1.3, and the 95 th percentile is 1.4. This su ggests that des

pite the uncertainty in the estimate of the 95 th percentile of variability, an H Q of 1.0 is lik ely to be exceeded . Sometim es such results are said to describe the 90% confidence interval

in the 95 th percen

tile HQ . The te

rm “c onfiden

ce inter val” i s used l oosely

in this context to convey

information abou

t uncertainty; howev

er, it is not the sam

e as a statistical confidence limit that one m

ight obtain by

estimating a population parameter from a sam ple. An alternative term that may be more appropriate in this case is “credible interval”. The three curves represent similar informa tion on uncertainty a

cross the complete range of percentiles for variability . The solid line shows the CDF for all of the 50 th percen tiles of unce rtain ty, whereas the dotted lines s how the 5 th and 95 th percentiles of uncertainty

. The box-and-

whisker plot is sim

ple to produce, conv

eys information

about the sym

metry and width o f the confidenc e interv al, and is easy to inte rpret (Tufte, 19 83). I

n general, box-an

d-w hisker plots are useful for sum

marizing results from two- dimensional Monte C arlo (2-D MCA ) simulations. The metho ds and inferenc es assoc iated with 2- D MCAs are dis cussed further in Appe ndix D. The resu

lts of a 2-DMonte C arlo sim

ulation represe nt a ran ge of possibl e estim

ates for the per centile give

n one or mor

e sources of uncertainty that were included in the simulation. I f the target audience for this graphic has a greater understanding

of statistics, it may

be less confusing if alternative phrases are used to describe the results, such as “credible in terval” or “probability

band”. Graphic s that show pro

bability density functions for u

ncertain ty (PD Fu’s) ar

e gene rally m ore meaningful to

a technical audience of risk assessors and u ncertainty analysts. Alternative g raphics may be need ed to co mmunicate other s ource s of unc ertai nty in ri sk esti mates (e .g., use of alter native probability mo dels for exposure variables, effect of chang es in the model time step, application of spatial weighting to concentration data, etc.). Additional inform ation on comm

unicating risks to

senior EPA managers is given by

Bloom et al. (1993). The results from the sen sitivity analy sis may b e useful to the senior m anagers in deciding whethe r additi onal sam

pling is neces sary. One is sue that may be im portant to addr ess with risk m anage rs and senior staff is that the width of the credible interv al (e.g., 5 th to 95 th percentiles of uncertainty ) will be determined in part by the num ber of sources of uncertainty that are quantified. A s additional sources of uncerta inty a re quan tified an d includ ed in the mod

el, the interval aroun

d the ri sk distribution will t end to widen. This sit

uation m ay app ear to b e count

erintuit ive for those mana

gers w ho expe ct confi dence t

o increase as uncertainty is quantified. Howev er, by uncov

ering and quantify

ing the sources of uncertainty, the benefits in the risk com munication and d

ecision-m aking process should become clear. The results of the sensitivity analysis should help to focus discussions, data collection efforts, and analy ses on t

he more sig

nificant source s of unc ertainty . In addition, by

devel oping

estimates of credible intervals of uncertainty

in risk estim

ates, the decision-making process using the tiered approach ma y becom e mor e transp arent.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-19 6.4.3 P RESS RELEASES For a press briefing presentation, care should b e given to identify messages and develop publi catio n quali ty graphi cs with clear descri ption s that can be pr ovided i n press packages . It is usua lly a good idea to provide the grap hics in both color and black and white so that the press can ch oose the most appropriate presentation sty le for the story. The RPMs generally should work with the CI C, the press sta ff i n the Communi cat ion D ivis ion, and s eni or man ager s to d evel op pr ess mater ial s. Ade quat e time should be left f

or the preparati on of m

aterials and int ernal Agency

revie w and a pproval befor

e information is released. 

### 6.5 P ERCEPTIO N OF 

### RISK 

### A ND 

### U NCERTAINTY The purpose of this section is to present current think ing about how people view risk and uncertainty. This section should provide use ful information for p lanning risk communication and addresses the first step in the seven step process (Section 6.2.2), “Identify the Audienc e.” There are many indiv idual di fference s in the

way people regard

the risk s and hazards

that are present in modern

life. These differences hav e their roots in the differences in perception of risk and uncertainty of the ind

ividual hum

an mind (Slov

ic, 1986). The risk

assessor and/or risk commun icator should keep i

n min d the g eneral p erceptions abou t risk held by

differe nt gro ups. C omm

unicatio ns shou ld be tailored to t

he spec ific aud

ience. This section sum

marizes som

e of the criteria used to judge r isks i n the absence of scientific data and the direction of the p otential bias that may be expected by

applying these criteria. Additional p ublications on this issue are identified in the reference section at the end of this cha pter. In the absenc e of scientific data, the general public evaluates risks using infere nces of judgmen

t as described be low (Sl ovic e t al., 1 979):

• Availability : Peopl e tend t o judge risks as mo re like ly if t

hey ar e easy to reca ll.

• Overconfidence : People tend to be overconfident about the judgments they make b ased on the use of heuristics.

• Desire for Certa

inty : Peopl e tend t

o mis gaug e risk

/benefit conflicts in fav

or of the benef

its as a res ult of

a desir e for ce rtain ty and anx iety abo ut unce rtain ty.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-20 Slovic et al. (197 9) identified nine characteristics of risk that may influence perceptions. These nine dimensions may prov ide a perspective on

whether a health risk

is perceived as “m

ore risky” or “less risky”, as described in the table below.

Dime nsion o f Risk M ore Risky Less Risky
Vo luntariness Involuntary Voluntary Immediacy of

the ef fect Delayed Imm ediate Exposed persons’ knowledge about risk Low High

Sciences’ kno wledge ab out risk Low High Contro l over risk Low High
New ness Unfamili ar or New Famili ar Chro nic/Ca tastrop hic Cata strop hic Chro nic Common/Dread Dreaded Common Severit y of t he consequence s High Low The presentation of uncertainty in a risk estim

ate can be interpreted with vastly different conclusions depending

on the audience and their perceptions. For example, a thorou gh scientific account of multiple sources of uncertainty presented to a group of interested risk assessors a nd environm

ental scientists may be clearly understood . Such a g roup will likely

conclude that the assum

ptions made in the risk a ssessm

ent wer e appro priate a nd that the resu

lts can be used

with co nfidenc e as a d

ecision suppor t tool. In c ontrast, a similar scien tific presentation given to the comm unity may

be misunderstoo d, and the perceiv ed risk may be gr eater. Citize

ns are o ften m

ore concerned about the poten tial im pact to their personal situation, than to the uncertainty in the risk estimate. Consequently, the comm unity may

react negatively

to a long, hig

hly scientific presentation on uncertainty. A good rule of th

umb is to lim

it the presentation to no m

ore than 15 m

inutes. Focusing heav

ily on uncertainty

may cau se citizens to conclud e that the risk m

ust be high. They may also conclude that the presenter is incompetent beca use he or she is not sure of anything , or that the presenter is trying

to hide someth

ing by cloak

ing the inform

ation in technical jargon, o r even that the presenter is intentionally a voiding the public’s issues of concern. To the extent possible, te chnical jargon during the presentation

should be avoid

ed or explained. A helpful presentation generally should incorp orate the following steps: (1) present inform ation about the conclusions that can be drawn from the risk assessm

ent; it is extremely

frustrating for decision-m

akers to recei

ve det ailed informa

tion on uncerta inty w ithout c onclusi ons (Ch

un, 19 96); (2) desc ribe the certain ty of t he infor mation that s

upports these c onclusi ons; (3)addres s the un certaint y and its implications for the conclusions; and (4 ) present the inform ation without jargon and in a frank and open manner. Section 6.4 provid es examples of graphics th at may be useful in presenta tions of PRA.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-21 

### 6.6 T RUST AND 

### C REDIBILITY The single mo st imp ortant q uality a prese nter m ay nee d to po ssess in

order t o com munic ate to others i s a sens e of tru st and credibili ty. Tr ust and credibi lity ar

e based on wor king with the com

munit y and providing

thoughtful, accurate

responses to questions a

nd concerns raised by

the comm

unity. Building trust and credibility is imp ortant, whether com municating to

a high- level technical audience,a RPM/decision-m

aker who wishe s to have the "big picture," or the public. Credibility can best b e established through a long history of frank and op en discussions with the comm unity. In addition, a pr

esenter can ga

in credi bility if he or she has the abi lity to

restate the available information so that it addresses the concerns an d interests of an audience. The ability to g arner trust and credibility come s from kn owing the audien

ce, respecting their o

pinion, and com

municating a

t an appropr iate lev el (U.S . EPA, 1994). 

### 6.7 C OMMUNICATIO N 

### I SSUE S FOR 

### RPM S Following the R

PM’s decision to conduct a site-specific PR

A, the level of stakeholder involvem ent in the developm

ent and review o f the PRA should be evaluated. E stablishing the appropriate level of stak

eholder involve

ment ma y include input from

the CIC, risk assessor and appropriate senior m

anagers (e.g. , Section Chief, B ranch Chief, etc.). The level of stakeh older involvem ent may v ary depending

on the site com

plexity and the interest of the comm unity. As an initial step, it may

be appropriate to condu

ct an exploratory ses sion where letters are sent to various stakeholders (e.g ., environm ental groups, CAG, etc.) inviting their participation in a g eneral meeting on the topic of PRA. If there is a strong interest am ong the stak

eholders, then a m

ore involved commun ication plan may

be appropriate including , but not limited to the following steps:

• Providing stak

eholders with an introductio n to the principles of PR

A in an inform

al session (e.g. , publi c avai lability

session ).

• Provi ding a

draft S cope of

Work ( SOW) to

interes ted stak

eholders followed shor

tly thereaft er by an ava

ilability session to disc uss com ments on the docum

ent.

• Providing a period of time for the stakeholders to rev iew and comm

ent on the selected distributions, including an availability ses sion for discussions with EPA staff where the commun ity may help to identify k

ey site-spe cific information such as exposure factors and receptor behav

ior.

• Provi ding t

he opportunity

for EP A risk

assessor to meet with

the TAG grant ee (if appropr iate) and stak

eholders to ask

questions reg arding

the SO W.

• Providing a revised SOW includin g a response to stakeholder com ments.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-22 • Provi ding a

n ove rview of the f inal PR A at a p ublic m eeting and providing

appropriate support ing P RA doc

umen ts in the reposit ories for stak

eholder revie w and c omm

ent. Thi s session may

be part of the g eneral session regarding the remedial inv

estigation when the risk assessm ent is d iscusse d. Bas ed on the com plexity of the P RA, it

may be appropriate

to hold a public avail ability

session where the stak

eholders (inclu

ding t he TAG g

rantee), if appropriate, are able to m eet with EPA staff to ask questions and offer suggestions regarding the doc umen t.

• Providing a response to co mments from stakeholders regarding the PRA. If the level of interest is low, then a less extensive C IP m

ay be appropriate. In this case, fact sheets (in plain langu

age) describing the general principles o f PRA to the stak eholders and the k

ey findings of the PR

A may be provided (U. S. EPA, 2000 a). At public m eetings where the risk assessmen t is discussed, a sh ort discussi on of the PRA findings and the ir significanc e may be appropriate. The PRA document should

be made av

ailable in the repositories for rev iew and comm

ent by the stak

eholders. For sites with m

edium interest, a combination of the activities identified abov e may b e appropriate. For exam

ple, it may

be appropriate to hav

e a public availability

session on the principles of PRA and then m

ake the docum

ents available for rev

iew and comm

ent. The RPM should consider a number of ad

ministrative issues in developing the plan for invo lving the stakeholders in the PRA. I ssues to consider include: staff resources, funds for obtaining meeting spac e, ava ila bil ity o f con tra cto r sup port , sign ifi canc e of P RA in deci sio n makin g, and the l engt h of t ime requi red to c omplete the RI/FS. To aid i n reduc ing cost s, it may be a ppropr iate t o combine meetings regarding PR

A and point estim

ate risk assessm

ent based on the close links between the documents.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-23 REFERENCES FOR C HAPTER 6 Bloom, D. L. et al. 1993. C ommunic ating Risk to Senior EPA Policy

Makers: A Focus Group Study. U.S. EPA Office of Air Quality Planning and Standards. Chun, A. 1996 . Strategies for Communicatin g Uncertainty to the Public. I BM Risk Conferen

ce Proceedings, October 31. Helsel, D.R. and

R.M. Hirsch. 199 3. Statistical Methods in W ater Resources . Elsevier Science. Amste rdam. Slovic, P., B . Fischoff, and S. Lichtenstein. 1979. Rating the Risk s. Environment

21(3):14–20 and 36–39. Slovic, P. 19 86. Inform

ing and Educating

the Public About R

isk. Risk Anal . 6(4):403–415. Tufte, E.R. 1983. The Visual Display of Qu antitative Info

rmation. Graphics

Press. Chesire, CT.
U.S. EPA. 1989. Risk Assessment Guida nce for Superfund (RAGS ): Volum e I. Human He alth Evalua tion Man ual (HHEM)(Part A, Baseline Risk Ass essment). Interim Final. Office of Emergency

and Remedial Response, Washing ton, DC. EPA/540/1–89/002. N TIS PB90- 155581.
U.S. EPA. 1 991a. Risk Assessment Guida nce for Superfund (RAGS) , Volum e I: Hum an Hea lth Evalua tion Man ual (HHEM), Part B, D evelopment of Risk- Based Preliminary Remediation Goals. Office of Em

ergency and Remedial Respo nse, Washington, DC. EPA/540/R -92/003. NTIS PB92- 963333.
U.S. EPA. 1 991b. Role of the Baseline R isk Assessment in Super fund Remedy Selection Decisions

. Offic e of Sol id Was te and Eme rgency Res ponse, Washi ngton, DC . OSWER Di recti ve No. 9355.0- 30.
U.S. EPA. 1 994. Seven Cardinal Rules of Risk Communica tion . Office of Policy Analysis. Washing

ton, DC. EPA/OPA

/87/020.
U.S. EPA. 1 998. Superfu nd Com munity Involve ment H andboo k and T

oolkit . Offic e of Emergency

and Remedial Response, EPA 540-R- 98-007.
U.S. EPA. 1 999a. Risk Assessment Guida nce for Superfund: Vo lume I–Human H ealth Evaluation Manual

. Supplem ent to P art A: C ommun ity Invo lvement

in Sup erfund Risk As

sessmen ts

. EPA/540/R- 98/042, March.
U.S. EPA. 1 999b. Superfund Risk Assessm ent and How You C an Help: An Overvie

w . Videotape. September 1999(English ve rsion) and August 2000 (Spanish version). English Version: EPA- 540- V-99- 003, O SWER D irective No. 9 285.7-29B. S panish Version ( norther n Mexica n): EPA-540-

V-00-0 01, OSWER Directiv e No. 9285.7-

40. Available thro

ugh NSCEP : 800.4909.198 or 513.489.8190.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-24 U.S. EPA. 2 000a. El Superfund Hoy D ía. La Estimación d e Reisgos: Cómo L ograr La participatión del la Com unidad. ¿Qué es la Estimación del R iesgo para la Salud H umana? OSWER Directi ve No . 9200 .2-2 6K. Ene

ro. (fact sheet)
U.S. EPA. 2 000b. Superfu nd Risk Assessment an d How You CanHelp . Videotape. (English version only). EPA- 540-V- 99-002, O SWER Directive No. 9285.7- 29A. Available through NSCE P: 800.49 09.198 or 513 .489.8 190, S eptem ber.
U.S. EPA. 2 001 . Early and Meaningful Community Involveme nt . Office of Solid Waste a nd Emerg ency Response. Washing

ton, DC. OSWE

R Directive No. 9230.0-9 9. October 12. Supplemental References Rega rding Risk Communication and P ublic Perception Connelly, N. A. and B.A. Knuth. 1998. Evaluating Risk Commu nication: Examining

Target Audience Percept ions Ab out Fou

r Presen tation F ormat s for Fi sh Con sump tion He alth Ad visory Informa tion. Risk Anal . 18:649–659. Covello, V.T. 1987. Decision Analys is and Risk Management Decision Mak ing: Issue s and Methods. Risk Anal . 7(2):131–139. Deisler, P.E. 1988 . The Risk Manag ement-R isk Assessm

ent Interface. L ast in a Five-Part Series on Cancer Risk A ssessment. Environ . Sci. Techno l. 22:15–19. Fischhoff, B. 1995 . Risk Perception and Comm

unication Unplugged: Twenty Years of Process. Risk An al. 15(2):137–145. Fischhoff, B. 1998 . Comm unicate unto others. Reliab. Eng. Syst. Saf . 59:63–72. Fischhoff, B., A . Bostrom an d M.J. Quadrel. 1997. Chapter 34. Risk Perception and Com

munication. In: Oxford

Textbo ok of P

ublic H ealth , Vol. 2, pp 987–1002. Londo n: Oxford Univ. Press (Ed. R. Defels, et al.). Hora, S.C. 19 92. Acquisition of Ex pert Judgment: E

xamples from

Risk Assessm

ent. J. Energy Eng. 118(2):136–148. Ibre kk, H. and

M.G. M organ . 1987 . Graph ical Comm

unicatio n of Un certain Quantit

ies to Non-Technical Pe ople. Risk Anal . 7:519–529. Johnson, B.B. and P. Slovic. 19 95. Presenting U ncertainty in Health R

isk Assessm

ent: Initial Studies of its E ffects on Risk

Percep tion and

Trust. Risk A nal. 15:485–494. Kaplan, S. 1992. ‘E

xpert Inform

ation’ Versus ‘Expert Opinions. ’ Another Approach to the Problem of Eliciting/Comb ining/Using E xpert Knowledge in P RA. Reliab. Eng. S yst. Sa f. 35:61–72. Morgan, M.G., A. Bostrom, L. Lave an d C. J. Atman . 1992. Com municating R

isk to the Public. Environ . Sci. Techno l. 26(11):2048–2056.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 6 ~ De cem

ber 31, 2001

Page 6-25 Ohania n, E.V. , J.A. M oore, J. R. Fow le, et a l. Work shop O verv

iew. 19 97. Ri sk Characteri zation : A Bridge to I

nformed Decision Making. Fundam . Appl. Toxico l. 39:81–88. Thompson, K.M. and

D.L. Bloom . 2000. Com munication of R

isk Assessm

ent Inform

ation to Risk Managers. J. Risk Res. 3(4):333–352.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-1 

## C HAPTER 

## 7
R OLE OF THE 

## PRA IN 

## D ECISION 

## M AKING


### 7.0 I NTRODUCTI ON When de ciding

whethe r or not to rem ediate a hazar dous w aste site , the ri sk m anage r needsto know if an un

acceptable risk is presen

t, and if so, what cleanup level to app ly to the contam inated media. For this inform ation, the risk

mana ger should tu

rn to the risk

assessor for he lp in in

terpreti ng the results of the r isk as sessment. Thi s chapte r prov ides g uidance on how to inte rpret the result s of a probabil istic risk assessmen

t (PRA) to help determ

ine if an unacceptable risk is present, and the criteria to consider when d erivin g a risk-based prelimi

nary r emed iation goal (PR

G) and a final r

emed ial go al. 

### 7.1 G ENERAL 

### P RINCIPLES OF 

### RISK 

### -B ASED 

### D ECIS ION 

### M AKING 

### I N 

### S UPERF UND Under Agency

policy, an indiv

idual with reasonable m

aximum exposure (RME) will g enerally be the principal basis for ev aluating potential hum

an health risks at S uperfund sites (see Risk Assessment Guidance for Superfun d (Section 6.1.2 of U .S. EPA, 198 9) and the National C ontingency P lan’s (NCP) Pream ble (U. S. EPA , 1990 )). The RME is d efined a s the hi ghest exposu re that i s reason

ably expected

to occur at a sit e, and i s inte nded to estimat e a cons ervati ve expos ure cas e (i.e. , well a bove the average case) that is still within the ran ge of possible exp osures. In general, where cu mulative carcino genic risk to the RME individ

ual is less than 1E- 04, and the non- carcinogenic Hazard Index (HI) is less than or equal to 1, rem

edial action is not warranted under Superfund unle ss there are adverse environmental impacts, or the applicable or relevant a nd appropriate requirem

ents (ARARs) are n

ot met. As discussed in Section 7.2.4, the RME receptor is often (although not always) an appropriate basis for evaluation of risks to ecolo gical receptors, as well. Once a determin

ation of unacceptable risk to humans an

d/or ecological receptors has been made, the risk mana gers w ill typ ically

ask the risk

assessor to dev

elop sit e-sp ecific P RGs. PRGs a re generally defined as health-

based chemical co

ncentrations in an env

ironmental m

edia for which the risk

s (cancer or noncancer) to the RME receptor would not exce ed some specified target level. For systemic or noncarcinogenic toxica

nts, the target risk

level is gene

rally a HI of unity (1). This is considered to be a threshold concentration to w hich the human

population (including

sensitive subg

roups) and ecologica

l receptors may

be exposed without adverse effect during less-than- lifetime (i.e., chron ic, subchronic, or short-term)exposu

res. F or carci

nogen s, the t

arget risk le

vel us ed to de

rive the PRG typica

lly rep resents a cumulative lifetime cancer risk to an individual of between 1E- 06 and 1E- 04 (equivalently expressed as 10 -6 and 10 -4 ). Fo r car cin ogeni c ris ks, le ss-th an-li fet ime ex posu res are c onver ted to eq uiva len t li fet ime values (U.S. EPA, 1989). The 1E-06 risk level is specified in the NCP as a poin t of departure for determ ining remed

iation goals when ARA

Rs are n ot ava ilable or not sufficient

ly protectiv e. I

t is important to rem

ember that risk

-based PRG s are initial guidelines an d do not represent final c leanup or remediation levels. Remediation levels are finaliz ed after appropriate analy sis in the remedial investigation/feasibility

study (RI

/FS) and record of dec ision (ROD). A final cleanup level m ay differ from a PRG

based on the risk

manager’s consideration of variou s uncertainties in the risk estimate, the technical feasibility of achie ving the PR

G, and the nine criteria outlined in the NCP (see Chapter 1, Exhibit 1-2 ).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-2 E XHIBI T 7-1
D EFI NIT IONS FOR C HAPTE R 7 Applicabl e or Rele

vant and A

ppropriate Requireme

nts (ARARs) - Federal or state environm ental standards; the NCP states that ARARs should be considered in determining remediation goals. ARA Rs may be se

lected as site-specific cleanup levels. Central Tendency

Exposure

(CTE) - A risk

descriptor

represe nting the av

erage or typica l individual i

n a population, usually considered to be the m

ean or m edian of t he distribution. Confidence Interval - A range of va lues that are likely to include a population parameter. Confidence intervals may describe a param eter of an input var

iable (e.g ., mea

n ingestion r

ate) or output variable

(e.g., 95 th percentil e risk). When used t

o charact erize unc ertainty

in a risk estim ate, it is assum ed that m ethods used to quantify

uncertaint

y in the model inputs are

based on sta

tistical principles s

uch as sa mpling dis

tributions or Ba

yesia n approaches

. For exam ple, give n a random

ly sam pled data s et, a 95% c onfidence interval for the m

ean can be

estim ated by deriving a sam

pling distribut

ion from a Student's

t distribution. Credible Interval - A rang e of va lues that r epresent plausible bounds on a population par amete r. Credible

interval s may describe a param

eter of an input var

iable (e.g ., mea

n ingestion r

ate) or output variable

(e.g., 95 th percentil e risk). T he term is introduce d as an al

ternative to the term

confidenc e interva

l when the

methods used to quantify uncertainty are not based entirely on statistical principles such as sam pling distributions or Bayesian approaches. For example, multiple estimates of an arithmetic mean m

ay be available from

different studies reported in the literature—using professional judgment, these estimates may support a decision to describe a range of possible v

alues f or the arithm

etic m ean. Hazard Index(HI) - T

he sum of more tha n one Haza

rd Quotient f

or multiple

substance s and/or m

ultiple ex posure pathways. The HI is c alculated separately for chronic, subchronic, and shorter-duration exposures. Hazard Q uotient (HQ) - The ratio of a single substance exposure level over a specified time period (e.g., subchronic) to a ref erence dos e (or conce

ntration) f

or that subst

ance deri ved from a sim ilar expos ure period. Preliminary Remediation Goal (PRG) - Initially developed chemical concentration for an environmental medium that is expec ted to be protective of

human hea

lth and ecos

ystem s. PRGs m

ay be de veloped bas ed on applica ble or relevant and appropriate requirements, or exposure scenarios evaluated prior to or as a result of the baseline risk assess ment. (U .S. EPA, 1991a, 1991b)

. Reasonable

Maxim um Exposur

e (RME) - T

he highest

exposure that is re asonably expected to occur at

a site ( U.S. EPA, 19 89). The in tent of the RME is to est imate a conservat ive exposur e case (i.e., well above the average case) that is still within the range of possible exposures. Remedi al Inves

tigation/Fe asibility

Study (RI/FS) - St

udies underta

ken by EPA to deline ate the na ture and ex

tent of contamina

tion, to eva

luate potenti al risk, and to deve lop alterna tives f or cleanup. RME Range - The 90th to 99.9 th percentiles of the risk distribution generated from a PRA, within which an RME risk value m ay be identified. The 95 th percentil e is gene rally recomm

ended as the

starting point for specify

ing the RME r isk in a Sup erfun d PR A. RME Risk

- The estimated risk corresponding to the reasonable maxim um exposure.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-3 Figure 7- 1. Hypo thetical PRA results showing a cum ulative distribution function (CD F) for lifetime excess cancer risk. 

### 7.2 I NTERP RETI NG 

### A RISK 

### D IST RIBUT ION 7.2.1 W HAT I S A D ISTRI BUTION O F RIS K A ND W HAT D OES I T L OOK L IKE ? In the traditi onal po int estim ate risk assessm ent approach, risks to the R ME indiv idual ar e characterized as single point values (e.g ., HI=2, or cancer risk=1E- 05). In the PRA approach, the output of the risk assessment is an estimate of the dist ribution of ri sks across all members of the popu lation. An example is show

n in Figure 7- 1.

In th is exam ple, th e x- axis of Figur

e 7- 1 repres ents the

excesslifetim

e cance r risk level and the

y- axis represents the cumu

lative probability o f the cancer risk lev el within the hypothetical population. The graph also shows

various landm

arks along the distribution curve su ch as the 50 th percentile, the 90 th , 95 th , etc. In this illustration, the 95 th percentile corresponds to a cancer risk of 1. 2E-06.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-4 7.2.2 W HAT IS THE RME R ANGE ? Given a risk

distribution such as show n in Figure 7- 1, what part of the risk distribution should a risk m anage r be con cerned about? As exp lained a bove, the risk

to the R ME receptor is a key factor i n making decisions regarding

the need for action at a Superfund site. E PA’s Guidelines for Exposure Assessm ent (U.S. EPA, 1 992) sta tes that the “hi gh- end” (or RME) o f expos ure for a popul ation oc curs between the 90 th and 99.9 th percentiles, with the 99 .9 th percentile considered a bo unding estim

ate. Similarly, P RAs developed to support RME risk estimates for Sup erfund should reflect this app roach.

L In this guidance, the 9 0 th to 99.9 th percentiles of the risk distribu tion are collectively referred to as the recommended RME range

. In utilizing PRA results to dete rmine if an unaccepta ble risk is present and to develop a PRG which is sufficiently p

rotective, risk m

anagers should add

ress two questions: (1) What per

centile of the r

isk di stributi on will

be sele cted to represen

t the RME receptor? (2) How will inform

ation on uncertainty in the high- end risk estim ates be used in this proc ess? The risk manager may co nsider a number of factors in choosing a specific percentile to represe nt the RME i ndiv idua l. Th is may i ncl ude b oth q uant ita tive info rmat ion a nd pr ofes sio nal jud gment . In particular, risk m

anagers may

need to understand w

hat sources of variab ility and uncertainty are already explicit ly acc ounted for by

the m odeling approa ch and inputs ( i.e., p oint est imate s and/or proba

bility distributions) used to estim ate the risk distribution, and what sources m ay be present but are not quantified. Approaches for selecting an appro priate percentile in hum

an health and ecological risk assessments are des cribed below.
7.2.3. RELATING THE RIS K D ISTRI BUTION TO THE RIS K M ANAGEMENT G OAL FOR H UMAN H EALTH In m ost case s, a rec omm

ended starting

point f or risk mana gement dec isions r

egard ing the RME is the 95 th percentile of the risk distribution. The 95 th percen tile for the risk

distribution is

an appropriate description of high- end exposure as identified by the Presidential/Co ngressional Com

mission on Risk Assessment and Risk Managem ent (1997).

L In human health PR

A, a recommende

d starting point for risk m anagement decisions regarding the R ME is the 95 th percentile of the risk distribu tion. Figure 7- 2 illustrates this approach for a site where cancer risk s are the risk driv

er. Assume the risk manager has selected an excess cancer risk of 1E- 05 as the risk m anagement goal, and the 95 th percentile as the definition o f the RME. I f line B on the g raph represents a 1E- 05 probability of cancer, a no-action d

ecision may be warranted because the 95 th percentile of the risk distribution is below the cancer risk lev

el of concern. Con

versely, if we were to assum e that the 95 th percentile is above the risk level of co

ncern (i.e., line A on the graph repre sents 1E-05), remedial action m ay be warranted.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-5 Figure 7-2. Example of a pr obabil ity di str ibut ion f or ri sk i ll ust

rat ing the 95 th

percentile and two different

risk l evels of concern (A and B). Assuming the 95 th perc entile co rresp ond s to the RME, the n eed f

or reme dial act ion de pends onhow the RM E risk comp ares with the risk level of concern. For Case A (RM E > level of concern), remedi al a cti on may be warr ante d. For Case B (R M E < level o f conc ern), re med ial action may b e unne cessa ry.

Although the 95 th percentile is recomm ended as a starting point for defining the RME in the majority of hum

an health risk assessm

ents conducted within the Superfund progra

m, the risk

manager may use discretion in selecting a different percentile within the R ME range (90 th to 99.9 th perc ent ile s). In situations where the risk

manager believes that a sufficient amount of site- specific information ha s been collecte d to ind

icate that the ri sk est

imate s are m

uch m ore lik ely to be hig h (e.g ., overestim

ated due to multiple health protectiv

e inputs), the risk manager m ay choose a low

er percentile within the recommen

ded RME risk range (e.g., the 9 0 th ) as the most rep resentative of the RME

estimate at the site. Conversely, when the risk m anager believes that the risk estim ates may ten

d to underestimate true risks, or if there is substantial uncertain ty in the accuracy

of the risk estim

ates, the risk m

anager may

choose a percentile higher than the 95 th in the r ecommende d RME ri sk range (e.g., the 98 th or the 99 th ). There are a variety of factors that can be considered w hen makin

g this decision, su

ch as the qualitative and quantitative uncertainty

in the exposure assessm

ent calculations, the uncertain ty in the toxicity

values, and the presence of bi ologic al or m easured data (in contrast to m

odeled data). These factors are discussed below in Section 7.3. I t is highly rec ommend

ed that the risk m

anager consult with the site risk assessor when apply

ing these factors to d

etermine an approp

riate percentile in the RME risk range.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-6 7.2.4 RELATING THE RIS K D ISTRI BUTION TO THE RIS K M ANAGEMENT G OAL FOR E COLOGICAL RIS K A SSESS MENT For ecological risk

assessments, the choice of the percentile of the variability distribu tion for exposure or risk tha

t will be protective d

epends on the receptor th at is being considered

as well as the nature of the endpoint u sed to establish the level of concern. For most species, the risk management objective will generally

be to ensure population

sustainability, even if some indiv idual mem

bers of the population (those at the u pper end of the expos

ure or risk distribution) may expe rience a higher risk

of adver se effects. The risk m anage ment goal o f popul ation st ability does not nece ssarily

corresp ond to protection of the central tende ncy receptor at or b elow the regulatory level of concern. As indicated in Chapter 4, without knowledge of the pro portion of the local popu lation that must survive and reproduce for the population to be stable, the cho ice of the central tendency exposure (CTE) receptor as the basis of the risk mana

gement goal may n

ot be protective. Sustainability of a loca l population often depends upon the amou nt of “reserves” with

in that subpopulation to fill in ecological niches left voided by toxicologically impaired indiv

iduals. At a very small nu mber of sites, a population biologist ma

y be a ble to provide

inform ation ab out the level of effect associ ated wi th a decrease in population sustainability. At the majority of sites, the use of the CTE receptor by risk management as the basis for adequate protection of local populations of eco logical receptors cannot be supported. Therefore, in the absence of such species-spe cific (trophic level) inform ation, it is prudent and appropriate to base PRG s and cleanup levels on the upper end of the distribution of variability in the Hazard Quotient (H Q) to provide g reater confidence that the receptor population of concern will be protected. For threatened or endangered species, it will norm ally be appropriate to provide protection to as high a percen tile of t he distr ibution (i.e., the RME

receptor) as is practica

ble (e.g ., hig

h-en d of the RME range of 90 th to 99.9 th percentiles), since injury to even a single individual is undes irable. 

### 7.3 F ACTORS T O 

### C ONSI DER IN 

### C HOOSING THE 

### P ERCENTILE FOR THE 

### RME
Risk assessm

ents (both poin

t estim ate and PRA) should be base d on th e best q uality data available. A k

ey compo nent of the risk m

anagement process is a careful rev iew and evaluation

of the potential limitations in the quality and relev ance of the data that are used in the risk as sessment (i.e., qualitative and quantitativ e uncertainties) in order to ev aluate the strengths an

d weaknesses of the assessment (U.S . EPA, 1993). Comm unication between risk

managers, risk assessors, and other technical team m

embers is v

ital at this stage. The m

ain question to be answered is, “How we ll do the inputs t o the ri sk ass essme

nt repre sent exp osure p athway s and b ehavi ors at a giv en site? ” The an swer to this question can be exp

ressed qualitatively (e.g., hig h, medium , or low) or quantitativ ely (e.g., confidence intervals or credible intervals). S ome exam ples of these types of evaluation are illustrated below. Use of Default Exp osure Distributions When sit e-sp ecific d

ata are n ot ava ilable, the best avai lable informa tion on some exposu re parameters m

ost lik ely will be fr

om st udies at other s ites (e. g., i

n other parts of

the cou ntry). In both point estimate risk

assessment and PRA, the use of surrogate data to sup port input parameters

raises questions about representativ eness for both current and future land use scenarios. A specific exam ple of potenti ally p oor representat ivene ss woul d be the

use of nationa

l data fo r estim ating the exp osureRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-7 E XHIBI T 7-2
E XAMPLES OF D EM OG RA PH IC , C ULTURAL , AND B EHAVIORAL F ACTORS THAT C AN A FFECT E XPOS URE

C Subsist ence fi shing , hunt ing, or inges tion of home-grown pr oduce

C Exp osure s to cultur al food s or m edicin es that co ntain co ntamin ants

C Preparation of foods in containers t hat contain contaminant s that may l each out into food or beverage

C Ho bbie s and other p erson al pra ctices re sulting in ex pos ure to c ontam inants

C Age of

the popula ti on (e. g., chi ldre n may h ave gr eate r expos ure a nd sus cepti bil ity than adu lt s (U. S. EPA, 1995b, 1996)

E XHIBI T 7-3
E XAMPLES OF P HYSI CAL OR G EOGRAPHICAL F ACTORS THAT C AN A FFECT E XPOS URE

C Geo graphical features that limit or enhan ce accessa bility (e.g., sl opes, valleys, mo untains)

C Land use, inclu ding w here e xpo sure o ccurs within the e xpo sure un it, and the curre nt or futur e ma nner in which the rec epto r con tacts the c ontam inated med ia

C Ava ilability of co ntamin ated m edium for exp osure (e.g., gra ss vs. ba re soil)

C De pth o f conta minatio n (e.g., sur face so il is of greate st conc ern for direc t conta ct)

C Bioavail abili ty of

contamin ant f

rom media or wat er (e. g., physi ochemical fact ors th at enhan ce or reduce absorption)

C W ater qual ity

and dist ribut ion sy stems, incl uding wat er hardnes s and use of lea d-solder ed pipes

C Te mpo rary ba rriers (e .g., fences , groun d co ver, an d co ncrete ) that affec t curren t (but no t neces sarily fut ure) exposures frequency of adult w

orkers when the receptor of concern is a ra ilroad worker. R

ailroad workers m

ay typica lly be on the site for only 1 00 day s/year . If the risk assessm ent wer e based on the nationa

l defaul t assumption of 25 0 days/year, this choice would give a high bias to the risk estim ate. Another example

of a site-specific exposure factor that m ay vary considerably am

ong different locations is fish ingestion rates. At sites where

ingestion of fish co

ntaminated with m

etals poses a concern, tissue concentration s from fish fillets collecte d on site are often used to determine the concentration term. However, a cultural practice of people harv esting fish on site m

ay include consum ing some of

the internal or

gans o f the fish in add

ition to the fill ets. I f the m

etal con taminants selectiv ely ac cumu

late in t he inter nal org ans inst

ead of t he fillet tissues , use o f data only on fillets contaminants wou

ld give a low bias to the risk estimate.
Other Factors that In

fluence Site-Specific Expo sures Exhibits 7-2 and 7-3 list other types of factors that may be important to con sider when evaluatin g the representativeness of an exposure or risk model. Given the source of the available data, the risk assessor should

identif y pote ntial un certaint ies and discuss

the lik elihood that th e valu

es used mayunderor ove restim ate actu al site- specific

exposu res. The risk m anage r should consid er this i nformation in decision mak

ing th roughout the tiered process f or PRA (see Ch apter 2)

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-8 For example, the features of a potentially exposed population an

d the physical and

geographical factors at a site can increase or decrease exposure to c ontaminated m edia. These factors should be considered in defining

exposure pathway s and characterizing exposure variables in the risk assessm ent. Such site-specific information m ay support a decisio

n to evaluate the e

ntire RME range (90 th to 99.9 th percentile) before selecting the percentile that represents R ME risk. A d

eparture from the 95 th percentile would depend on whether or not qualitative or quantitativ e factors sugg est an increased or decreas ed expo sure, a

nd henc e, risk . In

practic e, mu

ltiple an d sometime s com

peting factors may need to be bala nced in order t o determ ine an a ppropri ate perc entile fo

r the RME risk

(see hy potheti cal exam

ple in Section 7.5). Subpop ulati ons may be at incre ased ri sk from che mical ex posur es due t o incre ased se nsiti vity, behavior patterns that result in high expo sures, and/or current or p ast exposures from other sources. Environmen

tal health threats to children are a particular concern (U. S. EPA, 1995 b, 1996). Once identified, a subgrou

p can be treated as a population in itself, and c haracterized in the sam

e way as the larger popula tion usi ng sim ilar des criptors

for pop ulation and ind ividua l risk (U.S. EPA, 1 995a). This principle applies to both po int estimate risk assessments and P RA. Use of Biologi

cal Dat a Biological m

onitoring data and/or other biomark er data can be useful so

urces of information for evaluating unc ertainty in an expos ure or risk assessm

ent. These data can prov ide an indication of the mag nitude o f curren

t or pas t exposures and the de gree t o which the exp osures a

re corre lated with contam inated s ite me dia. E

xamp les of b iologi cal data that are useful in hum

an heal th asses sment s include lead in blood, trichloroethylene and its metabolites in blood or urine, arsenic or m

ethyl parathion metabolites in urine, and polychlorinated biphenyls (PCB s) or dioxins in blood or fat tissue. Tissue burdens of contam

inants are also widely

useful as bioma

rkers of exposure in

ecological risk assessments. Just as ai r or g roundwater m

onitorin g data can provide i

ncrease d (or de creased

) confid ence in the results of predictive air or g

roundwater mod

els, biomark

ers can be used in a similar mann er to evaluate how much confidence should be placed in pre dictive exposure assessment mo dels. Biological data can be subject to the same sho

rtcomings as other exposure data in term s of data quality an

d representativeness. The desig n and p erform

ance of the bio logica

l data c ollection effort

gener ally should b e carefu

lly evaluated for these factors (e.g., low , medium , and high qu ality or confidence; low

or high bias, etc.) before using the resu

lts in the risk decision . Currently, c ollection of biological m onitoring data is lim

ited at Superfund sit es and requires coordination wi th appropriat e agencies outside of EPA.
Issues Related to Tox

icity Factors
A var iety o f factor s may

affect t he ma gnitud e of adv erse res ponses expecte d to occ

ur in si milarl y exposed individuals such as age, p hysiological status,nutritional status, and genotype. I n general, these sources of inter-in

dividual variability , and related uncertainties, are taken into accou nt in the derivation of toxicity value s (e.g., reference concentration (RfC), referenc e dose (RfD), and carcinogenic slope factor ( CSF)) u sed in h

uman health risk a ssessm

ents. Th us, hum an heal th toxic ity values us

ually are derived to be health-protective for the most sensitiv e populations.

L Sources of variability or uncertainty are often accoun ted for in the derivation of toxicity values. The level of protectiveness afforded by the toxicity value m ay be a n impor

tant fac tor in d

eciding on the approp riate RME risk percentile to use.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-9 E XHIBI T 7-4
E XAMPLES OF T OXICITY C ONSI DERATI ONS

C Ho w seve re is the effe ct?

C Is the eff

ect rev ersi ble?

C Ho w steep is the slope o f the dose-respo nse curve at low dose?

C Is the contaminant persistent in the environm ent or in recep tors?

C Do es the co ntamin ant bio con centra te as it moves through the food chain?

C Ho w bio availab le is the co ntamin ant? Risk man agers, in collaboration

with risk assessors, should carefully consider whether the toxicity value is representative of the po pulation of concern. F or example, the toxicity value m ay be based on oral exposures to drinking water, whereas exposure to a site population being evaluated m

ay be via soil ingestion . Similarly , the toxicity v alue could be based on effects in a healthy worker population , whereas the site population encompasses all ages and a range of individual health conditions. Uncertainty in toxicity

values m ay reflect insufficient data to evaluate developmental toxicity concerns or to account for in utero exposures. Also, it may be unclear whether the pop ulation of concern has sim ilar characte ristics t o the se nsitiv e popul ation ac counted for in t he deriv ation of the toxicity v

alue. Th is determination m

ay require coordination

with a toxicologist to review the basis for the derivation of the toxicity values in

question. Even then, in m ost cases, the determ

ination will be very

difficult, because our understanding o f human variability in toxicologic responses is very limited for m

any chem icals. When data are insufficient to su pport a more q

uantitative representation of these sources of inter-individu

al variability an uncertainty factor m ay be used in the derivation of non-cancer hum an health toxicity values (RfD, RfC).
Some of the s ame f

actors t hat should be c onsider

ed whe n emp loying

toxicit y va lues to estimate risk are also relevant to the use of toxicok inetic and toxicodyn

amic mo deling in risk assessment. Fo r example, a toxicity assessmen t for methylm

ercury used a techn

ique called benchm

ark dose m

odeling (BMD) to relate the levels in maternal bloo d to adverse dev elopmental effects, b ased on data from a large epidemio logy study of Faroes I slanders (Grandjean et al., 19 97; Budtz- Jørgensen et al., 2000 ). The RfD determined is well-supported by the other large human studies from the Sey chelles (Davidson et al., 1995, 1998) and New Zealand (Kjellstrom et al., 1986, 1989 ) as well as a phy siologically- based pharmacokinetic (PBPK) model based on the Sey chelles data (Clewell et al.,1999). The RfD o btained with be nchmark do se mo deling

(BMD) w

as 1E- 04 mg

/kg- day. The PBP K mode l incorp orated v ariabili ty in toxicokinetics to obtain a range of acc eptable intakes of m ethylmercury

between 1E- 04 and 3E- 04 mg/ kg- day. Althou

gh the PBPK m odel wa s not us ed in the derivation of the ben chma rk dos e valu

e, it was used to support the choice of uncertainty factors in the derivatio

n of the RfD. At the time this guidance was finalized, the understan

ding of this ty

pe of toxic ity inf

ormatio n (i.e. , human vari abili ty) was not well developed. Althoug h such infor mation wa s not us ed to ch aract erize variability in hum

an health risks, the estimates of variability from the PBPK model did prov

ide additional informa

tion on uncertainty. For decision mak ers, the toxicity data and the choice of the endpoint (e.g, neurodev

elopmental effects in the case of methylm

ercury) can gu

ide qualitative risk managem ent choices regarding

the percentile representing the RME (within the 90 th to 99.9 th percentile range) and /or the appropr iate lev el of confidenc e in the

RME estimate. Exhib it 7-4 lists som e of the issues to consider when evaluating the uncertainty in a toxicity

value.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-10 Figure 7- 3. Box and whi sker plot s cha ract eri zing uncer tai nty in t he RME risk

est imat es ( 95 th

percentile of t

he Hazard I ndex) at f

our l ocati ons. The box re prese nts the int er- quart il e ran ge (25 th to 75 th percentiles) while the whiskers represent

the 90% credible interval (5 th to 95 th perc entiles). Use of Quantitative U

ncertainty Estimates
PRA m ethods such as

a two- dimen sional M onte Ca rlo anal ysis ( 2-DMCA) m

ay be used to quantif y the uncerta

inty or confid

ence su rroundi ng risk esti

mates , and t

his informatio

n may be help ful in selecting the RME risk

percentile. Figure

7-3 prov ides hypothetical results of a 2-D MCA where a credible interval has be en quantified for a 95 th percen tile of v ariabili ty in n oncance r HI . In exposu re units (EU) 1 and 3, the credible intervals fo r the 95 th percen tile a re fai rly narr ow, whic h suggest s a high degree of confidence

that the risks in E

U1 are negligible and that the risks in EU3 are unacce ptable. Conversely, the relatively w ide credible intervals in EU2 and EU4 give less confidence in the results, but suggest that the 95 th percentiles likely exceed a target HI of 1 in both cases. Further efforts to reduce o r characterize uncertainties m ay affect the risk

managem

ent decision in these two

areas.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-11 Summary: Multiple Criteria Form the Basis of th e Remedial Decision Final risk m

anagement decisions should be base d on a weighte d consideration of all of the relevant factors that influen ce confidence in the risk

distribution. For exam

ple, a risk m

anager may

be presented with a risk

assessment for a heavy m etal in residential soil in which the distribution of cancer risk estimates in

the RME range (i.e., 90 th to 99.9 th percen tiles) ov erlaps t he risk range of conc ern (1E- 06 to 1 E-04 ). The r

isk m anage r then should proceed with th e site te chnical team to ev aluate t

he data availa ble to d efine in puts for the risk assessm ent, as well as the site -spe cific factors, an d the av ailable biological mo

nitoring data. A ssume that sev eral factors that are likely to increase the confidence in the risk estimates were noted: (1) the soil collection and analysis effort was well-desig ned; (2) the predominant chem

ical and physical form

s of the metal in the soil are characterized by relatively low bioav ailabili

ty; (3 ) all of the ya

rds in t he resid ential neighb orhood

are cov ered wi th gra ss lawn s, a fea

ture gener ally expected

to redu ce direct exposure to s oil; and (4) biomonit

oring data from the

site are all within normal ph

ysiological ranges, suggesting little, if any, exce ss contaminant ex posure occurred at the site. In ad

dition, generic nation

al data were used in the absence of site- specific information on two input variables that rank

ed highest in the sensitivity analy sis, thereby reducing confidence in the risk

estimates. In this exam

ple, the consideration of these factors collectively suggests that the results of the risk assessment are lik

ely biased towards an overestima te of risk, and th

is information m

ay be used in a

risk managem ent selection of a percentile o f the risk distribution to represent the RME receptor (e.g., less than or equal to the 95 th percentile)

. 

### 7.4 U NCERTAI NTY 

### A SSOCIATED WITH THE 

### U SE OF THE 

### 99.9 TH 

### P ERCENTILE As previously

stated, this guidan

ce adopts the 90 th to 99.9 th percentiles of the risk distribution as the recomm

ended RME risk ra

nge for decision-

making purposes, consistent w

ith EPA’s Guidelines for Exposu re Asse

ssment (U.S. EPA, 1 992). A cautionary note should be adde d about the selection of the higher percentiles within that range, espec ially the 99.9 th percentile. The extrem e percentiles (“tails”) of an inpu t distrib ution ar e understandab ly the

most uncerta in part of a PD F, sinc e the nu mber of data values in these ranges are less abundant than in the center of the range. This uncertainty in the tails of the input distributions leads in turn to greater unc ertainty in the tails of the calculated exposure or risk distribution, and the m

agnitude of this un

certainty increases rapidly at the very high percentiles. I n many cases, estimates at the extreme tails, su ch as the 99.9 th percentile, may be neither accurate nor p lausible. For that reason, g

reat care should be tak

en when evaluatin

g an RME risk

in the upper percentiles o f the risk range. 

### 7.5 M OVING 

### F ROM 

### A PRG T O 

### A REMEDIAL 

### G OAL As discussed above, where an unacceptable risk is identified, the risk assessor is ty pically asked to develop site- specific PRGs (see Ch apter 5 for discussion on derivation of PRG s). PRGs m ay be developed using

a probabilistic approach m uch in the same

manner as they

are developed us ing a point estimate approach. The target risk lev el should be set for a specified percentile (corresponding to the RME receptor), and the concentration in contam inated media wh

ich corresponds with that target risk level should be calcu

lated. It is

impo rtant to unders tand tha

t the PR G is an early step, n ot the l ast step , in the selection of a final cleanu p level. During

the RI/FS , the risk m

anager should ev

aluate the remedial alternatives using the nine criteria described in the NCP (U.S. EPA, 1990) (C hapter 1, Exhibit 1-2). Achieving a

target level of protection for huma n and/or ecological receptors is one of the prim ary factors, but this objective shoul d be ba

lanced by cri

teria su ch as fe asibility, perm

anence, state an

d com munit y acceptance, and cost. Indeed, there may b e times when a purely risk- based PRG m

ay be imp racticable asRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-12 a final cleanup goal. In cases such as this, it is im portant to remem

ber that the RME is not a single, fixed perce ntile on the ri sk distr ibuti on, but inste ad repr esent s the po rtion of the r isk dist ribut ion cur ve between the 90 th and 99.9 th percentiles. Depending on the specific exposure and toxicity inform

ation available at a site, a PRG developed using the 90 th percentile of risk m ay be sufficient to p

rotect the reasona bly m aximu m exp osed individu

al. Al ternativ ely, a t some sites, the risk

mana ger m ay fee l that a PRG developed

using even

the 95 th percentile of risk is n ot sufficiently protective of the RME indiv idual and thus may

choose to develo

p a PRG using

a higher percentile.

L Selection of f

inal remediati on or cleanup level s during the RI/FS andROD may be an iterative pro cess, and may con sider a range of factors in addition to the i nitial P RG esti mate

. For example, at a former nuclear energy site, a PRG of 200 p icocuries/gram (p Ci/g) was developed for pluton

ium in soil based

on a one- dimensional Monte C arlo analysis (1- D MCA) and the recommen

ded starting point of the 95 th percentile for the RME ind ividual. At this particular site, the surrounding com

munities were stron

gly opposed

to this PRG as a cleanup level. They felt it was not adequately protective, and as a result, lim ited progress occurred in remediating the site over the y ears. The communities pointe d out to the risk manager that many of the expos ure assumptions used in the PRA were not site-sp ecific, and some members of the comm unity felt that exposu

res occurred more often (i.e, with higher frequenc

y) and for a longer period of time (i.e., for a greater duration) than were assum ed. Based on the exposu

re parameters recom

mended by

the comm

unity, the PRG

would have b een 75 pCi/g. At this point, the risk

mana ger co uld hav e chose n to eit her go back and col lect suf

ficient s ite-s

pecific demographic

and exposure data to refine the risk calculations and the PRG deriv ation, or evaluate the feasibility of a PRG

associated with hig

her percentiles on the risk distribution curve (e. g., 99 th percentile). I

n this particular exam

ple, the risk m

anager com

pared the costs associated w ith the cleanup that wo uld be r equired to satis fy the

comm unity concern s with t

he costs associa ted with collection of additional data an d recalculation of the risk and PRG. The risk

manager decided that the additional cost of cleanup was manag eable and expected that the PRG based on the 99 th percen tile wou ld be acce pted by the com munit y. I n additi on, rem

edial activity

could b egin q

uickly withou t mor e invest igation. When

the risk mana ger pr esented these f inding s to the

comm unity, the citizens q

uickly agreed with this ap

proach and reme diation activities m oved forward.
How does Variabilit

y and Uncertainty in Risk Relate to the Choi ce of a PRG? An effe ctive approac

h for comm

unicatin g the results of a pro

babilist ic analysis to ri sk m

anage rs is to develop g

raphics that relate variability and uncertainty in risk to the choice o f a PRG. Two g raphics are illustrated in

Figu res 7- 4 and 7 -5, based on the concept o f iterati ve sim ulation s presen

ted in C hapter 5 (Section 5.5). Cont inuing the PRG example discuss ed above, assume that multiple 1-D MCA simulations are run with PRGs for plutonium rangin g from 25 pCi/g to 250 pCi/g in increm ents of 25 pCi/g. A s the concentration term is changed to c orrespond with a PR G, each Monte Carlo simulation yields a different distribution of risk. Fig

ure 7-4 focus es on the RME range of percentiles from the risk distribution (i.e., 90 th - 99.9 th percentiles). A risk manager might use this graphic to ev aluate how the PRG could change based on the choice of the percentile used to re present the RME. A

hypothetical risk level of concern of 1E-05 corresp onds with the 90 th percen tile a t a PRG of a pproxi mately 125 pCi/g, whereas 1E- 05 intersects the 95 th percentile line at a PRG of approximately

75 pCi/g. Therefore, when variability in risk

is the focus of the de cision, the difference betwe en an RME set at the 95 th percen tile instead of the 90 th percen tile i s 50 pCi/ g.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-13 Figure 7- 5 presents information on uncertainty, ra

ther than variability . This graphic could be used to summ

arize results of a 2-

D MCA (see Appen

dix D), or a series of 1-D MCA simulations (see Chapter 3, Section 3 .4) applied to the sam e range of PRG s evaluated in Fig ure 7-4. In this case, the results yield a 90% cred

ible interval (CI

) for the risk distribution . Figure 7- 5 highlights the 90% CI for the 95 th percentile, assumin g that a risk manager sele cts the 95 th percentile to represent the R ME risk, and she is interested in the unc ertainty in the risk estimates. Us ing the sam e hypothetical risk

level of concern (1E-0 5), the 90% upper CI for the 95 th percentile corresponds with 1E-05 at a PRG of approx imatel y 25 pCi/g. The ri sk manager may ne ed to co nside r the co st and f easib ilit y of achi eving a PRG as low as 2 5 pCi/g. I n addition, the 90% lower CI correspo nds to a PRG of 250 pCi/g. The risk mana ger m ay det ermine that th

is rang e of unc ertainty

(i.e., an orde

r of magnitu

de) is to o wide to set a PRG, and tha

t furthe r steps are need

ed to re duce id

entify the m ajor sour ces (i.e ., sens itivity

analy sis). Variations on Figures 7-

4 and 7-5 can be developed to focus on different perc entiles of the risk range. This inform

ation, together with the results of the sensitiv ity analysis w

hich highlights the major sources of variability

and uncertainty, should help to gu ide the selection of final rem ediation or cleanup levels, or continu ed data collect ion and analysis fol lowing the tiered pr ocess for PRA.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-14 Fig ure 7-4 . Example of gr aphic showi ng var iabi li ty in r isk (i. e., RME range, or

90 th to 99.9 th perc entiles) as socia ted with differen t choic es of P RG for plu tonium in soil (pC i/g). The hypot

heti cal ris k le vel of con cern (1E-05) corre sponds to a 90 th percenti le ri sk at a a PRG of ~ 100 pCi/ g, a nd a 95 th percenti le a t a PRG of ~ 75 pCi/ g. I n th is exampl e, al l of the 99 .9 th

percentiles exceed 1E-05, leaving no choices f or PRG at the hi gh end of the RM E range.

Figure 7- 5. Example of gr aphic showi ng un cert aint y i n 95 th perc entile risk a ssocia ted with the same choices of PRGs given in Figure 7-4. Uncertaint y is gi ven by the 90% upp er and lowe r cred ible inter val (C I). Th e hypo thetical risk level o f conc ern (1 E-0 5) co rresp ond s with the 90% up per CI at a PRG of ~ 25 pCi/g, and the 90% lower CI at a PRG o f ~ 250 pC i/g.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-15 REFERENCES FOR C HAPTER 7 Budtz-Jørg ensen, E., P. Grandjean, N. Keiding, et al. 2000. Ben chmark D ose Calculations of Methylmercury-Associated N eurobehavioral Deficits. Toxicol. Lett. 112–113:193–199. Clewell, H.J., J.M. Gearha rt, P.R. Gentry , et al. 1999. Ev aluation of the Uncertainty in an Oral Reference Dose for Methylm

ercury Due to I

nterindividual Variability in Pharmacok

inetics. Risk An al. 19:547–558. Davidson, P ., G. Myers, C. Co x, et al . 1995 . Long itudina l Neuro devel opme

ntal Stu dy of Seychellois Children Following

in Utero Exposure to

Methylmercury

from Maternal Fish Ingestion: Outcomes at 19and 29 Months. NeuroToxicology 16:677–688. Davidson, P .W., G .J. Mye rs, C. Cox, e

t al. 19 98. Eff

ects of Prenata

l and P ostnata

l Methy lmerc ury Exposure from Fish Consumption on Neurodevelopment: Outcom es at 66 Months of Age in the Seychelles Child D evelopment Study. JAMA 280:701–707. Grandjean, P., P. Weihe , R. White, et al. 1 997. Cognitiv e Deficit in 7-year-old Children with Prenatal Exposure to Methylm

ercury. Neurotoxicol. Teratol. 20:1–12. Kjellstrom , T., P. Kennedy , S. Wa llis, et al. 19 86. Ph

ysical and Men tal Dev elopment of C

hildren with Prenatal Exposure to Mercury from Fish. S tage 1: Prelim inary Test at Age

4. Natl. Swed. Environ. Protec. Bd

. , Rpt 3080 (Solna, Sweden). Kjellstrom , T., P. Kennedy , S. Wa llis, et al. 19 89. Ph

ysical and Men tal Dev elopment of C

hildren with Prenatal Exposure to Mercury from Fish. S tage 2: I

nterviews and psy

chological tests at age 6. Natl. Swed. Enviro n. Prot. Bd. , Rpt 3642 (Solna, Sweden). Presidential/Congressional Commis sion on Risk A ssessment and R isk Managem

ent. 1997. Risk Assessment and Risk Management in Regulatory Decision Making . Final Report, Volum e 2.
U.S. EPA. 1 989. Risk Assessment Guida nce for Superfund (RAGS ): Volum e I. Hu man He alth Evalua tion Man ual (HHEM)(Part A, Baseline Risk Ass essment). Interim Final. Office of Emergency

and Remedial Response, Washing ton, DC. EPA

/540/1-89/002. NTIS PB9 0-155581.
U.S. EPA. 1 990. National Oil an d Hazardous Su

bstances Pollution Contingency Plan. Fina l Rule. 40 CFR 30 0: 55 Federal Register, 8666-88 65, March 8.
U.S. EPA. 1 991a. Role of the Baseline R isk Assessment in Super fund Remedy Selection Decisions

. Offic e of Sol id Was te and Eme rgency Res ponse, Washi ngton, DC . OSWER Di recti ve No. 9355.0- 30.
U.S. EPA. 1 991b. Risk Assessment Guida nce for Superfund (RAGS) ,Volum e I: Hum an Hea lth Evalua tion Man ual (HHEM), Part B, D evelopment of Risk- Based Preliminary Remediation Goals. Office of Em

ergency and Remedial Respo nse, Washington, DC. EPA/540/R -92/003. NTIS PB92- 963333.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Chapt er 7 ~ De cem

ber 31, 2001

Page 7-16 U.S. EPA. 1 992. Final Guidelines for Ex posure Assessment . EPA/600/Z- 92/001. 57 Federal Register, 22888-2293 8, May 29.
U.S. EPA. 1 993. Data Quality Objectives Process for Superfund . Office of Solid Waste a

nd Emergency Response. Washington, DC.
U.S. EPA. 1 995a. Memorandum from C arol Browner on R isk Characterization . Office of the Administrator. Washin

gton, DC. February 22.
U.S. EPA. 1 995b. Memorandum from C arol Browner on P

olicy on Evaluating He

alth Risks to Children

. Office of the Adm

inistrator. Washington, D C. October 20.
U.S. EPA. 1 996. Memorandum from C arol Browner on E

PA’s Report, Environ mental Health Threats to Children . Offic e of the

Adm inistrat or. Wash ington , DC. Septem

ber.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 1 

## A PPENDIX 

## A
S ENSITIVITY 

## A NALYSIS 

## : H OW 

## D O 

## W E 

## KNOW 

## W HAT 

## ’ S 

## I MPORTANT 

## ?
A.0 I NTRODUCTI ON Sensitivity analysis, as it is applied to risk assessment, is a

ny system

atic, comm

on sense technique used to understand how risk estimates and, in p articular, risk- based decisions, are dep endent on variability and u

ncertainty in the factors contributing to risk . In sho rt, sensitivity ana

lysis identifies what is “driv ing” t he risk estimates. I

t is use d in bo th poin

t estim ate and probab ilistic a

pproaches to id

entify and ran k importa nt sour ces of va riabi lity as well as importa nt sour ces of u ncert ainty. T he quan titat ive information prov

ided by sensitiv

ity analysis is im

portant for guiding

the complexity

of the analysis an

d commun icating importan

t results (see Chapter 6). As such, sensitiv ity analysis plays a central role in the tiered process for PRA (see Chapter 2). This A ppendix focuses on a set of graphical and statistical techniques that can be u sed to determine which variables in the risk mo del contribute most to the variation in estim

ates of risk. This v

ariation in risk could represent variability, uncertainty, or both, depending on the

type of risk model and chara cterization of input v

ariables. The re is a wid e arr ay of a nal ytic al met hods that may be r efe rre d to a s sen sit ivit y anal ysis , some of which are very

simple and intuitiv

e. For exam

ple, a risk assessor may hav e two comparab le studies from which to estimate a reasonable maximum exposure (RME) for child hood soil ingestion. One approach to evaluatin

g this uncertainty

would be to calculate the corresponding R ME risk twice, each time u sing a

differe nt plaus ible poi

nt estim ate for s oil ing estion r

ate. S imilar

ly, in a proba bilistic mode l, there may be uncertainty reg arding the choice o f a probability distribution . For exam ple, lognorm

al and gamm a distributions may

be equally plausib le for characterizing variability in an

input variable. A simpl e exploratory

approa ch wou ld be to run sep arate Monte Car

lo sim ulation s with e ach dist

ribution in order to determ ine the effect that this p articula r source of unce rtainty may have on risk

estimates wit

hin the RME range (90 th to 99.9 th percentile, see Chapter 1). Sensitivity ana

lysis can also inv

olve more complex m

athematical and statistical techniques such as correlation and reg

ression analysis to de

termine which facto

rs in a risk m

odel contribute mo st to the variance in the risk

estimate. The com

plexity generally

stems from

the fact that multip le sources of variability and u

ncertainty are influencing a risk estim

ate at the same tim

e, and sources m

ay not act indepen dently . An i

nput variable contributes sig nificant ly to t he outp ut risk distribution if

it is both highly variable and

the variability p

ropagates throug

h the algebraic risk

equation to the m

odel output (i.e., risk). C

hanges to the distribu

tion of a variable w

ith a high sensitiv

ity could have a profound im pact on the r isk est imate, wh ereas even lar ge changes to the di strib ution of a low s ensit ivity vari able may ha ve a minimal impact on the fina l result. Information from sensitivity analysis can be important when trying to determine whe

re to focus additional resourc es. The choice of techniqu e(s) should be determ

ined by the inf ormatio n needs for ris k management d ecisi on making. This app endix presents guida nce on both pr

actical decision mak

ing an d theore tical concepts associated with the sensitiv ity analysis that are comm only applied in risk

assessment. A

n overview of the type o f inform

ation pr ovide d by s ensitiv

ity analysis

is pres ented fi

rst, fol lowed by gu

idance on how

to decide what me

thod to use in each o f the tiers. A straig htforward example of applications of Tier 1 and Tier 2 sensitivity analysis methods is sh own, followed by

a more detailed discussion of the theory and equations associated with the different methods.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 2 E XHIBI T A-1
D EFI NIT IONS FOR A PPENDIX A Continuous Variables - A random variable that can a

ssume any va

lue within an interva

l of rea l numbers (e.g., body weight)

. Correlation

- A quanti

tative e xpression of the sta

tistical associat

ion betwe en two v

ariables ; usually

represe nted by the Pearson correlation coef

ficient for linea

r models , and the Spea

rman ra nk correla

tion coeff icient (see below)

for nonlinear models. Discrete Variables - A random variable that can a

ssume any va

lue within a finite

set of v alues (e. g., numbe

r of vis its to a site in one y

ear) or a t most a countably

infinite set of v

alues, m eaning tha

t you ca n count observ

ations, but the

re is no defined uppe

r limit. An ex

ample of countably

infinite would be the

number of

dust partic les in a v

olume of air (a Poisson distribut ion), whereas uncountably infinite would be the

number of

points in a li

ne segm ent. Local Se nsitivity Analy sis - Evalua

tion of the m

odel sensit

ivity at some

nominal poi

nts within the range

of value s of input variable (s). Monte Carlo A

nalysi s (MCA)

or Monte Car

lo Simulat

ion- The proc ess of r

epeatedly sampli ng from probability dist ribu tion s to d erive a dist ribu tion of out comes. MCA is o ne of se veral t echn ique s that may be use d in P RA. Multiple Reg

ression A nalysi s - A sta

tistical method tha

t describe s the ext

ent, direct

ion, and stre

ngth of the

relationshi

p betwee n severa l (usually continuous) inde

pendent var

iables (e .g., expos

ure duration, i

ngestion ra

te) and a s ingle continuous depende

nt variable (e.g., risk). Nonparam

etric T ests - Statistical test s that do not re

quire assum

ptions about the

form of the popula

tion probability distribution. Range Se nsitivity Analy sis - Evalua

tion of the m

odel sensit

ivity across the

entire ra nge of v alues of the input va

riable(s)

. Rank - If a set of values is so rted in asc ending o rder (smallest to largest) , the rank co rrespon ds to the relative po sition of a number in t

he sequence . For exa

mple, the set {7, 5, 9, 12} when sor ted give s the following sequence {5, 7, 9, 12} with rank s ranging from 1 to 4 (i.e., r

ank of 5 i s 1, rank of

7 is 2, rank

of 9 is 3, a

nd rank of

12 is 4). Sensitivit y Ana lysis - Sensitivity generally re fers to the variation in output of a model with respect to changes in the values of the model’ s input (s). Sensi tivity analysis at tempts to pr ovide a ran king of the mod el inpu ts based o n their re lative contributions to model outp ut variability and uncertainty. Common m etrics of sensitivity include:
< Pearson Correlation Coef ficient - A statistic r

that me asures the

strength a

nd direction of

linear as sociation betwee n the value s of tw

o quantitativ

e varia bles. The square of the coef ficient ( r

2 ) is the fraction of the variance of one va riable tha t is expla ined by least-squar

es regr ession on the other variable, als o called the coeff icient of determ ination..
< Sensitivit

y Rati o - Ratio of

the change in mode l output per unit c hange in a

n input varia

ble; also c alled elasticity

. < Sensitivit

y Score - A sensitivity ratio that is w eighted by some characteristic of the input variable (e.g., variance, coeff icient of variation, range).
< Spearma

n Rank Orde

r Correlat ion Coeff icient - A “distribution free” or nonparametric statistic r that measures the st rengt h and direc tion of asso ciati on b etween the ra nks of t he val ues (n ot th e valu es the mselves ) of two quantitativ

e varia bles. See Pearson (abov

e) for r

2

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 3 E XHIBI T A-2

U TILI TY OF S ENSITI VITY A NA LYS IS

C Decision maki ng with the tiere d approach

e.g., After

quantifyi

ng parameter uncertai nty, we are 95 pe rcent con

fident that the R ME risk is bel ow the ris k level

of concern— no further
ana lysis is nee ded

. Also

— selectio n of a beta dist ribut ion ove r a l ognormal di str ibut ion f

or inges ti on rat e changes the 95 th perce nti le of the ri sk dis tri bution by a fact or of 10—further evaluat ion may be nee ded.

C Resource allocat ion - e.g. , Two of the 10 exposure vari ables contr ibut e 90 perc ent of the vari abili ty i n the ri sk est imate.

C Risk c ommunicat ion - e.g. , For input vari able X, if

we were t

o use a di

str ibut ion bas ed o n site-sp ecific d ata in stead of a nati onal s urvey, we would ex pect a minimal change in the RME risk est imate. 

### A.1.0 U TILITY OF 

### S ENSI TIVI TY 

### A NAL YSIS As hig hlight ed in Exhibit A -2, sensitiv ity analy sis can provid

e valu able inf ormat ion for both risk a ssessor s and ri sk m

anage ment decision mak

ers throughout the tie red process for PRA. By highlighting

important sources of variability and uncerta inty i n the ri sk ass essme

nt, sen sitivit

y anal ysis is generally

an im portant comp onent o f the ov

erall uncertainty analy sis. For exam ple, methods that quantif y para meter uncerta inty a nd mo del unc ertainty may yield differe nt estim ates of the RME

risk. This information can be

used to guide the tiered process by supporting decisions to conduct additional analyses or prioriti ze res ource a

llocations for a

dditional data collection efforts. Results o f sensitivity analysis can also facilitate the risk com munication process by focusing discussions on the important features of the risk assessmen

t (e.g., constraints of available data, state of knowledge, significant scientific issu es, and significant policy choices that were m ade when alternative interpretations of d ata existed). Decision Making with the Tiered Approach In g eneral, the ty

pe of informa

tion provided

by a sensitiv

ity analysis

will vary with each

tier of a PRA. Table A- 1 provides an ov erview of the m

ethods that may

be applied in each tier based on the typ e of information need

ed. In Tier 1,sensitivity analysis typically inv olves changin

g one or m

ore input variables or assum

ptions and evaluating

the corresponding changes in the risk estimates. Ideally, the results for Tier 1 would be usef ul in de ciding

which exposu

re pathways, variab les, and assum ptions a re carried forward for further con sideration in subsequent tiers of analysis. By identifying the variables that are most im

portant in determining

risk, one can also decide whether poin t estimates, rather tha

n probabi lity d

istribut ion functions (PDFs), can be used wi th little conseq uence t o the m odel ou tput. Th is information is im

portant not only for designing 1 -D MCA models of variability, but also for designing more com

plex analyses of un

certainty discussed in Appendix D (e.g ., 2-D MCA models, geostatistical anal ysis , Baye sia n ana lysi s). S ect ion A .2.2 provi des a n over view o f the Tie r 1 met hods and s ome insights regarding

their limitations. Meth

ods associated with Monte Carlo simulations u sed in Tiers 2 and 3 can take advantage of the ability to vary multiple in puts simultaneously

and account for correlations. Sections A. 2.3 and A.3 provide an overview of the sens itivity analy sis methods that c an be applied in a probabilistic ana lysis.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A-4 Ta ble A -1. Overv iewof S ens it iv it y An aly si s Met hods Appl ica ble in Tier s 1, 2, a nd 3 of a PRA. Tier Goal SA Method(s) What to Look For Ration ale
1 Quanti fycontributi ons of each exposure pathway to ri sk, iden ti fymajor and mino r pathw ays Calculate % of total risk from each exposure pathway Exposure pathway s that con tribute a very sm all perc entage (e.g., < 5 %) to total risk Good preli minar y st ep inTier 1 f

or redu cing the number of exp osure variab les to foc us on in subse que nt tiers.

Exposure vari ables t

hat app ear in m ultiple exp osure pathw ays Risk es timat es are l ikel y to be more s ensitive t o variabl es that appe ar in multiple expo sure pathwa ys.

1 Identify

the form of the dose equati on for key pathways Inspe cti on Equation is multi plicative or additive SR values can be determi ned wit h mini mal eff

ort (s ee Ta ble A -3). Fo r multip licative eq uation s, SR= 1.0 fo r all variables in t

he numerator, and SR is a functi on of the percent chan ge for a ll variab les in the d eno minato r. Equation contains variab les with ex pon ents (e.g., powers, square roots) Ou tput is likely to be m ore se nsitive to v ariab les with exponen ts great er than 1. 0.

1 Quanti fycontributi ons of each expo sure va riable to tot al r isk , i denti fymajor and minor var iables Sensitivi ty Ratio (SR), unweigh ted SR = 1.0, or SR is the same for mu ltiple variabl es It’s likely that t his is a multipli cative equation (see above), and the SR approach will not be effecti ve at discrimi nating among relat ive cont

ribut ions. Explore sen sit ivi ty f urther with other method s. SR 1.0 SR may vary as a function of t he % change in the input variable. In t

his si tuation, it can be inf ormative to explore small deviati ons (± 5% ) and large deviations ( min, max) in the input va riable s. SR < 1.0 Implies an inverse r elationship between the input and output variables (e.g. , inputs in the denominator of a risk equati on).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Ta ble A -1. Overv iewof S ens it iv it y An aly si s Met hods Appl ica ble in Tier s 1, 2, a nd 3 of a PRA. Tier Goal SA Method(s) What to Look For Ration ale

Page A- 5 SR=0 Variable probab ly appears in both the numerat or and den omin ator a nd, the refore , cance ls out of the risk eq uation

. Exam ples include exp osure du ration (ED ) in noncanc er risk equati ons, and body we ight (BW ) if inges tion r ate i s expres sed as a func tion of b ody w eight.
1 (con t’d) Q uantify contri butions of each exposure var iable t o total risk Sensitivi ty Ratio (SR), weight ed— also cal led Sensitivi ty Score Diffe rences i n SR based on th e wei ght ing fac tor A more i nfor mati ve appr oach t

han unweig hted SR valu e for those variabl es that have s uff icie nt i nformat ion to def ine a weighting f

actor (e.g., coeffici ent of variat ion or range).

2 Quantify rel ative contributions of exposure pathways to risk 1-D M CA for var iability or un certain ty, with outputs specify ing % contribution of exposure pathw ays Co mpa re me an with high- and low-end perc entiles o f % contribution to risk The % co ntribution of each e xposure pathway will vary as a function of the variability (or uncertainty) in the inputs; exposu re pat

hway s t hat appear to be rel ati vely minor contri butors on average, or from Tier 1 assess ment, may in f act be a m ajor contrib utor to risk und er cer tain exp osure scena rios. The likel ihood that

a pathway

is nonn egli gible (e.g. , > 5%) can be useful information for risk managers.
2 Quantify rel ative contributions of exposure variables to risk 1-D M CA for var iability or un certain ty, Grap hical analysis— scatterp lots of inp uts and output No nlinear relation ship Easy and i ntui tiv e approach tha t may i dentif y rel ations hips t

hat other meth ods coul d miss . May s ugges t t rans forma ti ons of input or output

varia bles (e .g., logari thms, power transf ormat ions ) th at wou ld i mprove c orrel ati on and r egres sion analyses. 1-D MCA, Correl ati on Analy sis usi ng Pear son and /or Spearman R ank Very h igh or l ow corr elation coefficie nts
Diffe rences bet

ween relat ive rankings based on Pearson and Spearman Easy to implement with commercial sof tware; rank orders t

he variabl es based on t

he average contrib ution to varian ce. Diffe rences i n magnit

ude of coef

fici ents are expect ed between Pearson and Spear

man ra nk appr oaches , but rel ati ve orde r of importance is l ikel y to be t he same.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Ta ble A -1. Overv iewof S ens it iv it y An aly si s Met hods Appl ica ble in Tier s 1, 2, a nd 3 of a PRA. Tier Goal SA Method(s) What to Look For Ration ale

Page A- 6 1-D M CA , M ultiple Line ar Regr essi on Analys is ( e.g. , st epwise) Very h igh or l ow regre ssion c oefficien ts
R 2 and adj ust ed R 2 for totalmodel Easy to i mplement wi th commerci al sof tware; gives contr ibut ion t o reduc ti on inres idual sum of squar es ( RSS)
For risk eq uation s with large sets of inp ut variab les, a sma ll subset of input s may be able to explain the majorit y of the varian ce. 2 Quantify rel ative contributions of exposure variables to R M E risk range 1-D MC A; same as previ ous s tep, but f

or subset of risk dist ributi on (e. g., > 90 th percenti le) Difference in rel ative contributions f

or entire risk dist ribution and the RM E range of the risk dist ributi on Variab les may contribute d ifferently t o the high-end o f the risk distribu tion, esp ecially if the inp ut variab les are h ighly skew ed. This situati on would warrant a closer look at the as sumptions regardi ng the e sti mate of t he vari ance, di ffe rences i n the upper tail (high-en d pe rcentiles ) for alter native c hoice s of pro bab ility dist ributi ons, and as sumpt ions ass ociat ed wit h tr uncat ion limits.
1-D M CA, G ood nessof-fit, K-S o r Chisquare; Sort out

put as above; perf

orm GoF on input d istribution only, compari ng su

bset of input values corr espo nding with high- end ris k to subs et corr espo nding with remaind er of risk distri bution GoF result—rejection of null (dis tributi ons are the same) suggests t

he variabl e may be an important contributi ng factor to the RM E risk estima te A sec ond metho d for id entifying va riable s that co ntribute differently at the high-end of the risk distribution. GoF test results sho uld b e interp reted with cau tion be cause a M onte Car lo simu lation w ill genera lly yield large samp le sizes (e .g., n=5,000 iterations), whi ch is more likel y to result in a positi ve GoF test (i.e., rejection of t he null).

3 Quantify rel ative contributions of exposure pathways and vari ables to variabili ty and uncertainty in risk 2-D MCA, s ame sensitivity ana lysis methods as Tier 2 For variab ility, evaluate inn er l oop val ues; for para mete r unce rtainty, eval uate oute r loop values The result s of a s ensitivi ty an alys is depend on t he quest ion th at is being asked abo ut the risk e stimate— are we interested in variab ility or unce rtainty? T he ma jor so urces of varia bility in risk may point t o a different set of input variables than t

he major sources of uncertainty in ris k.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 7 Resource Allocation Decisions regarding

allocation of future resources and data collection efforts to reduce lack of knowledge generally shou ld take into considera

tion the most influe

ntial input factors in the m odel, and the cost of gaining

new information

about the factors. S ensitivity analy sis is a key feature of determining the exp ected v

alue of inform

ation (E VOI) (see Ap

pendix D). O nce a se

nsitivity analysis

is used to identify an input v

ariable as being im

portant, the source of its variability g enerally should be determined. If an input factor has a significant u ncertainty com

ponent, further research an d/or data collection can be conducted to reduce this uncertainty. Red ucing major sourc es of uncertainty, s uch as the mo st relevant probab ilit y model for variabi lity or the par ameter e stimat es for the model , will gen erall y improve confidence in the m

odel output, such as the estimated 95 th percentile of the risk distribution. An input factor may contribute little to the va riability in risk, but greatly to the uncertainty in risk (e.g., the concentration term). Likewise, a v ariable may contribute greatly to the variability in risk, but, bec ause the data are from a well characterized p opulation, the uncertainty is relatively low

(e.g., adult tap

water ingest ion rate ).
An exa mple of the o

utput fr om a 2-DMCA of

uncerta inty a nd va riability (see Ap pendix D) is shown in Figure

A-1. Assume for this example that the decision mak ers choose the 95 th percentile risk as the RME risk, an

d that a sensitivity

analysis is run to identify and quantitativ ely rank the important source( s) of pa

ramet er unce rtainty . The ba

r chart ( top panel) in F igure A-1 indicat es that the mean soil concentration contributes m ost to the uncertainty in the 95 th percentile risk estim ate. In ad dition, the mean exposu re frequ ency i s a gr

eater so urce of uncerta

inty t han the standar d deviation exposure frequency. Since

both the samp le size and variance impact the m agnitude of the confidence limits for an arithm etic mean soil

concen tration, one wa

y to re duce the confid

ence lim its (i.e ., the uncerta

inty) would be to co llect ad

ditiona l soil sa mples . As shown by the box- and-whisk er plots (bottom panel) in Figu re A-1 , inc rea sin g the s ample size (fr om n =25 to n =50) reduced the 90% confidence limits for the 95 th percentile risk to belo w 1E-05, assuming the additional observatio ns support the sam

e estimate of the mean and stand

ard deviation as the original samp le. Although the un

certainty in a risk

estimate can be reduced by further data collection if the sensitive input distribution represents uncertainty, this is not necessarily true for input distributions that represent variability. For example , variability in the distribution of body weights can be better characterized with add

itional data, but the coefficien t of variation (i.e., standard deviation divided by the mean) will not in

general be reduced

. Risk Communication
Even if addit

ional da ta are not colle cted to reduce uncerta inty, identif

ying the exp osure fa

ctors that contribute most to risk or haz ard may b e useful for risk com munication. F or example, assu me that the input for exposure freq uency has the stron

gest effect on the risk

estimate for a futu

re recreational open space. Further examination of this exposure variable rev eals that the wide spread (i.e., variance) of the PDF is a result o f multiple users (e.g ., mountain bikers, hikers, individuals who bring picnics, etc.) of the open space who may spend very different amounts of tim e recreating. As a result of this analysis , the decision mak

ers and comm

unity may

decide to focus rem

ediation efforts on protecting the high- risk subpopulation that is expe cted to spend the m ost time in the o

pen space. After determining which contaminants, media, and expos ure pathways to carry int o a PRA, numerical experim

ents generally should be perform ed to determine th

e sensitivity of the output to variou sRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 8 distributions and param

eter estimates that m

ay be supported by the availab le information. Variables that do not strongly affe ct the ri sk est imate s may

be char acteriz ed with

point e stima

tes with out sig nificant ly altering the risk estimates. This g uidance documen

t does not recom

mend a quantitativ

e metric or rule of thumb for determ

ining when a v

ariable strongly a

ffects the output; this would generally be determined on a case-by-case basis. A qualitative or qu antitative analysis m

ay be used dep ending on the complexity of the risk assessm

ent at this point. For example, incidenta l ingestion of soil by

children is often an influen tial fact or in de termin ing ri sk from soi l, a factor reco gniz ed by risk a ssessor s. This recog nition i s a de facto informal sensitiv

ity analysis. An array of qua ntitative techniques is also available, ranging from som ething as sim

ple as comparing

the range of po ssible values (i.e., maximum -minim um) for each variable, to m

ore complex statistical methods such a s multiple reg ression analysis. S everal of these methods are discu ssed in more de tail in this appendix. Often, sufficient inform

ation is available to ch

aracterize a PDF fo

r a minor variable without significant effort. This situation raises a question of w

hether the variable sho

uld be characterized w

ith a point estimate or a PDF. The results of se nsitivity analy sis should be viewed as supplemen tal information, rath er than an absolut e rule for deter mining when to use a PDF. There are at least two issues to consider related to risk comm unication. First, the risk

commu nication process ma

y be facilitated by narrowing

the focus of the ev

aluation to the key

factors. More attention can be given to the discussion of key

variables quantified by PDFs by describing the m

inor variables with p oint estimates. However, the decision to use a point estimate should be balanced by consid ering a second issu

e regarding perception and trust. There m ay be a concern that by reducing sources of variability

to point estimates, there would be a redu

ction (however sm

all) in the variability

in risk, especially

if multiple sm

all sources of var iability add up to a nonnegl igible

contrib ution. To addre ss these

concer ns, it m ay be prudent

to leave the PDFs in the calculations despite the results of a sensitivity analysis.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 9 Figure A-1. Result s of 2- D M CA in whichparameters of in put dis tri butions descri bing var iabil ity are ass umed to be rand om v alues. R esults of a sensitivity ana lysis (top g raph ) sugge st that mo re than 50% of the un certain ty in t he 95 th perc entile of the risk distrib ution is d ue to u ncerta inty in the arithm etic me an co ncen tration in so il. The bott om graph giv es box- and- whis ker pl ots for the 95 th perc entile of the risk distrib ution a ssocia ted with M onte Ca rlo simulations using different samp le sizes ( n =25 and n =50). For this example, the whis kers represent the 5 th and 95 th percenti les of

the di stri

bution f or uncert aint

y, ot herwis e descri bed as the 90% confidence in terval (CI). For n =25, the 90% CI is [1.0E-06, 2.2E-05]; for n =5 0, the 9 0% CI is red uced to [1.2 E-0 6, 9.5 E-0 6]. W hile incr easing n did not chang e th e 50 th

percenti le of t

he uncert aint

y dis tri bution, it did provide gr eater conf idenc e th at t

he 95 th percenti le r isk

is below 1x10 -5

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 10 E XHIBI T A-3

S OME K EY I NDICES O F S ENSITI VITY A NALYSIS

C Rela tive co ntributio n of ex pos ure p athwa ys

C Inspe cti on of r isk

equat ion

C Sens itivity ratios (i.e., ela sticity)

C Sens itivity score s (i.e., weigh ted sen sitivity ratios)

C Gra phica l techniq ues with r esults of M onte Carlo simu lations (e.g., scatter plots)

C Correl ati on coef

fi cien t (or

coef fi cien t of deter minatio n, r 2 ) (e. g., Pearson product moment, Spearman rank)

C Norma lized multi ple regression coeff icient

C Go odne ss-of-fit

test for subsets of the risk dist ributi on 

### A.2.0 C OMMON 

### M ETHOD S OF 

### S ENSI TIVI TY 

### A NAL YSIS Of the n umero us appr oaches to sensi

tivity analy sis that

are available (see Ex

hibit A -3), no sing

le approac h will s erve a s the be st analysis for a ll mode ling e fforts. Often, it will mak e sense to appl y multi ple approaches . The be st choic e(s) for a particular situation will depend on a number of factors, including the nature and comp lexity of the model and the resources available. A brief descrip tion of the more com mon a pproaches is provided in this ap

pendix. Sensitivity

analysis need not be l imited

to the m ethods discuss ed in th

is guidance, which

focuses on the m

ore comm

on approaches. A large body of scientific literature on various other m

ethods is available (e. g., I man et al., 1988, 1991; Morg an and Henrion, 19 90; Saltelli and Marivort, 1990; Ros e et al., 1991; Merz , Small, and Fischbeck, 1992 ; Shevenell and H offman 1993; Hamby, 1994; U.S. EP

A, 1997). An

y method used, however, g enerally should be documented clearly and concisely. This includes

providing a

ll information needed by a third

party to repeat the procedure and corroborate the results. R elevant inform

ation might include the following : exposure pathway s and equations; a table with the input variables with point estimates, pro bability distributions and parameters; and tables or graph s giving the results of the sensitiv ity analysis and

description of the m

ethod used. A hypothetical exam

ple is presented in this app endix to illustrate how to apply and present the results of selected approaches to sensitiv ity analysis. Hypothetical Example o

f a Noncancer Risk Equation To illustrate the application of sensitiv ity analysis con

cepts to Tier 1 and Tier 2, a hypothetical risk assessmen

t is presented based on the general equation for Hazard I ndex (HI) g

iven by E quation A-1. Note th at HI is equ al to th e sum of th e chemic al-spec ific Ha zard Quot ient ( HQ) value s, so te chnic ally, this exa mple reflects

exposu res from a single chem

ical.

Equa tio n A-1
The terms in Equation

A-1 can be defined as follows: co ncentration in the i th exposu re me dium (C i ), ingestion or inhalation rate of the i th exposu re me dium (I i ), absorption fraction of chemical in the i th exposure medium (AF i ), expo sure du ration (ED), exposure

frequen cy (EF ), body

weig ht (BW), averag ing ti me (AT=ED x 3 65 day s/year ), and referenc e dose (RfD ).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 11 For this exam ple, HI is calcu lated as the sum of the e xposure s to adu

lts from two ex posure pathwa ys: ta p water ingestion and

soil ingestion. Equat

ion A- 2 gives the e quation for HI while Table A-2 gives the inputs for a point estimate asses sment and a p robabilistic assessment o f variability.

Equa tio n A-2

Ta ble A -2. Poi nt

es ti mat es an d pr oba bi li ty di st ri butio ns f

or in put va ri abl es us ed in t

he hypot he ti ca l e xa mpl e of

HI assoc iated w ith occ upa tional ex pos ure via water a nd so il ingestion

.

Inpu t Va riab le in Equation A -2 Point Estimate Probabil ity Di stri buti on Units CTE RME Type Parameters Concentrat ion in

Water (C_w) 40 40 poin t estimate 40 mg/L Tap W ater I ngest ion Rate ( I_w) 1.3 2.0 lognormal 1 [1.3, 0.75] L/day Absorption Fr acti on Water (AF_w) 0.30 0.50 beta 2 [2.0, 3.0] unitless Conc entration in Soil (C_ s) 90 90 poin t estimate 90 mg/kg Soil Ingestion R ate (I_s) 0.05 0.10 uniform [0, 0. 13] kg/day Abso rption Frac tion Soil (AF_ s) 0.10 0.30 beta 2 [1.22 , 4.89] unitless Exposur e Frequ ency(EF) 250 350 tri angular [180, 250, 350] days/yr Exp os ur e Du ra ti on (ED) 1 7 empirica l 3 see be low years Body Wei gh t ( BW) 75 75 lognormal 1 [74.6, 12.2] kg Av er ag i ng Ti me (A T) 365 2555 empirica l 4 ED x 365 days RfD oral 5 0.5 0.5 poin t estimate 0.5 mg/kg- day

1 Para me ters of log norm al d ist ribu tion are [ari thm eti c m ean , st anda rd de via tion ]. 2 Paramet ers of be ta distributi on are [alpha,beta], wi th range de fined by

min=0 a nd max= 1.0. Parame

ter conve rsions for arithme tic me an and standa

rd deviation are giv

en in Table

A-7. 3 Paramet ers of e mpirica l cumula tive distr ibution functi on (ECDF) f or ED ~ [min,

max, {x}, {p}] = [0, 30, {0.08, 0.18, 0.30, 0.44, 0.61, 0. 84, 1.17, 1.72 , 3.1, 6.77, 14. 15, 23.94}, {0.1 , 0.2, 0.3, 0.4, 0 .5, 0.6, 0.7, 0.8 , 0.9, 0.95, 0.9 75, 0.99}], where xis the array of value s and p is the array of corresponding cumulative probabilities. 4 AT=ED x 365 for nonc arcinoge

nic risks (Hazar d Index). 5 For simpli city, RfD oral

is assum ed to be appli cable to the ingestion of

the chem ical in both w

ater and s oil. 

### A.2.1 T IER 

### 1 A PPROACHES Approaches for sensitiv

ity analysis in Tier 1 of a PRA are lim ited to calculations that are ba sed on changing

point estimates. They are genera lly easy to p erform and to communicate. As given by Table A-1, goals for the sensitivity analysis in Tier 1 includ e quantifying the relative contributions of the exposure pathway s, identifying potential nonlinear relationships that may ex ist between input variables and the risk estim

ate, and rank ord

ering the relative contribution of exposure v ariables to variability orRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 12 uncertainty in the risk

estimate. This last goal may b e the most difficult to achieve due to the limitations associa ted with the poi nt estim ate methodol ogy . Metho ds are a pplied t o the hy potheti cal exam

ple presented above (S ection A.2.0) in ord er to demonstrate the inherent limitations o f the Tier 1 approaches in some situations.
A.2.1.1 P ERCENTAGE C ON TRIBU TION OF E XPOSURE P ATHWAY S T O T OTAL RISK For cancer and

noncan cer risk assessm ents cen tral tend ency e xposure (CTE) an

d RME r isk is typically calculated as the sum of risks from m ultiple exposure pathways. Risks m ay be dom

inated by one or two exposure pathways, whic h can be determ

ined through a sim

ple calculation as shown below. The re lative contri butio ns of ex posur e pathw ays are l ikely to d iffer betwee n the CT E risk an d RME ri sk. The point estimates in Table A-2 were applied to Equation A-2 to obtain CTE and R ME point estimates of HI . Table A-3 gives the perc ent contributions of soil ingestion and tap water ingestion using Equations A- 3 and A-4 . Tap water ingestion contributes at least 90% to H I, and the total HI is g

reater than 1.0 for both C TE and RME point estim ates. If 1. 0 is the level of c oncern for HI , and a decision wa s made to expl ore va

riability and uncertaint

y in a probab ilistic a

nalysi s, this result m

ight s upport prioritizing the ev

aluation of data and assum ptions associated with the tap water ingestion pathway.

Table A-3. Percen t con tri buti on of e xposur e pat hway s t o HI for the exampl e inSect ion A. 2. Exposure Pathway CT E P oint E stimate RM E P oint E stimate
HI % of total 2 HI % of total Soil Inge sti on 0.02 6 % 0.15 13 % Tap W ater

Inge sti on 0.28 94 % 1.02 87 % Total 0.30 100 % 1.17 100 %

1 Equation A-3: HI total

= HI soil + HI water 2 Example using Equation A-4: % of total RME HI for soil ingestion = (0.15 / 1.17) x 100% = 13%.
Equa tio n A-3

Equa tio n A-4

In th is exam ple, th e choic e of CTE

and RME point

estimates ref

lects an effort t o explore variability in HI , rather than uncertainty . Even if the concentration terms re present the upper confiden ce limit on the m

ean (e.g., 95 % UCL), the point estimates chosen to represent the CTE and RME for other exposure variables reflect assumptions abou t the variability in

exposures. There is unc ertainty that the choices actuall y repr esent the centra

l tenden cy and reasonable m

aximu m exp osures. To expl ore this uncertainty, alternative choices for CTE and R ME may have been selected. This type of exploration of uncertainty in Tier 1 m

ay also be v

iewed as a form

of sensitivity a

nalysis. The percent c ontribution of exposure pathway s could be recalculated, and the sensitivity ratio approaches discussed below may also be applied.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 13 A.2.1.2 I NSPEC TION OF RIS K E QUAT ION For many S uperfun

d risk assessm ents, ri sk equ

ations c an be ch

aracteri zed as

relatively si

mple algebraic expression s invo lving addition, m

ultiplic ation, and div

ision of input v ariables . The te

rm “product-quotient”

model is often applied to describe equation s such as Equation A -1. For these risk equations, the input v

ariables that are likely

to contribute mo st to the variability or uncertainty in risk

can be identified by insp ection. In addition, inspection of the risk equation can help to identify which sensitiv ity analysis

methods are

unlik ely to revea l the rel ative impor

tance o f the in

put va riables. This concept is illustrated by

comparing

the results of the sensitiv ity ratio approach (Se ction A.2.1.3) w ith the Tier 2 approaches (Section A .2.2) applied to the h ypothetical exam

ple in Section A.2. 0. Some risk equations can be m ore complex, in

volving conditional probabilities, or ex pressions with exponents (e.g ., y=x 2 , or y=exp(1-

x) ). In these

cases, the Tier 1 sensitiv ity analysis m

ethods may be effective and hig

hlighting the v

ariables that contribute m ost to the risk estim

ates. A.2.1.3 S ENSITI VITY R ATI O (SR) A method of sensitivity analy sis applied in man y different m

odels in science, engineering, and economics is the Sensiti

vity Ra tio (SR), otherwise k

now as the elastici tyequation. The approach is easy to understand and apply . The ratio is equal to the percentage change in output (e.g. , risk) divided by the percentage chang e in input for a specific in put variable, as sh

own in Equation A -5.

Equa tio n A-5

where, Y 1 = the baseline value o f the output variable using baseline v alues of input variable s Y 2 = the va

lue of t he outp ut var iable af ter chan ging the v alue of one inp

ut var iable X 1 = the base line poi nt estim ate for a n input varia ble X 2 = the valu e of the input va riabl e afte r changi ng X 1 Risk estimates

are considered m

ost sensitive to input variables that yield the highest absolu

te value for SR. The basis for this equation can be understo od by exam ining the fundam

ental concepts associated with pa rtial derivat ives (se e Secti on A.3.2 ). In fact, SR is equ ivalen t to the normali zed part ial de rivati ve (see Equation A- 12). Sensitivity ratios c an generally b e grouped into tw o categories—local SR

and range SR . For the local SR method, an input v

ariable is var

ied by a sma ll amo unt, us ually ±5% of the nom

inal (de fault) point estimate , and t

he corre spondin

g chan ge in the model ou

tput is o bserv ed. For the ran

ge sen sitivit

y ratio method, an

input variable is v

aried across the entire rang e (plausible minim um and m

aximum values). Usually , the results of local and range SR calcu

lations are the same . When the results differ, the risk assessor can con

clude that different exposure variables are driv ing risk near the high- end (i.e., extreme tails of the risk distribution) than at the central tendency region.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 14 Demonstration of the L imitations of SR Appro ach Although SR is a relatively sim ple and intuitive app

roach, it does not prov

ide useful information under certain conditions for the more com mon risk equations. To dem onstrate the limitations, first Equation A- 5 is applied to the hy pothetical example g

iven in Section A .2.0. The results are the n extended to a m

ore general case of any of the m ore comm

on risk m odels that involve the products of terms (i.e., mult

iplicati ve m odel) or the sum of terms (i.e. , addit

ive m odel). Table A- 4 prese nts an examp le of th e local SR and range SR app roach a pplied t o the se t of RME inputs given in Table A-2. For the local SR, each input was increased by 5% (i.e., ) =+5%), while for the range SR, each

input was increased by

50%. Inp

uts for exposure frequency were truncated at the maximum

value of 365 d

ays/year, w

hich represents a 4.29% inc rease over the nom

inal RME value of 350 days/yea

r.
Ta ble A -4. Resu lts of the Se nsitivity Ratio (SR ) app roac h app lied to th e hypo thetical ex amp le of R M E H I given in Section A .2.0. Includes both soil ingestion and tap w ater ingestion pathwa ys.
Inpu t Va riab le , X in Equation A -2 1 No mina l RME value (X 1 ) Loc al SR

( ) = + 5 . 0%) Rang e S R ( ) = + 50% or max)

X 2 ) i n HI

( %) SR X 2 ) i n HI ( %) SR Tap W ater

Inge sti on Rate , I_w (L/da y) 2.0 2.1 4.35 0.87 3.0 43.5 0.87
Absorption Fraction W ater, AF_ w (unitless) 0.50 0.525 4.35 0.87 0.75 43.5 0.87

Soil In gestion Rate , I_s (kg /day) 0.100 0.105 0.65 0.13 0.150 6.5 0.13 Absorpt ion Fr acti on Soil , AF_s (unitless) 0.30 0.315 0.65 0.13 0.45 6.5 0.13
Exposur e Frequ ency , EF (days/yr) 350 365 2 4.29 1.00 365 2 4.29 1.00
Exp osure D uration, ED (years) 7 7.35 0.00 0.00 10.5 0.00 0.00 Bod y Weight, BW (kg) 75 78.75 - 4. 46 - 0. 89 112 .5 - 33. 33 - 0. 67

1 Only input variable s that repr esent va riability are included. Concentra tions are poi

nt estim ates of uncertaint

y. A veragi ng time is a function of exposure duration. RfD is a fixed point estimate. 2 Maxim um EF of 365 days/

yr repr esents a 4.29% change in the nom inal RME va

lue of 350 da

ys/y r. The following observ

ations can be m

ade from these res

ults:
< In decrea sing ord er of se nsiti vity:

Local SR ( ) = 5%) rankings: EF > BW > I_w = AF_w > I_s = AF_s > ED Range SR ( ) = 50%) ranking s: EF > I_ w = AF_w > B W > I_s = A F_s > ED

< EF is the mo st sensitive variab le with an SR v alue of 1.0. Since

EF is a variable in the numerator for both exposur e pathways, this resul t is to be expected , as will be explain ed below.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 15 < ED yields an S R=0, sugg esting it does not c ontribute to the HI estimate. Up on closer inspection of the r isk eq uation, it is ap

parent t hat ED occurs in the n

umera tor of E quation

A-2 , as we ll as in the denominator (A

T=ED x 365). Thus, E

D effectively c ancels out of the product quotient model and d oes n ot ef

fec t the esti mate o f HI.

< BW, the only v

ariable in the denom

inator of the risk equ

ation, is also the only

variable to y

ield a differen t SR value when com

paring the loc al and r

ange S R appro aches. Thus, B W is the only variab le for w

hich SR depends on the

percen t chang e in the

input ( ) ).

< BW is the only negative SR value, indicating that HI an

d BW are inversely

related. This is true in general for

any v ariable in the d

enom inator o f a prod

uct quo tient m

odel.

< For variables unique

to the water inge stion pathway (I _w, AF_w), S R=0.87. Sim ilarly, for variab les uniq

ue to the soil in

gestion pathway (I

_s, AF _s), SR =0.13 These SR

value s are ex actly the same as the percent contributions of the tap water ingestion pathway and so il ingestion pathwa y to HI (se e Table A -3). Since tap wat er inges tion i s the do minant p athway ( i.e., 8 7% of RM E HI), a rea sonabl e stra tegy for the Tier 1 sen sitivit

y ratio approa ch mi ght be

to lim it the su bsequen t probab ilistic a

nalysi s in Tier 2 to the tap water ingestio

n pathway; so th

at input variables uniq

ue to the soil ingestion pathway would be characterized by point estimates. For this relatively simple exam ple, this would m

ean that soil ingestion rate (I_s) and a

bsorption fraction from

soil (AF_s) would be described by p oint estimates instead of PDFs. The question to

address would then become—Of the e xposure variables in the tap water ingestio n pathway, which

ones contribute m

ost to HI? A sensitivity ratio approach was applied to the tap water ingestion pathway

to address this question. The results are presented in Table A-5.

Ta ble A -5. Resu lts of the Se nsitivity Ratio (SR ) app roac h app lied to th e hypo thetical ex amp le of R M E H I given in Section A .2.0. Includes only tap wa ter inges tion pa thway.

Inpu t Va riab le , X in Equation A -2 1 No mina l RME value (X 1 ) Loc al SR ( ) = + 5 . 0%) Rang e S R ( ) = + 50% or max)

X 2 ) i n HI

( %) SR X 2 ) i n HI ( %) SR Tap W ater

Inge sti on Rate , I_w (L/da y) 2.0 2.1 5.0 1.00 3.0 50 1.00
Absorption Fraction W ater, AF_ w (unitless) 0.50 0.525 5.0 1.00 0.75 50 1.00
Exposur e Frequ ency , EF (days/yr) 350 365 2 4.29 1.00 365 2 4.29 1.00
Exp osure D uration, ED (years) 7 7.35 0.00 0.00 10.5 0.00 0.00 Bod y Weight, BW (kg) 75 78.75 - 4. 46 - 0. 89 112 .5 - 33. 33 - 0. 67

1 Only input variable s that repr esent va riability are included. Concentra tions are poi

nt estim ates of uncertaint

y. A veragi ng time is a function of exposure duration. RfD is a fixed point estimate. 2 Maxim um EF of 365 days/

yr repr esents a 4.29% change in the nom inal RME va

lue of 350 da

ys/y r.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 16 The following observ

ations can be m

ade from these res

ults:

< In decrea sing ord er of se nsiti vity:

Local SR ( ) = 5%) ranking s: I_w = A F_w = EF > B W > ED Range SR ( ) = 50%) ranking s: I_w = A F_w = EF > B W > ED

< SR values for v

ariables in the num

erator (I_w, AF_w, and EF ) are all equal to 1.0, so the SR approach sugg ests that they contribute equally to the HI estima

te.

< BW values are the same as in Table A -4. They are negative, and the values change as a function of the percent chang e in the nomina l RME value ( ) ). Tables A-4 and A-5 suggest that the SR approach provides essentially the sam e information about sensitivity as other Tier 1 metho ds. Specifically, inspection of the risk equation reveals that E D does not contribute to HI. I n addition, for pathway

-specific va

riables in the numerator, like ing estion rates and absorption fractions, SR values are equal to the percent contrib utions of the exposure p athways. This actually reflects the fact that each factor in the num erator of a multiplicativ

e equation has an SR

of 1.0. The resul

ts of th e SR ap

proach applied

to the examp le abov

e can be gener alized to all multiplicative and ad ditive risk equatio ns, as discussed be low. Generalizing the Limitation s of the SR Approa ch
In m any cases, the general equation for SR (Equation A- 5) will give values that can be determined a priori , witho ut doin g many ca

lculations. To u

ndersta nd why this is true, it

is usef ul to simpl ify th e alge

braic ex pression given by

Equati on A- 5. Let ) equal the percentag e change in the input variable, X

1 . For SR calculations, ) may be either positive or ne gative (e.g ., ±5% for local SR; ±100 % for range SR), and the new value for the input variable (i.e., X 2 ) is given by Equation A- 6.

Equa tio n A-6

Therefore, the denom

inator in Equation A- 5 reduces to ) :

and Equ ation A -5 re duces to

Equati on A- 7:
Equa tio n A-7RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 17 E XHIBI T A-4
C ATEGORIE S OF S OLUTI ONS FO R S ENSITI VITY R ATIOS OF M ULTI PIC ATI VE OR A DDITIVE E QUATIONS Case 1 SR is a constant (e.g ., 1.0). SR is independent of the ch oice of nominal (default) values for inpu t varia bles and the cho ice of )

. Case 2 SR is a

constan t determ ined on ly by the nom inal v alues for the in

put va riables. SR is indepen dent of the cho ice of )

. Case 3 SR is c onstant

determ ined on ly by the cho ice of ) . SR is independ ent of the nomin

al values for the input variables. Case 4 SR is a

function of bo th the n

omina l values for the input variab les and the cho ice of )

. Case 5 SR is 0. The variable does not contribute to the risk estim

ate. Equation A- 7 can be used to ev aluate SR for different ty pes of exposure m

odels in which the intake equation is g enerally

expres sed as a

simp le alg ebraic c ombin ation of input v ariables . Solu

tions to SR calculations for input variables in both m ultiplicative and additiv

e equations are giv

en in Table A-6

. For any such risk

equation, the solution will fall into one of the fiv e categories giv

en by Exhibit A -4.

Ta ble A -6. Examples of al gebrai c sol uti ons t o Sensi ti vit y Rat io ca lcul ati ons f

or addi ti ve and mu lt ipl icative forms of risk equations. 1, 2

Equati on Type (Ou tp ut = Y, Input s = A, B, C, D) SR A = SR B = SR C = SR D =
1) Additive in Numerat or NA 3

2) Additive in Denomin ator 1.0 NA

3) Multipl icat ive inNumerat or 1.0 1.0 NA

4) Multipl icat ive in Denomina tor 1.0 NA

1 Sensitivit

y Rati o for input v

ariable A for an e quation that is

additive i

n the nume

rator: SR

A =A / (A + B). 2 ) =% change in input va riable. For

exam ple, ) for C=[(C 2 - C 1 )/C 1 ] x 100%, where C 1 =the origi nal point est

imate and C 2 =the modified point estim

ate. Sim ilarly , C 2 =C 1 (1 + ) ). 3 NA=not a

pplicable be cause the

variable is not in the equation.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 18 The fo llowi ng obser vation s can be made for t he four forms of t he ris k equati on, bas ed on one of the f ive cases described in Exhibit A-4: (1) Additive in Numerator

< Case 2 : SR values for variables in the num erator depend exclusiv

ely on the nom

inal point estimates for all v

ariables in the num

erator. The values are independent of the choice of percent change in the inp

uts ( ) ).

< Case 3 : SR values for variables in the denominator depend e xclusively on ) , and are negative (i.e., invers ely re lated to

the out put). A lso, th e lower

the cho ice for ) , the higher the resulting SR values. Therefore, SR is som ewhat arbitrary, espe cially for the rang e SR approach since input variables m

ay have differe

nt plausible minim

um and m

aximum values. (2) Additive in Denominator

< Case 1 : SR values for varia bles in the num

erator a re alwa ys equ al to 1. 0. Since they

are independent of the nom

inal values and ) , there is no way to distinguish the relative contributions to the o utput.

< Case 4 : SR values for variables in the denominator are a function of both the nom inal values of variables in the denominator and )

. (3) Multiplicative in N

umerator and (4) Multiplicative in Denominator

< Case 1 : SR values for varia bles in the num

erator a re alwa ys equ al to 1. 0. Since they

are independent of the nom

inal values and ) , there is no way to distinguish the relative contributions to the o utput.

< Case 3 : SR values for variables in the denominator depend e xclusively on the ) , and are negative (i.e., invers ely re lated to

the out put). A lso, th e lower

the cho ice for ) , the higher the resulting SR values. Therefore, SR is som ewhat arbitrary, espe cially for range SR since input variables may hav e different plausible m

inimum a nd maxim

um values. These generalized resu

lts highlight a major limitation in the use of the SR a pproach for obtaining information from

sensitivity analysis. For simp le exposure mod els in which the relationship between exposu re and r isk is li near ( e.g., mult iplic ative) , the ra tio of fers l ittl e infor mation r egardi ng the re lative contributions o f each i

nput variable to the r

isk es timate . In

many cases, all of t

he inpu t varia bles wi ll have the sam

e constant, either equal to 1.0 (in the case of a single exposure p athway) or equal to the relative contributions of the exposure pathways. For more complex m

odels that combine

additive, multiplicative, a

nd nonlinear relationships be tween inputs and outpu

ts (e.g., env

ironmental fate and transport models, pharmacokinetic models), the ratio is likely to be an effective screenin g tool for identifying poten

tially influential input v

ariables and assum

ptions. Another difficulty w

ith the SR approach is that it generally requires an assum

ption that the input variab les are i

ndependent. Two va

riables may

actuall y be p ositiv ely correlated (e.g. , hig h valu es of X 1 corresp ond wit

h high value s of X 2 ) or negativel y corr elated ( e.g., high value s of X 1 correspond with low value s of X 2 ). If input variables are correlated, holding the value for one variable fixed while allowing the other to vary

may pro duce misleading results, especially w

ith the range sensitiv

ity ratio approach. For exa mple, it ma

y not be reali stic to h

old bod y wei ght fixed at a

central tendency wh

ile allow ing sk inRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 19 surface area to vary

from the m inimu m to m aximu m v alues. An im

prove ment over t

he sens itivity

ratio approa ch woul d be to al low cor relat ed inpu t varia bles t o vary simul taneo usly.
A.2.1.4 S ENSITI VITY S CORE A var iation on the se

nsitivity rat

io appro ach may provide m ore informatio

n from a Tier 1 sensitivity analy sis, but it requires that add itional information be available for the input variables. The sensitivity scoreis the SR weig hted by a norm

alized measure

of the variability

in an input variab le (U.S. EPA, 1999). Examples of no rmalized m

easures of variability

include the coefficient of v ariation (i.e., standard deviation div

ided by the m

ean) and the norm

alized range (i. e., range div ided by the m

ean), as given by Equation A- 8. Equa tio n A-8

By norm alizing the m

easure of variability

(i.e., dividing

by the m ean), this method

effectively we ights the ratios in a manne

r that is independent of the units of the input v ariable, and provide s a more robust method of rank

ing contributions to the risk estima tes than the SR alone. This approach does requ ire that the coefficient of variation or range can b e calculated for each v ariable. Tables A- 7 and A-8 present the results of the sensitivity

scores based on the CV applied to the hy pothetical example from Section A. 2.0.

Ta ble A -7. Calculation of coef fici ent of varia tion ( CV = SD / Mean) for t he hypoth etic al exampl e of RME H I give n in Sec tion A .2.0.

Inpu t Va riab le , X in Equation A -2 1 Probabil ity Di stri buti on 2 M ean 3 SD 3 CV = SD/Mean
Ta p W ater Ing estion R ate, I_w (L/da y) logn ormal (1.3, 0.75) 1.3 0.75 0.58 Absorpt ion Fr acti on, Water , AF_w (unitless) beta (2.0, 3.0) 0.4 0.2 0.50

Soil In gestion Rate , I_s (kg /day) uni form ( 0, 0. 13) 0.065 0.038 0.58 2
Abso rption Frac tion, Soil, AF_s (un itless) beta (1.22, 4.89) 0.20 0.15 0.75 Exposure Freque ncy, EF (days/yr) tri angul ar (180, 250, 350) 260 35 0.13 3 Exp osure D uration, ED (years) emp irical CD F (see Ta ble A-2 for pa rameters) 1.75 3.86 2.21

Bod y Weight, BW (kg) logn ormal (74.6, 12.2) 74.6 12.2 0.16

1 Only input variable s that repr esent va riability are included. Concentra tions are poi

nt estim ates of uncertaint

y. A veragi ng time is a function of exposure duration. RfD is a fixed point estimate. 2 Beta (a,b): mea

n=a / (a+ b) and SD = ( (a x b) / [(a + b)^2 x (a+b+1)])^0.5) Uniform (min, m ax): m ean = (m in + ma x)/2 and SD = ((1/12)^0.5) x (ma x - mi n) = 0.289 x (m

ax - m in) Triangular (mi

n, mode, m

ax): m ean = (m in + mode

+ max )/3 and SD = (1/18) x (m

in^2 + m ode^2 + m

ax^2 - m in x ma x - mi n x mode - mode x max) Empirica l CDF ({x }, {p}): m

ean and SD w

ere esti mated by Monte Carlo simulat

ion. 3 Mean=ari

thmetic mean; SD=arithm

etic sta ndard devia

tionRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 20 Ta ble A -8. Result s of t he Sensi tiv ity

Score (Score) approach applie d to the hypothet ical example of RM E HI given in Sec tion A .2.0. C alculatio ns for S ensitivity Ra tio (SR ) and Co efficient o f Var iation (C V) a re give n in T able A -4 and Ta ble A -7, resp ectively.

Inpu t Va riab le , X in Equation A -2 1 No mina l RME value (X 1 ) CV

(Table A-7) Loc al SR ( ) = + 5 %) Rang e S R ( ) = + 5 0%) SR (Table A-4 ) Score 2 SR (Table A-4 ) Score 2
Tap W ater

Inge sti on Rate , I_w (L /day) 2.0 0.58 0.87 0.50 0.87 0.50
Absorption Fraction, Water, AF_w (unitless) 0.50 0.50 0.87 0.44 0.87 0.44

Soil In gesti on Rate, I_s (kg /day) 0.100 0.58 0.13 0.06 0.13 0.06
Absorption Fraction, Soil, AF_ s (unitless) 0.30 0.75 0.13 0.10 0.13 0.10
Exp osure Freq uenc y, EF (days/yr) 350 0.13 1.00 0.13 1.00 0.13
Exposure D uration, ED (years) 7 2.21 0.00 0 0.00 0
Body Wei gh t, BW (kg) 75 0.16 - 0. 89 - 0.14 - 0. 67 - 0.11

1 Only input variable s that repr esent va riability are included. Concentra tions are poi

nt estim ates of uncertaint

y. A veragi ng time is a function of exposure duration. RfD is a fixed point estimate. 2 Score=SR x

CV (see Equa

tion A-8)
The following observ

ations can be m

ade from these res

ults:
< In decrea sing ord er of se nsiti vity: - Score based on local SR ( ) = 5%): I_w > AF_w > BW > EF > AF_s > I R_s > ED - Score based on range SR ( ) = 50%): I_w > AF_w > EF > BW > AF_s > I R_s > ED

< Comp ared wi

th the S R appro ach alone in w hich sen sitivit

y can only b e expre ssed for exposu re pathways, the se

nsitivity score ap

proach provides a measure of sensitiv ity for exposure v

ariables withi n each e xposur e pathw ay.

< Alth ough E D has the h ighe st CV , it c onti nues to ha ve no c ontr ibut ion t o the HI.

< If Tier 1 sensitiv

ity analysis is b ased on the sensitivity score, the highe st ranked variables are gen

erally those with the highest CV in the ex posure pathway that contributes the most to the total risk (HI). For th is hypothetical exam

ple, I_w and AF_w are the two highest ranked variables.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 21 

### A.2.2 T IER 

### 2 A PPROACHES Approa ches for sensit

ivity analy sis in Tie

r 2 of a PRA u tilize the resu

lts of Monte Car

lo simul ations, which allows multi ple inpu t varia bles to vary

simu ltaneou sly. The methods a re relati vely simpl e to per

form with spreadshe ets or c omm ercial s tatistica

l softwa re. The results are ge

nerally easy to communicate, although the deta ils of the methodology are more complex tha n Tier 1 approac hes. As given by Table A-1, g

oals for the sensitivity

analysis in Tier 2 are the same as Tier 1:quan tifying the relative contributions of the exposure pathways, identifying potential nonlinear relationships that may exist between input v

ariables and the risk estimate, and rank ordering the relativ

e contribution of exposure variables to variability or unc ertainty in the risk estimate. I

n addition, since the ou

tput is a distribution, Tier 2 sensitivity analysis methods can also utilize graphical techniques to observe nonlinear relationships, as well as ev aluate potential changes

in relative im

portance of variables and assumptions for risk s in the RME risk range. Methods are applied to the h ypothetical examp le prese nted in Section A.2.0 in o

rder to demonstrate the advantages over the Tier 1 m

ethods.
A.2.2.1 G RAPHICAL T ECHNIQUES Simple scatter plots of the simulated in put and output (e.g ., risk vs. exposure frequency , or risk vs. arithmetic mean soil concen tration) can be used to q

ualitatively and q

uantitatively ev

aluate influential variables. A “tig

ht” best-fit line thro

ugh the scatter plot, as indicated by the magnitud e of the r

2 , sug gests that a v ariable may

signi ficantly

influen ce the v ariance in risk . Hy

potheti cal scat

ter plot s used t

o identi fy sensitive and insensitiv

e variables are show

n in Figure A-2. Another m ethod for visualizing the relationship between all of the inputs and outputs is to generate a scatterplot matrix (Helsel and Hirsch, 1992). This graphic shows both histog rams and scatter p lots for all variables on the same page. Figur e A- 3 illust

rates sc atter plots for the 1- D MCA simul ations a ssociate d with the exa mple from Section A .2.0. Based on the r 2 values (i.e., c oefficient of determination for simple linear reg

ression analy sis), the relatio

nship b etween HI a

nd I _w is v ery st rong ( r 2 = 0.47) while the relationship between HI a nd I _s is very w eak (r 2 < 0.01), suggesting that HI is m ore sensitive to v

ariability in I

_w than I_

s. A.2.2.2 C ORRELATION C OEFFI CIENTS The variance in a risk

estimate from

a Monte Carlo sim

ulation is due to the v

ariance in the probability distributions u sed in the risk equ ation. It is common

ly said that a Monte Carlo model propagates sources o f variability sim ultaneously in a risk

equation. Num

erous statistical techniques, know n collectively

as corr elation analy sis and regres sion analysis , can b e applie d to a li near eq uation t o estimate the relative change in the output of a Monte Carlo simulation based on changes in the input variables. Exam

ples of metrics of sensitivity include the simple correlation coefficient, the rank correlation coefficient, and a variety of coefficien ts from m ultiple regression techniqu es. The underlying assumptions associ ated with these ap proaches are dis cussed in greater de tail in Sectio n A.3. As explained in Section A. 3.3.1, correlation coefficien ts and regression co efficients are based on different interpretations of the input v ariables, but they c an be calculated with sim ilar equations. When the output distribution is compared with the distribution for one in put variable at a tim

e, two of the mo

re comm on approaches are to ca

lculate the Pearson product moment correlation and the Spearm an rank correla tion. C orrelati on anal

ysis w ith one input v

ariable will g enerally

yield reasonable results when the input v

ariables are sampled independently in a Monte Carlo simulation . Some statistica l packages offer the correlation coefficient as an index of sensitivity , so it is important to identify whichRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 22 coefficient is being calcu lated. Crystal

Ball ® and @Risk

can be used to calcu

late the Spearman rank correlat ion, w hich ten ds to be more robust when the relatio

nships between

inputs and out

puts are nonline ar. I f the rel ationsh ips are linear, such as

with the produ

ct quoti ent models presented in this appendix, the two m

etrics of correlation will y ield similar rank

ings of input v

ariables. Rank correlation coefficients shown in Crystal Ball ® and @Risk

are calculated by the standard method provided in m

ost statistics texts. Crystal

Ball ® also indicates that sensitiv ity can be determ

ined as contribution to v

ariance. This is not the relative partial sum of squares techniques discussed in S ection A.3.3.2 (E quation A-19 ). Instead, Crystal

Ball ® calculates the contribution to the variance by squaring the rank

correlation coefficients and norm

alizing them

to 100%. Many other comm only used com

mercial software packages will perform Spea

rman rank correlation. Pearson prod uct mom ent correlations ( r ) can be calcula ted in Microsoft Excel using the trendline feature in a scatter plot chart, or by using the function Correl(X array, Y array) , where X array corresponds with the Monte Carlo simulation of an input variab le, and Y arraycorresponds with the o utput of the simu

lation. Figure A- 4 illustrates results of the correlation analysis for the 1-

D MCA simu

lations associated with th e example from Sect ion A.2. 0. The gr aphic s were gen erate d using Crystal Ball ® 2000. The results are sum

marized in Tab le A- 9. I f the m odel ou tput variable (e.g. , HI) and in put va riable a re hig hly correlated, it means that the output is sensitive to that input variab le. By squarin g the coefficient, the results can be expressed in terms of the pe

rcentage contribution to

variance in the ou

tput (Figure A-4, top panel). To determine

if the correlation is positive or negative, the correlation coefficient shou ld not be squared (Figure A -4, bottom panel). For risk

equations, in ge

neral, variables in the

numerator of the equation (ing estion r

ate, ab sorption fracti on, exp osure fr equency, etc.)will te

nd to b e positi

vely correlat ed with risk, while v

ariables in the d enom inator ( body weigh

t) will t end to b e nega

tively correlated with risk. The greater the absolute value of the correlation coefficient, the strong er the relationship.

Table A -9 . Resul ts of

Tier 2 sens iti vity anal yses applied t o hypothet ical example i n Secti on A.2.0: Pear son product mom ent correlations and Spearm an rank co rrelations. 1

Exp osure Va riable Product M oment Correl ati on Spearman R ank Correl ati on 2

r r 2 x 100% r r 2 x 100% normal ized r 2 x 100%
Ta p W ater Ing estion R ate, I_w (L/da y) 0.644 41.4 0.603 36.3 39.5 Abso rption Frac tion W ater, AF_ w (unitless) 0.583 34.0 0.666 44.4 48.3 Bod y Weight, BW (kg) - 0. 216 4.7 - 0. 229 5.2 5.7 Exposure Freque ncy, EF (days/yr) 0.174 3.0 0.167 2.8 3.0 Abso rption Frac tion Soil, AF_ s (unitless) 0.109 1.2 0.149 2.2 2.4 Soil In gestion Rate , I_s (g/d ay) 0.061 0.4 0.099 1.0 1.1 Exp osure D uration, ED (years) 0.010 0.0 0.010 0.0 0.0

1 Monte Car lo simulati on usin g Crystal Bal l ® 2000 , Latin Hypercube sampling, and 5000 iterations.
2 Crystal Bal

l ® 2000 output includes Spearma n rank corr

elations, r , and normalized r 2 values, c alculate d by divi ding each r 2 value by the sum of all the r 2 values (i.e., 0.920 in this example). Figure A-4 illustrates the r and norma

lized r

2 values f or the Spearman rank correlation analysis.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 23 Figure A-2. Scatterplots of simulated random values from a 1-D MC A of variabilit y. The output from the model is a contaminant concentrati on in soil (C) t hat corresponds with a prescribed (fi xed) level of ri sk for a hypothet ical populati on (based on Ste rn, 1994). For each i terat ion of a 1-D MC A simulation, random values were simultan eou sly selected for all mo del va riable s and the co rresp ond ing co ncen tration (C ) was ca lculated

. Inputs were simulat ed as independent random variables. Scatterplots of 500 consecutive random values and est imat es of

C are s hown f

or tw o inpu t va ria

bles : re lat ive absorption f ract ion, RAF (top g raph) ; an d mass fr acti on of dust as soil , F (bottom graph). There is a moderate, indirect relati onship between C and RA F (r 2 =0.34), compared with t

he weak relationship between C and F (r 2 =0.02), suggesting t hat the model output (C) is more sens it ive to va ria

bil ity i n RAF th an F.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 24 Figure A-3. Scatter

plot s of si mulat ed ran dom val ues f rom a 1- D MC A of va ria bil ity f or exa mple i n Sect ion A.2.0. The output from the model is HI. For each iterati on of a 1-D M CA simulation, random values were simul taneously sel ected f or all model vari ables an d the corr esponding HI was calcul ated. Inputs were si mulated as indepen dent ran dom variable s. Scat terpl ots of

250 consecuti ve random valu es and estimates of HI are s hown for two input va riable s: soil inges tion rate , I_s (to p gra ph); an d tap water ing estion ra te, I_w (bo ttom g raph ). There is a negligibl e relati onship between HI and I_s (r 2 < 0.01), compared wi th t he str ong rela tions hip betwe en HI and I_w (r 2 =0 .47) , sugges ting that the m ode l outpu t (HI) is more sensitive to variab ility in I_w tha n I_s. B estfit li nes were gen erated with the Simp le Linear Re gression in M icrosoft Exce l’s trendline option for scatterplo ts; r 2 values represent t

he coefficient

of determination (s ee Section A.3).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 25 Figure A-4. Top panel - bar graph showing the r 2 values (squa re of S pea rman rank c orre lation co efficient), a metric for t

he dependence o f HI on exposure factors based on 1-D M CA for variabilit y. Bottom panel - bar graph, sometimes referred to as “ tornado plot”, showing rank correlat ion coeffici ent. This graph is effecti ve for showing both t he re lat ive magni tude and di rect ion of inf luen ce (pos it ive or neg ati ve) f or each vari able. Abbrevi ati ons f or inpu t va ria

bles are g iven

in Tabl e A-4. In t

his exampl e, t

he var iabl e wit

h th e gre ates t ef

fect on HI i s t he abs orpti on fraction in water ( AF_ w), follo wed by the w ater inge stion rate (I_w) . Con centra tion do es no t influence variab ility because, in thi s example, long-t erm average concentration is charact erized by a point esti mate (i.e. , 95% U CL), rather th an a p rob ability distrib ution. E xpo sure d uration doe s not influe nce va riability be cause variab ility in ED is exp ressed in both the num erato r (ED ) and deno minato r (AT =E D x 3 65 fo r non carcin oge nic effec ts), and canc els out. Output was generated wit h Crys tal Ba ll ® , which calcul ates the contri bution to variance by squari ng the rank co rr el at i on co ef

f i ci en t an d n or ma l i zi ng t o 1 00 %.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 26 In this exam

ple, seven expos

ure variables are used

to characterize variability in HI . The remaining v

ariables in the risk eq

uation (i.e., concentration terms, and R

fD) are characterized b

y point estimates. Becau se point estimates d o not vary in a Monte Carlo sim ulation, they do not contribute to the variance in the outp

ut. This result does not m ean that concentration is an unimportant v

ariable in the risk assessment. C oncentration may still contribute greatly

to the uncertainty in

the risk estim

ate. A sensi tivity a nalysi s of par ameter u ncert ainty i n a risk eq uatio n can be e xplor ed usin g itera tive simulations, such as with 2-D MCA. Results of the Pearson correlation and Spearm an rank correlation g

ive similar rank

ings of the input variables, with

absorption fraction of wa ter (AF_w) and tap w

ater ingestion rate (I

_w) being the two dom inant ex posure variab les. Pe arson c orrelati ons sug gest t hat I _w is the most sensit

ive variable ( r =0.644), whereas the highest Spearman rank correl ation is for AF_w ( r = 0.603). This m

ay reflect the fact that I_w is characterized by an untruncated log

normal distribution, whereas AF_w is b ounded between 0 and 1.0 . The effect on the correlation s of the occasional hig h-end value for I_w generated from random

sampling of the lognorm al distribution will tend to b e expressed by P earson correlations, but muted by

the Spearman rank correlations. A comparison of the Tie r 1 and Tier 2 resul ts is given below:

< Tier 1, Sensitivity R atios: - Local SR ( ) = 5%) rankings: EF > BW > I_w = AF_w > I_s = AF_s > ED - Range SR ( ) = 50%) ranking s: EF > I_ w = AF_w > B W > I_s = A F_s > ED

< Tier 1, Sensitivity Sc ores: - Score based on local SR ( ) = 5%): I_w > AF_w > BW > EF > AF_s > I R_s > ED - Score based on range SR ( ) = 50%): I_w > AF_w > EF > BW > AF_s > I R_s > ED

< Tier 2, Correlation Coefficients: - Pearson: I

_w > AF_w > BW > EF > AF_s > IR_s > ED - Spearm an Rank: AF_

w > I_w > EF > BW > A

F_s > IR_

s > ED The Tier 1 sensitivity scores and Tier 2 correlation coefficients yield similar resu lts, suggesting that, if sufficient inform

ation is available to estimate the coefficient o f variation in the inp ut variables, a Tier 1 analysis can help to focus eff orts on the variabl es that contri bute most to the variance i n risk. By contras t, the Ti er 1 sen sitivit y ratio approa ch suggested

that EF was th e most influe

ntial variable, when i

n fact it c ontribu

tes less than 5% to the v ariance in the HI. These resu lts suggest t

hat Tier 1 sensiti

vity ratios are best applied to ide ntify domina

nt exposure pathway s, rather than dom inant exposure variab les in the risk equation.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 27 Eq uation A-9

Equati on A-10 A.2.2.3 F OCUSING ON THE RME R ANGE OF THE RIS K D IS TRI BUT ION Monte Carlo metho ds can also be used to determine the se nsitivity over a subset of the outpu t distribution, such as the R ME range (i.e., 90 th to 99.9 th percentiles). For som e exposure mod

els, the relative contribution of exposure variables m ay be different for the

high-en d exposed individu

als than for the entire range of exposures. The general strategy for e xploring sensitivity

over subsets of risk

estimates is to first sort the distribution o f simulated output values in ascending (or descending) order, and then apply a sensitiv

ity analysis to the subset of interest (e.g ., > 90 th percen tile). F or the h

ypothetical examp le presented in this appendix, there was no difference in the relative rank ings of inputs in the RME range.
A.2.2.4 I NSP ECT ION With Monte Carlo analysis,the probability distributio ns assumed for the various input v ariables are used to ge nerate a samp le of a l arge n

umbe r of poi nts. S tatistica

l meth ods are applied to this sampl e to evaluate the influen

ce of the inputs on the

model output. A numbe r of different “indices” of sensitiv ity ca n be de rived from the sim

ulated s ample

to quan tify the influe

nce of t he inpu ts and i dentify the ke y con tributors. Most

of thes e are ba sed on an assu mption that t

he mo del outp

ut Y varies in a mono tonic, linear f

ashion with res

pect to variou s input variab les (X 1 , X 2 , etc.) . For examp le, an e stima te of avera ge daily i ntake (mg/kg-da y) from mult iple e xposur e pathw ays is li near wi th res pect t o the in take from each pathway. Since m ost risk mo dels are linear with respect to the input variables, the output distribution (particularly its up per percentiles) tends to be dictated by the inpu t variables with the largest coefficient of variation (CV), or the ratio of the standa rd deviation to the mean. For example, Equa tion A-9 represents a simple expression for intake rate as a

function of random

variables X 1 and X 2 :

where X 1 and X 2 may repre sent dietary intak e associated with prey

species 1 and 2, re spectively. If the same probability

distribution was used to

characterize X 1 and X 2 , such as a log norma l distrib ution w ith an arith metic mean of 100and sta

ndard d eviati on of 5 0 (i.e. , CV=50/100=0. 5), each vari able wo uld contribute equally to variance in Y. If, howev er, X 2 was characteri zed by a log norma l distrib ution w ith an arith metic mean of 100and sta

ndard d eviati on of 2 00 (i.e ., CV=2 00/100 =2.0), we would exp ect Y to be more sensitiv

e to X 2 . That is, X 2 would be a g reater contributor to varian ce in Y. While the coefficient of va riation may be a useful screening tool to develop a sense of the relative contributions of the different input variab les, a comm on exception is the case when X 1 and X 2 have different scales. For example, Equation A- 10 is an extension of Equation A- 9:

where a 1 and a2 are constants that m ay represent the algebraic combination of point estimates for other exposu re var iables. If t

he me ans of X 1 and X 2 are equ al, but a 1 >> a 2 , then X 1 would tend to be the domin ant con tributor to va riance, regar

dless of the CV f or X 2 . This concept was dem onstrated by the sensitivity score c alculations given in Table A-8. Water ingestion rate (I _w) and soil ing estion rate (I_s) had the same C

V (0.58), but I

_w was the dom

inant variable because tap water ingestion contributed appr oxima tel y 90% t o the HI.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 28 Equati on A-11

The mo st influential ra

ndom varia bles g enerally

have the hig

hest deg rees of skewn

ess or a re related to the output accord ing to a power function (Cullen and Fre y, 1999). F or example, E quation A-11 presents an extension of E quation A-10 in which there is a p ower relationship between X 2 and Y. In this example, assum

e Y represents the total d ietary intake rate of cadmium for musk rats, X 1 and X 2 represent the dietary intake

rate associated with prey species 1 and 2, re spectively, a 1 and a2 represent additional point estimates in the equation, and 2 is the p ower ex ponent. In gener

al, for 2 > 1, the total d

ietary intake rate (Y ) will b e mor

e sensit ive to the intake ra

te assoc iated with spec ies 2 (X 2 ) than species 1. Assum e (hyp othetica

lly) that the p ower re lationsh ip stems from

the fact that there is a direct r elationship between availability

of prey species X 2 and chemical b ody burdens of prey species X 2 because indivi duals that are more ac cessible to the mu skrat also happen to frequent areas of the site with higher concentrations. 

### A.3.0 A DVANCED 

### C ONCEPT S IN 

### S ENSI TIVI TY 

### A NAL YSIS This section provides add itional information on

the underlying

principles of sensitivity

analysis, although it is not a

comprehensive summ

ary and is not inte

nded to substitute for the numerous statistical texts and journal articles on sensitiv ity analysis. Section A.3.1 beg ins with a general framework for relating mode l outpu

t to m odel inp ut. Se ction A .3.2 explains the sen sitivit

y ratio approa ch and highli ghts some of its li

mitati ons. S ection A .3.3 re views some of the m etrics re ported b y the comm

ercial s oftware that report results of sensitiv ity analysis follow

ing Monte Carlo sim

ulations (e.g., Crystal

Ball ® , @Risk ). While statistical software for MCA provides convenient metrics for qu antifying and ranking these sources, it is strongly

recommended that risk assess ors and risk m anagers develo

p an understanding

of the underlying principles associated with these metrics. 

### A.3.1 RELATING THE 

### C HANGE I N 

### RISK TO THE 

### C HANGE I N 

### I NPUT 

### V ARIABLE 

### X For pur poses o f discus sion, l et Y denote a mod

el output (e.g., risk) and suppose that it depends on the input variable X

. In general, a risk assessm ent model m

ay use any

number of inp

uts; however, for purposes of illustrating concepts, it is convenient to restrict this discussion to one variable. The model relates the outp

ut Y to va lues of X (i.e., x 0 , x 1 , sss , x n ) based on the function expressed as Y =F(x) . The sensitiv ity of Y to X can be interpreted as the s lope of the tangent to the response surface F(X) at any point x i . This tw

o-di mensi onal su

rface ca n be a s imple

straig ht line, or it m

ay be very

comp lex with chang ing sl opes as

shown in Fig ure A- 5a. The

sensiti vity , theref

ore, m ay dep end on both th e valu e of X and the amoun

t of the change ) x about that point. This concept can be extende d to two input variables, X 1 and X

2 , where the response is characterized by a three-dim

ensional surface. The shape may be a simple plane (Fig

ure A-5c) or it may b e very com plex with many

“hills” and “valleys” depending on the definin g function F(X 1 , X 2 ). In a typical risk assessment with ten o r more variab les, the surface can be very com plex, but the shape is likely to be d ominated by

a small subset o f the input variables. A sensitivity a

nalysis based on a

relatively sm

all deviation about the point may be referred to as a loc al se nsi tivi ty ana lysi s, wh ile a lar ge devi ati on may b e ref err ed to as ra nge se nsi tivi ty ana lysi s. In either c ase, the objectiv

e is to e valua te the s ensitiv

ity at some nomin

al point ( X 1 *, X 2 * ) such as the point defined by the mea

n or m edian o f X

1 and X

2 . At any p oint, the sensitivity of the model output, Y*

= F( X 1 *, X 2 * ), to one of the inpu

ts ( X 1 or X

2 ) , is represented by

the rate of chang e in Y

per unit change in X

.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 29 Figure A-5 a. H ypothetical 2- D response surface for Y given one input variabl e: Y= F(X

). Th e se nsitivity of Y

wit h res pect to X is calcul ated as the slope at a specifi c point on the surf ace (x 0 , x 1 ), or t

he parti al deri vati ve, M Y/ M X i . This is the slope of the s

urface a t that nomina

l point in the d

irection of X and is expressed as M Y/

M X i , the partial derivative

of Y with respect to X

.

If the function F(

X 1 , X 2 ) is known exp licitly, it may be possible to determ

ine the partial derivatives analy

tically. This is not a re quirement, however, because an estimate can be o btained by incre menting X i by a sm all amount, )

X i , while keeping

the other inputs fixed an d reevaluating the model output Y . The resulting change in Y divided by )

X i will approximate M Y/

M X i at th e nomi nal poin t. In practice , analy

tical so lutions can be a

pproxim ated usi ng Mon te Carlo techniq ues. Th is infor mation is presented to highlig

ht the fundamenta

l concepts of sensitivity

analysis. The partial d

erivative, per se , would typically

not be one of the methods of sensitiv ity analysis used

in a PRA. H owever, all of the approac hes that are pre sented i

n this a ppendix are v ariation s on thi s conce pt. One dra wback to usin g the partial d

erivat ive to quantif y the influence of X i is that the partial deriva tive is influen ced by the uni ts of m easurem ent of X

i . For examp le, if the mea surement scal e for X i is changed from

grams to milligram s, the partial derivativ e M Y/ M X i will change by

a factor of 1,000. Therefore, it is necessary to normalize the partial derivative to remove the effects of units (see Section A.3.2 ). If the relationship between Y and all of the inputs is linear, then the response surface is a flat plane and each of the partial derivatives at each point, ( X i , Y ), will remai

n const ant reg ardless of wher

e the poi nt is in the sur face ( Figure A-5b ). In this c ase, it is a simpl e matter to dete rmine th e rela tive influence that the va

rious in puts ha ve on the m odel ou tput. W hen the relation ship is nonline ar, however, the situation is more

complex because the inf luence of a particul ar input

may vary depending on the

value of that input.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 30 Figure A-5b. Hypothetical 3-D response surface for Y gi ve n t wo input v ar ia

bl es : Y = f(X

1, X 2 ). Th e sensitivity of Y wit h res pect to X

iis cal culat ed as the slope at a specif ic point on the s urface , or t

he parti al deri vati ve, M Y/ M X i .

Figure A-5c. Hypothetical 3-D response surface when Y is a l in ea r f

un ct io n o f tw o i nput v ar ia bl es : Y = f (X 1, X 2 ). The sl ope (i.e ., t

he parti al deri vati ve, M Y/

M X i ) is con stant for a ny po int ( X i , Y ) on t he surfa ce i n th e dir ecti on of X i . In this case , M Y/

M X 1 =5 while M Y/

M X 2 =2.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 31 

### A.3.2 N ORMALIZ ED 

### P ARTIAL 

### D ERIVAT IVE Classical sensitivity analysis meth ods use estimates of the partial derivativ es of the mode l output with respect to each v

ariable. For the purpo se of evaluating the relative influence of the various inpu t variables on the m

odel output at a single point, the normalized partial d erivative provides a useful index.
If the input variab les are a ll discre te and t ake on a sma ll num ber of v alues, then it is possi

ble to evaluate the influence of the vari ous inp ut var iables a t each o f the po ints def

ined by consid ering all possible combination s of the inputs. Then the influence can be ev aluated for each input by

computing normalized p artial derivatives at each point. This approach is limited to situations where the num ber of inputs as well as the nu

mber of possible values for each inpu t is relatively sm

all; otherwise, the num

ber of combinations to

be evaluated w

ill be unmanageable. Furtherm ore, when evalu

ating the influence at different points on the inpu t-output surface simultaneously, it is important to take into account the probab ilit y associ ated wi th each of thos e point s. For ex ample, t he fact that a p artic ular i nput ha s a large influence on the mod

el outpu t at a pa rticular point w ould be discou nted if the prob ability

associa ted with that particular point is very low. A sim ilar approach m

ay be used to

analy ze inp uts that are con tinuous varia bles if a few po ints representing the range of values are selected. For exam ple, low, medium (or nom inal), and high v

alues may be selected for each of the co ntinuous input variab les and then the relative influence of each of the input variables can b e computed as in the case of discrete inpu ts. One limitation

of this approach, however, is t

hat the continu

ous nat ure of t he inpu ts ma kes it impo ssible t

o calcu late an exact probabil ity for each of the points. Generally, in a PRA, man y if not all of the inputs will be random variables described by proba

bility distributions and it will be necessary to quantify the influenc e of each input, X i , over t he entir e rang e of X

i . An esti mate of t he part ial de rivati ve can be o btain ed by incr ementin g X i by a sm all amount, say ) X i while k eeping the other input s fixed and reevalua ting t he mo del outp

ut Y . The resulting change in Y divide d by ) X i will approximate M Y/ M X i at the nominal point.

As previously

noted, one com

plication to using the partial derivative to quantify the influence of X i is that the part

ial deriv ative is influenced by the units of meas ureme nt of X

i . One way t

his is accomplished is to divide the partial deriv ative by the ratio of the nomin al point estimates, Y* / X i *(or equival ently mul tiply by X i */ Y* ). An approxim

ation of the norm

alized partial deriva

tive is given by Equation A- 12.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 32 E XHIBI T A-5

S IMPLIF YING A SSUMPT IONS IN R EGRES SI ON A NA LYS IS

C Y is a l inear funct ion of t he unknown coefficients ( $ i )

C Succ essive v alues o f Y are uncorr elat ed

C Va riance of Y

is const ant for all val ues of inputs (X

i ) Equation A- 12

This is the same as the

equation for ca

lculatin g sens itivity

ratios ( Section

A.2.1 .3), or

elastici ty (see Equation A- 5). As with the S R approach, the normalized partial derived can be weighted by characte ristics o f the in

put va riable ( Section

A.2.1 .4). One appr

oach is to div

ide by the rati o of sta

ndard deviations ( F Y

/ F X ), where F Y is the s tandard devia tion of Y and F X is the standard

devia tion of X . This method requires tha

t the standard deviations be known, or that a suitable estim ate can be obtained. As previously

noted, if the relationship between Y and all of the inputs is nonlinear, the influence of a par ticular i nput m ay vary de

pending on the value

of that input. One a

pproach to this problem

is to conside r a range of v alues for the in put and to exam ine the influence ove

r that ra nge. If the input is considered to be a random variable fo llowing som e specified probability d

istribution, then it m

ay be desirable to look at the influence that the random input has on the model ou

tput across the distribution of input values. This ca

n be accomplishe d with a Monte Carlo a pproach. Another tec

hnique that addresses nonlinearities is to calculate contrib utions to variance u sing input va riables that are transform ed (e.g., lognormal o

r power transformatio

n). 

### A.3.3 REGRES SIO N 

### A NAL YSIS 

### : R2 

### , P EARSON 

### R, AND 

### P ARTIAL 

### C ORRELATI ON 

### C OEFFICIENTS In order to u

nderstand R 2 , it is necessary to first understand simple and multiple linea

r regression. In regress ion analysis, we are interested in obtaining an equation that relates a dependent variable ( Y ) to one or more in

dependent variables (

X ): Equation A- 13
where 

### $ 0 and 

### $ 1 are regression coefficien ts, and g is called a random error. Equation A- 13 is the general equation for a simpl e linear regre ssion, because

there is only one Y a

nd one X varia

ble, an d their relation ship can

be desc ribed by a line w ith inte rcept 

### $ 0 and slope 

### $ 1 . Note that linear

regression refers to the linear relationship between parameters ( 

### $ 0 , 

### $ 1 ), not X and Y . Thus, the equation is considered linear. Multiple linear regression involves m

ore than one X

related to one Y , while multiva riate regression involv

es more than one Y

to more than one X

. The rand

om er ror, g , represents the difference between an ob served Y value (calculated from the observ

ed input variables), and a Y valueRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 33 predicted by the reg

ression line ( í ). It is also called the residual

(i.e., g =y– í ). The random e rror takes into account all unpredictable and unknown factors that are not included in the model. Exhibit A- 5 gives some of the s impli fying

assum ptions t hat app ly to r

egress ion ana

lysis. Assum

ptions a bout g are that the random error has mean = 0 and constant variance, and is uncorrelated am ong observa

tions. One m

ethod of findi ng the best re gressi on line is to m inimi ze the

residua l sum of squa res (i.e ., leas t-sq

uares m ethod), also cal led the sum o f square s due to error (S SE).
In term s of sensitivity analysis, we are interested in how m uch of the variation in Y

can be explained by the variation in X , and how m

uch is unexplained (due to random error). If a scatter plot of paired observations (

x , y ) shows

that ou r regre ssion li

ne intersects al l of the

observ ations exactly , then all of the variation in Y

is expl ained by X

. Another way

of stating this is tha

t the difference between the mean output ( ) and an observed y ( y i ), or ( y i - ), is equal to the difference between the m

ean output and a predicted y or ( ). In g eneral, the total dev

iation o f y i from is equal to the sum of the deviation d

ue to the regres sion line plus t

he dev iation d ue to ra ndom error:

Equation A- 14

Thus, the total sum of squares (SST) equals the sum of squares due to error (SSE) plus the sum of squares due to reg

ression (SSR).
A.3.3.1 C ALCU LATIO NS O F R2 AND A DJUSTED R2

The R 2 term is a m

easure of how well the regression line exp lains the variation in Y , or:

Equation A- 15

where R 2 is called the coefficient of multiple determin ation and R is called the multiple correlation coefficient . If R 2 =0.90 f

or a cer tain linear model, we could conclu de that the inpu

t varia bles (X 1 , X 2 ,...X k ) explain 90% of the v ariation in the output v

ariable ( Y ). R 2 reduces to the coefficient of determination r 2 for simple linear reg

ression when one inde

pendent variable (

X ) is in the regression model. The sample correlation coefficient , r , is a measure of the association between X and Y, and calculat ed by Equati on A- 16. I

t is also referred to as the Pears on prod

uct momen

t correl ation co efficien

t.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 34 Equation A- 16

In addition, r

is an estimate of the unknown p opulation parameter, D , defined by E quation A-17 :
Equation A- 17

where F X and F Y denote the population standard deviations of the random v ariables X and Y , and w here F XY is called the covarian ce between X

and Y . The covariance F XY is a population param eter describing the average am

ount that two va

riables “covary”. Thus, another way of thinking about a correlation coeffici ent (

R ) is that it reflects the ratio of the covariance betwee n two variables d

ivided by the product of their respective standa rd deviations; and the value alway s lies between - 1 and +1. @Risk and Crystal Ball® provide both the R 2 for the entire mo del, as well as the co rrelation coefficients for each inpu t variab le (or re gresso

r). The highe

r the value of R i for X i , the more sen sitive the output v ariable is to that input variable. Although the calcu

lations are the same , there is a subtle conce ptual difference between the coefficient of determinatio n ( r 2 ) from regress ion, and the square o f the correlation coefficient. When evaluating two

variables (X, Y

), the k ey is w hether X is interpreted as a “fixed” quantity (i.e., an explanatory va

riable), or a random

variable just like Y

. In reg ression analysis, r 2 measures how

well the regression line explains the variation in Y giv en a par

ticular v alue for X (Equation A- 15). Correlation requires that X be considered a random variable, typ ically having a bivariate norm

al distribution with Y (see Appendix B).
One art ifact of regre ssion an

alysis is that R 2 increases as you add more and more input v ariables to your m odel; however, the increased fit of the m odel due to one or m

ore of the input variables may be insignificant. Som

etimes an adjusted R 2 is calculated to take into account the num ber of input variables (called regres sors) in

the m odel (k ) as wel l as the

numb er of ob servat

ions in the data

set ( n ):
Equation A- 18

While R 2 giv es the proportio n of the total variation of Y

that is explained, (Equation A- 18) takes int o acc ount the d egre es of free dom ( df

), and give s the pr

oportion of the

total varianceof Y

that is expla ined (va rianc e = varia tion / df); o r stat ed simpl y, is the R 2 correcte d for df , where df is describ ed by

[1 -k/(n- 1)].

C If the relationship between an input v ariable and an output v

ariable is strong, bu

t nonlinear, the R 2 statistic will be misleadingly low.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 35 C If the m

eans of the samp ling data are used rather than the individ ual observations for e ach variable, R 2 will be misleadin

gly hig h. This is because tak

ing the mean of a sample reduces the fraction of the total variation due to random variation (see discussio n of random e

rror above). This is a

n impor tant con siderati on whe

n tryi ng to interpre

t the res ults of regres sion analyses that incorpora

te data averaged over different spatial scales (e.g., reg ression of PbB on s oil lead concentrations tak en at the city block

level may

give an inflated R 2 value if the sam pling data are av

eraged over a larger spatial scale, s

uch as t he cens us tract level ). A multiple reg

ression analysis can a

lso be performed to estimate the regress ion coe fficients (see Append ix A.3.3 ). Each c oeffi cient essent ially r epres ents an “average ” value o f the pa rtial derivat ive across the entire distribution of the input. The regress ion coefficient, like the partial derivative, depends on the units of m

easurement so, a s in the case of the pa rtial derivative, it must be norm alized. This can be accomplished by multiply ing the regressio

n coefficient by the ratio of estimated stan dard deviations s y /s x . A convenient way to carry o ut a sensitivity analysis is to perform a stepwise regression analysis. Some statistical software packages (e.g., SAS,SPSS) offer a v ariety of different approa ches for this; however, in general, they can be c lassifie d into t wo ge neral ca tegori es: forward sele ction an d back ward elimin ation. In t

he forward sele ction, the inpu ts are ad ded to t he mo del one

by one in the

order o f their contribution. I

n the backward elimination, all of the inputs are used in the model initially and then they are dropped one by

one, eliminating

the least important input at each step. A true stepwise procedure is a variation on the forward selection approach where an input can drop out again once it has been selected into the model if at some point other inputs enter the m odel that account for the same inform ation.
A.3.3.2 RELATIVE P ARTIAL S UM OF S QUARES (RPSS) The relative partial sum of squares (RPSS) meas ures the sensiti vity of the m odel ou tput to each of the input variables by partitioning the variance in the

output attributable to each variable using multiple regression techniques (Rose et a l., 1991). The RP SS is presented as a p ercentage reflecting the proportion of influence a g iven variable has on risk. The res ults of RPSS are intu

itive and generally easy to understand. Briefly, the RPS S represents the percentage of the total sum of squares attributable to ea ch of the variab les. To calculat

e RPSS for v ariable

V i , the difference between the regression sum of squares (RSS) for the full model and the regression s um of squares for the model w ith V i missing (RS S -i ) is divided by the total sum of squares (TSS) and expressed as a percentage:
Equation A- 19
This procedure can be thoug ht of as analogou s to least squares linear reg ression, but perform

ed in the n -dimen sional parameter spac e of the risk equatio n. Since this approac

h depends on the ad

equacy of the linear regression model between the output variable (e. g., risk) an d all the variables, an additional diagnostic is to check

how close R 2 is to 1.0. For eq uations with more than three param eters (such as those used in Superfund

risk assessm

ents), the compu

tational overhead of this process is large a nd requ ire s spe cif

ic co mpute r pro grams . The soft ware progr am Crystal Ball ® does not perform th

is calculat ion, bu t it can be dete

rmine d with most standar d statis tical so ftware p ackages that performRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 36 multiple regression. @Risk performs a calculation similar to this called multivariate stepw ise regression that yields correlation co efficients in lieu of percent con tributions to output variance.
A.3.3.3 S PEARMAN ’ S R ANK C ORRELATION C OEFFI CIENT (RHO ) The validity of usin

g indices such as regression coefficients, co rrelation coefficients, and partial corre latio n coeff

icien ts depe nds on t he assu mptions of the un derlyi ng linear mode l being

met. If t here is any doubt that a data set satisfies the m odel assumptions, a nonparametric measure of correlation based on the rank orders

of the inputs and asso ciated outputs can be us ed. The Spearm an Rank correlation coefficient is a nonparam etric statistic; it measures an association between v

ariables that are either count data or data meas

ured on an ordinal scale, as opposed to data measured on an interval or ratio scale. An exam ple of a n ordinal scale

would be the r

ankin g of s ites bas ed on their relat ive m ean soil concent rations. For ex ample , if the re are fo ur categories of

soil contami

nant concentra

tions, sites wi

th the highest concentrations may re ceive a rank of 1 while sites with lowest concentrations m ay receive a rank of 4. Ordinal scales indicate relative positions in an ordered series, not “how m uch” of a difference exists between successive positions on a scale. To calculate the Spearm

an rank correlation co efficient, assign a rank to each of the input variables (

X j ) and output variab les ( Y k ). For each ra nked pair (

X j, Y k ), calculate the difference, d , between the ranks. For example, if the first observation for v ariable X has a ranking

of 5 (relative to all of the observations o f X

), and the corr

espond ing value of Y

has a ranking

of 3 (relative to all of the observations of Y ), the difference ( d

) is equal to 5–3=2. Spearman rho ( r s ) is calculated as:

Equation A- 20
Hence (-1 # r s # 1.0), and r s =-1 describes a

perfect indirect or neg ative relationship betwe en ranks in the sense that if an X element incre ases, the cor respo nding Y element decreases. Similarly, r s =0 suggests that there is no relationship between X and Y

. The Pearson product m

oment correlation co efficient is equal to the Spe arman rank correlation coefficient when interva

l/ratio values of the m

easured observations (

X, Y ) are rep laced w

ith thei r respective ranks.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment Appendix A ~ Decem

ber 31, 2001

Page A- 37 

### REFERE NCES F OR 

### A PPE NDIX 

### A Cullen, A.C. an

d H.C. Frey . 1999. Probabilistic Techniq ues in Exposure Asse ssment. A H andbook for Dealing with Variability

and Uncertainty in Models and I nputs. Plenum Press. Hamby, D.M. 1994. A

Review of Techniq

ues for Parameter Sensitivity An alysis of Env

ironmental Models

. Environ. Monit. and A ssess. 32:135–154. Helsel, D.R. and R. M. Hirsc h. 199 2. Stat

istical M ethods in Water Resour ces. El sevier

Scienc e B.V. Iman, R.L. and J.C. Helton. 1988. An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Mod

els. Risk An al. 8:71–90. Iman, R.L. and J.C. Helton. 1991. The R epeatability of Uncertain ty and Sensitiv

ity Analyses for Complex Prob abilistic Risk Assessm ents. Risk An al. 11:591–606. Merz, J., M.J. Sma

ll, and P. Fischbeck . 1992. Measuring Decision Sensitivity : A Combine d Monte CarloLogistic Reg

ression Approach. Medical Decision Making, 12: 189–196. Morgan , M.G. an d M. Henr ion.19 90. Unce rtain ty: A Guid e to Deal ing with Uncert ainty i n Quant itati ve Risk and Policy

Analysis. Cam

bridge Unive

rsity Press. Palisade Corporation. 19 94. Risk An alysis a nd Simu lation A dd-In fo r Micros oft Exce l or Lot us 1-2-3

. Windows Versi on Release 3.0 User’ s Guide, Palisade Corporation, Newfi eld, NY. Rose, K.A., E.P . Smith, R .H. Gardner, A .L. Brenkert, and S.M. Bartell. 199 1. Parameter S ensitivities, Monte Carlo Filtering, and Model Forecasting Under Uncertainty

. J. Forecast 10:117–133. Saltelli , A and

J. Marivort. 19 90. Non-Pa ramet ric Stat istics in Sensiti vity analy sis for M odel Ou

tput: A Comparison of Selected Techniques. Reliab. Engin. Syst. S af. 28:299–253. Shevenell, L. and F.O. Hoffm an. 1993. Necessity

of Uncertainty A

nalyses in Risk

Assessment. J Hazard Mater . 35:369–385. Stern, A.H. 199 4. Derivation of a Target Level of Lead in Soil at Reside ntial Sites Corresponding to a de minimi s Contribution to Blood

Lead Concentration. Risk An al. 14:1049–1056.
U.S. EPA. 1 997. Guidin

g Princ iples fo r Monte C arlo An alysis . Risk Assessm ent Forum and

National Center for Environ

mental Assessm

ent. EPA/630/R- 97/001.
U.S. EPA. 1 999. TRIM, Total Risk Integrate d Methodology, TRIM F

ATE Technical Supp ort Document Volume I: Desc ription of Modul

e . Office of Air Qua

lity Planning and Standards. EPA/43/D-99/002A.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-1 

## A PPENDIX 

## B S ELECTION AND 

## F ITTING OF 

## D ISTRIBUTIONS 

### B.0 I NTRODUCTION An important step in Monte Carlo analysis (MCA) is to select the most appropriate distributions to represent the factors that have a strong influence on the risk estimates. This step in the development of a Monte Carlo model can be very challenging and resource intensive.



### L

Specifying probability distributions for all of the input variables and

parameters in a probabilistic risk assessment (PRA) will generally not be necessary. If the sensitivity analysis indicates that a particular input variable does not contribute significantly to the overall variability and uncertainty, then this variable may be represented as a point estimate. As discussed in Appendix A, however, different approaches to sensitivity analysis may be applied throughout the tiered approach (e.g., sensitivity ratios, correlation analysis), and the ability to reliably identify variables as being minor or major can vary. Sometimes it can be helpful to develop probability distributions based on preliminary information that is available from Tier 1 in order to explore alternative options for characterizing variability and uncertainty. Likewise, sometimes the important “risk drivers” are apparent, and resources can be allocated to fully characterize the variability and uncertainty in those input variables. Therefore, the process of selecting and fitting distributions may also be viewed as a tiered approach. This appendix reviews the methods available to select and fit distributions and provides guidance on the process for determining appropriate choices depending on the information needed from the assessment and the information available to define the input variables. In PRA, there are some important distinctions in the terminology used to describe probability distributions. A probability density function (PDF), sometimes referred to as a probability model, characterizes the probability of each value occurring from a range of possible values. Probability distributions may be used to characterize variability (PDFv) or uncertainty (PDFu). One advantage of using a PDFv and PDFu is that distributions represent a large set of data values in a compact way (Law and Kelton, 1991). For example, a lognormal distribution provides a good fit to a large data set of tap water ingestion rates ( n =5,600) among children ages 1 to 11 years (Roseberry and Burmaster, 1992). Therefore, the distribution type (lognormal) and associated parameters (mean and standard deviation) fully describes the PDFv for intake rates, from which other statistics of interest can be calculated (e.g., median, and 95 thpercentile). Reducing a complex exposure model to a series of representative and wellfitting distributions can facilitate both the quantitative analysis and the communication of the modeling methodology. Alternatively, a PDFu may be specified to characterize parameter uncertainty. For example, the sample mean ( ) is generally an uncertain estimate of the population mean ( 

### : ) due to 

### x measurement error, small sample sizes, and other issues regarding representativeness (see Section B.3.1). A PDFu can be used to represent the distribution of possible values for the true, but unknown parameter. Understanding whether uncertainty or variability is being represented by a PDF is critical to determining how the distribution and parameters should be specified and used in a PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-2 E XHIBIT B-1
D EFINITIONS FOR A PPENDIX B Bayesian Analysis - Statistical analysis that describes the probability of an event as the degree of belief or confidence that a person has, given some state of knowledge, that the event will occur. Bayesian Monte Carlo combines a prior probability distribution and a likelihood function to yield a posterior distribution (see Appendix D for examples). Also called subjective view of probability, in contrast to the frequentist view of probability. Bin - Regarding a histogram or frequency distribution, an interval within the range of a random variable for which acount (or percentage) of the observations is made. The number of bins for a histogram is determined on a caseby-case basis. In general, equal interval widths are used for each bin; however, in some cases (e.g., Chi-square test), individual bin widths are calculated so as to divide the distribution into intervals of equal probability. Countably Infinite - Used to describe some discrete random variables, this term refers to a set of numbers that can be counted with integers (e.g., one, two, three) and that has no upper limit. Examples include the number of tosses required for a coin to show a head—we can count each toss, but it is possible that at least one more toss is needed. The number of dust particles in a volume of air is another example. Countably finite implies there is an upper limit (e.g., days of work per year). Cumulative Distribution Function (CDF) - Obtained by integrating the PDF, gives the cumulative probability of occurrence for a random independent variable. Each value c of the function is the probability that a random observation x will be less than or equal to c . Empirical Distribution Function (EDF) -The EDF, also called the empirical CDF (ECDF), is based on the frequency distribution of observed values for a random variable. It is a stepwise distribution function calculated directly from the sample, in which each data point is assigned an equal probability. Frequency Distribution or Histogram - A graphic (plot) summarizing the frequency of the values observed or measured from a population. It conveys the range of values and the count (or proportion of the sample) that was observed across that range. Goodness-of-Fit (GoF) Test - A method for examining how well (or poorly) a sample of data can be described by a hypothesized probability distribution for the population. Generally involves an hypothesis test in which the null hypothesis H 0 is that a random variable X follows a specific probability distribution F 0 . That is, H 0 : F=F 0 and H

a : F ... F

0 . Independence - Two events A and B are independent if whether or not A occurs does not change the probability that B occurs. Likewise, knowing the value of B does not affect the value of A . Input variables, X and Y, are independent if the probability of any paired values (X, Y) is equal to the probability of X multiplied by the probability of Y. In mathematical terms, X and Y are independent if f(X, Y)=f(X) x f(Y). Independence is not synonymous with correlation. If X and Y are independent, then their correlation is zero, Cor(X, Y)= 0. But, the converse is not always true. There may be a nonlinear relationship between X and Y that yields Cor(X, Y)=0, but the variables are highly dependent. Nonparametric Method - Also called a distribution-free method, a procedure for making statistical inferences without assuming that the population distribution fits a theoretical distribution such as normal or lognormal. Common examples are the Spearman rank correlation, (see Appendix A) and the bootstrap-t approach.. Parameter - In PRA, a parameter is a quantity that characterizes the probability distribution of a random variable. For example, a normal probability distribution may be defined by two parameters (e.g., arithmetic mean and standard deviation). Parametric Distribution - A theoretical distribution specified by a distribution type and one or more parameters. Examples include the normal, Poisson, and beta distributions.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-3 E XHIBIT B-1

— Continued D EFINITIONS FOR A PPENDIX B
Probability Density Function (PDF) - A function representing the probability distribution of a continuous random variable. The density at a point refers to the probability that the variable will have a value in a narrow range about that point. Probability Distribution - The mathematical description of a function that associates probabilities with specified intervals or values for a random variable. A probability distribution can be displayed in a graph (e.g., PDF or CDF), summarized in a table that gives the distribution name and parameters, or expressed as a mathematical equation. In PRA, the process of selecting or fitting a distribution that characterizes variability or uncertainty can also be referred to as applying a probability modelto characterize variability or uncertainty. In this guidance, the probability model is considered to be one source of model uncertainty. Step Function - A mathematical function that remains constant within an interval, but may change in value from one interval to the next. Cumulative distribution functions for discrete random variables are step functions. Z-score - The value of a normally distributed random variable that has been standardized to have a mean of zero and a SD of one by the transformation Z=(X– : )/ F . Statistical tables typically give the area to the left of the z-score value. For example, the area to the left of z =1.645 is 0.95. Z-scores indicate the direction (+/-) and number of standard deviations away from the mean that a particular datum lies assuming X is normally distributed. Microsoft Excel’s NORMSDIST(z) function gives the probability p such that p=Pr(Z # z), while the NORMSINV(p) function gives the z-score z p associated with probability p such that p=Pr(Z # z p ).

B.1.0 C ONCEPTUAL A PPROACH FOR I NCORPORATING A P ROBABILITY D ISTRIBUTION IN A PRA Often, more than one probability distribution may appear to be suitable for characterizing a random variable. A step-wise, tiered approach is recommended for incorporating probability distributions in a PRA. This appendix provides guidance on selecting and fitting distributions for variability and parameter uncertainty based on the overall strategy given in Exhibit B-2. Many of the same principles of selecting and fitting distributions are also given in EPA's Report of the Workshop on Selecting Input Distributions for Probabilistic Assessments (U.S. EPA, 1999a). Probability distributions may be developed to characterize variability or uncertainty. Example flow charts for specifying a PDFv and PDFu are given in Figures B-1 and B-2, respectively. Both approaches outline an iterative process that involves three general activities: (1) identify potentially important sources of variability or uncertainty to determine if a PDF may be needed; (2) apply the general strategy given in Exhibit B-1 and evaluate plausible alternatives for distributions and parameter estimates; and (3) document the decision process. The flowcharts provide a general outline of the process and contain terms which are explained in subsequent sections. Just as with the point estimate approach, different sites may require different probability distributions for input variables, depending on the unique risk management issues and sources of uncertainty. E XHIBIT B-2 G ENERAL S TRATEGY FOR S ELECTING AND F ITTING D ISTRIBUTIONS (1) Hypothesize a family of distributions (2) Assess quality of fit of distribution (3) Estimate distribution parameters (4) Assess quality of fit of parametersRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-4 B.2.0 P RELIMINARY S ENSITIVITY A NALYSIS Selecting and fitting probability distributions for allof the input variables can be resource intensive and is generally unnecessary. Ideally, a subset of variables could be identified that contribute to most of the variability and uncertainty in a risk estimate. Sensitivity analysis can play an important role in helping to identify and quantitatively rank the major exposure pathways and variables. Since the information obtained from a sensitivity analysis may vary, depending on the approach(es) used and the information available to characterize the input variables, risk assessors should understand inherent limitations of each approach. A variety of approaches that are common for Tier 1 and 2 analyses are described and applied to a hypothetical example in Appendix A. In a Tier 1 assessment, sensitivity analysis is typically limited to exploring the effect of alternative point estimates on the risk estimate. These methods can be helpful if additional information regarding the variability in the input variables is incorporated into the analysis (i.e., sensitivity scores). Alternatively, a reasonable approach is to specify preliminary probability distributions for one or more inputs in order to maximize the advantages of probabilistic methods. The difference between a preliminary distribution and a subsequent distribution reflects the level of effort invested in characterizing variability and uncertainty. If a robust data set is available in Tier 1 to define point estimates, then a preliminary distribution may, in fact, fully characterize variability with very high confidence. For other variables, summary statistics, rather than sample data, may be available, allowing for estimates of central tendency or plausible ranges. The use of preliminary distributions reflects an effort to employ more robust sensitivity analysis techniques without expending the effort and resources that might otherwise be applied to a PRA in Tier 2. The goal of the preliminary analysis would not be necessarily to evaluate risks and/or develop a PRG; rather, the focus would be on identifying input variables that may be important to explore more fully. Preliminary sensitivity analysis can provide insight into the importance of selecting among alternative probability distributions and exposure scenarios. One-dimensional Monte Carlo simulations with preliminary (or screening-level) distributions can be run prior to engaging in a more involved process of selecting and fitting distributions. The distributions can be selected based on knowledge regarding the mechanisms that result in variability, and information already available for determining point estimates (e.g., summary statistics, U.S. EPA guidance, etc.). Table B-1 provides examples of preliminary distributions that might be selected based on the type of information available, sometimes referred to as the state of knowledge . In many cases, the distribution is intended to estimate the plausible bounds of a variable, while requiring no additional data collection effort. For example, given estimates of a lower bound [min], upper bound [max], and the assumption that each value is equally likely, a uniform distribution would be used to represent variability (or parameter uncertainty). If no mechanistic basis for selecting a distribution exists, then the preliminary distribution would be chosen based on the available information. For example, given the estimates of the arithmetic mean [ 

### : ] and a percentile value [a] for a random variable, an exponential distribution might be recommended with 

### 8 =1/ 

### :
. Guidance on matching the choice of the distribution to the state of knowledge is extended to a more diverse array of scenarios later in this appendix (see Table B-4).RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

1 The preliminary distributions are based in part on maximum entropy concepts. Maximum entropy is a technique for determining the distribution that represents the maximum uncertainty allowed by the available information and data (Vose, 1996). Although the approach can be used to quickly define distributions that maximize uncertainty, the credibility of the distribution depends on the use of accurate, unbiased information.

2 See Table B-2 for more detailed descriptions of selected distributions. Page B-5 Table B-1. Examples of Preliminary Distributions Based on Information Available 1 , 2

Information / Constraints Distribution Shape [a, b] uniform [a, m, b] triangular [ a, b, α 1 , α 2 , β ] beta [ 

### : , 

### F ] normal γ exponential [a, b, μ, σ ] Johnson Sb, Lognormal [ α, β ] gamma a=minimum, b=maximum, m=mode, α =shape parameter, : =mean, F =standard deviation, γ =average rate of occurrence of events, β= scale,
It may be informative to explore alternative choices for distributions applied to the same variable. For example, a simple yet informative approach is to run two 1-D MCA simulations for variability with an input variable characterized first by a Johnson Sb (i.e., a four-parameter lognormal distribution; Hahn and Shapiro, 1967) and then by a normal distribution. The difference in the risk distribution, especially at the percentile that is relevant to the risk management decision (e.g., 95 thpercentile), may offer insights regarding the importance of the shape of the PDFv.
B.3.0 W HAT D OES T HE D ISTRIBUTION R EPRESENT ? Distributions may be specified to characterize variability or uncertainty. Often, a Monte Carlo simulation of variability will focus on describing differences between individuals in a population (i.e., inter-individual variability). In this case, the goal is to select a distribution that is representative of the target population—the set of all receptors that are potentially at risk. There may be uncertainty that the choice of PDFv reflects variability in the target population. In general, risk assessors should fully disclose uncertainties in the PDFv, especially because the use of a distribution instead of a point estimate may inappropriately suggest that there is a greater state of knowledge. Following the tiered process (see Chapter 2, Figure 2-1), there are multiple opportunities to consider consequences of alternative modeling approaches early in the process of developing a probabilistic model. The importance of relating the distribution to the target population, clearly distinguishing between variability and uncertainty, and evaluating data representativeness is emphasized in Sections B.3.1, B.3.2 and B.4.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-6 B.3.1 C ONCEPTS OF P OPULATION AND S AMPLING The distinction between a target population, a sampled population, and a statisticalpopulation should be considered carefully when evaluating information for use in both Tier 1 and Tier 2 of a PRA. The target population is often considered to be the “population of concern”. A risk assessor is often interested in quantifying specific attributes of the population (e.g., exposure duration, exposure frequency, etc.). A sampled population is the set of receptors available for selection and measurement. For purposes of this appendix/guidance, the sampled population may be the target population or it may be a different population that is thought to be representative of the target population. For purposes of this guidance, a statistical population is an approximation of the target population based on information obtained from the sampled population. Distributions are generated from representative sample populations to make inferences about the target population. Ideally, a sampled population should be a subset of a target population and should be selected for measurement to provide accurate and representative information about the exposure factor being studied. However, defining representative samples is a matter of interpretation.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-7 Sensitivity Analysis (i.e., Tier 1 and Tier 2)

Is the factor influential? Use a health protective point estimate Present in workplan and report

Is the factor amenable to expert judgment?

Conduct expert elicitation for PDF Are the data representative of

of the target population? No

Yes

Yes Yes No

No No

Continued on next page No Do sufficient data exist or can they be

collected to run a refined 1-D MCA? Yes

Yes Present PDF/EDF in workplan and report Can the data be adjusted to better represent the target population (e.g., weighting factors)

Consider the mechanistic characteristics of the data (e.g., continuous or discrete variable) Figure B-1 (page 1 of 2). Conceptual approach for incorporating probability distributions for variability in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-8 Present PDF/EDF in work plan and report (Continued from previous page) Summary statistics and graphical data exploration (e.g., histograms)

Use an EDF?

Choose a type of PDF (Normal, Weibull, etc.) Estimate percentile values of EDF No Yes

Appropriate Goodness-of-Fit? Estimate parameters

Apply truncation limits as appropriate

Specify a mixture of distributions

Would a mixture of distributions better represent the data? Estimate tails and truncation limits No

Yes

No

Yes Present PDF/EDF in work plan and report (Continued from previous page) Summary statistics and graphical data exploration (e.g., histograms)

Use an EDF?

Choose a type of PDF (Normal, Weibull, etc.) Estimate percentile values of EDF No Yes

Appropriate Goodness-of-Fit? Estimate parameters

Apply truncation limits as appropriate

Specify a mixture of distributions

Would a mixture of distributions better represent the data? Estimate tails and truncation limits No

Yes

No

Yes

Figure B-1 (page 2 of 2). Conceptual approach for incorporating probability distributions for variability in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-9 

### Identify dominant exposure pathway(s) from sensitivity analysis (Appendix A, Section B.2.0)

Identify factor(s)that may contribute to uncertainty in the risk distribution (Appendix A, Section B.2.0) Select a plausible risk exposure model for the exposure pathway (Fig. B-2b)

Select probability distribution(s) for variability in exposure factor(s) (Fig. B-1)

Quantify parameter uncertainty with point estimates or distribution(s) (Fig. B-2c)

Run simulation to propagate variability and uncertainty (e.g., multiple

1-D MCAs; 2-D MCA; MEE, etc.)

Continue quantifying uncertainty? Present results in graphical and tabular format Yes No

Figure B-2a (page 1 of 3) . Conceptual approach for quantifying model and parameter uncertainty in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-10 Run simulation with candidate exposure model Identify dominant exposure pathway(s) from Sensitivity Analysis (Appendix A and Section B.2.0) Is more than one model plausible for an exposure pathway?

Identify and evaluate the exposure factor(s) quantified by a candidate exposure model

Purpose and Objectives • regulatory context • scientific questions addressed • application niche (physical,

chemical, biological system) • status of agency and/or peer review

Defining and Limiting Components • process(es) characterized

(e.g., transport, diffusion, volatilization, bioavailability, etc.) • temporal and spatial scales • level of aggregation/simplification

Theoretical Basis • mechanistic basis for algorithms

• numerical or analytic solution

Select alternative exposure model No Yes No

Yes

Is the exposure model

appropriate? Run simulation with candidate exposure model Identify dominant exposure pathway(s) from Sensitivity Analysis (Appendix A and Section B.2.0) Is more than one model plausible for an exposure pathway?

Identify and evaluate the exposure factor(s) quantified by a candidate exposure model

Purpose and Objectives • regulatory context • scientific questions addressed • application niche (physical,

chemical, biological system) • status of agency and/or peer review

Defining and Limiting Components • process(es) characterized

(e.g., transport, diffusion, volatilization, bioavailability, etc.) • temporal and spatial scales • level of aggregation/simplification

Theoretical Basis • mechanistic basis for algorithms

• numerical or analytic solution

Select alternative exposure model No Yes No

Yes

Is the exposure model

appropriate?
Figure B-2b (page 2 of 3). Detailed conceptual approach for incorporating model uncertainty in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-11 Present results in graphical and tabular format Estimate parameters (e.g., MLE, method of moments, etc.)

Is information available to quantify parameter uncertainty? Is the parameter amenable to expert elicitation?

Conduct expert elicitation Select distribution (or point estimate) for uncertainty

Run simulation to propagate variability and uncertainty

Should an alternative

probability model (i.e., PDFv or PDFu) be explored? Continue with process acknowledging

limits of data No

Yes Yes

Yes No No

Run sensitivity analysis to identify important sources of uncertainty Identify candidate probability distribution(s) for variability (Fig. B-1): • mechanistic basis for variability • exploratory data analysis • expert judgment

Present results in graphical and tabular format Estimate parameters (e.g., MLE, method of moments, etc.)

Is information available to quantify parameter uncertainty? Is the parameter amenable to expert elicitation?

Conduct expert elicitation Select distribution (or point estimate) for uncertainty

Run simulation to propagate variability and uncertainty

Should an alternative

probability model (i.e., PDFv or PDFu) be explored? Continue with process acknowledging

limits of data No

Yes Yes

Yes No No

Run sensitivity analysis to identify important sources of uncertainty Identify candidate probability distribution(s) for variability (Fig. B-1): • mechanistic basis for variability • exploratory data analysis • expert judgment

Figure B-2c (page 3 of 3). Detailed conceptual approach for incorporating parameter uncertainty in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-12 B.3.2 C ONSIDERING V ARIABILITY AND U NCERTAINTY IN S ELECTING AND F ITTING D ISTRIBUTIONS Multiple probability distributions may be used to describe variability and uncertainty in an input variable. For example, a normal probability distribution may be selected to characterize variability in body weight, whereas a uniform distribution may selected to characterize uncertainty in the estimate of the arithmetic mean of the normal distribution. The appropriate interpretation and analysis of data for an exposure variable will depend on whether one is specifying a PDFv or PDFu. Figure B-1 outlines one useful process for selecting distributions for variability, whereas Figure B-2 (three pages) outlines a useful process for quantifying both model and parameter uncertainty. Variability generally refers to observed differences attributable to true heterogeneity or diversity in a population (U.S. EPA, 1997b). Variability results from natural random processes. Inter-individual variability may stem from environmental, lifestyle, and genetic differences. Examples include human physiological variation (e.g., natural variation in body weight, height, breathing rates, drinking water intake rates), changes in weather, variation in soil types, and differences in contaminant concentrations in the environment. Intra-individual variability may reflect age-specific changes (e.g., body weight and height). Variability is not reducible by further measurement or study. A PDF for variability can usually be obtained by fitting a distribution to the sample measurements. Sources of Uncertainty Uncertainty generally refers to the lack of knowledge about specific factors, parameters, or models (U.S. EPA, 1997b). Although uncertainty in exposure and risk assessment may be unavoidable due to the necessary simplification of real-world processes, it generally can be reduced by further measurement and study. Parameter uncertainty may stem in part from measurement errors, sampling errors, or other systematic errors in the collection and aggregation of data. Model uncertainty may reflect the simplification of a complex process, a mis-specification of the exposure model structure, a misuse or misapplication of an exposure model, use of the wrong distributional model, and the use of surrogate data or variables. Scenario uncertainty may reflect uncertainty in an exposure model, such as the relevance of specific exposure pathways to the target population. A conceptual exposure model can be used to provide direction in specifying a probability distribution for uncertainty. For example, the concentration term in a Superfund risk assessment typically represents the long-term average concentration to which a receptor is exposed (see Chapter 5). An uncertainty distribution for the concentration term could be developed in part from ideas about the statistical uncertainty of estimating the long-term average from a small sample, and the assumption of random movement of the receptors within a defined exposure unit. Probability Distributions and Model Uncertainty This appendix primarily focuses on methods for quantifying uncertainty associated with both the selection of a variability distribution, and estimating parameters of a distribution. A probability distribution can be referred to as a type of model in the sense that it is an approximation, and often a simplified representation of variability or uncertainty that combines both data and judgment. A broader use of the term model refers to a representation of a chemical, physical, or biological process. In risk assessment, many different models have been developed, with varying objectives, major defining and limiting components, and theoretical basis. Figure B-2b provides a general process for exploring model uncertainty of this type. This figure reflects the concepts and spirit of the Agency Guidance for Conducting External Peer Review of Environmental Regulatory Modeling (U.S. EPA, 1994). In general, EPA regional risk assessors should be consulted in order to determine the types of exposure and risk models that may be plausible for quantifying exposure at a particular site.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-13 Parameter Uncertainty Quantifying parameter uncertainty in a probabilistic model typically requires judgment (see Appendix C). When data are uncertain due to, for example, small sample sizes or questionable representativeness (Section B.3.1), Monte Carlo simulation can be a useful tool for demonstrating the effect of the uncertainty on the risk estimates. It is most important to model uncertainty when the sensitive input variables are uncertain. Uncertainty can be quantified in both the point estimate approach (e.g., a range of possible central tendency exposure values) or a probabilistic approach (e.g., a range of possible values for the arithmetic mean of a distribution). While a quantitative uncertainty analysis may complicate a risk management decision by suggesting that risk estimates are highly uncertain, this information can be helpful by focusing additional efforts towards collecting data and reducing uncertainty in the most sensitive input variables. Likewise, if an estimated risk is below a regulatory level of concern, even after quantifying highly uncertain inputs to the exposure model, the risk manager may be more confident in a decision. As emphasized in Figures B-2a, B-2b, and B-2c, risk assessors should generally refrain from setting ad hoc probabilities to different candidate distributions in a single Monte Carlo simulation. Instead, this guidance strongly recommends exploring model or parameter uncertainty by running a separate simulation with each candidate model. For example, rather than randomly assigning a beta distribution or a lognormal distribution to an exposure variable for each iteration of a simulation, separate simulations should be run with the candidate probability distributions. Similarly, if a range of temporal or spatial scales is plausible for quantifying exposure, multiple simulations should be designed to demonstrate the importance of these assumptions on the risk estimates. Uncertainty in parameter estimates may be characterized using a variety of methods. Similar to a PDF for variability, a PDF for parameter uncertainty may be represented by a probability distribution with a unique set of parameters. Sometimes the distribution for uncertainty can be specified by knowing (or assuming) a distribution for variability. For example, if X is a normally distributed random variable, the Student’s t distribution and the Chi-square ( 

### P 2 ) distribution can be used to develop PDFu’s for random measurement error uncertainty in the sample mean and variance, respectively. The PDFu for both the Student’s t and Chi-square distributions is determined by the sample size ( n ). If a PDFu cannot be determined from the PDF for variability, or assumptions regarding the underlying distribution for variability are not supportable, nonparametric or “distribution free” techniques may be used (e.g., bootstrapping). Both parametric and nonparametric techniques may yield confidence intervals for estimates of population parameters.
B.4.0 D O D ATA E XIST T O S ELECT D ISTRIBUTIONS ? Developing site-specific PDFs for every exposure assumption (or toxicity value, in the case of ecological risk) can be time and resource intensive, and in many cases, may not add value to the risk management decision. For those exposure variables that do exert a significant influence on risk, a PDF may be developed from site-specific data, data sets available in the open literature (e.g., EPA’s Exposure Factors Handbook , U.S. EPA 1997a), or from existing PDFs in the literature (e.g., Oregon DEQ, 1998).At Superfund sites, perhaps the most common exposure variable that will be described by sitespecific data will be the media concentration term. The sample (i.e., collection of empirical measurements) will most often be used to estimate either a point estimate of uncertainty (e.g., an upper confidence limit for the arithmetic mean concentration—the 95% UCL), or a distribution that characterizes the full distribution of uncertainty in the mean. Exposure variables such as ingestion rates, exposure duration, and exposure frequency will most likely be derived from existing PDFs or data sets inRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-14 the open literature. The Agency supports the development PDFs that may be generally applicable to different sites (e.g., body weight, water intake, and exposure duration) (U.S. EPA, 1999b, 2001). Until final recommendations of PDFs are available for the more generic exposure variables, PDFs for exposure variables that lack adequate site-specific data will typically be selected from: (1) existing PDFs; (2) data on the entire U.S. population; or (3) data on subsets of the U.S. population that most closely represent the target population at a site. If risks to a sensitive subpopulation, such as young children, elderly adults, ethnic groups, or subsistence fishermen, are a concern at a site, then existing PDFs or data sets that best characterize these subpopulations would be preferable to national distributions based on the entire U.S. population. If adequate site-specific data are available to characterize any of the exposure variables, distributions can be fit to those data. Uncertainty Associated with Sample Size An appropriate question to consider when evaluating data sets for use in exposure and risk assessment is, “What sample size is sufficient?” Generally, the larger the sample size ( n ), the greater one’s confidence in the choice of a probability distribution and the corresponding parameter estimates. Conversely, for small n , Goodness-of-fit (GoF) tests (see Section B.6.2) will often fail to reject many of the hypothesized PDFs. In general, there is no rule of thumb for the minimum sample size needed to specify a distribution for variability or uncertainty. Increasing a sample size may be an appropriate option to consider when evaluating risk management strategies to reduce uncertainty. Statistical sampling, in general, is important to consider when estimating parameters of a probability distribution. One rule of thumb is that the parameters that reflect the central tendency of a distribution (e.g., arithmetic mean, median, mode) can be estimated with greater confidence than parameters that reflect the extremes of the distribution (e.g., 95 thpercentile). When deciding on appropriate truncation limits (minimum and maximum values), it is unlikely that the statistical sample actually includes the plausible bounds. See Section B.5.7 for more detailed guidance on specifying truncation limits for probability distributions.
B.4.1 W HAT ARE R EPRESENTATIVE D ATA ? The question, “What is a representative sample?”, is important to address when selecting and fitting distributions to data. Many of the factors that may determine representativeness (e.g., sample size and the method of selecting the target, and sample population (Section B.3.1)) are relevant to both point estimate and PRA. EPA’s Guidance for Data Usability in Risk Assessment, Part A (U.S. EPA, 1992) describes representativeness for risk assessment as the extent to which data define the true risk to human health and the environment. The goal of representativeness is easy to understand. However, evaluating data to determine if they are representative is more difficult, especially if the problem and decision objectives have not been clearly defined. The importance of representativeness also varies with the level of complexity of the assessment. If a screening level assessment is desired, for example, to determine if concentrations exceed a health protective exposure level, then representativeness may not be as important as health protectiveness. However, if a complete baseline risk assessment is planned, the risk assessor should generally consider the value added by more complex analyses (e.g., site-specific data collection, sensitivity analysis, and exposure modeling). A tiered approach for making these decisions for a PRA is presented in Chapter 2,RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-15 and examples of more complex analyses are presented in Appendix D. In addition, the Agency (U.S. EPA, 1999a) summarizes the advantages and weaknesses of proposed checklists for risk assessors to evaluate representativeness of exposure factors data. For purposes of this guidance, a surrogate study is one conducted on a sampled population that is similar to, but not a subset of, the target population. When using surrogate data, the risk assessor should generally exercise judgment about the representativeness of the data to the target population. For example, the distribution of body weights of deer mice from two independent samples from similar ecosystems may differ depending on the age structure, proportion of males and females, and the time of year that the samples were obtained. When in doubt about which study results to use in defining a probability distribution, one option is to develop a distribution and calculate risks with each sample independently, and compare the results. This approach can be a simple, but effective type of uncertainty analysis. At a minimum, uncertainties associated with the use of surrogate studies should be discussed in the assessment. In many cases, the surrogate population shares common attributes with the target population, but is not truly representative. The risk assessor should then determine the importance of the discrepancies and whether adjustments can be made to reduce those differences. There are a wide variety of methods that can be used to account for such discrepancies, depending on the available information. Summary statistics (e.g., as presented by the Exposure Factors Handbook , U.S. EPA, 1997a) can be used to estimate linear characteristics of the target population from the sample population. For example, if the mean, standard deviation, and various percentiles of the sample population are known, then the mean or proportion exceeding a fixed threshold can be calculated using a simple weighted average. Adjustment options are more numerous if the risk assessor has access to the raw data. Adjustments for raw data include: weighted averages, weighted proportions, transformations, and grouping of the data based on the available information (e.g., empirical data, and professional judgment). In most cases, the evaluation of data representativeness will necessarily involve judgment. The workplan should generally include a description of the data, the basis for the selection of each distribution, and the method used to estimate parameters (see Chapter 2). Empirical data (i.e., observations) are typically used to select distributions and derive parameter estimates. However, it may be necessary to use expert judgment or elicitation in cases where the quality or quantity of available data are found to be inadequate.
B.4.2 T HE R OLE OF E XPERT J UDGMENT Expert judgment refers to inferential opinion of a specialist or group of specialists within an area of their expertise. When there is uncertainty associated with an input variable, such as a data gap, expert judgment may be appropriate for obtaining distributions. Note that distributions elicited from experts reflect individual or group inferences, rather than empirical evidence. Distributions based on expert judgment can serve as Bayesian priors in a decision-analytic framework. The distributions and Bayesian priors can be modified as new empirical data become available. There is a rich literature base regarding the protocol for conducting expert elicitations and using the results to support decisions (Morgan and Henrion, 1990). Elicitation of expert judgment has been used to obtain distributions for risk assessments (Morgan and Henrion, 1990; Hora, 1992; U.S. EPA, 1997b) and for developing air quality standards (U.S. EPA, 1982).RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-16 E XHIBIT B-3 F ACTORS TO C ONSIDER IN S ELECTING A P ROBABILITY D ISTRIBUTION *

C Is there a mechanistic basis for choosing a distributional family?

C Is the shape of the distribution likely to be dictated by physical or biological properties or other mechanisms?

C Is the variable discrete or continuous?

C What are the bounds of the variable?

C Is the distribution skewed or symmetric?

C If the distribution is thought to be skewed, in which direction?

C What other aspects of the shape of the distribution are known ?

C How well do the tails of the distribution represent the observations? *Source: U.S. EPA, 1997b Bayesian analysis is a statistical approach that allows the current state of knowledge, expressed as a probability distribution, to be formally combined with new data to reach an updated information state. In PRA, Bayesian Monte Carlo analysis (Bayesian MCA) can be used to determine the reduction in uncertainty arising from new information. When combined with techniques from decision analysis, Bayesian MCA can help to determine the type and quantity of data that generally should be collected to reduce uncertainty. The benefits and limitations of expert elicitation, Bayesian statistics, Bayesian MCA, and decision analysis (i.e., value of information [VOI]), as applied to PRA, are discussed in greater detail in Appendix D. 

### B.5.0 F ITTING 

### D ISTRIBUTIONS TO 

### D ATA Sometimes more than one probability distribution may adequately characterize variability or uncertainty. The choice of a distribution should be based on the available data and on knowledge of the mechanisms or processes that result in variability. In general, the preferred choice is the simplest probability model that adequately characterizes variability or uncertainty and is consistent with the mechanism underlying the data. For example, a log-logistic distribution would not necessarily be selected over a 2-parameter lognormal distribution simply because it was ranked higher in a GoF test by a statistical software package. Some distributions (e.g., normal, lognormal) are well known among risk assessors. The statistical properties for these distributions are well understood and the formal descriptions can often be brief. Important factors to consider in selecting a PDF are described in Exhibit B-3. An initial step in selecting a distribution should be to determine if the random variable is discrete or continuous. Continuous variables take any value over one or more intervals and generally represent measurements (e.g., height, weight, concentration). For a continuous variable, a mathematical function generally describes the probability for each value across an interval. Discrete variables take either a finite or countably infinite number of values. Unique probabilities are assigned to each value of a discrete variable. The number of rainfall events in a month is an example of a discrete random variable, whereas the amount of rainfall is a continuous variable. Similarly, the number of fish meals per month is a discrete variable, whereas the average size (mass) of a fish meal is continuous. Another important consideration is whether there are plausible bounds or limits for a variable. For example, it is highly unlikely that an American adult will weigh less than 30 kg or more than 180 kg. Most exposure variables may assume any nonnegative value within a plausible range. Therefore, distributions will generally be truncated at a minimum of zero (or higher), or a probability distribution that is theoretically bounded at a nonzero value may be specified (see Table B-3). A more detailed discussion of factors to consider in selecting a PDF and specifying parameter values is provided below.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-17 B.5.1 C ONSIDERING THE U NDERLYING M ECHANISM There may be mechanistic reasons depending on known physical or biological processes that dictate the shape of the distribution. For example, normal distributions result from processes that sum random variables whereas lognormal distributions result from multiplication of random variables. A Poisson distribution is used to characterize the number of independent and randomly distributed events in a unit of time or space. An exponential distribution would describe the inter-arrival times of independent and randomly distributed events occurring at a constant rate. If, instead, the elapsed time until arrival of the k th event is of interest, then the appropriate probability distribution would be the gamma distribution (Morgan and Henrion, 1990).



### L

In all cases, it is incumbent on the risk assessor to explain clearly and fully the

reasoning underlying the choice of a distribution for a given exposure variable—primarily from a mechanistic standpoint if possible. Table B-2 lists some of the probability distributions that may commonly be used in PRA. This is not an exhaustive list, and the scientific literature contains numerous examples with alternative distributions. Where practicable, a mechanistic basis is presented for the choice of the distribution. For some distributions, such as beta, triangular, and uniform, a mechanistic basis is not offered because it is unlikely that a chemical or biological process will yield a random variable with that particular shape. Nevertheless, such distributions may be appropriate for use in PRA because they reflect the extent of information that is available to characterize a specific random variable. Preliminary distributions are discussed in Section B.2.0 and Table B-4. Because many of the distributions given in Table B-2 can assume flexible shapes, they offer practical choices for characterizing variability. Table B-2 also illustrates probability distributions (both PDFs and CDFs) commonly used in PRA. While intuitively appealing, identifying a mechanistic basis for a distribution can be difficult for many exposure variables; however, it may be relatively apparent that the variable is bounded by a minimum (e.g., ingestion rate 

### $ 0 mg/day) and a maximum (e.g., absorption fraction 

### # 100%), or that the relevant chance mechanism results in a discrete distribution rather than a continuous distribution, as described above. For each distribution, one or more examples with different parameter estimates are given to demonstrate the flexibility in the shape of the PDF. In addition to the descriptions of the distributions in Tables B-2, Table B-3 provides a summary of the parameters and theoretical bounds that define the PDFs. For a further discussion of characteristics of PDFs see Thompson, 1999. Figures (a-h) immediately following Table B-2 present examples of PDFs and the corresponding CDFs for distributions commonly used in PRA.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-18 Table B-2. Examples of Selected Probability Distributions for PRA

. Distribution Mechanistic Basis Example(s) Beta Figure (e) Describes a continuous random variable with finite upper and lower bounds. This distribution can take on very flexible shapes, but generally does not have a mechanistic basis. Absorption fraction bounded by 0 and 100%; fraction of time an individual spends indoors.

Binomial Describes a discrete random variable produced by processes that: (1) occur in a fixed number n of repeated independent “trials”; (2) yield only one of two possible outcomes (e.g., “success” or “failure”) at each trial; and (3) have constant probability p of “success”. A binomial distribution is characterized by parameters n , p , and x , representing the number of trials, the probability of success of each trial, and the number of successes, respectively. The number of animals with tumors (or some other quantitative outcome) in a chronic animal bioassay.

Exponential Figure (h) If instead of counting the number of events in the Poisson process (below), one measures the time (or distance) between any two successive, random, independent events. The length of time between two radiation counts; length of time between major storm events; distance between impact points of two artillery shells. Gamma Figure (g) Similar to exponential except that time until occurrence of the k th

event in the Poisson process is measured (rather than time between successive events). Reduces to exponential when k=1. Time until k th

radiation count; elapsed time until k

th

major storm event.

Lognormal Figure (b) Multiplication of a large number of random variables, or equivalently adding the logarithms of those numbers, will tend to yield a distribution with a lognormal shape. Chemical concentrations in environmental media; media contact rates; rates and flows in both fate and transport models. Because the basic risk equation is multiplicative, distributions of risk are generally lognormal. In practice, lognormal distributions often provide good fits to data on chemical concentrations in a variety of media (Gilbert, 1987; Ott, 1990). Normal Figure (a) Addition of independent random variables, with no one variable contributing substantially to the total variation of the sum, will tend to yield a distribution with a normal shape. This result is established by the central limit theorem. The “Gaussian Plume Model” for the dispersion of air pollutants is based on the idea that, at a micro level, individual parcels of air, or molecules of pollutants, are subject to many random collisions from other molecules that act together as if a large number of random numbers were being added/subtracted from an initial 3-dimensional description of a position.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Table B-2. Examples of Selected Probability Distributions for PRA

. Distribution Mechanistic Basis Example(s)

Page B-19 Poisson Observed when counting the frequency of discrete events, where the events are independent of one another, and randomly distributed in space or time. Approximates the binomial distribution when sample size, n , is large and probability, p , is small. The number of counts of radiation that occur in a particular time interval; the release of synaptic transmitter from nerve cells; the number of artillery shells falling within a fixed radius; the occurrence of major storm events in a month; number of leaks in average length of pipe. Triangular Figure ©) The PDF is shaped like a triangle, with parameters representing plausible bounds and a most likely value (i.e., mode). This is a “rough” probability model that generally describes a random variable based on limited information rather than mechanistic basis. Variability in shower droplet diameter. Uncertainty in the mean air exchange rate in a shower.

Uniform Figure (d) The PDF is shaped like a rectangle, with parameters representing plausible bounds. This is a “rough” probability model that generally describes a random variable based on limited information rather than a mechanistic basis. Variability in the air ventilation rate in a house.

Weibull Figure (f) Originated in reliability and (product) life testing as a model for time to failure or life length of a component when the failure rate changes with time. A very flexible model taking a wide range of shapes. If the failure rate is constant with time, the Weibull reduces to the exponential distribution. Examples for exponential and gamma would also be appropriate for Weibull.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-20 [200, 100]

(b2)cumulative probability

0 200 400 600 800 0 0.2 0.4 0.6 0.8

1 [100, 100]

[100, 365]



### (d2)cumulative probability

0 100 200 300 400 0 0.2 0.4 0.6 0.8

1u l [0, 30, 100]



### (c2)cumulative probability

0 20 40 60 80 100 0 0.2 0.4 0.6 0.8

1 [200, 100]

(a2)cumulative probability

-300 -100 100 300 500 700 0 0.2 0.4 0.6 0.8

1 [200, 100]

(a1) -300 -100 100 300 500 700 0 0.004probability density Normal

[0, 30, 100] 

### Triangular

(c1)probability density

0 20 40 60 80 100 0 0.01 0.02 [100, 365]

0 100 200 300 400 0 0.005probability density Uniform



### (d1) (b1)probability density

0 200 400 600 800 0 0.01 [100, 100]

[200, 100]

LognormalRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-21 

### (e2)cumulative probability

0 0.5 1 0 0.2 0.4 0.6 0.8

1
[1, 1]

[10, 2] [2, 6]

[100, 365]

(d2)cumulative probability

0 100 200 300 400 0 0.2 0.4 0.6 0.8

1



### (g2)cumulative probability

0 2 4 6 8 10 0 0.2 0.4 0.6 0.8

1 [1, 1]
[2, 1]
[10, 2] (f2)cumulative probability

0246 0 0.2 0.4 0.6 0.8

1

[1, 1]
[2, 1] [10, 2] 

### Weibull

(f1)probability density

0246 0 0.4 0.8 1.2 1.6

2

[1, 1] [2, 1] [10,2]

Gamma

(g1)probability density

0 2 4 6 8 10 0 0.2 0.4 0.6 0.8

1 [1, 1]

[10, 2] [2, 1]



### (h2)cumulative probability

0246 0 0.2 0.4 0.6 0.8

1 [1] [2] 

### Beta

(e1)probability density

0 0.5 1 0 1 2 3 4 5

[1, 1] [2, 6] [10, 2]



### Exponential

(h1)probability density

0246 0 0.2 0.4 0.6 0.8

1

[1] [2]RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-22 

### B. 5.2 E MPIRICAL 

### D ISTRIBUTION 

### F UNCTIONS 

### (EDF S 

### ) In some cases, an empirical distribution function (EDF) may be preferred over fitting the data set to a hypothesized distribution. EDFs, also called empirical cumulative distribution functions (ECDF), provide a way to use the data itself to define the distribution of the relevant variable. Briefly, an EDF for a random variable is described by a step function based on the frequency distribution of observed values. An EDF for a continuous random variable may be linearized by interpolating between levels of the various bins in a frequency distribution. The CDF for a linearized EDF appears as a line, rather than steps. Example B-3 at the end of this Appendix illustrates an EDF, linearized EDF, and beta distribution ( 

### " 1 =0.63, 

### " 2 =2.85, rescaled to min=0, max=364) fit to percentile data for soil ingestion rates in children (Stanek and Calabrese, 1995). A plausible range (i.e., minimum and maximum values) was imposed on the data set for this example. EDFs provide a complete representation of the data with no loss of information. They do not depend on the assumptions associated with estimating parameters for theoretical probability models. EDFs are designed to provide direct information about the shape of the distribution, which reveals skewness, multimodality, and other features of the data set. However, EDFs may not adequately represent the tails of a distribution due to limitations in data acquisition. In the simplest case, an EDF is constrained to the extremes of the data set. This may be an unreasonable restriction if limiting the EDF to the smallest and largest sample values is likely to greatly underestimate the distributional tails. If this is an important source of uncertainty, the risk assessor may choose to extend the tails of the distribution to plausible bounds or to describe the tails with another distribution (see Exhibit B-4). For example, an exponential distribution may be used to extend the tails based on the last 5% of the data. This method is based on extreme value theory, and the observation that extreme values for many continuous, unbounded distributions follow an exponential distribution (Bratley et al., 1987). As with other probability models, uncertainty in the plausible bounds of an EDF may be reduced by obtaining additional information. Advantages and disadvantages of using EDFs in PRA are discussed in detail in the Report of the Workshop on Selecting Input Distributions for Probabilistic Assessments (U.S. EPA, 1999a).
B.5.3 G RAPHICAL M ETHODS FOR S ELECTING P ROBABILITY D ISTRIBUTIONS Graphical methods can provide valuable insights and generally should be used in conjunction with exploratory data analysis. Examples of graphical methods are frequency distributions (i.e., histograms), stem-and-leaf plots, dot plots, line plots for discrete distributions, box-and-whisker plots, and scatter plots (Tukey, 1977; Conover, 1980; Morgan and Henrion, 1990).



### L Graphical methods are invaluable for exploring a data set to understand the characteristics of the underlying population

. E XHIBIT B-4
V ARIATIONS OF THE EDF Linearized - Linearly interpolates between two observations, yielding a linearized cumulative distribution pattern.
Extended - In addition to linearizing (see above), adds lower and upper bounds based on expert judgment. Mixed Exponential - Adds an exponential upper and/or lower tail to the EDF.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-23 

### Numberof Bins n = + 1 3322 10 

### . log Equation B-1



### Percentile i
n = × − 100 05

. Equation B-2



### Percentile i
n = × + 100 1 Equation B-3



### Percentile i a
n a = × − + − 100 1 2 Equation B-4 Together with statistical summaries, graphical data summaries can reveal important characteristics of a data set, including skewness (asymmetry), number of peaks (multi-modality), behavior in the tails, and data outliers. Frequency Distribution or Histogram The frequency distribution, or histogram, is a graphical approximation of the empirical PDF. Frequency distributions can be plotted on both linear and log scales. The general strategy for selecting the number of bins to partition the data is to avoid too much smoothing and too much jaggedness. Equation B-1 (U.S. EPA, 1999a) provides a starting point for estimating the number of bins based on the sample size ( n ).

Probability Plotting Another method that may be used to visualize distributions and estimate parameters is probability plotting, also referred to as linear least squares regression or regression on ordered statistics. This technique involves finding a probability and data scale that plots the CDF of a hypothesized distribution as a straight line. The corresponding linearity of the CDF for the sample data provides a measure of the GoF of the hypothesized distribution. The general approach involves sorting the sample data in ascending order and converting the ranks to percentiles. The percentile value for the i t hrank is calculated according to Gilbert (1987) as:

An alternative formula is provided by Ott (1995):

Plotting positions given by Equations B-2 and B-3 are special cases of the more general formula given by Equation B-4 (Helsel and Hirsch, 1992):

where a is a constant that varies from 0 (Equation B-3) to 0.5 (Equation B-2). The percentiles are used to calculate the z-scores , which represent the number of standard deviations away from the mean that a particular datum lies assuming the data are normally distributed. For normal distributions, the data are plotted against the z-scores; for lognormal distributions, the data are log-transformed and plotted against the z-scores. In both cases, parameters of the distribution can be estimated from the least-squares regression line. When the hypothesized distribution is a poor fit to the data, p-plots can yield misleadingly low estimates of the standard deviation (Cullen and Frey, 1999). Both Gilbert (1987) and Ott (1995) provide excellent descriptions of the use of probability plotting to derive parameter estimates for a given distribution. Probability plotting techniques with best-fit lines have been used to estimate parameters for a wide variety of distributions, including beta, Weibull, and gamma.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-24 

###  μ

a 

### X X 2 1 2 2 

### 2 = +

      1.



###  

### μ

b 

### X X 2 1 2 2 2 

### 2 = + 2.



### μ c 

### X X 2 1 2



### = × 3. Cullen and Frey (1999) point out that probability plotting may not be a primary choice for selecting a fitting distributions because the method violates an important assumption of least squaresregression—independence of the observations (see Appendix A, Exhibit A-5). This is because the rankordered data are no longer independent. Nevertheless, this approach may yield good results when the fit is good and the choice of distributions is somewhat subjective.
B.5.4 P ARAMETER E STIMATION M ETHODS As a rule, there are often a number of different methods available for estimating a given parameter. The most appropriate method to apply may require judgment, depending on the relative difficulty in applying a method for a particular parameter, as well as the desired statistical properties of the method. The following simple example provides a useful analogy. Suppose that the parameter of interest, A, is the total area of an approximately square exposure unit. If the exposure unit is a perfect square, and the length of one side (L 1 ) is known, the area would be equal to L 1 2 (i.e., for a square, A=L i 2 ). Suppose L is unknown, but two independent measurements, X 1 and X 2 , are available to estimate the length (see Exhibit B-5). If it is assumed that the random variable, L, has a probability distribution with mean 

### : , then the area of the square piece of property is A= 

### : 2 . What is a reasonable estimate of the area (i.e., ) based on X 1 and X 2 ? Three plausible methods for calculating are given below. 

### > > A = μ 2 

### > μ 2

Because these three estimators will, as a rule, give different answers, it may be useful to set criteria for selecting which one gives the “best” answer. Some of the statistical criteria that are used for this purpose are consistency , efficiency, robustness, sufficiency , and unbiasedness (see Exhibit B-6). It turns out, each method is relatively easy to implement, but the third method is preferred because it is a more efficient estimator. In many cases, particularly if a model is complex, potential estimators of the unknown parameters are not readily apparent. To assist in developing estimators, several general methods have been developed. Exhibit B-7 lists some of the more common parameter estimation methods. Perhaps the simplest method is the method of matching moments (MoMM), also called the method of moments. MoMM is appropriately named, as it involves expressing the unknown parameters in terms of population moments and then “matching”, or equating the sample moments to the population E XHIBIT B-5
E STIMATING THE A REA OF A H YPOTHETICAL E XPOSURE U NIT


### Exposure Unit

x 1 

### x 2 

### Exposure Unit

x 1 

### x 2RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-25 n moments. For example, the sample mean ( ) 

### x and standard deviation ( s ) are estimators for the corresponding population parameters ( 

### : and 

### F ). Maximum Likelihood Estimation (MLE) is a commonly applied method, that is often thought of as a parameter estimate for which the observed data are most “likely”. The likelihood function is defined for independent continuous random variables as follows:


## L( θ 1 

## , θ 2 

## ,... θ k 

## ) = 

## Π f(x 1 

## | θ 1 

## , θ 2 

## , ..., θ k 

## ) I=1
The likelihood function is evaluated based on the product of the PDF for each value of x. The parameters of the probability model, ( θ k ), are chosen to maximize the likelihood function value and thereby are most likely to produce the sample data set (Cullen and Frey, 1999).
It has also been demonstrated that MLE yields estimators that generally have good properties when evaluated by the criteria listed above. In some cases (e.g., for smaller sample sizes), these estimators are not unbiased ; however, this can often be accounted for by “adjusting” the estimator. A familiar example of this adjustment is in estimation of the variance of a normal distribution. The MLE for the variance is biased by a factor of (( n -1)/ n ), but this is easily corrected by multiplying the MLE by ( n /(–1)). For some distributions, calculations of the MLE are straightforward. For example, MLE for parameters of a normal distribution are given by the mean and standard deviation of the sample data, the same as MoMM. MLE for parameters of a lognormal distribution are given by the mean and standard deviation of the log-transformed data, which is different from MoMM. In general, MLE calculations are complex, and commercial software such as @Risk and Crystal Ball ® may be used. A more detailed discussion of the derivation and properties of MoMM and MLE can be found in the statistics literature (e.g., Chapter 5 of Mood and Graybill, 1963; Chapter 9 of Mendenhall and Scheaffer, 1973; Section 6.5 of Law and Kelton, 1991; Section 5.6 of Cullen and Frey, 1999). E XHIBIT B-7
P ARAMETER E STIMATION M ETHODS • Method of Matching Moments • Maximum Likelihood • Minimum Chi-Square • Weighted Least-Squares E XHIBIT B-6 C RITERIA FOR E VALUATING P ARAMETER E STIMATION M ETHODS * Consistency A consistent estimator converges to the “true” value of the parameter as the number of samples increases. Efficiency An efficient estimator has minimal variance in the sampling distribution of the estimate.
Robustness A robust estimator is one that works well even if there are departures from the assumed underlying distribution. Sufficiency A sufficient estimator is one that makes maximum use of information contained in a data set. Unbiasedness An unbiased estimator yields an average value of the parameter estimate that is equal to that of the population value. *Source: Cullen and Frey, 1999RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-26 

### B.5.5 D EALING WITH 

### C ORRELATIONS AMONG 

### V ARIABLES OR 

### P ARAMETERS Correlations between exposure variables or between parameters of the probability distribution may be important components of a probabilistic model. Correlation is a measure of association between two quantitative random variables. Two random variables may either be positively or negatively correlated. A positive correlation exists between two variables if the value of X 1 increases as the value of X 2 increases. For example, higher hand dust lead levels have been associated with higher pediatric blood lead levels (Charney et al., 1980). A negative correlation exists between two variables if the value of X 1 increases as the value of X 2 decreases. For example, studies suggest the ingestion of soil and dust particles increases as particle size decreases (Calabrese et al., 1996). A first step in identifying correlations is to assess the possible physical and statistical relationships that exist between variables. In an ecological risk assessment (ERA), for example, the largest surf scoter (diving duck) does not consume the least amount of food, nor does the smallest surf scoter consume the greatest amount of food. Random sampling of body weight and ingestion rate as separate parameters, however, allows for these two possibilities. Neglecting a correlation between two variables may restrict (underestimate) the tails of the ecological Hazard Quotient (HQ) for each chemical of concern (COC), which are frequently the areas of the distribution of most interest. The degree to which correlations affect the output of a risk model depends on: (1) the strength of correlations between the two variables, and (2) the contribution of the correlated variables to overall variance in the output (Cullen and Frey, 1999). Therefore, it is useful to conduct a preliminary sensitivity analysis to assess the impact of alternative correlation assumptions on the model output. If the impact is significant, correlations should be identified and accounted for in the PRA. There are several approaches to account for dependencies in MCA including: (1) modifying the model to include the correlation; and (2) simulating dependence between variables for sample generation (Cullen and Frey, 1999). Modifying the model is preferred as simulation techniques cannot capture the full complexity between model inputs. However, when this is not possible, dependencies between variables can be simulated and approximated by correlation coefficients and bivariate normal distributions. Correlation coefficients are a numerical measure of the strength and direction of the relationship between two variables. Sample correlation coefficients measure the linear relationship between variables. However, if two variables are from different probability distributions, it is unlikely that they are linearly related. Consequently, simulation software programs such as Crystal Ball ® and @Risk can be used to calculate and employ the nonparametric statistic, Spearman’s Rank Correlation Coefficients (Rho) in simulating correlation between inputs. Rank Correlation Coefficients measure the linear dependence not of the data values themselves, but of the rank value of the data. The ranks indicate relative positions in an ordered series, not the quantitative differences between the positions. The disadvantage of losing information by using the rank values (rather than the actual values) is offset by the ability to correlate random variables from different distribution types (See Appendix A). Exhibit B-8 gives an example of a straightforward approach to specifying a rank correlation between two input variables in a one-dimensional Monte Carlo analysis (1-D MCA) for variability. A range of correlations is explored as a form of uncertainty analysis on the distribution of intakes given a fish advisory of 7.0 

### : g/day for a chemical.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-27 E XHIBIT B-8 C ORRELATION OF I NPUT V ARIABLES FOR 1-D MCA OF V ARIABILITY Intake Equation Intake = (CF x IR x FI x EF x ED)/(BW x AT)

Variables Description and Units Units Point Estimate or PDFv CF concentration in fish ug/kg 25 IR fish ingestion rate kg/meal lognormal (0.16, 0.07) 1 FI fraction ingestion from source unitless 1.0 EF exposure frequency meals/yr lognormal (35.5, 25.0) 1 ED exposure duration years 30 BW body weight kg 70 AT averaging time days 10950

1 Lognormal PDF parameters: arithmetic mean, standard deviation



### < Correlation between IR and EF is suggested by Burger et al. ( 1999) study of 250 anglers on the Savannah River, South Carolina. Moderate correlation (Kendall’s tau=0.17, p=0.04)

< Uncertainty Analysis: 1-D MCA simulations of variability correlating IR and EF using Crystal Ball ® 2000 (5,000 iterations, Latin Hypercube sampling). Spearman rank correlations: 0.10, 0.50, 0.90
Statistics of PDFv for Intake (ug/day) compared to Fish Advisory of 7.0 ug/day
Rank Correlation (r) 0.10 0.50 0.90 Intake Statistics (ug/day) mean 1.6 1.8 2.0 50 th percentile 1.1 1.1 1.1 95 th percentile 4.4 5.4 6.5 97.5 th percentile 5.7 7.0 9.0

< For this example, only IR and EF are characterized by PDFs. They contribute approximately equally to the distribution of intakes. Positive rank correlations have little effect on the median (50 th percentile) of the output distribution, but tend to widen the tails of the distribution. Increasing the correlation from 0.10 to 0.90 increases the 90 th percentile from 4.4 to 6.5 ug/day, and the 97.5 th percentile from 5.7 to 9.0 ug/day.

< If the fish advisory is 7.0 ug/day, uncertainty in the correlation coefficient may have important consequences for the risk management decision.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-28 Correlations may also be specified for parameters of a probability distribution. This is an important concept when designing a two-dimensional Monte Carlo analysis (2-D MCA) in which parameters of the same PDFv might be otherwise be described by independent PDFu’s. A common approach for correlating two parameters is to specify a bivariate normal distribution (Nelsen, 1986, 1987; Brainard and Burmaster, 1992). A bivariate normal distribution allows for the distribution of one variable to be sampled conditional on the other. This is a special case of a joint distribution in which both x and y are random variables and normally distributed (as the conditional distribution of x or of y is always normal) (Wonnacott and Wonnacott, 1981). Example B-4 further explains bivariate normal distributions and demonstrates this approach applied to coefficients of a simple linear regression model that relates contaminant concentrations in soil and dust. The results of correlation analysis should be interpreted with caution. Two variables may be associated due to: (1) a dependency between the two variables; (2) chance (two independent variables appear dependent due to chance in the sampling procedure); and (3) variables not included in the analysis (lurking variables) are affecting the two variables being analyzed. Likewise, a low correlation measure does not necessarily mean the two variables are independent. As a lurking variable may cause the appearance of an association between the two independent variables, it may also mask the association between two dependent variables.



### L Correlation describes a degree of mathematical association, not a causal relationship between the two variables. Efforts to extrapolate or predict correlations outside the range of observed values should also be done with caution. For example, there may be a strong linear relationship between age and height in children; however, it would be inappropriate to apply this correlation to adults. Additional caution is needed when correlating more than two factors at a time. In general, because of the complexity of specifying a valid covariance matrix when correlating more than two factors at a time, risk assessors may need to consult a statistician to avoid generating misleading risk estimates. 

### B.5.6 C ENSORED 

### D ATA In order to define the exposure point concentration, estimates of summary statistics representative of the entire distribution of data are needed (Helsel and Hirsch, 1992). Censored data complicate the process of selecting and fitting PDFs and estimating parameter estimates. A censored data set is a data set for which measurements above or below a certain threshold are not available. Left censored data occurs frequently at Superfund sites, where samples for a number of chemicals are often below the reporting limit. A censored datum (often denoted by ND) commonly represents a value of half of the laboratory reporting limit. Three general methods for estimating summary statistics for left censored data sets include: (1) simple substitution; (2) distributional methods; and (3) robust methods (Helsel and Hirsch, 1992). These methods may be evaluated based on the root mean squared error (RMSE) estimate, a measure of the difference between the sample statistic (e.g., the sample mean, ) and the true population parameter x (e.g., population mean, μ ).RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-29 

## N x RMSE N i 

## μ μ 2 

## ) ( 1 

## − ∑

= =

Methods which yield estimates closer to the true parameter value have lower bias, higher precision, and lower RMSEs. Simple Substitution Methods Simple substitution methods entail substituting values equal to or lower than the reporting limit in the data set. These surrogate values are then included in the calculation of the summary statistics and in determining the distributional shape of the data set. Although this method is frequently used, it is important to understand its limitations; depending on the surrogate value used (e.g., half the reporting limit) the simple substitution method may yield biased parameter estimates (e.g., low estimates of the mean) and may yield misleading distributional shapes. Studies such as those reported by Gilliom and Helsel (1986) have determined, in terms of the RMSE, that simple substitution methods perform more poorly than the distributional and robust methods described below. Distributional Methods With distributional methods, the entire data set is assumed to follow a theoretical distribution (e.g., normal distribution). Assuming a theoretical distribution, MLE and probability plotting (p-plot) methods provide summary statistics that best match the reported values of the data and the percentage of samples below the threshold value. If the data fit the theoretical distribution exactly, or if the sample size is large, both MLE and p-plots are unbiased methods. Often, however, the sample size is small and the distribution deviates from a theoretical distribution. In this case, the MLE and p-plot methods may yield biased and imprecise methods (Hesel and Hirsch, 1992). Robust Methods With robust methods, a theoretical distribution is needed. A theoretical distribution is fit to the data above the detection limit by MLE or p-plot methods. Based on this assumed PDF, the value of the data points below the detection limit are extrapolated and used in the summary statistics calculation. Unlike the simple substitution method, these extrapolated values are not estimates for the data points; rather, they are only used jointly to calculate summary statistics (Hesel and Hirsch, 1992). The method is considered robust as it uses the actual values of the sample data, rather than the distribution above the detection limit.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-30 B.5.7 T RUNCATION Truncation refers to imposing a minimum and/or maximum value on a probability distribution. The main purpose of truncation is to constrain the sample space to a set of “plausible values”. For example, a probability distribution for adult body weight might be truncated at a minimum value of 30 kg and a maximum value of 180 kg in order to avoid the occasional selection of an unlikely value (e.g., 5 or 500 kg). Given the subjectiveness involved in selecting truncation limits, such choices should clearly be made with caution, and involvement of stakeholders who may be aware of site-specific circumstances. For example, there may well be individuals who weigh more than 180 kg and less than 30 kg. The purpose for truncating the tails of a distribution is to confine each risk estimate of a Monte Carlo simulation to a combination of plausible input values. The advantage of truncating unbounded probability distributions in PRA is that central tendency and high-end risk estimates will not be biased by unrealistic values. The disadvantage is that the original parameter estimates of the nontruncated distribution are altered by constraining the sample space. The bias in the parameter estimates increases as the interval between the minimum and maximum truncation limit is reduced. For example, a normal distribution with an arithmetic mean of 100 may be fit to a data set; imposing a truncation limit of 300 may result in a truncated normal distribution with an arithmetic mean of 85. The relationship between the truncated and nontruncated parameter estimates can be determined analytically (Johnson et al., 1995) or approximated using Monte Carlo simulations under both truncated and nontruncated scenarios.
Table B-3. Theoretical bounds and parameter values for selected distributions.

Probability Distribution Parameters 1 Theoretical Bounds Normal ( : , F ) (- 4 , + 4 ) Lognormal ( : , F ) [0, + 4 ) Weibull ( " , $ ) [0, + 4 ) Exponential ( $ ) [0, + 4 ) Gamma ( " , $ ) [0, + 4 ) Beta ( " 1 , " 2 , a, b) [a, b] Uniform (a, b) [a, b] Triangular (a, m, b) [a, b] Empirical ( bounded EDF) (a, b, {x}, {p}) [a, b]

1 a=minimum, b=maximum, : =mean, F =standard deviation, m=mode, " =shape parameter, $ =scale parameter, x=value, p=probability Truncation is typically considered when using unbounded probability distributions (e.g., normal, lognormal, gamma, Weibull) to characterize variability. Table B-3 gives the theoretical bounds for selected probability distributions that may be more commonly used in PRA. Truncating the minimum value may also be appropriate for distributions whose minimum is defined as zero (e.g., lognormal, gamma, Weibull). Truncation is generally less important when a PDF is used to characterize uncertainty in a parameter estimate (e.g., arithmetic mean), since distributions for uncertainty are often bounded byRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-31 definition (e.g., triangular, uniform). Bounded continuous distributions, such as the beta distribution or empirical distribution (see Section B.5.2) are not subject to the parameter bias of truncation, although plausible minimum and maximum values must still be identified. Identifying appropriate truncation limits that reflect “plausible bounds” for an exposure variable will often require judgment. Given that most data sets represent statistical samples of the target population, it is unlikely that the minimum and maximum observed values represent the true minimum and maximum values for the population. However, there may be physiological or physical factors that can aid in setting plausible truncation limits. For example, the maximum bioavailability of chemicals in the gastrointestinal (GI) tract is 100%. Similarly, the solubility of chemicals in aquatic environments (accounting for effects of temperature) will generally be less than the chemical solubility in water free of particulates. In general, sensitivity analysis can be used to determine if truncation limits are an important source of parameter uncertainty in risk estimates. For exposure variables in the numerator of the risk equation, the maximum truncation limit is of greatest concern. For exposure variables in the denominator of the risk equation, the minimum truncation limit is of greatest concern. Details regarding the fit of the tails of the probability distribution and the effect of truncation on the parameter estimates should generally be included in the workplan. 

### B.6.0 A SSESSING 

### Q UALITY OF THE 

### F IT The quality of the fit of a distribution may be evaluated in several ways. Standard statistical approaches are available to test the fit of a theoretical distribution to a data set (i.e., GoF tests). In addition, alternative choices for distribution shapes and plausible bounds might be explored as a form of sensitivity analysis. Together with graphical exploration (Section B.5.3), this information may be useful when deciding whether or not to incorporate a specific type of distribution for an exposure variable into a PRA.



### L GoF tests are one tool among several to assess the quality of a distribution. Although GoF testing is a necessary part of distribution fitting, and tests are readily available with commercial software, it is less important than mechanistic considerations or graphical data exploration for choosing a candidate distribution. Examples of GoF tests are discussed below, and cautions regarding GoF are outlined in Section B.6.3.
B.6.1 W HAT IS A G OODNESS - OF -F IT T EST ? Goodness-of-fit (GoF) tests are formal statistical tests of the hypothesis that the data represent an independent sample from an assumed distribution. These tests involve a comparison between the actual data and the theoretical distribution under consideration. In statistical hypothesis testing the null hypothesis ( H 0 ) is assumed to be true unless it can be proven otherwise. The “evidence” upon which we base a decision to reject or not to reject H 0is a random sample. Typically, we seek to reject H 0in favor of H a . For example, with the two sample t -test, the null hypothesis is that the means of two populations are equal (not different) and the alternative is that they are different. This is expressed as:RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-32 

### H H a 01 2
1 2 

### :
: μ μ μ μ = ≠



### HF F HF F

a 0 0
0 

### :
: = ≠ Most often, the hypothesis test is used to show that the means are not equal (i.e., reject H 0 in favor of H a ) in order to state that there is a significant difference between the two populations at a specified significance level (e.g., 

### " =0.05). Thus, the hypothesis test is often referred to as a significance test. The p-value in a statistical test is calculated from a sample and represents the probability of obtaining a value of the test statistic as extreme or more extreme as the one observed if H 0is in fact true. When the p-value is small it means either the null hypothesis is not true, or that we have witnessed an unusual or rare event (by chance we drew an unusual sample that resulted in the extreme value of the test statistic). Often a value of 0.05 or 0.01 is designated as a cutoff, or significance level 

### " . If the p-value is (e.g., p < 0.05), the null hypothesis is rejected in favor of the alternative, and we state that the test result is statistically significant at level 

### " . This does not mean that we have proven H a is true. Rather, we are saying that based on our sample results, it is unlikely that H 0is true. In a GoF test, the hypothesis test is set up the same way as a “traditional” hypothesis test, but the outcome is viewed a little differently. In GoF tests, we generally seek to failto reject H 0because the null hypothesis states that the data were obtained from a population described by the specified distribution ( F 0 ). The alternative hypothesis is that the data were obtained from a population described by a different distribution. In most applications of GoF techniques, the alternative hypothesis is composite—it gives little or no information on the distribution of the data, and simply states that H 0is false (d’Agostino and Stephens, 1986). This can be expressed as:

where F 0is a specific continuous distribution function, such as the CDF for a normal distribution.



### L

GoF tests do not prove that the population is described by the specified

distribution, but rather that this assumption could not be rejected. In general, p-values provide one metric of evaluating the fit of the distribution. For example, a p-value of 0.06 indicates that the null hypothesis (i.e., the assumption of a specified distribution) cannot be rejected at 

### " =0.05. Larger p-values indicate a better fit and stronger evidence that the distribution specified by the null hypothesis may be appropriate. This guidance does not recommend an arbitrary cutoff for the p-value . A risk assessor performing a GoF test generally should report the p-value and whether the fit is considered “good” or “poor”.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-33 B.6.2 W HAT A RE S OME C OMMON G OODNESS - OF -F IT T ECHNIQUES ? The following GoF tests can also be found in most general statistical and spreadsheet software. Both Crystal Ball ® and @Risk software present the results of chi-square, K-S, and Anderson-Darling tests in their fitting routines. Shapiro-Wilk Test The most widely used GoF test in risk assessment is the Shapiro-Wilk test for normality (Gilbert, 1987). This simple hypothesis test can determine whether or not a small data set ( n 

### # 50) is normally distributed. The test can also be run on log-transformed data to assess whether the data are lognormally distributed. D'Agostino's test may be used for samples sizes larger than those accommodated by the Shapiro-Wilk test (i.e., n > 50) (d’Agostino and Stephens, 1986). In addition, Royston (1982) developed an extension of the Shapiro-Wilk test for n as large as 2000 (Gilbert, 1987). Probability Plot Correlation Coefficient Test
The correlation coefficient r (or the coefficient of determination, r 2 ) between the data and the z-scores of a normal probability plot (Filliben, 1975; Helsel and Hirsch, 1992) is similar to the W statistic of the Shapiro-Wilk test. A detailed comparison of the Shapiro-Wilk test and the product correlation coefficient test is given by Filliben (1975) and d’Agostino and Stephens (1986). Helsel and Hirsch (1992) summarize critical r* values derived by Looney and Gulledge (1985) for the probability plot correlation coefficient test. Chi-Square Test The chi-square test is a general test that may be used to test any distribution (continuous or discrete), and for data that are ordinal (e.g., categories such as high/medium/low). Chi-square is a measure of the normalized difference between the square of the observed and expected frequencies. For example, by constructing a frequency distribution of the data with k adjacent bins, j =1... k , the number of data points in the j th bin can be compared with the expected number of data points according to the hypothesized distribution. Note that in the case of continuous, unbounded distributions (e.g., normal), the first and last intervals may include [- 

### 4 , a 1 ] or [a k , + 

### 4 ] (Law and Kelton, 1991). The chi-square test is very sensitive to the chosen number and interval width of bins—different conclusions can be reached depending on how the intervals are specified. Strategies for selecting bins (e.g., setting interval widths such that there are no fewer than 5 data points expected per bin) are given in the statistical literature (d’Agostino and Stephens, 1986; Law and Kelton, 1991). The test statistic is compared with a value of the chi-square distribution with ( k - r - 1) degrees of freedom, where k is the number of sample values and r is the number of parameters of the hypothesized distribution. As described in Section B.6.1, in general, higher p-values suggest better fits. Kolmogorov-Smirnov (K-S) Test The K-S test is a nonparametric test that compares the maximum absolute difference between the step-wise empirical CDF and the theoretical CDF. Because the maximum discrepancy is compared with the test statistic, K-S is sometimes referred to as a supremum test (Cullen and Frey, 1999). In general, lower values of the test statistic indicate a closer fit. The K-S test is most sensitive around the median of a distribution, and, hence, it is of little use for regulatory purposes when the tails of distributions are mostRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-34 generally of concern (U. S. EPA, 1999a). Although it does not require grouping data into bins like the chi-square test, critical values for the K-S test depend on whether or not the parameters of the hypothesized distribution are estimated from the data set (Gilbert, 1987; Law and Kelton, 1991). The Lilliefors test was developed to surmount this problem when the hypothesized distribution is normal or lognormal (Gilbert, 1987). Anderson Darling Test The Anderson-Darling test assesses GoF in the tails (rather than the mid-ranges) of a PDF using a weighted average of the squared differences between the observed cumulative densities. The Anderson-Darling test is sometimes referred to as the quadratic test (Cullen and Frey, 1999). The test statistic should be modified based on sample size prior to comparison with the critical value. Like the K-S test, in general, lower values of the test statistic indicate a closer fit (i.e., if the adjusted test statistic is greater than the modified critical value for a specified 

### " , the hypothesized distribution is rejected). The Anderson-Darling test may be particularly useful because it places more emphasis on fitting the tails of the distribution. B.6.3 C AUTIONS R EGARDING G OODNESS - OF -F IT T ESTS There are many statistical software programs that will run GoF tests against a long list of candidate distributions. It is tempting to use the computer to make the choice of distribution based on a test statistic. However, GoF tests have low statistical power and often provide acceptable fits to multiple distributions. Thus, GoF tests are better used for rejecting poorly fitting distributions than for ranking good fits. In addition, for many distributions, GoF statistics lack critical values when the parameters are unknown (i.e., estimated from the data). In practice, this limitation is often discounted and the critical values are interpreted as a semi-quantitative measure of the fit. It is most appropriate to form an idea of the candidate distributions based on some well reasoned assumptions about the nature of the process that led to the distribution, and then to apply a GoF test to ascertain the fit (U.S. EPA, 1999a). Whenever possible, mechanistic and process (i.e., phenomenologic) considerations should inform the risk assessor's choice of a particular distribution rather than the results of a comparison of GoF tests (Ott, 1995). In addition, the value of graphical evaluations of the fit cannot be overstated.
B.6.4 A CCURACY OF THE T AILS OF THE D ISTRIBUTION The tails of a distribution (e.g., < 5 thand > 95 thpercentiles) for an input variable are often of greatest interest when characterizing variability in risk. Distributions fit to data may not characterize the tails of the distribution in a way that represents the target population. In general, the importance of uncertainty in the fit of the tails of particular distributions should be determined on a site-specific basis. For exposure variables in the numerator of the risk equation, the upper tail is of greatest concern. For exposure variables in the denominator of the risk equation, the lower tail is of greatest concern. The tails of the input PDFs generally have a significant influence on the tails of the risk distribution, especially for those variables that are ranked highest in a sensitivity analysis. Different distributions may share the same mean and variance, but assume very different shapes. Experiments with Monte Carlo simulations have demonstrated that the shape of the input PDFs may have a minimal effect on the risk estimates in the tails of the probability distribution when the mean and variance of the input PDFs are held constant (Hoffman and Hammonds, 1992; Finley and Paustenbach, 1994). Nevertheless, it is generally a good practice in PRA to demonstrate that alternative choices of PDFs do not have a significant effect on percentiles in the RME risk range.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-35 A common question when developing and evaluating Monte Carlo models is, “How many iterations is enough?”. Since Monte Carlo sampling is approximately random, no two simulations will yield the same results (unless the same starting point, or seed, of the random number generator is used). A rule of thumb is that the stability of the output distribution improves with increasing numbers of iterations, although there will always remain some stochastic variability. The stability is generally better at the central tendency region of the output distribution than at the tails; therefore, more iterations may be needed when the risk management decision is associated with the higher percentiles (e.g., > 95 thpercentile). Risk assessors are encouraged to run multiple simulations (with the same inputs) using different numbers of iterations in order to evaluate the stability of the risk estimate of concern. The results of such an exercise should generally be reported to the Agency when submitting a PRA for review. Note that while the speed of modern computers has essentially eliminated the issue for 1-D MCA (e.g., 10,000 iterations of most 1-D MCA models can be run in less than 1 minute), it may still be an important issue for more complex modeling approaches such as Microexposure Event analysis (MEE) and 2-D MCA (see Appendix D). 

### B.7.0 S ELECTING 

### P ROBABILITY 

### D ISTRIBUTIONS 

### B ASED ON 

### S TATE OF 

### K NOWLEDGE Table B-4 summarizes preliminary strategies for proceeding with a PRA based on the amount of available information. Recommended starting points for each of the three steps in the general process are provided. This table provides guidance on candidate distributions that are consistent with the available information, however, it is not intended to discourage the use or exploration of alternative choices.



### L Table B-4 provides recommended preliminary strategies, not steadfast rules. As an analyst works through the PRA, alternative distributions, estimation methods, consideration of mechanism, and GoF tests may better guide the selection process. Case 1 represents the best scenario, in which the analyst has access to the raw data and a sufficiently large sample size (or > 6 percentiles). In this case, the analyst has a variety of choices for distribution fitting and estimating parameters. However, frequently raw data are inaccessible to the analyst. Cases 2 and 3 have limited information available (i.e., mean and upper percentile) and, therefore, have a narrower set of starting points. Case 4 is the most extreme scenario of data availability requiring expert judgment on selecting and fitting distributions.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-36 Table B-4. Strategies for conducting PRA based on available information. Preferred methods in Case 1 (most information) are identified by an asterisk (*). Evaluation Step Case 1 Case 2 Case 3 Case 4 Decreasing Information Data Availability raw data of sufficiently large sample size or six or more percentiles three to five statistics two statistics one statistic

Selection of Distribution Type Nonnegative Continuous any in this category Bounded

beta, Johnson’s SB Nonnegative Continuous lognormal, gamma, Weibull Bounded

beta, Johnson’s SB case-by-case basis using expert judgment

Selection of Parameter Estimation / Fitting Method maximum likelihood* regression methods matching moments minimize average absolute percent error (MAAPE) for available statistics exact agreement between 2-parameter PDF and available statistics Assessment of Quality of Fit Graphical Assessment P-log Q plot*, P-Q plot*

residual % error plot* P-P plot, Q-Q plot GoF Tests

Anderson-Darling* K-S Chi-square Graphical Assessment P-log Q plot, P-Q plot GoF Test

Chi-square, Estimate p -value for MAAPE using parametric bootstrap (if sample size is known) Graphical Assessment judgment based on comparative analysis of PDFs and CDFs

Estimation of Parameter Uncertainty Large Sample asymptotic normality assumption Medium Sample

nonparametric bootstrap Small Sample

parametric bootstrap Parametric bootstrap generate random samples using the fitted distribution (if sample size is known)RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-37 

### E XAMPLES OF 

### F ITTING 

### D ISTRIBUTIONS USING 

### G RAPHICAL 

### M ETHODS 

### , G OODNESS 

### - OF 

### -F IT 

### , AND 

### P ARAMETER 

### E STIMATION Example B-1. Empirical Distribution Function (EDF) for Soil Ingestion Rates This hypothetical example illustrates how graphical methods can be used to select probability distributions for variability based on percentile data reported in the literature. Table B-5 gives the summary statistics that are reported by Stanek and Calabrese (1995) for average daily soil ingestion rates among young children. Three options are explored for selecting a distribution: (1) empirical distribution function (EDF) represented by a step function; (2) linearized and extended EDF; and (3) continuous parametric distributions (beta and lognormal). In order to specify an EDF, a plausible range (minimum and maximum) must be inferred using judgment. Exposure factors such as ingestion rate are nonnegative variables (i.e., minimum 

### $ 0); given the relatively low value for the 25 thpercentile (10 mg/day), it is assumed that 0 mg/day is a reasonable minimum value for this example. If children with pica for soil are excluded from the population of concern, the maximum value may be inferred from the relatively shallow slope at the high-end of the distribution. That is, the 90 thpercentile is reported as 186 mg/day while the 99 th percentile is 225 mg/day, an increase of only 39 mg/day; it is assumed that 300 mg/day is a plausible maximum value for this example. Commercial software such as Crystal Ball ® and @Risk can be used to input EDFs. Figure B-3 illustrates the basic step-wise EDF represented by the reported percentile values, as well as the “linearized, extended EDF” (i.e., linear interpolation between reported values and extended lower and upper tails). An alternative to relying on a linear interpolation between the percentile values is to fit a continuous probability distribution to the reported percentiles. Since the original data are unavailable, standard GoF tests for the EDF, such as K-S and Anderson-Darling (d’Agostino and Stephens, 1986), cannot be applied. Note that computer software (e.g., Crystal Ball ® , @Risk ) will provide test statistics and corresponding p-values , however, these results will (inappropriately) reflect the number of percentile values reported rather than the sample size of the original data. Nevertheless, graphical methods may be employed to assess the adequacy of the fit of various PDFs. In this example, a beta distribution and lognormal distribution were fit to the EDF using Crystal Ball ® . Figure B-4 illustrates the selected statistics for both distributions. The beta distribution appears to more closely match the reported percentile values, especially at the upper tail of the distribution. The lognormal distribution has an unbounded maximum that, for this example, results in an extreme overestimate of the 95 thand 99 thpercentiles. The beta distribution, by definition, is bounded at 0 and 1, and rescaled in this example to a maximum of 364 mg/day. This analysis would support the use of a beta distribution in a Monte Carlo simulation.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-38 

### 0.00 0.25 0.50 0.75 1.00

0 50 100 150 200 250 300 Ingestion R ate (mg/day)cumulative probability



### Step-wise ED F Linearized ED F

Figure B-3. Comparison of step-wise EDF and linearized EDF for ingestion rate. The upper and lower tails of both distributions are extended to a plausible range of [0, 300] mg/day. Table B-5. Selected statistics for reported and fitted distributions for ingestion rate (mg/day). Summary Statistic Reported Values Linearized, Extended EDF Beta Distribution 1 Lognormal Distribution 2 minimum -- 0 0 0 25 thpercentile 10 10 13 11 50 thpercentile 45 45 44 31 75 thpercentile 88 88 100 86 90 thpercentile 186 186 165 216 95 thpercentile 208 208 205 375 99 thpercentile 225 225 322 3346 maximum -- 300 364 + 

### 4

1 Parameters of best-fit beta distribution: " 1 =0.63, " 2 =2.85, min=0, max=364. 2 Parameters of best-fit lognormal distribution: : =97.6, F =291.8.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-39 

### 0.00 0.25 0.50 0.75 1.00

0 50 100 150 200 250 300 Ingestion Rate (mg/day)cumulative probability



### Beta(0.63, 2.85, 0, 364) Lognormal(97.6, 291.8)

Figure B-4. Graphical assessment of beta and lognormal distributions fit to the cumulative distribution reported in the literature (circles). The beta distribution provides a closer fit to the percentile values in this example. .RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-40 Example B-2. Variability in Lead Concentrations in Quail Breast Tissue This hypothetical example demonstrates how the combination of graphical methods, GoF tests, and parameter estimation techniques provides strong evidence for selecting and fitting a lognormaldistribution. Assume lead concentration in quail is an important variable for a food web model. Sitespecific data ( n =62) are used to estimate inter-individual variability in concentration (Table B-6). The histograms in Figure B-5 show lead concentrations in quail breast tissue collected near a settling pond at a plating works. Equation B-1 indicated that 7 bins is an appropriate starting point. The result (top left panel, Figure B-5) suggests that approximately 80% of the values are < 200 ppm and that the probability distribution for variability may be described by a nonnegative, right-skewed distribution (e.g., exponential, Weibull, lognormal, etc.). However, additional bins are needed to better understand the low-end of the distribution. After increasing the number of bins from 7 to 16 (top right panel, Figure B-5), graphical evaluation continues to suggest that the distribution is unimodal right skewed. The bottom panel of Figure B-5 illustrates that increasing the number of bins would not provide better resolution of the low-end of the distribution. For these data, 16 bins appear to provide a reasonable balance between too much smoothing and too much jaggedness. Probability plots can be used to visually inspect the GoF of a specified distribution to the data, and, because the hypothesized distribution yields a straight line, the plots are particularly useful for evaluating deviations at the tails. In addition, parameter estimates can be obtained from the regression lines fit to the data, as discussed below. For this example, two lognormal probability plots are explored to evaluate how well the data can be described by a lognormal distribution (Figure B-6). The top panel gives the z-score on the abscissa (the “x” axis) and ln[concentration] on the ordinate (the “y” axis), while the bottom panel gives ln[concentration] on the abscissa and z-score on the ordinate. Plotting positions for both methods were calculated using Equation B-2. Equally plausible parameter estimates can be obtained from regression lines using either plotting method; however, the approach shown in the top panel may be easier to implement and interpret. Despite the relatively large sample size of n =62, GoF tests generally fail to reject lognormality (i.e., normality of the log-transformed data) in this example. For the probability plot correlation coefficient test (Filliben, 1975; Looney and Gulledge, 1985), if r < r* (the value for r at a specified 

### " ), normality is rejected. For this example, r is 0.988, and r* is between 0.988 and 0.989 for n =62 and 

### " =0.25; therefore, the p-value for the concentrations is approximately 0.25 and one fails to reject lognormality at 

### " # 0.25. D’Agostino’s test yields essentially the same conclusion, with a calculated Y

value of -1.9166. For this data set, with n =62 and 

### " =0.10, one rejects normality if Y < -2.17 or Y

> 0.997 (see Table 9.7 in d’Agostino and Stephens, 1986); therefore, since Y is within this interval, one fails to reject the normal distribution. However, for 

### " =0.20, the rejection criteria is [ Y < -1.64 or Y > 0.812], Y falls outside the low-end of the interval, resulting in a rejection of the normal distribution. For this data set, the p-value associated with d’Agostino’s test is slightly less than 0.20 and one fails to reject normality at 

### " < 0.20.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-41 0 2 4 6 8

0 2 0 4 0 6 0 8 0 1 0 0 1 2 0 1 4 0 1 6 0 1 8 0 2 0 0 2

2 0 2 4 0 Lead in Muscle (ppm) 0.00 0.03 0.06 0.10 0.13FrequencyProbability 0 10 20 30 40 50 60

0 20 0 40 0 6 00 8 00 10 0 0 >120 0
Lead in M uscle (ppm) 0.00 0.16 0.32 0.48 0.65 0.81 0.97FrequencyProbability 7 Bins

0 10 20 30 40

0 2 0 0 4 0 0 6 0 0 8 0 0
1 0 0 0
1 2 0 0
1 4 0 0 > 1 6 0 0
Lead in M uscle (ppm) 0.00 0.16 0.32 0.48 0.65FrequencyProbability 16 Bins

Table B-6. Sample values of lead concentration (ppm) in quail breast muscle ( n =62). 0.45 15.8 36.6 57 91 173 265

2.1 16 40 59.6 94.2 175.6 322 5.4 16.7 40.1 61.4 99 176 490 7.8 21 42.8 62 107 177 663.4 7.8 23 44 64 109 205 703 8.8 24 46 64 111 239 1231 11.8 24.8 47 84.6 149 241 1609

12 29.2 49 86.6 149 245 1634 15 35.5 53 86.8 154 264 Figure B-5. Histograms of lead concentrations in quail breast muscle ( n =62). The top left panel shows the result with seven bins; the top right panel shows the result with sixteen bins; the bottom panel uses bin widths of 10 ppm to highlight the lower tail (< 250 ppm) of the distribution.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-42 

### ln( ) [ ] [ ] x slope z = + intercept Equation B-5



# ( ) geometric mean

arithmetic mean =

= = +

= −

exp[ *] exp[ *] exp[ * . * ] exp[ *] exp[ * ]exp[ * ] μ
σ μ σ μσ σ geometric standard deviation

standard deviation 0 5 1 2 2 0.5 2 Different methods for obtaining the parameter estimates for the lognormal distribution can be explored in this example. For the lognormal distribution, MLE and MoMM simply require calculating the mean and standard deviation of the log-transformed sample data. For the lognormal probability plot method, the parameters can be obtained directly from the least squares regression line expressed as follows:

such that exponentiating the intercept will give the geometric mean (GM) and exponentiating the slope will give the geometric standard deviation (GSD) (see Footnote 3 of Table B-7). Both the MLE and MoMM estimates will generally match the arithmetic mean of the log-transformed data (i.e., intercept) determined from lognormal probability plots; however, estimates of the standard deviation (i.e., slope) will vary (Cullen and Frey, 1999). In general, the probability plot method yields estimates of the standard deviation that are less than or equal to that of MoMM and MLE, and the results yield closer estimates as the correlation coefficient of the probability plot increases (Cullen and Frey, 1999). Table B-7 summarizes the parameter estimates using MLE, MoMM, and the two lognormal probability plotting techniques described above. The corresponding parameter estimates for the untransformed data are also presented. In this example, the strong linearity of the probability plots (r 2 =0.98) shown in Figure B-6 is an indication that a lognormal distribution is a reasonable model for describing variability in concentrations. The tails of the distributions fit the data fairly well, although the bottom panel suggests that the lognormal distribution slightly overestimates the lower tail. Furthermore, the parameter estimates of the lognormal distribution using probability plotting closely match the estimates using MLE and MoMM.
Table B-7. Parameter estimates for lognormal distribution of lead concentrations (ppm).

Parameter Estimation Method Log-transformed Data Untransformed Data 3 Arithmetic mean [ ] 

###  μ Arithmetic stdev [ ] 

### < σ Arithmetic mean [ ] 

###  μ Arithmetic stdev [ ] 

### < σ Maximum Likelihood Estimate (MLE) 4.175 1.522 207 626 Method of Matching Moments (MoMM) 4.175 1.522 207 626 Log Probability Plot 1 4.175 1.507 203 597 Log Probability Plot 2 4.175 1.543 214 670

1 Least squares regression line for Figure B-6, top panel. 2 Least squares regression line for Figure B-6, bottom panel. 3 For a lognormal distribution, the following equations can be used to convert parameters of the normal distribution of log-transformed data to corresponding parameters of the lognormal distribution of untransformed data. Assume : * and F * are the arithmetic mean and standard deviation, respectively, for the normal distribution of log-transformed data.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-43 

### 0 1 2 3 4 5 6 7 8 9

-3 -2 -1 0 1 2 3

z-score ln(conc) = 1.507 z + 4.175
r 2 

### = 0.976; r = 0.988ln [Concentration (ppm)]



### -3 -2 -1

0 1 2 3

0 1 10 100 1000 10000 Concentration (ppm) z = 0.648 ln(conc) - 2.705
r 2 

### = 0.976 ln(conc) = (0.648) -1 

### z + 2.705 / 0.648 = 1.543 z + 4.175z-score

Figure B-6. Lognormal probability plots of lead in quail breast tissue. Top panel gives z on the abscissa and ln[concentration] on the ordinate. Bottom panel gives concentration (log scale) on the abscissa and z on the ordinate. Equally plausible parameter estimates can be obtained from regression lines using either plotting method. Bottom panel requires an additional step to express the equation that yields parameter estimates [ln(x)=(slope) z + (y-intercept)], where the slope estimates the standard deviation of ln(x) and the y-intercept (at z=0) estimates the arithmetic mean of ln(x).RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-44 Meal Size in Consuming Anglers

0 5 10 15 20 25 30

0 200 400 600 800 1000 1200 1400 g/mealFrequency Meal Size in Consuming Anglers

0 2 4 6 8 10 12 140100200300400500600700800900100011001200 g/mealFrequency Example B-3. Variability in Meal Sizes Among Consuming Anglers A creel survey of anglers consuming contaminated fish was performed to estimate variability in fish meal sizes. The anglers were asked how many people would eat their fish. The lengths of the fish were measured and a regression equation was used to calculate the corresponding weights. The portion of the fish mass that is consumed was assumed to be 40% (e.g., fillets). Results given in Table B-8 are expressed in units of grams of fish per meal. The appearance of the histograms (Figure B-7) suggests that the sample ( n =52) may have been selected from a single distribution. A normal probability plot of the meal sizes (Figure B-8) shows a departure from linearity. Specifically, there appears to be a “kink” in the probability plot at about 400 g/meal, suggesting that the sample may have been obtained from two unique distributions. Both the Filliben test and Shapiro-Wilk test indicated a significant departure from normality at 

### " =0.01. Parameters may be read directly from the equations of the regression lines on the right hand panel of the graph. MoMM and MLE gave similar estimates.

Figure B-7. Histograms of meal size ( n =52) among consuming anglers. Left panel uses 7 bins, while the right panel uses 14 bins. Table B-8. Meal size (g/meal) ( n =52). 65 182 310 405 74 208 314 415 74 221 318 416 77 226 318 477 90 241 327 531 110 248 332 572 111 253 336 608 133 260 337 745 143 261 350 831 150 281 351 907 163 303 360 1053 163 305 365 1189 174 305 390 1208RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-45 y = 241.55x + 357.38 R 2= 0.8056

-400 -200 0 200 400 600 800 1000 1200 1400

-3 -2 -1 0 1 2 3 y = 102.98x + 249.11 R 2 = 0.9529 y = 265.42x + 812.12 R 2 = 0.9369

-150

50 250 450 650 850 1050 1250

-3 -2 -1 0 1 2 3

Example B-4. Bivariate Normal Distributions This example introduces the bivariate normal distribution to illustrate two concepts: (1) use of information on correlations in a Monte Carlo simulation; and (2) specifying distributions for uncertainty in parameter estimates. A brief explanation of the bivariate distribution is presented followed by an example comparing assumptions of no correlation and perfect correlation. A less complex example of a method for addressing correlations in PRA is given in Exhibit B-8. Properties of a Bivariate Normal Distribution One approach that can be used to correlate two random variables is to specify a bivariate normal distribution, which allows for the distribution of one variable to be sampled conditional on the other. A bivariate normal distribution is a special case of a joint distribution in which both x and y are random independent normally distributed variables. A bivariate normal distribution can be specified for all correlation coefficients including ρ =0, ρ=1, and ρ=−1. The bivariate distribution has a three dimensional shape and for ρ= 0, from a bird’s-eye view, is perfectly circular. As correlation increases (i.e. moves towards -1 or 1) this circle narrows and flattens to an elliptical shape, and finally for perfect correlation → =1 and ρ =-1) becomes a straight regression line with a r 2 =1. In three dimensional space the probability of obtaining measurement pairs (x, y) in the region is equal to the volume under the surface in that region. To completely specify the bivariate normal, estimates of the arithmetic mean and variance of the two parameters, as well as the correlation coefficient ( 

### : X and 

### : Y , variances 

### F 2 X and 

### F 2 Y , and correlation coefficient 

### D ) are needed. Figure B-8. Probability plot of meal size data from consuming anglers. The left panel shows the combined data, with a departure from linearity at ~ 400 g/meal. The right panel shows the data split between high consumers (top line) and low consumers (bottom line); note that separate lognormal probability plots were reconstructed for both subsets of the data. The point at which to “split” the distribution in the left panel is somewhat subjective. The break would be more obvious if the two distributions did not overlap.

T HIS E XAMPLE P RESENTS ... • Description of the assumptions associated with the bivariate normal distribution • Guidance on simulating the bivariate normal distribution for two random variables • Application of bivariate normal to a simple linear regression equation relating contaminant concentrations in soil and dust (see Figure B-9). Results are compared to the assumption of no correlation and perfect correlationRAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-46 

### mean y and X X
Y Y

X 

### =+ −

= − μ ρ σ σ μ
σ ρ ( ), ( ) variance 2 2 

### 1 Equation B-6



### mean x and Y Y
X X

Y 

### =+ −

= − μ ρ σ σ μ
σ ρ ( ), ( ) variance 2 2 

### 1 Equation B-7



### X Z X X 

### =+× μ σ 1 Equation B-8



## Y Z Y Y 

## =+× μ σ 2 Equation B-9 In a bivariate normal distribution, values of y corresponding to each value of x follow a normal distribution (Snedecor and Cochran, 1989). Analogously, the values of x corresponding to each value of y follow a normal distribution. Furthermore, if two random variables, X and Y , jointly follow a bivariate normal distribution, the marginal distribution of X is normal with mean 

### : X and variance 

### F 2 X , and the marginal distribution of Y is normal with mean 

### : Y and variance 

### F 2 Y . Conditional Distributions Assume we are interested in the conditional distribution of X given a certain value for Y. For example, if X and Y are positively correlated, we would expect that relatively high values of X tend to correspond with relatively high values of Y. The conditional distribution of X given that Y =y, where y represents a specific value for the random variable Y , is a normal distribution with:

Likewise, the conditional distribution of Y given that X =x, is also normal with:

These general equations can be used to generate a correlated pair ( X , Y ), as described below.
*Note that the mean of the conditional distribution of X is a function of the given value of Y but the variance depends only on the degree of correlation. General Approach for Correlating X and Y To generate a correlated pair ( X , Y ), first generate X using a random value Z 1 from the standard normal distribution:

Next, express Y as a function of the conditional mean and variance of Y given X and a second standard normal variate Z 2 :RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-47 

# [ ] Y ZZ Y Y 

## =+ × +−×

μσρ ρ ( ) 1 2 2 

## 1 Equation B-10 and generate a correlated Y by plugging Equation B-7 into Equation B-9. Using algebra, the combined equations yield the following simplified expression for generating Y :

The important component of this equation is that two random variates are needed ( Z 1 and Z 2 ). An alternative, but less general approach would be to obtain Y by first generating a normal variate X (Equation B-8) and then plugging that value into the regression equation of Y on X to obtain the associated value of Y. While this method maintains a correlation between X and Y , it will underestimate parameter uncertainty. The results are equal only for the special case of perfect correlation ( 

### D =1.0) between X and Y. Therefore, the more general bivariate normal distribution approach (given by Equations B-8 to B-10) is recommended for correctly correlating X and Y because it provides a more robust estimate of parameter uncertainty. Application of Bivariate Normal Distribution to Correlate Concentrations of Zinc in Soil and Dust Assume random sampling of soil and dust zinc concentrations occurs in a residential area. Composite samples of soil and dust are collected from 21 locations such that samples are paired (i.e., each soil sample is co-located with a dust sample) (Table B-9). First the relationship between the zinc concentration in soil and dust is evaluated using simple least-squares regression. Next, the bivariate normal distribution for the slope ( 

### $ 1 ) and intercept ( 

### $ 0 ) is determined, yielding an arithmetic mean and standard deviation for each parameter ( 

### : b0 , 

### F 2 b0, 

### : b1 , and 

### F 2 b1 ), and correlation coefficient 

### D between 

### $ 1 and 

### $ 0 . In this context, the bivariate normal distribution may be considered a distribution for uncertainty in the parameter estimates. Three simulation methods are employed to demonstrate the effect of assuming a bivariate normal distribution for parameters vs. perfect correlation, or independent parameters. Specifically: (1) The slope and intercept of the regression line are described by a specific form of the bivariate normal distribution (i.e., follow Steps 1, 2 in Exhibit B-9, and use Equation B-10 instead of Step 4 ). (2) The slope and intercept of the regression line are described by a general form of the bivariate normal distribution (i.e., follow Steps 1 to 4 in Exhibit B-9). (3) The slope and intercept of the regression line are described by independent normal distributions (i.e., follow Steps 1–4 in Exhibit B-9, but omit the correlation coefficient 

### D in Steps 2 and 4 ). E XHIBIT B-9 S TEPS FOR S IMULATING U NCERTAINTY IN L INEAR R EGRESSION E QUATION U SING A

B IVARIATE N ORMAL D ISTRIBUTION TO C ORRELATE P ARAMETERS ( # 0 , # 1 ) (1) Select Z 1 from a standard normal distribution Z

~ N (0, 1) (2) Calculate $ 0 using Equation B-8, where X= $ 0 , : x = : b0 , and F 2 x = F 2 b0 (3) Select Z 2 from a standard normal distribution Z

~ N (0, 1) (4) Calculate $ 1 using Equation B-10, where Y= $ 1 , : y = : b1 , F 2 y = F 2 b1 , D =correlation between $ 0 and $ 1RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-48 0 500 1,000 1,500 2,000

0 500 1,000 1,500 2,000 Soil Zn Concentration (ppm)Dust Zn Concentration (ppm)
y = 0.193x + 174
r 2 = 0.24 For each approach, Monte Carlo simulations with I =5,000 iterations were run to determine the set of parameter values ( 

### $ 0 , 

### $ 1 ) for a simple linear regression equation. Typically, the uncertainty in the parameter estimates is not accounted for when simple linear regression equations are used to relate to exposure variables in a model. Such an approach may fail to account for important sources of parameter uncertainty. Figure B-10 (middle panel) illustrates the preferred approach for characterizing parameter uncertainty based on the bivariate normal distribution. (Note that the correlation coefficient relating the intercepts and slopes generated from the simulation is consistent with the correlation coefficient that describes the bivariate normal distribution; this is a good check that the simulation was set up correctly and run for a sufficient number of iterations). These results are contrasted with results using a form of the bivariate normal (Equation B-10) that underestimates uncertainty (top panel) unless parameters are perfectly correlated. In addition, the simplistic approach of sampling from independent normal distributions (bottom panel), yields a “shot gun” scatter plot. Sampling from independent normal distributions results in unlikely extreme combinations of the slope and intercept more often than the correct bivariate normal approach; propagating this bias through a risk model may severely bias estimates of uncertainty in risk.

Bivariate Normal Distribution for Parameters of the Regression Equation B 0 mean 173.9 variance 4162.2 B 1 mean 0.193 variance 0.0063

s 2 27857.4 Cov (B 0 , B 1 ) -4.2428 r -0.8254 Figure B-9. Simple linear regression of zinc concentrations in soil and dust

. Table B-9. Zinc concentrations in paired (i.e., co-located) soil and dust samples (ppm) for n =21 locations. Sample Soil (X i ) Dust (Y i ) Sample Soil (X i ) Dust (Y i ) 1 120 216 12 560 200 2 190 149 13 560 256 3 270 83 14 720 496 4 285 508 15 800 239 5 310 215 16 880 203 6 340 219 17 910 757 7 350 203 18 1035 676 8 380 101 19 1445 426 9 440 178 20 1600 522 10 480 232 21 1800 276 11 560 199RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-49 r 2 = 1.0

-0.20 0.00 0.20 0.40 0.60

-100 0 100 200 300 400 500 Intercept (B 0 )Slope (B1)

mean B 0 = 174 mean B 1 = 0.193

r 2 = 0.68

-0.20 0.00 0.20 0.40 0.60

-100 0 100 200 300 400 500 Intercept (B 0 )Slope (B1)

mean B 0 = 174 mean B 1 = 0.193

-0.20 0.00 0.20 0.40 0.60

-100 0 100 200 300 400 500 Intercept (B 0 )Slope (B1)

mean B 0 = 174 mean B 1 = 0.193 r 2 = 0

Figure B-10. Results of Monte Carlo simulation ( n =5000 iterations) to estimate the slope and intercept of a regression equation. Top panel reflects the bivariate normal distribution for the special case that fails to capture the parameter uncertainty; middle panel reflects the preferred bivariate normal distribution with D =-0.825 based on empirical paired data; bottom panel reflects sampling from independent normal distributions.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-50 R EFERENCES FOR A PPENDIX B Brainard, J. and D.E. Burmaster. 1992. Bivariate Distributions for Height and Weight of Men and Women in the United States. Risk Anal12(2):267–275. Brately, P., B.L. Fox, and L.E. Schrage. 1987. A Guide to Simulation . Springer-Verlag, NY. Burger, J., W. L. Stephens, Jr., C. S. Boring, M. Kuklinski, J.W. Gibbons, and M. Gochfeld. 1999. Factors in Exposure Assessment: Ethnic and Socioeconomic Differences in Fishing and Consumption of Fish Caught along the Savannah River. Risk Anal . 19(3):427–438. Calabrese, E.J., Stanek, E.J., and Barnes R. 1996. Methodology to Estimate the Amount and Particle Size of Soil Ingested by Children: Implications for Exposure Assessment at Waste Sites. Regul. Toxicol. Pharmacol . 24:264–268. Charney, E., J. Sayre, and M. Coulter. 1980. Increased Lead Absorption in Inner City Children: Where Does the Lead Come From? Pediatrics 65:226–231. Conover, W.J. 1980. Practical Nonparametric Statistics . John Wiley & Sons, NY. Cullen, A.C. and H.C. Frey. 1999. Probabilistic Techniques in Exposure Assessment. A Handbook for Dealing with Variability and Uncertainty in Models and Inputs. Plenum Press. d’Agostino, R.B. and M.A. Stephens. 1986. Goodness-of-fit techniques . Marcel Dekker, Inc, NY. Filliben, J.J. 1975. The Probability Plot Correlation Coefficient Test for Normality. Technometrics 17(1):111–117. Finley, B.L. and D.J. Paustenbach. 1994. The Benefits of Probabilistic Exposure Assessment: Three Case Studies Involving Contaminated Air, Water and Soil. Risk Anal14(1):53–73. Gilbert, R.O. 1987. Statistical Methods for Environmental Pollution Monitoring . Van Hostrand Reinhold, NY. Gilliom, R.J. D.R. Helsel. 1986. Estimation of Distributional Parameters for Censored Trace Level Water Quality Data, 1. Estimation Techniques. Water Resources Research . 22:135–146.. Hahn, G.J. and S.S. Shapiro. 1967. Statistical Models in Engineering . John Wiley & Sons, NY. Helsel, D.R. and R.M. Hirsch. 1992. Statistical Methods in Water Resources . Elsevier. Amsterdam. Hoffman, F.O. and J.S. Hammonds. 1992. An Introductory Guide to Uncertainty Analysis in Environmental and Health Risk Assessment. ES/ER/TM–35. Martin Marietta. Hora, S.C. 1992. Acquisition of Expert Judgment: Examples From Risk Assessment. J. Energy Eng

. 118(2):136–148. Johnson, N.L., S. Kotz, and N. Balakrishnan. 1995. Continuous Univariate Distributions . Volume 2, Second Ed. John Wiley & Sons, NY. Law, A.M. and W.D. Kelton. 1991. Simulation Modeling and Analysis . McGraw-Hill, NY.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-51 Looney, S.W. and T.R. Gulledge. 1985. Use of the Correlation Coefficient with Normal Probability Plots. American Statist. 39:297–303. Mendenhall, W. and R.L. Scheaffer. 1973. Mathematical Statistics with Applications . Duxbury Press. Mood, A.M. and F.A. Graybill. 1963. Introduction to the Theory of Statistics . Second Edition. McGrawHill, Inc. Morgan, G.M. and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis . Cambridge University Press, NY. Nelsen, R.B. 1986. Properties of a One-Parameter Family of Bivariate Distributions with Specified Marginals. Comm. Stat. (Theory and Methods) 15:3277–3285. Nelsen, R.B. 1987. Discrete Bivariate Distributions with Given Marginals and Correlation. Comm. Stat. (Simulation and Computation) B16:199–208. Oregon DEQ. 1998. Guidance for the Use of Probabilistic Analysis in Human Health Exposure Assessments . Waste Management and Cleanup Division. Interim Final. November. Ott, W.R. 1990. A Physical Explanation of the Lognormality of Pollutant Concentrations. J. Air Waste Manage Assoc. 40(10):1378–1383. Ott, W.R. 1995. Environmental Statistics and Data Analysis . CRC Press, Boca Raton. Palisade Corporation. 1994. Risk Analysis and Simulation Add-In for Microsoft Excel or Lotus 1-2-3

. Windows Version Release 3.0 User’s Guide, Palisade Corporation, Newfield, NY. Roseberry, A.M. and D.E. Burmaster. 1992. Lognormal Distributions for Water Intake by Children and Adults. Risk Anal. 12(1):99–104. Royston, J.P. 1982. An Extension of Shapiro and Wilk’s W test for Normality to Large Samples. Appl. Stat . 31:115–124. Snedecor, G.W. and W.G. Cochran. 1989. Statistical Methods . Eighth Edition. Iowa State University Press, Iowa. Stanek, E.J. and Calabrese, E.J. 1995. Daily Estimates of Soil Ingestion in Children. Environ. Health Perspect. 103:176–285. Thompson, K. 1999. Developing Univariate Distributions from Data for Risk Analysis. Hum. Eco. Risk Assess. 5(4):755–783. Tukey, J.W. 1977. Exploratory Data Analysis . Addison-Wesley, Boston.
U.S. EPA. 1982. Air Quality Criteria for Particulate Matter and Sulfur Oxides . ECAO, ORD. EPA 600/8–82-029.
U.S. EPA. 1992. Guidance for Data Useability in Risk Assessment, Part A. Office of Emergency and Remedial Response, Washington, DC. OSWER Directive No. 9285.7-09A.RAGS Volume 3 Part A ~ Process for Conducting Probab ilistic Risk Assessment Appendix B ~ December 31, 2001

Page B-52 U.S. EPA. 1994. Guidance for Conducting External Peer Review of Environmental Regulatory Models. Office of the Administrator, Washington, DC. EPA/100/B-94-001. July.
U.S. EPA. 1997a. Exposure Factors Handbook. Office of Research and Development, Washington, DC. EPA/600/P-95/002Fa, Fb, and Fc. U.S. EPA. 1997b. Use of Probabilistic Techniques (Including Monte Carlo Analysis) in Risk Assessment , Memorandum from Deputy Administrator Hansen and Guiding Principles for Monte Carlo Analysis . EPA/630/R-97-001.
U.S. EPA. 1999a. Report of the Workshop on Selecting Input Distributions for Probabilistic Assessments. Risk Assessment Forum. EPA/630/R-98/004. January. U.S. EPA. 1999b. Options for Development of Parametric Probability Distributions for Exposure Factors. Office of Research and Development. Research Triangle Institute Final Report. April 6.
U.S. EPA. 2001. Development and Evaluation of Probability Density Functions for a Set of Human Exposure Factors. Office of Emergency and Remedial Response. University of California Draft Report. May. Vose, D. 1996. Quantitative Risk Analysis: A Guide to Monte Carlo Modeling . John Wiley & Sons, NY. Wonnacott and Wonnacott. 1981. Regression: A Second Course in Statistics . John Wiley & Sons, NY.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-1 

## A PPENDIX 

## C
C HARACTERIZING 

## V ARIABILITY AND 

## U NCERTAINTY

IN THE 

## C ONCENTRATION 

## T ERM 

### C.0 T HE 

### C ONCENTRAT ION 

### T ERM AN D THE 

### E XPOSURE 

### U NIT Incomplet

e knowledg e of the

concen tration of one o

r mor e chem icals in vario us expo sure m

edia is often the major source of uncertainty in Su perfund risk assessm

ents. In a ny risk assessm

ent, the deriva tion of the concentrati on term will re flect as sump tions ab out: (1)proper ties of t he cont amina nt, (2) the s pat ial and t empor al va ria bil ity i n con tami nat ion, (3) t he be havi or of the r ecep tor , and (4) t he ti me scale of the toxicity o f the chemical(s). This appendix expands upo n concepts introduced in Chapter 5. This appendix does not provide detailed equations for performing

calculations, but instead re fers the reader to other Env

ironmental Protection Agency (E PA) guidance d

ocuments in w

hich both the recommen

ded approaches and ca

lculations are provided. The concentration term is linked to the con

cept of an exposure un

it (EU). For Supe

rfund risk assessments, an EU is the geog raphical area in which a receptor is randomly exposed to a contam

inated medium for a relevant exposu re duration. Env

ironmental sam

pling provides information about the contam ination within and aro und an EU. Mu

ltiple E Us may be defined

at a site based

on the choice of a receptor, the exposure m

edium, and the nature of contact with the medium . For exam ple, residential exposures to children m

ay involve exposures via so il and dust ingestion both at the prim

ary residence and recreati onal are as at a d

ay car e facilit y. S ite-s pecific inform

ation re gardin

g the activi

ties of r eceptors should guide assum

ptions about the receptor’s c ontact with exposure m edia.

L Definin

g the E U is cri tical tothe success of the rem edial st rategy, as it affects the calculation of the concentration to which receptors are exposed. 

### C.1.0 V ARIABI LIT Y IN 

### PRA In general, variability an

d uncertainty should

be kept separate to the extent possible in a ny probabi listic ri sk ass essme nt (PRA ). For exam ple, as sume a one- dimen sional M onte Ca rlo Ana lysis (1-D MCA) was developed to characterize variab ility in risk, b ut it combined a

distribution for uncerta inty i n mea n conce ntration with d istribut ions for varia bility in expo sure variables . The re sult would yield a sin

gle distribution for risk, however, ea ch risk estima

te would reflect both un

certainty and variability and d

istinguishing between the two would not be possible. Therefore, EPA’s Guiding Principles for Monte Carlo analysis recommen ds against m

ixing distributions of v

ariability and uncertainty in a 1-

D MCA (U.S. EPA, 1997b) to avoid such am biguities. A fundamental concept in Monte Carlo an alysis is that there is v

ariability in exposure b etween receptors (inter-indiv

idual variability) as well as day- to-day v ariability for each indiv

idual (intraindivi dual var iabil ity). In most Tier 2 analys es (se e Chapt er 2), t he goal of a 1-D MCA is t o chara cteri ze inter- indivi dual variabili

ty in exposure

and risk. Ty

pically , proba

bility distributions f

or expo sure represent variability (PDFv’s) between individuals in the average value over the entire exposure duration. In this case, the exposure point conce ntration (EPC) should rep resent the averag e exposure concentrationRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-2 over the entire expo sure duration. Because an EPC is calculated from a sam ple, there is uncertainty

that the sample m

ean equals the true m

ean concentration within the EU; therefore, to acc ount for associated uncertainty, the 95% upper confidence lim it for the mean (9 5% UCL) is generally used for Sup erfund risk a ssessm

ents (U .S. EPA, 199 2).
In a 1-D MCA, a point e stima

te for the EPC is com

bined w ith PDF v’s for other variab les to y ield a probability distribution for risk. An a lternative approach is to

simulate long -term a

verage expos

ures as a series of consecutive

short-term

exposure events. This approach is referred to as MicroExposure Event (MEE) Monte Car

lo mo deling , and i s discus sed in d etail in Append

ix D. In MEE mo deling , the goal is to develop PDFv’s for exposure v ariables that capture the ev ent-to- event variability

in exposures at the individual level. The concept of an av eraging tim

e still applies, but g enerally to a shorter tim e frame. For example, seasonal variability in exposure frequency might be expected amon

g outdoor occupa

tional workers so that d

ifferent PDFv’s are representative of inter- individual for each se ason. In this case, the EPC continues to rep

resent an averag e concentration within the EU, but it would be linked to season

specific activity pa

tterns. It m

ay be imp ortant to develop tw o different weighted averages to reflect season-specific activ

ity patterns and location s that are more freq uently contacted in the summ er compared with the winter, for exam ple. As the tim

e frame for the ex

posure scenario is shortened from the enti re expo sure du

ration, to a sea son, to

a day , to an

indiv idual ev ent, th e conce

ntration term should be reev aluated to asse ss the r

elevan ce of the assum

ption that conc entrations cont acted by the receptor are represented by the m

ean of the meas

ured sample. The following discuss ion intr oduces concept s of tem

poral an d spatia l varia bility as they apply to the estimate of the EPC for different exposu re media and ex

posure scenarios. While the general rule of thumb applies to all Monte Carlo mo dels—use a measure of the average concentration within the EU over t he tim e fram e of exp osure—it

is im portant to appl y the site sam

pling data in a way

that is consistent with the exposu re scenario. 

### C.1.1 T EMPORAL 

### V ARIABILITY Temporal variability

in chemical con

centrations may

be an importan

t consideration when devel oping

a prelim inary remed

iation goal (PR

G) for a ny ex posure mediu

m (ref er to Chapter 5 for a compreh ensive discus sion of using PRA t o evalua te PRGs) . For exa mple, win d erosi on may change chemical concentration s in surface soil ove r time; leaching

may change concentrations in both subsurface soil and groun dwater; and bio accum

ulation may result i n increasing c oncentr

ations i n preda tory f ish with time. If p ossible , such factors should be cons idered e arly in the risk assessm ent proc ess and

include d in the conceptual

site m odel. Devel opme

nt of the EPC norma

lly wi ll depen

d on th e aver

aging time releva

nt to the expos

ure scenario and he alth end

point o f conce

rn. I n the shorter te

rm, i t may be unli kely that rec eptors a

re exposed throug hout th e entire

EU due to tem

poral (a nd spat ial) va riability in the c ontam

inant and interindividual variability

in activity pa

tterns. Therefore, inter- individual variability in the EPC m

ight be expected, and a distribution of EPCs m ay be deve loped to represent differences in exposure amo ng the populat ion. Va riability in shor t-ter m exp osure m

ay be an im

portant factor f or asses sing v

ariabili ty in acute toxicity. H owever, over time, short- term variability

in the EPC will ten

d to smooth o

ut and approach a long-

term average concentration. A s ingle estimate of the long-term average EPC may be reasonable to use in assess ing risks to the receptor population. This is true regardless of the underlying distribution of the env

ironmental sam

pling data (e.g ., lognorm

al, normal, beta , etc.).RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-3 While m ost chem icals re gulate d by t

he Supe rfund program

are bas ed on c oncerns

for chr onic toxi cit y (e.g. , lif

eti me can cer risk from e xpos ure t o a ca rci nogen for t en or more ye ars ), fo r some chemicals, toxic effe cts occur with shorter exp osure durations (e.g ., nitrate in drinkin g water and methemog

lobinemia in infants). Differences between a cute and chronic health e ndpoints are impo

rtant to consider for ecological receptors such as transient migratory species. Superfund guidance distinguishes between acute and chronic expos ure to provide risk

assessors the option of evaluating risk under different time frames. The EPC should be estim

ated within an EU d

uring a period of time that has toxicological releva

nce for the exposed po

pulation.

L The time scale of the concentration term should match the time scale of the toxicity criterion and exp osure duration. 

### C.1.2 S PATIAL 

### V ARIABI LIT Y Spatial variability in

chemical concen

trations is also an im

portant property to co

nsider when developing a PRG. Spatial v ariability arises from

many factors, including the mechanism of contamination, ph

ysical and chem

ical dilution and transform ation processes, and ph

ysical characteristics of the site (Cullen and Frey, 1999). Similarly, rece ptors may exhibit spatial variability in their contact with an exposure m

edium. I n general, receptors are assumed to h ave equal access to

all areas within an EU so that the conce

pt of a long-

term average concentration is applicable. Often, the EPC is estimated without re gard to the spatial p atterns in contamination. The samplin g design yie lds a meas ure of t he varia bilit y in conce ntrat ions th at is as sumed to b e repr esent ative of the r eceptor ’s conta

ct with the exp osure m

edium . How ever, even when the sam pling design

is representative (e.g ., both are sim ple random sam

ples within the EU), the concentrations m ay exhibit clear spatial patterns that could be used to reduce uncertainty in the EPC. Geostatistics (see Section C.5 .2 and Appendix D) offers a wide range of tech niques for incorporating

spatial information in

to estimates of the EPC. These techniques

are par ticularl y usef ul when there is uncerta inty i n the re present ativeness of s ite sampling, due to a difference in sca le between site sam

pling and the size of the EU, or the use of targeted sampling de signs that ove rsample areas within

an EU believe d to contain the hig hest levels of contamination. In point e stimat e risk as sessmen ts (T ier 1 of the PRA) , the EPC is most of ten cha racte rized by a point estimate of the

mean concentration, typically g iven by the 95% UCL for the m ean to account for uncertainty in the site characterization (U.S. EPA, 1992). Variability in concentrations is an important consideration for determ

ining appropriate statistical m ethods used to estim

ate the 95% UCL. In addition, for some Monte C arlo models, a PDFv m ay be deve loped to determine the EPC for the exp osure model. A PDFv for the EPC ma

y be warranted in

short-term

exposure scenarios, particu larly when the sam

pling density is relatively

sparse in relation to the s ize of the EU (i.e., poor site characterization). For exam ple, a risk assessm ent may inc lude a f

uture us e reside

ntial sc enario ( e.g., current ly the site is undeveloped)in which t he EPC th at is r elevan t to a pot entia lly expo sed pop ulati on of chi ldren is the a verage concentration within a 0. 5 acre lot. I f the soil sampling yields 100 m

easurements, but a small subset of the samples (e.g ., less than three) are av ailable for any 0.5 acre area, the m ost appropriate meas

ure of the average conce

ntration for a h ypothetical re sidence

may be the maxim

um de tected c oncentr

ation or a single value from the PDFv in concentration am

ong hypothetical receptors. I n general, for any

of the EU’s that define a randomly located residence, the poor site ch aracterization would b e a source of uncertainty in both a

point estimate and

probabilistic risk assessm ent.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-4 At the v ast majority of sites , conce ntration data is the eas iest dat

a to obt ain of a ll the ex posure variables. I

n cases of poor site cha

racterization, risk managers m ay opt to perform

a point estimate risk assessment only

using the m

aximum detected concentration and h ighly protectiv

e exposure assum

ptions. In the scenario described above for 0.5 acre residential lots, it is possible that a residence would be located in an area in w

hich the averag e concentration is represented by the m aximum detected concentration in the sam

ple. Should the risk

manager opt for a Tier 1 point estim ate risk assessm

ent, the use of t he ma ximum detecte d conce

ntration of a chemica

l on the site should en sure the

perform ance of a health-protective risk assessm ent within a sm

aller EU. Consideration of va

riability is also warranted in short-term scenarios for ecological risk assessment (ERA

) when the EU is much sm aller than the site (see Section C.3.1.1). F or example, the home rang e of the receptor population s may be relatively sm

all in comparison to the spatial distribution of sam pling location s (e.g ., ben

thic inv ertebrat es living in

the sed iment at the b ottom

of a riv er or so il invertebrates in a terrestrial habitat). In these cases, the receptor would be exposed to a n area smaller than the sampling

grid or m

easure of areal sam

pling density. A value from the PDFv that characterizes variab ility i

n the concentra

tions ac ross a r elativ ely la

rge sp atial sca

le ma y be u sed to d

efine the EPC for a receptor popul ation at

a sma ller sca le. Again, risk asse ssors sh ould tak e care in desig ning a 1-D Monte Carlo model w

hen using a PD Fv for the concen tration term. I

t is unadvisable to mix a PDFv for the concentration term w

ith PDFv’s for other exposure scenarios w hen estimating

risks within one EU. Use of the PDFv in this manner w ould incorrectly sug

gest that the m

ean concentration varie d for each individual within the same EU acco rding to the variability in concentration measured across a much larger area. A p

referred approach is to use a PDFv to o btain a point estima

te that represents the EPC , and then combine

this po int estim

ate with PDFv ’s for o ther v ariables in the M onte Ca

rlo sim ulation to estimate risk

s in the smal l EU. If there are many EU’s a t a site, or if the bound

aries of EUs ar e undefined, more

advanced m

odeling approaches can be developed to efficiently run m

ultiple scenarios. Methods for characterizing exposure point concen

trations for ecological rece ptors are further discussed in Sections C.2 and

C.3.
C.1.3 E XAM PLE O F T EMPORAL AND S PATIAL V ARIABIL ITY Exposure scenarios often require consideration of both temporal and spatial variability. The MEE might be used to assess tem poral variability by simulating long-term

intake as the sum

of individual exposure events. The time step for MEE is an importan t consideration and will d epend on the rate of change of the m

ost rapidly changing exposure v ariable. In addition, there should be a correspondence between the time periods over which data were obtained an

d the time step u sed in the MEE m odel. For example, whe

n a MEE is used for the risk assessm ent, the concentration term selected at each tim

e period should match the

“average” co

ncentration within the EU appropriate for that particular tim e period. Assume that the receptor is a residential child, and the time perio d is a single day , and the child m ay contact only 1,000 square feet wit hin the 0.5 acr

e (20,0 00 squa

re feet) residen

tial EU . The sp ecific 1,000 square foot are a may chang e with e ach day

as the c hild cho oses dif

ferent a reas in the ya

rd to frequent. Hence, the

variability in the

sample m

ay be a m ore appropriate mea sure of the concentration contacted by residential child receptor on a day -to-d ay basis than the long-term average w

ithin the 0.5 acr e EU. Over t he long-te rm, this recept or will be expos ed to th e enti re EU and hence t he avera ge contaminant concen

tration within the 0.5 ac

re EU. Note that the day- to-day v ariability in concentration unde rgoe s the famil iar phen omeno n of “ regr ess ion t o the mean” when cons ide red o ver t he lo ng-ter m.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-5 C.1.4 S PATIAL AND T EMPORAL V ARIABIL ITY FOR D IFF ERENT E XPOSURE M EDIA C.1.4.1 V ARIAB ILITY O F C ONCENTRATIONS IN S OIL Surface soil is subject to

erosion by wind and

surface water r unoff. Over

time, concen

trations in surface soil may

change, but generally at a slow rate relative to other m edia. The spatial variability of chemical contam

ination is most often

due to the m

echanism by

which the contam

ination occurred. For example, particulate stack emission s will tend to fall in an even pattern down wind of the stack

whereas over-app lication of pesticides and che mical spills can result in a patchy pattern of contamination. Subsur face soi l is not subject to

wind e rosion, so concentrati ons cha nge m ostly due to degradation processes or leaching of the contaminant to g roundwater. At m

ost Superfund sites, concent rations of chem

icals in subsur face soi l will re

main relatively constant. C.1.4.2 V ARIAB ILITY O F C ONCENTRATIONS IN G ROUNDWATER Exposure to grou

ndwater contamina

tion mostly occurs at a fixed point in space (e.g., the wellhead). Groundwater is subject to a variety of influences that can alter chemical concentration s within this m

edium such as aerobic and anaerobic

biodeg redation , vol atizat

ion, an d absor ption. Due to the se in flu ence s, mon ito red n atu ral atte nuat ion i s an a ppro pri ate remed y unde r cer tai n sit e con dit ions . If a risk assessor wishes to use a measure of the long- term average of a concentration in g roundwater, a hydrogeolog

ist should be consulted.
C.1.4.3 V ARIAB ILITY O F C ONCENTRATIONS IN S URFACE W ATER Concentrations in surface water can be very dynamic. Streams are co nstantly flowing

and the effects of mixing , dilution and evaporation can chang e the chemical co ncentrations in surface water over relative short time periods. Any sampling of surface water is truly a “snapshot” in tim

e. The sampling methods used to charac terize spati al and t emporal variabi lity of concen trati ons in s urfac e water will ha ve a dir ect effe ct on the u ncer tai nty i n est imat es of the a vera ge con cent rat ion o ver bo th sh ort and l ong ti me frames.
C.1.4.4 V ARIAB ILITY O F C ONCENTRATIONS IN S EDIMENT In som e situations, sedime

nt may b e considered a relatively stable medium, similar to soil. Alternatively, s ediment m ay be phy sically mov

ed by currents, tid

es, the mov

ement of ships a

nd other event s. Trend

analy sis may be used to

establi sh the l ong-

term a verag e sedim

ent tran sport at

a site. This information could p

rovide the basis for choosing a represen tative “averag e” concentration in the sediment ava

ilable to ecological receptors (Piest and Miller, 1975; Van Sickel and Bes chta, 1983; Walling , 1983 ; Meade e

t al., 1 990).
C.1.4.5 V ARIAB ILITY O F C ONCENTRATIONS IN F ISH Concentrations in fish m

ay vary due to a chang e in the availability of food and env

ironmental conditions. Factors that may be used to model population dynam ics may inc lude intensity of ang ler harves t, deat h/att riti on of th e popul ation , and the intro ducti on of a pr edato r speci es or a mor e adapt ive species. In risk assessmen ts that include a fish ing estion exposure pathway , the activities of the angler may be a more im

portant factor in determ

ining the EPC than the changes in concentrations in fish ov erRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-6 time. For example, an avid re creat ional angler may ha rvest f ish fr om diffe rent l ocati ons wit hin a la ke and consume fish

of different sizes and

species. I

n this way, w

ith the consumption of contaminated fish, both the contamin

ated medium

and the exposure po int change throug hout the exposure duration

. Unless, samples of fish are collected over time, kn owledge of these factors will generally be unknown. Concentrations of bioaccum ulative chemic als in territorial fish (e.g. , largemou th bass) obtaine d in dif

ferent l ocation s will generally reflect the concentrati ons in t he sedim ent in t he indiv idual’s home territor y. C oncentr

ations o f bioacc

umula tive chemic als in m

igratory fish

will be more difficu lt to predict as the fish will con tact areas with vary

ing sedimen

t and surface water conc entrations.
C.1.4.6 E XAM PLES O F T EMPORAL AND S PATIAL V ARIABIL ITY I N THE C ONCENTRATION T ERM F OR S ELECTED E XPOSURE M EDIA Whatever me dium is consi dered in the dev elopment of E

PCs, t he risk assessor should

be awa re that the EPC em

bodies aspects of both the spatial distribution of contam ination, the mov

ement of the receptor, and possibly

the contaminated medium within the EU. Table C-1 presents examples of sourc es of temporal and spatial variability in the concentration term based on both the contamination in se lected exposu re me dia and

the rec eptor.

Table C -1 . Exampl es of tempor al a nd spatial vari abil ity i n sel ecte d media for the concen tra ti on te rm in common expo sure scenarios.

Factor Soil Groundwater Fish Temporal Variability Contamina

nt • none, if c ontaminant source is inactive • seasonal fluctuation in groundwat

er table • seasonal changes in species

availability

• aerial de

position from ongoing sourc e emis sions affe cted by

wind patte

rns • migra

tion of conta

minant plume • bioconcentra tion

• degra dati on o ver ti me • natural at tenuation • long-term changes in population dynamics

• volatilization • fish tissue concentrations linked to temporal variability in water and sedim

ent concentr

ations

• migration to groundwater • physical and chemical processes

• radioactive growth and decay Receptor • changes in activi ty patte rns and behavior

s over tim e (e.g., with age) • none, fixed location at specific wellhead • dietary preferences f or fish species

• changes in well location over t ime • cooking practices Spatial Variability Contamina

nt • heteroge

neity in concentrat

ions over a

small area and with depth, including presence of hotspots • migra tion of conta

minant plume, based on hydrogeology and source emiss ions (e.g.,bulk flow or continuous source) • migration of fish

• heteroge

neity in soil properties that influence bioavaila bility • changes in fish popula tion structure Receptor • daily activit y patterns in volve

contact w ith diff erent are as of the EU • none, fixed location at specific wellhead • change in recreational habits, and areas fished

• changes in well location over t imeRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-7 Figure C-1. Spatial and temporal variabil ity in contaminant concentrati ons in groundwater. 

### C.2.0 N ONRAND OM 

### E XPOSURES As discussed in Section

C.1.2, in the lo

ng-term it is generally assumed receptors exhibit random movem ent, such that there is an

equal probability of contacting any area within the entire EU . Therefore, the long-term

exposu re conc entration will m

ost lik ely be the arit hmeti c mea

n of the concen tration within the EU. Howe

ver, in m any situations, the assumption of ran dom exposures in space may clearly be an oversimplification. People’s behav ior and preferences will cau se them to access specific areas within an EU wit h grea ter freq uency than ot hers. The same is true

in terms of eco logica

l recept ors with specifi c habitat preferences.

For exa mple, groun

dwater concent

rations may

show a large varia tion when sam

pled from we lls in different locations (Fig ure C-1). Typically, residential receptors do not sam ple randomly

from different wells, but draw chronically from individual wells. In such a case, the EU is a single wellhead. Fluctuations in the g

roundwater plume will depend on the hydrogeolog y of the site as well as the seasonal fluctuations in the w ater table. In

this hypothetical exa

mple, concentration s are declining ov er time at distances ne arest to the source, and concentrations are increasing as the plume moves farther from the source.
Incom plete information regarding the behav ior patterns of people and environmental systems can be a lar ge sou rce of u ncertain ty in a risk a ssessm

ent. B ecause of this, methods are being devel oped to model spa tial r elati onshi ps (bet ween th e conta minant a nd rece ptor) and nonr andom exp osure s. Recen tly, a quantitative technique

to model non

random exposure

has been proposed fo

r ERA (Hope, 2 000, 2001). Briefly, this techniqu

e divides the EU

into smaller subu

nits and uses inform

ation about the attractive

nessRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-8 of each subunit to assig

n a probability of the receptor occupyin g a given subunit for a period of time. Receptor mov

ements are m

odeled stochastically an

d a time- weighted average of all the sub units provides a measure of the EPC. I

n some ecolog

ical risk assessm

ents, telemetry

data can be used to better characterize the areas of contamination that overlap with habitats o f selected species. Hoff (1998) demonstrates an ap

proach for Ame

rican badgers ( Taxidea taxus ) in which telemetry data and geostatistical mod

eling provide an improve d relationship between contaminant concentration s, tissue residues, and effects. 

### C.3.0 S OURC ES OF 

### U NCERTAINTY IN THE 

### C ONCENTRAT ION 

### T ERM There are numerous p otential sources of uncertainty in the estimate o f the true mean concentration within an E U. As discussed in Chapter 5 (Section 5 .1.1), sources of un certainty can be grouped into four broad categories: sam ple data, location of the EU, behavior of the receptor, and from misce llaneou s source s (e.g ., physical an d chem ical pro cesses). Dev elopment of a n uncer tainty distribution for the avera ge concentration requ

ires knowledg e of the variability in chemical concentrations within the E U (unless distribution- free approaches are used),the toxicity of the chemicals, and the receptor’s behav

ior. These distributions should be developed by risk assessors with the concept of the EU in m

ind. Differences in scale (e.g., sm all home range of an ecological receptor population relative to the site sam

pling design) can be a major source of uncertainty in e cological risk ass essments. Methods for add ressing such uncert ainties in the concentratio n term are presented be low. By incorporating

these m ethods into the

quantit ative uncerta

inty a nalysi s, risk

mana gers m ay m ore effectively evaluate the importance of data-g aps and design s ubsequent rounds of site sampling to reduce the uncertainty in the EPC. 

### C.3.1 Q UANT IFICATION O F 

### U NCERTAI NTY 

### B ASED ON THE 

### S IZE OF THE 

### E XPOSURE 

### U NIT Site characterization som etimes occurs before an EU has been d efined. Therefore, an EU may be small er than an entir

e site, equal to

the site itself, or larg er than the site . These three condition s lead t o differen t conclu sions an

d met hods ab out the determ

ination of the E PC. Th e most com

plex sit uation i s when the EU is sm

aller than the site and the site can contain m ultiple EUs. For futu

re scenarios in which the land use differs from

the current land use, the difficulty in predicting the exact size and

location of EUs necessitat es accounting for th e uncertainty in th e EU. Comp osite sa

mplin g is o ften use d to m

aximize site

inform ation. Howev

er, it is important to

note that the use of com

posite sampling

influences the concentration term. I f composite sam pling is used exclusively at a

site, the actual m

aximum concentration present or the best estimate of this maximum concent ration w ill not b e avai lable. Depend ing on the tim e scale of the t

oxic eff ect or w hether a cute toxicity should be c onsidered, this lack of knowledg e of the maxim um concentration present may be a large data gap . Risk assessors are urged to consider composite sa mpling and

its ramifications for the conc ent rat ion t erm. C.3.1.1 W HEN THE E XPOSURE U NIT I S S MALLER THAN THE S ITE The size of the EU will be different depending on the length o f exposure. A receptor can access a greater area if given more time. I n almost all cases, the size of the E U for short-term

exposure will be small er than the EU for long-term exposu re. Ther efore, in addit

ion to t he unce rtainty associa ted with sampling an

d analysis (which can be quantified with existing metho ds for calculating confidence intervals), there is unce

rtainty about the location of the EU within the site.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-9 If contam ination is evenly

spread across the site, the location of the EU may not have any b earing on the E PC. I n such a case, uncerta

inty m ay dep end on the sam

ple siz e or den sity o f mea surements within the EU relative to the entire site. I n point estimate risk

assessments, the concentrations of chemicals at the sam

pling location that po ses the greatest risk may be considered as estim

ates of the EPC for this small EU . Using this “riskiest” samplin g location as an e stimate of the m ean within an EU of unknown location

accounts for both the uncertainty associated w ith limited sam

pling within a sing

le EU and the uncertai nty of the location wi thin the site of the EU. To express the uncertainty in location of the EU as a distribution, m ethods have been developed to place an EU of a given size randomly about a site (Burm

aster and Thompson, 1997). A conce ntration term is deve loped for each of a large number of ran domly loca

ted EUs. The distribution of these concentrati on terms will express the uncertaint y in the location of the EU. Risk assessors are c autioned to consider whether the statistical metho d used to estima te the EPC in an EU accounts for all sources of uncertainty in the concentration term . If only a few samples

are used to characterize the av

erage concentration w

ithin an EU, then the uncertainty in the EPC is large an d should be presented in the risk characteriza tion. These conditions m ay warrant additional sampling or the use of analytical methods that acco unt for spatial variability within the entire site.
At so me sit es, ge ost ati sti cal metho ds, p att ern r ecog nit ion, and ge ogra phic al in for mati on sys tems (GIS ) meth

ods may provide a

dditional insig

ht and w ill aid i n the de velopment of the c oncentr ation te rm (see Section C.5.2). Although Table 3 -1 shows several statistical m ethods for estimating

both point estimates and

distributions t

hat enc ode unc ertainty

in the c oncentr

ation te rm, a risk a ssessor ’s underst anding of thes e uncer

tainties should be conceptual

as opp osed to

purely statisti cal.
C.3.1.2 W HEN THE E XPOSURE U NIT I S THE S AME S IZE AS T HE S ITE In this case, the entire environm ental data set within the site boundaries can be used for the determination of the concentration term. Assuming the EU occupies the

entire site, then the sourc e of uncertainty associated w

ith knowing

the average concentration within the E U is the samp ling and analytical uncertainty

. C.3.1.3 W HEN THE E XPOSURE U NIT IS L ARGER THAN THE S ITE In th is case, the EU

extend s beyo

nd the s ite boun

daries. Therefore, the entire e nviron menta l data set within the site bound

aries can be used for de

termination of the c oncentration term. However, an additional term in the exposure assessm ent may b e needed to account for the fraction of the expos ures that are expected to occur off site. Essentially, the contribution of the c hemical concentrations

measured on and off site are we ighted by the fraction ingested or contacted in each area. Similarly, the term “area use factor” is used in eco logical risk assessments to refer to the percentage of tim e or area an anim

al inhabits a contamin

ated area. An expo sure scenario in which the EU is defined by the multiple location s that may b e visited would b e a comm on extension of this con

cept. One reasonable assumption reg arding off site exposu

res is that the c oncentr

ations w ould be equal t o the “b ackg

round” concent

rations. If t

his assumption is m

ade, a site risk asse ssor should be consulted to determine appro priate methods for incorporating back

ground concentrations

into the risk assessm

ent. Alternatively , additional samp ling at off site locations would b e needed to estim ate the concentrations.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-10 

### C.4.0 S UMM ARY O F 

### RECOMM ENDATIONS FOR THE 

### C ONCENTRAT ION 

### T ERM Table C-2 prese

nts ge neral g uidelines for e stablish

ing a concent

ration term in

vario us me dia based on expos

ure tim e and the size

of the E U. Thes e gene

ral gu idelines along with si te-sp ecific exposure conditions are the driving factors in risk assessm ent decision mak

ing for establishing

the conc ent rat ion t erm.
Table C-2. Sum mary o f factors tha t may b e con sidere d in de velop ing an E PC

.

M edium Exposure

Time Random NonRandom Size of EU relative to

the s ite /samp li ng density Recommendati on (H uma n H ealth and Eco logic al) Soil Short-term X small HH - consi der va ria bil ity i n conce ntr ati on relative to the time sc ale of to xicity. ECO - time wei ghted aver age of s maller subunits. Soil Long-term X variab le HH , ECO - consider uncertainty i n the average con ce ntra ti on wi th in

an EU. Fish Short-term X variab le HH , EC O - co nside r variab ility in samp le concentr ations relat ive t o the exposur e ti me. Fish Long-term X variab le HH - consider uncertai nty in the average concentration in consumed portion of fi sh. ECO - consider uncertai nty in average con centra tion of w hole fish.Groundwater Short-term X smal l - si ngl e wel l he ad HH - consider ei ther t he high est det ected concentration or uncertai nty around the concen tration at the center of the plum e as a meas ure of a

si ngl e wel l a nd r elate to t he t ime scale o f the toxic effect. EC O - no t app licableGroundwater Long-term X smal l - si ngl e wel l he ad HH - consider var iabil ity among the hi gher con centra tion sam ples as a pro tective E PC

. Alt ernatively, hydr ogeolog ic model ing may be used t o obtai n a l ong- term aver age con cent rat ion in t he most cont aminate d area. EC O - no t app licable 

### C.5.0 M ETHODS FOR 

### E STI MATING 

### U NCERTAINTY IN THE 

### M EAN 

### C ONCENTRAT ION Confidence intervals (CIs) and UCLs are comp uted to characterize un

certainty in a param

eter estimate. CI s can be comp uted for any param

eter. The general m

ethod for estimating

confidence intervals is presented in equation C-1

. CI = param

eter estimate ± (critical v alue) x SE Eq uation C-1 The parameter estim

ate is the estimated value for the unk nown population param

eter. The critical value is the num

ber, z , with probability, p , lyin g to it s righ

t (for an upper c ritical v alue) or left (for a lower crit

ical v alue). For a st

andard norma

l distrib ution (i .e., a

rithmetic mean=0, standar dRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-11 deviation=1), critical v

alues are referred to as the z-score or z-statis tic . These value s are commonly given in statistics texts, and may also be calculated using

the Microsoft Excel function Normsinv(p) , where p corre sponds to the pr obabi lity lyi ng to the right of the valu e. Dist ribut ions th at char acter ize parameter uncertainty

are sometim

es referred to as sam

pling distributions. The standard error (SE) is the standard deviation of the sampling distribution for the param eter estimate. The confidence interval conveys two concepts: (1) an uppe r and lower confidence limit (for a 2- sided CI), and (2) a confidence level (1– " ), which gives the probability that the method y ields an interval that en

closes the parameter (Moore and McCabe, 199 3). Methods for estim ating SE vary for specific param eters. For exam

ple, the SE of a mean concen tration may

be calc ulated b ased on

the sam ple va riance a nd the s ample

size ( due to Central Limit Theorem

). Methods for calculating the SE for other param

eters, such as the 95 th percentile, are more com

plex, and may

be estimated from

a series of nested bootstrap simulations (Efron and Tibshirani, 1993; U.S. EPA, 2001a). When co mpari

ng alt ernative appro aches fo

r quanti fying

param eter unc ertainty , criter

ia that a re impor tan t to c onsi der i ncl ude t he var ian ce of the o rigi nal data set, and t he bi as an d cove rage of th e CIs gener ated by each m ethod. In statistic s, a m

ethod is unbias ed if th e mea

n of the samp ling d

istribut ion is equal to the t rue valu e of the paramet er. Simil arly, a met hod has accura te cover age if th e proba bilit y p that a CI d

oes not cover the true parameter is equ al to the probability le

vel used to constru

ct the CI. For risk a ssessm

ent, th e most desira

ble method is

one tha t deals well wi

th hig h vari ance, y

ields C Is that are sufficiently wide (i.e. , the CI does not underestim ate the probability of enclosing the popu lation parameter), and, more specifically , yields upper co nfidence limits that are not biased low. The ch oice of the most appropriat e method will depend on the ch aracterist ics of the data set and a balance betwee n two objectives: (1) the desire to b e health protective and , therefore, have a low probability o f underestimating the mean, and (2) a desir e to be ac curat e, in th e sense of choos ing a metho d whose e xpect ed cover age equal s the tr ue cover age. As a gen eral p rinci ple for quanti tative uncert ainty an alysis , if alt ernat ive methods y

ield very differen

t answers, it is helpful to

explore the reasons for the differences. The objective is to explain w

hy the estimates of the 95% UCL di ffer, an d to det ermine if the differen

ces are sufficiently great tha

t they could alter the

risk mana

gement dec ision or PRG. This information should b e presented as part of the risk commu nication process associated w ith the scientific mana

gement dec ision points of the tiered process for PRA (see Chapter 2). As discussed in Chapter 5, in Superfund risk assessment, the EPC is usually calculated as the 95% UCL for the m

ean to account for the u

ncertainty in estim

ating the average concentration within an EU. The 95% UCL

is defined as a value that, when repeated ly calculated for randomly drawn subsets of size ( n ), equals or exceeds the true population m

ean 95% of the tim

e. In other words, it is calculated and applied as a 1- sided confidence lim it. The 95% UCL is o

ne percentile on the prob ability distribution that characterizes uncertainty in the mean (i.e., the PDFu fo r the mean). It is equal to the 95 th percentile of the sampling distribution for the mea n. EPA’s g

uidance on calculating

the concentration term

describes the rational e and m

ethodol ogy

for sele cting the 95% UCL as the poi nt estim ate for t he conc entration term (U.S. EPA, 1 992).
Commo n methodolog

ies for characterizing

the 95% UCL for the arithmetic m ean concentration include the following : (1) application of Equation C-1 using

Student’s t-statistic (for normal distributions), (2) Land m ethod using H- statistic (for lognorm al distributions) (Land 1971 , 1975), and (3) bootstrap and Jacknife resampling tec hniques (Efron and Tibshirani, 1993). Details on these methods and on choosing

an appropriate m

ethod are provided in the ORD/OSWER guidance bulletin, Lognormal Distribution in Environm ental Applications(U.S. EPA, 1997a), and the m ore recent OSWER g

uidance bulletin, Guidance on Calcula tion of UCLs at Su perfund Sites (U.S. EPA, 2001a)

. An overview ofRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-12 metho ds that may

be used when d ata are n ot norm al or lo gnorm

al is als o prov

ided by Schulz and Gr iffin (1999). It is the responsibility of the regional risk assessor to ensure that an appropriate method for calculating a UCL

or for developing

an uncertainty distribu

tion is chosen. Cha

pter 3 (Table 3-1) provid es an ov erview of appr oaches for char

acterizing un

certaint y in the conce

ntration term in both 1-D MCA and 2-D MCA. 

### C.5.1 Q UANTIF YING 

### U NCERTAINTY W ITHOUT 

### I NFORMATI ON 

### A BOUT 

### L OCATIO NS OF 

### S AMPLES AND 

### RECEPTOR S Knowledge of both the sampling locations and the receptor’s activity patterns w ith the EU can be used to derive a more representative estimate of the 9 5% UCL. I f a risk assessor has access to an enviro nmen tal data set wit hout in forma

tion abo ut the s ample

locations, the

risk a ssessor

is force d to assume that the sam

ple con sists of

a num ber of i ndependent ob servat

ions. The vali dity o f this assumption depen

ds on the unkn

own spatial variability

of contamination at the site. The size a nd location of an EU, as well as the choice o f a statistical method for estimating the distribution of uncerta inty a round t he me an conc entration will r equire o ften im

plicit (a nd poss ibly i

ncorrect) assumptions abou

t the spatial distribution of con tamination. Sim

ilarly, if inform

ation regarding receptor activity patterns is unavailable, one m ust assume that an

y area within the EU is equally rep resentative of potenti al expo sures. The risk assesso r is urg ed to ex

plore the effect s of the se var ious ass umpti ons and

to mak e choic es that are prot

ective of hum an heal th and t he env ironm ent. 

### C.5.2 Q UANTIF YING 

### U NCERTAI NTY WITH 

### I NFORMATI ON 

### A BOUT 

### L OCAT IONS OF 

### S AMPLES AND 

### RECEPTOR S In classical statistics, observations are assum ed to be independent. This assumption is often invalid at contam

inated sites where the m

ethod by which

a chemical is relea sed into the environ ment (e.g. , depos ition from airb orne em issions; mig

ration o f contam

inant pl ume f rom a point s ource) r

esults i n positive spatia

l autocorrelation. I

n other words, observ

ations l ocated next to each ot her tend to cont ain similar levels of contamination (i.e. , redundant inform ation) (Griffith and Lay

ne, 1999). For example, the higher the spatial autocorrelation, the less increm ental information is p

rovided by adding observations i

n close proxim

ity toexistin

g obs ervati ons. Th

is decre ase in t

he infor mation conte

nt of a site sample is exace

rbated by the tende

ncy to choose sa

mpling locations in the most contaminated areas rathe r than d istri buted at regul ar spat ial in terval s or spe cifie d using r andom sampl ing method ology. At many hazardous waste sites, environm ental sampling

plans are designed

with remedial actions rather than risk assessm

ent in mind. Therefore, the risk assessor must establish a co rrespondence between the actual sam

pling locations and the locations a receptor wo uld be expected to frequ

ent. Geostatistics may

provide inform

ation to establish this correspondence. Geostatistics is a branch of spatial statistics that can be used to model spatial variability and parameter uncertainty . Geostatistics offers two fundamental contribu tions to risk assessm

ent: (1) a group of methods to d

escribe the spatial distribution of a contaminant in a quantitative fashion, and (2) the ability to max

imize the info

rmation availab le in the data set (Deutsch and Journel, 1988; I saacs and Srivastava, 1 989).
Geostat istics is capabl e of usi

ng the inform ation re vealed by a correlat

ion ana lysis of the d

ata to estimate concentrations at unsampled loca tions. For exam

ple, geostatistics is able to use the spatial inform ation contained

in the d ata to m odel uncertaint

y in c ontam

inant concentra

tions for areas w

hereRAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-13 data are sparse, a situation common ly encountered in s ite assessment work. Using geostatistics, information from

samples collected from outside an EU can be used to model the un

certainty in the m

ean concentration within an E U. Approaches that do not consider the geospatial inform ation present in the data are limit ed to the subset of sam

ples wi thin an EU. H owev er, this ability to mo del unc ertainty

in areas w here da ta are sp arse is a

lso lim ited, an d a wel l characterize d site is still the best p ath to understanding the risk

at that site. Geostatistical methods may be used to calculate a distribution of uncertainty in the mean of the concentration term for use in PRAs. In the past, geostatistics has not be en widely applied to risk assessm ent, ev en thou gh uncertaint

y in the expos

ure concentrati on is of

ten a m ajor sour ce of un certaint y in risk estimates . Most risk assessors quantify uncertainty

in the long-

term average concentration without explicitly con sidering the spatial inform ation present in data obtaine d from env ironmental sampling or knowledge of the receptor’s movement and activities within the E

U. When spatial information does n

ot exist, the inherent assum ption is that environm

ental sampling

yields a data set that is representative of the spatial variability in concentrations encountered b y a receptor. This a ssumption represents one source of u ncertainty in the EP C. In addition, data collected ou tside an EU are often ignored in the analysis, even thoug h they can prov

ide a more com

prehensive view

of patterns of contam ination across the site , inclu

ding t he EU o f intere st. I gnorin g site -wid e inform ation m ay res ult in less informed estim

ates of risk and, therefore, less effective rem edial designs (i.e., too little or too m uch remed iat ion) . In the past five years , wit h rap idl y expa ndin g soft ware and h ardw are capa bil iti es, s ome example s of the applic ation of geost atist ics ca n be foun d in exp osure assess ment and r emedial design (e.g., Gom

ez-Herna ndez, 1996; Goovaerts, 1996, 1997; Kriakidis, 1996; Ginevan and S plitstone, 1997; McKenna, 1997, 1998) a s well as site assessm ent guidance (e.g ., U.S. EPA , 2000).
A limit t o applyi ng geosta tisti cs at ha zardous waste s ites i s that the metho d is res ource intens ive and req uires pe rsonnel experie nced with the softwar

e and te chnique s. Risk asse ssors an d risk mana gers should ensure that contractors and other personnel have the necessary capabilities before apply ing geostatistical meth

ods to risk assessm

ent or site cleanup. G eostatistics is a powerful tool, but it cannot incorporate qua

ntitativ e knowledg e rega

rding all sour

ces of u ncertain ty. The risk

assessor is cau

tioned t o conside r all po ssible s ources of unce rtainty as described in

Chapte r 5. A s indica

ted prev iously , a full discuss ion of geostatistics is b eyond the sco pe of th is gui dance, and int

erested readers are urg ed to consult the OSWER g

uidance documen

t, Guidance on Strategy for Surface Soil Cleanup at Superfund Sites (U.S. EPA, 2001b)

. EPA ha s produ

ced sev eral sof tware p ackages used

for g eostatistical est

imatio n. Am ong these are GEO-EA S and GEO-PACK. Expertise in geostatistics can be obtained from OR D/Las Vegas.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-14 REFERENCES F OR A PPENDIX C Burmaster, D. E. and K.M. Thom pson. 1997. Estim

ating Exposure P oint Concentrations for Su rface Soils for use in Determ

inistic and Probabilistic Risk Assessments. Human Eco. R isk Assess . 3(3): 363-84. Cullen, A.C. an

d H.C. Frey . 1999. Probabilistic Techniques in Exposure Assessment. A Handbook for Dealing with V ariability and U

ncertai nty in Mo dels an d Input

s . Plenum Pre ss. Deutsch, C.V. and Journel, A.G. 1998. Geosta tistical Softwar e Libra ry and User’s Guid e, 2nd Ed., Oxford University Press , NY. Efron, B. and Tibshirani, R.J. 1993. An Introduction to the Bootstrap . Chapman and Hall, CRC Press. Ginevan, M.E. and D.E. Splitstone. 1997. I mproving Remediation D ecisions at Hazardou s Waste Sites with Risk- Based Geostatistical Analysis. Environ. Science Tech

. 31(2):92A–96A.
Gome z-H ernande z, J.J. 1996. Issues on E nviron menta l Risk Assessment. In: Proceedings of the Fifth International Geostatistics Co ngres s, Vol. 1. Baafi, E .Y. and N.A . Schofield, eds.). Kluwer Academic Pub lishers, Dordrecht, 15-26. Goovaerts, P. 1996. Accounting for Local Uncertainty

in Environm

ental Decision-Mak

ing Processes. In: Proceedings of the Fifth International Geostatistics Co ngres s, Vol. 2. Baafi, E.Y and N.A. Schofield, eds.). Kluwer Academic Pub lishers, Dordrecht, 929- 940. Goovaerts, P. 1997. Geostatistics for Natural R esources Evaluation. New York: Oxford Univers ity Press. Griffith, D.A. and L.J. Layne. 199 9. A Casebook for Sp atial Statistical Analysi s. Oxfor d Univer sity Press, NY. Hoff, D .J. 199 8. I

ntegra ted Lab oratory

and Fi eld I

nvesti gation s Asses sing C ontam inant R isk to American Badgers ( Taxidea taxus ) on the Rocky Mountain Arsenal Nationa l Wildlife Refuge. Ph.D. Dissertati on, Clemson University, Clemson, S.C. Hope, B.K. 2000. Generating Proba bilistic Spatially- Explicity I ndividual and Po

pulation Exposure Estimates for Ecolo gical Risk A ssessment. Risk An al. 20(5):575–590. Hope, B.K. 20 01. A

Case S tudy C ompa

ring S tatic and Spati ally E xplicit Ecological Exp osure A nalysi s Methods. Risk An al. 21(6):1001–1010. Isaacs, E.H . and R.M. Sriv astava. 1989. An Intr

oductio n to Ap

plied G eostatis tic s. Oxfo

rd Univ ersity Press, NY. Kriakidis, P.C. 19 96. Selecting Pa nels for Remediatio

n in Contamina

ted Soils via Stochastic Im aging. In: Proceedings of the Fifth International Geostatistics Co ngres s, Vol. 2. Baafi, E.Y. and N.A. Schofield, eds.). Kluwer Academic Pub lishers, Dordrecht, 973- 983.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-15 Land, C.E. 19 71. Confidence I ntervals for Linear Functions of the Norm al Mean and Variance. Ann. Math. Stat . 42:1197–1205. Land, C.E. 19 75. Tables of Confidence Limits for Linear F unctions of the Norm

al Mean and Variance. In: Selected Tables in Mathema tical Statistics , Vol. 3 . America n Math ematica l Socie ty, Provi denc e, RI. McKenna, S.A. 1997. Geostatistical Analysis of P u-238 Contamination in Release Block D, Mound Plant, Miamisburg, O

hi o. SAN D97- 0270, Sandia Nationa l Labor atories, Albuqu

erque, NM. McKenna, S.A. 1998. Geostat istical A pproach for Mana

ging Uncert ainty in Env

ironmental Remediation of C ontaminated Soils: C ase Study. Environmental and E ngineering Geoscienc e, 4(2):175-184. Meade R.H., T.R. Yuzyk, and T.J. Day. 1 990. Movement and Storage o f Sediment in Rivers of the United States and Canada . In : Wolman et al . (eds) Surface Water H ydrolo gy. The Geo logy

of Nor th America. Geological Socie ty of America, Boulder, CO. Moore, D.S. and G.P. McCabe. 19 93. Introduction to the Practice of Statistics . W.H. Freem an and Company, NY. Piest R.F. and C .R. Miller. 1975. Sediment Yields and Sedime nt Sources . In : Vanoni

V.A. (e d.) Sedimentation Engineer ing, American Society of Civil Engineer s, NY. Schulz, T.W. and S. Griffin. 1999. Estimating R isk Assessm

ent Exposure Point C oncentrations when the Data ar e not N ormal

or Log norma l. Risk Anal . 19(4): 577– 584.
U.S. EPA. 1 992. Supplemental Guidance to RAGS: Calcula ting the Concentration T erm . Offic e of Solid Waste and Em

ergency Res

ponse. Washington, DC. OWSER Directive No. 92 85.7-081.
U.S. EPA. 1 997a. Lognormal Distribution in Environmental App lications . Office of Research a nd Development and Office of Solid Waste and Emergency Response. Washington, DC

. EPA/600/R- 97/006.
U.S. EPA. 1 997b. Memorandum from Deputy Administrator Fred Hansen on the U

se of Probabilistic Techniques (including

Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte C arlo An alysis . Office of Research a nd Developm

ent, Washington, D C. EPA/630/R- 97/001. May 15.
U.S. EPA. 2 000. Statistical Estimation and V isualization of Ground-w ater Contamination D

at a. Office of Research and Dev

elopment, Washing

ton, DC. EPA

/600/R-00/034

. U.S. EPA. 2 001a. Draft Guidance on C alculation of Upper C

onfidence Limits for Exp osure Point Concentrations at Super

fund Sites . Office of Em

ergency and Remedial Respo nse, Washington, DC.RAGS Volume 3 Part A ~ Process for Conducti ng Probabilisti c Risk Assessment App end ix C ~ Decem

ber 31, 2001

Page C-16 U.S. EPA. 2 001b. Supplemental Guidance for Developing Soil S

creening Levels for Supe

rfund Sites

. Peer review draft. Office of Solid Waste and Emergency Response. Washing

ton, DC. OSWE

R No. 9355.4- 24. March. Van Sick el J. and R.L. Beschta . 1983 . Supp ly- Based M odels o f Suspe nded Se dimen

t Transport in Streams. Water Resour. Res. 19:768–78. Walling D.E. 19 83. The Sediment Delivery Prob lem. J. Hydrol . 65:209–37.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

1 Note that the term “credible interval” may be more appropriate than “confidence interval” given that the range is based on subjective as well as statistical considerations. Brattin, Barry, and Chiu (1996) provide additional examples of uncertain PDFs that illustrate this concept. Page D-1 

## A PPENDIX 

## D
A DVANCED 

## M ODELING 

## A PPROACHES FOR 

## C HARACTERIZING 

## V ARIABILITY AND 

## U NCERTAINTY 

### D.0 I NTRODUCTION This appendix briefly describes the following advanced modeling approaches that can be used in probabilistic risk assessment (PRA) to characterize variability and uncertainty: two-dimensional MCA (2-D MCA), microexposure event analysis (MEE), geospatial statistics, and Bayesian analysis. Except for 2-D MCA, these approaches can also be applied to point estimate risk assessment. The application of many of these approaches will require access to expertise in specialized areas of statistics and, in some cases, specialized or even custom-designed computer software. The intent here is to introduce some of the basic concepts and terminology, as well as to provide references where the reader can find more exhaustive coverage of these topics. 

### D.1.0 E XPRESSING 

### V ARIABILITY AND 

### U NCERTAINTY 

### S IMULTANEOUSLY A Monte Carlo analysis that characterizes either uncertainty or variability in each input variable (see Chapter 1) can be described as a one-dimensional Monte Carlo analysis (1-D MCA). A 2-D MCA is a term used to describe a model that simulates both uncertainty and variability in one or more input variables. All probability distributions that are used to describe variability in a PRA model have a certain degree of associated uncertainty. For example, suppose variability in soil concentration (ppm) is estimated using a normal probability density function (PDF) defined by a mean (μ soil =5) and standard deviation ( σ soil =1), and subjectively truncated (min, max) at (0, 50). Uncertainty in the parameter estimates can be represented in a PRA model by assuming both parameters are also random variables. To illustrate this concept, assume normal PDFs for uncertainty can be specified for both parameters. Uncertainty in the mean is described by the normal PDF with parameters (μ mean =5, σ mean =0.5); similarly, uncertainty in the standard deviation is described by the normal PDF with parameters (μ SD =1, σ SD =0.5). Model variables are represented in this manner when there is a compelling reason to believe that a unique probability distribution does not adequately describe one’s knowledge of each variable in the model. A variable described in this way is called a second order random variable. Figure D-1 (Panel A) shows a collection of n =20 cumulative probability distributions (CDFs), each curve representing a unique set of (mean, SD) parameter estimates for the normal PDF for variability. Panel B shows the 90% confidence interval 1 based on 2,500 simulated CDFs. The 95% lower and upper bounds correspond to the distribution of 5 thpercentiles and 95th percentiles, respectively (i.e., CDF for 2,500 5 thpercentiles and CDF for 2,500 95 thpercentiles). The 90% credible interval (CI) for the 50 thpercentile is (3.4, 6.7).RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-2 

### Panel A

Panel B 0.00 0.20 0.40 0.60 0.80 1.00

0 2 4 6 8 10 12Cumulative Probabilit Y ~ Normal(mean, stdev) - mean ~ Normal(5, 0.5) - stdev ~ Normal(1, 0.5)

CDF 1 N~(4.0, 1.3)

CDF 2 N~(5.4, 0.3)

0.00 0.20 0.40 0.60 0.80 1.00

0 2 4 6 8 10 12Cumulative Probability Lower 95% bound
Upper 95% bound Median CDF

90% CI for median (3.4, 6.7)

Figure D-1. Panel A shows a family of 20 CDFs for a hypothetical random variable, Y (e.g., concentration in units of ppm), characterized by a normal PDF where both the mean and SD are also random variables representing uncertainty in the parameter estimates: Mean~ Normal(5, 0.5), SD~ Normal(1, 0.5). Each CDF represents a single simulation of n =2500 iterations using a unique set of parameters. For example, CDF 1 represents N~(4.0, 1.3) while CDF 2 represents N~(5.4, 0.3). Panel B shows the “90% credible interval” for the CDF based on 2,500 simulations , each simulation using n = 2500 iterations (i.e., a 2-D MCA with 2,500 outer loop iterations and 2,500 inner loop iterations). Lower, median, and upper bounds represent the simulated 5 th , 50 th , and 95 th percentiles, respectively. The 90% confidence interval for the estimate of the 50 th percentile is: {3.4, 6.7}.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-3 E XHIBIT D-1
D EFINITIONS FOR A PPENDIX D Bayesian Statistics - A specialized branch of statistics that views the probability of an event occurring as the degree of belief or confidence in that occurrence. Geospatial Statistics - A specialized branch of statistics that explicitly takes into acc ount the georeferenced context of data and the information (i.e., attributes) it contains. Frequentist- A term referring to classical statistics in which the probability of an event occurring is defined as the frequency of occurrence measured in an observed series of repeated trials. Image Analysis - A technique in geostatistics used to restore a degraded image or interpret images that have been contaminated by noise or possibly some nonlinear transformation. Kriging - A geostatistical method of spatial statistics for predicting values at unobserved locations. Likelihood Function - A Bayesian term referring to a probability distribution expressing the probability of observing a piece of new information given that a particular prior belief is true. Location Tag - The spatial coordinates of a sampling location (e.g., longitude, latitude). Microexposure Event Analysis (MEE) - An approach to modeling exposure in which long-term exposure of an individual is simulated as the sum of separate short-term exposure events. Point Pattern Analysis - A technique in geostatistics of restricting the analysis to location information, ignoring attribute information, addresses two location problems: (1) describing points according to spacing, and (2) describing points according to density. Posterior Distribution - A Bayesian term referring to a probability distribution that has been updated with new information. Prior Distribution - A Bayesian term referring to the hypothesized, expected, or calculated probability distribution for an event prior to the collection of new information. Spatial Autocorrelation - The tendency of data from locations that are relatively close together to be geographically correlated. Thiessen (Voronoi) Polygon Analysis - A method of spatial statistics in which an area is s ubdivided into subregions, or polygons, in order to predict values at unobserved locations. Time Step - A modeling term used to describe the time interval within which variable values do not change. Two-Dimensional Monte Carlo analysis (2-D MCA) - Separate representation of variability and uncertainty in an MCA, usually accomplished using nested computation l oops.

In the example shown in Figure D-1, the mean and standard deviation for soil concentration were allowed to vary independently. Thus, a distribution could be defined by a combination of a low mean and a high standard deviation, high mean and low standard deviation, or any other combination in between. The assumption of independence of variable parameters may not be valid in all cases. It may be unreasonable to assume that a high mean soil concentration would occur with a low standard deviation. An alternative assumption would be that the standard deviation of the mean is a constant proportion of the mean (i.e., a constant coefficient of variation). Correlations between parameters should be considered in the design of the PRA. One approach that is especially useful for characterizing relationships between the slope and intercept of a simple linear regression is to specify the bivariate normal distribution for the parameter estimates.
D.2.0 T WO -D IMENSIONAL M ONTE C ARLO A NALYSIS (2-D MCA) Two-dimensional MCA is an approach for computing risk (or hazard) when combining distributions that represent variability and uncertainty. In 2-D MCA, distributions representing variability and uncertainty are sampled using nested computational loops (Figure D-2). The inner loop simulates variability by repeatedly sampling values for each variable from their defined probability distributions. With each circuit of the outer loop, new parameter values for each variable are selected, and the inner loop sampling is repeated. The result is a collection of inner loop simulations, one for each parameter value selected. If the inner loop samples 5,000 times, and the outer loop samples 1,000 times,RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-4 Input PDFs for exposure variables Input PDFs for paramaters

of exposure variable PDFs Select number of 1-D iterations

Select number of 2-D iterations

Save intake, exposure

variable parameter values to file

1-D iterations

complete? Simulate N months Simulate N months of exposure and intake of exposure and intake

2-D iterations

complete? Select parameter values for exposure variable PDFs

Yes No No

Yes 

# EndOuter loop (Uncertainty)Inner loop (Variability) 

## Simulation Logic for 2-Dimensional MCA

Figure D-2. Diagram showing of a 2-D Monte Carlo model in which the variability and uncertainty dimensions are computed in nested loops. In this example, values for exposure variables in the inner loop represent monthly averages. then each variable is sampled 5,000,000 times and 1,000 simulated probability distributions of risk are generated from the PRA model. These probability distributions can be analyzed to estimate the distributions for specific risk estimates. For example, confidence limits on the estimate of specific risk percentiles can be simulated using 2-D MCA (Figure D-3).RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-5 

### 0 5 10 15 20AM5th %ile25th %ile50th %ile75th %ile90th %ile95th %ile



### Variability StatisticsHazard Quotient 95th %ile 75th %ile 50th %ile 25th %ile 5th %ile

Figure D-3. Output from a 2-D MCA showing the estimated mean Hazard Quotient (HQ) and the 90% confidence interval for the arithmetic mean (AM) and selected percentiles of the HQ distribution. The 95 th %ile HQ would be the reasonable maximum exposure (RME) risk estimate. The simulation suggests that there is a 95% probability that the RME HQ (95 th percentile) is below 16.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-6 DOSE ! C x IR x EF x ED BW x AT Equation D-1

DOSE ! 1 AT ! ED j ! 1 1 BWj ! Events j
i ! 1 C ij ! IR ij Equation D-2 

### Standard Time-Averaging

Microexposure Event Modeling

C = Concentration; I = exposure event; j = year of life IR = Intake Rate EF = Exposure Frequency ED = Exposure Duration BW = Body Weight AT = Averaging Time 

### D.3.0 M ICROEXPOSURE 

### E VENT 

### A NALYSIS The standard dose equation generally used in Superfund site risk assessments represents exposures averaged over a specified time period that is relevant to the health endpoint of concern (Equation D-1). If the riskassessment is directed at assessing lifetime risk to humans, the averaging time used in Equation D-1 would generally be 70 years (i.e., estimated average human lifetime), and the calculated chemicalintake would generally represent the lifetime average daily dose (LADD). Where information is available to characterize variability on a smaller time scale than life-time, an alternative expression of dose that accommodates such variability may be desirable.
Concentrations in various environmental media can be expected to vary over time. For example, wind erosion may change chemical concentrations in surface soil. Leaching may change concentrations in both subsurface soil and groundwater. The change in the concentration term is most readily apparent when considering anglers harvesting fish. If an angler consumes a large amount of fish from a single location (e.g., a specific lake, pond, or river), then the average chemical concentration in the fish consumed by that angler can be expected to be similar to the average of the chemical concentration of fish in the population. However, if an angler consumes fish only occasionally, or harvests fish from different locations, there will be considerably more uncertainty in the concentration term. In addition, a harvesting angler may consume varying amounts of fish over the period of the exposure duration due to changing tastes, changes in the fish population size or other factors. Daily activity patterns, food intake, soil ingestion and other behavioral factors are measured in a time period of less than a year. The extrapolation of these short term results to the chronic exposure situation is a source of uncertainty. Exposure events are real but unknowable, whereas data regarding the nature and magnitude of these events is known but its application to a real world situation is uncertain. Microexposure event analysis (MEE) attempts to explicitly quantify this uncertainty. Figure D-5 presents the general approach for MEE analysis. (Price et al., 1996, 2000). MEE modeling provides an alternative to the standard time-averaging approach represented by Equation D-1. In the MEE approach, long term intake is viewed as the sum of individual exposure events (Equation D-2). Implementing the MEE approach in a PRA requires dividing the exposure duration into short epochs, or time steps, within which the values assigned to exposure variables remain constant, but are allowed to vary from one time step to the next. In a PRA model, exposure variables are adjusted at each time step by selecting values from the probability distributions representing each variable (Figure D-4). Discussion of theRAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-7 x 2
Annual Time Step n = 1 Monthly Time Step n = 12 Quarterly Time Step n = 4 x

1 x

1 x 3 x 4 x 5 x 6 x 2 3 mo. 6 mo. 9 mo. 12 mo. Begin

x 1 

### X X
n Avg Time t t n

. 

### = = 

### ∑

1 Arithmetic mean intake for averaging time = Time Step Issues: How many random values? What averaging time for intake?

Figure D-4. Time Step for MEE. implementation of MEE analysis in risk assessment and its merits and limits can be found in Wallace et al. (1994), Price et al. (1996), Slob (1996), and Buck et al. (1997). In MEE modeling, the time step becomes an important variable, with associated uncertainty. The time step should be selected based on information available to describe how exposures change over time. For example, a model of a moving plume of solvents in groundwater might suggest that chemical concentrations in a given location are dropping by between 16 and 25% quarterly. Several rounds of sampling may support this prediction. This rapid decline in concentrations suggests that an appropriate time step might be one quarter (i.e., three months). On the other hand, where risk is being assessed for metals, dioxin, or PAHs in soil, the concentrations might be expected to change much more slowly, if at all, and the basis of the time step might be the increase in age and corresponding changes in behavior of the receptor. The time step may be global; that is, one time step may apply to all variables in the model. In this case, the same number of random values would be selected for each exposure variable in a Monte Carlo simulation. A more complex model may use different time steps for different variables, requiring some probability distributions to be sampled more often than others. The selection of a value for a time step implies that the value represents the average value for that variable during the time step.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-8 Figure D-5. Flowchart showing general approach for Microexposure Event (MEE) analysis.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-9 

### 0.00 0.10 0.20 0.30

0 20 40 60 80 100 120 140 160 180 200 220 

### Soil and Dust Ingestion Rate (mg/day) Annual Quarterly Monthly Arithmetic MeanProbability

Figure D-6. Hypothetical example showing the effect of model time step on the probability distribution for soil and dust ingestion rate in children over a 1-year period. Number of samples ( n ) needed to simulate exposures: Annual (1), Quarterly (4), Monthly (12). Two important issues related to time step should be considered in implementing the MEE approach in PRA models. The first is the relationship between the length of the time step and the number of times random values are generated from a defined probability distribution. As the time step decreases, more time steps are needed to simulate exposures over a specified duration. For example, given a time step of one year and an exposure duration of 30 years, each random variable will be sampled 30 times (once per year); for a time step of one month and an exposure duration of 30 years, each random variable would be sampled 360 times (i.e., 12 months/year x 30 years). The Central Limit Theorem indicates that as n increases, the distribution of sample means is approximately normal, and the standard deviation of the sample distribution is inversely proportional to the square root of n . Thus a highly skewed input distribution (e.g., lognormal) may tend to become less skewed with increasing n (Figure D-6). A biased estimate of the RME risk in a PRA model may result if an inappropriately small or large time step is used in the model. This emphasizes the importance of having an empirical basis for selecting the time step and of exploring the time step as a variable in a sensitivity analysis of the model. The second issue related to the time step concerns temporal correlations. Is it reasonable to assume that random values selected for consecutive time steps are completely independent? For example, consider body weight. The body weights of an individual measured at different times would be expected to show positive temporal autocorrelation; that is, body weight is likely to be similar (but not constant) from one time step to the next. For example, if an individual weighs 60 kg during one month, it is unlikely that they will weigh 80 kg the next month. If this scenario is accepted, then body weight should not be allowed to vary independently from one monthly time step to the next in the model. At shorter time steps, temporal correlation becomes more likely as a result of temporal autocorrelation. For example, one can expect a higher correlation between body weights on an individual measured on two successive days (one-day time step) than between weights measured at the midpoint of two successive years. Approaches to simulating temporal correlations in probabilistic models might include fixing an individual within a percentile range of a distribution (e.g., randomly assigned quartile) or using randomly assigned fluctuations (e.g., BW t = BW t-1 ± x ).RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-10 E XHIBIT D-2
P OSITIVE S PATIAL A UTOCORRELATION • Locations with a high value of Y tend to be surrounded by nearby high values of Y. • Locations with a medium value of Y tend to be surrounded by nearby medium values of Y. • Locations with a low value of Y tend to be surrounded by nearby low values of Y.

E XHIBIT D-3
E XAMPLES OF R ISK A SSESSMENT I SSUES

L INKED TO G EOSPATIAL S TATISTICS • Sampling tends to disproportionately represent “hot spots” (i.e., a relatively large portion of a data set with a small sample size (n) tends to be concentrated at “hot spots”). • The upper confidence limit (UCL) for the arithmetic mean exposure concentration (e.g., chemical concentrations in soil) depends on the sample size. • Additional sampling may be needed, especially to better define the spatial patterns or the extent of contamination. • There is uncertainty about locations not sampled at a site, as well as uncertainty regarding the representativeness of neighboring samples in nearby EUs. 

### D.4.0 G EOSPATIAL 

### S TATISTICS Spatial statistics is a specialized branch of statistics, falling under the heading of multivariate statistics, that explicitly takes into account the georeferenced or locational tagged context of data. Generally, environmental samples collected at Superfund sites have this geolocational information By acknowledging the geography of site chemicals, information about the spatial distribution of contamination can be incorporated into an exposure assessment. In addition, knowledge about a receptors home range or patterns of movement may also be incorporated into the definition of the exposure unit (see Appendix C, Section C.2.0). Explicitly accounting for spatial relationships may lead to a more accurate estimate of the confidence limits for the arithmetic mean concentration. Geospatial statistics quantifies the spatial autocorrelation (Exhibit D-2) of sample measurements and allows for the exploration of the spatial distribution of exposure and risk using techniques of map generalization. By recording locational tags for each sample, information about spatial patterns within an exposure unit (EU) can be exploited to estimate both pre- and post-remediation exposure and risk. In the past five years, with rapidly expanding software and hardware capabilities, some examples of the application of geostatistics can be found in exposure assessment and remedial design (e.g., Gomez-Hernandez, 1996; Goovaerts, 1996, 1997; Kriakidis, 1996; Ginevan and Splitstone, 1997; McKenna, 1998; Hope, 2000; 2001) as well as site assessment guidance (e.g., U.S. EPA, 2000). Several important risk assessment issues are closely linked to geospatial statistics, as described in Exhibit D-3. Geospatial statistics comprises: • spatial autoregression • geostatistics • point pattern analysis • image analysis The first three of these subjects can contribute to spatial statistical support of site risk assessments. The key concept linking all three is spatial autocorrelation, which refers to covariation among samples for a single chemical, or the tendency of data from locations that are relatively close together to be geographically correlated. By analogy, classical statistics treats soil samples as though they are balls, each having a battery of attributes, that can be placed into anRAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-11 urn for statistical analysis; geospatial statistics treats soil samples as though they are clusters of grapes, with the branchy stems representing locational tags. Concentrations located on the same “branch” will be more strongly correlated than concentrations on different branches. How is Geostatistics Different from Classical Statistics? In general, geostatistics provides information beyond that provided by classical statistical techniques for at least two reasons. First, in classical statistics, observations are assumed to be independent. This assumption is often invalid at contaminated sites where the method by which a chemical is released into the environment (e.g., deposition form airborne emissions; migration of contaminant plume from a point source) often results in positive spatial autocorrelation (see Section D.4.1). In other words, observations located next to each other tend to contain similar levels of contamination (i.e., redundant information). For example, the higher the spatial autocorrelation, the less incremental information is provided by adding observations in close proximity to existing observations. This issue is compounded when the sample locations have been preferentially determined (e.g., “hot spot” sampling) rather than distributed at regular intervals or specified using random sampling methodology. Second, geostatistics is able to use the geospatial information contained in the data to model uncertainty in contaminant concentrations for areas where data are scarce, a situation commonly encountered in site assessment work. Using geostatistics, information from samples collected from outside an EU can be used to model the uncertainty in the mean concentration within an EU. Approaches that do not consider the geospatial information present in the data are limited to the subset of samples within an EU. D.4.1 C ORRELATION AND S PATIAL A UTOCORRELATION Several simple bivariate statistical approaches may be used to introduce the concept of spatial autocorrelation. Consider two variables, X and Y. For positive correlation there is a tendency for high values of X to be paired with the high values of Y, medium values of X to be with the medium values of Y, and low values of X with the low values of Y. The tendency is in the opposite direction for negative correlation; high values of X tend to be paired with low values of Y, and so on. Spatial autocorrelation, which virtually always is positive, directly parallels these definitions, but is written in terms of a single variable as shown in Exhibit D-2. Just as the bivariate relationship between two variables, X and Y, can be portrayed by a scatter plot (Y versus X), the spatial autocorrelation relationship can be portrayed for a single variable, Y, (e.g., Y versus Y). A good example is the Moran scatterplot, which plots the sum or average of nearby values of Y versus Y. This plot is most effective when Y has been converted to z-scores. As shown in Figure D-7 and Section D.4.2, scatter plots can be used to illustrate some important issues related to sample size.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-12 

### 0 5 10

0 5 10 

### XY 

### outlier (n = 1)

cluster (n = 99)

Figure D-7. Effect of an outlier on measured correlation: r=0.956 with outlier ( n =100), whereas r=0.086 excluding outlier ( n =99 clustered points).
If no soil samples were collected at a site ( n =0), there is no information about the chemical concentrations in soil, and any guess may be considered an estimate. However, if the chemical concentration of a single sample ( n =1) is measured, some information is obtained that partly restricts this estimate. As each additional independent sample is taken, more information is obtained, and the restriction on the estimate becomes more binding. If the same location is selected repeatedly for sampling, then the repeated measures, which may vary through time, will tend to be highly positively correlated; part of the information obtained from each sample is the same, and should not be counted more than once in estimating the site-wide soil concentration. Similarly, if immediately adjacent locations are sampled, the measures will often tend to be highly positively correlated (spatial autocorrelation). Once the first sample is taken, each additional sample provides only a fractional increment of new information about the site in its entirety.
D.4.2 E FFECTIVE S AMPLE S IZE ( N *) AND D EGREES OF F REEDOM Repeated measures can result in data clustering, which can be illustrated in a scatter diagram. Because two points determine a straight line, if ( n–1 ) points cluster together on a scatter diagram while a single additional point occurs far away from this cluster (i.e., an outlier), then the resulting bivariate correlation will be very high (see Figure D-7). This situation alludes to the notion of effective sample size ( n* ): the n* is no longer equal to the number of observations ( n ), but rather is dramatically reduced by the presence of inter-observational correlation. For the example shown in Figure D-7, n* is slightly greater than 2 rather than 100 (i.e., n ).RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-13 E XHIBIT D-4
E FFECT OF S PATIAL A UTOCORRELATION

(r) ON E FFECTIVE S AMPLE S IZE ( n* )

r n* 0.000 625 0.050 514 0.539 64 0.957 3 1.000 1 Spatial autocorrelation plays an analogous role in georeferenced data. If a sampling network is arranged as a 25-by-25 square grid (one sample point per grid cell), and superimposed over a large site so that a very large distance separates nearby sample locations, then essentially zero spatial autocorrelation should be present in the geographic distribution of the concentrations of any given chemical. Concentrations will appear to be haphazard across the site, rendering the effective sample size as n* =625. If the distance between nearby locations on the sampling mesh is decreased so that the spatial correlation is only r=0.050, then the effective sample size decreases to n* =514. The effect of reducing the intersample distance on spatial autocorrelation and n* for a 25-by-25 grid is shown in Exhibit D-4. If r increases to 1, then n* reduces to 1. Therefore, obtaining a measure of latent spatial autocorrelation is essential to estimating n* ; this in turn is critical to determining confidence limits for estimates of mean concentrations, which are sensitive to sample size. The UCL for the mean will be biased only when very high levels of spatial autocorrelation are present; this is because the Student-t statistic used to estimate the UCL (assuming a normal distribution) changes very little as the degrees of freedom (related to sample size) increases above 10; part of the difference between n and n* is offset by an inflation of the variance. The concept of effective degrees of freedom is important in exposure assessment because high positive spatial autocorrelation can bias the estimate of the UCL concentration if geospatial statistics are not considered. This should be of particular concern when specific locations at a site are intensively sampled (e.g., suspected “hot spots”), and other locations are relatively undersampled. Accordingly, the design of the sampling network itself can be evaluated from the perspective of geospatial statistics in order to ascertain the quality of sample information. The ideal sampling network should provide geographic representativeness, should be roughly uniformly distributed over a site, and is best implemented as a stratified random sampling design; that is, the site is partitioned into geographic stratum (e.g., EUs), and then a random sampling of points is selected within each strata. In practice, sample designs may need to focus on objectives that are in conflict with the above ideals. For example, intense sampling of suspected “hotspots” may be necessary at some sites, at the expense of a more representative spatial coverage of the site. In such cases, several statistical techniques are available for assessing the statistical benefit (in terms of reducing uncertainty) of additional sampling at undersampled locations. D.4.3 A SSESSMENT OF A DDITIONAL S ITE S AMPLING Thiessen Polygons . In addition to calculating nearest neighbor statistics, the adequacy of a sampling network can be assessed by Voronoi (i.e., Thiessen polygon) surface partitioning, a popular approach used in mapping intra-site geographic distributions. This procedure divides a site into a mutually exclusive set of polygons, each polygon containing a single measured concentration. Each polygon has the unique property that any location within the polygon is closer to the polygon’s sample location than to any other sample point (Clifford et al., 1995). The concentration measured at the sampleRAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-14 

### SI ! 1
m ! m
i ! 1 

### A " 1 i Equation D-3



### SI Variance 

### ! 1
m

" 1 ! m
i ! 1 

### A " 2 i 

### " 1
m ! m
i ! 1 

### A " 1 i 2 Equation D-4



### C soil,w 

### ! ! n
i ! 1 

### C i 

### A i 

### A T Equation D-5 point in the polygon is assigned to the entire area of the polygon. The intensity of sample points on a surface can be measured by Equation D-3 mean inverse polygon areas:

where SI is a measure of the sampling intensity, A iis the area of the i th polygon, and m is the number of interior polygons (those not along the edge of the site); m < n . The variance of the sampling intensity can be expressed by Equation D-4:

If the sampling network is uniform (i.e., polygon areas are equal), the variance will be essentially zero. The variance will increase as the network deviates from uniform. This measure can be used to assess whether or not additional samples will improve the spatial coverage.

! Sampling locations that would yield a dramatic reduction in the variance should be given priority for future sampling efforts. Thiessen polygons can be used to develop area-weighted estimates of the arithmetic mean concentration (C soil,w ) according to the following general equation:

where Ciis the concentration in the i th polygon, A iis the area of the i th polygon in the EU, and A T is the total area of the EU. The weight for each measurement is essentially the ratio of the area of each polygon to the total area of the site. Clifford et al. (1995) applied this approach to an ecological risk assessment of the burrowing owl with the following simplifying assumptions: habitat range is circular, size of EU is constant (75 ha) although location may vary, and organisms spend equal time in all portions of their habitat. Given these assumptions, a nonparametric bootstrap method can be used to determine the approximate 95% UCL for the mean concentration (see Appendix C). Using Monte Carlo analysis, C soil,w can be estimated for different locations of the EU according to Equation D-5, and confidence limits can be generated from the multiple bootstrap estimates. Burmaster and Thompson (1997) demonstrate a similar approach in which the EU (with constant area but random rectangular dimensions) is overlayed on the Theissen polygon surface and 95% UCL for the mean is calculated from the bootstrap sample. Linear Regression. Another diagnostic is found in the linear regression literature. The locational tag coordinates (e.g., longitude, latitude) can be converted to z-scores (say z u and z v ) for the following calculation:RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-15 

### Y ! 1
n # z 2 u 

### # z 2 v 

### " 2ru,v 

### z u 

### z v 

### (n

" 1)(1

" r 2 u,v 

### ) Equation D-6

where Y is a measure of the sampling network, r uvis the correlation between the coordinate axes, and n is the number of samples. Any sampling location (z u , z v ) in which Y > 9/ n may be considered too isolated in the sampling network. Additional sampling locations would be positioned closer to it to improve the overall coverage of the sampling network.
D.4.4 M AP G ENERALIZATION Another important application of geospatial statistics to risk assessment is that of map generalization, which draws on the subjects of geostatistics and spatial autoregression. Techniques developed for both topics exploit spatial autocorrelation in order to produce a map. Kriging and Semivariograms. Geostatistics may employ kriging, which yields statistical guesses at values of a chemical at unsampled locations based on information obtained from sampled locations. Kriging assumes that the underlying geographic distribution is continuous, evaluates spatial autocorrelation in terms of distance separating sample points, and employs a scatter diagram similar to the Moran scatter plot to portray this relationship (i.e., the semivariogram plot: half the squared difference between measured concentrations for two sampled locations versus distance separating these two locations). The best-fit line to this scatter of points is described by one of about a dozen equations (semivariogram models). Many different kriging approaches can be applied to quantify the spatial relationships among geographic attributes within an exposure unit. For example, site-specific chemical concentrations may be correlated with geologic information, such as glacial deposits, soil characteristics of core samples, and attributes that represent favorable habitats for ecological receptors. This information can be used to expand the available data and improve estimates of chemical concentrations at unsampled locations by employing a technique called co-kriging. Thiessen Polygons and Spatial Autoregression. Spatial autoregression assumes a discretized surface, uses the Thiessen polygon surface partitioning to construct a Moran scatter plot, and can be used to estimate values at selected points with a regression-type equation. Theoretically, the exponential semivariogram model relates to the conditional autoregressive model, and the Bessel function semivariogram model relates to the simultaneous autoregressive model; in practice, though, the spherical semivariogram model often provides the best description of a semivariogram plot. Regardless of which approach is taken to map generalization, one relevant contribution of these two subjects is the following observation:

! Including positive spatial autocorrelation results in more accurate variance estimates; this in turn yields more accurate estimates of the 95% UCL for the mean concentration.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-16 D.4.5 I MPLEMENTATION I SSUES R ELATED TO G EOREFERENCED D ATA Estimation of parameters, for either geostatistical or spatial autoregressive models, cannot be achieved with ordinary least squares (OLS) techniques; nonlinear least squares must be used. While OLS provides unbiased regression coefficients, these estimates are not necessarily sufficient (i.e., they do not summarize all of the information in a sample pertaining to the population), efficient (i.e., the standard errors often are incorrect), and consistent (i.e., the asymptotic sampling distribution concentration will not be at the parameter value). In other words, OLS essentially uses the wrong degrees of freedom in its calculations, as described in Section D.4.2. Two additional complications of georeferenced data that do not appear in other types of data are (1) spatial autocorrelation might be directional (i.e., directional dependency); and (2) variance might be nonconstant over space as well as over the magnitude of the dependent variable, Y (e.g., chemical concentration). Several statistical approaches, which are beyond the scope of this guidance, are available for analyzing these potential sources of bias in the exposure concentration estimates (Isaaks and Srivastava, 1989; Cressie, 1991; Griffith, 1993; Ginevan and Splitstone, 1997). 

### D.5.0 E XPERT 

### J UDGMENT AND 

### B AYESIAN 

### A NALYSIS Up to this point in RAGS Volume 3: Part A, risk has been characterized as having a population probability distribution with parameters (e.g., mean, standard deviation) that can, theoretically, be estimated from observation. In theory, risk estimates could be derived by repeatedly measuring risk in subsets of the population of interest (e.g., repeated measurements of site-related cancer risk). The unstated expectation, or goal, is that the PRA model will accurately simulate this realrisk distribution. This approach derives from a classicalview of probability. The classicalor frequentist view defines the probability of an event as the frequency with which it occurs in a long sequence of similar trials. From the frequentist perspective, the probability of having a flipped coin land heads-up is given by thefrequency distribution of heads-up results derived from repeated similar trials of coin flips. For realworld decisions such as those informed by Superfund risk assessments, there is uncertainty that the sample data are representative of the population (see Chapter 1, Section 1.2.4). Bayesian View of Probability . A Bayesian perspective on probability allows distributions to be constructed based on the judgment of an expert in the field. The subjectivist or Bayesian view is that the probability of an event occurring is the degree of belief a person has in the occurrence. Probabilities can be assessed by experts using scientific knowledge, judgment, data, past experience, and intuition. Different people may assign different probabilities to an event, and a single individual may assign different probabilities to the same event when considered at different times. The consequence is that probabilities become conditional and the conditions must be explicitly stated (Howson and Urbach, 1989; Morgan and Henrion, 1990; Ott, 1995; Sivia, 1996). These conditional probabilities can, of course, be updated with new information. Using the coin flip analogy above, a Bayesian perspective might be that, based on experience with coins, assuming that most coins are fair , and that a fair coin would be expected to land heads-up half the time, the expected probability of the tossed coin landing heads-up is 0.5. If the outcome of repeated trials was different from the expected, the Bayesian approach would be to update the probability based on the new data. In the coin flip example, both the Bayesian and frequentist approaches will arrive at the same conclusions, because the outcome is amenable to rigorous experimentation. Where the two approaches can be expected to differ is in the assignment of probabilities to events that cannot be rigorously measured; for example, the probability of a site-related cancer risk, or the probability of a child ingesting a specific amount of soil.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-17 E XHIBIT D-5 C OMPONENTS OF B AYES T HEOREM IN PRA • Input probability distributions for exposure (or toxicity) based on available data or expert judgment • Prior probability distribution for risk based on input probability distributions (output from PRA) • New data • Likelihood function, expressing the probability of observing the new data conditional on prior risk estimates • Posterior (updated) probability distribution for risk The subjective judgment of experts is, therefore, an important tool in the Bayesian approach to risk assessment. For example, the input distributions for a PRA may be based upon the judgment of one or more experts who rely upon estimates from the literature, data from experimental studies, and any other information they consider relevant. Even when formal elicitations of expert opinion are not done, the final selection of the form and parameters of the input distributions usually involves some subjective judgment by the analyst. One of the challenges of incorporating judgments from experts or lay people is that there can be overconfidence bias (i.e., people tend to underestimate their uncertainty). There is a rich literature about the protocol for conducting expert elicitations and using the results to support decisions (Lichtenstein and Fischoff, 1977; Morgan and Henrion, 1990; Shlyakhter and Kammen, 1992). Elicitation of expert judgment has been used to obtain distributions for use in risk assessments (Morgan and Henrion, 1990; Hora, 1992; U.S. EPA, 1997;) and in developing air quality standards (U.S. EPA, 1982). In addition to providing input distributions for PRAs, Bayesian analysis allows the current state of knowledge, expressed as a probability distribution, to be formally combined with new data to reach an updated information state. The distribution expressing the current knowledge is the prior distribution and may be the output of a PRA (Figure D-8). An appropriate likelihood function for the data must also be formulated. The likelihood function is based upon an understanding of the data gathering process and is used to determine the probability of observing a new set of data given that a particular risk estimate is true. Once the prior distribution is determined, the new data values are collected, and the likelihood function is assumed, Bayes theorem (Exhibit D-5) provides a systematic procedure for updating the probabilistic assessment of risk. The updated information state is called the posterior distribution and reflects the reduction in uncertainty arising from the new information.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-18 

### Bayes Theorem $ 

### : P(R i 

### /D) ! P(D/R i 

### ) P(R i 

### ) ! N j ! 1 

### P(D/R j 

### ) P(R j 

### ) Equation D-7

D = new data R i = i thrisk prediction associated with new data Rj = j thrisk estimate simulated from PRA model N = number of risk estimates from the PRA model Figure D-8. Conceptual model of Bayesian Monte Carlo analysis. A PRA simulation yields a prior distribution of risk based on probability distributions for input variables. Given new data for an input variable, and a likelihood function for risk, Bayes Theorem (Eq. D-7) can be used to generate a posterior distribution of risk. The expression P(D/R) refers to a conditional probability, “the probability of D, given R” . Conditional probabilities can be thought of as relative frequencies, where R is the information given, and D is the event being computed when a particular value of R occurs.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-19 For example, suppose a model is available to relate soil 

### tetrachlorodibenzodioxin (TCDD) concentrations at a site with serum concentrations of TCDD. A probability distribution of soil concentrations is created based upon expert judgment and a limited amount of site specific data. Using the model, the soil concentrations can be associated with a distribution of serum TCDD concentrations (P®), the prior distribution). New site-specific data (D) are subsequently collected on serum TCDD concentrations in order to reduce uncertainty in the risk estimate. Assume that it is known that serum TCDD concentrations generally follow a lognormal distribution and that the best estimate of the parameters of this distribution come from the prior distribution on serum TCDD. This creates the likelihood function (P(D|R)). Using Bayes Theorem, the new data are used to form a revised distribution of serum TCDD. This is the posterior distribution (P(R|D)). Bayesian Monte Carlo analysis. In the past, the use of Bayesian analysis was limited by the degree of mathematical complexity involved. Using Monte Carlo analysis to carry out the PRA, rather than mathematical equations to describe the distributions, allows the calculations to be done much more easily. This variation on traditional Bayesian methods is called Bayesian Monte Carlo analysis (Patwardan and Small, 1992; Dakins et al., 1996). In the TCDD example discussed above and illustrated in Figure D-7, the required calculations are carried out for each of the N iterations of the Monte Carlo analysis (I and j go from 1 to N). Bayesian Monte Carlo analysis is appropriate in several situations. If a model has been created and a distribution developed using PRA, new information may be incorporated without the need to repeat the entire analysis. This information could be on one of the uncertain parameters of the model or on the model output variable. Similarly, a generalized risk model with generic parameter distributions may be used for a Superfund risk assessment with the model predictions fine-tuned using data from a particular site of interest. Finally, after a distribution is developed, the amount of uncertainty that exists may be too large for the risk manager to make a decision. In this case, the risk manager might seek out new information that would refine the analysis and decrease the uncertainty. Bayesian Monte Carlo analysis can also be combined with techniques from decision analysis to help determine the type and quantity of data that should be collected to reduce uncertainty. Decision analysis is a technique used to help organize and structure the decision maker’s thought process and identify a best strategy for action. To determine the appropriate action, one defines the range of possible decisions, evaluates the expected value of the utility or loss function associated with each decision, and selects the decision that maximizes the expected utility or minimizes the expected loss.

! Decision analysis provides a quantitative approach for evaluating the benefits of including an expanded assessment of uncertainty and the subsequent benefits of reducing this uncertainty. Value of Information. Value of information (VOI) analysis involves estimating the value that new information can have to a risk manager before that information is actually obtained (Clemen, 1996). It’s a measure of the importance of uncertainty in terms of the expected improvement in a risk management decision that might come from better information. Examples of VOI quantities are the expected value of including uncertainty (EVIU), the expected value of sample information (EVSI), the expected value of perfect information (EVPI). Calculation of these quantities can be done using mathematical methods, numerical integration (Finkel and Evans, 1987), or Monte Carlo techniques (Dakins, 1999)RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-20 Value of information calculations require the specification of either a utility or a loss function. A loss function states the losses associated with making different types of decision errors including both direct monetary costs and losses associated with other consequences. Loss functions take various forms depending on the risk management situation (Morgan and Henrion, 1990). Expected Value of Including Uncertainty . The expected value of including uncertainty, EVIU, is a measure of the value of carrying out a PRA. It’s the difference between the expected loss of a decision based on a point estimate risk assessment and the expected loss of the decision that considers uncertainty (Figure D-9). If uncertainty in a risk assessment has been estimated using Monte Carlo techniques and a loss function has been specified, the EVIU can be easily calculated. First, the management decision from the point estimate assessment is determined. The loss from making this decision is calculated for each iteration of the Monte Carlo, each time assuming that the risk estimate from that iteration is true. The expected loss is the average of these individual losses. The expected loss for the PRA is determined by calculating the expected loss for a full range of management decisions and selecting the decision with the lowest expected loss. The EVIU is calculated by subtracting the loss associated with the PRA from that associated with the point estimate risk assessment. Expected Value of Sample Information . The expected value of sample information is the difference between the expected loss of the decision based on the PRA and the expected loss of the decision from an improved information state. As such, the EVSI is a measure of the value that may result from the collection and use of new information (Figure D-9). Calculation of the EVSI involves a technique called preposterior analysis and is somewhat more complicated. This type of analysis is termed “preposterior” because it involves the possible posterior distributions resulting from potential samples that have not yet been taken. For each replication from the Monte Carlo simulation, the predicted value from the model is used to randomly generate a set of K data points. Each set of data points is then used to calculate the posterior probabilities for the N Monte Carlo simulated values. These posterior probabilities are then used to obtain the optimal answer to the management question at this new level of uncertainty by selecting the decision that minimizes the expected loss over all possible management decisions. This procedure is repeated for each of the N replications of the Monte Carlo analysis resulting in N posterior distributions, N management decisions, and N associated expected losses. Because each of these outcomes is equally weighted, the expected loss associated with the state of uncertainty expected to exist after the data collection program is carried out is simply the average of the N expected losses. The EVSI is the difference between the expected loss based on the results of the PRA and the expected loss from the updated information state. Expected Value of Perfect Information . The EVPI is the difference between the expected loss of the decision based on the results of the PRA and the expected loss of the optimal management decision if all uncertainty were eliminated. In actual application, no research plan or data collection program can completely eliminate uncertainty, only reduce it. The EVPI is an upper bound for the expected value of efforts to reduce uncertainty and so provides the ultimate bound on what should be spent on research and data collection efforts.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-21 Nominal Decision Ignoring Uncertainty

Decision Under

Uncertainty

Decision with Additional

Imperfect Information

Decision with Perfect Information EVIU

EVSI

EVPI

Expected Loss EVIU = Expected Value of Including Uncertainty EVSI = Expected Value of Sample Information EVPI = Expected Value of Perfect Information

Figure D-9. Expected Loss associated with various types of information incorporated into a generic uncertainty analysis. The x-axis reflects different categories of value of information (VOI) quantities. The y-axis reflects the increasing Expected Loss with increasing uncertainty.
When a PRA has been carried out using Monte Carlo techniques, the expected loss associated with perfect information is calculated by determining the expected loss for each iteration of the Monte Carlo, assuming that the correct management decision, if that iteration were true, is made. As always, the expected loss is the average of these losses, and the EVPI is calculated by subtraction.

Uses of Value of Information in Risk Assessment . VOI analysis has many benefits for risk managers. First, VOI analysis makes the losses associated with decision errors explicit, balances competing probabilities and costs, and helps identify the decision alternative that minimizes the expected loss. VOI analysis can help a decision maker overcome a fear of uncertainty by developing a method to handle it. If the losses associated with making a poor decision are unclear, small uncertainties can take on major importance. Conversely, if the losses associated with different risk management decisions are similar, little additional effort need be expended to continue to consider the alternatives. In addition, VOI analysis helps prioritize spending on research. It provides insights into how resources could be spent to achieve the most cost-effective reduction in uncertainty by identifying which sources of uncertainty should be reduced, what type of data should be obtained, and how much data isRAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-22 needed. Finally, VOI analysis may help decision makers explain the rationale for their decisions to the public and help the public understand the multiple objectives considered in managing risks. Expected Loss is usually greatest when uncertainty in risk estimates is ignored. For example, by quantifying uncertainty in risk (e.g., 2-D MCA, Bayesian Monte Carlo analysis) a risk manager may determine that the cleanup level associated with the 90 thpercentile of the risk distribution (rather than the 95 thpercentile) is adequately protective. Quantifying uncertainty may also result in lower expected loss when more soil remediation is required due to the losses associated with possible under-remediation, e.g., cost of additional sampling or lost revenue due to failure to meet land use requirements. The expected loss may be further reduced by collecting additional soil samples, which would presumably reduce uncertainty in estimates of mean exposure point concentrations. The expected loss may be minimized by obtaining "perfect" information (i.e., no uncertainty); however, as shown in Figure D-9, EVPI spans a wide range of expected loss because the value associated with reducing uncertainty may be tempered by costs associated with additional sampling and analysis. In practice, risk assessors consider this issue when deciding to obtain additional samples for site characterization. The decision to obtain additional information in order to reduce uncertainty should be made on a site-specific basis, taking into account the potential impact that reducing uncertainty may have on the overall remedial decision. Important questions to consider include: (1) Are the risk estimates sufficiently sensitive to an exposure variable that collecting further data will reduce uncertainty? and (2) Are the confidence limits on the 95 thpercentile risk estimate sufficiently wide that reducing uncertainty may alter the cleanup goal? An example of decision framework applicable to PRA is presented in Figure D-10. The framework has three tiers. Tier 1 includes the point estimate approach and an assessment of the need for PRA. In Tier 2, the EVIU is calculated and, if warranted, a PRA is conducted. In Tier 3, the value of additional information is assessed and Bayes Theorem would be used to incorporate the new information and update probability distributions. Limitations of These Techniques . Figure D-10 illustrates situations where Bayesian analysis and value of information quantities may not be helpful. For example, if point estimate risk assessment is selected as the appropriate method, these techniques do not apply. In addition, as site-specific data become available that are increasingly comprehensive and representative of the population of interest, Bayesian Monte Carlo analysis and the Monte Carlo analysis using the classical (frequentist) methods will approach the same result. This is because the site-specific data are incorporated into both approaches. To be representative and comprehensive, the data set must be sufficiently large, randomly selected, and represent the full range of variability that exists in the population (e.g., temporal, spatial, inter-individual). However, data sets are rarely perfect, often too small, suffer from relatively high sampling and/or measurement errors, or don’t represent the entire population variability over time, space, age, gender, or other important variables. If the data cannot be assumed to describe the population distribution sufficiently well, then PRA will help to more fully develop the entire range of the population distribution and the Bayesian Monte Carlo analysis will act to refine the model estimates.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-23 Figure D-10. Conceptual model for evaluating the expected value of including uncertainty in a Bayesian Monte Carlo analysis.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-24 In order to carry out VOI calculations, a loss function must be assumed. Definition of the loss function may be complex due to multiple decision goals and/or multiple decision makers and may be difficult to capture in an equation. Finally, for Bayesian analysis and the calculation of the EVSI to be helpful, one or more sources of new data must exist. In addition, some information must be available about these data since a likelihood function describing its probability distribution must be assumed.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-25 R EFERENCES FOR A PPENDIX D Brattin, W.J., T.M. Barry, and N. Chiu. 1996. Monte Carlo Modeling with Uncertainty Probability Density Functions. Hum. Eco. Risk Assess . 2(4):820–840. Buck, R.J., K.A. Hammerstrom, and P.B. Ryan. 1997. Bias in Population Estimates of Long-Term Exposure From Short-Term Measurements of Individual Exposure. Risk Anal. 17:455–466. Burmaster, D.E. and K.M. Thompson. 1997. Estimating Exposure Point Concentrations for Surface Soils for Use in Deterministic and Probabilistic Risk Assessments. Hum. Eco. Risk Assess. 3(3):363–384. Clemen, R.T. 1996. Making Hard Decisions: An Introduction to Decision Analysis . Duxbury Press, Pacific Grove, CA. Clifford, P.A., D.E. Barchers, D.F. Ludwig, R.L. Sielken, J.S. Klingensmith, R.V. Graham, and M.I. Banton, 1995. An Approach to Quantifying Spatial Components of Exposure for Ecological Risk Assessment. Environ. Toxicol. Chem . 14(5):895–906. Cressie, N. 1991. Statistics for Spatial Data. Wiley, New York, NY. Dakins, M.E., J.E. Toll, M.J. Small, and K.P. Brand. 1996. Risk-based Environmental Remediation: Bayesian Monte Carlo Analysis and the Expected Value of Sample Information. Risk Anal. 16:67–79. Dakins, M.E. 1999. The Value of the Value of Information. Human Eco. Risk Assess . 5(2):281–289. Finkel, A.M. and J.S. Evans. 1987. Evaluating the Benefits of Uncertainty Reduction in Environmental Health Risk Management. J. Air Pollut. Control Assoc . 37:1164–1171. Ginevan, M.E. and D.E. Splitstone. 1997. Improving Remediation Decisions at Hazardous Waste Sites with Risk-Based Geostatistical Analysis. Environ. Sci. Technol . 31(2):92A–96A. Gomez-Hernandez, J.J. 1996. Issues on Environmental Risk Assessment. In: Proceedings of the Fifth International Geostatistics Congress , Vol. 1. (Baafi, E.Y. and N.A. Schofield, eds.). Kluwer Academic Publishers, Dordrecht, 15–26. Goovaerts, P. 1996. Accounting for Local Uncertainty in Environmental Decision-Making Processes. In: Proceedings of the Fifth International Geostatistics Congress , Vol. 2. (Baafi, E.Y. and N.A. Schofield, eds.). Kluwer Academic Publishers, Dordrecht, 929–940. Goovaerts, P. 1997. Geostatistics for Natural Resources Evaluation. Oxford University Press, NY. Griffith, D.A. 1993. Spatial Regression Analysis on the PC: Spatial Statistics Using SAS . Association of American Geographers. Washington, DC. Hope, B.K. 2000. Generating Probabilistic Spatially-Explicity Individual and Population Exposure Estimates for Ecological Risk Assessment. Risk Anal. 20(5):575–590.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-26 Hope, B.K. 2001. A Case Study Comparing Static and Spatially Explicit Ecological Exposure Analysis Methods. Risk Anal. 21(6):1001–1010. Hora, S.C. 1992. Acquisition of Expert Judgment: Examples From Risk Assessment. J. Energy Eng

. 118:136–148. Howson, C. and P. Urbach. 1989. Scientific Reasoning: The Bayesian Approach . Open Court, LaSalle, IL. Isaaks, E. and R. Srivastava. 1989. An Introduction to Applied Geostatistics . Oxford University Press, Oxford. Kriakidis, P.C. 1996. Selecting Panels for Remediation in Contaminated Soils via Stochastic Imaging. In: Proceedings of the Fifth International Geostatistics Congress , Vol. 2. (Baafi, E.Y. and N.A. Schofield, eds.). Kluwer Academic Publishers, Dordrecht, 973–983. Lichtenstein, S. and B. Fischoff, 1977. Do Those Who Know More Also Know More About How Much They Know? Organizational Behavior and Human Performance 20:159. McKenna, S.A. 1998. Geostatistical Approach for Managing Uncertainty in Environmental Remediation of Contaminated Soils: Case Study. Environ. Engin. Geosci. 4(2), Summer, 175–184. Morgan, G.M. and M. Henrion. 1990. Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis . Cambridge University Press, NY. Ott, W.R. 1995. Environmental Statistics and Data Analysis . CRC Press. Boca Raton. Patwardhan, A. and M.J. Small. 1992. Bayesian Methods for Model Ancertainty Analysis with Application to Future Sea Level Rise. Risk Anal. 12:513–523. Price, P.S., C.L. Curry, P.E. Goodrum, M.N. Gray, J.I. McCrodden, N.H. Harrington, H. Carlson-Lynch, and R.E. Keenan. 1996. Monte Carlo Modeling of Time-Dependent Exposures Using a Microexposure Event Approach. Risk Anal. 16:339–348. Price, P.S., J.Y. Young, C.F. Chaisson. 2000. Assessing Aggregate and Cumulative Pesticide Risks Using LifeLineTM Version 1.0. A Report Submitted to U.S. EPA Science Advisory Panel. August 31. Shlyakhter, A.I. and D.M. Kammen. 1992. Sea-level Rise or Fall? Nature 253:25. Sivia, D.S. 1996. Data Analysis: A Bayesian Tutorial . Clarendon Press. Oxford. Slob, W. 1996. A Comparison of Two Statistical Approaches to Estimate Long-Term Exposure Distributions from Short-Term Measurements. Risk Anal. 16: 195–200.
U.S. EPA. 1982. Air Quality Criteria for Particulate Matter and Sulfur Oxides . ECAO, Office of Research and Development. EPA/600/8-82/029.RAGS Volume 3 Part A ~ Process For Conducting Probab ilistic Risk Assessment Appendix D ~ December 31, 2001

Page D-27 U.S. EPA. 1997. Exposure Factors Handbook. Update to Exposure Factors Handbook. Office of Research and Development, NCEA. EPA/600/8-89/043, May 1989. August.
U.S. EPA. 2000. Statistical Estimation and Visualization of Ground-water Contamination Data . Office of Research and Development, Washington, DC. EPA/600/R-00/034. Wallace, L.A., N, Duan, and R. Ziegenfus. 1994. Can Long-term Exposure Distributions be Predicted From Short-Term Measurements? Risk Anal. 14:75–85.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-1 

## A PPENDIX 

## E
D EFINITIONS OF 

## T ERMS 

## R ELEVANT TO 

## PRA AND 

## R EFERENCES FOR 

## F URTHER 

## R EADING


### E.0 D EFINITIONS OF 

### T ER MS Definitions for the

specializ ed terms

pertain ing to probab ilistic a

nalysi s are pr

esented in this appendix. Some of the same term s are also defined at the beginni ng of each chapter, s ometi

mes w ith additional exam

ples tha t are rel evant

to concepts prese

nted in the cha

pter. The defini

tions in this guidance are intende d to be consistent with definit ions us ed in the National Contingen cy Plan (NCP ) and other Environmen

tal Protection Agenc

y (EPA) g uidance, includ ing the d efini tions of varia bilit y, uncertainty, and Monte Carlo simulation

found in EPA’s Guidin

g Princ iples fo r Monte C arlo Analys is (U.S. EPA, 1997a). Note that if a def inition uses a t

erm th at is defined

elsewhere in the appen

dix, it is hig

hlight ed in bo ld text.

Def in it io ns of

Ter ms Use d i n PRA 50 th perc entile The number in a di stri bution s uch tha t hal f th e values in t he dist ribut ion are g reater than the number

and hal f t

he val ues a re l ess. The 50 th per cen tile is equivalent to the median

. 95 th perc entile The numbe r in a distribution such that 95% of the values in the distribution are less than or equal to the number and 5% of the values are greater t han the number.
95% Upper Confidence Limi t for a M ean The 95 percent upper con fiden ce limit (95% UC L) for a mean is defined as a value that , when r epeatedly

calcul ated f

or randomlydrawn subs ets of size n , equ als or exceeds the true population mean 95% of the time. The 9 5% UC L prov ides a mea sure o f u ncertainty in t he mean ; it is not a me asure of variability and sh ould not be conf

used w ith a 95 th per cen tile . As sample si ze incr eases, the di ffe rence between t

he UCL for the mean and the t rue mean decreases , whi le the 95 th per cen tile of the dist

ribution remains r elativel y unchanged, at the upper end of the dis tr ibu ti on. EPA’s Supe rf und p rogr am ha s tradi ti onal lyuse d th e 1- si ded 95% UCL for t he mean as th e concentration ter m in point est imates of reasonable maximum exposure ( RM E) for human health ris k ass essme nt (U.S. EPA , 1992, 1997 b). Applic able or Relevant and A ppropriate Req uiremen ts (AR AR s) Federal or st ate envi ronment al s tandards; the NCP stat es t hat ARARs shoul d be considered in determining rem edia tion g oals . ARARs may be sel ected as sit e-speci fic clean up le vels

. Ar ithme tic M ean (AM ) A number equal to t he ave rage valu e of apopula ti on or s ample. Usual ly obtai ned by summ ing all the v alues in th e sam ple an d div iding b y the num ber o f values (i.e ., sample size).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-2 Assessment End point A term usually as sociated with ecologi cal r isk asse ssmen t ; anexpl ici t ex press ion of an env ironm ental va lue (ec olog ical reso urce) that is to be pro tected , ope rationa lly define d by ris k managers and ris k asses sors as valuabl e att ribut es of anecologic al entity . Examples include 1) sustai ned aquatic community struct ure, incl uding spec ies com pos ition and relative a bund ance and tro phic stru cture; 2 ) redu ctions in populations of f

ish -eating birds ; an d 3) re ducti ons i n sur viv al, reprodu cti on or species diversit

y of indigenous benthi c communities (U.S. EPA, 199 7c, 1999a).
Backcalc ulati on A method of cal culat ing a preliminary remediation go al (PRG ) that invol ves alge brai c rear

rang ement

of t he ri sk equ ati on to s olve for concen tra ti on as a fun cti on of risk, ex pos ure, an d tox icity. Background Exposure Exp osure s that are not rela ted to the site. Fo r exam ple, ex pos ure to c hem icals at a diff erent time or from locations ot her th an the ex pos ure uni t ( EU) of co ncern
. Background sources may be either natur ally occurri ng or anthropogenic (man-made).
Ba yesia n A naly sis Stat istical anal ysi s t hat descr ibes the probabi li ty of anevent as t he degr ee of

beli ef or con fidenc e that a p erson has, give n som e state o f know ledge , that the ev ent will occur. Bayesian M ont e C arlo combines a prior probabil ity d ist ribut ion and a li kel ihood f

uncti on to yield a poste rior d istribution (see A ppe ndix D for exa mple s). Also c all ed subj ecti ve vi ew of probabi li ty , i n cont

rast to t he f requenti st viewof pro bab ility. Bootstrap M ethods A m ethod of sam pling a ctual d ata at ran dom , with rep lacem ent, to d erive a n estima te of a population parameter such as the arit hmeti c mean or the standard error of the mean . The sample size of each bootstrap sample is equal t o the sample size of t he origi nal dat a set. Both parametr ic and nonpar ametri c bootstr ap methods have been devel oped. Boxplot Graphical r epresent ation showing the cen ter an d spread of a di stri bution, someti mes with a display

of outlier s (e.g., Figure 7-3). This guidance uses boxplots t o represent the foll owing percentil es : 5 th , 25 th , 50 th , 75 th , an d 95 th

. Cancer Slop e Factor (CSF) A pla usible u ppe r-bou nd es timate o f the pro bab ility of a resp onse per u nit dos e of a chemi cal over ali fet ime. The CSF is used t o est imat e an uppe r-bound probabi li ty of an indi vidual developing cancer as a resul t of a lif etime of exposure t o a parti cular level o f a po tential car cinog en.
Central Limit Theorem If r andom samples of size n are repeatedly drawn from a population of any distribut ion, t

he dist ribut ion of s ample means conv erges to the no rmal d istribution

. The appro ximation imp roves as n increases. Central Tendency Expos ure ( CTE) A ris k desc ript or representing t he average or typical i ndividual in t he population, usual ly cons idered t o be the arith metic mean or median of the ri sk distribution.
CT E R isk The estimat ed risk correspondi ng to t

he central te ndency exposure.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-3 Cleanup Level A ch emi ca l c onc en tr at io n c hos en by th e r is k mana ge r a ftercon si der in g b ot h RGs and the ni ne se lection- of- remedycri ter ia of the NCP (U.S. EPA, 1990; 40CFR 300.430 (e)(9)(iii )). Also ref erred to as Final Remediation Levels (U.S. EPA, 1991), chem ical-spe cific clea nup le vels are doc ume nted in the Re cord of D ecision (RO D). A cleanup l evel maydiff er fr om a PRG for several reas ons, including various uncert aint ies i n the r isk es timat e, th e techni cal f easibi lit y of ach ievi ng the PRG , and appl ic at io n of

th e nine c ri te ri a ou tl in ed i n the NC P.
Coeffi cient

of Variat ion Ratio of the standar d deviat ion (SD) to th e arit hmeti c mean (AM ) (CV =SD /AM ). Dimens ionl ess measur e of t he spr ead of

a dis tri buti on, t heref ore, usef ul f or comparing prob ability den sity functio ns (PD Fs) for differe nt random variables

. Community Advisory Grou p ( CAG) A group formed to pro vide a public forum for community members to present and discu ss their ne eds a nd co ncern s related to the S upe rfund d ecision -makin g pro cess. A CAG serves as th e focal point f or the ex change of inf

ormation among the l ocal community , EPA, Stat e regul atory agency, and other perti nent Feder al agenc ies invol ved in t

he cleanup of a Superf

und sit e.
Com munity Invol vement Coor di nat or ( CIC) As a member of t he CAG and site team, t he CIC coo rdinates communication plans (i. e., t he Communicty I nvolvemen t Plan (CI P) and a ddr esses site-sp ecific CAG organizationa l issues.
Com munity Invol vement Pla n (CI P) A plan that iden ti fi es commu nit

y conc erns and the pre fer ences of t he commun ity f

or the comm unication of site-related issues.

Concentration Term The conce ntrati on var iable used in exposur e ass essme nt . Concentration terms are exp ressed in units ap plicab le to the m edia o f conc ern (e .g., mg/L for wate r, : g/m 3 for air; mg/k g for so il and d ust.
Confidenc e Int erval A range of values that are likel y to incl ude a population parameter

. Confidence interva ls may describe a parameter of an inp ut var iable (e.g., mean inges tion r ate) or o utput var iable (e.g., 95 th perc entile ris k). When used t o charact erize uncert ainty in a risk estim ate, it is assum ed tha t metho ds use d to q uantify uncertainty in the model inputs ar e based on statist ical princi ples such as sampling distri butions or Bayesian app roac hes. Fo r exam ple, give n a ran dom ly samp led d ata set, a 95% confidence i nterva l for the mean can be estimated by deriving a s ampling distribu tion from a Stud ent's t distributio n.
Co nfide nce Limit The upper or lower va lue of a confi dence i nterval

. Co ntinu ous V aria ble A ran dom var iable that can assume any value within an interval of r eal numbers (e.g., concentration).
Coun tably Infinite Used to descri be some discr ete random variables , this ter m ref ers to a s et of numbers that can be counted with int egers (e.g., one, two, three) and that has no upper li mit. Examples i nclude t he number of tosses r equired f or a coin t o show a head

— we ca n cou nt each toss, bu t it is possib le that at lea st one m ore to ss is need ed. Th e num ber o f dust pa rticles in a v olum e of air is a nothe r exam ple. C ounta bly finite implie s there is a n upp er limit (e.g ., days o f work p er year ).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-4 Correlat ion A qu antitative re lationsh ip be tween two o r mo re inpu t varia bles of a m ode l (e.g., body weight , inhalat ion rat e, skin surf ace area). In anal yses invol ving t imedep end ent variables , a chang e in one variable is acco mpa nied b y a chan ge in anoth er t ime- dependen t, corre lat ed var iable . Ignoring correl ations in pro bab ilistic ri sk as se ss men t ( PRA) may lead to un realistic combination s of values in a risk calculati on. Correlations can als o be defined as relationships between inputs and outputs. Coverage Co nfide nce interv als are expected to enclose a true but unknown parameter according t o a specif

ied probabil ity , such

as 90% or 95%. This is t he expected coverage of

the confi dence i nterval , given a spec ified significa nce lev el (alph a). Th e differe nce b etwee n the ex pec ted co verag e and the actu al cov erage is one m etric for evaluati ng stati stical methods that yi eld different con fiden ce inte rva ls

. Credibl e Int erval A range of values that represent plausibl e bounds on a po pulation parameter

. Credible i nterva ls maydescri be a parameter of an inp ut variable (e.g., mean ingestio n rate) o r outp ut variable (e.g., 95 th perc entile ris k). The t erm is introduced as a n al ter

nat iv e t o th e t erm confi dence i nterval when the me thod s used to qua ntify uncert ainty are not based entirely on statis tical pri nciples such as s ampling distri butions or Bayesian approaches. For example, multiple est imates of an arit hmeti c mean may be available from diff erent studies reported in the literature

— using p rofessio nal jud gme nt, these e stimates m ay sup por t a dec ision to describe a

range of possibl e values for t

he arit hmeti c mean.
Cumulati ve Dist ribut ion Func ti on ( CDF) A graph that shows t he cumulative probabil ity of occurr ence for a random indep end ent var iable (e.g. , Fig. 6-1). The cumulati ve probabili ty i s ty pically giv en as the y-ax is, rangi ng fr om 0 to 1.0. Each va lue c of the func tion is the p rob ability that a random observation x wil l be l ess than or equa l t o c . Mathematically, the funct ion th at def

ines t he CDF is obtai ned from t he PDF by i nteg rat ion ( in t he cas e of a conti nuous ran dom var iable ) or by summation (for discret e random variables ).
Disc rete V aria ble A ran dom var iable that can assum e any va lue within a finite set of va lues (e.g ., number of ra inf

all events in one mont

h) or at most a countably infinite set of values. Empiri cal Di stri buti on A dist ributi on obta ined from a ctua l dat a and pos sibly smoot hed wi th i nter

polat ion techniques. Data are not f it t o a particular parametri c dis tri buti on (e.g., no rmal, lognormal ), but

are descr ibed by t

he per cen tile values. Expecte d Value of Inform ation (E VO I) The expec ted increase in t he value (or decrease in the l oss) associated wit h obtaining more inform ation abo ut quantities relevant to the decision proce ss. EVO I is a mea sure o f the imp ortan ce of uncert ainty in risk and the potential f or changing a ris k managementdecisio n if uncert ainty is reduced (see Appendix D). Expert Ju dgment An infe rential o pinio n of a sp ecialist o r grou p of sp ecialists with in an are a of their experti se. Expert

judgment ( alte rnati vely refer red to as profess ional judgment) may be base d on an a sses sment of dat a, as sumpt ions , cr it eri a, model s, and parameters in respo nse to q uestion s pos ed in the releva nt area of exp ertise (se e Ap pen dix D ).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-5 Exposure Asses sment The qualit ativ e or quanti tati ve est imate ( or measurement) of t he magnit

ude, fre quency , dur ati on, a nd rout e of e xposur e. A pr ocess tha t i nteg rat es i nfor mati on on chemical fate and tr ansport, environmental measurements, human behavior, and human physi ology t o esti mate th e average dose s of che micals receiv ed by indi vidual recept ors. For simpli cit y i n th is gui dance, exposure en compass es con cepts of absorbed do se (i.e. , uptake and bioavailabili ty). Exposure Point Conc ent rat io n ( EPC) The contaminant concentration withi n an expo sure unit to which receptors are exposed. Est imates of th e EPC represent t he concentration ter m used in exposure asses sment

. Exp os ure Uni t ( EU) A geog raph ic are a wh ere expos ure s oc cur to t he r ecep tor

of c once rn d uri ng t

he t ime of interes t. Receptors may be human or ecological (e.g., plants, bi rds, fi sh, mam mals). F or p urpo ses of PRA , prob ability distribu tions for exposure and toxicity variables apply equally t o all members of a population at a given exposure unit . Ecologi cal exposure uni ts of ten cons ider habi tat and seasonal ity fact ors th at enhance exposu re in a spatial area usually related to ho me range s.
For wa rd C alculation s A method of calculating a ri sk estimate t hat involves t

he standard arrangement of the risk eq uation to solve for risk as a functio n of co ncen tration, e xpo sure, an d tox icity. Frequency Di stri buti on A graph or plot that shows t he number of obs ervati ons that occur wit

hin a given int erval; usual ly pre sented a s a his togr am showing the re lati ve probabili ties for each value. It conv eys t

he range of

values and the coun t (or pr oportion of the s ample) that was observed a cross t

hat range. Frequen tist A term referring t o classical s tatis tics i n which the probabilit y of an event occurring is def ined as t he fre quency of

occurrence measured i n an observed s eries of repeat ed trials.
Geom et ri c Mean (GM) The n th roo t of the p rod uct of n obse rvation s. For lo gnor mal d istribution s, the G M is equal t o the median

and is l ess t

han the arit hmeti c mean . For norm al distributions, all three m easur es of ce ntral tend ency (G M , AM , median ) are eq ual.
Geostatist ics Bra nch o f statistics that foc uses o n data that hav e a sp atial or g eog raph ic component s. In r isk

ass essme nt, geost ati sti cs i s a ge neral ter m for a var iet

y of techniqu es t

hat are t ypi cally appl ied t o chemi cal concen tra ti ons i n soi l or groundwater in which the s ampling locations are consi dered in quantifying the exposur e point

concent rati on
. Goodn es s- of- Fit(GoF) Test A m ethod for exa mining how w ell (or p oor ly) a sam ple o f data c an be desc ribed by a hypot hesi zed probabil ity d ist ribut ion for t he populati on. General ly i nvolves an hypo thesis test in which the null hyp oth esis H 0is t hat a ran dom var iable X foll ows a specif ic probabil ity d ist ribut ion F 0 . Tha t is, H 0 : F = F0and H a : F F 0 . Hazard Index (H I) The sum of more th an one hazard q uotient for mu ltiple sub stance s and /or mu ltiple exposure pathways. The HI is calculated separatel y for chronic, subchronic, and shorter-duration e xposure s.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-6 Hazard Quot ie nt (HQ) The ratio of estimated si te-specif ic exposure to a singl e chemical from a sit e over a spec ified pe riod to the es timated daily ex pos ure lev el, at which no ad verse health effects are likel y to occur.
Hazardous Substance Research Centers (HSR C) Rese arch c enters p rovid ing free te chnica l assistanc e to co mm unities with environmental contaminati on programs through two distinct outreach p rogram s: Tec hni cal

Out re ach Se rvi ce s f

or C omm uni ti es (T OSC) and Technical Assis tance to Brownfields Community (TAB ). Hig h-end Risk A ris k desc ript or representing t he high-end, or upper tail of the ri sk distribution, usua ll y cons ider ed to be equal to or grea ter tha n th e 90 th per cen tile

. Histogram A graph ing tec hni que whi ch gr oups t he dat a in to i nter

vals and di splays t

he coun t of the observations wit

hin each inter

val. It conveys the range of val ues and the relati ve frequenc y (or proport ion of t he sample) that

was observe d across t

hat range. Hypothesi s Test ing Stat istical tes t of an as sumpt ion about achar acte ris ti c of apopula ti on. The g oal of the statistical infe rence is to dec ide wh ich of two com plem entary h ypoth eses is likely to be true.
Ima ge A naly sis A technique in geostatis tics used to r estore a degraded image or int erpret images that have been contaminated by noise or possibly some nonlinear transf

ormation. Independence Two events A and B are ind epe nde nt if know ing whe ther or not A occur s does not chan ge the p rob ability that B occur s. Two random variables X and Y are indep end ent if the jo int probabil ity d ist ribut ion of X and Y can be expressed as the pro duc t of the ind ividua l marg inal probabil ity d ist ribut ions . Tha t i s, f( X, Y) = f(X) A f(Y). In dep end ence of X and Y is not syn onymous wit h zer o correl ation (i.e., Co r(X , Y) = 0). If X

and Y are independent, then Cor( X , Y ) = 0; however, the converse i s not ne cessar ily true bec ause X

and Y may be rel ated in

a nonli near fas hionbut s ti ll main tai n zer o correl ation (Law and Kelton, 1991 ). Independ ent and Identical ly Distributed (I ID) Random v ariables that are in dependent and hav e the s ame probabil ity d ist ribut ion of occurr ence.

Individual -Level Effect An asses sment endpoint that focuses on protect ing a hy potheti cal or r eal i ndivi dual in a population. Indivi dual-based models may account for unique exposure and toxicologica l response to c hemicals am ong individu al receptors.
Iter ative Reduc tion (IR) A method of cal culat ing a PRG that i nvolves successively lowering t he conce ntra ti on te rm unt il the cal culated r isk is accept able. This me thod c an be appl iedto any medi um.
Iter ative Truncati on A method of cal culat ing a PRG that i nvolves developing an expression for t he concentration ter m in which hi gh-end values are “t runcated” to reduce the maximum concent

rati on, and cal culat ing ri sks as sociat ed withthe re duced concentration. The method may be repeated with consecutively l ower truncat ion limi ts un tilris k is a cceptable . It erati ve truncat ion methods avoi d diff icul ties associated with apply ing M ont e C arlo ana lysis to a backcal culat ion

.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-7 Kriging A statistical int erpolation method that sel ects the best linear unbiased est imate of the parameter in ques tion. Often us ed as a geo statistic al method of s pati al s tat istics for pred icting valu es at uno bserv ed lo cation s base d on data fro m the su rroun ding a rea. Inf ormat ion onfat e and t

rans port of

chemi cals wit

hinthe area lack ing datacan be incorpo rated into kriged estimates.
Ku rtosis The measure of peak edness of a dist ribut ion. A un ifor

m distr ibuti on has a l ower kurto sis than a p eake d distrib ution su ch as the norm al and logno rmal d istribution

. Kurtosis is r eferr ed to as t he 4 th centra l moment of a dis tri buti on
. Land M ethod The conventional method for calculating uncertainty in t he mean concen tra ti on (e.g., 95% UCL ) whe n th e sampl e dat a are obtai ned f rom a l ognormal

dis tri buti on (U.S. EPA , 1992). Latin Hyperc ube Sampli ng (LHS) A variant of th e M onte Carl o sampl ing method that ensures selec tion of

equal numbers of values from all segments of the dis tributi on. LHS divides the distri bution into regi ons of equal sampling coverage . Hence, t he valu es obtai ned will be fo rced to co ver the entire d istribution . It is more efficient than simple random s ampli ng, i. e., it requi res fewe r it erat ions to ge nerat e th e dis tri buti on sufficiently.
Likel ihood Funct ion A te rm f rom Bayesian stat ist ics r eferr ing t o a probability d ist ribut ion that expresse s the probabili ty of observin g new in formati on given t hat a particu lar bel ief is t rue. Local Sensitivity An alysis Evaluati on of th e model sens iti vity at s ome nominal poi nts wi thi n the r ange of va lues of input vari able(s).
Location Tag The spatial coordinates of a sampling l ocation (e.g., longitude, l atitude).
Low -end R isk A ris k desc ript or representing t he low-end, or lower tai l of the r isk dist ribution, such as the 5 th

or 25 th per cen tile

. M aximum Detected Conc ent rat io n ( MDC) The maximum concentr ation detecte d in a sample.

M ean Arithme tic mean or ave rage; the sum of allobservati ons di vided by the numbe r of obs ervatio ns. Re ferred to as the first central moment of a dis tri buti on
. M icroe xposure Event (M EE ) An alysis A method of assessing ris k based on an aggregate sum of a receptor's contact with a contaminat ed medium. MEE analy sis si mulat es l if eti me expos ure a s t he sum of many s hort -t erm, or “mi cro” exposure s (s ee Appendi x D). MEE approach es ca n be used to expl ore uncertainty assoc iated w ith the mo del time st ep in PRA (e.g., use of a sing le value to rep resen t a long-te rm av erage phen ome non, se ason al patte rns in exp osure , or intra-ind ividua l variability ).
M ode The most probable value of a ran dom var iable ; a value with the larg est pro bab ility or highe st probabi lit

y densi ty ( or mass f

or discr ete ran dom var iable ). The second parameter of a triang ular distr ibution

.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-8 M oments of a Distr ibut ion Simil ar to a parameter ; consta nt that rep resen ts a math ema tical de scriptio n of a ran dom var iable . Centr al moments are defi ned wit h respect to th e mean
. M ean , variance , skew ness , and kurtosis are th e fi rst, second, t hird, and fourt

h centr al moments of a probabil ity d ist ribut ion

. M ont e C arlo An alysis (MC A) or Simul ation A tech niqu e for charact erizi ng the uncert ainty and variabi li ty in r isk est imat es by repeatedl y sampling the prob ability distribu tions of the ris k equation inputs and usi ng these inpu ts to ca lcul ate a dis tri buti on of r isk

val ues. A set

of i ter ati ons or calcu lations fro m M onte C arlo sa mplin g is a simu lation. F or ex amp le, a single iterati on for risk f

rom ingestion of water may represent a hypotheti cal indivi dual who drinks 2 L/day and we ighs 65 kg; anot her it erati on may repres ent a hy potheti cal individual who drinks 1 L/day and weighs 72 kg.
M onte Carl o Sampli ng A method of si mple random sampli ng used t o obtain a di stri bution of

values which may serv e as an i nput t o a PRA . The pro bab ility of obta ining an y given sa mple is similar to the probability of a sample occurring withi n the distr ibution. Hence, for a given sample size, simple random sampling tends to produce values cluster ed around the mean of the dist

ribution.
M ultiple Regre ssi on An alysis A statistical method that describes the extent, direction, and st rength of the relation ship b etwee n seve ral (usua lly continu ous) in dep end ent variables (e.g., exp osure dura tion, inge stion rate ) and a single c ontinu ous d epe nde nt var iable (e.g., risk). Nonparametri c Method A procedur e for maki ng st ati sti cal inf

erenc es wi thou t as sumi ng that the population distribut ion has any speci fic form such as normal or lognormal . Someti mes ref erred to as dist ributi on-free metho ds. Co mm on ex amp les are th e sign test, Spearman rank corr elat ion , and t he bootst rap -t approach.
Num erical Stability The property of a probabili sti c simul ation such t hat t he a parameter value of the outp ut distrib ution (e .g., per cen tile , mean , variance , etc.) rem ains sufficien tly con stant for a spec ified num ber o f M ont e C arlo iteration s. Num erical stabi li ty is a measur e of t

he pre cis ion of

the outpu t f

rom a s imul ati on; t

he t ail s of

the dis tri

buti on are t ypi cally l ess sta

ble t han t he cen ter . Suf

fi cien t pre cis ion i s det ermi ned by pro fessiona l judgm ent.
One-d imensional M onte Carlo A nalysis (1-D MCA) A method of si mulating a dis tri

buti on for

an endpoint of

concer n as a fun cti on of probabil ity d ist ribut ions that charact erize variability or uncertainty . In this guidance, dist

ribut ions us ed to charac teri ze vari abil ity maybe abbre viat ed PDF v, where as di str ibut ions used t o char acte riz e unce rta int y maybe abbre viat ed PDF u. It is good pract ice not to combine PDF s for variabi li ty and uncert ainty in 1-D MCA.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-9 Parameter A value t hat char acteri zes th e probabil ity d ist ribut ion of a ran dom var iable . For exam ple, a n orm al probabil ity d ist ribut ion may b e defin ed b y two p aram eters (e .g., AM and SD). It is important to distingui sh between this definit ion, and a second popular use of the term parameter when referr ing to an input variable in a mat hema ti cal equa ti on or model . Fo r thi s gu ida nce, th e t erm varia ble will be use d to describe inputs to a model. For example, if body weight is a vari able in the exposure assessm ent that w e defin e with a p rob ability distrib ution (e .g., norm al) we w ould stat e that the var iable i s body weig ht and t he parameter s are t

he ari thmeti c mean and standar d deviat ion valu es t hat chara cter ize the normal dis tri buti on
Parametri c Dist ribut ion A theoret ical dist ribut ion defined by one or more para meters. Examples are the normal distribut ion, the l ognormal distributi on, the tri angular distr ibution, and the beta distri bution.
Per cen tile The p th perc entile of the d istribution is the value such tha t p percent of the observ ati ons f

all at or below it . Al so cal led quantiles or fract iles ; percentil es are expressed as a percent, ranging f rom 0 to 100 , whereas quantil es or fracti les range from 0 to 1.
Point Estimate A quant ity cal culated f rom values i n a sample t o repre sent an un known popu lat ion parameter . Point esti mates typical ly represent central t endency or upper bound estima te of variability

. Point E stimate R isk Asses sment The familiar ris k ass essme nt metho dolo gy in whic h a single estimate of risk is calcu lated fro m a se t of point est imates . The results provi de point estim ates of risk for t he CTE and RM E exp osed individ uals. Variabi li ty and uncert ainty are discussed in a quali tative manner.
Po int P atte rn A naly sis A technique in geostatis tics of restri cting the analysis t o location informati on, ignoring att

ribute inf

ormation, addresses two locati on problems: (1) describing poin ts acco rding to spa cing, an d (2 ) desc ribing p oints ac cord ing to d ensity. Population-Level Eff

ect An ecological term for an asses sment endpoint tha t f ocuse s on pr otecting a grou p of indi vidu als wit

hina spec if

ied expo sure unit and ti me that have si milar

exposures and toxico logical respon ses to chemica ls.
Posteri or Dist ribut ion A te rm f rom Bayesian stat ist ics r eferr ing t o a probability d ist ribut ion that has been updated with new information.
Po tentia lly R espo nsible Part y (PRP) Individuals, companies, or any other party t hat is potent ially

liable f or Superfund cleanup c osts.
Pow er Th e pro bab ility that a test pro ced ure d etects a false null hyp oth esis ; Po wer eq uals (1- $ ), wher e $ is t

he probabili ty of

a Type II error (i.e. , accepting H 0when H a is true). P owe r curve s are a fu nction of a fixed significanc e level ( " ), sample size, and variability ( SD ).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-10 Prelimi nary Remediat ion Goal (PRG ) A chemical concentr ation in an environment al medium as sociat ed witha parti cular exp osure scena rio that is ex pec ted to be p rotec tive of hu man health a nd ec osystem s. PRG s may be developed ba sed on ( ARARs ), or exposure s cenar ios eval uate d prior to a ris k ass essme nt (e.g. , gener ic PRG) or as a res ult of th e baseli ne risk asses sment (site- specific PRG). Exhibit 5-1 provides furt her detail on generic and site-specific PRG s.
Prior Di stri buti on A Bayesian ter m ref erri ng t o the hypot

hesi zed, e xpect ed, or

cal culated probab ility dis tri buti on for an event prior t o the collecti on of new information.
Pro babilistic Risk Ass es sm ent

(PR A) A ris k ass essme nt tha t us es pr obabil istic met hods t o deri ve a di str ibut ion of ris k or hazard based on m ultipl e sets of values sampled for random variables

. Proba bility Density Func ti on ( PDF) A graph that shows the probabilit y of occurrence of an unknown or var iable quanti ty. A PDF is used to character ize a continuous random variable; the i ntegra l of al l pos sibl e val ues i s equa l t o 1.0 ( i. e., the area under the curv e). In PRA, PDFs can be used t o display

the s hape of t

he dist ribut ion for

an input varia ble (e. g., normal distribut ion for

inges tion r ate) as well as th e output from a M onte Carl o si mulati on (e.g., risk d istribution ).
Probabil ity Di stri buti on A funct ion th at ass ociates probabili ties with the val ues taken by a random var iable

. A pr oba bility distribu tion ca n be d isplayed in a grap h (e.g., PDF or CDF ), summarized i n a tabl e that gives the di stri

bution n ame and parameter s, or e xpressed as a ma thema ti cal equat ion. In PRA, t he proc ess of s elec ti ng or f itti ng a di str ibut ion that characterizes variability or uncertainty can also be referred to as applyi ng a probabili ty model to chara cteri ze variability or uncertainty . In thi s guidance, the pro bab ility mod el is con sidere d to b e one sourc e of m ode l uncer tainty.
Pro bability M ass Func ti on ( PMF) A his togram that shows t he probability of occurrence of an unknown or var iable quanti ty. A PM F is us ed to charac teri ze a discr ete ra ndo m va riab le ; simil ar to the PDF , the sum of al l possible values of a PM F is equal to 1.0. The mass at a point refer s to t

he probabili ty t

hat t he variable will have a value at that po int.
Ra ndo m V aria ble A var iable that ma y assum e any va lue from a set of va lues ac cord ing to ch ance
. Dis cre te ran dom varia bles can assume only a fi nite or countably infinite number of values (e.g., num ber o f rainfall eve nts per year). A rand om v alue is co ntinuo us if its set of possible values is an ent ire int erval of numbers (e.g. , quantit y of rain i n a year) variable that ma y assum e any o f a set of va lues. T he likeliho od o f each v alue is described by

a probabil ity d ist ribut ion

. Rang e Sensitivity An alysis Evaluation of the model sensitivity across the enti re range of values of the input variable(s).
Rank If a set of values is sorted in ascending order (small est to lar gest), t he rank corresponds to the relati ve position of a number in the sequence. For example, the set {7, 5, 9, 12} when sorted gives t he following sequence {5, 7, 9, 12} with r anks rang ing fr om 1 to 4 ( i. e., rank of 5 i s 1, rank

of 7 i s 2, rank

of 9 i s 3, and rank of

12 is 4).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-11 Rank Correlat ion (Spearman Rank O rder Correlation C oefficient) A “dist ribut ion fr ee” or nonparamet ric s tati sti c r that measures t he strength and direction of asso ciation betwee n the ranks of the values (no t the values themselves) of two quan tit ativ e variables

. Remedial Investigation/Feasibility Study (RI/

FS) Stud ies und ertake n by E PA to de lineate the nature and e xtent o f conta minatio n, to

evalua te po tential risk, an d to d evelo p altern atives fo r clean up.

Reasonable M aximum Expos ure ( RM E) The hig hest exposure t

hat is reas onably

expect ed to oc cur a t a s it e (U.S. EPA, 1989, 199 0). T he inten t of the R M E is to e stimate a cons ervativ e exp osure case (i.e ., well abov e the average case) that is still within the range of po ssible exposu res. RM E R isk The estimat ed risk correspondi ng to t

he reasonable maximum exposure

. Ref er enc e Do se (Rf

D) An estimate of an exposure level for the human population, i ncluding sensi tive subp opu lations, tha t is likely to be withou t an ap prec iable risk of de leteriou s effects during a lif

etime. Chronic RfDs are specifi cally developed to be protective for a long-term exposure to a chemical (e. g., >7 years) and account for uncertainty spanning perhaps an order of magnitude or greater.
Remediat ion Acti on Level (RAL) Ge nerally, a c onc entratio n such that rem ediatio n of all co ncen trations a bov e this level in an expo sure unit will resul t in the 95% UCL being re duced to a l evel t hat doe s not p ose a n unac cep table risk to an ind ividua l expe riencin g rand om e xpo sures. The RAL will depend on t he mean , variance , and sample size of t he concentrations within an exposure unit as well as cons idera tions o f acute to xicity of the c hem icals of co ncern
. Remediat ion Goal Generally, a healt h-based chemical concentration i n an environmental medium chose n by t

he ri sk man ager as appr opria te f or a l ikely l and us e sce nari o.
Risk As sess ment Th e use o f availab le inform ation to make inferenc es ab out the health e ffects assoc iat ed wit

h expos ure of

indi vidu als or popul ati ons t o hazar dous mat eri als or sit uati ons. Components of ri sk ass essment incl ude: hazar d identi fication, doserespo nse asse ssmen t, exposur e ass essme nt , an d ris k charact eriz ation (NRC, 1983).
Risk Charac teri zati on A co mpo nent o f ris k ass essme nt that describes t he nature and magnitude of ris k, including uncertainty . In assess ments of Superfund sit es, it includes the s ummary and int erpreta tion of

inf ormation g athere d from previ ous st eps in t he sit e risk asses sment (e.g., da ta evalu ation, exposur e ass essme nt , toxici ty asses sment), including the results of a pro babilistic analysis.
Risk Des cript or A stat ist ic (e. g., arit hmetic mean , 95 th

perc entile ) that descri bes the risk t o the asses sment endpoint

. Risk Management The process by whi ch regul atory decisi ons are made us ing al l avai lable risk asses sment inf

ormation ( incl uding, but not limi ted to, the re sult s of t he PRA ). The NCP provides nine crit eria for remedial decis ions (e.g., protection of human health, compliance with ARARs , etc .). Risk manag ers may include t he Remedial Projec t Manager (RPM ), se ction and branch chi efs, etc.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-12 RM E Range The 90 th to 99 .9 th percentil es of th e ris k dist ribut ion gener ated f

rom a PRA , within which an RME ris k value may be ident ifi ed. The 95 th percentil e is gener ally recommended as th e star tin g point for speci fyi ng the RM E ris k ina Super fun d PRA

. Scien tif ic/M anagement Deci si on Poi nt ( SMDP) A point duri ng the ris k ass essme nt process whe n the r isk as sessor communicate s results of the assessment at that stage to the risk mana ger. At this point, the risk manage r det ermi nes w heth er t he i nfor mati on is suf

fi cien t t o arri ve at

a deci sion regarding ris k managementstrategies and/o r if add itional info rmatio n is need ed to characterize ri sk.
Sen sitivity An alysis Process for identif ying the i mportant sources of variabil ity and uncert ainty i n a model’s out

put. Dif ferent techni ques can be us ed in each of the 3 tier s of t he ti ered process for PRA (see Cha pter 2). In Tier 1, sensi tivit y rati os are used to quantify the effects of changes in one or more model inputs on the model output. In Tiers 2 and 3, cor rel ati on analysi s canbe use d to r ank i nput s bas ed on their rel ati ve cont

ributi on to variance in ris k. Local sensitivity refers t o nominal changes in inputs within a plausible range, whereas range sensit ivity refers t o changes in inputs across the minimum and maximum values of the plausible range. Further explanati ons of the dif fe ren t me th ods f

or s ens it iv it

y an aly si s ar e gi ven in Appen dix A.
Sen sitivity Ra tio Ratio of the ch ange in mo del o utput p er unit ch ange in an inp ut var iable ; al so call ed elasticity

. Skew ness The measure of asy mmetry of

a dis tri buti on. Coef

fi cien ts of s kewne ss a re zer o for symm etric distrib utions (e .g., norm al), po sitive for rig ht-skewe d distrib utions (e .g., lognormal), and negative f or left- skewed distributi ons (e.g., specific f orms of beta) . Refe rred to as the third ce ntral moment of a dis tri buti on
. Spati al Autocor relat ion The ten dencyof dat a fr om locations tha t ar e rel ati vely clos e tog ethe r to be geogr aphic all y cor

rel ated.
Stakeholder Any ind ividua l or gro up wh o has an intere st in or m ay be a ffected by E PA ’s site decision-ma king proce ss. Stability Stochastic variability , or “w obble” associ ated wi th ran dom sampling, calcul ated as the average percent change in t he model output after rerunni ng M ont e C arlo simulation s with the s ame set of input ass umptions. Used as a metric for evaluating the adequacy

of th e number of i terat ions i n a MCA

. Standard Devi ation, Arithme tic and Ge ome tric Stan dard dev iat ion ( or ari thmetic s tandard devi ati on, SD) i s a commonmeasu re of the spr ead of a dis tri bution. Calcul ated as t he square r oot of t

he var ianc e. The geometric standard deviation (GSD) is the anti -log of the standard deviation of t he logarithms of each value. The GS D is a unitl ess quantit y that gi ves a measure of the rati o of the varian ce to the mean, s imil ar in c oncept to t he coeff ici ent of variat ion

. Step Funct ion A mathematical functi on that remains constant within each of a series of adjacent interva ls but ch ange s in value from o ne interv al to the n ext. Cumulati ve dis tri buti on funct ions for discret e random variables are step function s.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-13 Stochastic Dominance Implies no int ersect ion betwe en the CDFs ; dis tri bution A s tochast ical ly domin ates distribu tion B if, for every per cen tile of th e CDF , A > B. This char acteri sti c may not be apparent

from th e PD Fs of t

he di str ibut ions , whi ch mayoverl ap.
Stoch astic Pro cess A process involving random variables , and c harac terized by variability in s pace or time. Target Popul ation The set of all receptors t hat are potential ly at r isk. Sometimes ref erred to as the “po pulatio n of co ncern ”. A sam ple p opu lation is sele cted fo r statistical sam pling in or der to mak e i nf

er en ce s r eg ar di ng th e t ar ge t p opu la ti on (s ee App en di x B, Section B.3.1, Concepts of Pop ulations and Sampling).
Technical Assi stance Gran t ( TAG) A federal grant that is int ended to provide a com munity wit h the opportunity t o hire independent

experts to hel p evaluat e and explai n the r esult s of a ris k ass essme nt
Technical Outreach Servic es f

or Comm uni ti es (T OSC) A servi ce of t he HSRC wit

h th e ai m to pr ovide indepe ndent tec hni cal inf

ormat ion and assistance to h elp com munities with hazardo us substance p ollution prob lems.

Th iessen (Vo ron oi) Po lygo n A naly sis A method of spat ial st ati sti cs i n whi ch an area i s su bdivi ded in to s ubreg ions , or polyg ons, in o rder to pre dict va lues at un obs erved locatio ns. Time Step A variable in al l exposur e models that re fers to th e unit of ti me for whi ch a random value is consid ered repre sentative of intra-ind ividua l variabi li ty (e.g., ave rage d aily ing est ion r ates for

an i ndiv idual

fr om one ye ar t o the next ). A ti me st ep may be equal to an entire exposure durati on (e.g., 30 years), or a fracti on of the exposure dura tion du ring wh ich cha nges in in put varia bles may b e exp ected (e.g., on e year)

. Time steps n eed not be i dentic al f or all exposure variables , and should address the most rapidly changing var iable in the ri sk equation. Time step can be an important consideration for M EE ana lysis

. Toxicity Reference Val ue (TRV ) A num erical e xpre ssion o f a chem ical’s do se-resp onse relation ship tha t is used in ecologi cal r isk

asse ssmen t

. True Mean Concentrat ion The actual average concentration in an expo sure unit . Even with extens ive samplin g, the tru e mean cannot be kn own. Onlyan est imate of the t rue mean is possible. A greater number of representative samples increases confidence that t he esti mate of t

he mean more closel y repres ents t he tru e mean
. Truncati on The process of s etting lower and upper li mits on the r ange of a di stri bution, in order to avoid unreali sti c values for exposure variables (e.g., > 1 00% bioa vailab ility). Most oft en us ed for

cont inu ous, unboun ded probabil ity d ist ribut ions (e.g., no rmal).RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001
Def in it io ns of

Ter ms Use d i n PRA

Page E-14 Tw o-dimensional M onte Carlo A nalysis (2-D MCA) An ad vanced mod eling technique that uses two stages of rando m samp ling, also called nested loops, to dis tinguis h between variability and uncert ainty in exposure and toxicity variables . The first stage, often called the inner loo p, involves a com plete 1- D MCA simulatio n of variabi li ty in ri sk. I n the s econd stag e, oft en called the ou ter loo p, parameters of th e probabil ity d ist ribut ions are red efined to reflect uncertainty . These loo ps are rep eated ma ny times resulting in multiple risk distri butions, f

rom which con fiden ce inte rva ls are calculated t o represent uncert ainty in the po pulatio n distrib ution o f risk. Type I Errors False positive ; the erro r mad e whe n the nu ll hypo thesis is rejected i n favor of the alterna tive, whe n in fact the null hypo thesis is true. Type II Errors False negativ e; the err or m ade when the null hypo thesis is accepted when in fact the alte rnati ve hypo thesis is true.
Uncerta inty Lack of knowl edge about speci fic variables , parameters , mod els, or o ther facto rs. Examples in clude l imit ed data regarding t he concentrati on of a cont aminant in an enviro nme ntal me dium and la ck of info rmatio n on lo cal fish co nsum ption prac tices. Un certain ty may b e red uced throug h further study. Variability True het erogen eit y or di vers ity i n char acte ris ti cs amon g members of a popu lat ion (i. e., inter- indiv idual variabi lit y) or f or one indi vidual over ti me (int ra-i ndivi dual var iab il it y) . Fo r ex ampl e, body wei ght s of a st udy popul ati on at

one p oin t i n t ime will exhibi t vari abili ty, and body weigh t wil l chang e as an i ndivi dual ages . Furt her study (e .g., increa sing sam ple size , n ) wi ll not r educe v ari abil ity, but i t ca n provi de greate r con fidenc e in qu antitative c harac terizatio ns of va riability.
Va riab le A qua ntity that can assume many va lues.
Variance Measure of th e spread of a dist ribut ion, equa l to t

he square of

the standard deviat ion (SD). Calculated as the average of t he squares of the deviati ons of the observations from their mean . Var iance is referre d to a s the sec ond centra l moment of a di stri buti on
. Z-score The val ue of a norma ll y di str ibut ed ran dom var iable that has b een sta nda rdize d to have a mean of

ze ro an d a SD of

on e by th e t ra ns fo rm at io n Z =( X– : )/ F . Statist ical tables typ ically give th e area to the left o f the z-sco re valu e. Fo r exam ple, the area to the left of z =1.645 i s 0. 95. Z- score s i ndicate t he di rect ion ( +/- ) and number of standar d deviat ions away f rom the mean that a parti cular datum lies assuming X is norma lly distributed. Microso ft Excel’s NO RM SD IST(z) functi on gives the pro bab ility p suchtha t p=Pr( Z # z), wh ile the NORM SINV(p) functi on gives the z-scor e z

p assoc iated w ith pro bab ility p suchtha t p=Pr( Z # z p ). 

### E.1.0 A DDITIONAL 

### I NFORMATI ON Uncertainty: A Guid

e to Dealing with Uncer

tainty in Quantitative Risk and Policy Analysis (Morgan and Henrion, 1990) an

d Probabilistic Techniques in ExposureAssessment (Cullen and Frey , 1999) provide exce llent philoso

phical a nd practical tre atises on proba bilistic risk a ssessm

ent. The se work

s are hi ghly recommen

ded to risk assessors w

ho wish to kno

w more about p

robabilistic risk assessm

ent. The Summary Report for the Worksh op on Monte Carlo A nalysis (U .S . EPA, 1996) and the Summary Report for the Workshop on Selecting In

put Dis tributio ns for P

robabil istic As sessmen ts (U.S. EPA, 1999b ) are other sources of

information to learn

more about

PRA. Other additional refe rences f or readi ng are listed i n this Appendix.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-15 REFERENC ES FOR APPEN DIX E Cullen, A.C. an

d H.C. Frey . 1999. Probab ilistic T echniqu es in Ex

posure Assessm ent. A Handbook for Dealing with Variability

and Uncertainty in Models and I nputs. Plenum Press. Law, A.M. and W.D. Kelto n. 1991. Simulation Mode ling and Analysis. McGraw-Hill , Inc., NY. Morgan, G.M. an

d M. Henrion, 1990

. Uncertainty: A Guide to Dealing with Unc ertainty in Quantitative Risk an d Polic

y Analy sis . Cambridge University Press, NY. National Research Cou

ncil (NRC). 1983. Risk Assessment in the Federal Government: Managing the Process . National Academy Press. Was hington, DC.
U.S. EPA. 1 989. Risk Assessment Guida nce for Superfund (RAGS ): Volume I. Hum an Health Evaluation Manual (HHEM)(Part A, Baseline Risk Ass essment). Interim Final. Office of Em

ergency and Remedial Respo

nse, Washington, DC. EPA/540/1- 89/002. NTIS PB90-15 5581.
U.S. EPA. 1 990. National Oil an d Hazardous Su

bstances Pollution Contingency Plan. Fina l Rule. 40 CFR 30 0: 55 Federal Register, 8666-88 65, March 8.
U.S. EPA. 1 991. Risk Assessment Guida nce for Superfund (RAGS) , Volume I: Hum an Health Evaluation Manual (HHEM)

, Part B, Development of Risk-Based P reliminary Remediation G oals. Office of Emergency

and Remedial Response, Washing ton, DC. EPA

/540/R-92/003 . NTIS P B92-96333 3.
U.S. EPA. 1 992. Supplemental Guidance to RAGS: Calcula ting the Concentration T erm . Offic e of So lid Waste and Emergency Response, Washington, DC . OSWER Directiv e No. 9285.7- 081.
U.S. EPA. 1 997a. Memoran dum fro m Depu ty Admi nistrato r Fred Hansenon the Use ofProbab ilistic Techniques (including

Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte C arlo An alysis . Office of Research a nd Developm

ent, Washington, D C. EPA/630/R- 97/001. May 15.
U.S. EPA. 1 997b. Lognormal Distribution in Environmental App lications . Office of Research a nd Development, and Office of So lid Waste and Eme rgency Resp onse, Washington, DC. EPA/60 0/R- 97/006 . December.
U.S. EPA. 1997c. Ecological Risk Assessm ent Guidance for Sup erfund: Process for Designing and Conducting Eco logical Risk Assessments. Interim Final. U.S. Environmen tal Protection Agency , Environm ental Response Team

(Edison, NJ). June 5.
U.S. EPA. 1999a. I ssuance of Final Guidan ce: Ecological Risk

Assessment an

d Risk Manag ement Princip les for S uperfun

d Sites. U.S. Envi

ronme ntal Protection Agency, O ffice of Solid Waste and Emergency Response. OSWER Directi ve 9285.7-28 P. Stephen D. Luftig for Larry D. Reed. October 7.
U.S. EPA. 1 999b. Summa ry Repo rt for th e Wor kshop o n Selec ting Inp ut Distr ibution s for Pr obabili stic Risk Assessment . Risk Assessm ent Forum. E

PA/630/R-9 8/004.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-16 REFERENCES FOR F URTHER READING Baird, BF. 1989

. Manager ial Dec isions U

nder Un certain ty . John Wiley & Sons, Inc., NY. Bevington, P.R. 1969. Data Reduction and Error Analysis for the P hysical Sciences . McGraw-Hill, NY. Bratley, P., B .L. Fox, and L .E. Schrage. 1987. A Guide to Simulation . Springer-Verlag, NY.
Burmaster, D .E. an

d P.D. Anders on. 19 94. Pri nciples of Goo d Practi ce for the Use o f Monte C arlo Techniques in Hum

an Health and Ecolog

ical Risk Assessm

ent. Risk An al. 14(4):477–481. Clemen, R, 1990. Making Hard Decisions . Duxbury P ress. Conover, W.I . 1971. Practical Nonparametric Statistics . John Wiley & Sons, NY. Cox, D.C. an

d P. Baybutt. 1981. Methods for Uncertainty An alysis: A Comparative Survey

. Risk Anal 1(4):251–258. Cullen, A.C. an

d H.C. Frey . 1999. Probabilistic Techniques in Exposure Assessment . Plenum Press, NY. D’Agostino, R . and M.A. Stephens, eds. 1986. Goodness-of-Fit Techniqu es . Marcel Dekker, Inc., NY. Devroye, L. 1986. Non-Uniform Rando m Deviate Generation . Springer-Verlag, NY. Evans, M., N . Hastings, and B. Peacock. 1993. Statistical Distributions . John Wiley & Sons, NY. Finkel, A.M. 1 990. Confronting Uncertainty in Risk Management: A Guide for Decision-Makers

. Resources for th e Future, Washin gton, DC. Gilbert, R.O. 19 87. Statistical Methods for Environm ental Pollution Monitoring . Van Nostrand Reinhold, NY. Hamby, D.M. 1994. A

review of Techniques for Parameter Sens itivity Analy sis of Environm ental Models

. Environ. Monit. and A ssess . 32:135–154.
Hamme rsley, J.M. and D.C . Handscomb . 1964. Monte Carlo Methods . John Wiley & Sons, NY. Hertz, D.B. and H. Thom as. 1983. Risk Analysis and Its A pplications . John Wiley & Sons, NY. Hertz, D.B. and H. Thom as. 1984. Practical Risk Analysis - An Approach Thro ugh Case Studies . John Wiley & Sons, NY. Hoffman, F. O. and J.S. Ham monds. 1992

. An Intr

oductor y Guide to Unc ertainty

Analys is in Environmental and H

ealth Risk Assessment . ES/ER/TM-35, Martin Marietta. Hoffman, F. O. and J.S. Ham monds. 1994

. Propag

ation o f Uncer tainty i n Risk A ssessme

nts: Th e Need to Disting uish Be tween U

ncertai nty Due to Lac k of Kn owledg

e and U ncertai nty Due to Var iability

. Risk Anal 14(5):707–712.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-17 Iman, R.L. and W.J. Co

nover. 1982. A Distribution- Free Approach to I nducing Rank

Correlation Am

ong Input Variables. Commun. Stat, Part B 11:311–33 1. Iman, R.L. and J.C. Helton. 1988. An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Mod

els. Risk Anal . 8(1):71–90. Iman, R.L., J.M. Dav

enport, and D.K. Z eigler. 1980. Latin Hypercube Sam pling (A Program

Users Guide) . Technical Report SAN D 79:1473, Sandia

Laboratories, Albuque

rque. Johnson, M.E. 1987. Multivariate Statistical Simulation . John Wiley & Sons, NY. Johnson, N.L. and S. Kotz. 1970. Continuous Univariate Distributions . Vols. 1 & 2. John Wiley & Sons, NY. Johnson, N.L., S. Kotz, and A.W. Kemp. 1992. Univariate Discrete Distribu tions . John Wiley & Son s, NY. Kendall, M. and A. S tuart. 1979. Advanced Theory of St atistics, Vol ume I - Distributi on Theory, Volume II - Infe rence a nd Rela tionship . MacMillan, Inc., NY. Kennedy, W.J. and E. Gentle. 1980. Statistical Computing . Marcel Dekker, Inc., NY. LePage, R. and L. Billard. 199 2. Exploring the Limits of Bootstrap . John Wiley & Sons, NY. Lipton, J., W.D. Shaw, J. Holmes, and A. Patterson. 1995. Short Com munication: Selecting

Input Distributions for use in Monte Carlo Analysis. Regul. Toxico l. Phar macol. 21:192–198. McKone, T.E. and K.T. Bogen. 1992. Uncertainties in Health Risk Assessment: An Integrated

Case Based on Tetrachloroethy lene in California Ground water. Regul. Toxico l. Phar macol. 15:86–103. Megill, R.E., ed. 1985. Evaluating and Managing Risk . Penn

Well Bo oks, Tulsa, O K. Morgan, G.M. an

d M. Henrion. 1990

. Uncertainty: A Guide to Dealing with Unc ertainty in Quantitative Risk an d Polic

y Analy sis . Cambridge University Press, NY.
NCRP. 1996. Comme

ntary No. 14. A Guid

e for U ncertai nty Ana lysis inDose a nd Risk Assessments Related to Environmen tal Contamination . National Com mittee on Rad iation Program

s, Scie ntific Committee 64-17. Washington, DC. Palisade Corporation. 19 94. Risk An alysis a nd Simu lation A dd-In fo r Micros oft Exce l or Lot us 1-2-3

. Windows Versi on Release 3.0 User’ s Guide, Palisade Corporation, Newfi eld, NY. Press, W.H., B. P. Flannery, S .A. Teulolsky , and W.T. Vetterling. 1 989. Numerical Recipes in Pascal: the Art of Scientific Computin g . Cambridge University Press, NY. Press, W.H., S.A . Teulolsky, W. T. Vetterling, and B.P . Flannery. 19 92. Numeri cal Rec ipes in FORTRAN: the A rt of Scientific Computing . Cambridge University Press, NY.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-18 Press, W.H., S.A . Teulolsky, W. T. Vetterling, and B.P . Flannery. 19 92. Numerical Recipes in C: The Art of Scientific Computing . Cambridge University Press, NY. Read, T. and N. Cressie. 1988. Goodne ss-of-F it Statis tics for

Discre te Multiv

ariate D ata . SpringerVerlag, NY. Rohatgi, V.K. 1984

. Statistical Inference . John Wiley & Sons, NY. Rubenstein, R.Y. 1981. Simulation and the Monte Carlo Method . John Wiley & Sons, NY. Sachs, L. 1984. Applied Statistics - A Ha ndbook of Techniques . Springer-Verlag, NY. Saltelli , A and

J. Marivort. 19 90. Non-Pa ramet ric Stat istics in Sensiti vity Analy sis for M odel Ou

tput: A Comparison of Selected Techniques. Reliab. Eng. Syst. Saf . 28:299–253. Schneider, H. 198 6. Truncated and Censo red Distributions from N ormal Populations . Marcel Dekk er, Inc., NY. Seiler, F.A. 1987 . Error Propagation for Large Errors. Risk Anal . 7(4):509–518. Seiler, F.A. and J.L. Alvarez. 1996. On the Sele ction of Distributions for S tochastic Variables. Risk An al. 16(1):5–18. Slob, W. 1994. U

ncertainty Analy sis in Multiplicative Models. Risk Anal . 14(4):571–576. Smith, A.E ., P.B. Ry

an, and J.S. Evans. 1992. The Effect o f Neglecting Correlations when Propa gating Uncertainty and E stimating the Population Distribution of Risk. Risk Anal . 12(4):467474. Smith, R.L . 1994. Uses of Monte Carlo Simu lation for Human

Exposure Assessm

ent at a Superfund Site. Risk Anal 14(4):433–439. Sokal, R. an

d R. Rohlf. 198 1. Biometry: The Principles and Practice of Statistics in Biological Research

. Second Edition. W. H. Freeman & Co., NY.
U.S. EPA. 1 978. Source Assessment: An alysis of Uncertainty - Pr inciples and Applications

. EPA/600/2- 79-004.
U.S. EPA. 1 992a. Guidelines for Exposure Assessment

. Federal Register . 57(104):22888- 22938. May 29.
U.S. EPA. 1 992b. Guidelines for Carcinog enic Risk Assessment

. Federal Register

. 51(185):33992- 34003. May 29.
U.S. EPA. 1 996. Summa ry Repo rt for th e Wor kshop o n Monte Carlo A nalysis . Office of Research a nd Development, Washington, D C. EPA/630/R- 96/010.
U.S. EPA. 1 999. Guidelines for Carcinog enic Risk Assessment . Revi ew Dr aft . Ris k Asse ssmen t For um. Washington, DC. NCEA- F-0644.RAGS Volume 3 Part A ~ Process For Conducti ng Probabilisti c Risk Assessment Appendix E ~ Decemb er 31, 2001

Page E-19 U.S. EPA. 2 001. Guidelines for Carcinog enic Risk Assessment

. Federal Register . 66(230):59593- 59594. Novembe r 29. Wilks, D.S. 19 95. Statistical Methods in the Atmo spheric Sciences, An In troduction . Academic P ress, San Diego.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001

Page F-1 E XHIBIT F-1
E XAMPLES OF E LEMENTS OF THE W ORKPLAN FOR PRA 1. Statement of the ecological assessment endpoints and/or human risk 2. Summary of the point estimate risk assessment 3. Potential value added for risk management by conducting a PRA and proceeding to the subsequent tiers (quantify variability, uncertainty, or both) 4. Discussion of adequacy of environmental sampling for PRA (e.g., data quality issues) 5. Description of the methods and models to be used (e.g., model and parameter selection criteria) 6. Proposal and basis for probability distributions and point estimates 7. Methods for deriving the concentration term 8. Proposal for probabilistic sensitivity analysis 9. Method for dealing with correlations 10. Bibliography of relevant literature 11. Software (i.e., date and version of product, random number generator) 12. Simulation approach (e.g., iterations, Monte Carlo or Latin Hypercube sampling, time step) 

## A PPENDIX 

## F
W ORKPLAN AND 

## C HECKLIST FOR 

## PRA
F.0 I NTRODUCTION This appendix provides guidance on developing a workplan prior to the initiation of a probabilistic risk assessment (PRA), and using a checklist when reviewing a PRA. Like the quality assurance project plan (QAPP), the workplan for PRA generally should document the combined decisions or positions of the remedial project manager (RPM), risk assessor, and stakeholders involved in the risk assessment. Often there are many stakeholders in a risk assessment, and it is important to involve andengage all stakeholders early in the decisionmaking process. These are important steps that should save time and effort. 

### F.1.0 W ORKPLAN In general, PRAs may be developed by Environmental Protection Agency (EPA), EPA contractors, or a potentially responsible party (PRP) with appropriate EPA oversight. In each case, it is important to develop a workplan early in the risk assessment process. PRAs to be submitted by a contractor or PRP should generally be submitted for EPA review before commencing the analysis. The workplan should describe the software to be used, the exposure routes and models, and input probability distributions and their basis (e.g., relevance to the site-specific contamination and pathways), including appropriate literature references. Examples of the elements of a workplan are given in Exhibit F-1, as well as Exhibit 4-8 in Chapter 4 (Example Elements of a Workplan for Ecological PRA). It is important that the risk assessor and risk manager discuss the scope of the probabilistic analysis and the potential impact on the Remedial Investigation/Feasibility Study (RI/FS).

!

Given the time and effort that can be expected to be invested in conducting a

PRA, it is important that a workplan undergo review and approval by EPA, prior to proceeding with the assessment.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001

Page F-2 E XHIBIT F-2
K EY F OCAL P OINTS FOR PRA R EVIEW 1. Clarity of and conformation to objectives. 2. Scientific basis and documentation of input distributions and assumptions. 3. Model structure and computational mechanics. 4. Results, including, limitations, reasonableness, and clarity of documentation. The EPA generally will not accept probabilistic analysis where a workplan for the analysis has not been initially submitted to the Agency and approved by the Regional risk assessor and RPM. Exceptions to this process may be considered on a case-by-case basis. Conducting a PRA is an iterative process. In general, as new information becomes available, it should be used to evaluate the need to move to a higher tier. The decision to move an assessment to a higher tier of complexity should result in a revised workplan and consultation with the Agency. The previous PRA, and its sensitivity analysis, should be included in the revised workplan, along with a point estimate risk assessment based on any data collected as part of a lower tier. The assessment will often be restricted to the chemicals and pathways of concern that contribute the greatest risk. Throughout the process of developing the PRA, the EPA risk assessor and the personnel involved in developing the assessment should have a continuing dialogue to discuss the many Agency decisions and their potential impact on the assessment. This dialogue, along with interim deliverables, will help to ensure that the risk assessment report will meet the needs of the Agency and that any problems are identified and corrected early in the process. 

### F.2.0 F OCAL 

### P OINTS FOR 

### PRA R EVIEW In reviewing a PRA, it is recommended that a systematic approach be adopted to ensure that all key technical elements of the PRA are evaluated and potential weaknesses are identified. A review check list can facilitate this process and promote consistency in the reviews of PRAs. Such a list can be developed from EPA’s guiding principles (U.S. EPA, 1997) and other reviews on the subject of PRA quality review (e.g., Burmaster and Anderson, 1994). In general, the review of a PRA can be organized into four focal points listed in Exhibit F-2. PRAs can vary in complexity, from relatively simple to very complicated; thus, the review strategy may need to be customized for specific sites. 

### F.3.0 C HECKLIST FOR 

### R EVIEWERS The exposure pathways and chemicals considered in a PRA should be clearly stated and related to the assessment endpoint. Often, the simplest way of doing this is to use the site conceptual model. Table F-1 provides a list of major points that may be used to evaluate the quality of a probabilistic assessment. This is not an exhaustive list. The ultimate judgment of the acceptability of a PRA is the responsibility of the regional EPA personnel. The issues that a reviewer should focus on may be different for each assessment. The workplan and the assessment should address each of the items on the checklist, but the workplan may includeRAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001

Page F-3 additional items. The reviewer is responsible for ensuring that the workplan and the assessment are complete and of sufficient quality to help support a risk management decision under the National Contingency Plan (NCP). The report should include a discussion of the results of assessment and how they relate to the point estimate of risk and hazard. A clear and concise description of what the results mean is an important part of each report. 

### F.4.0 I NTERNAL AND 

### E XTERNAL 

### R EVIEW There are two levels of review that may be appropriate for a PRA. If an EPA reviewer feels the need for help with a review, other EPA personnel may be contacted formally or informally to provide additional review capabilities. The EPA personnel should also review the draft workplan for PRA to evaluate the appropriateness and consistency with Agency guidance. If EPA personnel are contacted early in the risk assessment process, the review can occur in a more productive and timely manner. When the issues at a particular site are complex or contentious, EPA reviewers may also wish toobtain the services of outside experts for peer review (U.S. EPA, 2000). According to EPA’s PeerReview Policy Statement dated June 7, 1994 (U.S. EPA, 1994), “Major scientifically and technically based work products related to Agency decisions normally should be peer-reviewed.” External peer review should be considered when allocating resources for a PRA. The EPA reviewers generally should select external peer reviewers who possess no bias or agenda regarding the process or methods of PRA.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001

Page F-4 Table F-1. Example of a Generic Checklist for Reviewers [2 pages]
Focal Point " Evaluation Criterion Objectives and Purpose Assessment Endpoints " Are the human health and/or ecological assessment endpoints clearly stated and consistent with the workplan? Benefits " Are the rationales for, and benefits of, performing the PRA clearly stated and consistent with the workplan? Site Conceptual Model " Is there a description or graphic representation of the receptors and pathways considered in the assessment? Has the PRA addressed each of the pathways for completeness (e.g., sources, release mechanisms, transport media, route of entry, receptor)? Separation of Variability and Uncertainty " What is the modeling strategy for separating variability and uncertainty in the PRA? Is this strategy consistent with the assessment endpoints? Model Structure and Computational Mechanics Flow Chart " Is a diagram of the computational sequence provided so that the pathways of inputs and outputs and data capture can be understood and easily communicated? 1-D MCA / 2-D MCA " Is a 1-D MCA or 2-D MCA being implemented in the PRA? What is represented by either or both dimensions? Algorithms " Are all algorithms used in the model documented in adequate detail to recreate the analysis? Integration " Are the algorithms used in numerical integration identified and documented? Dimensional Analysis " Has a unit analysis been conducted to ensure that all equations balance dimensionally? Random Number Generation " What random number generator is used in model computations? Is it robust enough? What reseeding approach is used to minimize repeated sequences? Input Distributions and Assumptions Variability and Uncertainty " Is there a clear distinction and segregation of distributions intended to represent variability from distributions intended to represent uncertainty? Data sources " Are the data or analysis sources used in developing or selecting the input distributions documented and appropriate for the site? Distribution Forms " Are the analyses used in selecting the form of the distribution adequately documented (i.e., understandable and repeatable by a third party?) Distribution Parameters " Are the analyses used to estimate the distribution parameters adequately documented? Distribution Tails " Do the estimation methods precisely depict the tails of the input distributions; how was this evaluated? Is there sufficient information to depict tails for empirical distributions? Are these estimated as exponential tails with bounding values? Truncations " Are any input distributions truncated? Do these truncations make sense? Should truncations be applied to any of the distributions? Concentration Term " Is the derivation of a point estimate or distribution for the concentration term adequately documented? Is sufficient information provided to enable the reviewer to recreate the concentration term? Variable Correlations " Have variable independence and correlations been addressed? Has the methodology for representing variable correlations in the model been documented and is it reasonable in terms of the variables, the site, and the statistical approach?RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001 Focal Point " Evaluation Criterion

Page F-5 Time Step " Has the basis for the time step used in the model been documented? Is a single time step used, or do variables have different time steps? Are the time steps conceptually reasonable for the variables; for the site? Has the time step been evaluated in the sensitivity analysis? Sensitivity Analysis " Has a sensitivity analysis been conducted? Are the methods used in the analysis statistically valid? What did the analysis reveal about uncertainties in the assessment and the relative contributions of input variables to uncertainty? Results of Modeling Completeness " Are all the exposure routes identified in the site conceptual model and workplan addressed in the model results? Has the PRA fulfilled the objectives and satisfied the purpose stated in the workplan? Point Estimate Calculation " Has a point estimate calculation, using mean or median values of the input distributions, been performed? How do these results compare with the central tendencies calculated with the probabilistic model? How do the reasonable maximum exposure (RME) estimates compare? Have the similarities or differences between risk estimates from the point estimate and probabilistic approaches been adequately addressed? Stability of Output Tails " Has the stability of the high-end tail of the risk distribution been adequately evaluated? How stable are the estimated tails (in quantitative terms?) Is this level of stability adequate to support the risk management decisions that the model is intended to support? Significant Figures " Is the number of significant figures used in the output reasonable and consistent with model uncertainty? Limitations " Are the strengths and weaknesses of the PRA methodology and limitations of the results for decision making clearly presented? Clarity " Are the results and conclusions clearly presented and consistent with model output (e.g., central tendency exposure (CTE) and RME identified in the Executive Summary along with discussion of uncertainty)? Graphics " Are there graphics included that show both the risk distribution and PRA results (e.g., CTE and RME risk)?RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix F ~ December 31, 2001

Page F-6 R EFERENCES FOR A PPENDIX F Burmaster, D.E. and P.D. Anderson. 1994. Principles of Good Practice for the Use of Monte Carlo Techniques in Human Health and Ecological Risk Assessment. Risk Anal. 14(4):477–481. U.S. EPA. 1994. Memorandum from Deputy Administrator Carol Browner on Peer Review and Peer Involvement at the U.S. Environmental Protection Agency . June 7. U.S. EPA. 1997. Memorandum from Deputy Administrator Fred Hansen on the Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment , and Guiding Principles for Monte Carlo Analysis . Office of Research and Development, Washington, DC. EPA/630/R–97/001. May 15.
U.S. EPA. 2000. Peer Review Handbook: 2 nd Edition. Science Policy Council. Washington, DC. EPA/100/B-00/001. December.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-1 

## A PPENDIX 

## G
F REQUENTLY 

## A SKED 

## Q UESTIONS FOR 

## PRA
I NTRODUCTION This section presents a few questions and answers relating to probabilistic risk assessment (PRA). The purpose of the frequently asked questions (FAQs) is to facilitate the understanding of PRA using a comparison with the traditional point estimate approach to risk assessment. The FAQs presented here provide an overview of PRA with pointers to more detailed, and often more technical, discussions in other parts of the guidance. (1) What is a risk assessment? Risk assessment is a tool for organizing available information to make inferences about the potential human health or ecological effects associated with exposure to hazardous materials. The National Contingency Plan (NCP) addresses the use of a baseline risk assessment at Superfund sites to determine whether risks to human health and the environment are unacceptable. The NCP implements the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) of 1980. Risk assessments traditionally provide single point descriptors of risk (e.g., a central tendency exposure (CTE) risk descriptor or a reasonable maximum exposure (RME) risk descriptor). As such, these types of risk assessments have been referred to as point estimate risk assessments. In 1983, the National Research Council (NRC) described the following four steps for conducting human health risk assessments:

• Hazard identification : the determination of whether a particular chemical is or is not causally linked to a particular health effect.

• Dose-response assessment : the determination of the relation between the magnitude of exposure and the probability of occurrence of the health effects in question.

• Exposure assessment : the determination of the extent of human exposure before or after application of regulatory controls.

• Risk characterization : a description of the nature and often the magnitude of human risk, including attendant uncertainty (NRC, 1983). Readers are referred to risk assessment guidance documents such as Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment) (U.S. EPA, 1989a), Risk Assessment Guidance for Superfund: Volume II. Environmental Evaluation Manual(U.S. EPA, 1989b), and Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments (U.S. EPA, 1997a) for more information about point estimate risk assessment methods and policies.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-2 (2) What is a probabilistic risk assessment (PRA)? Superfund risk assessments have traditionally provided single point estimates of risk. More recently, PRAs have been developed. A PRA is a risk assessment that provides a probability distribution, rather than a point estimate, of risk. A probability distribution conveys both a range of values and a likelihood of occurrence of each value. This may allow a risk assessor to make statements about the likelihood that risks will exceed a level of concern. The probability distribution for risk often represents variability in risk estimates for a potentially exposed population. This variability may be due to variability in exposure and/or toxicity. PRA may also be used to quantify uncertainty in risk estimates. This can be useful because it allows a risk assessor to make statements about the level of confidence in the likelihood that risks will exceed a level of concern. Probabilistic methods often use computer simulations to combine multiple probabilistic distributions in a risk equation. Monte Carlo analysis (MCA) is perhaps the most widely used probabilistic method in PRA (see Question #7). (3) How does PRA compare with the point estimate approach? A single point estimate of risk does not explicitly characterize associated variability or uncertainty. However, multiple point estimates of risk (e.g., CTE or RME) can begin to characterize variability in risk as they use different points on each input distribution for exposure). A PRA can characterize variability in risk by using the full distribution of variability in exposure parameters in the risk equations. Advanced PRA techniques can also quantitatively characterize uncertainty. In appropriate circumstances, results of a PRA can lead to more informed risk management decisions. A PRA can be more resource intensive than a point estimate risk assessment. Some PRAs can require greater effort than point estimate approaches to define model inputs (i.e., select and fit probability distributions), as well as additional steps in the planning, review, and communication of the risk assessment assumptions and results (see Chapter 6 and Appendix F). A PRA does not necessarily require more data than a point estimate approach, although it does provide a framework for incorporating more of the available information into the risk assessment. When information on important exposure variables is lacking, results from a point estimate approach and a probabilistic approach will be equally uncertain. If a decision is made to conduct a PRA, this does not replace a point estimate risk assessment. Results of point estimate approaches should still be presented along with results of probabilistic approaches in Tier 2 or Tier 3. (4) Why should I consider using PRA? PRA can have several advantages over the traditional point estimate approach to risk assessment. PRA can often provide a more complete characterization of risk; a quantitative description of the uncertainties in the risk estimates; more informative sensitivity analysis; the ability to make probabilistic statements about risk; the ability to know where specific risk levels are on the potential distribution of risk; an increased understanding of risks; and opportunities for improved communication and risk management decision making.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-3 (5) When should I consider using PRA? A PRA may be considered as early as the planning stages of a point estimate risk assessment or as late as after the completion of a point estimate risk assessment. Ideally, PRA should be considered as early as possible in the planning of risk assessment activities at a site so that sampling plans and data collection efforts may be appropriately directed. A PRA may be used when the risk management decision is not apparent and when the results of a PRA may inform the risk management decision. Often a risk management decision is not apparent when the site-specific risk estimate is close to the regulatory level of concern. The NCP discusses a generally acceptable range for cumulative excess cancer risk of 1E-06 to 1E-04 for protecting human health (U.S. EPA, 1990). Noncancer risks to human health and ecological health are generally characterized by a ratio of exposure to toxicity, called a Hazard Quotient (HQ) or Hazard Index (HI) for multiple contaminants. The point of departure for evaluating noncancer risks may vary from site to site, but a HQ of 1 may be a good starting point for risk management decisions. PRA may also be considered when the results of the point estimate risk assessment suggest that risks are clearly above a risk level of concern, and a preliminary remediation goal (PRG) is needed. Because PRA and point estimate risk assessments use different techniques for quantifying variability and uncertainty, they may support different PRGs. If the results are dramatically different, further steps may be warranted to reevaluate the choices for input variables - both the point estimates, and the probability distributions and parameters (including truncation limits) for the 1-D MCA. PRA will not be needed in many cases. Point risk estimates often produce results which are sufficient for making remedial decisions (e.g., sites are usually either heavily contaminated or only marginally contaminated). A tiered approach to risk assessment has been developed by Environmental Protection Agency (EPA) and is recommended for use in deciding when to move from point estimate risk assessments to PRAs of varying complexities. A workplan should be developed and submitted for review before beginning a PRA at any stage in the tiered process. As a general rule, if the potential value added by a PRA outweighs the additional resource required to conduct it, PRA may be warranted (see Chapter 2). (6) How is the risk distribution from PRA used for decision making?
The EPA’s RAGS Volume I (U.S. EPA, 1989a) and the NCP Preamble (U.S. EPA, 1990) state that the RME will generally be the principal basis for evaluating potential human health risks at Superfund sites. Ecological assessments also often consider an RME endpoint. The point estimate Superfund risk assessments use a combination of average and high-end input values to arrive at the RME. In PRA, risks are described by a probability distribution instead of a point estimate. To use a risk distribution for decision making, one needs to identify a percentile value that corresponds to the RME. EPA’s Guidelines for Exposure Assessment (U.S. EPA, 1992a) states that, “the high-end risk means risks above the 90 thpercentile of the population distribution”, and “the high-end estimator should not exceed the 99.9 thpercentile” due to uncertainty in specifying the upper tail of the input distributions in a Monte Carlo analysis. Similarly, the 90 thto 99.9 thpercentiles of the risk distribution are recommended in this guidance as the RME range for decision making in PRA. Selection of a single point within the RME range generally requires consideration of the level of uncertainty in the risk distribution. The EPA recommends that the 95 thpercentile of the risk distribution be used as a starting point for risk management decisions in the absence of site-specific information.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-4 (7) What is Monte Carlo Analysis (MCA)? MCA is a numerical technique for PRA. MCA was developed in the 1940's during the beginnings of the nuclear power industry. MCA combines statistical analysis with modern computational techniques to calculate risk estimates, by randomly choosing different sets of input values each time. Each calculation is an iteration and a set of iterations is called a simulation. The output of a simulation used for risk assessment is a continuous probability distribution, which can be displayed in a graph in the form of either a probability density function (PDF) or corresponding cumulative distribution function (CDF). Both displays represent the same distribution, but are useful for conveying different information. For example, the PDF for risk is a good way for displaying relative probability using an area under the bell-shaped curve. The CDF for risk is generally S-shaped and can be especially informative for illustrating the percentile corresponding to a particular risk level of concern (e.g., 95 thpercentile=1E-06). Other uses of PDFs and CDFs are presented in Chapter 1, Exhibit 1-3. In 1997, EPA published a policy accepting the use of MCA to perform human health and ecological risk assessments (U.S. EPA, 1997a). This guidance focuses on MCA as a method of quantifying variability and uncertainty. (8) What is the policy on using PRA to characterize variability or uncertainty in toxicity or dose response? In human health risk assessments, probability distributions for risk should reflect variability or uncertainty in exposure. In ecological risk assessments, risk distributions may reflect variability or uncertainty in exposure and/or toxicity (see Chapter 1, Sections 1.4 and 1.4.1, Item 3). Approaches to characterizing variability and uncertainty in toxicological information should reflect both the latest developments in the science of hazard and dose-response evaluation and consistent application of EPA science policy. This statement is consistent with the 1997 EPA Policy Statement presented in Section 1.4 above (U.S. EPA, 1997g). Probabilistic approaches to ecological dose-response assessment may be explored, as discussed and demonstrated in Chapter 4. This guidance does not develop or evaluate probabilistic approaches for dose-response in human health assessment and, further, discourages undertaking such activities on a site-by-site basis . Such activities require contaminantspecific national consensus development and national policy development. Parties wishing to undertake such activities should contact the OERR to explore ways in which they might contribute to a national process for the contaminant of interest to them. (9) What is the policy on using PRA at EPA and in Superfund? In the spring of 1997, EPA released the memorandum, Use of Probabilistic Techniques (including Monte Carlo Analysis) in Risk Assessment (U.S. EPA, 1997b)

. The policy states that probabilistic analysis techniques, “given adequate supporting data and credible assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk assessments.” As such, a PRA, “will be evaluated and utilized in a manner that is consistent with other risk assessments submitted to the Agency.” Together with this Policy Statement, the Agency released a set of guiding principles for use and review of probabilistic analyses. Hence, both RAGS and Agency-wide guidance emphasize the importance of review of the scientific and technical merit of a probabilistic analysis to determine whether or not the assessment is of sufficient quality to support a remedial decision. This guidance, RAGS Volume 3: Part A , provides risk assessors with comprehensive guidance on when and how to conduct PRAs using MCA within the Superfund program (see Preface and Chapter 1).RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-5 (10) What are the challenges of using PRA? Although PRA may have several advantages over the traditional point estimate approach to risk assessment, the use of PRA tends to be more resource intensive and may introduce some additional challenges to risk communication efforts. Risk communication helps build trust with the stakeholders and disseminate the risk information. In general, EPA staff and stakeholders are accustomed to a point estimate of risk and are unfamiliar with PRA and the quantitative estimates of uncertainty that PRA can support. Although, quantitative risk estimates may be more informative, they also may be more difficult to communicate and may not be well received due to stakeholder desires for certainty (Slovic, et al. 1979). Early and frequent communication with stakeholders is key in implementing PRA successfully. Often PRA requires additional data collection efforts as well as more time and resources to select and fit probability distributions.RAGS Volume 3 Part A ~Process For Conducting Probab ilistic Risk Assessment Appendix G ~ December 31, 2001

Page G-6 R EFERENCES FOR A PPENDIX G National Research Council (NRC). 1983. Risk Assessment in the Federal Government: Managing the Process . National Academy Press. Washington, DC. Slovic, P., B. Fischoff, and S. Lichtenstein. 1979. Rating the Risks. Environ. 21(3):14–20 and 36–39.
U.S. EPA. 1989a. Risk Assessment Guidance for Superfund (RAGS): Volume I. Human Health Evaluation Manual (HHEM) (Part A, Baseline Risk Assessment). Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/002. NTIS PB90-155581. U.S. EPA. 1989b. Risk Assessment Guidance for Superfund. (RAGS) : Volume II. Environmental Evaluation Manual . Interim Final. Office of Emergency and Remedial Response, Washington, DC. EPA/540/1-89/001. U.S. EPA. 1990. National Oil and Hazardous Substances Pollution Contingency Plan. Final Rule. 40 CFR 300: 55 Federal Register , 8666-8865, March 8.
U.S. EPA. 1992a. Final Guidelines for Exposure Assessment . EPA/600/Z-92/001. 57 Federal Register , 22888-22938, May 29.
U.S. EPA. 1997a. Ecological Risk Assessment Guidance for Superfund: Process for Designing and Conducting Ecological Risk Assessments . Interim Final. Environmental Response Team, Edison, NJ. EPA/540/R-97/006, OSWER Directive No. 9285.7-25, June. U.S. EPA. 1997b. Memorandum from Deputy Administrator Fred Hansen on the Use of ProbabilisticTechniques (including Monte Carlo Analysis) in Risk Assessment, and Guiding Principles for Monte Carlo Analysis . Office of Research and Development, Washington, DC. EPA/630/R-97/001. May.RAGS Volume 3 Part A ~Process For Conducting Probabilist ic Risk Assessment Appendix H ~ Decemb er 31, 2001

Page H-1 A PPENDIX H
I NDEX Applicabl e or Rele

vant and A

ppropriate Requirem

ents (AR AR ), 5-3, 1 8, 19 ; 7-1

backca lcul ati on, 5- 3, 10- 11 Bayesi an ana ly sis , D-16, 17-18 Benchmark Dose Sof

twar e (BMDS), 4-16, 17-19, 32-35 biom arker , 7-9 bivar iat e norma l di str ibution, B-45, 46- 49 bootst rap re sampl ing , 3- 12; 5- 7, 15; C-11

CDF (see cumulative dis tributi on function)
cens ored dat a, B- 27 cent ral li mit the orem, C-11 cent ral ten dencyexposure (CTE), 1-15, 16-17 chec klist, 1-29 ; 4-41 ; F-1, 4 cleanup goal , l evel , 5- 1, 3, 18-21; 7-13 conf idenc e in terval, 1-19; 3-12; 5-7; 6-16; C-11 continuous res ponse, 4-28, 29 correlation, and bi vari ate normal , B-46, 47-50 compari son wi th r egres sion anal ysi s, A-21, 34 part ial , A- 33 Pearson , A- 2, 26, 33
r-s quare, 4-12; 6-13; A-33, 34 simpl e, A- 2, 26 Spearman

rank , 3- 26; A- 26, 36; B-26 credi ble i nter

val, 1-19; 3-12, 16; 6- 16, 17 CDF (see cumulative dis tributi on function) CTE (see cen tral tendency exposure) cumul ati ve di str ibut ion f unct ion ( CDF)
com pare d with P DF , 1-12 ; 3-6, 7 -8; 4-1 4; 7-3
deterministic risk assessment (see p oint estimate risk
assessm ent) dichotomous r espons e, 4- 26, 27 distri bution (see probability distri bution) empir ical dis tri buti on functi on (EDF), 4-15; 5-13; B-8, 22, 37- 38 Expect ed Value of I nfor mati on (EVOI), 1-21; D-19, 20-24 expert judgme nt, 6-5; D-16, 17- 19 exposu re poi nt c oncent rat ion ( EPC), 3- 10; 5- 4, 6, 12 exposu re un it , 1- 18; 3- 10; 5- 4, 5- 20; C- 1, 2- 13

forwa rd ca lculation , 5-3

geost ati sti cs, 5-14; C-12, 13; D-10, 11-16 goodnes s-of

-f it (GoF) t est , 1- 29; B-31, 32-35 Anders on Darl ing (AD), B- 34 Chi- Square , A- 6; B-33 Kolmogorov -Smi rnov ( KS), B-33, 34 pro bab ility plot (see pro bab ility plot) Shapi ro-Wil k, B- 33

iter ativ e, reduction, 5-12, 13, 19- 21 truncation (s ee truncation)
join t proba

bil ity cur

ve, 4-30

kri gin g, 5- 9; D-15
Land Method, 5-7; C-12 Lati n Hyper cube Sampl ing , 3- 15, 17 logn ormal dis tri

buti on, 1- 11, 25; 3-4, 12-14; 5-7, 15; C-11

max imum entrop y, B-5 maximum li kelihood estimat ion ( MLE), B-25
measur ement

of at tai nment , 5- 21 method of mat chin g moment s, B-24, 25 M icroe xpo sure E vent A nalysis (M EE ), C-2; D -6, 7-9RAGS Volume 3 Part A ~Process For Conducting Probabilist ic Risk Assessment Appendix H ~ Decemb er 31, 2001

Page H-2 Monte Carl o, analysis, 1 -D M CA , 1-14 ; D-1 ; G-4 analysis, 2 -D M CA , 1-19 ; D-1 , 2-5 simu lat ion, 1-13
NCP, nin e cri ter ia, 1-6; 2-12, 16; 5- 1, 4; 7-1, 12 normal ized part ial deri vative, A-13, 32

paramet er es ti mati on cri ter ia, B-24, 25 part ial deri vative, A-13, 29-32, 36 PDF (see probability density f unction) poin t estimate risk asses smen t, compared w ith PRA, 1- 11, 17, 20-23; 4-7, 8-11; G-2 , 4, 5 prel imi nary

remedi ati on goal (PRG), al l of Chapte r 5; 7-11, 12-14 probability densit

y functi on (PD F),
com pare d with C DF , 1-12 ; 3-6, 7 -8; 4-1 4; 7-5
con cep t of pro bab ility density, 3-4 PDFu, 1-19; 3-12, 13-15; 4-31, 33-34; 5-8; 6-16 PD Fv, 1 -12, 1 -19, 2 0; 3-4 , 12-1 4; 4-3 1; 5-8 ; C-1, 2 -4 probability dist

ribution, continuous , 4- 16 disc ret e, 4- 16 prel imi nary

or sc reening l evel , 2- 6; 4- 44; B-1, 4-5 sel ecti on of, 3-5; B-34, 35 for dose r espons e or t oxici ty , 1- 27; 3- 6; 4- 15, 16-3 9; 7-8 , 9; G -4 probabi li ty mass fun cti on (PMF), 1- 11 probabi li ty plot , 5- 16; B-23, 24, 33- 34, 40- 46 proble m formu lat ion, 1-7, 22, 24, 28; 4- 2, 11, 42; 5-4, 15

quant it ati ve unc ert aint

y ana ly sis , C- 11

rand om v ariab le, 1-11 , 14; 4 -12; 5 -6 rank corre lat ion c oeff

ici ent, 3-26; 6-14
reas onable maxi mum expos ure ( RME), 1-15, 16-17 RME range, 1- 21, 26- 27; 7- 4, 11- 13 regr essi on analysi s, A-1, 32-36 mult ipl e, A- 2, 6, 8, 36 ste pwis e, A- 36 remedi ati on acti on le vel, 5-1, 7-8, 17 goa l, 1-6, 2 8; 2-1 4, 15 ; 5-1; 7 -1; C-2 repr esenta tiveness , 1-17 ; 3-5, 6 ; 4-7; 6 -10; 7 -6, 7-8 , C-3 risk chara cter izat

ion, 1-5, 8; 3- 1, 6, 9; 4- 2; 6- 10; 7- 6, 7-8; C -3 CTE and manageme nt of ecol ogica l ri sk, 4-38 communi cati on, 1- 4, 10, 25-26; 2-16; allof Chapte r 6; C-11 RM E (see reas onable maxi mum exposure)
sampl e si ze, 1- 18; 3- 6; 5- 8, 14; 6-9; C-9, 11; D-12, 13 Scienti fi c/Manage ment Decis ion Pl an (SMDP), 1- 9; 4-5, 8, 44, 46, 48 sensiti vity anal ysis, all of Appendix A role in th e tiered app roac h, 3-9, 2 1; A-3 simpl e corr elation c oeff ici ent, A-21 spat ial aut ocorre lat ion, C-12 speci es s ensi ti vit

y di str ibut ion ( SSD), 4- 20, 21, 24-25 stability, 3-1 7; 4-3 8; 7-6 , F-5 numeri cal, 1-15, 25; 3- 17 stakeh olde rs, types o f, 2-7, 8; 6 -4, 5 role in tier ed proces s, 1-4, 7; 2- 16; 3- 17, 18- 26; 4-39; 5-12; 6-1, 4, 7, 19

Thiess en pol ygons , D-13, 14-15 ti ered appr oach, 1-9, 26-28; 2-9, 10-18; 4-40, 41; 3- 17; 5-9, 1 0; 6-6 time step , C-4; F -5 toxi cit y re fer ence v alue (TRV), 4- 6; 4- 9; 4- 15, 16, 20-24, 32-35, 38 trunca tion, probabi li ty dis tri

buti ons, 3-6, 13-15, 25; B-30, 31-32 iterative m ethod for P RG s, 5-12 13-2 1; F-4RAGS Volume 3 Part A ~Process For Conducting Probabilist ic Risk Assessment Appendix H ~ Decemb er 31, 2001

Page H-3 unce rtainty, mod el, 1-17 , 18; 3 -11, 1 7; 4-6 paramet er, 1-17, 18; 3- 11, 12- 16
scen ari o, 1- 18; 3- 11, 17 upper c onfi dence l imi t (UCL), 5-4, 5; C- 11

value of inf

ormation (VOI) (see EV OI) variab ility, and concentration term, all of Appendix C inter-ind ividua l, 3-1; C -1 intra-ind ividua l, C-1, 2 spatial, C -3, 4-7 temp oral, C -2, 3-7
workpl an, 1-27; 2-1, 4; 4- 39, 40, 44, 46; allof Append ix F

z-s core, C-10, 11