EPA/630/P-02/001F 

May 2003 

Framework for Cumulative Risk Assessment

Risk Assessment Forum

U.S. Environmental Protection Agency

Washington, DC 20460
Disclaimer 

This document has been reviewed in accordance with the U.S. Environmental Protection 

Agency (EPA or the Agency) peer and administrative review policy. It has had extensive 

peer involvement in development (including other Federal, State, and public involvement), it has 

completed the formal external technical peer review process, and it has been administratively 

reviewed under the auspices of the Agency Science Policy Council. The mention of 

commercial products is for illustration only and in no way implies EPA endorsement of these 

products. 

ii 
Table of Contents 

List of Figures ............................................................... vi

List of Text Boxes ........................................................... vii

Acknowledgments ............................................................ ix

Foreword ....................................................................x

Preface ..................................................................... xi

Authors, Contributors, and Reviewers ............................................ xiii

List of Abbreviations and Acronyms ............................................. xvi

Executive Summary ......................................................... xvii

1. INTRODUCTION ..........................................................1

1.1. Purpose and Scope of the Framework Report ..............................4

1.2. Intended Audience ...................................................6

1.3. Key Definitions in Cumulative Risk Assessment ...........................6

1.4. The Cumulative Risk Assessment as a Tool for a Variety of Users

and Purposes .......................................................9

1.5. The Broader Decision-Making Context for Cumulative Risk Assessment .......11

1.6. Organization of This Report ..........................................12

2. THE PLANNING, SCOPING, AND PROBLEM FORMULATION PHASE ...........14

2.1. Planning and Scoping ...............................................14

2.1.1. Defining the Purpose of the Assessment .........................17

2.1.2. Defining the Scope of Analysis and Products Needed ...............18

2.1.3. Agreeing on Participants, Roles, and Responsibilities ...............19

2.1.4. Agreeing on the Depth of the Assessment and the Analytical

Approach ..................................................22

2.1.5. Agreeing on the Schedule and Resources Available ................22

2.1.6. Review of Lessons Learned ...................................22

2.2. Problem Formulation, Conceptual Model, and Analysis Plan .................24

2.2.1. Problem Formulation ........................................24

2.2.2. Developing the Conceptual Model ..............................25

2.2.3. Constructing the Analysis Plan .................................28

2.2.4. An Early Look at Uncertainty ..................................29

2.3. Ecological versus Human Health versus “Integrated” Cumulative

Risk Assessment ...................................................30

2.4. The Final Step Before the Analysis Phase: Discussion of Possible Outcomes ....31

iii 
3. THE ANALYSIS PHASE ...................................................34

3.1. General Process ....................................................34

3.2. Available Methods and Approaches ....................................36

3.2.1. Examples of Increased Complexity of Cumulative Risk Assessment ...37

3.2.2. Approaches for Predicting Risk of Multiple Stressors ...............43

3.2.2.1. Single stressor information ............................44

3.2.2.2. Information on stressor interactions and multiple

exposures ..........................................46

3.2.2.3. Decision indices .....................................48

3.2.2.4. Probabilistic approaches ..............................50

3.3. Areas of Complexity and Current Research ..............................50

3.3.1. Interactions Between Stressors and Other Factors ..................51

3.3.2. The Promise of Biomarkers and Biomonitoring ....................52

3.3.3. A Single Metric for Multiple Types of Hazard .....................54

3.3.3.1. Creating a common metric .............................55

3.3.3.2. General issues regarding a single metric ..................57

3.3.4. Qualitative Approaches .......................................58

4. THE RISK CHARACTERIZATION PHASE ....................................59

4.1. Risk Description ...................................................61

4.2. Uncertainty Analysis ................................................63

4.2.1. Assumptions in the Assessment ................................64

4.2.2. Uncertainty and Variability ...................................65

4.2.3. Uncertainty and Risk Addition .................................66

4.2.4. Other Cumulative Risk Assessment Uncertainties .................67

4.3. Information Provided by Cumulative Risk Assessment .....................67

4.3.1. Making Sense of Multiple-Stressor Effects .......................67

4.3.2. Cumulative Risk Assessments in a Public Health Context ............68

4.3.3. How the Scope and Purpose of the Assessment Affect Results ........69

4.3.4. Documenting Stakeholder Input ................................69

4.4. Using the Results of the Assessment ....................................69

5. GLOSSARY ..............................................................72

REFERENCES ..............................................................75

APPENDIX A: RESEARCH AND DEVELOPMENT NEEDS ........................91

APPENDIX B: SELECT RESOURCES FOR EXPOSURE AND RISK ASSESSMENT ....94

B.1. Resources Relevant to Chemical Exposures ..............................94

B.2. Resources Relevant to Exposures to Nonchemical Stressors .................96

APPENDIX C: SOME THOUGHTS ON BACKGROUND EXPOSURES ...............98

iv 
APPENDIX D: EXAMPLES OF OUTLINES OF ANALYSIS PLANS .................100

D.1. Outline for Human Health Analysis Plan for Pesticides Under the 1996 Food

Quality Protection Act (FQPA) ......................................100

D.2. Outline for Ecological Analysis Plan ..................................101

APPENDIX E: TOXICOLOGIC SIMILARITY—ORGANOPHOSPHORUS

PESTICIDES .........................................................102

APPENDIX F: OTHER TYPES OF CUMULATIVE ASSESSMENTS .................105

F.1. Quality-of-Life Assessments .........................................105

F.2. Cumulative Impact Assessments ......................................107

F.3. Empirically Derived Medical Models ..................................108

F.4. Risk Surrogates ...................................................109

v 
List of Figures 

1-1. A chemical- (or stressor-) focused assessment starts with a source and evaluates how the

chemical gets to various populations or ecological targets ........................1

1-2. Population-based assessments start with the receptors and determine which chemicals,

stressors, or other risk factors are affecting them ...............................2

1-3. Framework for cumulative risk assessment ...................................13

2-1. The Planning, Scoping, and Problem Formulation phase ........................15

2-2. An example of a generic conceptual model ...................................26

2-3. Specific conceptual model for a complex project, OAQPS’ National Scale Air

Toxics Assessment ......................................................27

3-1. The Analysis phase .....................................................35

4-1. The Risk Characterization phase ...........................................60

vi 
List of Text Boxes 

Examples of Cumulative Risk Assessment Activities within EPA in 2002 ............... xix

EPA’s Risk Assessment Guidelines ...............................................5

FQPA’s Terminology Interpretations ..............................................7

Cumulative Risk Assessment Features ............................................10

Core Principles of Community-Based Environmental Protection ........................11

Possible Management Goals ....................................................17

Examples of Cumulative Risk Assessment Scopes ...................................18

Examples of Possible Interested or Affected Parties (Stakeholders) ......................20

Guidelines for Stakeholder Involvement ...........................................21

Reed Holderman’s Lessons Learned ..............................................23

The Science Advisory Board’s Desired Outputs for Problem Formulation ................24

Important Details for an Analysis Plan ............................................28

Cumulative Risk Initiative (CRI) for Cook County (IL) and Lake County (IN) .............32

Scope of EPA’s National-Scale Assessment for Hazardous Air Pollutants ................36

The National-Scale Air Toxics Assessment ........................................38

Examples of Exposure Models that Consider Time Aspects ............................40

Toxicologic Similarity: The Dioxin Reassessment ...................................46

The Hazard Index .............................................................49

Risk Characterization Guiding Principles ..........................................61

Some Thoughts on Risk Characterization ..........................................62

Morgan and Henrion’s Ten Requirements for Good Policy Analysis .....................63

vii 
The Cumulative Exposure Project ................................................65

The European Environment Agency’s 12 Lessons Learned Late ........................71

Vermont’s Quality-of-Life Criteria ..............................................106

NEPA’s “Cumulative Impact” Definition .........................................107

viii 
Acknowledgments 

This U.S. Environmental Protection Agency (EPA or the Agency) report has been 

developed under the auspices of EPA’s Risk Assessment Forum, a standing committee of EPA 

scientists charged with developing risk assessment guidance for Agency-wide use. An 

interoffice Technical Panel chaired by Michael A. Callahan (Region 6) was commissioned to 

write this report. Other members of the Panel are Edward S. Bender (Office of Science Policy), 

George L. Bollweg (Region 5), Vicki L. Dellarco (Office of Pesticide Programs), Lynn A. 

Delpire (Office of Pollution Prevention and Toxics), Martin P. Halper (Office of Environmental 

Justice), Richard C. Hertzberg (National Center for Environmental Assessment), Elizabeth Lee 

Hofmann (Office of Emergency and Remedial Response), R. Craig Matthiessen (Chemical 

Emergency Preparedness and Prevention Staff), Alexander McBride (Office of Solid Waste), 

Deirdre L. Murphy (Office of Air Quality Planning and Standards), Henry C. Topper (Office of 

Pollution Prevention and Toxics), and Winona Victery (Region 9). 

In addition, Carole Braverman (Region 5), Loren Hall (Office of Civil Rights), and 

Denis R. Borum (Office of Water) participated for part of the duration of the panel. Steven M. 

Knott of the Risk Assessment Forum staff coordinated the project and also participated as a 

member of the panel. The Technical Panel would also like to thank Forum staff members 

Lowell F. Smith, Marilyn D. Brower, and Crystal M. Samuels for their assistance. 

Peer involvement and peer review were provided by scientists from EPA, other Federal 

agencies, State agencies, academia, public interest groups, and the private sector; their names are 

listed in the section titled “Authors, Contributors, and Reviewers.” 

ix 
Foreword 

Several reports have highlighted the importance of understanding the accumulation of 

risks from multiple environmental stressors. Among these reports are the National Research 

Council’s 1994 Science and Judgment in Risk Assessment and the 1997 report by the 

Presidential/Congressional Commission on Risk Assessment and Risk Management, Risk 

Assessment and Risk Management in Regulatory Decision-Making. In addition, legislation such 

as the Food Quality Protection Act of 1996 (FQPA), has directed the U.S. Environmental 

Protection Agency (EPA, or the Agency) to move beyond single chemical assessments and to 

focus, in part, on the cumulative effects of chemical exposures occurring simultaneously. Some 

of the cases filed with EPA under Title VI of the 1964 Civil Rights Act further emphasize the 

need for EPA to develop methods to assist consideration of cumulative risks. 

The Superfund program began conducting cumulative risk assessments at hazardous 

waste sites as early as the 1980s. More recently, in response to the increasing interest in 

cumulative risk, several other EPA programs have begun to explore approaches to cumulative 

risk assessment. In 1997, the EPA Science Policy Council issued a guidance on planning and 

scoping for cumulative risk assessments. More recently, the Office of Pesticide Programs has 

developed cumulative risk assessment guidance focused on implementing certain provisions of 

FQPA. In addition, the Office of Air Quality Planning and Standards is performing a nationalscale cumulative assessment of human health risks posed by outdoor air exposures to a set of 33 

priority urban air toxics. 

The EPA Science Policy Council has asked the Risk Assessment Forum to begin 

developing Agency-wide cumulative risk assessment guidance that builds from these ongoing 

activities. As a first step, a technical panel convened under the Risk Assessment Forum has been 

working to develop a framework for cumulative risk assessment. This document is the result of 

that technical panel’s efforts. Building from the Agency’s growing experiences, this framework 

is intended to identify the basic elements of the cumulative risk assessment process. It should 

provide a flexible structure for the technical issues and define key terms associated with 

cumulative risk assessment. Further efforts and experience in the coming years should advance 

our knowledge beyond the framework stage to a future set of Agency guidelines for cumulative 

risk assessment. 

William P. Wood, Ph.D.

Executive Director

Risk Assessment Forum

x 
Preface 

In the past several years, cumulative risk assessment, aggregate exposure assessment, 

and research on chemical mixtures has taken on increased importance, as evidenced by reports 

such as the National Research Council’s (NRC’s) Pesticides in the Diets of Infants and Children 

(NRC, 1993) and Science and Judgment in Risk Assessment (NRC, 1994), the National Academy 

of Public Administration’s Setting Priorities, Getting Results (NAPA, 1995), the 

Presidential/Congressional Commission on Risk Assessment and Risk Management’s Risk 

Assessment and Risk Management in Regulatory Decision-Making (PCCRARM, 1997), and the 

U.S. Environmental Protection Agency’s (EPA’s or the Agency’s) Science Advisory Board 

Toward Integrated Environmental Decision-Making (USEPA, 2000a). 

In addition, recent legislation mandates consideration of cumulative risk and

variability factors in the risk characterization process. Specifically, the Food Quality

Protection Act of 1996 (FQPA) (PL 104-170, August 3, 1996) directs EPA in its assessments

of pesticide safety to focus, in part, on the cumulative effects of pesticides that have a common 

mechanism of toxicity, considering aggregate dietary and nonoccupational pathways of 

exposure. 

Assessing cumulative risk through complex exposures is one of the Agency’s high 

priorities, especially in light of FQPA mandates, and it is germane and of great interest to all 

program and regional offices. This area of research is also directly applicable to children’s risk 

issues. The framework presented in this document is meant to lay out broad areas where analysis 

might be conducted if needed. It is not suggested that cumulative risk assessment is a tool that 

should be used with every issue or that all areas of analysis outlined or discussed here must—or 

even should be—conducted in every such assessment. The scope of the assessment will define 

the areas to be analyzed. For some areas discussed in this framework, the methodology for 

conducting the risk analysis may not yet exist. 

According to an expert panel report (USEPA, 1992a), a key role of science at EPA is to 

reduce uncertainties in the information used for environmental decision making. The report 

points out that although many EPA programs have historically focused on chemical-specific 

impacts, methods to assess or control the effects of chemical mixtures and general stressors on 

human health and ecosystems remained to be developed. 

NRC (1993) has recommended that all exposures to pesticides—dietary and 

nondietary—need to be considered when evaluating the potential risks to infants and children. 

Estimates of total dietary exposure should be refined to consider intake of multiple pesticides 

that have a common toxic effect. Further, the report identifies important differences in 

susceptibility with age. NRC (1994) has also stated that health risk assessments should generally 

consider all possible routes by which people at risk might be exposed and recommends this 

approach universally in the assessment of hazardous air pollutants regulated by EPA under the 

Clean Air Act Amendments of 1990 (P.L. 101-549, November 15, 1990). 

xi 
Regarding variability, the NRC (1994) recommends that EPA assess risks to infants and 

children whenever it appears that their risks might be greater than those of adults. The report

also encourages EPA to recognize the possibility of synergistic interactions when multiple chemical

exposures occur, and to consider extreme variability among individuals in their responses to 

toxic substances. A related issue is the problem of how risks associated with multiple chemicals are

to be combined. EPA hopes to begin systematically addressing these issues in this framework.

Finally, FQPA requires research on the influence of complex exposures on noncancer 

human health effects of pesticides and other toxic substances. 

Cumulative risk is also an important issue with the general public. In public meetings of 

Superfund stakeholders held in late 1996 in San Francisco and Washington, DC, and in early 

1998 in Atlanta, the issue of cumulative risk was raised several times in each session (USEPA, 

1996a, 1998a). 

Cumulative risk assessments will identify the need for many different kinds of 

data—some of which are not commonly used in current risk assessment—and they will often 

demand large quantities of such data. Until such data can be provided, identification of critical 

information and research needs may be the primary result of many cumulative risk assessment 

endeavors. 

As of August 1, 2001, there were 19,533 pesticide products on the market (USEPA, 

2001a) and 79,120 existing chemicals on the Toxic Substances Control Act inventory (USEPA, 

2001b). Each year, a number of chemicals are added. Assessing the cumulative effect of these 

chemicals will be a great challenge to to the field of risk assessment and to the Agency.

xii 
Authors, Contributors, and Reviewers 

Authors: 

EPA Risk Assessment Forum Cumulative Risk Technical Panel: 

Michael A. Callahan, Office of the Regional Administrator, Region 6 (Chair)

Edward S. Bender, Office of Science Policy, ORD

George L. Bollweg, Air and Radiation Division, Region 5

Vicki L. Dellarco, Office of Pesticide Programs, OPPTS

Lynn A. Delpire, Office of Pollution Prevention and Toxics, OPPTS

Martin P. Halper, Office of Environmental Justice, OECA

Richard C. Hertzberg, National Center for Environmental Assessment, ORD

Elizabeth Lee Hofmann, Office of Emergency and Remedial Response, OSWER

Steven M. Knott, Risk Assessment Forum, ORD

R. Craig Matthiessen, Chemical Emergency Preparedness and Prevention Staff, OSWER

Alexander McBride, Office of Solid Waste, OSWER

Deirdre L. Murphy, Office of Air Quality Planning and Standards, OAR

Henry C. Topper, Office of Pollution Prevention and Toxics, OPPTS

Winona Victery, Policy and Management Division, Region 9

Contributors and Reviewers: 

EPA Reviewers: 

Donald G. Barnes, EPA Science Advisory Board

Michael Firestone, Office of Children’s Health Protection

Debra L. Forman, Region 3

Audrey Galizia, Region 2

Mark Maddaloni, Region 2

Elizabeth H. Margosches, Office of Pollution Prevention and Toxics 

Marian Olsen, Region 2

James Rowe, Office of Research and Development

Scott Schwenk, Office of Research and Development

Michael Shapiro, Office of Solid Waste

Sally Shaver, Office of Air and Radiation

Margaret Stasikowski, Office of Pesticide Programs

Jeannette Wiltse, Office of Water

Jeffrey Yurk, Region 6

The following persons participated in peer involvement workshops held during 2001 to discuss and comment on the

draft framework.

Federal Scientists’ Meeting, May 1, 2001: 

Michael A. Babich, Health Sciences Division, U.S. Consumer Product Safety Commission

Michael Buchman, Office of Response and Restoration, National Oceanic and Atmospheric Administration

Stephen Channel, Department of the Air Force, U.S. Department of Defense

Christopher T. DeRosa, Division of Toxicology, Agency for Toxic Substances and Disease Registry

Dennis Druck, Army Center for Health Promotion and Preventive Medicine, U.S. Department of Defense

Frank Hearl, Division of Respiratory Disease Studies, National Institute for Occupational Safety and Health

Keith Hoddinott, Army Center for Health Promotion and Preventive Medicine, U.S. Department of Defense

Steve Jones, Office of Regional Operations, Agency for Toxic Substances and Disease Registry

xiii 
Brian Kaplan, Division of Health Assessment and Consultation, Agency for Toxic Substances and Disease Registry

Ralph Kodell, National Center for Toxicological Research, U.S. Food and Drug Administration

Dan Lewis, Health Effects Laboratory Division, National Institute for Occupational Safety and Health

Margaret MacDonell, Argonne National Laboratory, U.S. Department of Energy

Mark McClanahan, National Center for Environmental Health, Centers for Disease Control and Prevention

Anita K. Meyer, HTRW Center of Expertise, U.S. Army Corps of Engineers

Moiz G. Mumtaz, Division of Toxicology, Agency for Toxic Substances and Disease Registry

Leslie G. Rubin, Animal and Plant Health Inspection Service, U.S. Department of Agriculture

Antoinette Sebastian, Office of Community Planning and Development, Dept. of Housing and Urban Development

Scott D. Sudweeks, Div. of Health Assessment and Consultation, Agency for Toxic Substances and Disease Registry

Timothy Taylor, Army Center for Health Promotion and Preventive Medicine, U.S. Department of Defense

John Wheeler, Division of Toxicology, Agency for Toxic Substances and Disease Registry

Jody Wireman, Department of the Air Force, U.S. Department of Defense

States’ Meeting, May 3, 2001: 

George Hagevik, National Conference of State Legislators (Moderator)

Glen Andersen, National Conference of State Legislators

Ligia Mora-Applegate, Florida Department of Environmental Protection

Alex M. Barron, Virginia Department of Environmental Quality

Sharon Baxter, Office of Pollution Prevention, Virginia Department of Environmental Quality

Joseph P. Brown, Office of Environmental Health Hazard Assessment, California Environmental Protection Agency

Michele F. Burgess, Georgia Department of Natural Resources

Perry Cohn, Environmental Health Service, New Jersey Department of Health

Linda J. Cullen, New Jersey Department of Environmental Protection

Susan Dempsey, Nebraska Department of Health and Human Services

Michael J. DiBartolomeis, Office of Environmental Health Hazard Assessment, California Env. Protection Agency

Michael Ellenbecker, Toxics Use Reduction Institute, University of Massachusetts at Lowell

Joann Held, Office of Air Quality, New Jersey Department of Environmental Protection

Bruce Hope, Oregon Department of Environmental Quality

Tom Hornshaw, Toxicity Assessment Unit, Illinois Environmental Protection Agency

Paul B. Jiapizian, Maryland Department of the Environment

David R. Johnson, Bureau of Environmental Epidemiology, Florida Department of Health

Michael Juras, Community Right-to-Know, South Carolina Department of Health and Environmental Control

Kevin Kessler, Air and Waste Division, Wisconsin Department of Natural Resources

Rosanna Kroll, Health and Toxicology Section, Maryland Department of Environment

Charles Moore, Office of Environmental Management, South Carolina Department of Natural Resources

Rebecca Pottratz, Communication Specialist, Wisconsin Department of Natural Resources

John Ridgway, Environmental Justice Program, Washington State Department of Ecology

Kirpal S. Sidhu, Environmental Epidemiology Division, Michigan Department of Community Health

Lawrence P. Sirinek, Division of Emergency and Remedial Response, Ohio Environmental Protection Agency

Alan Stern, Bureau for Risk Analysis, New Jersey Department of Environmental Protection

Chuck Stroebel, Environmental Research Scientist, Minnesota Department of Health

Barbara S. Taylor, Department of Health and Human Services, West Virginia Bureau for Public Health

Mark Werner, Wisconsin Division of Public Health

Carol Rowan West, Office of Research and Standards, Massachusetts Department of Environmental Protection

Victor R. Wilford, Department of Health and Human Services, West Virginia Bureau for Public Health

Stephen L. Wust, Drinking Water Bureau, New Mexico Environment Department

Public Meeting, August 22-23, 2001: 

xiv 
Ken W. Sexton, Center for Environment and Health Policy, University of Minnesota (Moderator)

Elizabeth Boa, American Chemistry Council, Inc.

Kenneth T. Bogen, Health and Ecological Assessment Division, Lawrence Livermore National Laboratory

James P. Butler, Environmental Assessment Division, Argonne National Laboratory

Douglas J. Crawford-Brown, Dept. of Environmental Sciences and Engineering, University of North Carolina

Amy D. Kyle, School of Public Health, University of California-Berkeley

Norris McDonald, Center for Environment, Commerce and Energy

Beth E. Mileson, Technology Sciences Group, Inc.

Stephen S. Olin, Risk Science Institute, International Life Sciences Institute

William R. Rhyne, H&R Technical Associates, Inc.

Jennifer Sass, Health Program, Natural Resources Defense Council

Margo Schwab, School of Public Health, Johns Hopkins University

External Peer Review Workshop, June 4-5, 2002: 

Peter deFur, Environmental Stewardship Concepts (Peer Review Panel Chair)

Gail Charnley, Health Risk Strategies

Robert W. Collin, University of Oregon

David F. Goldsmith, George Washington University

Dale B. Hattis, Clark University

Joann Held, New Jersey Department of Environmental Protection

Paul A. Locke, Johns Hopkins University

Margaret M. MacDonell, Argonne National Laboratories, U.S. Department of Energy

Pavel Muller, ToxProbe, Inc.

P. Barry Ryan, Emory University

Jennifer Sass, Natural Resources Defense Council

Paul M. Schlosser, CIIT Centers for Health Research

Curtis C. Travis, Quest Technologies

Written Public Comments: 

James M. Solyst, American Chemistry Council, Inc.

Andy Lawrence, U.S. Department of Energy

Joe J. Mayhew, American Chemistry Council, Inc.

xv 
List of Abbreviations and Acronyms 

APEX - Air Pollution EXposure model

ATSDR - Agency for Toxic Substances and Disease Registry

CARES - Cumulative and Aggregate Risk Evaluation System

CBEP - Community-Based Environmental Protection

DALY - Disability-Adjusted Life Year

EPA - U.S. Environmental Protection Agency

FQPA - Food Quality Protection Act

GIS - Geographical Information System

HAP - Hazardous Air Pollutant

HEC - Human Equivalent Concentration

HRS - Hazard Ranking System

HUD - U.S. Department of Housing and Urban Development

IED - Integrated Environmental Decision-making

ILSI - International Life Sciences Institute

LADD - Lifetime Average Daily Dose

LLE - Loss of Life Expectancy

LOAEL - Lowest-Observed-Adverse-Effect Level

MSDS - Materials Safety Data Sheet

NAAQS - National Ambient Air Quality Standards

NATA - National Air Toxics Assessment

NEPA - National Environmental Policy Act

NHEXAS - National Human Exposure Assessment Survey

NOAEL - No-Observed-Adverse-Effect Level

NRC - National Research Council

OAR - Office of Air and Radiation (EPA)

OP - Organophosphorous

OPP - Office of Pesticide Programs (EPA)

OPPTS - Office of Prevention, Pesticides, and Toxic Substances (EPA)

ORD - Office of Research and Development (EPA)

PAH - Polycyclic Aromatic Hydrocarbon

PCB - Polychlorinated Biphenyl

pNEM - Probabilistic NAAQS Exposure Model

QALY - Quality-Adjusted Life Year

RfC - Reference Concentration

RfD - Reference Dose

SAB - Science Advisory Board

SHEDS - Stochastic Human Exposure and Dose Simulation model

TEAM - Total Exposure Assessment Methodology

TEF - Toxicity Equivalence Factor

xvi 
Executive Summary 

This report, Framework for Cumulative Risk Assessment, is the first step in a long-term 

effort to develop cumulative risk assessment guidelines. Its primary purpose is to offer a simple, 

flexible structure for conducting and evaluating cumulative risk assessment within the U.S. 

Environmental Protection Agency (EPA, or the Agency). Although this framework report will 

serve as a foundation for developing future guidelines, it is neither a procedural guide nor a 

regulatory requirement within EPA, and it is expected to evolve with experience. This report is 

intended to foster consistent approaches to cumulative risk assessment within EPA, identify key 

issues, and define terms used in these assessments. 

This framework is meant to lay out broad areas where analysis might be conducted if 

needed. It does not suggest that cumulative risk assessment is a tool that should be used with 

every issue, nor does it suggest that when cumulative risk assessment is applied, that all areas of 

analysis outlined or discussed here must—or even should be—conducted in every assessment. 

The scope of the assessment will define the areas to be analyzed. In some areas discussed in this 

framework, the methodology for doing the risk analysis may not yet exist. Appendix A includes 

a summary of areas where research is needed. 

In this report, “cumulative risk” means “the combined risks from aggregate exposures to 

multiple agents or stressors.” Several key points can be derived from this definition of 

cumulative risk. First, cumulative risk involves multiple agents or stressors, which means that 

assessments involving a single chemical or stressor are not “cumulative risk assessments” under 

this definition. Second, there is no limitation that the “agents or stressors” be only chemicals; 

they may be, but they may also be biological or physical agents or an activity that, directly or 

indirectly, alters or causes the loss of a necessity such as habitat. Third, this definition requires 

that the risks from multiple agents or stressors be combined. This does not necessarily mean that 

the risks should be “added,” but rather that some analysis should be conducted to determine how 

the risks from the various agents or stressors interact. It also means that an assessment that 

covers a number of chemicals or other stressors but that merely lists each chemical with a 

corresponding risk without consideration of the other chemicals present is not an assessment of 

cumulative risk under this definition. 

“Cumulative risk assessment” in this report means “an analysis, characterization, and 

possible quantification of the combined risks to health or the environment from multiple agents 

or stressors.” One key aspect of this definition is that a cumulative risk assessment need not 

necessarily be quantitative, so long as it meets the other requirements. 

The framework itself is conceptually similar to the approach used in both human health 

and ecological assessments, but it is distinctive in several areas. First, its focus on the combined 

effects of more than one agent or stressor makes it different from many assessments conducted 

today, in which, if multiple stressors are evaluated, they are usually evaluated individually and 

presented as if the others were not present. Second, because multiple stressors are affecting the 

same population, there is increased focus on the specific populations potentially affected rather 

xvii 
than on hypothetical receptors. Third, consideration of cumulative risk may generate interest in 

a wider variety of nonchemical stressors than do traditional risk assessments. 

The framework describes three main phases to a cumulative risk assessment: (1) 

planning, scoping, and problem formulation, (2) analysis, and (3) risk characterization. In the 

first phase, a team of risk managers, risk assessors, and other stakeholders establishes the goals, 

breadth, depth, and focus of the assessment. The end products of this phase are a conceptual 

model and an analysis plan. The conceptual model establishes the stressors to be evaluated, the 

health or environmental effects to be evaluated, and the relationships among various stressor 

exposures and potential effects. The analysis plan lays out the data needed, the approach to be 

taken, and the types of results expected during the analysis phase. 

The analysis phase includes developing profiles of exposure, considering interactions (if 

any) among stressors, and predicting risks to the population or populations assessed. It is in this 

phase that difficult technical issues such as the toxicity of mixtures, the vulnerability of 

populations, or the interactions among stressors that may be chemical or nonchemical are 

addressed and, hopefully resolved. The end product of this phase is an analysis of the risks 

associated with the multiple stressors to which the study population or populations are exposed. 

The third phase, risk characterization (interpretation), puts the risk estimates into 

perspective in terms of their significance, the reliability of the estimates, and the overall 

confidence in the assessment. It is also in this phase that an evaluation is made of whether the 

assessment met the objectives and goals set forth in phase one. 

The discussion of cumulative risk in this framework report takes a broad view of the 

topic and includes many aspects of an assessment that might conceivably be conducted in the 

future, even though techniques may not currently exist to examine every question. It also 

includes aspects of cumulative risk that may be outside of EPA’s current legislative mandates 

and where expertise outside of the Agency would be needed to address certain questions if they 

should arise. These aspects are discussed here for the sake of technical completeness and not as 

a recommendation that EPA perform all possible aspects of a cumulative risk assessment in all 

its risk assessments—even all its cumulative risk assessments. This framework may, however, 

provide an opportunity for the Agency to start to integrate the requirements of its various 

legislative mandates, at least in the area of risk assessment. 

EPA is currently engaged in activities that fall under various aspects of the cumulative 

risk assessment umbrella. Some of these activities are listed as illustrations in the box on the 

next page. The broad interpretation of cumulative risk in this framework report allows these 

activities to be put into perspective relative to one another and can illustrate how the activities fit 

together under the framework. Individual Program Offices and Regions may have to make 

decisions that affect the scope, types of stressors, or methods used for their programs’ 

cumulative risk assessments, based on legislative mandates or other criteria. Nothing in this 

report should be 

xviii 
Examples of Cumulative Risk Assessment Activities within EPA in 2002 

! The Superfund Program has updated its guidance on risk assessment to include planning and scoping 

for cumulative risk assessment and problem formulation for ecological risk assessments. The plan for 

the Office of Solid Waste’s Surface Impoundment Study includes both a conceptual model and an 

analytical plan, per the agency guidance on planning and scoping for cumulative risk. 

! The Office of Water is planning a watershed-scale risk assessment involving multiple stressors in 

ecological risk. This approach was developed through a collaboration with external scientists and is 

now being field evaluated. 

! Several Regional Offices are evaluating cumulative hazards, exposures, and effects of toxic 

contaminants in urban environments. In Chicago (Region 5), citizens are concerned about the 

contribution of environmental stressors to endpoints such as asthma and blood lead levels. In 

Baltimore (Region 3), a regional/Office of Prevention, Pesticides, and Toxic Substances/community 

partnership tried to address the long-term environmental and economic concerns in three 

neighborhoods that are adjacent to industrial facilities and tank farms. Region 6 (Dallas) is developing 

a geographic information system approach for planning and scoping cumulative risks. 

! The Food Quality Protection Act of 1996 requires that EPA consider the cumulative effects to human 

health that can result from exposure to pesticides and other substances that have a common mechanism 

of toxicity. The Office of Pesticide Programs has developed guidance for conducting cumulative risk 

assessments for pesticides and has prepared a preliminary cumulative risk assessment for 

organophosphorous pesticides. 

! The Office of Air and Radiation’s (OAR’s) air toxics program has a cumulative risk focus. Under 

the Integrated Urban Air Toxics Strategy, OAR will be considering cumulative risks presented by 

exposures to air emissions of hazardous air pollutants from sources in the aggregate. Assessments will 

be performed at both the national scale (a national-scale assessment for base year 1996 was completed 

in 2002) and at the urban or neighborhood scale. In partnership with the Office of Research and 

Development (ORD) and the National Exposure Research Laboratory, the Office of Air Quality 

Planning and Standards is developing the Total Risk Integrated Methodology (TRIM), a modular, 

modeling system for use in single or multimedia, single or multipathway human health and ecological 

risk assessments of hazardous and criteria air pollutants at the neighborhood or city scale. The 

Agency’s guidance for planning and scoping cumulative risk was used to develop a conceptual model 

and analysis plan for the national-scale air toxics risk assessment. 

! ORD’s National Center for Environmental Assessment (NCEA) has completed ecological risk 

assessment guidelines that support the cumulative risk assessment guidance. Five watershed case 

studies are being assessed to demonstrate the guidelines approach. Each of these cases deals with 

cumulative impacts of stressors (chemical, biological, and, in some cases, physical). In addition, 

NCEA has prepared a draft reassessment of dioxin and related compounds. 

! The Risk Assessment Forum convened a technical panel to develop guidance for conducting 

cumulative risk assessments, of which this framework is a first step. 

xix 
interpreted as mandating that a cumulative risk assessment be conducted or be conducted in a 

certain way for any specific case. Likewise, this report is not an attempt to lay out protocols to 

address all the risks or considerations that are needed to adequately inform community decisions. 

Rather, it is an information document, focused on describing various aspects of cumulative risk 

whether or not the methods or data currently exist to adequately analyze or evaluate those 

aspects of the assessment. Because of the limitations of current science, cumulative risk 

assessments done in the near future will not be able to adequately answer all the questions posed 

by stakeholders or interested parties. This does not mean, however, that they cannot answer 

some of the questions; in fact, cumulative risk assessment may be the best tool available to 

address certain questions dealing with multiple-stressor impacts. 

xx 
1. INTRODUCTION 

During much of its early history, the U.S. Environmental Protection Agency (EPA, or the 

Agency) focused its efforts on cleaning up the overt pollution problems of the 1960s and 1970s. 

Until the Agency was established, in 1970, relatively uncontrolled air emissions, water effluents, 

and dumped wastes had led to pollution of the environment that was easily detected by the five 

senses. The most effective and efficient way of approaching these overt problems of the 1970s 

was to find the entry point of the pollutant into the environment and to control it at that point. 

Looking back, we see a strategy that moved to control stack emissions, industrial and municipal 

effluents, pesticide application, land applications, burial of chemical wastes, and other “sources” 

of pollution. In addition, criteria and standards were established as goals for cleaning up the 

various environmental media. By the 1980s, this “command and control” strategy was well 

established in environmental laws and regulations but was reaching the point of diminishing 

returns from a cost-benefit viewpoint. 

The development of risk assessment methodology during the 1970s and early 1980s 

Figure 1-1. A chemical- (or stressor-) 

focused assessment starts with a source 

and evaluates how the chemical gets to 

various populations or ecological 

targets. Individual assessments may 

pursue some or all pathways, media, or 

population segments. 

closely followed the Agency’s strategy for control 

of pollution, because risk assessments were being 

used as a factor in EPA’s regulatory decision 

making. The focus on sources led naturally to 

analyses of what types of pollutants were in 

effluents, air emissions, and waste sites and the 

detection of chemical, biological, and—sometimes 

—radiological agents. By the 1970s, the link 

between some chemicals and certain diseases such 

as cancer had been established through a series of 

bioassays or, in the case of chemicals such as vinyl 

chloride and asbestos, through epidemiological 

studies. New analytical techniques also made it 

possible for the first time to detect very minute 

concentrations of chemicals. The focus of the EPA 

strategy to control pollution (and the risk 

assessment methodology being used to partially 

support decisions) gradually leaned toward 

assessing and controlling the individual chemicals. 

Congressional legislation tended to underwrite this 

approach by focusing on controlling sources and 

even including lists of individual chemicals to be 

controlled. 

The risk assessment methodology of the 1970s and early 1980s, therefore, tended toward 

single-chemical assessments (Figure 1-1). A National Research Council (NRC) report (NRC, 

1983) was focused largely on the single-chemical risk assessment approach when it spoke of the 

four parts of a Federal risk assessment: hazard identification, dose-response assessment, 

1
exposure assessment, and risk characterization. EPA’s 1986 risk assessment guidelines 

(USEPA, 1986a), with the exception of the mixtures guidelines (USEPA, 1986b), also focused 

largely on single-chemical assessment. 

However, research conducted or sponsored 

by EPA in the early 1980s was taking the first steps 

toward investigating a different type of risk 

assessment methodology, one that focused on 

identifying the persons exposed, investigating the 

chemicals or stressors to which they were exposed, 

and determining consequent risks (Figure 1-2). 

This approach differs from those that focus on 

either a chemical (and investigates the chemical 

environmental fate, exposed populations, and risks 

[Figure 1-1]) or a source (and investigates 

environmental releases from that source, exposed 

populations, and risks). The goals of the 

population-focused approach1 Figure 1-2. Population-based were much more

assessments start with the receptors and 

determine which chemicals, stressors, 

or other risk factors are affecting them. 

useful to decisionmakers who were dealing with 

public health or ecological health questions rather 

than controlling sources of pollution. 

The challenges posed by the population-based assessment can be daunting, even if only a 

few of the stressors affecting a population are evaluated together (i.e., cumulatively). Taken to 

the extreme, Figure 1-2 represents a concept of “total risk” for the population or population 

segment being evaluated, with each chemical, biological, radiological, or other stressor2

 adding 

some fraction of the total risk. Looking at the problem from an individual stressor viewpoint, to 

do this type of assessment would require not only evaluating each individual stressor, but also 

developing a way to add up all the risks among stressors across a population of individuals with 

1

 A chemical-focused assessment may look at several populations affected by exposure to the chemical but 

not at other chemicals. A population-focused assessment looks at one population for perhaps many stressors but not 

at other populations. Consequently, for traditional, chemical-focused assessments, we say we conduct a “risk 

assessments for a certain chemical.” In contrast, the essence of a cumulative risk assessment is that the assessment is 

conducted “for a certain population.” This difference is shown schematically by comparing Figures 1-1 and 1-2. 

How the population is identified for a cumulative assessment is not addressed here. 

2

 A stressor is a physical, chemical, biological, or other entity that can cause an adverse response in a 

human or other organism or ecosystem. Exposure to a chemical, biological, or physical agent (e.g., radon) can be a 

stressor, as can the lack of, or destruction of, some necessity, such as a habitat. The stressor may not cause harm 

directly, but it may make the target more vulnerable to harm by other stressors. A socioeconomic stressor, for 

example, might be the lack of needed health care, which could lead to adverse effects. Harmful events, such as 

automobile crashes, could also be termed stressors. Obviously, calculating risks from different types of stressors can 

use widely differing methods, including probabilistic estimates of disease via dose-response relationships or looking 

up rates in statistical tables of historical events, among others. 

2 
different exposures and susceptibilities. In the early 1980s, the state of the science was unready 

for virtually any part of the methods for doing this type of assessment. 

But progress was being made toward developing a population-based methodology. 

Starting in the late 1970s, a group of EPA researchers and contractors began developing what 

would become the Total Exposure Assessment Methodology (TEAM) study (USEPA, 1987). 

TEAM measured the concentrations of a number of chemicals simultaneously at the point of 

exposure. This project led to a larger study, the National Human Exposure Assessment Survey 

(NHEXAS) in the 1990s (Sexton et al., 1995). Both TEAM and NHEXAS developed analytical 

tools and methodologies to do population-based exposure assessments. 

Some progress was also being made in the early 1980s on the question of how to 

cumulatively consider the risks from different chemicals or stressors. EPA’s 1986 risk 

assessment guidelines (USEPA, 1986a) included a guideline on chemical mixtures (USEPA, 

1986b), which discussed how the risks from multiple chemicals could be evaluated as a whole. 

Work on this guidance has continued most recently with Supplementary Guidance for 

Conducting Health Risk Assessment of Chemical Mixtures (USEPA, 2000e), which expands and 

supplements the 1986 effort. 

As progress was being made on single chemical and chemical mixture risk 

assessment with the 1986 guidelines, some different kinds of risk assessment problems

began to catch the Agency’s attention. In 1986, 11 Chicago-area community groups 

joined to file a petition under Section 21 of the Toxic Substances Control Act, asking for

a community assessment in Southeast Chicago. A series of community-based actions

that started in 1982 and grew throughout the 1980s focused on disparities of risk among

various population subgroups, calling specific attention to cumulative effects of pollution

on minority subgroups (GAO, 1983; United Church of Christ, 1987). This series of 

community-based actions, chronicled by Bullard (1990), eventually became known as

the movement. The issues raised by the movement were the basis of a 1994 Presidential

Executive Order (Executive Order 12898, February 11, 1994), which told federal agencies,

among other things, to consider multiple and cumulative exposures whenever practicable

and appropriate. In the 1990s, Environmental Justice cases, including the cases filed under

Title VI of the 1964 Civil Rights Act, [P.L. 88-352, July 2, 1964] have further emphasized the

need for a cumulative human health risk assessment methodology. 

It was apparent that in addition to chemical- or stressor-focused assessments 

(as shown in Figure 1-1), population-focused assessments (as shown in Figure 1-2) would 

be needed if EPA was going to be able to answer the questions and issues being raised by the

public. Community spokespersons and other stakeholders, as well as scientific panels,

were increasingly coming to the Agency with problems that demanded a multi-stressor approach

 (e.g., NRC 1994). Ecological problems in particular were demanding a “place-based” context 

(such as the Chesapeake Bay watershed) in which the various populations within the area were

 looked at from a “total system” viewpoint. This place-based focus was part of Framework

3
for Ecological Risk Assessment (USEPA, 1992b) and Guidelines for Ecological Risk

Assessment (USEPA, 1998b). 

Although clearly addressing more than cumulative human health or ecological risk 

assessment, the National Environmental Policy Act of 1969 (NEPA) (P.L. 91-190, 42 U.S.C. 

4321-4347, January 1, 1970, as amended by P.L. 94-52, July 3, 1975, P.L. 94-83, August 9, 

1975, and P.L. 97-258, §4(b), Sept. 13, 1982), which was passed at about the same time EPA 

was established, requires assessments on the cumulative impacts of federal or federally funded 

projects (such as roads, dams, power lines, military projects, and infrastructure development) on 

natural ecosystems, endangered species, habitats, and opportunities for public enjoyment and 

natural resource use. A primary concern for NEPA is “cumulative effects analysis,” defined as 

“the incremental impact of the action when added to other past, present, and reasonably 

foreseeable future actions. . . . Cumulative impacts result from individually minor but 

collectively significant actions taking place over a period of time” (CEQ, 1997). Much of the 

NEPA cumulative effects analysis is qualitative, but risk assessments and cause-and-effect 

relationships are key parts of the analysis process for controversial projects. 

In 1997, the Agency issued a policy memo, Guidance on Cumulative Risk Assessment, 

Part 1: Planning and Scoping (USEPA, 1997a), which took the first formal step toward 

developing guidance and guidelines for cumulative risk assessment. 

Cumulative risk assessment applications have become relatively common, not only for 

assessments of chemicals that operate by the same mode of action, as mandated for the EPA’s 

pesticides program, but also for community-based, population-based, assessments that may 

include more varied stressors than just chemicals alone. Much like the “place-based” ecological 

assessments, which may cover a wide variety of physical, chemical, and biological stressors, 

some communities have added human health and perhaps “quality of life” to the endpoints of 

interest in their place-based assessments. The demand for more sophisticated human health risk 

assessments has driven the need for research into cumulative risk assessment, populationfocused assessments, aggregate exposure assessment, and risk from chemical mixtures. 

1.1. Purpose and Scope of the Framework Report 

An understanding of the finite purpose and scope of this framework report is important. 

EPA and other organizations need detailed, comprehensive guidance on methods for evaluating 

cumulative risk. Before such detailed Agency-level guidance is developed on a relatively new 

field of risk assessment, it has been the recent policy of the Agency to first develop a simple 

framework as a foundation for later comprehensive guidance. This Framework for Cumulative 

Risk Assessment emphasizes chemical risks to human health in its discussion and also in the 

context of the effects from a variety of stressors, including nonchemical stressors. Some 

important topics that could be characterized as “cumulative risk,” such as global climate change, 

are beyond the scope of this report. 

4
Given this background, the 

framework has two simple purposes, 

one immediate and one longer term. As 

a broad outline of the assessment 

process, the framework immediately 

offers a basic structure and provides 

starting principles for EPA’s 

cumulative risk assessments. The 

process described by the report 

provides wide latitude for planning and 

conducting cumulative risk assessments 

in many diverse situations, each based 

on common principles discussed in the 

report. The process also will help foster 

a consistent EPA approach for 

conducting and evaluating cumulative 

risk assessments, for identifying key 

issues, and for providing operational 

definitions for terms used in cumulative 

risk assessments. 

In the longer term, the 

framework report offers the basic 

principles around which to organize a 

more definitive set of cumulative risk 

assessment guidance. With this in 

mind, this report does not provide 

substantive guidance on certain issues 

that are integral to the risk assessment 

process (see box on this page and 

Appendix B for a listing of useful 

resources). These issues include 

specific analytical methods, techniques 

for analyzing and interpreting data, and 

guidance on issues influencing policy. 

Rather, on the basis of EPA experience 

and the recommendations of peer 

reviewers, EPA has reserved discussion 

EPA’s Risk Assessment Guidelines 

Chemical mixtures (USEPA, 1986b)

Mutagenicity risk assessment (USEPA, 1986c)

Carcinogen risk assessment (USEPA, 1986d)

Developmental toxicity risk assessment (USEPA, 1991a)

Exposure assessment (USEPA, 1992c)

Reproductive toxicity risk assessment (USEPA, 1996b)

Proposed carcinogen risk assessment (USEPA, 1996c,

1999a, b)

Ecological risk assessment (USEPA, 1998b)

Neurotoxicity risk assessment (USEPA, 1998c)

Selected Policy and Guidance Documents 

Risk assessment guidance for superfund (USEPA, 1989a)

Locational data policy (USEPA, 1991b)

Framework for ecological risk assessment (USEPA, 1992b)

Application of refined dispersion models (USEPA, 1993a)

Policy/guidance for risk characterization

(USEPA, 1995a, b)

Benchmark dose (USEPA, 1995c, 2000b)

Cumulative risk planning and scoping (USEPA, 1997a)

Guiding principles for Monte Carlo analysis 

(USEPA, 1997b)

Acute inhalation exposure (USEPA, 1998d)

Chemical emergency risk management (USEPA, 1998e)

Draft comparative risk framework (USEPA, 1998f)

Aggregate exposure and risk (USEPA, 1999g)

Community involvement in Superfund risk assessment

(USEPA, 1999c)

Guidance for offsite consequence analysis (USEPA, 1999d)

Guideline on air quality models (USEPA, 1999e)

Framework for community-based environmental protection

(USEPA, 1999f)

Handbook for risk characterization (USEPA, 2000c)

Handbook for peer review (USEPA, 2000d)

Supplementary guidance for conducting health risk 

assessment of chemical mixtures (USEPA, 2000e) 

Cumulative risk assessment of pesticide . . . common 

mechanism of toxicity (USEPA, 2002a) 

of these important aspects of cumulative risk assessment for future guidance, which will be 

based on the risk assessment process described in this framework report. 

This report lays out broad areas where analysis might be conducted if needed. It does not 

suggest that cumulative risk assessment is a tool that should be used with every issue, nor does it 

suggest that when cumulative risk assessment is applied, all areas of analysis outlined or 

5
discussed here must or even should be conducted in every assessment. The scope of the 

assessment should be defined in the planning and scoping stage (see Section 2.1) and may 

include or exclude stressors or pathways as relevant to the particular context or application. In 

some areas discussed in this report, the methodology for doing the risk analysis may not yet 

exist. 

Following completion of this framework report, EPA plans to initiate development of a 

more detailed guidance document. As a first step in this process, EPA will oversee the 

preparation of a number of case studies and issue papers on select topics. In addition, the 

Agency plans to hold workshops to further evaluate those issues. Following these activities, 

EPA will begin drafting the more detailed guidance in the form of Guidelines for Cumulative 

Risk Assessment. At this time, the Agency does not have a definite schedule for these activities. 

1.2. Intended Audience 

This framework report is primarily intended for EPA risk assessors, EPA risk managers, 

and other persons who either perform work under EPA contract or sponsorship or are subject to 

EPA regulations concerning risk assessments. The terminology and concepts described here also 

may be of assistance to other Federal, State, and local agencies as well as to members of the 

general public, including stakeholders, who are interested in cumulative risk assessment issues. 

The style and language used in this report were chosen so as to be understandable by as wide a 

variety of interested parties as possible, from the policy maker to the risk assessment scientist to 

the concerned nonscientist member of the general public. It is hoped that this report will be the 

first step in developing a broad scientific consensus about cumulative risk assessment, and that 

further guidelines and guidance will build upon this foundation. 

1.3. Key Definitions in Cumulative Risk Assessment3 

In this report, “cumulative risk” and “cumulative risk assessment” are defined as follows, 

assuming a defined population: 

Cumulative risk: The combined risks from aggregate exposures4

 to multiple agents or 

stressors. 

Cumulative risk assessment: An analysis, characterization, and possible quantification 

of the combined risks to health or the environment from multiple agents or stressors. 

3

 In this section, a few basic definitions related to cumulative risk assessment are discussed. For a glossary 

of terms, the reader is directed to Chapter 5. 

4

 See the box on the next page for a definition of aggregate exposure. Some references (e.g., Berglund et 

al., 2001) refer to this type of total exposure over time for a single stressor as “cumulative exposure.” To avoid 

confusion, we have not used the term “cumulative exposure” in this report. 

6 
Several key points arise from this 

definition of cumulative risk. First, 

cumulative risk involves multiple agents or 

stressors, which means that assessments 

involving a single chemical or stressor are 

not “cumulative risk assessment” under this 

definition. Second, there is no limitation that 

the “agents or stressors” be only chemicals. 

“Agents or stressors” may be chemicals, of 

course, but they may also be biological or 

physical agents or even the absence of a 

necessity such as habitat. Third, this 

definition requires that the risks from 

multiple agents or stressors be combined. 

This does not necessarily mean that they be 

“added,” but rather that some analysis 

should be conducted on if and how the 

effects or risks from the various agents or 

stressors interact. It also means that an 

assessment that covers a number of 

FQPA’s Terminology Interpretations 

The Food Quality Protection Act of 1996 [P.L. 

104-170] discusses the addition of exposure for a single 

chemical across sources, pathways, routes, and time as 

aggregate exposure. To be consistent with that 

terminology, the Agency has elected to speak of 

multiple source/pathway/route single-stressor 

exposures and risks as “aggregate exposures” and 

“aggregate risks.” The EPA Science Policy Council’s 

Cumulative Risk Subcommittee has developed the 

following working definitions for single-chemical or 

single-stressor situations: 

Aggregate exposure: The combined exposure of an 

individual (or defined population) to a specific agent or 

stressor via relevant routes, pathways, and sources. 

Aggregate risk: The risk resulting from aggregate 

exposure to a single agent or stressor. 

chemicals or other stressors but that merely lists each chemical with a corresponding risk without 

consideration of the other chemicals present is not an assessment of cumulative risk under this 

definition. 

The definition of cumulative risk assessment follows from the definition of cumulative 

risk, but again there is a key point: cumulative risk assessments can be qualitative as well as 

quantitative. 

Some examples of cumulative risk assessments as well as assessments that would not be 

considered as cumulative risk assessments are listed below. Each example presupposes a 

defined individual or population5

: 

1. Single-agent or -stressor assessment. Risks can be added or accumulated over time 

for a single agent or stressor across sources, environmental pathways, or exposure routes. This 

concept is consistent with “aggregate risk” in the terminology of the Food Quality Protection Act 

of 1996 (FQPA), shown in the box on this page. Although this type of assessment might 

conceivably be termed a cumulative risk assessment by some scientists, in this framework report 

such single-stressor assessments are termed “aggregate risk assessments.” Examples might be a 

multisource assessment of benzene risk in a community or an assessment of individual risk to a 

5

 Populations can be defined by geophysical boundaries, such as a watershed, or geopolitical boundaries, 

such as city or county limits, or by cultural, racial, economic, or other criteria within a certain geographic boundary 

such as a neighborhood. The definition of a population needs to be clear enough so that it can be agreed upon 

whether any specific individual is included in or excluded from the population. 

7 
specific pesticide from all uses combined. This type of assessment is not discussed in this report 

except to be referred to occasionally for clarity and contrast to cumulative risk assessments. 

There are several publications that discuss aggregate exposure and risk assessment in detail (e.g., 

ILSI, 1998, 2001; USEPA, 1999g). 

2. Multiple-stressor assessment. Exposures can be accumulated over time, pathways, 

sources, or routes for a number of agents or stressors. These stressors may cause the same 

effects (e.g., a number of carcinogenic chemicals or a number of threats to habitat loss) or a 

variety of effects. A risk assessment for multiple stressors may evaluate the risks of the 

stressors, associated health effects or ecological impacts one effect or impact at a time or it may 

evaluate the combined risk from some or all the effects or impacts; in either case, it is considered 

to be a cumulative risk assessment. 

A multiple-stressor cumulative risk assessment is distinct from a series of aggregate risk 

assessments because it considers any combined impact of the stressors, including the potential 

for interactions among stressors (e.g., synergism or antagonism). One example of a multiplestressor, single-effect cumulative risk assessment would be the combined risk to an individual or 

population from a series of pesticides acting by the same mode of action and causing the same 

effect. 

Another example would be a dioxin assessment, where toxicity equivalency factors 

(TEFs) are used to combine the toxicities of dozens of different congeners of chlorinated 

dibenzo-p-dioxins and dibenzofurans, resulting in a single estimate of risk for a specific effect 

from the combination of congeners (Eadon et al., 1986; Barnes et al., 1991). 

Another example is a physician’s use of a model derived empirically from 

epidemiological studies to estimate the probability of a woman’s developing breast cancer over 

the next 10 years. The “stressors” in the example of the breast cancer model are those risk 

factors known to be correlated with that form of cancer, such as the woman’s age at first 

childbirth, age at menarche, or having a previous biopsy with atypical hyperplasia. This 

example shows that stressors may not necessarily be chemical stressors, nor do they even need to 

be the same types of stressors. These models (discussed further in Appendix F) are in some 

ways quite different from the predictive risk assessments generally done for regulatory and other 

purposes. 

Another type of cumulative risk assessment discussed in this report is the multiplestressor, multiple-effects assessment. Again, stressors need not be limited to chemicals, nor do 

they even have to be the same types of stressors or have similar effects to be included in this type 

of assessment. For example, chemical, biological, radiological, and other physical and even 

psychological stressors can cause a variety of human health or ecological health effects. 

Assessing the risk for these situations is considerably more complex methodologically and 

computationally than for the examples of aggregate risk assessments or single-effect cumulative 

risk assessments given in the above paragraphs. 

8
As complex as this may sound, there are several examples of this type of assessment. 

Although these analytical approaches may start with the stressors and predict the risk of effects, 

more generally these types of assessments start with a defined geographical area or defined 

population (e.g., a population damaged in some endpoint or due to a reduction in a resource 

value) and try to determine what stressors are important. 

For example, cumulative ecological risk assessments such as those that have been 

conducted in the Columbia River Basin and the Chesapeake Bay focus on a number of observed 

adverse conditions, then attempt to determine, among all of the possible stressors, which 

particular combination is responsible for the observed adverse conditions (Barnthouse et al., 

2000). 

NRC (1994, Appendix I) lays out the general mathematics for a quantitative approach to 

multiple-stressor, multiple-effect assessments. Recently, Bogen (2001) used this approach to 

quantify combined risk of cancer and noncancer endpoints induced by the chemical 

trichloroethylene (TCE), including quantitative characterization of associated interindividual 

variability and associated uncertainty (including uncertainty regarding mechanism of 

carcinogenic action). Technical hurdles involved in implementing this approach include 

defining the set of relevant (preferably independent) endpoints and quantifying the likelihood of 

inducing each adverse health or ecotoxic response considered unacceptable as a function of the 

endpoints. 

Another example of a type of multiple-stressor, multiple-effect assessment would be a 

cumulative community health risk assessment. 

We believe that the definition of cumulative risk used in this framework report is 

consistent with the sense of most definitions of “cumulative,” such as are included in NEPA or 

FQPA. A summary of the features and options of a cumulative risk assessment, by the definition 

used in this report, is given in the box on the next page. 

1.4. The Cumulative Risk Assessment as a Tool for a Variety of Users and Purposes 

As discussed in the introduction of this report, the results of an assessment should reflect 

the purpose for conducting the assessment. However, information from cumulative risk 

assessments can also serve a variety of other purposes. Insights gained may also be used to 

partly meet regulatory mandates and to help identify targets for enforcement actions, or they may 

be considered when shaping policy and regulation. Assessments may conceivably be used in 

long-term planning with regard to siting new sources of potential pollution in specific areas. 

Assessments also may be used for general educational purposes not directly related to an 

immediate decision on a course of action. Assessment results can help guide priorities for 

voluntary or regulatory action or mobilize community efforts to address concerns. They can be 

conducted retrospectively (to determine past or current risks), prospectively (to assess the risks 

of, say, proposed facilities), or even creatively (to design a development plan for a community). 

As helpful as results may be in any of these other uses, however, some consideration should be 

9
given to the appropriateness of using the 

assessment for these purposes, given the 

objectives and scope of the assessment. 

Risk analysis, including 

cumulative risk analysis, is conceptually 

an analytic-deliberative process (NRC, 

1996). The analytic component includes 

rigorous, replicable methods that are 

evaluated under the agreed protocols of 

an expert community; the deliberative 

component is based on stakeholder value 

and judgment. Much of what is 

discussed in Chapter 2, “The Planning, 

Scoping, and Problem Formulation 

Phase,” is deliberative in nature, which 

means it depends on input from experts 

other than those who know how to do 

risk assessments, including persons who 

are knowledgeable about a community 

and its values. Although much of 

Chapter 3, “The Analysis Phase,” is 

given over to the analytic process, where 

risk assessment experts apply science to 

a problem, the deliberative aspect returns 

in Chapter 4, “The Risk Characterization 

Phase,” especially where risks of 

different types are being evaluated and 

combined. 

Because of this analyticdeliberative process, the cumulative risk 

assessment can be applied to a variety of 

Cumulative Risk Assessment Features 

Although many different types of exposures, stressors 

and other factors can be included, the definition of 

cumulative risk might be better understood by contrasting 

the featured and optional considerations. By the definition 

given above for this Framework report, the following 

features are included: 

• Multiple stressors. 

• Consideration of how the stressors act together rather 

than individually. 

• Population-focused assessment. Although this does 

not mean that the assessment must start with a 

population and work “backwards” toward the source, 

it does mean that the population needs to be defined, 

and multiple stressors are assessed with regard to 

impact on that population, although not every 

individual will see the same (or all) effects. 

Additional layers of complexity, such as those listed 

below, may or may not be addressed: 

• Multiple durations, pathways, sources, or routes of 

exposure. 

• Multiple effects or impacts. 

• Nonconventional stressors or risk factors (e.g., 

lifestyle, access to health care). These in general need 

continued research. 

• Quantification of risks. 

different problems where analysis of the overall impacts of multiple sources, stressors, pathways, 

or routes is necessary. It can be used as a regulatory analysis tool, such as in reviewing the 

overall impact of several different pesticides that act by the same mode of action (ILSI, 1999) or 

in NEPA analyses (CEQ, 1997). It can be used to analyze the overall impacts of permit 

decisions or the results of compliance with permits in a given community. 

Cumulative risk assessment can also be used in a community-based assessment approach, 

as outlined in USEPA (1999f). The community-based environmental protection (CBEP) 

approach (see box on next page) encompasses both ecological and human health assessments. 

Cumulative risk assessment, being a population-based or place-based analytic-deliberative 

process, is ideal for CBEP-type applications. 

10
Cumulative risk assessments are 

also applicable in ecological 

assessments. EPA’s definition of 

cumulative ecological risk assessment is 

a process that involves consideration of 

the aggregate ecological risk to the 

target entity caused by the accumulation 

of risk from multiple stressors (USEPA, 

1998b). A report by Foran and Ferenc 

(1999) discusses multiple stressors in 

ecological risk assessment and gives a 

good overview of the topic of 

cumulative ecological risk assessment. 

When should a cumulative risk 

Core Principles of Community-Based 

Environmental Protection 

1. Focus on a definable geographic area. 

2. Work collaboratively with stakeholders. 

3. Assess the quality of all resources in a place. 

4. Integrate environmental, economic, and social 

objectives. 

5. Use the most appropriate tools. 

6. Monitor and redirect efforts through adaptive 

management. 

Source: USEPA, 1999f 

assessment be done? Recognizing that the scope and nature of a cumulative risk assessment may 

range from a very limited qualitative assessment of a local situation, to a comprehensive 

assessment of the cumulative risk patterns for a large community, to a national assessment 

conducted within one of EPA’s programs, the simple answer is that one should be conducted 

whenever the combined impact of multiple stressors should be considered. Only experience with 

these assessments over a period of time will provide the wisdom needed to develop practical 

guidelines on this question. 

1.5. The Broader Decision-Making Context for Cumulative Risk Assessment 

Cumulative risk assessments may be used to form hypotheses that can be tested, but it is 

more likely that these assessments will be used as decision-making tools. The levels of decision 

making may vary widely, from a neighborhood group evaluating ways to improve or safeguard 

its health and environment to a Federal official weighing options for action at a much broader 

geographical level. Although the decision-making method is beyond the scope of this report, 

such decisions usually involve more than the basic science and analysis that make up the 

“scientific” part of risk assessment. Clemen (1996) notes that in one type of decision-making 

approach (called decision analysis): 

Managers and policy makers frequently complain that analytical procedures from 

management science and operations research ignore subjective judgments. Such 

procedures often purport to generate “optimal” actions on the basis of purely objective 

inputs. But the decision-analysis approach allows the inclusion of subjective judgments. 

In fact, decision analysis requires personal judgments: they are important ingredients for 

making good decisions. 

Regardless of the type of decision being made or the decision-making approach, a 

cumulative risk assessment’s analytic component is not the decision-making vehicle in itself. 

That is, “cranking out the numbers” will not be the sole basis for a decision. Although in some 

cases the estimated risks can weigh heavily in the decision, understanding the risk estimate is but 

11
one factor in a broader decision-making process that includes risk management components such 

as technical feasibility, economic costs and benefits, political realities, and other analyses usually 

associated with the field of policy analysis (e.g., OMB, 2000; Freeman, 1999; Hattis and Gobel, 

1994; Ashford et al., 1981). 

EPA’s Science Advisory Board (SAB) (USEPA, 2000a) has constructed a framework for 

what it terms integrated environmental decision making. The SAB noted that “The IED 

Framework recognizes that risks often are experienced simultaneously and are cumulative. . . .” 

It speaks of risk assessments in a very broad way and includes human health effects, ecological 

effects, and quality-of-life effects. The first phase (“Problem Formulation”) and part of the 

second phase (“Analysis and Decision-making”) of the IED essentially correspond to the three 

phases discussed in this framework report. Decision making—and the SAB’s third phase, 

“Implementation and Performance Evaluation,” are beyond the scope of this report. 

The SAB report gives a good insight into the broader context for cumulative risk 

assessment and some of the aspects of the analytic-deliberative parts of the assessment. The 

analytical-deliberative process is discussed more in Chapters 2 through 4 of this report as these 

phases of the cumulative risk assessment process are examined. 

NRC (1996) also provides much information on the analytic-deliberative aspects of a risk 

assessment and devotes a great deal of discussion to risk characterization. Needless to say, it is 

very important to apply cumulative risk assessment in the context of the decision or decisions to 

be made. This is most efficiently done by early and continued attention to the “risk 

characterization” step in the risk assessment process (NRC, 1996; USEPA, 2000c). The box in 

Section 4.1 summarizes some of the points made in the NRC report. 

1.6. Organization of This Report 

Figure 1-3 shows the basic structure of this report. Each of the three general process 

steps are described in detail in later chapters. The framework is organized to follow the outline 

in Figure 1-3, namely (a) a planning, scoping, and problem formulation phase (Chapter 2), (b) an 

analysis phase (Chapter 3), and (c) a risk characterization phase, where the interpretation of 

findings and explanation of the results are completed (Chapter 4). Chapter 5 is a glossary of 

terms, followed by references in Chapter 6. Additional information on selected resources and 

cumulative risk related topics are provided in the appendices. 

12
Planning, Scoping, and 

Problem Formulation 

Analysis 

Interpretation and 

Risk Characterization 

Figure 1-3. Framework for cumulative risk assessment. 

13
2. THE PLANNING, SCOPING, AND PROBLEM FORMULATION PHASE 

The first step in any risk assessment process is to define the problem to be assessed. This 

step has been called “problem formulation” (for example, USEPA, 1992b, 1997a, 2000a; NRC, 

1996). It is a phase where “public officials, scientists, and interested and affected parties clarify 

the nature of the choices to be considered, the attendant hazards and risks, and the knowledge 

needed to inform the choices” (NRC, 1996). 

The planning and scoping of an assessment are often thought to be part of the problem 

formulation phase, although EPA guidance (USEPA, 1997a) treats planning and scoping as 

activities that take place before problem formulation begins. Whether they are considered a 

separate phase or not, they take place at the very beginning of a cumulative risk assessment. For 

convenience, this chapter incorporates planning, scoping, and problem formulation into a single 

phase (see Figure 2-1). 

2.1. Planning and Scoping 

Risk assessments are conducted within some context, that is, they are usually conducted 

because of a regulatory requirement, a community need, a health crisis, or some other driving 

force. This context generates individuals or groups with interest in having the assessment done; 

several summary articles or books discuss the challenges of successful participation by these 

interested parties (e.g., Chess and Purcell, 1999; Frewer, 1999; Thomas, 1995). These parties 

may include public officials, risk experts, community leaders, or any number of others, including 

those, if any, who are legally mandated to be part of the process. Planning and scoping begins 

with a dialogue among these individuals or groups. 

Among these interested parties will be a person or a group of people charged with 

making decisions about how a risk may be mitigated, avoided, or reduced. For the sake of 

simplicity, we call this person or group the “decisionmaker,” or “risk manager6

,” and for ease of 

discussion will discuss the risk manager as if he or she were a single person. 

During planning and scoping, risk experts (including those involved in assessing risk, 

such as ecologists, toxicologists, chemists, and other technical experts such as economists and 

engineers), and decisionmakers work together as a team, informed by stakeholder input, to 

develop the rationale and scope for the risk assessment and characterization. 

6

 We use the term “risk management” to include actions that the risk assessment team recommends or 

implements that are not taken by the risk assessment team, per se. These include actions to address the problems 

taken by others outside the process who may not be identified until the analysis is underway or complete. 

14 
Planning, Scoping, and 

Problem Formulation 

Analysis 

Interpretation and 

Risk Characterization 

Planning, Scoping, and 

Problem Formulation 

Analysis 

Interpretation and 

Risk Characterization 

Planning and Scoping 

- Purpose - Approach 

- Scope - Resources 

- Participants - Past Experiences 

Problem Formulation 

Conceptual Model 

- Sources - Receptors 

- Stressors - Endpoints 

- Pathways/Routes 

Analysis Plan 

- Methods - Data Gaps 

- Models - Uncertainties 

Discussion of Possible Outcomes 

Figure 2-1. The Planning, Scoping, and Problem Formulation phase. 

As part of the initial discussions concerning the need for a risk assessment, other 

“interested and affected parties” besides the risk manager and risk assessor may help define 

purpose, scope, and approach. This “risk assessment planning team” seeks agreement through 

extensive dialogue and discussion on what analytical and deliberative steps need to be taken and 

by whom, when, and why (USEPA, 2000a). USEPA (2000a) explains some of the roles of the 

various participants on the risk assessment planning team during the planning and problem 

formulation phase: 

Scientists play an important role in [this phase] by collecting, analyzing, and 

presenting data in such a way that all parties can appreciate the type and 

magnitude of the problem(s) under discussion. This activity will generally 

involve all four parts of risk assessment, including assessment of exposures 

15
experienced by special populations and/or ecological resources. Planning, 

scoping, and screening—including selection of endpoints of concern—also 

requires explicit input of societal values and stakeholder participation. For 

instance, while some of the ecological endpoints may be chosen because of their 

role in a valued ecosystem, there may also be ecological endpoints chosen 

because of their direct significance to society. Examples of the latter include both 

economically important species and ‘charismatic’ species. Similarly, in 

integrated decision-making, judgments may have to be made about diverse health 

endpoints, such as cancer risks in the general population and the risk of 

reproductive/developmental risks in children. While scientists can help 

characterize such risks, they are not uniquely qualified to set priorities among 

them and broader deliberation is essential. Finally, decision-makers also play an 

important role during problem formulation; in addition to bringing the scientific 

and other resources of the Agency to bear on the problem, they also should help 

to identify the range of potential decisions and viable management options, while 

examining economic, political, or other constraints on those options. Decisionmakers also serve as managers of the overall process. (USEPA, 2000a) 

Another role of the risk assessment planning team is documentation. The activities 

discussed in the following sections are important and should be documented by the team for 

several reasons. Written records can be referred to by assessors and attendees at public meetings. 

They can also help in responding to comments and in establishing a record for any later 

decisions or plans that need to be peer reviewed (USEPA, 2000d). The risk assessment planning 

team should consider whether or not the overall project is to be peer reviewed and, if so, what 

type of peer review will be conducted. The team should plan and execute the peer review at the 

appropriate time. A peer review by an independent review group will help not only to establish 

the validity of the science, but it can also provide neutral comments on some of the 

interpretations of the assessment. 

In some cases, it may be useful for the stakeholders to appoint a “point person” to serve 

as the contact for communications. This is not to imply that stakeholders must speak with a 

single voice (which is not likely in any case), but that they have at least one person to help 

facilitate interactions and identify available technical resources and other sources of information. 

The Agency or stakeholders may also consider a public Web site for the project. A variety of 

resources can be posted, including cumulative risk tools and databases, project-related news, list 

of experts, glossary, reports, and related links. An online discussion forum could also be 

included on the Web site as a more interactive way of exchanging information with stakeholders. 

Finally, while including stakeholders in the risk assessment process, a regulatory agency 

such as EPA should balance stakeholder participation with the Agency’s need to retain the 

ability to carry out its responsibility to protect public health and the environment. For this 

reason, EPA will usually need to set some reasonable boundaries around the process to ensure 

that progress is being made in a timely and efficient fashion. 

16
17

Possible Management Goals

The goals of risk management are varied. They

may be risk related, aiming to:

• Reduce or eliminate risks from exposure to

hazardous substances.

• Reduce the incidence of an adverse effect.

• Reduce the rate of habitat loss.

They may be economic, aiming to:

• Reduce the risk without causing job loss.

• Reduce the risk without reducing property values.

They may involve public values, aiming to:

• Protect the most sensitive population.

• Protect children.

• Preserve a species from extinction.

Source: Presidential/Congressional Commission, 1997

2.1.1. Defining the Purpose of the Assessment

As discussed in Section 1.5 above, the risk assessment should be developed to inform the risk

management decision by constructing an appropriate, decision-relevant risk characterization. After the

risk assessment planning team is assembled, the dialogue between the decisionmaker and risk experts

begins with a discussion of risk management objectives and information needed to manage risks in the

particular situation. The manager and assessment planning team should discuss any regulatory or legal

basis for the risk assessment and what kind of information is needed to satisfy such requirements. If

interested and affected parties are part of the risk assessment planning team, it is especially important

that the entire team agree on the purpose of the assessment because a differing sense of purpose among

the team will lead to problems later on.

The purpose and risk management objectives guide the risk assessment strategy (see box

below for some possible management goals from which risk management objectives can be derived,

e.g., in terms of key participants, data sources, selection of assessment endpoints, approach, and the

schedule for developing the assessment). Other possible management goals include identifying options

for control or abatement of hazards or risks, where decisions can then be made after considering costs

and benefits of the various risk management options.

The previous discussion follows the

typical situation where the risk manager is

presented as an independent decisionmaker,

such as a senior official in a regulatory agency

who is responsible for establishing permit

conditions for a facility of some type. There are

situations, however, where the risk manager

may be one of the interested parties outside the

Agency, such as a local citizens’ board. For

example, mitigation of risks may not be

significantly affected by any Agency or State

permit decisions but will depend instead on local

zoning decisions or on decisions that affect

traffic patterns in a community. This is one of

the reasons why the discussion of possible

outcomes (discussed in Section 2.3) the final

step in the planning and problem formulation

phase is so important.
2.1.2. Defining the Scope of Analysis and Products Needed 

Scoping a cumulative risk assessment effort involves defining the elements that will or 

will not be included in the risk assessment7 (USEPA, 1997a). These include the stressors, 

sources, pathways, routes, populations, and effects or assessment endpoints to be evaluated. 

As illustrated by the examples in the 

adjacent text box, the scope of a cumulative 

risk assessment may be narrow or broad. 

Initially, the risk assessment planning team 

should select the kind of risk information, 

exposure scenarios, and assessment issues 

that need to be covered. These should be 

directly linked to the risk-related questions 

being asked when establishing the purpose. 

Scope can be limited geographically (e.g., by 

political or ecological boundaries), 

environmentally (e.g., by assessing only 

certain media), demographically (e.g., by 

assessing only risks to children or 

asthmatics), legally (e.g., by statute or 

regulation), or by lack of methods or data in 

certain areas. The issue of background 

exposures to stressors should be discussed 

and agreements reached (see Appendix C). 

An adequate assessment scope should 

make it clear what is included in and what is 

excluded from the assessment. Care should 

Examples of Cumulative 

Risk Assessment Scopes 

• Health risks associated with the aggregate 

exposure (via all pathways and routes) to 

insecticides acting by a common mode of action. 

• Human health risks associated with outdoor 

inhalation exposures of the general population to 

33 priority air pollutants nationwide or via all 

routes to all pollutants present or being released 

from a hazardous waste site. 

• Human health and ecological risks associated with 

multiple stressors resulting from developing a site 

or corridor of land for transportation, 

infrastructure, or a stationary facility. 

• Human health risks for a specific neighborhood 

associated with exposure via all routes to all 

pollutants present or being released from a set of 

adjacent sources, including several industries, two 

hazardous waste sites, traffic, and a municipal 

landfill. 

be taken to reconcile the limitations of the scope with the list of questions to be answered in the 

statement of purpose. If, for example, data limitations preclude addressing certain questions 

outlined in the purpose, the list of questions should be modified and the risk assessment planning 

team agree to the narrower scope of the assessment. Defining the scope of an assessment is a 

process that can include both analytical and deliberative aspects. 

The reasons for choosing the particular scope of the assessment and the manner in which 

the assessment will address the questions posed in the purpose statement should be stated 

explicitly. Defining the scope of the assessment should include details on the limitations of 

resources, limitations of data, the impact of risk elements on the risk estimate (i.e., some 

pathways may be seen as having negligible impact on the risks related to the questions being 

addressed), and limitations of the methods available. In cases where an element of risk is likely 

7

 An assessment that looks at all stressors over a period of time for a specific population would be a “total 

risk” assessment, which is difficult to perform with our current methods. 

18 
to be important but no valid data are available, the risk assessors highlight this deficiency or use 

judgment or assumed values to approximate the missing data. Such judgments and 

approximations should be clearly documented and explained to the manager in the risk 

characterization. 

Once the elements (sources, stressors, populations, etc.) have been identified by 

brainstorming with all the participants, the participants should discuss the need for and 

availability of technical information and how such information might affect the overall 

uncertainty of the assessment. Using input from the risk assessor, the risk assessment planning 

team should determine what elements will and will not (or can and cannot) be included in the 

risk assessment. Some of the stakeholder concerns may not be suitable for analysis by risk 

assessment, so other expertise and evaluation may be required to provide this additional analysis. 

Information gathered at this stage is preliminary and may be modified during the analysis phase. 

Identification of potential stressors, populations to be assessed, and potential effects are all part 

of the scoping process and help define the method of approach. 

Stressors can include physical (including radiological) stressors or chemical or biological 

agents that may cause an adverse effect. The sources of the stressors can be human activities in 

sectors of society (e.g., manufacturing, transportation, agriculture, land development), personal 

activities (e.g., smoking, diet, and other lifestyle activities), or natural phenomena (e.g., forest 

fires, floods). Stressors that are not physical, chemical, or biological, such as economic or other 

quality-of-life stressors, may also be identified, but good techniques for including the effect 

these have on risk currently may not exist. 

Population elements are usually entities that are at risk, for example, communities, 

portions of ecosystem functions (e.g., those species that provide food for others within the food 

chain), or vulnerable subpopulations such as persons with certain diseases or persons at 

vulnerable life stages, such as children. The more specifically these can be defined, the more 

focused the analysis can be. This will be helpful in interpreting the results of the assessment. 

2.1.3. Agreeing on Participants, Roles, and Responsibilities 

The risk assessment planning team will usually recommend other groups or individuals 

who should participate in the assessment planning, scoping, and risk analysis phase. Depending 

on the schedule, approach, and level of effort envisioned for the risk assessment, there may be no 

additional participants or there may be many. The analytic portions of the assessment will 

usually require substantial technical expertise. Some disciplines that may be pertinent include 

toxicology, epidemiology, ecology, exposure assessment, fate and transport modeling (e.g., 

indoor and outdoor air, surface and drinking water), computer science (including geographical 

information systems [GISs]), chemistry, biology, various engineering fields (e.g., chemical, 

mechanical, industrial, civil), economics, sociology, and others. 

19
For the deliberative portions of the 

assessment, it may be that a number of 

stakeholders and other interested parties 

should be considered for participation (see 

box for examples). For community-based 

assessments, in particular, it is important that 

community involvement be sought and 

encouraged. The Presidential/Congressional 

Commission on Risk Assessment and Risk 

Management (PCCRARM, 1997) suggests the 

following questions to identify potential 

interested or affected parties (stakeholders): 

• Who might be affected by the risk 

management decision? (This includes 

not only groups that already know or 

believe they are affected, but also 

Examples of Possible Interested 

or Affected Parties (Stakeholders) 

State governments Affected industry 

Tribal governments Civic organizations 

Local governments Business owners 

Community groups Trade associations 

Grassroots organizations Labor unions 

Environmental groups Public health groups 

Consumer rights groups Academic institutions 

Religious groups Outdoors clubs 

Fishers and hunters Impacted citizens 

Civil rights groups Other federal agencies 

Source: Adapted from USEPA, 1999b 

groups that may be affected but as yet do not know it.) 

• Who has information and expertise that might be helpful? 

• Who has been involved in similar risk situations before? 

• Who has expressed interest in being involved in similar decisions before? 

• Who might be reasonably angered if not included? 

The importance of involving stakeholders in risk assessment is being increasingly 

recognized (e.g., NRC 1996; PCCRARM, 1997; USEPA 1996a, 1997a, 1998a, 1999c, 1999f, 

2000a). The Commission’s suggested guidelines for stakeholder involvement are shown in the 

box on the next page. 

There are several issues concerning the stakeholders’ capacity to participate that should 

not be overlooked by the risk assessment planning team. First, some stakeholders may need 

training to be able to participate in technical and risk management discussions. Second, as noted 

in the box on the next page, some stakeholders may require incentives such as travel funds or 

lodging at sites of meetings outside the area where they live. The risk assessment planning team, 

along with the potential source of funds for such incentives, should decide to what extent, if any, 

such incentives can be provided, based on the scope, level of effort, and financial constraints of 

the risk assessment project. 

The roles and responsibilities of technical and nontechnical participants (i.e., ground 

rules for participants) should also be proposed by the planning team, depending on the schedule, 

approach, and level of effort that is envisioned for the risk assessment. There will be several key 

20
Guidelines for Stakeholder Involvement 

• Regulatory agencies or other organizations 

considering stakeholder involvement should be 

clear about the extent to which they are willing or 

able to respond to stakeholder involvement before 

they undertake such efforts. If a decision is not 

negotiable, do not waste stakeholders’ time. 

• The goals of stakeholder involvement should be 

clarified at the outset and stakeholders should be 

involved early in the decision-making process. Do 

not make saving money the sole criterion for 

success or expect stakeholder involvement to end 

controversy. 

• Stakeholder involvement efforts should attempt to 

engage all potentially affected parties and solicit a 

diversity of perspectives. It may be necessary to 

provide appropriate incentives to encourage 

stakeholder participation. 

• Stakeholders must be willing to negotiate and 

should be flexible. They must be prepared to listen 

to and learn from diverse viewpoints. Where 

possible, empower stakeholders to make decisions, 

including providing them with the opportunity to 

obtain technical assistance. 

• Stakeholders should be given credit for their roles 

in a decision, and how their input was used should 

be explained. If stakeholder suggestions were not 

used, explain why. 

• The nature, extent, and complexity of stakeholder 

involvement should be appropriate to the scope and 

impact of a decision and the potential of the 

decision to generate controversy. 

Source: Presidential/Congressional Commission on 

Risk Assessment and Risk Management, 1997 

points in the risk assessment process where 

stakeholder input will be critical, such as 

agreement on purpose, scope, and approach. Each 

project should define and approve a list of 

critical points for stakeholder input. The 

team may even decide to assign stakeholders 

to subgroups that have specific tasks, such as 

understanding the technical information and 

reporting back to the larger group, elevating 

and clarifying stakeholder issues as needed, 

or providing information and facts to their 

peers and to the analysts. 

Sometimes citizens choose not to 

participate because they feel that they will 

not influence the outcome, that the issue is 

too complex or technical, or that the effort is 

too great or because the decision process is 

unclear (USEPA, 2001c). Moreover, despite 

the increased emphasis on stakeholder 

participation, there are instances where largescale stakeholder involvement may not be 

appropriate. EPA (as the decisionmaker) 

should determine whether, and to what 

degree, stakeholder involvement in a 

cumulative risk decision will be useful and 

what objectives it may accomplish. There is 

a continuum of objectives that may apply to 

individual cases, from exchanging 

information on one end, through obtaining 

stakeholder recommendations, to developing 

agreements for joint activities at the other 

end (USEPA, 1998g). 

Many of the activities and much of 

the data needed for cumulative risk 

assessment draw upon broad expertise, 

experience, and legal mandates found not 

only in EPA, but in other public health agencies and academia. The most successful cumulative 

risk assessments will likely be those where cooperation among organizations (Federal, State, 

private, environmental, academic, etc.) leads to use of the best data and tools for the various 

parts of the assessment. 

21
2.1.4. Agreeing on the Depth of the Assessment and the Analytical Approach 

The analysis approach (discussed further in Section 2.2.3 and Chapter 3) may fall 

anywhere on a continuum from relatively unsophisticated methods that rely heavily on default 

(and often conservative) assumptions, and consequently have greater uncertainty, to increasingly 

refined assessments in which data are substituted for assumptions and uncertainty is reduced. 

Some of the factors that go into deciding on the approach and associated plans for data collection 

include the level of uncertainty in the risk estimates that is acceptable to the participants, the 

intended use and audience for the assessment, the time and money resources available, and the 

amount, quality, and accessibility of data. 

Cumulative risk assessments, because of their nature, may require more attention to time, 

space, and route of exposure than do many traditional assessments, and methods should be 

chosen, if available, to accommodate these needs. In making the decision on approach, there 

will need to be an understanding of both the level of effort necessary for conducting the 

assessment selected, with an insight to alternatives, and the features and limitations of the 

selected approach in comparison to other approaches. 

2.1.5. Agreeing on the Schedule and Resources Available 

Schedule and resources are often interrelated. They may also determine whether the 

work is performed in-house by the organization or team desiring the assessment or by a 

contractor or other external source. The need to meet external deadlines or to coordinate with 

the schedules of other organizations may become overriding factors in defining what will be 

prepared. Assessments that require short-term, low-budget efforts or preliminary screening 

assessments may not have the scope, time, or resources for extensive stakeholder involvement. 

When there is extensive stakeholder involvement, it is especially important that a budget and 

time schedule be developed and known by all participants. 

2.1.6. Review of Lessons Learned 

Much time and effort can be saved by taking the advice of those who have already been 

through this process or similar processes. Risk assessment reports will often have a review 

chapter of lessons learned (or, “if I had to do this over again, I would. . . .”). We have tried to 

include some discussion of recent Agency experiences to illustrate parts of this framework 

report. In addition, the reader is encouraged to find similar advice in other reports (e.g., USEPA, 

2002b). For example, EPA’s Office of Water has conducted several watershed studies over the 

past decade and has compiled a Web page with lessons learned (USEPA, 2001d) (see box on the 

next page for one of the lists). Even though not all studies were cumulative risk assessments, 

much of the wisdom gained is relevant. 

22
Reed Holderman's Lessons Learned 

(California Coastal Conservancy, 

Santa Ynez Watershed) 

1. Be sure that [the project] is needed, and if it is, 

build community support for it before proceeding. 

2. Invite everyone into the process and ask political 

leaders to select the steering committee. Otherwise, 

people will ask, “Who appointed you?” 

3. Don't be presumptuous. On the Santa Ynez River, 

we assumed everybody would appreciate a well 

thought out scope of work, budget, and schedule. 

Wrong. They said it only proved that the whole 

thing was a set-up. Next time, let [the whole 

planning team] figure it out! 

4. When the majority of stakeholders tell you that 

they want to deal with their issue first, believe them. 

I remain convinced that our failure to sustain interest 

in the Santa Ynez River plan was primarily because 

we were not willing to assist the County in carrying 

out its proposed channel-clearing activities in the 

Lompoc valley as a separate and distinct project. 

5. Do whatever you can to break down barriers and 

perceptions people have of each other. Be creative. 

Family BBQs, softball games, and parties have done 

wonders to improve relationships among 

stakeholders and build trust. 

6. Maintain constant communication among 

stakeholders throughout the process—and especially 

in the beginning—to pass information along, answer 

questions, or deal with rumors. Whether it's through 

regular meetings, newsletters, web sites, phone trees, 

or all four, good communication is a must. 

7. And finally, line up your money and in-kind 

services in advance of starting your [assessment] 

project, or else two bad things will happen: (a) your 

stakeholders will buy into a process and scope of 

work only to find out they can't afford it; and (b) you 

will spend more time looking for cash than 

participating in the planning process. Either way, 

you lose. 

Source: USEPA, 2001d 

23
2.2. Problem Formulation, Conceptual Model, and Analysis Plan 

One outcome of the problem formulation phase is a conceptual model that is intended to 

identify relevant stressors, sources, pathways, exposure routes, receptors, and effects and the 

relationships among them. The conceptual model serves as a basis for the analysis plan, which is 

used to focus the analysis phase of the assessment. 

2.2.1. Problem Formulation 

Problem formulation is a systematic planning step that identifies the major factors to be 

considered in a particular assessment. It is linked to the regulatory and policy context of the 

assessment. Problem formulation is an iterative process within which the risk assessor develops 

preliminary hypotheses about why adverse effects might occur or have occurred. It provides the 

foundation for the technical approach of the assessment. The outcome of the problem 

formulation process is a conceptual model that identifies the stressors, the population exposed, 

and the assessment endpoints that will be addressed in the risk assessment and describes the 

relationships among them. One of the major differences between a cumulative risk assessment 

and a more traditional, single-chemical assessment is that in a cumulative assessment special 

attention should be given to identification of stressors and endpoints and the relationship 

between them. 

The box below shows desired outputs from the problem formulation phase of an 

environmental decision-making exercise. Although such an exercise is not precisely the same as 

a risk assessment, some of the outcomes are applicable, depending on the scope of the 

assessment. 

The Science Advisory Board’s Desired Outputs for Problem Formulation 

! The initial goals for the decision-making exercise, including environmental goals to be achieved 

! Which environmental problems/stressors/systems will be included and which will not, and the reasons for 

these decisions 

! The health, ecological, and quality-of-life effects of concern 

! The spatial, temporal, and organizational dimensions to data analysis 

! Scoping of the uncertainties involved and research needed to significantly reduce critical uncertainties 

! Initial review of the range of options available to reduce risks, considering likely economic, political, or 

other constraints 

! The endpoints upon which the condition of the ecological, human health, or societal systems ultimately will 

be judged 

Source: USEPA, 2000a 

24
2.2.2. Developing the Conceptual Model 

A conceptual model includes both a written description and a visual representation of 

actual or predicted relationships between humans (or populations or population segments) or 

ecological entities and the chemicals or other stressors to which they may be exposed. 

Conceptual models represent many relationships and may describe primary, secondary, 

or tertiary exposure pathways. The model is developed by the risk assessor and may include 

input from other experts (including stakeholders). The model narrative should distinguish—to 

the extent possible—between what is known or determined and what is assumed. Also, it should 

include a discussion of uncertainties in the formulation of the assessment and state how the 

assessment is cumulative, that is, for which sources, stressors/agents, pathways/exposure routes, 

receptors/populations, and endpoints. In some cases, conceptual models will be submitted for 

peer review. 

A general conceptual model (Figure 2-2) defines the components of such a model and 

shows the theoretical pathways and routes of exposure between the stressors (and sources of 

stressors) and effects (endpoints) for human and ecological receptors. The conceptual model 

includes factors and endpoints that may not be analyzed in the risk assessment but may be 

evaluated in the overall decision-making process. 

The conceptual model and the associated narrative show the basic rationale for the 

decisions made in pursuing a particular course of action. It provides a record of decisions for 

future reference during risk analysis and characterization and communication of the risk 

management decision. It is also valuable as a risk communication tool both within the Agency 

and in interactions with the public. The conceptual model provides a scientific or technical work 

product that includes (1) the scientific rationale for selecting the stressors, sources, receptors, 

exposed populations, exposure or environmental pathways, endpoints, or effects; (2) the 

scientific, technical, economic, or sociologic basis for the construction of the conceptual model; 

and (3) the scientific implications of additional data gathering. 

Figure 2-3 is an example of a conceptual model from the National Air Toxics Assessment 

(NATA).8 

It is not inconceivable, given the deliberative nature of the process of developing a 

conceptual model, that more than one model will be considered. If the team decides to 

ultimately use more than one model and to evaluate each as part of hypothesis testing, a careful 

consideration of time and monetary resources should be made, as well as a very careful 

consideration of how the results will be interpreted (see Section 2.3). 

8

 NATA is the technical support component of EPA’s National Air Toxics Program (64FR38706-38740; 

USEPA, 2001e). 

25 
A Generalized Conceptual Model

with Examples of Possible Elements and Linkages

Sources Stressors Pathways / Exposure

Routes

Receptors Endpoints

Activities that generate or release stressorsChemical, physical, or

biological 

agents that cause an effect

Surface water,

air, indoor air, groundwater, or soil are pathways.

Populations, ecological 

communities, and ecosystems may be receptors for some stressors. 

Ecological Ecological Endpoints 

Habitat structure, 

species distribution,

diversity 

Ecosystem conditions 

•Population levels 

•Environmental 

process rates 

Status of critical 

species or species of special concern. 

-Manufacturing

-Fossil fuel 

combustion 

(e.g., for transportation,

heating, electricity)

-Waster processing -Mining

-Agricultural 

activities 

-Natural 

processes 

Chemical 

-Organic 

-Inorganic 

Biological 

-Pathogens -Exotic spp. 

Physical 

-Thermal 

-Erosion/ sedimentation 

-Habitat 

alteration 

A variety of other factors (e.g., health status, access to health) may impact individual or 

population susceptibility to above stressors 

For individuals,

ingestion,

inhalation, or

absorption are the routes of 

exposure.

For ecological 

entities, biotic 

and abiotic 

interactions are keys to how 

stressors are presented to receptors. 

Human Health 

Individuals and 

groups of 

people are also receptors -Infants 

-Sensitive pop -Occupational 

-Minorities 

-Env. Justice 

Communities 

Mortality and 

Illness, such as: Cancers 

•Leukemia, lung, etc 

•Other adverse health effects •Asthma, respiratory impacts •Kidney disease •CNS effects 

•Etc. 

Endpoints are measures of

effects of stressors

Human Health Endpoints 

Figure 2-2. An example of a generic conceptual model 

(adapted from USEPA, 2002b).

26
27

Figure 2-3. Specific conceptual model for a complex project, OAQPS’ National Scale Air Toxic

PBT = persistent, bioaccumulative, and toxic 

CNS = central nervous system 

Source: USEPA, 2001e 
2.2.3. Constructing the Analysis Plan 

The analysis plan is the final stage of the planning and scoping process (see discussion in 

USEPA, 1998b). It describes how hypotheses about the relationships among the sources, 

stressors, exposure conditions, populations, and adverse effects/endpoints (see box) presented in 

the conceptual model and narrative will be considered during the risk analysis phase of the 

assessment. The plan includes the rationale for which relationships (referred to as “risk 

hypotheses” in USEPA, 1998b) are addressed, methods, models, and a discussion of data gaps 

and uncertainties. It also may include a comparison between the level of confidence needed for 

the management decision and that expected from alternative analyses in order to determine data 

Important Details for an Analysis Plan

Sources: 

Identification of sources to be included and methods and 

associated data for including them. 

Stressors: 

Identification of stressors to be included and methods and 

associated data for including them. 

Clarification of direct- and indirect-acting stressors. 

Exposure Conditions: 

Specification of exposure conditions to be assessed, along 

with methods. 

Populations: 

Identification of the populations on which analysis will 

focus. 

Endpoints or Adverse Effects: 

Identification of one or more unique, well-defined 

endpoint for analysis. Note that a concept such as “health 

of the community” is not a well-defined endpoint. 

Identification of linkages between assessment endpoints 

and measurable attributes. 

Specification of those endpoints or exposures that will be 

measured directly and those that will be estimated or for 

which surrogates will be used. 

Identification of common endpoints/effects for groups of 

stressors for which risks or impacts are to be combined. 

Description of methods to be employed for combining 

risks in terms of endpoints. 

needs and evaluate which analytical 

approach is best. In some cases, a phased 

or tiered risk assessment approach can 

facilitate management decisions, 

particularly in cases involving minimal 

data sets. 

The analysis plan provides a 

synopsis of measures that will be used to 

evaluate risk hypotheses (as shown in 

Appendix D). The plan is strongest when 

it contains explicit statements of how 

measures were selected, what adverse 

effect (or assessment endpoint) they are 

intended to evaluate, and which analyses 

they support. Uncertainties associated 

with selected measures and analyses and 

plans for addressing them should be 

included in the plan when possible. The 

analysis plan can be a brief summary of 

the key components of the risk 

assessment and how each component will 

be measured or calculated. 

In a cumulative risk assessment, a 

key aspect is considering whether and 

how multiple stressors interact or act 

together in contributing to risks; thus, 

some early thought should be given to the 

strategy for addressing this aspect of the 

assessment. The strategy should address 

methods to be employed for considering 

potential joint action of multiple stressors 

on a single endpoint as well as whether 

28
the assessment will attempt to describe cumulative impact on multiple endpoints. The discussion 

of this issue in the analysis plan can include both qualitative and quantitative approaches (see 

Chapter 3). 

As with the conceptual model, societal importance, complexity, and available data and 

resources will determine the degree of sophistication and detail needed in the analysis plan. Key 

data gaps should be identified. The plan should also include thoughts about how to fill the 

information needs in the near term using existing information, in the midterm by conducting tests 

with currently available methods to provide data on the agent(s) of interest, and over the long 

term to develop better, more realistic understandings of exposure and effects and more realistic 

test methods to evaluate agents of concern. The plan should explain how measures were 

selected, what they are intended to evaluate, and which analyses they support. Uncertainties 

associated with selected measures and analyses and plans for addressing them should also be 

explicitly stated. 

The analysis plan should include (where feasible) milestones for completing the risk 

assessment. The plan may be revisited and revised periodically. If new information is acquired, 

such revisions may refine hypotheses of exposure and toxicity, modify the risk hypotheses 

addressed, or compare public concerns with the projected risk management options. 

2.2.4. An Early Look at Uncertainty 

In preparing the conceptual model and analysis plan, there should be some early thinking 

about uncertainty. In Section 4.2.1, there is a discussion of different types of uncertainty that 

should be considered in the analysis: (1) parameter uncertainty (uncertainty about technical, 

scientific, economic, and political quantities), (2) model uncertainty (uncertainty about the 

appropriate functional form of technical, scientific, economic, and political models), and (3) 

disagreements among experts (e.g., about the values of quantities or the functional form of 

models, as when different health scientists use different forms of dose-response models). These 

considerations are important for interpreting the results of the study and should be considered in 

the selection of methods as part of the planning, scoping, and problem formulation process. 

The first of these uncertainties facing the planning team is the so-called epistemological 

uncertainty (not yet even knowing what questions to ask). It is likely that in planning any 

complex assessment, some questions will only become evident after the data collection or 

analysis has begun. It is therefore important that the planning team make provisions for 

revisiting the analysis plan—or even the conceptual model—at intervals during the process. 

Even more helpful would be an agreed-upon mechanism for changing the analysis plan or 

conceptual model before the need for revision arises, as it almost assuredly will. 

The second general aspect of uncertainty that should be dealt with in the planning, 

scoping, and problem formulation phase is “acceptable uncertainty.” How much uncertainty is 

the planning team willing to accept in the results of the study? Typically, this is a very difficult 

question for risk assessors and decisionmakers to answer, but it is a key question that 

enormously affects the cost and usefulness of the study. 

29
At one end of this acceptable uncertainty spectrum are risk assessments that are based 

only on readily available information. At the other end is an assessment that starts with carefully 

reasoned and detailed quality assurance parameters, leading to specific data being accepted or 

rejected for the study based on predetermined quality assurance guidelines. This process in turn 

leads to results with known and acceptable uncertainties, but it may either require expensive data 

collection or cause the study to fail when none of the data meet the quality assurance 

requirements. The planning team should decide where on this spectrum it wants to be for the 

study under consideration and whether the results will allow meaningful decisions. 

Again, the decision on acceptable uncertainty is difficult, but consideration early in the 

process will improve the potential for producing an analysis suited to the needs of the 

stakeholders. 

2.3. Ecological versus Human Health versus “Integrated” Cumulative Risk Assessment 

Cumulative risk assessments may include both human health and ecological aspects. 

Several reports have dealt with cumulative ecological risk assessment in some detail (e.g., Foran 

and Ferenc 1999; Ferenc and Foran 2000; USEPA 1998b). USEPA (2002c) noted some of the 

major differences between human health and ecological assessments (see list below), and these 

differences need to be considered when planning a cumulative risk assessment that includes both 

aspects: 

C Ecological systems are not as well known biologically as are human health systems, 

either at the population and at the individual level; 

C For this reason, and because biological communities and ecosystems are inherently 

more complex, ecological risk assessment requires more preliminary analysis and 

deliberation regarding endpoints and protective standards; 

C Ecosystems, habitats, and ecological communities have traits and properties that 

individuals do not or that are not applicable to individuals or populations; 

C Ecological risk assessment has been generally applied to populations, not individuals, 

whereas the reverse is true for human health risk assessments; and 

C Ecological risk assessment should assess risk at multiple levels or organization, that 

is, the molecule, cell, organism, population, community, and ecosystem. 

The World Health Organization (WHO, 2001) has published approaches to integrating 

human health and ecological risk assessments to improve data quality and understanding of 

cumulative risks for decision making. The organization’s approach includes an integrated 

framework (modified from USEPA, 1998b) and case studies. 

Many tribal cultures view ecological and human health in an integrated way such that 

they cannot be easily separated. Similarly, there is some effort (especially in Canada) toward 

integrating human health and ecological assessment as well as decisionmaking in a field known 

as “strategic environmental assessment” (Bonnell and Storey, 2000). This approach has not been 

applied widely in the United States, and it remains to be seen how it will develop in the next few 

years. 

30
2.4. The Final Step Before the Analysis Phase: Discussion of Possible Outcomes 

Before the analytical efforts of the cumulative risk assessment are started, it is useful for 

the entire team to hold some preliminary discussions about the possible results and their 

implications. Given that statutory mandates, regulations, property rights, or due process may 

constrain or define most or all acceptability criteria, what conclusions of the team will be 

associated with various results or risk levels? For example, for a risk assessment team with 

members from the community, industry, and local and other government entities, what would 

happen if the assessment shows risk levels to be “low”? Would members accept this? 

Conversely, if “unacceptable” risks are determined, will all team members accept the results and 

their potential responsibility to do something about that risk? Do team members understand the 

limitations of the information to be generated? 

Discussions like these will help determine whether the assessment can really address the 

questions of the team. If not, the assessment may not be worth doing as planned. If members of 

the team will not accept the possibility of a range of results, then it is important to reopen the 

entire planning and scoping discussion before anything is done in the analysis phase, because the 

planning and scoping phase has not been satisfactorily completed. Although it is not necessary 

to have unanimity among stakeholders before proceeding with the plan, knowing where some of 

the potential disagreements may occur after the analysis and risk characterization phases are 

started allows the stakeholders as a group to plan beforehand for how such disagreements will be 

addressed, should they occur. Although it is possible to ensure that all stakeholders have been 

heard and their opinions given due consideration and weight, that does not necessarily mean that 

all of them will get what they want. 

As an example, USEPA (2000f) is a case study where the stakeholders thought they had 

agreement on roles, responsibilities, and approach, only to find that the group acrimoniously 

splintered after the analysis results came back. The Baltimore report contains valuable lessons 

learned in the area of stakeholder disagreements and agendas and can provide some insight for 

planning teams.9 

Discussions just prior to the analysis phase may lead to an assessment that is very 

different from the one originally envisioned. For example, in the case of the cumulative risk 

initiative for Cook County (IL) and Lake County (IN) (see box on next page) the original plan 

was for a quantitative cumulative risk assessment, but because of the lack of some critical 

information, the scope was changed. This led to an assessment that, although not as broad as in 

the original plan—and that did not even directly calculate risk—had better stakeholder buy-in 

and a better chance of success in providing useful information. 

Finally, it should be acknowledged by all practitioners of cumulative risk assessment that 

in the current state of the science there will be limitations in methods and data available. It will 

9

 This case study, along with several others, will be examined more fully in followup work to this 

framework report. 

31 
Cumulative Risk Initiative (CRI) for Cook County (IL) and Lake County (IN) 

(formerly the Chicago Cumulative Risk Initiative, CCRI) 

CRI BACKGROUND AND OVERVIEW 

In 1995 the Chicago Legal Clinic and 11 Chicago-area community advocacy groups filed a petition under the 

Toxic Substances Control Act requesting that the EPA Administrator prohibit or further regulate emissions 

from eight proposed or constructed incinerators in the Chicago metropolitan area and Northwest Indiana. The 

petitioners believed that neither current statutes nor local siting laws adequately addressed cumulative impacts 

of multiple sources of toxic pollutants in a geographic area. They requested that the Administrator restrict 

emissions of dioxins, furans, mercury, lead, and cadmium from these sources. In May 1996 the petition was 

withdrawn in response to an EPA offer to participate in an investigation of multimedia pollutant impacts in 

Cook County, Illinois, and Lake County, Indiana. This effort became the CRI. A CRI is an attempt to 

investigate cumulative loadings and hazards from pollutant sources, to develop community-based activities to 

help address these concerns, and to use analytic results to help prioritize use of regulatory agency resources. 

EPA and the petitioners agreed to a four-phase project: (1) an environmental loadings profile (EPA 747-R-1-

002); (2) a petitioner risk workshop (completed); (3) a hazard screening assessment (peer review draft available 

January 2002); and (4) a risk-hazard management response. 

HAZARD SCREENING ASSESSMENT 

The CRI hazard screening assessment was authored primarily by Argonne National Laboratory, with input from 

local, State and Federal participants. Reflecting stakeholder deliberations, the report focuses on cumulative 

hazard (not “risk” as typically defined by EPA) associated with noncriteria air pollutants (“air toxics”) in the 

two-county study area. It relies on “off-the-shelf” air pollutant information, including EPA’s Toxics Release 

Inventory, Cumulative Exposure Project, Regional Air Pollutant Inventory Development System, and outdoor 

air monitoring data. Emission estimates are “toxicity weighted,” and modeled/monitored outdoor air pollutant 

concentrations are compared with reference values to develop hazard index-like ratios. The ratios or toxicityweighted emission estimates are used to derive indicators of cumulative hazard and then mapped over study 

area locations. To identify geographic areas where potentially elevated hazards and individuals with potentially 

greater susceptibility are collocated, another part of the study assembles pollutant hazard information and data 

on existing human disease rates and indicators. 

PRELIMINARY LESSONS LEARNED 

1. A major planning/scoping/problem formulation effort by a broad group of stakeholders narrowed the scope 

of the CRI hazard screening assessment and seemed to increase stakeholder “buy-in” with the process. This 

was valuable, given the complexity, expense, effort, time requirement, and difficulty encountered in addressing 

even the narrowed scope. 

2. Large data gaps make risk and hazard assessment of environmentally relevant chemical exposures highly 

uncertain, even for single agents. Expanded assessments that address cumulative risk considerations (e.g. 

mixtures, developmental toxicity, nonchemical agents) are a better match for real-world circumstances but 

require acknowledgment of even more uncertainty. 

3. Obtaining and managing input from a large group of technical stakeholders is cumbersome and timeconsuming, but that group’s perspective and expertise greatly improved the CRI assessment. 

4. Given that the National Research Council’s 1983 four-step “framework” required several years for broad 

use and acceptance in the United States, the greater complexity of cumulative risk (for CRI, cumulative hazard) 

assessment suggests that an equally long period may be needed for terminology standardization, refinement of 

approaches, and development of consensus methods. 

32
be important to identify these limitations and discuss them frankly in the cumulative risk 

assessment report. Data limitations may be somewhat mitigated by qualitative information; the 

collection of qualitative data may be valuable in cumulative risk assessment. Still, limitations in 

methods or data should not be seen as a convenient reason for completely ignoring or not posing 

questions for which stakeholders may be seeking answers. Lack of an appropriate methodology 

may indeed be a reason why certain questions cannot be addressed in the analysis phase, but 

capturing the questions and having some discussion about why the questions could not be 

addressed in the assessment is often helpful. 

33
3. THE ANALYSIS PHASE 

The analysis phase (Figure 3-1) is primarily an analytic process in which risk experts 

apply risk assessment approaches to evaluating the problem at hand.10 The risk assessment 

paradigm most widely used by risk assessors during the past two decades was first documented 

by the National Research Council (NRC) (NRC, 1983). It consists of four parts: hazard 

identification, dose-response assessment, exposure assessment, and risk characterization. This 

paradigm was developed when almost all risk assessments were being conducted on single 

chemicals. Nevertheless, it is a useful place to start when considering cumulative risks. 

This framework follows the NRC risk assessment paradigm in all respects except that the 

exposure and hazard/dose-response components should be evaluated together rather than 

separately. As a prerequisite to using this framework, assessors considering cumulative risk 

assessments should be familiar with the 1983 NRC risk paradigm as well as the various EPA risk 

assessment guidelines (see text box titled “EPA’s Risk Assessment Guidelines” in Section 1.1). 

In both single-stressor and multiple-stressor risk assessments, the analyst will look at 

hazard and dose-response relevant to the stressor(s) of interest and perform an analysis of 

exposure(s) to those stressor(s). This chapter begins with a basic discussion of this general 

process and its basic ingredients (Section 3.1). The second part of this chapter (Section 3.2) 

discusses some of the situations that arise in cumulative risk assessment, methods currently 

available for addressing them, steps in the process, and some limitations to these methods. 

Finally, Section 3.3 identifies areas of ongoing work that are particularly relevant to cumulative 

risk assessment. 

3.1. General Process 

In developing the conceptual model and analysis plan (see Section 2.2), the scope of the 

assessment was specified (see example in box on page 36). Some of the aspects of scope include 

stressors, sources, pathways and media, exposure routes, populations and subpopulations, 

endpoints, and measures. 

The analysis plan should specify how data, modeling, or assumptions will be 

obtained, performed, or defined for all of the details concerning the characterization of exposure 

of the defined population and subpopulations to the defined set of stressors. Additionally, the 

analysis plan specifies the strategy for obtaining and considering hazard and dose-response 

10 Although the analysis phase is primarily an analytic process, with heavy emphasis on the role of the 

scientist, risk assessor, or other technical expert, other stakeholders can be involved in various ways, as agreed upon 

before the analysis phase begins. Some roles stakeholders might have in the analysis phase include (1) suggesting 

sources of data or providing data for the assessment; (2) helping clarify issues identified during problem formulation; 

(3) working alongside the risk assessment experts to see what data and assumptions are being used and why and to 

better understand how the risk assessment process works; and (4) suggesting alternate scenarios that may reflect 

more realistic exposure conditions in the community. A variety of roles for stakeholders in the analysis phase can be 

proposed and adapted for the particular circumstances of the individual case, assuming that the roles can be agreed 

upon by the team. 

34 
Figure 3-1. The Analysis phase. 

information for these stressors and the method for combining the exposure information with the 

hazard and dose-response information to generate risk estimates or measures. As the risk 

analysis is refined, it may be appropriate to revisit and refine the exposure, hazard, and doseresponse information in an iterative fashion. 

In the integration of exposure, hazard, and dose-response information for a cumulative 

risk assessment, several aspects of the assessment may be particularly important. These include 

multiple-stressor hazard, dose-response and exposure issues, exposure time or duration-related 

issues, vulnerability (including susceptibility) of the study population along with the influencing 

factors (including life stage), and subpopulations with special exposures. These items are 

discussed in the following section along with the currently recognized methods for evaluating 

the toxicity or risk associated with mixtures. 

35
The area of identifying and 

assessing risk to susceptible 

subpopulations has an increased profile in 

cumulative risk assessments. A variety of 

factors may be influential in affecting 

population susceptibility. The extent to 

which these can be considered will be 

heavily dependent on existing knowledge 

and available information. 

3.2. Available Methods and Approaches 

Many aspects of traditional risk 

assessment methodology apply to 

cumulative risk assessment. Predicting 

cumulative risk of multiple stressors, 

however, has required the development of 

additional specific methods or approaches. 

Additionally, there are some aspects of risk 

assessment that, although common to both 

single-stressor and multiple-stressor 

assessments, may increase in complexity 

or significance in a cumulative risk 

assessment. 

Scope of EPA’s National-Scale 

Assessment for Hazardous Air Pollutants 

(also see Figure 2-3): 

Stressors 33 priority urban hazardous air 

pollutants (HAPs) 

Sources Major industrial, small “area,”

mobile (on- and off-road), and

extrinsic “background” in air 

Pathways/media Outdoor air, indoor air 

Microenvironments 

Routes Inhalation 

Subpopulations General population only 

Endpoints Cancers, developmental, central 

nervous system, kidney, liver, 

respiratory effects 

Metrics For cancer: distribution of high-end

cancer risk estimates, predicted

percent of population within predicted

cancer risk ranges, predicted number

of cancer cases, HAP-specific and 

cumulative 

For other effects: distribution of 

estimated hazard index values and 

estimated percent of population within

specified ranges of index values 

Although the aspects common to single-stressor and multiple-stressor assessments may 

be many (e.g., the added dimension of multiple stressors influences consideration of stressor 

sources, routes of exposure, environmental media/pathways, and other factors), several examples 

are cited here. As one example, the assessment of the dose-response relationship and the 

corresponding characterization of exposures in terms of duration, timing relevant to life stage, 

and exposure history gain an additional dimension with the need to consider them cumulatively 

in some way. The consideration of population susceptibility (as a part of vulnerability), as 

recommended by EPA (USEPA 1995a, b, 2000c), also increases in complexity. A third example 

of a complicating aspect in cumulative risk assessment is the consideration of subpopulations 

that have particularly distinctive exposures. These examples are further discussed in Section 

3.2.1. 

Although it is beyond the scope of this framework report to describe all risk methods in 

detail, Appendix B lists a variety of resources relevant to various exposure assessment methods. 

Relatively speaking, there is a great deal of information on assessing human and environmental 

exposures to chemical stressors and there is some information on biological and radiological 

stressors, but there is comparatively little information on many other types of stressors. 

36
The most prominent aspect of cumulative risk assessment is often the prediction of the 

combined effects of multiple stressors. Past and current activities in the development of 

approaches for predicting risk of multiple stressors are described in USEPA (1986b, 2000e). 

Concepts, approaches, or methods described in these documents or elsewhere are discussed in 

section 3.2.2, with clarification of their applicability, limitations and notable points regarding 

interpretation of the results they produce. 

3.2.1. Examples of Increased Complexity of Cumulative Risk Assessment 

Cumulative risk assessments can be quite complex (see text box on the following page 

for an example). Three factors that can increase complexity in a cumulative risk assessment are 

(1) time-related aspects, (2) vulnerability (including susceptibility), and (3) subpopulations with 

special or particularly distinctive exposures. All three are relevant in single-stressor 

assessments, but they have the potential to be more complicated in multiple-stressor assessments 

Time-related aspects. The issue of repeated exposures to a single stressor or exposures 

to multiple stressors that may vary in time dimensions may have implications for susceptibility, 

which, consequently, has implications for the dose-response relationship. Traditionally in doseresponse assessment, there is an inherent presumption that, for many stressors and effects, it is 

the aggregate exposure (the combination of intensity and duration) to which the organism 

responds (e.g., Haber, 1924). Thus dose-response assessments based on one pattern of exposure 

(e.g., 6 hours per day, 5 days per week over a lifetime) are routinely applied to the assessment of 

risk associated with a variety of patterns of exposure. 

In the case of linear carcinogens, this aggregate exposure assumption has been carried as 

an explicit assumption in the risk assessment step. Regardless of the details of the exposure 

circumstances in the study on which the cancer potency was based, it is assumed that there is a 

linear relationship between amounts of exposure and associated cancer risk. For nonlinear 

carcinogens11, and conceivably for linear carcinogens, if data indicate deviation from the 

assumption that cancer risk is proportional to lifetime dose, the details and sequence of exposure 

may be important, both in developing the dose-response relationship and in predicting risk 

associated with exposures and life stages of interest. 

Because some chemicals may have the ability to affect an organism’s response to other 

chemicals, consideration of the time sequence of exposure may take on an additional layer of 

complexity in multiple-chemical cumulative risk assessments. For example, persons with 

relevant past exposures might have increased susceptibility to the effects of a particular chemical 

due to a previous exposure to the same—or a second—chemical. 

11 The draft cancer guidelines (USEPA, 1999l) explicitly recognize the potential for nonlinear doseresponse. It is only in the case where nonlinear response is modeled that time sequence of exposure can be 

considered in the risk assessment. 

37 
The National-Scale Air Toxics Assessment 

The National Air Toxics Assessment (NATA), which is based on 1996 emissions data is an ongoing series of 

studies—some of which are completed—that will ultimately provide results that are useful in understanding the 

quality of air and its possible effect on human health nationwide. The assessment includes 32 air toxics (a 

subset of EPA's list of 188 air toxics) and also diesel particulate matter (which is used as a surrogate measure for 

diesel exhaust). Specifically, the assessment consists of four steps that will produce nationwide estimates of (1) 

the release of these pollutants into the air from various sources, (2) the concentration of these compounds in the 

air, (3) the exposure of populations to this air, and (4) the risk of both cancer and noncancer health effects 

resulting from this exposure. 

Purpose: The results of the national-scale assessment will provide important information to help EPA continue 

to develop and implement various aspects of the national air toxics program. They will not be used directly to 

regulate sources of air toxics emissions. Although regulatory priority setting will be informed by this and future 

national assessments, risk-based regulations will be based on more refined and source-specific data and 

assessment tools. More specifically, the assessment results will help identify air toxics of greatest potential 

concern, characterize the relative contributions to air toxics concentrations and population exposures of different 

types of air toxics emissions sources (e.g., major, mobile), and set priorities for the collection of additional air 

toxics data and research to improve estimates of air toxics concentrations and their potential public health 

impacts. Important additional data collection activities will include upgrading emission inventory information, 

ambient air toxics monitoring, and information on adverse effects to health and the environment; establishing a 

baseline for tracking trends over time in modeled ambient concentrations of air toxics; and establishing a 

baseline for measuring progress toward meeting goals for inhalation risk reduction from ambient air toxics. 

The Four Steps: The national-scale assessment includes the following four major steps for assessing air toxics 

across the contiguous United States (also Puerto Rico and the Virgin Islands). 

(1) Compiling a 1996 national emissions inventory of air toxics emissions from outdoor sources. The types of 

emissions sources in the inventory include major stationary sources (e.g., large waste incinerators and factories), 

area and other sources (e.g., dry cleaners, small manufacturers, wildfires), and both onroad and nonroad mobile 

sources (e.g., cars, trucks, boats). EPA made some modifications to the 1996 National Toxics Inventory to 

prepare the emissions for computer modeling. 

(2) Estimating 1996 ambient concentrations based on the 1996 emissions as input to an air dispersion model 

(the ASPEN model). As part of this modeling exercise, EPA compared estimated ambient concentrations to 

available ambient air toxics monitoring data to evaluate model performance. 

(3) Estimating 1996 population exposures based on a screening-level inhalation exposure model (HAPEM4) 

and the estimated ambient concentrations (from the ASPEN model) as input to the exposure model. Estimating 

exposure is a key step in determining potential health risk. People move around from one location to another, 

outside to inside, etc., so exposure is not the same as concentration at a static site. People also breathe at 

different rates depending on their activity levels, so the amount of air they take in varies. For these reasons, the 

average concentration of a pollutant that people breathe (i.e., exposure concentration) may be significantly 

higher or lower than the concentration at a fixed location (i.e., ambient concentration). 

(4) Characterizing 1996 potential public health risks due to inhalation of air toxics. This includes both cancer 

and noncancer effects using available information on air toxics health effects, current EPA risk assessment and 

risk characterization guidelines, and estimated population exposures. Using the toxicological independence 

formula and the default assumption of additivity of risks (USEPA, 1986b, 2000e), this assessment combines 

cancer risk estimates by summing them for certain weight-of-evidence groupings and also across all groupings. 

For noncancer effects, the assessment assumes dose additivity and aggregates or sums hazard quotients for 

individual air toxics that affect the same organ or organ system (USEPA, 2000e), in this case combining air 

toxics that act as respiratory irritants. 

38
These considerations suggest that for cumulative risk assessment, chemical exposures 

need to be characterized in terms of which other chemicals are present, and when. As noted in 

ILSI (1999), “Data collected specifically to support a cumulative exposure assessment should 

conserve the covariance and dependency structures associated with the chemicals of concern.” It 

is important to note, however, that the level of detail to which exposures are characterized should 

be closely tied to the level of detail of information available in the dose-response assessment, 

because a lack of corresponding detail in the dose-response assessment can pose a limitation on 

the interpretation and usefulness of detailed exposure estimates. 

Cumulative risk assessment can present challenges in matching exposure estimates with 

dose-response relationships. Ideally, the dose-response assessment will indicate whether the 

time sequence for the chemical(s) or stressors of interest in the assessment is important for risk 

estimation. In cumulative assessments involving chemicals for which the time sequence of 

exposure is important, it may be necessary to characterize the details and sequence of exposure 

to the exposed population (see text box on the following page), so that there will be a match in 

not only the form, but also in the assumptions between the dose-response relationship and the 

exposure/dose estimate. 

Vulnerability. One of the concepts that can be used in risk assessments (both for human 

health and ecological assessments) is that of vulnerability of the population or ecosystem. 

Vulnerability has been a common topic in socioeconomic and environmental studies. The 

European Commission’s TEMRAP (The European Multi-Hazard Risk Assessment Project), 

studying vulnerability to natural disasters such as floods, windstorms, fires, earthquakes, and 

others, defines vulnerability as “the intrinsic predisposition of an exposed element [organism, 

population, or ecologically valuable entity] to be at risk of suffering losses (life, health, cultural 

or economic) upon the occurrence of an event of [a specific] intensity” (European Commission, 

2000). Kasperson et al. (1995) defines vulnerability as “The propensity of social or ecological 

systems to suffer harm from external stresses and perturbations. Involves the sensitivity to 

exposures and adaptive measures to anticipate and reduce future harm.” Kasperson (2000) 

identified four types of vulnerability, discussed further below. 

The Agency’s risk characterization policy and guidance (USEPA, 2000c) touches on this 

concept by recommending that risk assessments “address or provide descriptions of [risk 

to]...important subgroups of the population, such as highly exposed or highly susceptible 

groups.” Further, the Agency’s guidance on planning and scoping for cumulative risk 

assessments (USEPA, 1995b) recognizes the importance of “defining the characteristics of the 

population at risk, which include individuals or sensitive subgroups....” That guidance also 

recognizes the potential importance of other social, economic, behavioral, or psychological 

stressors that may contribute to adverse health effects (e.g., existing health condition, anxiety, 

nutritional status, crime, and congestion). As discussed below, the ways in which the Agency 

and others describe these concepts in the context of human health risk assessment overlap the 

various ways described by Kasperson (2000) in which human and biological ecosystems, 

communities, and populations may be vulnerable: susceptibility/sensitivity, differential 

exposure, differential preparedness, and differential ability to recover. 

39
Examples of Exposure Models that Consider Time Aspects 

Calendex (Novigen Sciences, Inc) integrates different pathways (e.g., dietary [food and water] and residential) 

and routes of exposure (oral, dermal, inhalation) using a calendar-based probabilistic approach. One of the 

important factors of this approach is that it provides estimates of risk that reflect aggregate and cumulative 

exposure to discrete individuals, with exposure pathways and routes appropriately linked for the scenarios being 

assessed. Calendex also allows one to estimate exposure before and after the use of a chemical, as well as 

during degradation periods. Calendar-based assessments maintain the integrity of the individual by capturing 

the location of the exposed individual, the time of year in which he or she was exposed, and the patterns of 

exposure. Calendex also allows for a variety of time-breakout options for the analysis of exposure. 

APEX - The Air Pollution Exposure (APEX) model is based on the probabilistic National Ambient Air Quality 

Standards exposure model (pNEM) for carbon monoxide (Johnson et al., 2000). This model mimics the basic 

abilities of the pNEM/CO model; it calculates the distributions of human exposure to selected airborne 

pollutants within a selected study area as a function of time. As a dose model (for carbon monoxide), it 

calculates the pollutant dose within the body, specifically summarized by the blood carboxyhemoglobin 

(COHb) concentration. APEX is a cohort-microenvironment exposure model in that it combines daily activity 

diaries to form a composite year-long activity pattern that represents specific population cohorts as they move 

from one microenvironment to another. A cohort consists of a subset of the population that is expected to have 

somewhat similar activity (and hence exposure) patterns; it is formed by combining demographic groups and 

geographic locations (districts). Once each cohort has been modeled and its relative size determined, an 

exposure distribution for the entire population can be assembled. A microenvironment is a description of the 

immediate surroundings of an individual that serves as an indicator of exposure (e.g., inside a residence, school, 

or car; outdoors; etc.). APEX has been developed as one of the inhalation exposure models accessible in the 

Exposure Event Module of the Total Risk Integrated Methodology (TRIM.Expo) for assessment of exposures to 

either criteria or hazardous air pollutants (USEPA, 1999j) 

Other models include LifeLine, developed under a cooperative agreement between EPA/OPP and Hampshire 

Research Institute (HRI, 1999, 2000); Stochastic Human Exposure and Dose Simulation (SHEDS), under 

development by EPA’s Office of Research and Development (Zartarian et al., 2000), and Cumulative and 

Aggregate Risk Evaluation System (CARES), under development by member companies of the American 

Crop Protection Association (ACPA, 1999) along with Residential Exposure Year (RExY), which is being 

developed by Infoscientific.com. 

The first of Kasperson’s categories is susceptibility or sensitivity. Although these two 

words may have slightly different meanings, they are often used interchangeably. They refer to 

an increased likelihood of sustaining an adverse effect, and they are often discussed in terms of 

relationship to a factor describing a human subpopulation. For example, susceptible persons or 

populations may be those who are significantly more liable than the general population to be 

affected by a stressor due to life stage (e.g., children, the elderly, or pregnant women), genetic 

polymorphisms (e.g., the small but significant percentage of the population who have genetic 

susceptibilities), prior immune reactions (e.g., individuals who have been “sensitized” to a 

particular chemical), disease state (e.g., asthmatics), or prior damage to cells or systems (e.g., 

individuals with damaged ear structures due to prior exposure to toluene, making them more 

sensitive to damage by high noise levels) (Morata et al., 1997). Confronted with equal 

concentrations of a chemical for equal durations, for example, a biologically susceptible or 

40
sensitive individual may show effects, whereas the typical individual within the population 

would have no or less severe effects. This category would also include generation-skipping 

effects. Although we generally do not have a lot of data available on this topic, susceptibilities 

or sensitivities may also exist among races or genders. 

Kasperson’s second category of vulnerability is differential exposure. Although it is 

obvious from examining a dose-response curve that two individuals at different exposure levels 

may have a different likelihood of effects, this category extends to differences in historical 

exposure, body burden, and background exposure, which are sometimes overlooked in an 

assessment. When looking at the dose-response curves for a typical individual and an individual 

vulnerable due to differential exposure, the curves may be the same, but the vulnerable 

individual may be currently at a higher dose due to greater current or prior exposure and body 

burden, so an increment of additional exposure may (due to slope of the curve at that point) 

produce a more pronounced effect than in a typical individual. 

Kasperson’s third category of vulnerability is differential preparedness to withstand the 

insult of the stressor. This is linked to what kind of coping systems and resources an individual, 

population, or community has: the more prepared, the less vulnerable. As an example, consider 

two individuals, one of whom has had a childhood disease immunization shot and the other has 

not. The two may be exposed to the same insult, but due to a difference in preparedness, the 

effects on the person with the immunization shot may be much milder or nonexistent. As 

another example, hurricanes typically cause less damage to boarded up homes than they do to 

homes without this reinforcement, even though the weather insult to both homes may be the 

same. 

Kasperson’s fourth category is the differential ability to recover from the effects of the 

stressor. This again is linked to what kind of coping systems and resources an individual, 

population, or community has. One aspect of differential ability to recover is illustrated by 

differing survival rates for the same disease (e.g., Lantz et al., 1998). Put in terms of progression 

of disease, for example, two persons in an early stage of cancer have different prospects for 

recovery if one is treated immediately while the other does not have access to, or does not trust, 

health care. On the ecological side, opportunistic infections in marine mammals12 appear to be 

related to accumulation of polychlorinated biphenyls and organotin compounds, which cause an 

immunosuppression response in laboratory animals (Tanabe, 1998). 

Preparedness and ability to recover are often crucial factors in ecological assessments. In 

human health assessments, lack of access to health care, income differences, unemployment, or 

lack of insurance, for example, may affect a community’s ability to prepare for or recover from 

a stressor. 

Cumulative risk assessments may be uniquely suited to addressing the issues related to 

vulnerability. In order to do so, however, there should be some relationship between the factors 

12 That is, infections easily warded off by healthy marine mammals. 

41 
discussed above and changes in risk. Many of these factors have not yet been extensively 

developed beyond correlations between mortality rates and several socioeconomic factors, such 

as income (e.g., Lynch et al., 1998). Susceptibility has been more developed than the other 

factors, and current approaches implemented by EPA and others to address risk of noncancer 

endpoints routinely employ a 10-fold factor to address heterogeneity in sensitivity. Variability 

with regard to susceptibility is discussed in detail by NRC (1994), and the current state of 

knowledge concerning epidemiologically based (e.g., oncogene-specific) risk factors provides 

empirical data upon which at least crude estimates of the magnitude of heterogeneity in 

susceptibility to toxic response can be based. However, much research in this area remains to be 

done. 

Subpopulations with Special Exposures. Certain subpopulations can be highly exposed 

to stressors because of geographic proximity to the sources of these stressors, coincident direct 

or indirect occupational exposures, activity patterns, or a combination of these factors. The 

Agency’s risk characterization policy and guidance (USEPA, 2000c) includes recognition of the 

need for risk information to include, as available, information on highly exposed subgroups. 

Accordingly, risk assessments, including cumulative assessments, may need to put special 

emphasis on identifying and evaluating these subpopulations. 

Subpopulations at risk of high exposure due to geographic proximity could include 

workers at a facility that is a source of a stressor or residents near such sources. Specific 

examples might be people living downwind from a coal-burning power plant, those near and 

using a polluted water body (e.g., for fishing or recreation), or those living or working near 

roadways with high levels of vehicular traffic. Occupational exposures may be either direct 

(occurring in the workplace) or indirect (occurring at home). Indirect occupational exposures 

include those experienced by family members who may be exposed to occupational chemicals 

brought into the house by the worker (e.g., on clothing). Thus, workers or family members may 

be subject to greater exposures than others in the population who do not have this additional 

burden. 

Examples of subpopulations at high exposure due to activity patterns may include people 

who exercise heavily in polluted air, recreational or subsistence fishers or hunters who consume 

large quantities of fish or game, farmers or others who get a large proportion of their food from a 

location near a source of pollution and live in areas with high pesticide use, individuals who 

have long commutes in automobiles, or children (because they consume a larger amount of food, 

drink, and air relative to their body weight and because of additional exposure routes such as 

incidental soil ingestion). Additionally, some subpopulations may be affected by the combined 

impact of high geographic exposure and high-exposure activity patterns (e.g., runners who run 

along heavily traveled roadways and people who fish for food in heavily polluted urban rivers). 

It is important to recognize that some heavily exposed populations may also be 

particularly vulnerable or susceptible to the effects associated with the stressors of concern. 

Examples of those who could be particularly vulnerable to certain stressors include children 

during certain stages of development, people with chronic respiratory problems, the elderly, and 

those who are economically disadvantaged and do not have access to medical care. A 

42
cumulative risk assessment may need to take into account potential combinations of high 

exposure and high vulnerability, but few, if any, methods are currently available and accepted to 

address the combined effects of exposure and vulnerability. This is an important area for further 

research and methods development. 

3.2.2. Approaches for Predicting Risk of Multiple Stressors 

Combination toxicology (Carpy et al., 2000) is the study of the toxicity of mixtures. In 

such studies, one may either measure the mixture toxicity directly (whole mixture toxicity), or 

one may develop an estimate of the combined toxicity from information on the multiple 

component stressors acting in concert with each other. (Toxicity of chemical mixtures has also 

been modeled on a physiologically based pharmacokinetic basis [e.g., Haddad et al., 2000, 

2001].) If evaluated using its component chemicals, the mixture toxicity data set should be 

treated only as a snapshot of a multidimensional dose-response relationship, because the joint 

toxicity and interactions can change with changes in exposure route, duration, relative 

proportions of the components, or the effect being tracked. The application of such a data set to 

a specific situation then requires careful matching of the test mixture composition and exposure 

conditions to those of the target situation. In whole mixture toxicity, once the mixture toxicity is 

known, a risk evaluation can be done on the mixture using the 1983 NRC risk assessment 

paradigm. On the other hand, component-based mixture assessments are rarely evaluated using 

the strict NRC paradigm, because the exposure and toxicity information should be compatible, 

requiring some iteration to obtain toxicity information that is relevant to the actual exposure 

estimates (USEPA, 2000e). 

To address concerns over health risks from multichemical exposures, EPA issued 

guidelines for health risk from exposure to chemical mixtures in 1986 (USEPA, 1986b). The 

guidelines described broad concepts related to mixtures exposure and toxicity and included few 

specific procedures. In 1989, EPA published guidance for the Superfund program on hazardous 

waste that gave practical steps for conducting a mixtures risk assessment (USEPA, 1989a). Also 

in 1989, EPA published the revised document on the use of TEFs for characterizing health risks 

of the class of toxicologically similar chemicals that included the dibenzodioxins and 

dibenzofurans (USEPA, 1989b). In 1990, EPA published a technical support document to 

provide more detailed information on toxicity of whole mixtures and on toxicologic interactions 

(e.g., synergism) between chemicals in a two-chemical mixture (USEPA, 1990a). Wholemixture assessments, toxicologic independence and similarity, and risk methods using 

toxicologic interactions are discussed at length in USEPA (2000e). 

Risk assessment for mixtures usually involves substantial uncertainty. If the mixture is 

treated as a single complex substance, these uncertainties range from inexact descriptions of 

exposure to inadequate toxicity information. When viewed as a collection of a few component 

chemicals, the uncertainties also include the generally poor understanding of the magnitude and 

43
nature of toxicologic interactions, especially those involving three or more chemicals. Because 

of these uncertainties, the assessment of health risk from chemical mixtures should include a 

thorough discussion of all assumptions and the identification, when possible, of the major 

sources of uncertainty. 

3.2.2.1. Single stressor information 

Assessments that evaluate the risk from a single stressor do not fall into the category of 

cumulative risk assessments under the definition given in Section 1.3, whether these singlestressor assessments address a single (dominant) endpoint or multiple endpoints or whether the 

exposures are simple or complex (e.g., multisource, multipathway, multiroute exposure). Some 

may be termed “aggregate risk assessments” by extension of the FQPA terminology. They can, 

however, provide useful information for cumulative assessments. 

A cumulative risk assessment considers the joint impact of multiple stressors. Studies on 

individual stressors can, however, provide informative qualitative information for multistressor 

assessments, particularly regarding hazard identification. The collection of single-stressor 

effects can indicate the variety of types of adverse effects likely to result from the stressor 

combination, although perhaps not the magnitude or extent of the effects. Factors affecting 

population susceptibility to the individual chemicals are also likely to be important with the 

combined exposure. To go further in terms of quantitative risk assessment requires 

consideration of the potential for joint toxicity. For most exposure situations, hazard and doseresponse studies of all of the joint effects from the multiple stressors will not be available, so that 

conclusions will have to be based at least partly on the single stressor information. 

Exposure assessments for single stressors also need further consideration before they can 

be used to characterize long-term exposure to all the stressors by all pathways. Transport and 

environmental transformation of a chemical can be influenced by the presence of other 

chemicals. Consequently, both the exposure levels and the relative proportions of chemicals at 

future times may not correspond well to present measurements of a combination of chemicals 

unless these influences are taken into account. In addition, exposure to one stressor may 

influence the uptake of a second stressor. For example, a nonchemical stressor that increases 

ventilation rate will increase the inhalation uptake of airborne chemicals. 

Toxicologic independence. Two situations allow plausible approximations of the joint 

exposure-response relationship using only the single stressor information: toxicologic 

independence and toxicologic similarity (USEPA, 2000e). In the case of toxicologic 

independence, if the toxicity modes of action are biologically independent, then as long as there 

are no pre-toxicity interactions (e.g., metabolic inhibition, influence on uptake), the single 

stressor information is sufficient to approximate the joint exposure-response relationship. When 

the effects from two or more stressors are different, the cumulative response, if toxicologically 

independent, is merely all the single-stressor responses, as if the other stressors were not present. 

For example, joint but low exposure to heat (causing minor elevated heart rate and toluene 

(causing minor hearing loss) would be expected to cause both the minor heart rate elevation and 

minor hearing loss, but to the same extent as expected for each stressor alone. If each stressor is 

44
below its toxicity threshold, then for stressors exhibiting toxicologic independence, there will be 

no estimated cumulative response, because the set of individual responses is then a collection of 

zeros. 

When the single stressor and cumulative toxicities are each represented by a frequency or 

probability for affected individuals—also termed a probabilistic risk—then independence means 

that “response addition,” as defined in USEPA (2000e), can be applied for each adverse effect 

that the stressors have in common. When all the single-stressor risks are low, the joint risk of a 

common effect under response addition can be approximated by the simple sum of the singlestressor risks. For example, if reproductive toxicity is the general effect common to the multiple 

chemicals, the cumulative risk of reproductive effects (at low single-chemical risk levels) is 

approximately the sum of the single-chemical reproductive risks. Risk addition under 

independence places no constraints on the individual chemical dose-response curves. 

Toxicologic Similarity. In the second situation, the stressors are grouped according to 

the common mode of action for each effect of concern determined in the planning and scoping 

phase (USEPA, 2002a). For all effects caused by that mode of action, “dose addition” (USEPA, 

2000e) can be applied to the stressor group. Thus far, this approach has been used only with 

combinations of toxicologically similar chemicals, not with combinations of chemicals with 

other kinds of stressors such as radiation, physical factors, or health status. With similar 

chemicals, each chemical exposure is converted into the equivalent exposure level of one of the 

chemicals, called the index chemical. The joint toxicity or risk from the combined exposure is 

then estimated by determining the effects or risk for that equivalent exposure level using the 

dose-response information for the index chemical. For example, with the dioxins and furans (see 

text box on next page), each congener exposure level is converted into its equivalent exposure as 

the index chemical, 2,3,7,8-TCDD (USEPA, 1989b). 

Although the assumption itself is not complicated, the decision to assume toxicologic 

similarity can be complicated, depending on the level of assessment decided on in the planning 

and scoping phase and described in the analysis plan. The implementation used in Superfund 

assessments (USEPA, 1989a, Part D) is a rough approximation to dose addition where a 

hazardindex is determined whenever chemicals have a common target organ. The 

implementation by the Office of Pesticide Programs in support of FQPA (USEPA, 2002a) is 

much more extensive and requires knowledge of modes of action in order to calculate the 

Relative Potency Factors (RPFs) for the effect of concern (see example in Appendix E). The 

TEF method used for the dioxins is a special case of the RPF method (see Appendix E); it 

requires the most toxicologic similarity because the similarity applies to every toxic effect by 

any type of exposure (USEPA, 2000e). 

Single stressor information can also be used with dissimilar chemicals to gauge the 

potential for toxicologic interaction. For example, chemicals with long whole-body half-lives or 

long tissue residence times have the potential to be present in those tissues at the same time. 

Such overlapping exposures can result in a higher effective tissue dose, altered tissue doses 

caused by toxicokinetic interactions, or altered toxicity from interacting toxic mechanisms. 

45
Toxicologic Similarity: The Dioxin Reassessment 

Scientists from EPA, other Federal agencies, and the general scientific community have been involved in a 

comprehensive reassessment of dioxin exposure and human health effects since 1991 (USEPA, 2002d). The 

final dioxin reassessment will consist of three parts. Part 1: Estimating Exposure to Dioxin-Like Compounds 

will include four volumes that focus on sources, levels of dioxin-like compounds in environmental media, and 

human exposures. Part 2: Human Health Assessment Document for 2,3,7,8-Tetrachlorodibenzo-p-Dioxin 

(TCDD) and Related Compounds will consist of two volumes that include information on critical human health 

endpoints, mode of action, pharmacokinetics, dose-response, and toxicity equivalence factors (TEFs) Part 3: 

Integrated Summary and Risk Characterization for 2,3,7,8-Tetrachlorodibenzo-p-Dioxin (TCDD) and Related 

Compounds will be a stand-alone document. In this summary and characterization, key findings pertinent to 

understanding the potential hazards and risks of dioxins are described and integrated, including a discussion of 

all important assumptions and uncertainties. 

2,3,7,8-Tetrachlorodibenzo-p-dioxin (TCDD) is highly toxic to many animal species, producing a variety of 

cancer and noncancer effects. Other 2,3,7,8-substituted polychlorinated dibenzo-p-dioxins (PCDDs) and 

dibenzofurans (PCDFs) and coplanar polychlorinated biphenyls (PCBs) exhibit similar effects, albeit at different 

doses and with different degrees of confidence in the database. The similarities in toxicity between species and 

across different dioxin congeners stem from a common mode of action via initial binding to the aryl hydrocarbon 

(Ah) receptor. This common mode of action is supported by consistency in effects evident from data from 

multiple congeners. This has led to an international scientific consensus that it is prudent science policy to use 

the concept of TEFs to sum the contributions of individual PCDD, PCDF, and coplanar PCB congeners with 

dioxin-like activity (van den Berg et al., 1998). The data supportive of dioxin-like toxicity, both cancer and 

noncancer, are strongest for those congeners that are the major contributors to the risk to human populations. In 

addressing receptor-mediated responses resulting from complex mixtures of dioxin-like congeners, this 

assessment has provided a basis for the use of integrated measures of dose, such as average body burden, as more 

appropriate default metrics than daily intake. The Agency recognizes, however, that the final choice of an 

appropriate dose metric may depend on the endpoint under evaluation. 

In this study, 2,3,7,8-TCDD was chosen as the index chemical, and the other dibenzo-p-dioxins and 

dibenzofurans and coplanar PCB doses were adjusted to 2,3,7,8-TCDD-equivalent toxicities so the doses could 

be added. 

When a careful evaluation indicates no internal dose overlap, including metabolites, the single 

exposures might be considered independently. 

3.2.2.2. Information on stressor interactions and multiple exposures 

One important simplification that is common in single-stressor human health assessments 

is the separate evaluation of many of the key steps. That is, simplifying assumptions have often 

been made regarding many characteristics of exposure (e.g., continuous vs. intermittent 

variations in magnitude). For a given exposure route, for example, only one dose-response curve 

may be used for the bounding case of setting a cleanup or action level of exposure and also the 

predictive case of estimating existing risk. These simplifying assumptions allow the doseresponse step to be performed in isolation from the exposure assessment step, with the two steps 

executed in either order. For health-protective action levels, one may use bounds, such as the 

upper bounds on toxic potency and exposure and lower bounds on the resulting acceptable 

46
exposure level. Such bounds may be much easier to calculate, but they may be more difficult to 

interpret in terms of the uncertainties, likelihood, and closeness to the best or central estimate. 

The incorporation of multiple chemicals, other stressors, and multiple exposure 

conditions obviously complicates the assessment and the use of simplifying assumptions.13 In 

cumulative assessments, performing the exposure and dose-response steps of the risk assessment 

paradigm separately is an approximation that obviously invokes a simplifying assumption. If the 

dose-response data do not represent the same conditions as the exposure being assessed, an 

extrapolation has to be made, which introduces additional uncertainty that should be clearly 

stated. Joint or cumulative toxicity depends on the total dose or exposure, relative exposure 

levels, and the many characteristics of exposure (e.g., duration, continuous vs. intermittent 

presence, route, co-occurrence with other chemicals). In many cases, the complexities 

introduced by multiple stressors will not allow use of some of the common simplifying 

assumptions of single-stressor assessments. For example, toxicologic interactions have been 

shown to change when the same doses are used but the sequence of exposure is reversed (i.e., 

chemical B then A instead of A then B), so that the exposure and dose-response analysis should 

be compatible and performed together. 

Nonchemical stressors (e.g., biological entities or even physical stressors such as noise) 

can also interact with chemicals to change the risks either that would cause separately. For 

example, chemicals such as toluene can damage the auditory system and have been shown to 

potentiate the effects of a physical stressor, noise, on hearing loss (Morata et al., 1997; Morata, 

2000). For aquatic organisms, the toxicity of polyaromatic hydrocarbons increased with 

exposure to ultraviolet radiation (Oris and Geisy, 1985). 

Toxicity and interaction data that cover the full range of exposures for the exposureresponse relationship for the mixture of interest is usually impossible to obtain because of limits 

on budget and other resources. More feasible approaches to cumulative risk characterization, 

beyond that with various simplifying assumptions, then require close matching of the exposure 

and dose-response steps to minimize the data requirements. In many cases, screening-level 

ranking may be the only practical assessment. In some cases there will be sufficient information 

for some quantitative evaluation of cumulative health risks that reflect both the complex 

exposures and toxicologic interactions. 

“Joint chemical toxicity” means the outcome of exposure to multiple chemicals that 

includes the single-chemical effects along with any toxicologic interactions. Chemical 

interactions can be divided into two major categories: those resulting from a toxicokinetic mode 

of action and those resulting from a toxicodynamic mode of action (USEPA, 2000e). 

Toxicokinetic modes of interaction involve alterations in metabolism or disposition of the toxic 

chemicals, for example, by the induction or inhibition of enzymes involved in xenobiotic 

activation and detoxification. Toxicodynamic modes of interaction include those processes that 

13 For ecological risk assessments, Gentile et al. (1999) review the theory and several methods for 

evaluating stressor/response linkages and stressor interactions for multiple stressors. 

47 
affect a tissue’s response or susceptibility to toxic injury. A simplifying observation is that most 

interactions seem to involve pharmacokinetics. Unfortunately, most studies of toxicologic 

interaction to date have involved only two chemicals, and few have quantified the magnitude of 

the interaction or its dependence on exposure conditions. 

Toxicologic interactions are commonly described in terms of synergism or antagonism. 

These terms are only marginally useful, in part because the underlying toxicologic concepts are 

only defined for two-chemical mixtures, and most environmental and occupational exposures are 

to mixtures of many more chemicals. Further, the mathematical characterizations of synergism 

and antagonism are inextricably linked to the prevailing definition of "no interaction," instead of 

to some intrinsic toxicologic property (Hertzberg and MacDonell, 2002). EPA (USEPA, 2000e) 

has selected “dose addition” as the primary “no interaction” definition for mixture risk 

assessment, so that synergism would represent observed toxic effects that exceed those predicted 

from dose addition . The EPA mixture risk guidance also describes a modified hazard index that 

incorporates evidence of pairwise toxicologic interactions but notes that the pairwise evidence 

may be specific to the exposure conditions of the study. The guidance further encourages 

development of full biomathematical models for the joint toxicity—such as those based on 

pharmacokinetics—so that qualitative interaction labels such as synergism are replaced by 

quantitative estimates of mixture response that directly reflect the actual environmental exposure 

levels. 

3.2.2.3. Decision indices 

Among the complexities of cumulative risk assessments is the frequent need to combine 

widely differing types of data. Exposure data for some stressors may be available only as timeweighted averages, whereas other data reflect daily human activity patterns. Toxicity data for 

some chemicals may allow estimation of probabilistic risk for one endpoint while providing only 

qualitative descriptions of other endpoints. It is possible to develop the risk characterization 

using the original information in a matrix, but such a summary will be difficult to evaluate and 

communicate. One approach to diverse multivariate data that has been used successfully for 

weather forecasting is the decision index, with examples such as the smog index, the pollen 

count, and the mold index commonly used to assist in public and personal decisions about 

environmental exposure. A similar approach can be taken for cumulative risk assessment 

(Hertzberg, 2000). 

The advantage of a decision index is the simplicity in converting highly multivariate 

technical information into a single number. The most common example used for cumulative 

health risk is the hazard index for mixture risk (see box on next page). Although specific for a 

single affected target organ, each hazard index reflects multiple studies of multiple chemicals, 

often involving multiple test animal species and test exposures and highly varied measures of 

toxicity. 

48
The main disadvantage of a simple 

index is that the uncertainties in its 

calculation are largely hidden. Another key 

disadvantage is in quantifying what are often 

scientific judgments. For example, the 

hazard index implemented under Superfund 

(USEPA, 1989a) is a number whose decision 

threshold is usually given as 1.0, so that when 

the hazard index is greater than 1, additional 

action is indicated. The actual value of a 

hazard index is not that informative: a value 

of 6 is not necessarily twice as bad as a value 

of 3. This is partly due to the uncertainty 

factors necessary to develop the reference 

dose (RfD) or the reference concentration 

(RfC). The total value of these factors can be 

as low as 3 or as high as 3000, depending on 

The Hazard Index 

The hazard index (HI) for oral exposure is 

implemented by Superfund assessors by the formula: 

HI = sum[ HQj ] = sum[ Ej

/RfDj ] 

where Ej and RfDj are the daily exposure and 

reference dose of chemical j. 

The RfD is itself a kind of decision index in that it 

reflects a dose that is selected to be sufficiently low 

that any toxic effects are judged highly unlikely. All 

available dose-response data on all effects are 

considered in determining each RfD. Uncertainties 

in the RfD will differ across the chemicals, making 

the uncertainty in HI difficult to characterize. 

the data upon which the RfD or RfC for a specific chemical is based (Barnes et al., 1988; Beck et 

al., 1993; USEPA, 1994; Dourson et al., 1996; USEPA, 2002e). 

One alternative for addressing multiple effects is to recast these qualitative judgments in 

terms of severity categories or levels of concern (e.g., high/medium/low) and then use statistical 

methods such as categorical regression that use only the ordering of the severity scores but not 

their actual values. The result is not a risk of a particular toxic effect but rather a risk of 

exceeding a certain minimum toxic severity level or level of minimal concern (Hertzberg, 1989; 

Guth et al., 1993). In the best situations, such as the EPA interaction-based hazard index 

(USEPA, 2000e), the decision index formula is modular, so component pieces can be evaluated 

separately for accuracy and improvements in one area can be easily incorporated to give an 

improved index. 

Another example of a decision index with more overt display of its diverse parts is the 

Hazard Ranking System (HRS) (47 Fed. Reg. 31219, dated July 16, 1982, and amended 55 Fed. 

Reg. 51532, dated December 14, 1990), a formula developed for characterizing the relative 

hazards of a particular waste site. These hazards were highly diverse and include corrosivity, 

explosivity, toxicity, and soil conditions. As with the hazard index, different uncertainties in the 

components make the uncertainty of the HRS index difficult to describe. Instead of merely 

presenting the index as a number, a graphical presentation such as the star plots of multivariate 

data could be used (Chambers et al., 1983; Hertzberg, 2000), where each arm of the star 

represents one of the sub-indices. Although this approach shows the relative contribution of 

each factor, it again hides the uncertainties of the factors as well as of the HRS index itself. 

Hybrid methods that combine judgment with numerical descriptions of risk or doseresponse have also been used for complex risk assessments. The EPA interaction-based hazard 

index (USEPA, 2000e) and the mixture risk approaches of the Agency for Toxic Substances and 

Disease Registry (ATSDR) (Hansen, et al., 1998) both include a judgmental weight-of-evidence 

49
(WOE) score to reflect the strength of evidence for toxicologic interactions and relevance to 

human health risk. The ATSDR WOE is used in communicating risks and intervention options, 

whereas the EPA WOE is used to calculate a modified hazard index. A slightly different 

approach is the Integral Search System database program for combinations of carcinogens (Woo 

et al., 1994) by which available studies on pairwise interactions of carcinogenicity are used to 

modify the risk range of the combination from that predicted by response addition (USEPA, 

2000e). In all these cases, scientific judgment is used to alter the risk description or quantitative 

estimate, but only in terms of an approximate risk interval or a decision threshold. 

3.2.2.4. Probabilistic approaches 

The recent report by Bogen (2001) illustrates an alternative probabilistic approach to 

noncancer endpoints in which methods used for integrated quantitative treatment of uncertainty 

and variability are made consistent with those used for probabilistic assessment of cancer risk. 

This report addresses many issues concerning the implementation of probabilistic methods for 

noncancer endpoints and cites a number of related references (e.g., Lewis, 1993; Dourson et al., 

1997; Slob and Pieters, 1998). 

Any approach to cumulative risk assessment should carefully define the set of relevant 

endpoints. Precisely how this is done has important logical and practical implications for how 

the cumulative risk may be calculated and interpreted. For example, the risk of inducing a given 

endpoint may differ among different people in a population at risk for some endpoints, (e.g., 

cancer conditional on all carcinogen exposures) but may be unaffected by interindividual 

variability (e.g., in exposure or susceptibility) for other endpoints (such as ecological or aesthetic 

effects). Defining the latter risks in terms of individual risk per se will thus complicate 

calculating cumulative risk if a probabilistic approach to cumulative risk assessment is used and 

perhaps if other approaches are used as well. 

In contrast, the probabilistic approach to cumulative risk assessment may be facilitated 

by defining the risk of a given endpoint in terms of population risk, that is, in terms of the 

predicted number of cases of that endpoint. Alternatively (or additionally), similar simplification 

can be achieved for all heterogeneous endpoints by defining the risk only with respect to those 

persons in the population at risk who are reasonably maximally exposed (e.g., individuals 

adjacent to a proposed source) or to those persons who will incur the greatest increased risk (e.g., 

persons at a vulnerable life stage, such as children, or other members of a sensitive 

subpopulation who might be located adjacent to a proposed source). 

3.3. Areas of Complexity and Current Research 

One reason for the somewhat limited availability of cumulative risk assessments may be 

the accompanying complexity that arises in various aspects of the assessment. Some of this 

complexity is discussed in the previous section, along with currently available methods specific 

to human health risk assessment. In this section, some areas where research is ongoing are 

discussed, and some existing methods for quantitatively assessing multiple types of risk or 

hazard using a single metric are described. 

50
3.3.1. Interactions Between Stressors and Other Factors 

In identifying and characterizing susceptible subpopulations, it may be important to 

consider a variety of factors, such as current physical and mental health status and past exposure 

histories, that may exacerbate the effects of the stressors of interest. Social factors such as 

community property values, source of income, level of income, and standard of living may also 

affect vulnerability of subpopulations to certain other stressors. Risks associated with chemical 

or biological stressors may be significantly affected by “vulnerability factors” such as lack of 

health care or genetic predisposition to some diseases and effects. Community traditions and 

beliefs may affect activity patterns and behaviors and therefore affect exposure to stressors as 

well as the acceptability of the risk management options. Depending on the scope of the 

assessment and the stressors included, lifestyle factors such as smoking and nutritional habits, 

among others, may be important to susceptibility. 

In what could be characterized as an exploration of how somewhat abstract factors may 

affect susceptibility, ATSDR held an expert panel workshop on the subject of psychological 

responses to hazardous substances (ATSDR, 1995). In its report, the panel noted that there is “a 

significant lack of information” about how often communities near hazardous waste sites or 

spills suffer chronic stress reactions, but that psychological stress causes both psychological 

changes that can be measured by self-reports and objective tests as well as physical changes such 

as increased blood pressure and heart rate and biochemical parameters such as changes in stress 

hormones. Assessing the levels of stress and their potential contribution to risk is difficult for a 

variety of reasons. The report notes that 

unlike the damage and injuries caused by a natural disaster, many toxic 

substances are invisible to the senses.... In the face of no external cues and 

uncertain circumstances, each person affected by a hazardous exposure develops 

their own beliefs about the nature of the resultant harm. These beliefs are based 

on the facts available to them, pre-existing opinions, cultural factors, sensory 

cues, and the beliefs of leaders and others in the community.... Unlike a natural 

disaster, which hits and has a low point after which recovery can begin, the 

response to a hazardous waste site can take 12 to 20 years. 

Although the ATSDR report indicates that stress related to hazardous chemicals in the 

community can show measurable physical effects, it stopped short of saying that long-term 

health effects from this stress can be converted to risk estimates at this time. One of the 

questions the panel was asked to address was, “Given what is known regarding the psychology 

of stress, are there interactions between chronic stress and exposure to neurotoxicants that could 

shift the dose-response curve for neurotoxins?” The panel concluded: 

A methodology does not exist that would allow for discrimination between stress 

or neurotoxicant-mediated effects in community-based studies.... Experimental 

animal data exist to suggest that stress levels can modulate a toxic response; 

however, the question of specificity remains. Given that stress can induce or 

unmask a latent effect of a toxicant, there is the possibility that chronic stress 

51
could alter basal levels of neurofunctioning and shift the threshold for 

neurotoxicity. Indeed, one may find a shift in the dose response to a 

neurotoxicant; however, a specific effect of the neurotoxicant should be examined 

in greater detail than the generalized non-specific endpoints. Detecting such a 

shift would require the knowledge of toxicant-specific biological mechanisms of 

actions, which most often are not known. 

The ATSDR report contains many suggestions for research to fill data gaps in this area, 

and scientists may make significant progress in the coming years. 

“Quality-of-life” issues may also influence risk to health or the environment, and 

evaluating those issues may require an approach that differs from the traditional NRC risk 

paradigm. Although a cumulative human health or ecological health risk assessment is not a 

cumulative impact analysis such as is conducted under NEPA, changes in quality-of-life factors 

may affect the vulnerability of a population to health or ecological risks and consequently may 

be part of the considerations in a cumulative risk assessment. Because few, if any, established 

and accepted relationships are currently available to quantitatively link quality-of-life factors and 

health or ecological risk, this further research in this area may prove valuable. 

To evaluate the effects on human or ecological health from these types of stressors, a 

more deliberative approach (in the analytical-deliberative process) is needed than is used in, say, 

cancer risk analysis. EPA (USEPA, 1993b) suggests a six-step process that may help 

characterize quality-of-life factors, some of which may be relevant to the assessment (e.g., in 

considering population susceptibility). An example of a set of quality-of-life criteria developed 

by the State of Vermont’s Agency of Natural Resources is provided in Appendix F; however, it 

should be noted that quality-of-life issues can encompass much more than the criteria mentioned 

in this example. Some human health or ecological cumulative risk assessments may consider 

quality-of-life factors as having a role in susceptibility to the stressors being assessed. 

3.3.2. The Promise of Biomarkers and Biomonitoring 

There are a variety of measures that are inherently cumulative. These include biomarkers 

(they give the full effect or full exposure, regardless of source) and measures of the incidence 

and prevalence of disease in a community. The latter give an indication of the total effect of 

multiple sources of exposure. In light of our understanding of the multifactorial basis of disease, 

a public health approach that says “regardless of the cause, a community has x level of disease” 

can be informative. Such statistics can be compared across geographical areas that have 

different sources or groups that have different levels of vulnerability. The approach is based 

strongly in the field of epidemiology. Indeed, the most often-heard critique of 

epidemiology—that it is the prevalence or incidence of disease documented as a function of the 

combined effect of many exposures (over time and/or space)—is exactly what makes it so well 

suited for cumulative risk assessment. It is likely that epidemiological concepts will figure 

prominently in cumulative risk assessment, both in identifying the underlying vulnerability of a 

population and in generating hypotheses regarding the determination of relative contributions of 

multiple stressors (IPCS, 1983). 

52
Sources of data include cross-sectional analyses that determine prevalence levels as well 

as basic surveillance techniques. With respect to the latter, The Pew Environmental Health 

Commission (http://pewenvirohealth.jhsph.edu/html/home/home.html) has recently completed a 

series of reports that document the extent of national- and state-level resources for chronic 

disease surveillance. Reports focus on the type of surveillance systems needed as well as the 

status of registries for birth defects and asthma. Health Track (http://health-track.org/ and 

http://healthyamericans.org/) is the outgrowth of that research; it is devoted to tracking and 

monitoring chronic disease to help communities to identify patterns of health problems. 

Like epidemiologic data, some biomarkers reflect the cumulative history of individuals 

and populations. The use of biomarkers is based on the concept that the biological unit (organ, 

body, etc.) can be an effective and accurate element for integrating the aggregate exposures or 

doses or cumulative risks. Using biological measurements—biomarkers—to determine prior 

exposures (biomarkers of exposure) or the current health status of individuals (biomarkers of 

effect) holds some promise for cumulative risk assessments of the future (IPCS, 1993, 2001). 

Use of biomarkers for a group of chemicals or stressors that act upon individuals in the same 

way can give the assessor a picture of where an individual currently falls on the continuum from 

exposure to effects, making it much easier to predict risks if additional exposure occurs. 

A few biomarkers (or even a single one) can possibly represent exposure to a suite of 

chemicals. Although this approach reduces the analytical burden and simplifies the process of 

estimating cumulative risk, it loses some of the advantages of single-chemical assessment 

(especially being able to quickly discern the importance of different pathways and routes of 

exposure contributing to the risk). 

Biomarkers have a number of advantages; one disadvantage, however, is that they 

generally cannot link an effect to any particular exposure. For example, information on the 

cumulative risks in a local population of a group of chemicals that are toxic to the liver might be 

provided by selective liver function tests, but causal inferences would have to take account of 

many other factors that may affect liver function. Likewise, body burden data for chlorinated 

dioxins and related compounds may show that exposure has occurred, but assumptions would 

need to be made as to the pathways, route, and timing of exposures and scenarios developed for 

future exposures if risks are to be estimated. For a regulatory agency such as EPA, a decision to 

act to reduce risk often depends on separating contributions from exposure pathways so that 

effective policies can be determined. 

One of the benefits of the biomarker approach—the development of data that show the 

actual current exposure and risk status of a population—is also its major impediment: it can 

require extensive (or for humans, possibly invasive) monitoring. Monitoring data can be not 

only costly, but also difficult to obtain. This approach uses primarily measurement methods; it 

can also be used to develop statements of probability of adverse effects of additional incremental 

exposures. This approach holds great promise for simplifying cumulative risk assessments, but 

few methods exist at this time for such applications. Although this framework report provides 

53
only a cursory discussion of the biomarker approach, it is hoped that the planned guidelines for 

cumulative risk assessment will discuss this approach in greater detail. 

3.3.3. A Single Metric for Multiple Types of Hazard 

The most complex cumulative risk assessments will evaluate both multiple exposures 

(potentially, multiple sources, stressors, pathways, and durations) and multiple effects. Ideally 

this evaluation would provide projections regarding the potential for a particular complex 

exposure to cause particular effects to different physiological systems and also provide an 

integration of these projections into a qualitative characterization of overall potential impact to 

human health. Some applications have attempted to integrate the potential impacts across the 

different physiological systems. Approaches vary from treating the assessment as a number of 

multistressor, single-effect assessments, where the risks are combined only at the final step, to 

assessments that are more integrated throughout all the steps in the assessment process. 

For example, cumulative ecological risk assessments conducted in the Columbia River 

Basin and the Chesapeake Bay (Barnthouse et al., 2000) focused on a number of observed 

adverse conditions, and then determined from among all the possible stressors, which particular 

combination was most influential in creating those conditions. Stressors such as over harvesting 

of natural resources; modification of natural hydrology; land use change; point-source and 

nonpoint-source pollution, including toxic chemicals; and the presence of exotic species were 

analyzed, with the goal of designing effective restoration strategies to eliminate or ameliorate the 

conditions. 

If it is considered desirable for the assessment, an important activity may be to determine 

how (if at all possible) to combine risks from different effects—or the even-more-problematic 

disparate measures of risk—and present them in an integrated manner. Depending on the 

purpose and risk management objectives (see Section 2.1.1), some cumulative risk assessments 

may employ some sort of single, common metric to describe overall risk. 

One—but certainly not the only—approach to simplifying this problem is to collapse this 

“n-dimensional matrix” of hazards and risks into a few or even a single measure (Murray, 1994). 

However, this requires converting the various measures of risk into a common metric or 

otherwise translating them into a common scale or index. Some methods for combining 

disparate measures of risk are briefly described below. 

54
3.3.3.1. Creating a common metric 

As discussed earlier in this chapter, there are several different theoretical approaches to 

cumulative risk assessment. Some of them require synthesizing a risk estimate (or risk 

indication) by “adding up” risks from different parts of the risk picture. Actual mathematical 

addition, of course, requires a “common denominator,” or a common metric. Frequently used 

common metrics are risk, money, time, and effort. Finding a common metric for dissimilar risks 

(e.g., cancer vs. noncancer, human vs. ecological) is not strictly an analytic process, because 

some judgments should be made as to how to link two or more separate scales of risks. These 

judgments often involve subjective values, and because of this, it is a deliberative process. 

EPA’s Office of Pollution Prevention and Toxics has released a CD-ROM (USEPA, 

1999i)14 that shows an example of combining different effects into a common metric and the 

consequent judgment needed to achieve a common metric. In this model, emissions for both 

carcinogens and noncarcinogens are weighted by a toxicity factor so that they can be combined 

in a risk-based screening “score” for a particular geographic area. The scale for this weight for 

carcinogens is related to the unit risk factor, and the weight for the noncarcinogens is based on 

the RfD. According to the authors, it is possible to link these two different scales by making a 

deliberative judgment or assumption as to their relationship. They note that in their case, “when 

combining cancer and noncancer endpoints, it is assumed that exposure at the RfD is equivalent 

to a 2.5 x 10-4 cancer risk” (Bouwes and Hassur, 1998; USEPA, 1998h). 

Obviously, as Bouwes and Hassur acknowledge, equating an HQ15 value of 1.0 (i.e., 

exposure is at the RfD) with a cancer risk of 2.5 x 10-4 is a judgment that is outside the strictly 

analytic part of an assessment; the equating of the two points in the respective scales represents a 

value judgment and as such can be debated. Therefore, this particular part of the assessment is 

deliberative in nature. In making these types of judgments in a risk assessment, some care 

should be taken not to lose information in the aggregation, especially if all stakeholders do not 

agree on the relative tradeoffs necessary to arrive at the common scale of risk. If there is 

disagreement on constructing the scale, or even if more clarity is desired in the final report, the 

disaggregated risks can also be presented. Equity issues may also arise here, making it necessary 

to break out risks into relative burdens for different subpopulation. 

In most cases, construction of a single scale for different types of endpoints will involve 

comparative risk, a field where different types of risks or endpoints are ranked, compared, 

weighted, or converted to a scale on the basis of the judgments and values of the persons doing 

the assessments (USEPA, 1990b, 1993b, 1998f, 1999j). Groups of stakeholders such as are 

14 As of this writing, version 2.0 is in beta test. Details are available at 

<www.epa.gov/oppt/env_ind/beta_test.htm>. 

15 A hazard quotient, or HQ, in this context is the estimated exposure or dose level for a given individual 

chemical divided by the RfD (or RfC) for that chemical. Values of less than 1.0 for HQ indicate levels that are 

generally expected to be without effect during a lifetime. 

55 
gathered for cumulative risk assessment can provide ranking of various effects in terms of 

importance even if the effects cannot be put on a single scale or metric. This information may 

subsequently be used by decisionmakers for dealing with “worst risks first.” 

There have been some attempts to allow for transparent and quantitative incorporation of 

values into a common metric. One example flows from the suggestion that “time is the unit of 

measure for the burden of disease,” whether the disease results in disability or premature 

mortality (Murray, 1994). On this premise, economic analyses of the costs and benefits of 

disease intervention strategies have used quality-adjusted life years (QALYs) and disabilityadjusted life years (DALYs) as the metrics for the adverse effects of disease. These metrics are 

intended to reflect the years of life spent in disease states (considering the variation in severity of 

effects) and the years of life lost due to premature mortality resulting from disease as a surrogate 

measure for risk from a variety of different types of effect. 

Even if this conversion of effects into QALYs or DALYs is successful, for diseases that 

result in periods of morbidity and disability (but not death), weighting factors (based on 

judgments) are used to equate time spent in various disease states with years lost to mortality. In 

this way, dissimilar adverse effects can be combined to provide a single measure of disease 

burden. However, it should be noted that aggregation of effects in this manner obscures the 

meaning of the final measure. QALYs and DALYs do not represent an actual shortening of the 

life span but are indicators of the overall degradation of well-being that results from various 

disease states. 

Experience with applying such measures as QALYs and DALYs to environmental risk 

problems is extremely limited. Some very early methods development work has been initiated 

that explores the use of QALYs for combining microbial and disinfection by-product risks 

(USEPA, 1998f). However, some concerns have been raised about the adequacy of such 

measures, especially when integrated with economic information for decision making (USEPA, 

2000g). Further methods development work is needed to improve the usefulness of QALYs and 

DALYs for environmental risk assessments, especially with respect to the incorporation of 

uncertainty (USEPA, 1999j). 

Categorical regression may provide another tool for combining disparate effects using a 

common metric. In this approach, adverse effects are assigned to severity categories (again, a 

judgment making the process deliberative) and the ordered categories are regressed against 

increasing dose (Teuschler et al., 1999). This use of categorical regression puts definite limits on 

the interpretation of the results. Because the toxicities are only represented by categories and 

judgment is used to place the observed response into a severity category, the results are rather 

coarse. But because the analysis is almost totally empirical—that is, no low-dose extrapolation 

is required—the results can still be quite useful. 

EPA has also used decision indices (see Section 3.2.2.3) that are based on dissimilar 

measures, and although they do not produce risk estimates, they can still prove useful. The 

approach involves developing a composite score—or index—from measures of various risk 

dimensions. Various environmental risk indices have been developed and applied to ranking and 

56
comparative analyses. Often, these indices use surrogate measures for risk rather than actual 

calculations of the probability of adverse effects. One such index is the HRS, which is used to 

place uncontrolled waste sites on the National Priorities List for Superfund. This index is based 

on the likelihood of off-site movement of waste, the toxicity of the waste, and the people and 

sensitive environments that may be affected. It also uses corrosivity, toxicity, fire hazard, and 

other factors, which are scored and combined into one numerical indicator of overall hazard 

potential. Such an approach for a composite index has been suggested for the communication of 

cumulative risk (Hertzberg, 2000). 

Fischhoff et al. (1984) provides an example of this approach as applied to the evaluation 

of energy technologies. In this case, disparate risks are assigned a score from a fixed scale (e.g., 

from 0, representing no risk, to 100, representing the worst risk for that dimension). The scores 

are then weighted to reflect value judgments about the importance of the various risk 

dimensions, and the composite score is calculated by summing the individual weighted scores. 

Again, the aggregation of dissimilar adverse effects obscures the meaning of the final score, 

making this approach more appropriate for ranking and comparative analyses. 

Recently, EPA has been working on several index-based approaches to dealing with 

cumulative risk issues. EPA Region 3 and the Office of Research and Development have been 

jointly working to develop a potential risk indexing system (USEPA, 1993c, 1995d, 1997c). 

This index also uses a vulnerability index, and it gauges the overall well-being of a locale and 

various subpopulations. Again, the volume and toxicity of released stressors serve as surrogate 

measures of risk in developing this index. 

Combining the diverse effects and risk using either common metrics or indices has both 

pros and cons. A weakness of the index approach is that, by aggregating dissimilar information, 

information is “lost,” and the meaning of the final score can be obscured. However, both 

approaches have one strength in common: the ability to incorporate social values into the risk 

assessment in an explicit and quantitative manner. For example, in the derivation of DALYs, 

weights can be used to reflect the different social roles people play as they age (Murray, 1994). 

In the composite scores developed by Fischhoff et al. (1984), public concern was incorporated as 

an adverse effect. The ability to incorporate issues suach as public concern into the composite 

scores is an important feature for methods that will be applied to cumulative risk assessments, 

especially for communities. Given that cumulative assessments have a community/population 

focus, the ability to incorporate social values into an overall assessment of well-being will be 

critical. 

3.3.3.2. General issues regarding a single metric 

As described above, each approach to portraying the results of a cumulative risk 

assessment has desirable and undesirable features. Although common metrics and indices can 

incorporate social values in an explicit and quantitative manner, the meaning of the final 

measure can be obscured by the aggregation of dissimilar effects. The abstract nature of the final 

measure could lead to difficulties when communicating the results of the assessment to the 

public. The use of graphical and mapping techniques do not necessarily overcome 

57
communication problems. Although these techniques do not have some of the problems 

associated with the mathematical aggregation of dissimilar effects, it still may be difficult, for 

example, to accurately describe the information that a graphic is intended to convey. 

Because we have relatively little experience in combining different types of risk, a key 

issue is the need for methods development in this area. The approaches described above indicate 

a beginning. Additional exploratory work is needed, however, to further develop existing 

methods and to find additional methods that are flexible, that can incorporate social values, that 

are easy to communicate, and that provide an integrated portrayal of the overall well-being of a 

community and its various subpopulations. 

3.3.4. Qualitative Approaches 

There will be cases where cumulative risk cannot be quantified in any meaningful or 

reliable way. Qualitative approaches can be valuable for cumulative risk assessment and, in the 

near term, they may be the only practical way to address many of the complexities involved. 

Qualitative approaches may be used as a way to overcome the complexity and data deficiencies 

that hinder quantitative approaches. In many assessments, risk may not be a quantifiable 

variable. 

For these cases, there may be qualitative approaches that provide some insight. Broad 

indicators related to exposure in complex ways (e.g., production volumes, emissions 

inventories, environmental concentrations, etc.) and indicators of toxicity can be communicated 

using geographic information systems. Displaying complex, multidimensional matrices on a 

map can help in visualizing locations of areas with multiple stressors. Furthermore, 

geographically based measures of hazard are potentially useful cumulative measures; although 

they do not provide information on the risks, the locations of hazards can be used as an indicator 

of aggregate exposures and, thus, cumulative risks from all of the potential chemicals associated 

with that site. The environmental justice literature has used this approach. 

Quantitative results might eventually be reduced to a more qualitative scale (high, 

medium, or low), or the qualitative results could provide “comments” tacked onto the 

quantitative results. The assessment might simply raise red flags associated with specific issues 

(e.g. density of emitters in a community, presence of minority populations, special exposure 

pathways, etc); a high number of such flags would indicate unacceptable cumulative risk, even if 

the risk is not quantified. This approach has been used in the European Union (CEU, 1996), and 

its experience in using qualitative methods for permitting suggests that “qualitative” is not 

“irrational.” Other relevant tools include expert judgment techniques, focus groups, opinion 

surveys, citizen juries, and alternative dispute resolution. 

58
4. THE RISK CHARACTERIZATION PHASE 

The last phase of cumulative risk assessment, risk characterization, integrates and 

interprets the results of the analysis phase and addresses the problem(s) formulated in the 

planning and scoping phase (Figure 4-1). It should describe the qualitative and/or quantitative 

risk assessment results; list the important assumptions, limitations, and uncertainties associated 

with those results; and discuss the ultimate use of the analytic-deliberative outcomes. Given the 

complexity of cumulative risk issues and the need for clarity and transparency in risk 

characterization, such “full disclosure” presents a major communication challenge. 

There is a substantial analytical component of the risk characterization phase, but there is 

also a considerable need for deliberation. At a minimum, stakeholders in this phase should (1) 

understand the outcome of the cumulative risk assessment, (2) ask questions about how best to 

frame the interpretation, and (3) confirm that the cumulative risk assessment met the goals set in 

the problem formulation, or if not, why not. As in the previous phase, the stakeholders’ role is 

only limited by what is proposed and agreed upon in the particular case being assessed. 

Risk estimation in a cumulative risk assessment will involve some combination of risks, 

whether the risks from different stressors cause similar effects or different types of effects. The 

stressors themselves may be similar or widely different. Combinations of many types of 

stressors that have different endpoints will quickly cause the risk estimation step to become very 

complex and difficult. 

Because of its potential complexity, and because in some cases the cumulative risk 

assessment will be dealing methodologically with “uncharted territory,” it is very important that 

the planning, conduct, analysis, and characterization of an assessment be transparent. As stated 

by the Office of Management and Budget (OMB, 2002), the “benefit of transparency is that the 

public will be able to assess how much an agency’s analytic result hinges on the specific analytic 

choices made by the agency.” The process, methodology, data, assumptions, and selection 

among alternate interpretations should be very carefully documented and very clearly stated. 

This is noted again in the next section. 

59
Planning, Scoping, and 

Problem Formulation 

Analysis 

Interpretation and 

Risk Characterization 

Planning, Scoping, and 

Problem Formulation 

Analysis 

Interpretation and 

Risk Characterization 

Risk Description 

- Central Tendency and High-End 

Individual Risk 

- Population Risk 

- Risk to Important Subpopulations 

Uncertainty Analysis 

-Being Explicit about Uncertainty 

- Uncertainty and Variability 

- Uncertainty and Risk Addition 

- Sensitivity Analysis 

Information Provided by Cumulative Risk Assessment 

Using the Results of Cumulative Risk Assessment 

Figure 4-1. The Risk Characterization phase. 

60
4.1. Risk Description 

The ultimate product in the risk assessment process is the risk characterization, in which 

the information from all the steps is integrated and an overall conclusion about risk that is 

complete, informative, and useful for decisionmakers is synthesized. The nature of the risk 

characterization will depend on the information available, the regulatory application of the risk 

information, and the resources available (including time). It is important to identify and discuss 

Risk Characterization Guiding Principles 

Regarding information content and uncertainty aspects: 

< The risk characterization integrates the information 

from the exposure and dose-response assessments 

using a combination of qualitative information, 

quantitative information, and information regarding 

uncertainties. 

< The risk characterization includes a discussion of 

uncertainty and variability. 

< Well-balanced risk characterizations present risk 

conclusions and information regarding the strengths 

and limitations of the assessment for other risk 

assessors, EPA decisionmakers, and the public. 

Regarding risk descriptors: 

< Information about the distribution of individual 

exposures is important to communicating the 

results of a risk assessment. 

< Information about population exposure leads to 

another important way to describe risk. 

< Information about the distribution of exposure and 

risk for different subgroups of the population are 

important components of a risk assessment. 

< Situation-specific information adds perspective on 

possible future events or regulatory options. 

< An evaluation of the uncertainty in the risk 

descriptors is an important component of the 

uncertainty discussion in the assessment. 

Source: USEPA, 1995b 

all major issues associated with determining 

the nature and extent of the risk. Further, 

USEPA (1995a) specifies that a risk 

characterization “be prepared in a manner 

that is clear, transparent, reasonable, and 

consistent with other risk characterizations of 

similar scope prepared across programs in 

the Agency.” In short, estimates of health 

risk are to be presented in the context of 

uncertainties and limitations in the data and 

methodology. 

USEPA (1995b) lists several guiding 

principles for defining risk characterization 

in the context of risk assessment (see text 

box), both with respect to information 

content and uncertainty aspects and with 

respect to descriptions of risk. EPA has also 

published a handbook on risk 

characterization (USEPA, 2000c). 

Risk assessments are intended to 

address or provide descriptions of risk to one 

or more of the following: (1) people exposed

at average levels and people in the high-end

portions of the risk distribution, 

(2) the exposed population as a whole, and 

(3) important subgroups or life stage strata of 

the population (e.g., children) or other highly 

susceptible groups or individuals, if known. 

Risk predictions for sensitive subpopulations 

are a subset of population risks. Sensitive 

subpopulations consist of a specific set of 

individuals who are particularly susceptible 

to adverse health effects because of 

physiological (e.g., age, gender, pre-existing 

61
Some Thoughts on Risk Characterization 

Understanding Risk (NRC, 1996) focuses on risk both the magnitude and character of uncertainty.... 

characterization and reaches the following Getting the right science. The analysis has addressed conclusions: 

the significant risk-related concerns of public officials 

1. Risk characterization should be a decision-driven and the spectrum of interested and affected parties, 

activity, directed towards informing choices and such as risks to health, economic well-being, and 

solving problems. The view of risk characterization ecological and social values, with analytic priorities

as a translation or summary is seriously deficient.... having been set so as to emphasize the issues most

Risk characterization should not be an activity added relevant to the decision.

at the end of risk analysis; rather, its needs should Getting the right participation. The analytic- largely determine the scope and nature of risk deliberative process has had sufficiently broad analysis. participation to ensure that the important, decision2. Coping with a risk situation requires a broad relevant information enters the process, that all 

understanding of the relevant losses, harms, or important perspectives are considered, and that the 

consequences to the interested and affected parties. parties’ legitimate concerns about inclusiveness and 

A risk characterization must address what the openness are met. 

interested and affected parties believe to be at risk in Getting the participation right. The analytic- the particular situation, and it must incorporate their deliberative process satisfies the decision makers and perspectives and specialized knowledge. interested and affected parties that it is responsive to 

3. Risk characterization is the outcome of an analytic- their needs: that their information, viewpoints, and 

deliberative process.... Analysis and deliberation can concerns have been adequately represented and taken 

be thought of as two complementary approaches to into account; that they have been adequately consulted; 

gaining knowledge about the world, forming and that their participation has been able to affect the 

understandings on the basis of knowledge, and way risk problems are defined and understood. 

reaching agreement among people. Developing an accurate, balanced, and informative 

4. The analytic-deliberative process leading to a risk synthesis. The risk characterization presents the state 

characterization should include early and explicit of knowledge, uncertainty, and disagreement about the 

attention to problem formulation. risk situation to reflect the range of relevant knowledge 

and perspectives and satisfies the parties to a decision 5. The analytic-deliberative process should be that they have been adequately informed within the

mutual and recursive.... A recurring criticism of risk limits of available knowledge. characterization is that the underlying analysis failed

to pay adequate attention to questions of central 6. Those responsible for a risk characterization should

concern to some of the interested and affected parties. begin by developing a diagnosis of the decision situation

This is not so much a failure of analysis as a failure to so that they can better match the analytic-deliberative

integrate it with broadly based deliberation: the process leading to the characterization to the needs of

analysis was not framed by adequate understanding the decision, particularly in terms of level and intensity

about what should be analyzed.... Structuring an of effort and presentation of parties.... Diagnosis of risk

effective analytic-deliberative process for informing a decision situations should follow eight steps: (1)

risk decision is not a matter for a recipe. Every step diagnose the kinds of risk and the state of knowledge,

involves judgment, and the right choices are situation (2) describe the legal mandate, (3) describe the purpose

dependent. Still, it is possible to identify objectives of the risk decision, (4) describe the affected parties

that also serve as criteria for judging success: and anticipate public reactions, (5) estimate resource

needs and timetable, (6) plan for organizational needs, Getting the science right. The underlying analysis (7) develop a preliminary process design, and (8)

meets high scientific standards in terms of 

measurement, analytic methods, databases used, 

summarize and discuss the diagnosis with the 

plausibility of assumptions, and respectfulness of responsible organization. 

62
conditions), socioeconomic (e.g., nutrition), or demographic variables or because of significantly 

greater levels of exposure (USEPA, 1992a). Subpopulations can be defined using age, race, 

gender, and other factors. If enough information is available, a quantitative risk estimate for a 

subpopulation can be developed; if not, then any qualitative information about subpopulations 

gathered during hazard identification should be summarized as part of the risk characterization 

(USEPA, 2000c). The box on the previous page summarizes some of the points made in 

Understanding Risk (NRC, 1996), which devotes a great deal of discussion to risk 

characterization. Risk characterization is most efficiently conducted with early and continued 

attention to the risk characterization step in the risk assessment process (NRC, 1996; USEPA, 

2000c). 

4.2. Uncertainty Analysis 

Morgan and Henrion (1990) note that, historically, the most common approach to 

uncertainty in policy analysis (including in risk assessment) has been to ignore it. In a section 

titled “Why Consider Uncertainty?” they advance three primary reasons, all of which are 

especially relevant to an analytic-deliberative process such as cumulative risk assessment. They 

suggest that it is important to worry about uncertainty 

• “when one is performing an analysis in which people’s attitude toward risk is likely 

to be important, for example, when people display significant risk aversion; 

• “when one is performing an analysis in which uncertain information from different 

sources must be combined. The precision of each source should help determine its 

weighting in the combination; and 

• “when a decision must be made about whether to expend resources to acquire 

additional information. In general, the greater the uncertainty, the greater the 

expected value of additional information.” 

Morgan and Henrion provide 

ten requirements for good policy 

analysis, and although all are 

commendable and several have been 

discussed elsewhere in this framework 

report, we should look more closely at 

numbers 6–8 in the box at right for 

some insight into uncertainty analysis. 

There are many resources 

available that talk in detail about how 

to perform uncertainty analysis (e.g., 

USEPA, 1997b; Morgan and Henrion, 

1990). Although detailed instruction is 

beyond the scope of this framework 

report, we believe that a discussion of 

some general principles is in order. 

Morgan and Henrion’s Ten Requirements 

for Good Policy Analysis 

1. Do your homework with literature, experts, and users. 

2. Let the problem drive the analysis. 

3. Make the analysis as simple as possible, but no simpler. 

4. Identify all significant assumptions. 

5. Be explicit about decision criteria and policy strategies. 

6. Be explicit about uncertainties. 

7. Perform systematic sensitivity and uncertainty analysis. 

8. Iteratively refine the problem statement and the analysis. 

9. Document clearly and completely. 

10. Expose the work to peer review. 

Source: Morgan and Henrion, 1990 

63
4.2.1. Assumptions in the Assessment 

Cumulative risk assessment will typically be used in a decision-making process to help 

inform the decisionmaker(s). For this reason, it is important that the decisionmakers be made 

explicitly aware of any assumptions that may significantly affect the conclusions of the analysis 

(requirement number 4 in the box on previous page). Morgan and Henrion (1990) suggest that 

these assumptions include: 

• the main policy concerns, issues, or decisions that prompted the assessment, 

• the evaluation criteria to be used to define issues of concern or options, 

• the scope and boundaries of the assessment and ways in which alternate selections 

might influence the conclusions reached, 

• soft or intangible issues that are ignored or inadequately dealt with in the quantitative 

analysis (e.g., intrinsic value of wilderness, equity of distribution of risks and 

benefits), 

• approximations introduced by the level of aggregation or by level of detail in models, 

• value judgments and tradeoffs, and 

• the objective function used, including methods of combining ratings on multiple 

criteria (or combining risk scales). 

Identifying significant assumptions can often highlight “soft” uncertainties that are not 

easily quantified and are therefore often left out of a quantitative uncertainty analysis. 

Nevertheless, these “soft” uncertainties can often contribute more to the overall uncertainty of 

the assessment than the factors more easily quantified. 

Morgan and Henrion’s sixth requirement for good policy analysis (see box on previous 

page) includes three types of uncertainty that analysts should explicitly address: 

• Uncertainty about technical, scientific, economic, and political quantities (e.g., 

quantities such as rate constants often lend themselves to quantitative uncertainty 

estimates relatively easily); 

• Uncertainty about the appropriate functional form of technical, scientific, economic, 

and political models (e.g., are the models used, such as dose-response models, 

biologically sound?); and 

• Disagreements among experts about the values of quantities or the functional form of 

models (e.g., different health scientists using different forms of dose-response 

models). 

In requirement number 7, Morgan and Henrion suggest that an assessor should find out 

which assumptions and uncertainties may significantly alter the conclusions, and that this 

process can be conducted using sensitivity and uncertainty analysis. Techniques include: 

• Deterministic, one-at-a-time analysis of each factor, holding all others constant at 

nominal values; 

64
• Deterministic joint analysis, changing the values of more than one factor at a time; 

• Parametric analysis, moving one or a few inputs across reasonably selected ranges to 

observe the shape of the response; and 

• Probabilistic analysis, using correlation, rank correlation, regression, or other means 

to examine how much of the uncertainty in the conclusions is attributable to which 

inputs. 

Finally, Morgan and Henrion answer the question of why we should consider uncertainty 

analysis with the following point. “Policy analysts have a professional and ethical responsibility 

to present not just ‘answers’ but also a clear and explicit statement of the implications and 

limitations of their work. Attempts to fully characterize and deal with important associated 

uncertainties help them to execute this responsibility better.” 

4.2.2. Uncertainty and Variability 

NRC (1994) notes a clear difference between uncertainty and variability and recommends 

that the distinction between these two be maintained: 

A distinction between uncertainty (i.e., degree of potential error) and interindividual variability (i.e., population heterogeneity) is generally required if the 

resulting quantitative risk characterization is to be optimally useful for regulatory 

The Cumulative Exposure Project 

EPA’s Cumulative Exposure Project (CEP), 

completed in 1998, modeled 1990 outdoor 

concentrations of hazardous air pollutants (HAPs) 

across the United States, which were combined with 

unit risk estimates to estimate the potential increase 

in excess cancer risk from multiple HAPs. The 

cancer risks of different HAPs were assumed to be 

additive and were summed across pollutants in each 

census tract to estimate a total cancer risk in each 

census tract. 

Consideration of some specific uncertainties, 

including underestimation of ambient concentrations, 

combining upper 95% confidence bound potency 

estimates, and changes to potency estimates, found 

that cancer risk may be underestimated by 15% or 

overestimated by 40-50%. Other unanalyzed 

uncertainties could make these under- or 

overestimates larger. 

Source: Woodruff et al., 2000 

purposes, particularly insofar as risk 

characterizations are treated 

quantitatively. The distinction 

between uncertainty and individual 

variability ought to be maintained 

rigorously at the level of separate riskassessment components (e.g., ambient 

concentration, uptake, and potency) as 

well as at the level of an integrated 

risk characterization. 

Variability and uncertainty have been 

treated separately and distinctly in singlechemical assessments such as the assessment 

of TCE in ground water at Beale Air Force 

Base in California (Bogen, 2001). The 

treatment of variability and uncertainty will 

also be an important issue in cumulative risk 

assessments, although at the time of this 

writing there are no good examples of an 

elegant treatment of this issue for cumulative 

risk. 

65
4.2.3. Uncertainty and Risk Addition 

Calculating individual stressor risks and then combining them largely presents the same 

challenges as combination toxicology but also adds some statistical stumbling blocks. Toxicity 

addition, independence, synergism, or antagonism still need to be evaluated, but because risk 

estimates for various stressors are often presented as values on the same numeric scale (e.g., as 

cancer probabilities), cancer risks are often simply added together. 

Because cancer slope factors are not “most probable estimates,” but rather 95% upper 

confidence levels, adding traditional risk levels can cause the resulting sum to overestimate a 

95% upper confidence level risk for a mixture. There have been several recent papers discussing 

this problem and how it may affect the resulting estimates. Kodell and Chen (1994) looked at 

several binary mixtures and calculated that the summation of individual upper 95% confidence 

intervals for chlorobenzene and hexachlorobenzene would overestimate the upper-bound risk of 

a binary mixture of these compounds by 2 to 6%, whereas for chlorobenzene and TCE the 

overestimate would be in the range of 12 to 15%. Seed et al. (1995) noted that, “in most cases, 

the magnitude of the difference in cancer risk estimates calculated by [Kodell and Chen’s] 

various methods will be greatest for mixtures of equipotent compounds. However, even for 

mixtures of equipotent compounds, the differences in joint risk estimated by summing the upper 

95% confidence levels...are not great.” 

After analyzing four cases, Cogliano (1997) concluded that “as the number of risk 

estimates increases, their sum becomes increasingly improbable, but not misleading.” For 

example, in adding 20 different cancer risk estimates based on a 95% upper bound, the resulting 

sum of the upper bounds was no more than 2.2 times the true upper bound. Cogliano went on to 

suggest that, for certain cases not involving synergistic or antagonistic interactions, “depending 

on the number of carcinogens and the shape of the underlying risk distributions, division by a 

factor of 2 can be sufficient to convert a sum of upper bounds into a plausible upper bound for 

the overall risk.” 

The assumption of toxicologic independence (see Section 3.2.2) may not be a bad one if 

other evidence supports it, but it should be addressed in the assessment if used (i.e., if risks are 

added). Although some scientists believe that toxicologic interactions are of minor consequence 

at concentrations observed in the environment (see discussion in USEPA, 2000e), the scientific 

evidence for such an assumption has not been firmly established. 

Notwithstanding the statistical limitations of adding traditional risk estimates and the 

implicit assumption that the toxicities will be additive16 (i.e., no interactions such as synergism 

or antagonism occur), the numerical ease for combining risks in this way may make it the most 

16 At risk levels often seen with pollutant concentrations observed in the environment, the combined risks 

calculated assuming “response additivity” (that is, each component acts as if the other were not present) are 

approximately the same as with dose additivity (USEPA, 2000e). 

66 
popular method for approximating cumulative risks in the short term, at least at a screening level 

of assessment. 

4.2.4. Other Cumulative Risk Assessment Uncertainties 

This framework does not, and cannot, provide an exhaustive list of uncertainties unique 

to cumulative risk assessment. Without question, however, there will be uncertainties inherent in 

a cumulative risk analysis that have not been as important in traditional assessments. As an 

example, because cumulative risk assessments can be geographically based and GIS technology 

seems to be a potentially useful tool for displaying results, there will be issues concerning how to 

present uncertainty information, for example, by overlaying impacts or risks for several 

chemicals, on a GIS map. 

Specific uncertainties can arise when adding doses for chemicals that operate by the same 

mode of action, such as the organophosphorous (OP) pesticides. In this case, USEPA (2002c) 

notes that uncertainties arise in estimating the RPFs of the OP pesticides. These RPF 

uncertainties can be partitioned into three groups: those that are basic (e.g., uncertainty in the 

dose-response relationship for the reference chemical), those that deal with chemicals in relation 

to one another (relative potencies of other chemicals relative to the reference chemical), and 

those concerning joint mode of action (e.g., members of the common mechanism group may 

have other modes of action that are not fully captured via the common-mechanism potency 

calculation). As risk assessors develop more experience with cumulative risk assessments, many 

more of these uncertainties may arise, but it is not possible to foresee all of them. 

4.3. Information Provided by Cumulative Risk Assessment 

It is important to clarify how cumulative risk assessment and this framework report relate 

to community assessments and community decision making. Certainly, the Agency’s risk 

characterization handbook (USEPA, 2000c) emphasizes that whatever information is imparted 

should be transparent, clear, consistent, and reasonable. For example, if it is known that the 

results of a particular cumulative risk assessment will be severely limited because of a lack of 

data or available methods, it may be advisable to start with a screening analysis to set priorities 

for a subsequent study that is more detailed and focused. In simple terms, what can a cumulative 

risk assessment tell us, and what can’t it tell us? 

4.3.1. Making Sense of Multiple-Stressor Effects 

The information provided by cumulative risk assessment is only a portion of that needed 

by communities and governments to make informed decisions about risks. There are almost 

always a multitude of factors that affect health in a community (e.g., crime, drugs, health care 

access, vehicle safety, climate, infectious disease, diet) that may not have been considered within 

the scope of a given cumulative risk assessment. Community decision making will typically take 

into account risks to the environment as well as consideration of historical and cultural values 

and questions of fairness and distribution of risks. The methodology is not currently available to 

understand how these factors (or stressors) may affect cumulative health risk. 

67
Additionally, benefits such as jobs and useful products or services that may be associated 

with chemical or other stressor exposures may be important contexts for decisions on the risks 

considered in cumulative risk assessments. 

This framework report is not an attempt to lay out protocols to address all the risks or 

considerations that are needed to adequately inform community decisions. Rather, its focus is on 

describing various aspects of cumulative risk, whether or not the methods or data currently exist 

to adequately analyze or evaluate those aspects of the assessment. It devotes considerable time 

to a discussion on improving the methods for a single part on the broader picture: characterizing 

health risks associated with exposures to multiple chemicals via multiple routes. Because of the 

limitations of the current state of the science, cumulative risk assessments in the near future will 

not be able to adequately answer all the questions posed by stakeholders or interested parties. 

This does not mean, however, that they would not be useful in providing insights to some of the 

questions asked; in fact, cumulative risk assessment may be the best tool available to address 

certain questions dealing with multiple-stressor impacts. 

4.3.2. Cumulative Risk Assessments in a Public Health Context 

The public often asks—in a variety of ways—for clarification of the relationship between 

environmental pollution (and risk assessments concerning it) and public health. Although 

cumulative risk assessment holds the promise of better public health-related information for 

communities, it is not a panacea. To draw relationships between environmental pollutant 

exposures and disease incidence, a body of epidemiological study is necessary. Trying to “work 

backwards” from health statistics to risk factors requires full knowledge of the risk factors 

associated with the relevant disease(s). This is challenging under the best of circumstances, with 

good data; many times it is not possible with the data at hand. 

Health statistics, including death rates and incidence of various diseases, illustrate the 

impact of a variety of risk factors (e.g., smoking as well as environmental pollutants) and risk 

reduction factors (e.g., exercise and good nutrition as well as pollution control measures). 

Indeed, population health statistics are reflective of all risk and risk reduction factors in a 

population’s history to date. Even the best cumulative risk assessment, given today’s state of the 

science, could not include an evaluation of the magnitude and interactions of all stressors and 

their effects. At best, the risk estimates of a cumulative risk assessment will reflect some of the 

risks that may be reflected in community health statistics. With rare exceptions17, cumulative 

risk assessment estimates would not be expected to match exactly with community health 

statistics, even for specific health endpoints such as specific cancers. 

17 It is conceivable that high risks to rare specific effects could be comparable for a risk assessment and 

community health statistics, given current state of the art. To be sure this is not coincidental, a substantial effort to 

match risk assessment scenarios with actual histories or exposures would have to be made. 

68 
4.3.3. How the Scope and Purpose of the Assessment Affect Results 

Historically, the Agency’s risk assessments have focused on assessing the risks from 

environmental pollutants to public health or the environment, usually for the purposes of 

prioritizing risk management activities or triggering regulatory action. Given the need for public 

health-protective decisions, traditional risk assessment tools usually focus on predicting high 

ends of the risk distribution. Also, the traditional tools are not designed to predict risk of 

diseases other than cancer. Additionally, the many environmental pollutants make up only some 

of the categories of risks to public health. Although quite adequate for their original purpose, 

when the results of these types of assessments are viewed from another perspective, such as that 

of a community concerned about the cumulative health impacts of five industrial and commercial 

facilities within a two-block area, they may not be useful. 

The Agency is doing more place-based human health and ecological assessments (i.e., 

compared to source- or media-specific assessments) than in the past, but it will be some time 

before they become commonplace. Consistent with good practices for planning and scoping, 

they often may be driven by specific risk-management needs. The desired objectives and purpose 

of parties who were outside the process may differ from those for which the assessment was 

designed. For this reason, users of cumulative risk assessments are advised to carefully study the 

scope and purpose of the assessment at hand as well as the analysis plan and resulting 

characterization to determine whether it is suitable (or partly suitable) to answer questions 

outside its stated objectives and purpose. 

4.3.4. Documenting Stakeholder Input 

Somewhere in the discussion of how the assessment meets or does not meet the 

objectives laid out in the planning and scoping phase, it is useful to document how stakeholder 

input has influenced the process, noting also those suggestions that were not included and why. 

This documentation supports stakeholder participation and provides assurance that individuals 

have been heard. 

4.4. Using the Results of the Assessment 

Once the results of an assessment are in hand, the assessment participants will usually 

focus primarily on the communication and use of those results. The intended use of the 

assessment was considered at the beginning, in the problem formulation phase, both to plan the 

assessment work and to set the stage for possible actions that might be taken at this point. A 

detailed discussion of the communication and use of the results of a cumulative risk assessment 

is beyond the scope of this document, but it should be noted that in deciding on a course of 

action, considerations other than the results of the assessment will also need to be taken into 

account. 

If the goals of a cumulative risk analysis are to estimate the risk from multichemical and 

multipathway exposure to people living within a geographical area of concern, then an 

69
important objective in presenting the results is to identify the major risk contributors in order to 

understand the sources, pathways, and stressors that contribute most to that overall risk. The 

results of a cumulative risk assessment provide an additional tool for the risk manager, one that 

permits a more complete accounting and more explicit analysis to target follow-up risk 

mitigation strategies toward those stressors that most contribute to the population’s risk. 

If action to mitigate or prevent risk is the goal of the stakeholders, then the options for 

action discussed in the planning of the assessment can be re-evaluated in light of the results of 

the assessment. Some questions that might arise from this re-evaluation include: “Is regulatory 

authority available to address concerns or are voluntary actions better suited to address the 

risks?” or “Can the concerns be addressed by the stakeholders involved in the assessment or are 

the options for mitigation and prevention beyond the scope of their control?” In the latter case, 

for example, siting issues are usually decided locally and may be within the authority of the 

participants of a local assessment. In contrast, risk from mobile sources or acid rain are likely to 

require action that is beyond the scope of a single local community. In that case, taking action 

will require working with other communities and is likely to take more time. Discussion of the 

options available for addressing the results of a risk assessment will help to keep expectations in 

line with possibilities. 

With regard to taking—or not taking—action after a cumulative risk assessment has been 

interpreted, the team may benefit from lessons learned by others, just as in the planning, scoping, 

and problem formulation phase. In early 2002 the European Environment Agency (EEA, 2001) 

released an extensive study of 12 classic case studies in human and environmental health 

protection and the lessons learned from them (see text box on the next page). The report is 

available on the Internet and should be food for thought for any group contemplating protective 

actions, particularly for community assessments. 

Finally, it is important to keep in mind that the results of the risk assessment are only one 

of the factors to be considered in making a decision on action to address the risk. Risk 

information can make an important and valued contribution to the decision-making process, but 

it cannot by itself determine the decision. Factors such as the availability of resources for 

change; perceived fairness; politics; business and employment; quality-of-life issues; the 

religious, cultural, aesthetics, or social values of a community; or concern for future generations 

may also influence decisions. 

In the siting example mentioned above, the assessment may determine that the new 

facility does not significantly increase risk to the community but a decision not to site the facility 

might still be made on the basis of a quality-of-life issue that is unrelated to risk. Or, a 

community may decide that the economic and employment benefits outweigh the risks 

associated with the siting. Other risk factors not considered in the assessment may also enter 

into the decision-making process, including both the environmental risks not covered in the 

cumulative risk assessment and the nonenvironmental risks that may affect a community. With 

limited resources, a community may use all available risk information to most effectively target 

its resources. 

70
The European Environment Agency’s 

12 Lessons Learned Late 

• Acknowledge and respond to ignorance, as well as 

uncertainty and risk, in technology appraisal and 

public policy-making. 

• Provide adequate long-term environmental and 

health monitoring and research into early warnings. 

• Identify and work to reduce blind spots and gaps in 

scientific knowledge. 

• Identify and reduce interdisciplinary obstacles to 

learning. 

• Ensure that real world conditions are adequately 

accounted for in regulatory appraisal. 

• Systematically scrutinize the claimed justifications 

and benefits alongside the potential risks. 

• Evaluate a range of alternative options for meeting 

needs alongside the option under appraisal, and 

promote more robust, diverse and adaptable 

technologies so as to minimize the costs of 

surprises and maximize the benefits of innovation. 

• Ensure use of “lay” and local knowledge as well as 

relevant specialist expertise in the appraisal. 

• Take full account of the assumptions and values of 

different social groups. 

• Maintain regulatory independence from interested 

parties while retaining an inclusive approach to 

information and opinion gathering. 

• Identify and reduce institutional obstacles to 

learning and action. 

• Avoid “paralysis by analysis” by acting to reduce 

potential harm when there are reasonable grounds 

for concern. 

Source: EEA, 2001 

71
5. GLOSSARY 

Adverse effect - A biochemical change, functional impairment, or pathological lesion that either 

singly or in combination adversely affects the performance of the whole organism or reduces an 

organism’s ability to respond to an additional environmental challenge. 

Agent - a chemical, physical, or biological entity that may cause deleterious effects in an 

organism after the organism is exposed to it. 

Aggregate exposure - The combined exposure of an individual (or defined population) to a 

specific agent or stressor via relevant routes, pathways, and sources. 

Aggregate risk - The risk resulting from aggregate exposure to a single agent or stressor. 

Benchmark dose (BMD) - The dose producing a predetermined, altered response for an effect. 

A BMD10, for example, would be calculated on the basis of a benchmark response of 10%. 

Benchmark response (BMR) - A predetermined level of altered response or risk at which the 

benchmark dose is calculated. Typically, the BMRs used are 1%, 5%, or 10%. 

Conceptual model - A written description and/or a visual representation of actual or predicted 

relationships between humans or ecological entities and the chemicals or other stressors to which 

they may be exposed. 

Cumulative risk - The combined risks from aggregate exposures to multiple agents or stressors. 

Cumulative risk assessment - An analysis, characterization, and possible quantification of the 

combined risks to health or the environment from multiple agents or stressors. 

Dose additivity - In a mixture, when each chemical behaves as a concentration or dilution of 

every other chemical. The response of the combination of chemicals is the response expected 

from the equivalent dose of an index chemical (the chemical selected as a basis for 

standardization of toxicity of components in a mixture). The equivalent dose is the sum of 

component doses scaled by their toxic potency relative to the index chemical. For example, for 

chlorinated dibenzodioxins (CDDs), 2,3,7,8-TCDD is selected as the index chemical; other CDD 

concentrations are adjusted for their potency relative to 2,3,7,8-TCDD and then treated as if they 

were 2,3,7,8-TCDD “equivalents.” 

Dose-response relationship - A relationship between (1) the dose, either “administered dose” or 

absorbed dose and (2) the extent of toxic injury produced by that chemical or agent. Response 

can be expressed either as the severity of injury or the proportion of exposed subjects affected. 

Endpoint - An observable or measurable biological or chemical event that is used as an index of 

the effect of a stressor on a cell, tissue, organ, organism, etc. 

72
Exposure pathway - The physical course that a chemical or pollutant takes from the source to 

the organism exposed. 

Exposure route - The way a chemical or pollutant enters an organism after contact, for example, 

by ingestion, inhalation, or dermal absorption. 

Lowest-observed-adverse-effect level (LOAEL) - The lowest dose or exposure level at which 

there is a statistically or biologically significant increase in the frequency or severity of an 

adverse effect in the exposed population as compared with an appropriate, unexposed control 

group. 

Model - A mathematical representation of a natural system that is intended to mimic the 

behavior of the real system, allowing description of empirical data and predictions about 

untested states of the system. Use of models is usually facilitated by computer programming of 

the mathematics and construction of a convenient input and output format. 

No-observed-adverse-effect level (NOAEL) - An exposure level at which there are no 

statistically or biologically significant increases in the frequency or severity of adverse effects 

between the exposed population and its appropriate control; some effects may be produced at 

this level, but they are not considered to be adverse or precursors to adverse effects. In an 

experiment with several NOAELs, the common usage of the term NOAEL is the highest 

exposure without adverse effects. 

Reference concentration (RfC) - An estimate (with uncertainty spanning perhaps an order of 

magnitude) of a continuous inhalation exposure to the human population (including sensitive 

subgroups) that is likely to be without an appreciable risk of deleterious noncancer effects during 

a lifetime. 

Reference dose (RfD) - An estimate (with uncertainty spanning perhaps an order of magnitude) 

of a daily exposure to the human population (including sensitive subgroups) that is likely to be 

without an appreciable risk of deleterious noncancer effects during a lifetime. 

Response additivity - In a mixture, when the toxic response (rate, incidence, risk, or probability 

of effects) from the combination is equal to the conditional sum of component responses as 

defined by the formula for the sum of independent event probabilities. For two chemical 

mixtures, for example, the body’s response to the first chemical is the same whether or not the 

second chemical is present. 

Risk - Absolute risk: The probability of injury, disease, or death under specific circumstances. 

In quantitative terms, risk is expressed in values ranging from zero (representing the certainty 

that there is no chance of harm) to one (representing the certainty that harm will occur). 

Incremental risk: The probability of injury, disease, or death under specific circumstances, 

relative to the background probability. In quantitative terms, risk is expressed in values ranging 

73
from zero (representing the certainty that the probability of harm is no greater than the 

background probability) to one (representing the certainty that harm will occur). 

Stakeholder - An interested or affected party in an ongoing or contemplated project (usually 

involving a group or team planning the project, analyzing one or more problems, and making 

decisions for possible actions on the basis of the interpretation of that analysis). 

Stressor - Any physical, chemical, or biological entity that can induce an adverse response. A 

stressor may also be the lack of an essential entity, such as a habitat. 

74
REFERENCES 

ACGIH (American Conference of Government Industrial Hygienists), 1998. Threshold Limit 

Values for Chemical Substances and Physical Agents. Cincinnati, OH. 

ACPA (American Crop Protection Association), 1999. “Cumulative & Aggregate Risk 

Evaluation System: CARES. Conceptual Model.” Washington, DC. Internet: 

www.alphacares.org/. 

AIChE (American Institute of Chemical Engineers), 1992. Guidelines for Hazard Evaluation 

Procedures, 2nd Edition with Worked Examples. Center for Chemical Process Safety, New York, 

NY. 

AIChE, 1996. Guidelines for Use of Vapor Cloud Dispersion Models, 2nd Ed. Center for 

Chemical Process Safety, New York, NY. 

AIHA (American Industrial Hygiene Association), 2000. Emergency Response Planning 

Guidelines Series. Emergency Response Planning Guidelines Committee, Fairfax, VA. 

Albert, Roy E., 1999. Unifying the standard setting process for carcinogens and noncarcinogens. Applied Occupational and Environmental Hygiene 14:742–747. 

Albert, Roy E., Joellen Lewtas, Stephen C. Nesnow, Todd W. Thorslund, and Elizabeth L. 

Anderson, 1983. Comparative potency method for cancer risk assessment application to diesel 

particulate emissions. Risk Analysis 3:101–117. 

American Heart Association, 2000. Stroke Risk Factors. Internet: www.americanheart.org. 

Ashford, N.A., D. Hattis, E.M. Zolt, J.I. Katz, G.R. Heaton, and W.C. Priest, 1981. “Evaluating 

Chemical Regulations: Trade-Off Analysis and Impact Assessment for Environmental DecisionMaking. Massachusetts Institute of Technology, Center for Policy Alternatives. Cambridge, 

Massachusetts. CPA-80-13 (NTIS #PB81-195067). 

ATSDR (Agency for Toxic Substances and Disease Registry), 1995. Report of the Expert Panel 

Workshop on the Psychological Responses to Hazardous Substances. U.S. Department of Health 

and Human Services. Atlanta, Georgia. Internet: www.atsdr.cdc.gov/HEC/PRHS/. 

Barnes, Donald G., Ann Alford-Stevens, Linda Birnbaum, Frederick W. Kutz, William Wood, 

and Dorothy Patton, 1991. Toxic equivalency factors for PCBs? Quality Assurance 1:70–81. 

Barnes, Donald G., and Michael L. Dourson, 1988. Reference dose (RfD): description and use 

in health risk assessments. Regulatory Toxicology and Pharmacology 8:471–486. 

75
Barnthouse, Lawrence W., David R. Marmorek, and Calvin N. Peters, 2000. Assessment of 

multiple stresses at regional scales. In: Multiple Stressors in Ecological Risk and Impact 

Assessment: Approaches to Risk Estimation. (Ferenc and Foran, Eds.) Society of Environmental 

Toxicology and Chemistry, SETAC Press, Pensacola, Florida. ISBN 1-880611-40-6. 

Beck, Barbara D., Rory B. Connolly, Michael L. Dourson, Daniel Guth, Dale Hattis, Carole 

Kimmel, and Stephen C. Lewis, 1993. Improvements in quantitative noncancer risk assessment. 

Fundamental and Applied Toxicology, 20:1–14. 

Berglund, Marika, Carl-Gustaf Elinder, and Lars Järup, 2001. Human Exposure Assessment: An 

Introduction. World Health Organization, Geneva. WHO/SDE/OEH/01.3. 

Bogen, K.T., 2001. “Methods for Addressing Uncertainty and Variability to Characterize 

Potential Health Risk from Trichloroethylene-Contaminated Ground Water at Beale Air Force 

Base in California: Integration of Uncertainty and Variability in Pharmacokinetics and DoseResponse.” Lawrence Livermore National Laboratory, U.S. Department of Energy, Livermore, 

CA. UCRL-ID-135978 Rev 1. 

Bonnell, Steve, and Keith Storey, 2000. Addressing cumulative effects through strategic 

environmental assessment: A case study of small hydro development in Newfoundland, Canada. 

Journal of Environmental Assessment Policy and Management 2: 477–499. 

Bouwes, Nicolaas W. and Steven M. Hassur, 1998. “OPPT’s Risk-Screening Environmental 

Indicators: Toxic Weights for Toxic Release Inventory (TRI) Chemicals and Chemical 

Categories.” Office of Pollution Prevention and Toxic Substances, Office of Prevention, 

Pesticides, and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. 

April 28, 1998. 

Bullard, Robert D., 1990. Dumping in Dixie: Race, Class, and Environmental Quality. 

Westview Press, Boulder, CO. ISBN 0-8133-7954-7. 

Carpy, Serge A., Werner Kobel, and John Doe, 2000. Health risk of low-dose pesticides 

mixtures: A review of the 1985–1998 literature on combination toxicology and health risk 

assessment. Journal of Toxicology and Environmental Health Part B, 3:1–25. 

CEU (Council of the European Union), 1996. Council directive concerning integrated pollution 

prevention and control. Brussels. 96/61/EC dated 24 September 1996, EU official Journal L 

257, 10/10/1996; 0026–0040. 

CEQ (Council on Environmental Quality), 1997. “Considering Cumulative Effects Under the 

National Environmental Policy Act.” Executive Office of the President, Washington, DC. 

76
Chambers, J.M., W.S. Cleveland, B. Kleiner, and P.A. Tukey. 1983. Graphical methods for data 

analysis. Wadsworth International Group, Belmont, CA, 145–171. 

Chess, C., and K. Purcell, 1999. Public Participation and the Environment: Do We Know What 

Works? Environmental Science & Technology 33(16):2685–2692. 

Clemen, Robert T., 1996. Making Hard Decisions: An Introduction to Decision Analysis. 2nd Ed. 

Duxbury Press, Wadsworth Publishing Co., Belmont, CA. ISBN 0-534-26034-9. p. 5. 

Cogliano, Vincent James, 1997. Plausible upper bounds: Are their sums plausible? Risk Analysis 

17:77–84. 

Cohen, Bernard L., 1991. Catalog of risks extended and updated. Health Physics 61:317–335. 

DOT (U.S. Department of Transportation), 1998. “High-Speed Ground Transportation Noise and 

Vibration Impact Assessment.” Final Draft. Office of Railroad Development, Federal Railroad 

Administration. Washington, DC. Report No. 293630-1 Internet: 

http://project1.parsons.com/ptgnechsr/noise_manual.htm. 

Dourson, M.L., S.P. Felter, and D. Robinson. 1996. Evolution of science-based uncertainty 

factors in noncancer risk assessment. Regulatory Toxicology and Pharmacology 24:108–120. 

Dourson, Michael L., Linda K. Teuschler, Patrick R. Durkin, and William M. Stiteler, 1997. 

Categorical regression of toxicity data: A case study using aldicarb. Regulatory Toxicology and 

Pharmacology 25:121–129. 

Eadon, George A., Laurence S. Kaminsky, Jay Silkworth, K. Aldous, D. Hilker, P. O'Keefe, R. 

Smith, J. Gierthy, J. Hawley, N. Kim, et al., 1986. Calculation of 2,3,7,8-TCDD equivalent 

concentrations of complex environmental contaminant mixtures. Environmental Health 

Perspectives 70:221–227. 

EEA (European Environment Agency), 2001. “Late Lessons from Early Warnings: The 

Precautionary Principle 1896-2000.” Copenhagen. Environmental Issue Report No. 22 (Released 

January 10, 2002) Internet: http://reports.eea.eu.int/environmental_issue_report_2001_22/en. 

Environmental Defense, 2001. “Scorecard.” Internet: http://www.scorecard.org. 

European Commission, 2000. “The European Multi-Hazard Risk Assessment Project 

(T.E.M.R.A.P.)” Directorate General XII for Science Research and Development, Environment 

and Climate - 1994/1998 - Climatology and Natural Hazard, European Commission. Brussels, 

Belgium. Internet: http://phypc9.geo.ulg.ac.be/nouveau/temrap/MainWebPage.htm. 

77
Ferenc, Susan A., and Jeffrey A. Foran (Eds.), 2000. Multiple Stressors in Ecological Risk and 

Impact Assessment: Approaches to Risk Estimation. Society of Environmental Toxicology and 

Chemistry, SETAC Press, Pensacola, FL. ISBN 1-880611-40-6. 

Fischhoff, Baruch, S. Watson, and C. Hope, 1984. Defining risk. Policy Sciences 17:123–139. 

Foran, Jeffrey A., and Susan A. Ferenc (Eds.), 1999. Multiple Stressors in Ecological Risk and 

Impact Assessment. Society of Environmental Toxicology and Chemistry, SETAC Press, 

Pensacola, Florida. ISBN 1-880611-32-5. 

Freeman, A. Myrick, III, 1999. Economics, Incentives, and Environmental Regulation. In: 

Environmental Policy in the 1990s: Reform or Reaction? Fourth Ed., Chapter 9. Norman J. Vig 

and Michael E. Kraft [Eds.]. Congressional Quarterly Press, Washington, DC. ISBN 1-56802-

341-3. 

Frewer, L., 1999. Risk Perception, Social Trust, and Public Participation in Strategic Decision 

Making: Implications for Emerging Technologies. Ambio 28(6):569–574. 

GAO (U.S. General Accounting Office), 1983. “Siting of Hazardous Waste Landfills and Their 

Correlation with Racial and Economic Status of Surrounding Communities.” Washington, DC. 

GAO/RCED 83–168. 

Gentile, John H., Keith R. Solomon, Jonathan B. Butcher, Michael C. Harrass, Wayne G. Landis, 

Michael Power, Barnett A. Rattner, William J. Warren-Hicks, and Robert Wegner, 1999. 

Linking stressors and ecological responses. In: Multiple Stressors in Ecological Risk and Impact 

Assessment. Jeffrey Foran and Susan Ferenc, Eds. Society of Environmental Toxicology and 

Chemistry, SETAC Press, Pensacola, FL. ISBN 1-880611-32-5. 

Guth, Daniel J., Richard C. Hertzberg , and Annie M. Jarabek, 1993. Exposure-response 

analysis: modeling severity against concentration and duration. In: “Improvements in 

Quantitative Noncancer Risk Assessment: Symposium Overview,” Society of Toxicology 

meeting; February 1992; Seattle, WA. 

Haber, F., 1924. Zur Geschichte des Gaskrieges [On the History of Gas Warfare], in: Funf 

Vortrage aus den Jahren 1920-1923 (Five Lectures from the Years 1920–1923), Springer, Berlin, 

76–92. 

Haddad, S., G. Charest-Tardif, and K. Krishnan, 2000. Physiologically based modeling of the 

maximal effect of metabolic interactions on the kinetics of components of complex chemical 

mixtures. Journal of Toxicology and Environmental Health, Part A 61:209–223. 

Haddad, S., M. Beliveau, R. Tardif, and K. Krishnan, 2001. A PBPK modeling-based approach 

to account for interactions in the health risk assessment of chemical mixtures. Toxicological 

Sciences 63:125–131. 

78
Hampshire Research Institute, 1999. “Review of an Aggregate Exposure Assessment Tool.” 

Presentation to the U.S. EPA FIFRA Scientific Advisory Panel, September, 1999. Hampshire 

Research Institute, Alexandria, VA. Internet: 

www.epa.gov/scipoly/sap/1999/september/aggbkgd.pdf. 

Hampshire Research Institute, 2000. “Overview of the Fundamentals of Version 1.0 of LifeLine 

Software for Modeling Aggregate and Cumulative Exposures to Pesticides.” Hampshire 

Research Institute, Alexandria, VA. Internet: www.epa.gov/scipoly/sap/2000/september 

/final_fundamentals.pdf. 

Hansen, Hugh, Christopher T. De Rosa, Hana Pohl, Michael Fay, and Moiz M. Mumtaz, 1998. 

Public health challenges posed by chemical mixtures. Environ Health Perspect 106:1271–1280. 

Hattis, Dale and Rob L. Gobel, 1994. Current Priority-Setting Methodology: Too Little 

Rationality or Too Much? In: Adam M. Finkel and Dominic Golding [Eds.]. Worst Things First? 

The Debate over Risk-Based National Environmental Priorities. Resources for the Future. 

Washington, DC. ISBN 0-915707-76-4. 

Hertzberg, Richard C., 1989. Fitting a model to categorical response data with application to 

species extrapolation. Health Physics, 57(Supplement 1):405–409. 

Hertzberg, Richard C., 2000. “Communicating cumulative risk: sound bites from chaos.” 

Luncheon address, presented at the conference, Toxicology and Risk Assessment Approaches for 

the 21st Century, April 10–13, 2000, Kings Island, OH. 

Hertzberg, Richard C., and M.M. MacDonell, 2002. Synergy and other ineffective mixture risk 

definitions. The Science of the Total Environment 288:31–42. 

HUD (U.S. Department of Housing and Urban Development), 1991. “The Noise Guidebook.” 

Office of Community Planning and Development, Washington, DC. HUD-953-CPD(1). 

ILSI (International Life Sciences Institute), 1998. Aggregate Exposure Assessment. Washington, 

DC. ISBN 1-57881-040-X. 

ILSI, 1999. A Framework for Cumulative Risk Assessment. Washington, DC. ISBN 1-57881-

055-8. 

ILSI, 2000. Revised Framework for Microbial Risk Assessment. Washington, DC. ISBN 1-

57881-081-7. 

ILSI, 2001. Aggregate Exposure Assessment: Model Evaluation and Refinement Workshop 

Report. Washington, DC. ISBN 1-57881-120-1. 

79
IPCS (International Programme on Chemical Safety), 1983. Environmental Health Criteria 27. 

Guidelines on Studies in Environmental Epidemiology. International Programme on Chemical 

Safety. World Health Organization. Geneva. ISBN 92 4 154087 7. 

IPCS, 1993. Environmental Health Criteria 155. Biomarkers in Risk Assessment: Concepts and 

Principles. World Health Organization, Geneva. ISBN 92 4 157155 1. 

IPCS, 2001. Environmental Health Criteria 222. Biomarkers in Risk Assessment: Validity and 

Validation. World Health Organization, Geneva. ISBN 92 4 157222 1. 

Johnson, Ted, Gary Mihlan, Jacky LaPointe, Kris Fletcher, Jim Capel, Arlene Rosenbaum, 

Jonathan Cohen, and Pat Stiefer, 2000. “Estimation of Carbon Monoxide Exposures and 

Associated Carboxyhemoglobin Levels for Residents of Denver and Los Angeles using 

pNEM/CO (version 2.1).” Draft report prepared for Office of Air Quality Planning and 

Standards, Office of Air and Radiation, U.S. Environmental Protection Agency. Research 

Triangle Park, NC. Contract No. 68-D6-0064. 

Kasperson, Jeanne X., Roger E. Kasperson, and Billie L. Turner, II (Eds.), 1995. Regions at 

Risk: Comparisons of Threatened Environments. United Nations University Press, Tokyo. 

Kasperson, Roger E., 2000. Personal communication to Michael Callahan, Chair of the 

Cumulative Risk Technical Panel, as part of an EPA Science Advisory Board Consultation on 

the draft Framework for Cumulative Risk Assessment. 

Kodell, Ralph L., and James J. Chen, 1994. Reducing conservatism in risk estimation for 

mixtures of carcinogens. Risk Analysis 14:327–332. 

Kroschwitz, Jacqueline I., and Mary Howe-Grant (Eds.), 1994. Kirk-Othmer Encyclopedia of 

Chemical Technology. 4th Ed. John Wiley and Sons, New York. ISBN 0-47152-677-0. 

Laden, F., L.M. Neas, D.W. Dockery, and J. Schwartz, 2000. Association of fine particulate 

matter from different sources with daily mortality in six U.S. Cities. Environmental Health 

Perspectives 108:941–947. 

Lantz, Paula M, James S. House, James M. Lepkowski, David R. Williams, Richard P. Mero, 

and Jieming Chen, 1998. Socioeconomic factors, health behaviors, and mortality: Results from a 

nationally representative prospective study of US adults. Journal of the American Medical 

Association 279:1703–1709. 

Lewis, S.C., 1993. Reducing uncertainty with adjustment factors: Improvements in quantitative 

noncancer risk assessment. Fundamentals of Applied Toxicology 20:2–4. 

80
Lewtas, Joellen, 1985. Development of a comparative potency method for cancer risk 

assessment of complex mixtures using short-term in vivo and in vitro bioassays. Toxicol. Ind. 

Health 1:193–203. 

Lewtas, Joellen, 1988. Genotoxicity of complex mixtures: Strategies for the identification and 

comparative assessment of airborne mutagens and carcinogens from combustion sources. 

Fundamentals of Applied Toxicology 10:571–589. 

Lynch, John W., George A. Kaplan, Elsie R. Pamuk, Richard D. Cohen, Katherine E. Heck, 

Jennifer L. Balfour, and Irene H. Yen, 1998. Income inequality and mortality in metropolitan 

areas of the United States. American Journal of Public Health 88:1074–1080. 

Morata, Thais C., 2000. “Suggested Guidelines to Studying the Combined Effects of 

Occupational Exposure to Noise and Chemicals on Hearing.” National Institute for Occupational 

Safety and Health. Cincinnati, OH. Dated February, 2000. Internet: 

http://www.ucl.ac.uk/noiseandhealth/WP3%20draft%20protocol%20-%20Morata.doc. 

Morata, T.C., A.C. Fiorini, F.M. Fischer, S. Colacioppo, K.M. Wallingford, E.F. Kreig, D.E. 

Dunn, L. Gozzoli, M.A. Padrão, and C.L.G. Cesar, 1997. Toluene-induced hearing loss among 

rotogravure printing workers. Scandanavian Journal of Work, Environment & Health 

23:289–298. 

Morgan, M. Granger, and Max Henrion, 1990. Uncertainty: A Guide to Dealing with 

Uncertainty in Quantitative Risk and Policy Analysis. Cambridge University Press, New York, 

NY. ISBN 0-521-36542-2. 

Murray, C., 1994. Quantifying the burden of disease: the technical basis for disability-adjusted 

life years. Bulletin of the World Health Organization 72(3):429–445. 

NAPA (National Academy of Public Administration), 1995. Setting Priorities, Getting Results: 

A New Direction for EPA. Washington, DC. LCCN 95-68048. 

NIOSH (National Institute for Occupational Safety and Health), 1997. “NIOSH Pocket Guide to 

Chemical Hazards.” Centers for Disease Control and Prevention. Washington, DC. NTIS PB 97-

177-604. 

NOAA (National Oceanic and Atmospheric Administration), 1999. “An Assessment of Lower 

Snake River Hydrosystem Alternatives on Survival and Recovery of Snake River Salmonids.” 

National Marine Fisheries Service. Seattle, WA. Dated April 14, 1999. 

NRC (National Research Council), 1983. Risk Assessment in the Federal Government: 

Managing the Process. Committee on the Institutional Means for Assessments of Risk to Public 

Health, Commission on Life Sciences. National Academy Press, Washington, DC. ISBN 0-309-

03349-7. 

81
NRC, 1993. Pesticides in the Diets of Infants and Children. National Academy Press, 

Washington, DC. 

NRC, 1994. Science and Judgment in Risk Assessment. Committee on Risk Assessment of 

Hazardous Air Pollutants, Board on Environmental Sciences and Technology, Commission on 

Life Sciences. National Academy Press, Washington, DC. ISBN 0-309-04894-X. 

NRC, 1996. Understanding Risk: Informing Decisions in a Democratic Society. Committee on 

Risk Characterization, Commission on Behavioral and Social Sciences and Education. National 

Academy Press, Washington, DC. ISBN 0-309-05396-X. 

OMB (U.S. Office of Management and Budget), 2000. “Guidelines to Standardize Measures of 

Costs and Benefits and the Format of Accounting Statements.” Memo dated March 22, 2000, 

from Jacob J. Lew, Director, to Heads of Departments and Agencies. Washington, DC. 

OMB, 2002. “Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by Federal Agencies.” Washington, DC. Federal Register 

67:369–378. 

Oris, J.T., and J.P. Geisy, 1985. The photo-enhanced toxicity of anthacene to juvenile sunfish 

(Lepomis spp.). Aquatic Toxicology 6:133–146. 

Osowski, Sharon L., Joseph D. Swick, Jr., Gerald R. Carney, Hector B. Pena, Jeffrey E. 

Danielson, and David A. Parrish, 2001. A watershed-based cumulative risk impact analysis: 

Environmental vulnerability and impact criteria. Environmental Monitoring and Assessment 

66:159–185. 

Perry, Robert H., Don W. Green, and James O. Maloney (Eds.), 1997. Perry’s Chemical 

Engineers’ Handbook. 7th Ed. McGraw Hill Professional Publishing, New York. ISBN 0-07049-

841-5. 

PCCRARM (Presidential/Congressional Commission on Risk Assessment and Risk 

Management), 1997. Risk Assessment and Risk Management in Regulatory Decision-Making. 

Washington, DC. 

Seed, Jennifer, Ronald P. Brown, Stephen S. Olin, and Jeffery A. Foran, 1995. Chemical 

mixtures: Current risk assessment methodologies and future directions. Regulatory Toxicology 

and Pharmacology 22:76–94. 

Sexton, Ken, David E. Kleffman, and Michael A. Callahan, 1995. An introduction to the 

National Human Exposure Assessment Survey (NHEXAS) and related phase I field studies. J. 

Expos. Anal. Environ. Epidem. 5:229–232. Related papers are in the same issue of Journal of 

Exposure Analysis and Environmental Epidemiology 5:233–444. 

82
Slob, W., and M.N. Pieters. 1998. A probabilistic approach for deriving acceptable human intake 

limits and human health risks from toxicological studies: General framework. Risk Anal. 

18:787–798. 

State of Vermont, 1991. “Environment 1991: Risks to Vermont and Vermonters.” Agency of 

Natural Resources. Waterbury, VT. 

Tanabe, Shinsuke, 1998. Butyltin contamination in marine mammals: A review. Marine 

Pollution Bulletin 39:62–72. 

Teuschler, Linda K., Michael L. Dourson, William M. Stiteler, Peter McClure, and Heather 

Tully, 1999. Health risks above the reference dose for multiple chemicals. Regulatory 

Toxicology and Pharmacology 30:S19–S26. 

Thomas, J., 1995. Public Participation in Public Decisions: New Skills and Strategies for Public 

Managers. Jossey-Bass, San Francisco, CA. 

United Church of Christ, 1987. Toxic Waste and Race in the United States: A National Report on 

the Racial and Socio-Economic Characteristics of Communities with Hazardous Waste Sites. 

United Church of Christ Commission for Racial Justice. New York, NY. 

USEPA (Environmental Protection Agency), 1986a. “The Risk Assessment Guidelines of 1986.” 

Risk Assessment Forum, Office of Research and Development. Washington, DC. EPA/600/8-

87/045. 

USEPA, 1986b. “Guidelines for the Health Risk Assessment of Chemical Mixtures.” Risk 

Assessment Forum, Office of Research and Development. Washington, DC. EPA/630/R-98/002. 

USEPA, 1986c. “Guidelines for Mutagenicity Risk Assessment.” Risk Assessment Forum, 

Office of Research and Development. Washington, DC. EPA/630/R-98/003. 

USEPA, 1986d. “Guidelines for Carcinogen Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development. Washington, DC. EPA/630/R-98/001. 

USEPA, 1987. “The Total Exposure Assessment Methodology (TEAM) Study.” Office of Acid 

Deposition, Environmental Monitoring and Quality Assurance, Office of Research and 

Development. Washington, DC. EPA/600/6-87/002. 

USEPA, 1988. “Federal Guidance Report No. 11: Limiting Values of Radionuclide Intake and 

Air Concentration and Dose Conversion Factors for Inhalation, Submersion, and Ingestion.” 

Office of Radiation Programs. Washington, DC. EPA-520/1-88-020. 

83
USEPA, 1989a. “Risk Assessment Guidance for Superfund, Volume 1: Human Health 

Evaluation Manual. Office of Emergency and Remedial Response, Office of Solid Waste and 

Emergency Response. Washington, DC. EPA/540/1-89/002. 

USEPA, 1989b. “Interim Procedures for Estimating Risks Associated with Exposure to 

Chlorinated dibenzo-p-dioxins and -dibenzofurans (CDDs and CDFs) and 1989 update. Risk 

Assessment Forum, Office of Research and Development. Washington, DC. EPA 625/3-89/016. 

USEPA, 1990a. “Technical Support Document on Health Assessment of Chemical Mixtures.” 

Office of Research and Development. Washington, DC. EPA/600/8-90/064. 

USEPA, 1990b. “Reducing Risk: Setting Priorities and Strategies for Environmental Protection.” 

Science Advisory Board. Washington, DC. SAB-EC-90-021. 

USEPA, 1991a. “Guidelines for Developmental Toxicity Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development. Washington, DC. EPA/600/FR-91/001. 

USEPA, 1991b. “Locational Data Policy.” Office of Policy, Planning, and Evaluation. 

Washington, DC. IRM Policy Manual 2100-CGH2, dated April 8, 1991. 

USEPA, 1992a. “Safeguarding the Future: Credible Science, Credible Decisions. A Report of the 

Expert Panel on the Role of Science at EPA.” Washington, DC. EPA/600/9-91/050 

USEPA, 1992b. “Framework for Ecological Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development, Science Advisory Board. Washington, DC. EPA/630/R-92/001. 

USEPA, 1992c. “Guidelines for Exposure Assessment.” Risk Assessment Forum, Office of 

Research and Development. Washington, DC. EPA/600/Z-92/001. 

USEPA, 1993a. “Guidance on the Application of Refined Dispersion Models for 

Hazardous/Toxic Air Release.” Office of Air Quality Planning and Standards, Office of Air and 

Radiation. Research Triangle Park, NC. EPA 454/R-93-002. 

USEPA, 1993b. “A Guidebook to Comparing Risks and Setting Environmental Priorities.” 

Office of Policy, Planning and Evaluation. Washington, DC. EPA 230-B-93-003. 

USEPA, 1993c. “Chemical Indexing System for the Toxic Chemical Release Inventory, Part I: 

Chronic Index.” Air, Radiation and Toxics Division, EPA Region III. Philadelphia, PA. 

EPA/903/R-93/002. 

USEPA, 1993d. “Federal Guidance Report No. 12: External Exposure to Radionuclides in Air, 

Water, and Soil.” Office of Air and Radiation. Washington, DC. EPA-402-R-93-081. 

84
USEPA, 1994. “Methods for Derivation of Inhalation Reference Concentrations and Application 

of Inhalation Dosimetry.” Office of Health and Environmental Assessment, Office of Research 

and Development. Research Triangle Park, NC. EPA/600/8-90/066F. 

USEPA, 1995a. “Policy for Risk Characterization.” Memorandum from U.S. Environmental 

Protection Agency Administrator Carol M. Browner, dated March 21, 1995. Washington, DC. 

USEPA, 1995b. “Guidance for Risk Characterization.” Policy paper dated February, 1995. 

Science Policy Council. Washington, DC. 

USEPA, 1995c. “The Use of the Benchmark Dose Approach in Health Risk Assessment.” Risk 

Assessment Forum, Office of Research and Development. Washington, DC. EPA/630/R-94/007. 

USEPA, 1995d. “Chemical Indexing System for the Toxic Chemical Release Inventory, Part I: 

Chronic Index; Addendum.” Air, Radiation and Toxics Division, EPA Region III. Philadelphia, 

PA. EPA/903/R-93/002a (August, 1995). 

USEPA, 1995e. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” 5th Ed. Office of Air Quality Planning and Standards, Office of Air and 

Radiation. Research Triangle Park, NC. EPA AP-42. 

USEPA, 1996a. “RAGS Reform Stakeholder Forums: Synopsis of Participants’ Comments. San 

Francisco, California, October 30-November, 1, 1996, and Washington, DC, November 6-

November 8, 1996.” Office of Emergency and Remedial Response, Washington, DC. 

USEPA, 1996b. “Guidelines for Reproductive Toxicity Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development. Washington, DC. EPA/630/R-96/009. 

USEPA, 1996c. “Proposed Guidelines for Carcinogen Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development. Washington, DC. EPA/600/P-92/003C. 

USEPA, 1996d. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” 5th Ed. Supplements A & B. Office of Air Quality Planning and Standards, 

Office of Air and Radiation. Research Triangle Park, NC. EPA AP-42. 

USEPA, 1996e. “Radiation Exposure and Risk Assessment Manual (RERAM).” Office of Air 

and Radiation. Washington, DC. EPA 402-R-96-016. 

85
USEPA, 1997a. “Guidance on Cumulative Risk Assessment, Part 1. Planning and Scoping.” 

Science Policy Council. Washington, DC. Attachment to memo dated July 3, 1997 from the 

Administrator, Carol Browner, and Deputy Administrator, Fred Hansen, titled “Cumulative Risk 

Assessment Guidance-Phase I Planning and Scoping.” 

USEPA, 1997b. “Guiding Principles for Monte Carlo Analysis.” Risk Assessment Forum, 

Office of Research and Development, Washington, DC. EPA/630/R-97/001. 

USEPA, 1997c. “Chemical Indexing System, Part II: Vulnerability Index.” Waste and 

Chemicals Management Division, EPA Region III. Philadelphia, PA. EPA/903/R-97/021. 

USEPA, 1997d. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” Fifth Edition, Supplement C. Office of Air Quality Planning and Standards, 

Office of Air and Radiation. Research Triangle Park, NC. EPA AP-42. 

USEPA, 1998a. “Risk Assessment Guidance for Superfund (RAGS) Stakeholder Forum: 

Synopsis of Participants’ Comments. Atlanta, Georgia, March 2-4, 1998.” Office of Emergency 

and Remedial Response. Washington, DC. 

USEPA, 1998b. “Guidelines for Ecological Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development, Washington, DC. EPA/630/R-95/002F. 

USEPA, 1998c. “Guidelines for Neurotoxicity Risk Assessment.” Risk Assessment Forum, 

Office of Research and Development. Washington, DC. EPA/630/R-95/001F. 

USEPA, 1998d. “Methods for Exposure-Response Analysis for Acute Inhalation Exposure to 

Chemicals: Development of the Acute Reference Exposure.” Review Draft. Office of Research 

and Development. Washington, DC. EPA/600/R-98/051. 

USEPA, 1998e. “General Guidance for Risk Management Programs (40 CFR Part 68).” 

Chemical Emergency Preparedness and Prevention Office, Office of Solid Waste and Emergency 

Response. Washington, DC. EPA 550-B-98-003. 

USEPA, 1998f. “Comparative Risk Framework: Methodology and Case Study.” SAB Review 

Draft dated November 9, 1998. National Center for Environmental Assessment, Office of 

Research and Development. Cincinnati, OH. 

USEPA, 1998g. “Report of the Common Sense Initiative Council’s Stakeholder Involvement 

Work Group.” Common Sense Initiative Council. Washington, DC. 

USEPA, 1998h. “An SAB Report: Review of Disproportionate Impact Methodologies.” Science 

Advisory Board. Washington, DC. EPA-SAB-IHEC-99-007. 

86
USEPA, 1998i. “Handbook for Air Toxics Emission Inventory Development, Volume I: 

Stationary Sources.” Office of Air Quality Planning and Standards, Office of Air and Radiation. 

Research Triangle Park, NC. EPA-454/B-98-002. 

USEPA, 1998j. “Human Health Risk Assessment Protocol for Hazardous Waste Combustion 

Facilities.” Peer Review Draft. Office of Solid Waste and Emergency Response. Washington, 

DC. EPA530-D-98-001 Internet: www.epa.gov/epaoswer/hazwaste/combust/riskvol.htm. 

USEPA, 1998k. “A Common Mechanism of Action: The Organophosphate Pesticides.” 

Scientific Advisory Panel. Washington, DC. Internet: 

www.epa.gov/scipoly/sap/1998/march/comec.htm. 

USEPA, 1999a. “Guidelines for Carcinogen Risk Assessment.” Review draft dated July, 1999. 

Risk Assessment Forum, Office of Research and Development. Washington, DC. NCEA-F-0644. 

USEPA, 1999b. “Review of Revised Sections of the Proposed Guidelines for Carcinogen Risk 

Assessment.” Science Advisory Board. Washington, DC. EPA-SAB-EC-99-015. 

USEPA, 1999c. “Risk Assessment Guidance for Superfund: Volume 1 - Human Health 

Evaluation Manual. Supplement to Part A: Community Involvement in Superfund Risk 

Assessments.” Office of Solid Waste and Emergency Response. Washington, DC. EPA 540-R98-042/PB99-963303. 

USEPA, 1999d. “Risk Management Program Guidance for Offsite Consequence Analysis.” 

Chemical Emergency Preparedness and Prevention Office, Office of Solid Waste and Emergency 

Response. Washington, DC. EPA 550-B-99-009. 

USEPA, 1999e. “Guideline on Air Quality Models.” Office of Air Quality Planning and 

Standards, Office of Air and Radiation. Research Triangle Park, NC. 40CFR Ch. I (7-1-99 

Edition), Appendix W to Part 51, pages 390–481. 

USEPA, 1999f. “EPA’s Framework for Community-Based Environmental Protection.” Office of 

Policy/Office of Reinvention. Washington, DC. EPA 237-K-00-001. 

USEPA, 1999g. “Guidance for Performing Aggregate Exposure and Risk Assessments.” Office 

of Pollution Prevention and Toxic Substances, Office of Pesticide Programs. Washington, DC. 

Item 6043, dated October 29, 1999. Available on the internet at: 

http://www.epa.gov/fedrgstr/EPA-PEST/1999/November/Day-10/6043.pdf. 

USEPA, 1999h. “Guidance for Identifying Pesticide Chemicals and Other Substances that Have 

a Common Mechanism of Toxicity.” Office of Pesticide Programs, Office of Prevention, 

Pesticides, and Toxic Substances. Washington, D.C. Internet: 

http://www.epa.gov/fedrgstr/EPA_PEST/1999/February/Day_05/. 

87
USEPA, 1999i. “Risk-Screening Environmental Indicators: 1988-1997 TRI Data ‘Air-Only’ 

Model.” Office of Pollution Prevention and Toxic Substances, Office of Prevention, Pesticides, 

and Toxic Substances. Washington, DC. CD-ROM Version 1.0, dated July 6, 1999. (Version 2.0 

is now in beta-testing.) 

USEPA, 1999j. “An SAB Report on the National Center for Environmental Assessment’s 

Comparative Risk Framework Methodology.” Science Advisory Board. Washington, DC. EPASAB-DWC-99-016. 

USEPA, 1999k. “Handbook for Criteria Pollutant Inventory Development: A Beginner’s Guide 

for Point and Area Sources.” Office of Air Quality Planning and Standards, Office of Air and 

Radiation. Research Triangle Park, NC. EPA-454/R-99-037. 

USEPA, 1999l. “Federal Guidance Report No. 13: Cancer Risk Coefficients for Environmental 

Exposure to Radionuclides.” Office of Air and Radiation. Washington, DC. EPA-402-R-99-001. 

USEPA, 2000a. “Toward Integrated Environmental Decision-Making.” Science Advisory 

Board. Washington, DC. EPA-SAB-EC-00-011. 

USEPA, 2000b. “Benchmark Dose Technical Guidance Document” Draft report. Risk 

Assessment Forum, Office of Research and Development. Washington, DC. EPA/630/R-00/001. 

USEPA, 2000c. “Science Policy Council Handbook: Risk Characterization.” Science Policy 

Council. Washington, DC. EPA 100-B-00-002. 

USEPA, 2000d. “Science Policy Council Handbook: Peer Review.” 2nd Edition. Science Policy 

Council. Washington, DC. EPA 100-B-00-001. 

USEPA, 2000e. “Supplementary Guidance for Conducting Health Risk Assessment of Chemical 

Mixtures.” Risk Assessment Forum, Office of Research and Development. Washington, DC. 

EPA/630/R-00/002. 

USEPA, 2000f. “Baltimore Community Environmental Partnership Air Committee Technical 

Report. Community Risk-Based Air Screening: A Case Study in Baltimore, MD.” Office of 

Pollution Prevention and Toxics, Office of Prevention, Pesticides, and Toxic Substances. 

Washington, DC. EPA 744-R-00-005. 

USEPA, 2000g. “Handbook for Non-Cancer Health Effects Valuation.” Non-Cancer Health 

Effects Valuation Subcommittee of the EPA Social Science Discussion Group, Science Policy 

Council. Washington, DC. Dated November, 2000. 

USEPA, 2000h. “AP-42: Compilation of Air Pollutant Emission Factors, Volume II: Mobile 

Sources.” Office of Transportation and Air Quality, Office of Air and Radiation. Washington, 

DC. EPA AP-42, Volume II Internet: www.epa.gov/otaq/ap42.htm. 

88
USEPA, 2000i. “Catalog of Hazardous and Solid Waste Publications.” 13th Edition. Office of 

Solid Waste and Emergency Response. Washington, DC. EPA530-B-00-001 Internet: 

www.epa.gov/epaoswer/osw/catalog.htm. 

USEPA, 2000j. “Guide to Field Storage of Biosolids, Appendix A: Odor Characterization, 

Assessment and Sampling.” Office of Wastewater Management, Office of Water. Washington, 

DC. EPA/832-B-00-007. Internet: www.epa.gov/owm/bio/fsguide/. 

USEPA, 2001a. Personal communication. Debby Sisco,. Biological and Economic Analysis 

Division, Office of Pesticide Programs, Office of Prevention, Pesticides, and Toxic Substances. 

Washington, DC. August 1, 2001. 

USEPA, 2001b. Personal communication. Anna Koutlakis, Office of Prevention, Pesticides, and 

Toxic Substances. Washington, DC. August 1, 2001. 

USEPA, 2001c. “Stakeholder Involvement & Public Participation at the U.S. EPA: Lessons 

Learned, Barriers, & Innovative Approaches.” Office of Policy, Economics and Innovation. 

Washington, DC. EPA-100-R-00-040. 

USEPA, 2001d. “Top 10 Watershed Lessons Learned.” Office of Wetlands, Oceans and 

Watersheds, Office of Water. Washington, DC. Internet: 

http://www.epa.gov/owow/watershed/lessons/top10.pdf. 

USEPA, 2001e. “National-Scale Air Toxics Assessment for 1996.” SAB Review Draft. Office of 

Air Quality, Planning and Standards, Office of Air and Radiation. Washington, DC. EPA-453-R01-003. 

USEPA, 2002a. “Guidance on Cumulative Risk Assessment of Pesticide Chemicals that Have a 

Common Mechanism of Toxicity.” Office of Pesticide Programs, Office of Prevention, 

Pesticides, and Toxic Substances. Washington, DC. January 14, 2002. Internet: 

http://www.epa.gov/pesticides/trac/science/cumulative_guidance.pdf. 

USEPA, 2002b. “Lesson Learned on Planning and Scoping for Environmental Risk 

Assessments.” Science Policy Council. Washington, DC. January, 2002. Internet: 

http://www.epa.gov/ORD/spc/2cumrisk.htm. 

USEPA, 2002c. “Summary Report of the Technical Peer Review Workshop on the EPA Risk 

Assessment Forum Draft Framework for Cumulative Risk Assessment.” Risk Assessment 

Forum, Washington, DC; EPA/630/R-03/002. Internet: http://www.epa.gov/ncea/raf. 

USEPA, 2002d. “Exposure and Human Health Reassessment of 2,3,7,8-Tetrachlorodibenzo 

-p-Dioxin (TCDD) and Related Compounds.” National Center for Environmental Assessment, 

Office of Research and Development. Washington, DC, (to be published). 

89
90

USEPA, 2002e “A Review of the Reference Dose and Reference Concentration Processes.” Risk

Assessment Forum, Washington, DC; EPA/630/P-02/002F. Internet: http://www.epa.gov/ncea/raf.

van den Berg, Martin, Linda Birnbaum, Albertus T.C. Bosveld, Björn Brunström, Philip Cook, Mark

Feeley, John P. Giesy, Annika Hanberg, Ryuichi Hasegawa, Sean W. Kennedy, Timothy Kubiak, John

Christian Larsen, F.X. Rolaf van Leeuwen, A.K. Djien Liem, Cynthia Nolt, Richard E. Peterson,

Lorenz Poellinger, Stephen Safe, Dieter Schrenk, Donald Tillitt, Mats Tysklind, Maged Younes,

Fredrik Wærn, and Tim Zacharewski, 1998. Toxic equivalency factors (TEFs) for PCBs, PCDDs,

PCDFs for humans and wildlife. Environmental Health Perspectives 106:775–792. 

WHO (World Health Organization), 2001. “Approaches to Integrated Risk Assessment.” International

Programme on Chemical Safety. Geneva. WHO/IPCS/IRA/01/12 Internet:

http://www.who.int/pcs/emerg_main.html.

Woo, Yin-Tak, Fred J. DiCarlo, Joseph C. Arcos, Mary Argus, Greg Polansky, and Jeff DuBose,

1994. Assessment of carcinogenic hazard of chemical mixtures through analysis of binary chemical

interaction data. Environmental Health Perspectives 102 (Supplement 9):113–118.

Woodruff, Tracey J., Jane Caldwell, Vincent J. Cogliano, and Daniel A. Axelrad, 2000. Estimating

cancer risk from outdoor concentrations of hazardous air pollutants in 1990. Environmental Research

Section A 82:194-206.

Zartarian, Valerie G., Halûk Özkaynak, Janet M. Burke, Maria J. Zufall, Marc L. Rigas, and Edwin J.

Furtaw, Jr., 2000. “A Modeling Framework for Estimating Children’s Residential Exposure and Dose

to Chlorpyrifos via Dermal Residue Contact and Non-Dietary Ingestion.” Environmental Health

Perspectives 108:505-514.
APPENDIX A: RESEARCH AND DEVELOPMENT NEEDS 

Framework for Cumulative Risk Assessment is intended to provide a basic structure for 

the issues and define key terms and concepts. In some cases, the concepts introduced in the 

framework report require the application of knowledge and methods that are not currently 

available. The following is a discussion of the needed areas of research and methods 

development highlighted within the report that may be most important to an evaluation of 

cumulative risks. This is not intended to be a comprehensive listing of cumulative risk 

assessment research needs. 

EPA and other scientists are currently investigating the use of similar approaches for 

cancer and noncancer assessments. Although we do not discuss this research need here, it would 

be useful to cumulative risk assessment to have similar approaches, and it is a topic of current 

discussion within scientific circles (e.g., Albert, 1999). 

Understanding the Timing of Exposure and its Relationship to Effects 

A key concept in the definition of cumulative risk is that it represents an accumulation of 

risk over time. However, unlike the traditional approach to risk assessment, where exposure 

events are summed and averaged over a period of time, cumulative risk assessment involves 

developing an understanding of how the sequence and timing of exposures influence the ultimate 

risk for effects. For example, for multiple stressors, it is important to understand how prior 

exposures to one or several stressors influence the risks from subsequent exposures to the same 

or different stressors. In addition, it is important to understand the implications of these 

exposures occurring during critical periods of an individual’s life (e.g., important periods of 

development or periods of disease). Several exposure models are under development that 

recognize the need to understand the timing of various exposure events (e.g., Calendex, APEX, 

Lifeline, SHEDS, and CARES/RExY). 

In addition to gaining a better understanding of the sequence and timing of exposures and 

their relationship to effects, it is important to understand how acute, nonlethal exposures from 

accidents contribute to chronic or long-term effects. 

Understanding the Composition and Toxicity of Mixtures 

Chemical mixtures can change or degrade over time and space, making the assessment of 

exposure a particular challenge. For cumulative risk assessment, the composition of the mixture 

at the point of contact with the receptor should be well characterized. Measurement techniques 

(at the receptor) and predictive models are both applicable in this characterization. 

EPA’s guidance for the health risk assessment of chemical mixtures (USEPA, 2000e) 

presents approaches for combining the toxicities of multiple chemical stressors. These 

approaches necessarily involve a number of simplifying assumptions when the mixtures are 

complex. Although the current methods provide a valuable resource for assessing cumulative 

91
risks, future cumulative risk assessment will need a more complete understanding of the 

interactions among chemicals in complex mixtures. Some current research efforts are seeking to 

identify toxicologic principles of joint action that are applicable to mixtures involving many 

chemicals. 

Applying the Risk Factor Approach to Environmental Health Risks 

The risk factor approach has been used in the medical profession to predict the chances 

of individuals developing various diseases. It has proved to be a useful approach not only in 

assessing certain cumulative risks, but also in communicating with patients. In this approach, 

characteristics of a population (e.g., age, ethnicity, personal habits, genetic polymorphisms, prior 

diseases, etc.) are correlated with the incidence of disease. For some diseases (e.g., breast 

cancer, coronary artery disease, stroke) these correlations are well established. However, there 

are substantial data gaps in terms of the role played by exposures to environmental stressors in 

the development of human disease, and correlations of environmental exposures with disease 

outcomes are generally not available. 

Using Biomarkers and Biomonitoring 

The use of biomarkers of exposure or effect holds a great deal of promise for cumulative 

risk assessment. This approach can provide a method for assessing stressors in groups. 

Currently, however, this approach is not practicable when considering a large number of diverse 

stressors, because appropriate biomarkers for many types of stressors have not yet been 

developed. 

Considering Hazards Presented by Nonchemical Stressors 

Cumulative risk assessment could encompass the interactions of chemical stressors with 

biological, radiological, and other physical stressors; socioeconomic stressors; and lifestyle 

conditions. In trying to assess all these different types of stressors, it is helpful to determine 

what types of effects the stressors produce and then to try to group stressors by like effects. 

Ideally, one would like to know the mechanism or mode of action by which various stressors 

cause effects to allow a more refined grouping. Currently, however, there are few methods for 

understanding how these disparate stressors interact to result in risk. 

Considering Psychological Stress as Part of Cumulative Risk 

Psychological stress causes both psychological and physiological changes that can be 

measured. However, assessing levels of stress and their potential contribution to risk is difficult 

for a variety of reasons. The Agency for Toxic Substances and Disease Registry began the 

process of identifying research needs in this area through an expert panel workshop held in 1995. 

Considering All Aspects of Vulnerability 

92
The issue of the vulnerability of a population can be thought of as having four 

components: susceptibility of individuals, differential exposures, differential preparedness to 

withstand the insult, and differential ability to recover from effects. Traditional risk assessment 

may consider one or more of these categories, but rarely are all considered. The overall 

consideration of all four categories may be more important in cumulative risk assessment than in 

traditional one-chemical assessments. A cumulative risk assessment, for example, may need to 

consider potential combinations of high exposure and high vulnerability across stressors. 

Methods development work is needed in this area. 

Methods for Combining Different Types of Risk 

Another key concept in the definition of cumulative risk assessment is that such an 

assessment represents the combined risk from multiple stressors. This implies that, in some 

cases, it may be necessary to combine disparate measures of risk (i.e., different types of effects) 

to simplify the expression of cumulative risks. There have been some attempts to collapse 

complex arrays of risk into a few or even a single measure. These approaches have involved the 

use of common metrics (e.g., quality-adjusted life years, disability-adjusted life years, loss of life 

expectancy, etc.) and indices (e.g., hazard ranking system, etc.) and the categorization of effects 

(e.g., as for categorical regression). Alternatively, geographic information systems and mapping 

techniques can be used to graphically portray integrated information on risks without 

mathematically combining disparate measures. Much methods development work remains to be 

completed in each of these areas. 

Development of Default Values for Cumulative Risk Assessments 

Conventional risk assessments use a series of default values for screening or other 

applications, and it may be necessary to investigate whether certain defaults need to be 

established specifically for cumulative risk assessments. 

Development of Case Studies and Issue Papers on Specific Cumulative Risk Topics 

The more detailed technical issues and methodologies should be developed as a series of 

issues papers that would augment the framework report. The level of detail would, of course, 

vary, depending on the topic. The issues papers (or white papers) should also include details on 

additional approaches to cumulative risk assessment that are currently being explored (including 

screening-level analyses, place-based assessments, comparative risk assessments, National 

Environmental Policy Act cumulative effects analyses, and hazard assessments). In addition, the 

issues papers could include summaries of case studies of cumulative risk projects that would 

extend the framework from theoretical to practical approaches and applications. 

93
APPENDIX B: SELECT RESOURCES FOR EXPOSURE AND RISK ASSESSMENT 

B.1. Resources Relevant to Chemical Exposures 

EPA Guidelines: 

Most of EPA’s general guidelines are listed in the text box in Section 1.1. 

Air-related sources and activities: 

EPA’s Clearinghouse for Inventories and Emission Factors (CHIEF) website 

(www.epa.gov/ttn/chief/) is an excellent starting place. It has many of the relevant 

documents on methods and data for constructing emissions inventories available for 

download, including Handbook for Criteria Pollutant Inventory Development: A 

Beginner’s Guide for Point and Area Sources (USEPA, 1999k), Handbook for Air Toxics 

Emission Inventory Development, Volume I: Stationary Sources (USEPA, 1998i), and the 

two volumes and supplement of Compilation of Air Pollutant Emission Factors (for both 

stationary and mobile sources) (USEPA, 1995e, 1996d, 1997d, 2000h), as well as many 

other documents and software. 

EPA’s Support Center for Regulatory Air Models (SCRAM) website 

(www.epa.gov/ttn/scram/) provides extensive information on the models discussed in 

Guideline on Air Quality Models (USEPA, 1999e), including downloadable software and 

users guides for many of the models. 

The Ambient Monitoring Technology Information Center (AMTIC) website 

(www.epa.gov/ttn/amtic/) contains information on monitoring programs and methods and 

other monitoring-related information. 

The umbrella website for all three of the above is the Technology Transfer Network 

(www.epa.gov/ttn/), which also has other useful information and links in addition to 

those noted above. 

Sources for land and waste-related activities: 

EPA’s Office of Solid Waste and Emergency Response has compiled an extensive 

catalog summarizing their publications (USEPA, 2000i). It has also published a “peer 

review draft” document titled Human Health Risk Assessment Protocol for Hazardous 

Waste Combustion Facilities (USEPA, 1998j), which deals with how to assess risks from 

hazardous waste incinerators. These reports are available on-line. 

94
Chemical accidents and transportation-related spills: 

Assessing an accidental chemical release exposure involves several steps. The typical 

analytical steps in the overall assessment are process analysis, likelihood or frequency of 

accidents, source term modeling, dispersion or consequence modeling, and the exposure 

assessment. 

< The process analysis is a formal, systematic analysis of the process where a 

chemical is handled to determine the probabilities and consequences of acute, 

catastrophic failures of engineered systems leading to an accidental release of the 

chemical. This analysis is often called a process hazards analysis (PHA). Several 

formal PHA evaluation techniques are available, including “What-If,” “Failure 

Mode and Effect Analysis,” “Event-Tree,” and “Fault-Tree” analyses (USEPA, 

1998e; AIChE, 1992). 

< The likelihood or frequency of accidents step is an evaluation of each of the 

scenarios uncovered in the process analysis step for likelihood or frequency of 

occurrence. 

< Source term modeling, which estimates the amount or rate of release in case of 

accident, is performed once the failure scenarios are determined. A wide variety 

of published calculation methods or models are available (USEPA, 1998e, 1999d) 

to determine the source terms for an accidental chemical release. 

< Dispersion or consequence modeling is performed once the source terms (rate and 

duration of the release) are known. A wide variety of dispersion and consequence 

modeling tools, ranging from simple screening models to sophisticated and 

complex computer applications, are available for this step (USEPA, 1993a, 

1999d; AIChE, 1996). In addition to the source terms generated above, several 

other data elements are needed, such as physical/chemical properties (e.g., 

whether the vapor cloud is heavier than air or water reactive), meteorological 

conditions (e.g., wind speed and direction, temperature, humidity), and terrain 

surrounding the facility (e.g., buildings or valleys that may channel or disperse a 

vapor cloud). Physical/chemical properties can be found in chemical reference 

texts such as Kirk-Othmer’s Encyclopedia of Chemical Technology (Kroschwitz 

and Howe-Grant, 1994), Perry’s Chemical Engineers’ Handbook (Perry et al., 

1997), on Material Safety Data Sheets (MSDS)18, or Risk Management Guidance 

for Offsite Consequence Analysis (USEPA, 1999d). Meteorological conditions 

are often collected on-site or at local airports. Information about terrain can be 

collected from topological maps or by visual inspection. Guidance on all these 

parameters is available in USEPA (1999d). 

18 There are many searchable MSDS databases on-line that can be located with most search engines. 

95 
< The final step is the exposure assessment, which is related to, and builds from, the 

dispersion or consequence modeling step. The dispersion or consequence 

modeling depends on a health endpoint and the exposure level related to that 

endpoint. Besides lethality, concentrations for certain health effects (e.g., odor 

thresholds, eye irritation) are available for several common toxic substances 

(NIOSH, 1997; ACGIH, 1998; AIHA, 2000). 

B.2. Resources Relevant to Exposures to Nonchemical Stressors 

Biological stressors: 

The International Life Sciences Institute’s Risk Science Institute has published a 

workshop report entitled “Revised Framework for Microbial Risk Assessment” (ILSI, 

2000), which looks at methods for assessing risks to microorganisms such as 

Cryptosporidium, which has caused disease outbreaks when it contaminates drinking 

water. The methodology is superficially similar to that of a risk assessment conducted 

for a chemical pollutant, but only at the most general level. For example, the 

characterization of exposure in the ILSI framework differs from that in an environmental 

chemical exposure assessment; it includes (1) pathogen characterization, (2) pathogen 

occurrence, (3) exposure analysis, and, finally, developing (4) an exposure profile. 

Radiological stressors: 

EPA’s Office of Air and Radiation maintains a web page at 

<http://www.epa.gov/radiation/assessment/>. This page provides (or cites) much of the 

needed documentation for performing risk assessments for radionuclides, including 

Radiation Exposure and Risk Assessment Manual (RERAM) (USEPA, 1996e) and several 

Federal guidance reports (USEPA, 1988, 1993d, 1999l). 

Noise, vibration, and congestion: 

The U.S. Department of Housing and Urban Development (HUD) has issued The Noise 

Guidebook (HUD, 1991), which implements the existing noise regulations (24 CFR 51-

B) and includes the HUD noise assessment guidelines. (The guidebook is available in 

hard copy only.) 

The Federal Railroad Administration has developed a manual titled High-Speed Ground 

Transportation Noise and Vibration Impact Assessment (DOT, 1998), which provides the 

theory, equations, and applications of noise and vibration analysis for high-speed 

railroads. Much of the theory and information is also applicable to other noise and 

vibration problems. Appendix A of the DOT guide is a general discussion of noise 

concepts, with references. The guide is available on-line 

(http://project1.parsons.com/ptgnechsr/noise_manual.htm). 

96
The National Institute of Occupational Health and Safety has done much research on the 

interaction of noise with chemical exposures (Morata, 2000). 

Odor: 

EPA’s Office of Wastewater Management has issued a report titled Guide to Field 

Storage of Biosolids (USEPA, 2000j), which contains an appendix on “Odor 

Characterization, Assessment, and Sampling.” Odor assessment is an analyticdeliberative process involving both science-based analytical methods and more 

subjective analysis. The appendix of the guide discusses sensory characterization of 

odors (character, intensity, pervasiveness, quantity), some practical options for assessing 

odors in a community, and the chemistry of odors (including range of odor thresholds). It 

also discusses odor sample collection and analysis and has several dozen references for 

further information. This report is available on-line (www.epa.gov/owm/bio/fsguide/). 

97
APPENDIX C: SOME THOUGHTS ON BACKGROUND EXPOSURES19 

When looking at aggregate exposures or cumulative risks of citizens, “background 

exposures” to specific chemicals are no less “real” than the exposures to pollution usually 

studied for regulatory purposes. Whereas in historical single-chemical assessments conducted 

for limiting pollution, background sources of the chemical were often irrelevant to the questions 

being asked of the assessment (or ignored as having negligible effect on risk), background 

sources in cumulative risk assessments are rarely irrelevant.20 

Background concentrations can be categorized as either naturally occurring, that is, 

chemicals that are naturally present in the environment before it was influenced by humans, or 

anthropogenic, that is, present in the environment due to historical human-made sources. 

Naturally occurring background chemicals may be either localized or ubiquitous. Anthropogenic 

background sources can be either localized from a point source or generalized from unidentified 

sources or nonpoint sources. 

Assessments of morbidity incidence and death rates, market basket surveys, and pesticide 

residue surveys also provide information that can be reflective of background chemical 

concentrations as well as overt pollution. Background issues extend across all media, beyond 

regulated sources, and beyond direct exposure. Many chemicals are naturally present in the 

environment (e.g., soils, water, vegetation, and other biota) and are consequently part of dietary, 

dermal, and inhalation exposures. In some cases, naturally occurring substances may be present 

at levels that exceed health-based or risk-based regulatory standards (e.g., drinking water 

standards) or other levels established to protect human health and the environment. Because 

cumulative risk assessments are population based, exposures due to naturally occurring 

background concentrations should typically be considered important. 

19 Several terms are used to discuss background, and there are several ways to describe different aspects of 

this issue. It has been suggested (deFur, 2002) that a more appropriate term for present conditions is “ambient,” and 

that “background” should be reserved for some untouched, even pristine state or condition. Although the Technical 

Panel discussed this use of the word “background” as a pristine reference area, the discussion in this appendix is 

meant to more closely reflect the way the word is used in practice within EPA. It is acknowledged that not all 

programs or scientists even within EPA use this term to mean the same thing. 

20 The word “background” is often used to describe exposures to chemicals or other stressors that derive 

from sources other than the sources being assessed. For example, in the Agency’s assessment of residual risk 

associated with hazardous air pollutant emissions from particular categories of sources that remain after the 

implementation of technology-based controls, “background” is defined as all hazardous air pollutant exposures (via 

inhalation or other routes) not associated with the source(s) being assessed. At a Superfund site, “background 

contamination” refers to contamination that is not related to the site release of chemicals, as defined by 

Comprehensive, Environmental Response, Compensation and Liability Act (CERCLA) (P.L. 96-510, December 11, 

1980, as amended by P.L. 98-802, August 23, 1983, and P.L. 99-499, October 17, 1986). Such focusing or 

segregation in a risk assessment can be useful to decisions involving pollution sources covered by particular 

statutory authorities, but it is typical of a chemically focused assessment rather than a population-focused assessment 

such as a cumulative risk assessment. 

98 
There are several important issues related to natural or anthropogenic background 

concentrations in cumulative risk assessment. First, if the risks posed by background 

concentrations of certain chemicals are significant (and some may approach or exceed health 

reference levels), their exclusion from the cumulative risk estimates and characterization may 

seriously distort the portion of the total estimated risk thought to be posed to the population by a 

specific evaluated source. A second issue is the problem of whether background chemical 

exposures can be clearly distinguished from specific source-related chemicals and how to 

quantify these exposures. It may be important in a cumulative risk assessment to estimate 

background exposures separately from specific source-related exposures, so that the risk assessor 

can provide the community with a more complete picture of both total and known source-related 

risks. This also provides a clearer, more complete picture for making risk management 

decisions. Finally, there may be problems in identifying representative geographic areas for 

determining background levels for comparison. 

Finally, background exposures for a community or population may also include both 

voluntary and involuntary exposures and subsequent risks. Involuntary exposures are associated 

with the naturally occurring or anthropogenic background concentrations described above. 

Voluntary exposures, such as are associated with lifestyle decisions, are exposures due to 

activities such as smoking, consuming char-grilled meats with polycyclic aromatic 

hydrocarbons, or other choice-based exposures and may also sometimes be defined in the 

assessment as background exposures if they are not assessed directly in the cumulative risk 

assessment. 

99
APPENDIX D: EXAMPLES OF OUTLINES OF ANALYSIS PLANS21 

D.1. Outline for Human Health Analysis Plan for Pesticides Under the 1996 Food Quality 

Protection Act (FQPA) 

Risk management/regulatory goal: Protection of the general human population and susceptible 

subpopulations to adverse effects from exposure to pesticide “X” under FQPA. 

Assessment Endpoints: 

- human or animal health status of exposed versus unexposed populations/cohorts/dose 

groups 

Measures of Effects: 

- general types of toxicological effects grouped according to acute, subchronic, and 

chronic exposure durations 

- organ-specific toxicity such as reproductive effects, developmental effects, 

neurotoxicity, developmental neurotoxicity, immunotoxicity, hepatotoxicity, 

pulmonary effects, cardiovascular effects, etc. 

- general classes of toxic effects such as carcinogenicity, mutagenicity 

Measures of Exposure: 

- monitoring of food, water, residential, occupational exposures, etc. (direct or surrogate) 

- monitoring of biological fluids or biomarkers (blood, urine, DNA or other 

macromolecules) 

What Can and Cannot Be Done Based on Planning and Scoping 

- pathways and relationships to be evaluated 

- resource restraints 

- milestones for completion of risk assessment 

Methods for Conducting Risk Analysis 

- RfD 

- margin of exposure 

- probabilistic risk assessment based on dose-response or exposure parameters 

- quotients (e.g., ratio of exposure level to toxicity threshold) 

- narrative discussions 

- other considerations (e.g., mechanisms of action, toxicokinetic models, timing of dose, 

sensitive population characteristics) 

Data Needs and Uncertainties 

21 Conceptual models are not included here. 

100 
D.2. Outline for Ecological Analysis Plan 

Risk management/regulatory goal: Viable, self-sustaining coho salmon population that supports 

a subsistence and sport fishery. 

Assessment endpoints: Coho salmon breeding success, fry survival, and adult return rates. 

Measures of Effects: 

- egg and fry response to low dissolved oxygen 

- adult behavior in response to obstacles 

- spawning behavior and egg survival with changes in sedimentation 

- population data over time in relation to fish passage 

Measures of Ecosystem and Receptor Characteristics: 

- water temperature, water velocity, and physical obstructions 

- abundance and distributions of suitable breeding substrate 

- abundance and distribution of suitable food sources for fry 

- feeding, resting, and breeding behavior 

- natural reproduction, growth, and mortality rates 

Measures of Exposure: 

- number of hydroelectric dams and associated ease of fish passage 

- toxic chemical concentrations in water, sediment, and fish tissue 

- nutrient and dissolved oxygen levels in ambient waters 

- riparian cover, sediment loading, and water temperature 

What Can and Cannot Be Done Based on Planning and Scoping 

- pathways and relationships to be evaluated 

- resource restraints 

- milestones for completion of risk assessment 

Methods for Conducting Risk Analysis 

- quotients 

- narrative discussions 

- stressor-response curves with probabilities 

Data Needs and Uncertainties 

101
APPENDIX E: TOXICOLOGIC SIMILARITY—ORGANOPHOSPHORUS 

PESTICIDES 

The Food Quality Protection Act of 1996 (FQPA) requires that EPA reassess pesticide 

tolerances (legal limits for residues in food) that were in effect as of August 1996. As part of the 

reassessment, EPA should consider available information concerning the cumulative effects on 

human health resulting from exposure to multiple chemicals that have a common mechanism of 

toxicity. In this context, pesticides are determined to have a common mechanism of toxicity if 

they produce the same toxic effect in the same organ or tissue by essentially the same sequence 

of major biochemical events (USEPA, 1999h). 

Shortly after enactment of FQPA, EPA began developing new methods and tools that 

would allow the consideration of combined risks from exposure to several pesticides via several 

pathways and routes of exposure. Actual data sets for organophosphorus (OP) pesticides were 

used in pilot analyses to test these methods. The methods and pilot analyses were subjected to 

peer review through the FIFRA Scientific Advisory Panel to ensure the use of sound science. As 

part of this ongoing effort, on December 28, 2001 EPA’s Office of Pesticide Programs (OPP) 

announced the availability of the preliminary organophosphorus cumulative risk assessment 

[66FR67249-67250]. The risk assessment is available electronically at 

<http://www.epa.gov/pesticides/cumulative>. In preparing the cumulative risk assessment for 

the OP pesticides, OPP followed five major steps. 

1. Selection of the specific pesticides, pesticide uses, and pathways and routes of exposure to 

include in the quantitative analysis. 

The selection of the specific OP pesticides began with identifying a “common 

mechanism group.” This was accomplished following Guidance For Identifying 

Pesticide Chemicals And Other Substances That Have A Common Mechanism Of 

Toxicity (available at http://www.epa.gov/pesticides/trac/science). All 39 registered OP 

pesticides share inhibition of acetylcholinesterase as a common mechanism for causing 

adverse effects (USEPA, 1998k). 

The common mechanism group was further refined to reflect current use patterns and 

information on the detection of residues from USDA’s Pesticide Data Program. This 

resulted in the following recommendations for quantitative analysis: include 22 OP 

pesticides for the food pathway of exposure; 24 OPs for the water pathway and 10 OPs 

for residential exposures were identified on the basis of use patterns and their individual 

assessments. 

102
2. Dose-response analysis for toxic potencies, relative contribution from each OP pesticide, and 

selection of an index chemical to use as the point of reference in the dose-response analysis. 

To determine the combined risk from multiple OP pesticides, EPA used the relative 

potency factor approach (for additional examples of comparative potency approaches, see 

Albert et al., 1983; Lewtas, 1985, 1988). The index chemical was selected on the basis of 

the quality of the dose-response data. Then the relative potency of each OP pesticide was 

estimated by taking the ratio of its toxic potency to that of the index chemical. 

In selecting studies for evaluating toxic potencies, EPA used relative potency factors and 

points of departure developed from cholinesterase inhibition in rats exposed to pesticides 

for 21 days or more. This practice was adopted to reflect cholinesterase inhibition at a 

point in the treatment schedule at which a steady state had been achieved. OPP elected to 

use data that reflected a steady state in the interest of producing relative potency factors 

that are reproducible and reflect less uncertainty due to rapidly changing time-sensitive 

measures of cholinesterase. 

Also, EPA considered that people generally have some level of prior exposure to OP 

pesticides. Further, the effects of exposure can persist for several days to weeks. 

Therefore, people may be more vulnerable to subsequent exposures to OP pesticides than 

might be predicted if these prior exposures are not considered. 

3. Estimation of the risks associated with all pertinent pathways of exposure in a manner that is 

both realistic and reflective of variability due to differences in location, time, and demographic 

characteristics of exposed groups. 

Evaluation of the OP pesticide use profiles allowed for the identification of exposure 

scenarios that may overlap, co-occur, or vary between chemicals. In addition, the use of 

profiles allowed for the identification of populations of potential concern. On the basis 

of this analysis, EPA considered exposure to OP pesticides in food to be uniform across 

the nation (i.e., there are no significant differences in food exposure due to time of year 

or geographic location). For the residential and drinking water pathways of exposure, 

EPA divided the nation into 12 regions for assessment. This allowed for the 

consideration of such factors as the location of vulnerable surface watersheds and regionspecific pest pressures. To estimate risks, EPA used Calendex, a calendar-based 

computer model. This model integrates the various pathways of exposure while 

simultaneously incorporating the time dimensions of the data. The model produces a 

detailed profile of the potential exposure to individuals across a calendar year. 

4. Identification of the significant contributors to risk. 

Although interpretation of the preliminary organophosphorous cumulative risk 

assessment is ongoing, there are some early indications concerning contribution to risk. 

The drinking water pathway for exposure does not appear to be a major contributor to the 

103
total cumulative risk. Residential exposure appears to be a contributor to risk, 

particularly inhalation exposures from certain no-pest strips and crack and crevice 

treatments. Childhood exposure from mouthing hands also appears to be a contributor, 

but there is a great deal of uncertainty associated with the estimates. 

5. Characterization of the confidence in the results and the uncertainties encountered. 

In addition to some uncertainties noted above, EPA identified many areas for additional 

analysis, including sensitivity analyses on input parameters, verification of residential use 

patterns, closer examination of the tails of the food consumption distribution, and 

evaluation of the effect of assumptions about residue concentrations in baby foods. 

104
APPENDIX F: OTHER TYPES OF CUMULATIVE ASSESSMENTS 

Several other types of cumulative assessments are related to the types of human health 

and ecological cumulative assessments done by the Agency. It is beyond the scope of this 

framework to discuss these in detail, but a short explanation of several other types of cumulative 

assessments are given in this appendix. 

F.1. Quality-of-Life Assessments 

One type of assessment that resembles a cumulative risk assessment—but whose 

evaluation may require a different approach from the traditional National Research Council risk 

paradigm—is the quality-of-life assessment. These assessments define “harm” to an individual 

or community broadly, then evaluate the importance of the various threats of harm to a set of 

“quality-of-life” criteria. These assessments do not usually attempt to predict probability that the 

harm will occur (as would a cumulative risk assessment), but rather aim to apply the 

community’s values to deal with the most important perceived threats. 

Although a quality-of-life assessment is not a risk assessment in most cases, changes in 

quality-of-life factors may affect the vulnerability of a population to health or ecological risks 

and consequently may be part of the considerations in a cumulative risk assessment. Because 

few, if any, established and accepted relationships are currently available quantitatively linking 

quality-of-life factors and health or ecological risk, this is an area in which further research may 

prove valuable. 

To evaluate the effects on human or ecological health from these types of impacts, a 

more deliberative approach (in the analytical-deliberative process) is needed than is used in, say, 

cancer risk analysis. To better help characterize these impacts, EPA’s A Guidebook to 

Comparing Risks and Setting Environmental Priorities (USEPA, 1993b) suggests a six-step 

process in quality-of-life analysis: 

1. Identify impacts and determine the values of the community. 

2. Identify and define evaluative criteria. 

3. Collect and analyze data on impacts. 

4. Characterize impacts for all problem areas. 

5. Present findings and rank problem areas for quality-of-life impacts. 

6. Analyze future environmental conditions and risk management considerations. 

105
Vermont’s Quality-of-Life Criteria 

Impacts on Aesthetics: Reduced visibility, noise, 

odors, dust and other unpleasant sensations, and 

visual impact from degradation of natural or 

agricultural landscapes. 

Economic Well-Being: Higher out-of-pocket 

expenses to fix, replace, or buy items or services 

(e.g., higher waste disposal fees, cost of replacing a 

well, higher housing costs), lower income or higher 

taxes paid because of environmental problems, and 

health-care costs and lost productivity caused by 

environmental problems. 

Fairness: Unequal distribution of costs and benefits 

(e.g., costs and benefits may be economic, healthrelated, aesthetic). 

Future Generations: Shifting the costs (e.g., 

economic, health risks, environmental damage) of 

today’s activities to people not yet able to vote or not 

born yet. 

Peace of Mind: Feeling threatened by possible 

hazards in air or drinking water or potentially risky 

structures of facilities (e.g., waste sites, power lines, 

nuclear plants), and heightened stress caused by 

urbanization, traffic, etc. 

Recreation: Loss of access to recreational lands 

(public and private) and degraded quality of 

recreation experience (e.g., spoiled wilderness, 

fished-out streams). 

Sense of Community: Rapid growth in population or 

number of structures or development that changes the 

appearance and feel of a town; loss of mutual respect, 

cooperation, ability, or willingness to solve problems 

together; individual liberty exercised at the expense 

of the community; the loss of Vermont’s landscape 

and the connection between the people and the land. 

Source: State of Vermont, 1991 

Quality-of-life impacts are determined 

by analyzing a set of criteria developed for 

each community, depending on what it 

values. Stressors are those things that 

threaten to degrade the quality-of-life criteria 

for that community. An example of a set of 

quality-of-life criteria and their descriptions 

is shown in the box on this page. These 

criteria were developed by the State of 

Vermont’s Agency of Natural Resources 

(State of Vermont, 1991). Vermont’s 

experience in evaluating these criteria was 

described as a qualitative description of harm 

or, in their terms, “risk”: 

Because most of these seven criteria 

are intangible, they are extremely 

difficult to measure or quantify. The 

Quality-of-Life Work Group 

described how each problem area 

affects each criterion and how 

widespread or intense the effects are. 

Although these non-quantitative 

descriptions of risk often lack 

precision and scientific objectivity, 

they focus attention on specific 

critical issues and thus are useful tools 

for comparing the problems 

systematically and consistently. (State 

of Vermont, 1991) 

Quality-of-life issues can encompass 

much more than the criteria shown in the 

example and thus may introduce much 

additional complexity into the analysis. For 

instance, there may be feedback loops that 

cannot be easily evaluated, for example, loss 

of property value or aesthetics tends to 

negatively affect the socioeconomic system, 

which tends to increase rates of crime, traffic 

accidents, and communicable pathogen transmission, all of which in turn ultimately reflect on 

overall community health or ecological risk. Some cumulative risk assessments may 

consequently include quality-of-life impacts as indirect measures of health effects if sufficient 

links can be established between the two. 

106
F.2. Cumulative Impact Assessments 

The National Environmental Policy Act (NEPA) defines “cumulative impact” (see box), and has 

certain requirements for a cumulative impacts analysis. Although the Council on Environmental 

Quality’s guidelines for cumulative impact analysis (CEQ, 1997) take a primarily qualitative 

approach to the analysis, this is a multiple-stressor, multiple-effect analysis that looks at a 

variety of impacts on the environment. 

The projects or actions that NEPA addresses can be viewed as sources of stressors. 

Under NEPA, a description of the affected environment in an environmental impact assessment 

contains four types of information: (1) data on the status of important natural, cultural, social, or 

economic resources and systems; (2) data that characterize important environmental or social 

stress factors; (3) a description of pertinent regulations, administrative standards, and 

development plans; and (4) data on environmental and socioeconomic trends. Health effects on 

populations and susceptible individuals are part of the affected environment as considered by the 

NEPA cumulative effects analysis, but the NEPA analysis may also consider effects on historic 

and archaeological resources, socioeconomic factors such as employment, human community 

structure, and quality of life changes. 

Although there is not always a clear relationship between these NEPA cumulative 

impacts and effects relevant to human health, the NEPA methods and tools for cumulative 

impact analysis may be useful for cumulative risk assessments. For example, cumulative impact 

analysis begins with an extensive scoping process and relies on conceptual models to plan the 

analysis. NEPA effects data may help risk assessors identify susceptible subpopulations, 

environmental pathways, or exposure patterns. 

EPA Region 6 has developed a 

system called the Cumulative Risk Index 

Analysis (CRIA), primarily for NEPA-type 

assessments (Osowski et al., 2001). The 

CRIA contains some 90 criteria with which 

to evaluate the health of an area and its 

ecosystem/human populations. These criteria 

help evaluate such diverse factors as human 

health, ecosystem health, and environmental 

justice considerations. Each criterion, which 

leads to an indexing of 1 through 5, has been 

through the deliberative process and peer 

review and is well documented. 

We also acknowledge that other 

Federal agencies have been preparing 

“cumulative risk analyses” for various 

NEPA’s “Cumulative Impact” Definition 

Council on Environmental Quality Regulation 1508 

for Implementing the National Environmental Policy 

Act of 1969 [P.L. 91-190, 42 U.S.C. 4321-4347, 

January 1, 1970, as amended by P.L. 94-52, July 3, 

1975, P.L. 94-83, August 9, 1975, and P.L. 97-258, 

§4(b), Sept. 13, 1982] defines “cumulative impact” 

as “the impact on the environment which results from 

the incremental impact of the action when added to 

other past, present, and reasonably foreseeable future 

actions regardless of what agency (Federal or 

non-Federal) or person undertakes such other actions. 

Cumulative impacts can result from individually 

minor but collectively significant actions taking place 

over a period of time.” 

Source: CEQ, 1997 

107
purposes related to their own mission as part of environmental impact statements (e.g., NOAA, 1999). 

F.3. Empirically Derived Medical Models 

The medical profession has long used empirically derived models to predict the chances 

of particular health effects in individual patients. In this approach, the characteristics of 

individuals within the population are correlated with the incidence of specific diseases or effects. 

For example, the risk factors for stroke are increasing age, heredity (family history) and race, 

prior stroke, high blood pressure, cigarette smoking, diabetes mellitus, carotid and other artery 

disease, heart disease, transient ischemic attacks, high red blood cell count, sickle cell anemia, 

socioeconomic factors, excessive alcohol consumption, and certain types of drug abuse 

(American Heart Association, 2000). Each of these risk factors can be correlated with stroke 

incidence, and then the risk of stroke from various combinations of these factors can be explored. 

In this way, the analysis is “cumulative,” but “risk factors” are not always synonymous with 

“stressors.” 

Physicians use models containing effect-specific risk factors to advise patients of the 

probabilities of future effects (e.g., stroke, breast cancer) on the basis of their medical history. 

Although the medical data upon which these factors are based have been well developed for 

many effects in humans, there are substantial data gaps in terms of the role played by exposures 

to many chemicals in the environment in the development of human disease. This empirically 

derived medical model approach to cumulative risk may be built on links between risk factors 

and effects for better-studied stressors but may be limited or nonexistent for less robust health 

effects databases. Although this approach may some day be applicable to human health and 

environmental risk assessment such as EPA conducts, at present the data and methods are not 

available. 

In a larger sense, although empirically derived models may be cumulative risk models, 

the approach to determining risk is substantially different from the risk assessment approach 

used by EPA, where a combined effect is estimated as the predicted aggregation of the effects of 

several different stressors. In an empirical model such as physicians use, the focus is on an 

effect of concern, and the model derives the influence of various “stressors” or “risk factors” 

from actual observations, usually through the use of multiple regression analyses. Although 

ideally the equations derived to represent the influences of various factors on the measured 

outcome (the effect of concern) would be causal-predictive models, in practice they are usually 

the most parsimonious “best fit” equations that satisfy statistical criteria. The versatility of this 

approach, however, is the ability to tease apart contributions of different sources of 

environmental exposures of interest. This is illustrated by Laden et al. (2000) in the association 

of particulates from different sources with short-term mortality changes. This approach also has 

considerable potential to be used in conjunction with biomarkers as dependent variables (Hattis, 

2002). 

The topic of cumulative risk models will likely be covered in more detail in the future 

guidelines for cumulative risk assessment. 

108
F.4. Risk Surrogates 

Geographic information systems and related mapping techniques (see, e.g., 

Environmental Defense, 2001) appear to hold some promise as tools for presenting integrated 

information concerning cumulative risks without mathematically combining disparate measures. 

Considerable methods development work remains to be completed. 

Not all statements of probability of harm are expressed as probabilities of specific health 

effects. Cohen (1991) uses mortality ratios to derive “loss of life expectancy” (LLE) estimates 

for a wide variety of risk-related activities. For example, workers in all occupations have a 60-

day LLE as a result of working, but workers in agriculture have a 320-day LLE and construction 

workers a 227-day LLE as a result of their particular occupation. These types of statements are 

empirically derived, probability-based statements of harm that do not use “probability of adverse 

health effect” as the basis for the risk statement. For estimates such as LLEs, one could 

theoretically add up the various activities and the corresponding LLEs in days to estimate a 

cumulative risk in terms of loss of life expectancy. These “other” types of risk-surrogate 

probability statements could conceivably be used in cumulative risk assessment, although 

currently they are not widely used, perhaps due to lack of methods. 

109