EPA/630/R-03/002

September 5, 2002 

Summary Report of the Technical Peer Review Workshop 

on the EPA Risk Assessment Forum 

Draft Framework for Cumulative Risk Assessment

U.S. Environmental Protection Agency

Washington, DC

June 4-5, 2002

Risk Assessment Forum

U.S. Environmental Protection Agency

Washington, DC 20460
NOTICE 

This document has been reviewed in accordance with U.S. Environmental Protection Agency 

(EPA) policy and approved for publication. Mention of trade names or commercial products 

does not constitute endorsement or recommendation for use. 

This report was prepared by Versar, Inc., an EPA contractor (Contract No. 68-C-99-238, Task 

Order No. 68), as a summary of the discussion held at the Technical Peer Review Workshop on 

the Risk Assessment Forum Draft Framework for Cumulative Risk Assessment (June 4-5, 2002). 

This report captures the main points and highlights of the meeting. It is not a complete record of 

all details discussed, nor does it embellish, interpret, or enlarge upon matters that were 

incomplete or unclear. Statements represent the individual views of each workshop participant. 
TABLE OF CONTENTS

Page No. 

EXECUTIVE SUMMARY . . .................................................... i

1.0 INTRODUCTION . . .......................................................1

1.1 Workshop Purpose . ................................................1

1.2 Workshop Participants ..............................................1

1.3 Charge Questions . .................................................1

1.4 Agenda . .........................................................2

1.5 Workshop Summary Report .........................................2

2.0 SUMMARY OF OPENING REMARKS . . ......................................4

2.1 Welcome . .......................................................4

2.2 Chair’s Introduction . ...............................................4

2.3 Background on Risk Assessment Forum and Framework Development . ......5

2.4 Presentation of the Draft Framework for Cumulative Risk Assessment . . ......6

2.5 Discussion on Opening Presentations ..................................8

3.0 SUMMARY OF COMMENTS AND RECOMMENDATIONS . .....................9

3.1 Charge Questions #1 and #2 .......................................14

3.1.1 Terminology . ..............................................15

3.1.2 Structure for Cumulative Risk Assessment .......................16

3.1.3 Methods . .................................................19

3.2 Charge Question #3 . ..............................................21

3.3 Charge Question #4 . . .............................................23

3.3.1 Vulnerability ..............................................23

3.3.2 Chemical and Non-Chemical Stressors ..........................25

3.3.3 Different “Types” of Risks ...................................27

3.4 Summary List of Suggested Topics for Issue Papers . .....................29

4.0 DOCUMENT REVIEW . ...................................................30

4.1 Section 1 .......................................................30

4.2 Section 2 . ......................................................31

4.3 Section 3 . ......................................................33

4.4 Section 4 . ......................................................34

APPENDICES

APPENDIX A LIST OF PEER REVIEWERS

APPENDIX B LIST OF OBSERVERS

APPENDIX C CHARGE QUESTIONS

APPENDIX D AGENDA

APPENDIX E PRESENTER OVERHEADS

APPENDIX F WRITTEN COMMENTS

APPENDIX G DRAFT FRAMEWORK FOR CUMULATIVE RISK ASSESSMENT
EXECUTIVE SUMMARY 

The Peer Review Workshop on the Draft Framework for Cumulative Risk Assessment was held 

on June 4 and 5, 2002, in Crystal City, VA. The purpose of the meeting was to provide 

comments on the current draft of the Framework, both generally and in response to a series of 

charge questions. The reviewer suggestions and recommendations will be used by EPA in 

finalizing the Framework document. In addition, the reviewers raised issues on topics related to 

cumulative risk assessment that may warrant additional development in issue papers or other 

future activities. 

The meeting was opened by Versar staff, following which the Chair briefly discussed the process 

for the meeting. After two presentations by EPA staff providing a summary of the Framework 

and background information on the draft Framework, the Chair summarized premeeting 

comments. The reviewers then engaged in an open discussion on their general impressions of 

the document. Each person highlighted the issues that they felt should be discussed in greater 

detail during the meeting. This discussion of overarching issues was followed by responses to a 

series of charge questions, lasting the remainder of the first day and until noon on the second 

day. The last part of the meeting was devoted to a review of the document for any items that had 

not yet been discussed. Time was set aside on both days of the workshop for comments to be 

made by observers, though no observer comments were provided. 

Most reviewers prepared written premeeting comments in which they presented their individual 

reviews of the Framework. During the two-day review meeting, the reviewers raised a number 

of important points that they wished to emphasize in the written report to EPA. The suggested 

changes to the Framework document presented in this report generally represent the opinion of 

multiple reviewers. Recommended revisions to the Framework presented in this report were 

considered highly significant by the Chair because they were voiced by multiple reviewers or 

applied broadly to most of the document. The following list summarizes some of the most 

significant suggested changes to the document provided by the peer reviewers. 

Summary of Suggestions and Recommendations from Peer Reviewers: 

1) EPA did a good job with the Framework and the reviewers generally found the document easy 

to read, well written, and organized. It was clear that the authors found it difficult to convey the 

issues and concepts related to cumulative risk assessment. In addition, the draft reflects 

comments made during the August 2001 peer consultation and other public meetings. The text 

boxes add something to the content without distracting from the flow of the text – make sure the 

boxes are in a readable font. 

2) Terminology should be reviewed to make sure it is correct and does not change the use of a 

common term with an existing understood meaning. Reviewers found several places in which 

the Framework was unclear or created some ambiguity. The text should be consistent with the 

glossary; at present the two are not consistent. The glossary needs to be augmented and the 

number of entries greatly increased. 

i 
3) EPA should confirm that the Framework is consistent with existing disciplinary uses of 

terminology, methods, references, data, and present state of the art. EPA should use caution with 

terminology, to avoid revising the current use of terms and methods when they already have use 

and meaning in other disciplines. 

Reviewers discussed the definitions of cumulative risk and stressor, along with other terms. The 

reviewers acknowledged that EPA should set an operational definition for cumulative risk 

assessment that is practical within the existing regulatory and assessment programs. Caution 

should also be used with the term “stressor,” because there is already a field of experts using that 

term and the Framework should reflect that existing use of the term. 

4) The Framework should provide a window to the literature and the reviewers said that in many 

respects the document does accomplish this goal. However, the references cited by EPA and 

listed in several sections of the Framework are focused on EPA documents and products. EPA 

should provide citations to more than their own Agency documents, so the literature reflects a 

broad perspective and does not seem insular. The other area where the reviewers recommended 

that EPA broaden the literature cited pertains to ecological risk assessment, especially case 

studies. 

5) EPA should continually answer the query – What is unique to cumulative risk assessment? 

Reviewers raised this point in premeeting comments, in discussion of the whole Framework, and 

in discussion of every section and charge question. The reviewers also frequently asked this 

question of one another. 

6) Public health issues were raised in several contexts. Several reviewers said that public health 

needs more emphasis in the Framework. Others recommended that EPA use more public health 

data and emphasis in framing cumulative risk assessments for communities and populations. 

Public health data are already available to better inform such cumulative risk assessments, such 

as birth weight and biomarker data, which are natural integrators of multiple effects and have the 

potential to be used to assess the separate and combined influences of multiple stressors. 

7) Both qualitative and quantitative aspects of cumulative risk assessment are applicable in 

many, if not all areas of the Framework and the document should clearly articulate this fact. The 

reviewers focused on the analysis phase and uncertainty analysis, but this point applies broadly. 

8) The role of decision-makers and risk managers in the cumulative risk assessment process is 

important and should be spelled out in somewhat more detail, even for a framework document. 

The reviewers discussed the need for balance between the necessity to inform the process with 

management goals and objectives, and the requirement to prevent management from dictating 

the assessment outcome. 

9) Several reviewers raised issues of how this Framework document will fit into the process of 

developing guidelines for cumulative risk assessment. EPA did address this issue on pages 5-6, 

where the Framework discussed this point, but reviewers felt that more information would be 

ii 
useful. Maybe EPA could insert a text box that includes the other major steps in the whole 

process, such as issue papers and case studies leading to development of guidelines. 

10) Some aspects of the conduct of a cumulative risk assessment need to be highlighted in the 

initial steps of planning, scoping, and problem formulation. These aspects include population (or 

ecosystem) identification or selection, vulnerability, uncertainty analysis, data collection and 

analysis, statistical analysis, and statutory constraints. These steps are not unique to cumulative 

risk assessment, but are particularly important because the entire risk assessment depends on 

these first steps. Raising certain issues at the outset also serves to highlight them to the risk 

assessment team and the interested and affected stakeholders. 

11) The reviewers recommended addition of examples and references in some key parts of the 

Framework. The additions are intended to accomplish two purposes: (1) add ecological cases 

and perspectives in areas that are now exclusively human health discussions and (2) broaden the 

base, geographic range, or types of studies presented. 

12) The reviewers supported the use of indices or other methods to combine, analyze, and 

present data in a cumulative risk assessment. However, they had some concerns that data and 

qualitative information could be lost or otherwise obscured in moving from conceptual model to 

analysis phase, in analysis, in moving from analysis to characterization, in uncertainty or 

sensitivity analysis, etc. It is important to preserve data throughout the process of assessing risks 

in cumulative risk situations, in no small part because of the number and complexity of stresses 

and agents, endpoints, and methods. 

13) EPA should acknowledge and incorporate more of the international efforts on cumulative 

risk assessment. Some of these international efforts are noted in the present version of the 

Framework, but more notation would be an important enhancement. 

14) EPA should consider producing issue papers or background materials on the following: 

• Background conditions, especially compared with ambient, control etc. 

• Precautionary Principle 

• Uncertainty analysis 

• Vulnerability (defined and parsed) 

• Biomarkers of exposure and of effects 

• Single metrics in cumulative risk assessment (including combining chemical and 

non-chemical stressors) 

• A database of literature on case studies and methods. 

iii 
Summary of Responses to Charge Questions 

Question #1. Comment on whether the Framework adequately captures, describes, and 

reasonably organizes the key issues for cumulative risk assessment so as to serve as an adequate 

foundation for the development of future guidance. Does the Framework provide adequate 

coverage of: terminology, structure, and methods? 

The reviewers were largely satisfied that EPA had covered the subjects of terminology and 

methods in the Framework, with a discussion of the meaning of cumulative risk and the nature 

and scope of a framework document. Several suggestions were made for improving the 

treatment of these aspects of the Framework. The reviewers discussed the definitions of 

“cumulative risk” and “stressor” at some length, with some recommendations for EPA. The 

main improvements suggested by the reviewers were in: identifying and describing the issues 

that are unique for cumulative risk assessment, clarifying terminology in the context of related 

disciplines that use the same terms, and providing the reader with a clearer sense of what 

constitutes cumulative risk assessment and when such assessments may be required. Reviewers 

suggested that EPA use more examples of ecological risk cases and provide more references in 

the text. 

Question #2. Does the Framework document include any scientific or technical inaccuracies in 

its presentation of terminology, assessment structure, and methods? Please identify any problem 

areas and propose revisions or other actions that will result in a scientifically sound and 

supportable discussion. With respect to methods, comment on whether the Framework 

adequately conveys the state of the science with respect to currently available cumulative risk 

assessment methods/approaches and the areas that are in need of further research and 

development. 

The reviewers were satisfied that there were few scientific inaccuracies, but did make 

recommendations to clear up some ambiguities. The methods for cumulative risk assessment are 

mostly those developed for other risk assessment applications, so EPA does not have much to 

cover here, but should note the fact that new tools are required. 

Question #3. Comment on whether the Framework adequately characterizes the importance of 

uncertainty analysis in cumulative risk assessment. What additional discussions of uncertainty 

should be included in the Framework (and in what sections of the document)? 

The reviewers thought that the Framework covered the subject of uncertainty analysis 

sufficiently and suggested some changes to improve the document. One of these 

recommendations was to insert consideration of uncertainty analysis into the planning phase, so 

that much like statistical analysis, the preparation of the risk assessment will be done with the 

uncertainty analysis in mind. The reviewers also recommended that the process begin with 

qualitative uncertainty analysis and move into more quantitative aspects. 

iv 
Question #4. Comment on the adequacy and accuracy of the Framework’s presentation in each 

of these areas: vulnerability, chemical vs non-chemical stressors, and different types of risk. 

There was an extended discussion of vulnerability and most reviewers agreed that it is a critical 

issue in cumulative risk assessment, warranting introduction earlier in the Framework. 

Reviewers initially felt that the topic could be arranged in a manner that was different from how 

EPA had presented it in the draft Framework. During this discussion, a few reviewers 

introduced some alternate schemes for organizing vulnerability, but these did not differ 

substantially from the approach presented in the document. The issue of non-chemical stressors 

raised the incorporation of disease predictions and exotic species introductions, for which the 

reviewers had some recommendations for EPA. The topic of types of risk is one of the most 

complex areas of cumulative risk assessment and it seemed to have been adequately covered in 

the Framework, but the reviewers provided some feedback on possible improvements. 

v 
1.0 INTRODUCTION

1.1 Workshop Purpose 

The Technical Peer Review Workshop on the Risk Assessment Forum Draft Framework for 

Cumulative Risk Assessment was held on June 4-5, 2002, in Arlington, VA. The workshop was 

sponsored by the U.S. Environmental Protection Agency’s (EPA’s) Risk Assessment Forum and 

was convened and facilitated by Versar, Inc. The purpose of the meeting was to provide a 

scientific peer review of the draft Framework document. 

1.2 Workshop Participants 

A group of 13 experts, from different disciplines and types of organizations, was assembled by 

Versar to peer review EPA’s Draft Framework for Cumulative Risk Assessment. Twelve of these 

experts attended the June 4-5 workshop (one of the reviewers were unable to attend). Versar 

selected experts with experience related to cumulative risk assessment from a variety of 

perspectives: academia, consulting, industry, environmental groups, and community advocates. 

In addition, the experts were selected such that the following topic areas would be covered: 

aggregate exposure, risk assessment for chemical mixtures, accident and transportation risks, 

epidemiology, community-based risk assessments, socioeconomic issues, uncertainty analysis, 

and other topics of importance in cumulative risk assessment. Versar identified more than 50 

candidate reviewers, from which the final group of experts were selected. The list of peer 

reviewers is presented in Appendix A. In addition to these reviewers, about 20 observers 

attended the workshop. The list of observers is presented in Appendix B. 

1.3 Charge Questions 

A list of charge questions, presented in Appendix C, was prepared by EPA in advance of the peer 

review and was distributed to the reviewers to stimulate feedback on technical issues related to 

the draft Framework document. These charge questions addressed the document's technical 

1
content, accuracy, and clarity, as well as: (1) overall presentation of an assessment structure as 

well as information, terminology, and methods for cumulative risk assessment, (2) presence of 

any scientific or technical inaccuracies, (3) handling of uncertainty analysis in cumulative risk 

assessment, and (4) other technical challenges which have been under deliberation within the 

EPA Risk Assessment Forum Technical Panel, such as vulnerability, chemical and non-chemical 

stressors, and different types of risk. The charge questions were a starting point for the dialogue 

and participants were encouraged to raise other issues or topics. However, regulatory and policy 

issues were beyond the scope of the peer review. 

1.4 Agenda 

The workshop agenda is presented in Appendix D. The meeting began with opening remarks 

from Versar and the Chair including an overview of the agenda for the two-day meeting and a 

review of the objectives and process for the meeting. This was followed by presentations from 

EPA staff on the background of the draft Framework document. The agenda was organized 

around the four charge questions. After the morning presentations, most of the remainder of the 

first day was devoted to raising over-arching issues and then responding to the first two charge 

questions. Discussion on the second day focused on third and forth charge questions, which 

addressed technical issues such as vulnerability, uncertainty, and combining different types of 

risks. The end of the second day included a chapter-by-chapter review of the document and 

comments on specific sections. Time was set aside on both days of the workshop for observer 

comment, though no observer comments were provided. 

1.5 Workshop Summary Report 

This report summarizes the workshop presentations and discussion, with appendices that provide 

handouts, materials used in presentations, and written comments from the peer reviewers. The 

remainder of the report is organized as follows: 

• Section 2 of this report summarizes the opening presentations. Overheads used by the 

presenters are provided in Appendix E. 

2
• Section 3 provides summaries of the reviewers’ suggested changes to the document. 

Overarching comments are provided first, followed by the suggestions and 

recommendations made by the reviewers in response to the four main charge questions. 

• Section 4 summarizes the document review and lists the chapter- and page-specific 

comments that were raised at the end of the meeting. 

• The appendices to this report present the handouts from the meeting (e.g., lists of peer 

reviewers and observers, agenda, charge questions, and presentation materials/slides) as 

well as written comments from the peer reviewers. 

3
2.0 SUMMARY OF OPENING REMARKS 

2.1 Welcome 

David Bottimore of Versar, Inc., opened the meeting by welcoming participants and observers. 

He presented an overview of the agenda, introduced the participants, and described the goals and 

intended outcome of the workshop. During his opening remarks, he emphasized that the meeting 

was intended to promote dialogue and provide input on technical issues associated with the draft 

Framework. Mr. Bottimore talked about the effort to assemble a group of experts with diverse 

backgrounds and expertise to contribute suggestions and recommendations on ways to improve 

the draft Framework report. He also noted that in addition to the main discussion sessions, time 

would be set aside each day for comments from observers. His opening remarks were concluded 

by introducing the peer reviewers, including the Chair, Peter deFur. 

2.2 Chair’s Introduction 

Peter deFur, from Environmental Stewardship Concepts, was the Chair for the peer review 

workshop and served as facilitator. He started by describing the peer review process and setting 

the ground rules for the two-day workshop. The goal of the meeting was restated to emphasize 

that technical input was sought from each reviewer, noting that there would be no attempt to 

achieve consensus through this meeting. Rather, the discussion should bring out the diverse 

perspectives of individual experts in the group. He introduced ground rules and guides to keep 

the discussion focused on technical issues related to the draft Framework document. Finally, he 

talked about post meeting activities to prepare a workshop report that summarizes the discussion 

and made the request that each participant’s individual comments be represented by either preor post-meeting comments, which will be appended to the workshop report. 

4
2.3 Background on Risk Assessment Forum and Framework Development 

Steve Knott, from EPA’s Risk Assessment Forum (RAF), provided a short overview of the RAF 

and its activities working toward developing cumulative risk assessment guidelines. He stated 

that comments and recommendations from this peer review will be used by the Cumulative Risk 

Assessment Technical Panel in finalizing the Framework. His presentation began with 

background on the RAF, explaining that their mission is to promote consensus across the 

Agency and apply the best-available science in risk assessments. They usually are called upon to 

address difficult, precedent-setting scientific issues that are relevant to programs across the 

Agency. As a result, the RAF, and technical panels assembled to address particular issues, are 

composed of scientists from the Agency’s program, regional, and research offices. The RAF 

provides guidance to EPA risk assessors through three principal types of products (1) agency 

guidelines, (2) guidance documents, and (3) technical issue papers. Major previous efforts 

include the exposure assessment guidelines and the ecological risk assessment guidelines. 

Development of the Framework was initiated in 1999 in response to a request from the EPA 

Science Policy Council that the RAF begin developing guidance on cumulative risk assessment. 

The draft Framework document has been developed by a Technical Panel composed of scientists 

from more than ten Agency offices and regions. They have been working over the last two 

years, consulting twice with the Executive Committee of the Science Advisory Board (SAB) and 

meeting with Federal and State government scientists, to obtain input for the Framework. In 

addition, a peer consultation workshop was held in August 2001 to provide interested 

stakeholders with an opportunity to see an early draft of the Framework and to obtain input from 

12 experts in various fields related to cumulative risk assessment. Steve Knott emphasized that 

the Framework is the first step in the overall process of developing guidelines on cumulative risk 

assessment, and as such, is different than guidelines. He also noted that this process is similar to 

that used by other Agency efforts, most notably the ecological risk assessment guidelines. 

5
The Framework document is intended to provide a flexible structure, capable of evolving with 

experience, that captures the basic elements of cumulative risk assessment. It should serve as a 

basis for developing guidelines by defining key terms and concepts to promote a common 

language and furthering development of the approaches. He stressed that the Framework is not 

intended to present detailed technical guidance, but rather provide a general overview of issues, 

topics, and approaches that can be considered in conducting a cumulative risk assessment. 

Future guideline development efforts will build on the Framework, including case studies and 

issue papers on certain cumulative risk assessment topics. 

2.4 Presentation of the Draft Framework for Cumulative Risk Assessment 

Mike Callahan, from EPA Region 6 and Chair of the RAF’s Cumulative Risk Assessment 

Technical Panel, provided a more detailed presentation on the Framework document. He opened 

his talk by stating that the Framework is an information document, presenting what cumulative 

risk assessment is, rather than describing how to do it, which is the purpose of future guidelines. 

The Framework is intended to be a brief, 50 to 75 page document that introduces scientific 

concepts, presents definitions, and describes available approaches for cumulative risk 

assessment. It is not a protocol or a guideline on how to perform such an assessment. EPA will 

be developing guidelines over the next several years, building on the Framework and other 

subsequent efforts. Furthermore, cumulative risk assessment is not a replacement for traditional 

risk assessment techniques. Rather, it is an evolution in risk assessment that responds to the 

growing need to focus on populations or communities, rather than on sources of pollution. Mike 

Callahan pointed out that several EPA offices are already doing cumulative risk assessments, 

such as the Office of Pesticide Programs (under the Food Quality Protection Act) and the Office 

of Air (as part of the National Scale Air Toxics Assessment), the Dioxin Reassessment, and cityspecific assessments in Chicago and Baltimore. 

The Framework for Cumulative Risk Assessment presents information that can be considered in 

planning and performing such an assessment. Included in the document are working definitions 

and descriptions of the overall process, organized into the three main phases: (1) planning, 

6
scoping, and problem formulation, (2) analysis, and (3) risk characterization. Cumulative risk 

assessment is not appropriate for every task, but it is a tool available to help risk assessors and 

decision makers address situations involving cumulative risks (e.g., multiples chemicals or other 

stressors, multiple exposure routes, etc.). It is particularly applicable to community-based 

studies. It is also important to recognize what a cumulative risk assessment can and cannot do, 

so people have realistic expectations of the possible outcomes. This is something that groups 

performing assessments should discuss during the planning and scoping of a project, so all 

stakeholders come to a common understanding that a cumulative risk assessment can only 

answer certain types of questions. The science behind such efforts is still being developed, 

particularly to meet needs of the analysis and risk characterization phases. He acknowledged the 

scientific challenges in cumulative risk assessment related to integrating different stressors and 

types of risks, incorporating vulnerability, and evaluating uncertainty. The Framework describes 

methods that the Agency is aware of, recognizing the strengths and limitations of the approaches. 

He pointed out that there are qualitative aspects to such assessments, in addition to the 

quantitative analyses, so the results will be more meaningful to decision makers. Mike 

Callahan’s presentation introduced the definition of cumulative risk assessment and he described 

the many considerations that went into the definition, such as the language from the Food 

Quality Protection Act and usage from other Agency efforts. 

Mike Callahan concluded his presentation by reiterating that while EPA is already performing 

cumulative risk assessments in certain situations, more studies are needed to develop, test, and 

improve methods. The Framework should be completed in the fall of 2002 following revisions 

after the peer review and approval by the Science Policy Council. He again stated that policy 

issues will not be part of the Framework and are being addressed separately from the Technical 

Panel’s efforts. Mike Callahan described EPA’s plans for developing guidelines for cumulative 

risk assessment; the Agency is following a process similar to Ecological Risk Assessment 

Guidelines. This process includes preparing a series of case studies on cumulative risk 

assessments that have been performed, some of which are in initial stages of planning now. EPA 

will also prepare issue papers on topics that need additional consideration before finalizing the 

guidelines. Although EPA is already starting or planning these, the full process will require 

7
several years to complete, and guidelines are expected in 2005. 

2.5 Discussion on Opening Presentations 

During the opening presentations by Steve Knott and Mike Callahan, several peer reviewers 

asked questions about the draft Framework. Many of the issues raised were related to 

definitions, the need for the Framework to reflect information in the scientific literature, and 

challenges in performing cumulative risk assessments involving chemical/non-chemical 

“stressors” and other types of risks. Peter deFur noted that these topics would be the focus of 

discussion later in the workshop, in response to the charge questions. 

8
3.0 SUMMARY OF COMMENTS AND RECOMMENDATIONS 

The peer reviewers met over two days and evaluated EPA’s draft Framework for Cumulative 

Risk Assessment generally and in light of the charge questions. Most, but not all reviewers 

provided premeeting written comments in which they commented on the overall quality of the 

Framework and on the charge questions. During the two-day review meeting, the reviewers 

raised a number of important points that they wished to emphasize in the written report to EPA. 

The summary below begins with the major suggestions and recommendations for changes to the 

Framework, followed by responses to the charge questions. Charge questions #1 and 2 were 

addressed simultaneously and are presented together in Section 3.1. Subsequent subsections 

detail the responses on the remaining questions, followed by a list of topics that reviewers 

believed warrant further development by EPA, possibly in issue papers. 

The Chair and Versar staff who took notes consulted on these topics and agreed that these issues 

were the most important and encompassing issues raised during the two-day meeting. The 

suggested changes to the document presented in this report generally represent the opinion of 

multiple reviewers. Recommended revisions to the Framework were considered highly 

significant by the Chair because they were voiced by multiple reviewers or applied broadly to 

most of the document. The following list summarizes some of the most significant suggested 

changes to the document provided by the peer reviewers. 

1) EPA did a good job with the Framework and the reviewers generally found the document easy 

to read, well written, and organized. It was clear that the authors found it difficult to convey the 

issues and concepts related to cumulative risk assessment. 

In addition, the draft reflects comments made during the August 2001 peer consultation and 

other public meetings. The text boxes add something to the content without distracting from the 

flow of the text – make sure the boxes are in a readable font. 

9
2) Terminology must be correct and not change the use of a common term with an existing 

understood meaning. Reviewers found several places in which the Framework was unclear or 

created some ambiguity. The text should be consistent with the glossary; at present the two are 

not consistent. The glossary needs to be augmented and the number of entries greatly increased. 

3) EPA should confirm that the Framework is consistent with existing disciplinary uses of 

terminology, methods, references, data, and present state of the art. The reviewers realized that 

cumulative risk assessment demands the use of expertise and methods from other fields, 

including epidemiology, public health, toxicology, etc. This point was raised several times in the 

discussions of charge questions and in the overall evaluation of the Framework. EPA should use 

caution with terminology, to avoid revising the current use of terms and methods when they 

already have use and meaning in other disciplines. 

Reviewers discussed the definitions of cumulative risk and stressor, along with several other 

terms. The reviewers acknowledged that EPA should set an operational definition for 

cumulative risk assessment that is practical within the existing regulatory and assessment 

programs. Caution should be used with the term “stressor,” because there is already a field of 

experts using that term. 

4) The Framework should provide a window to the literature and the reviewers said that in many 

respects the document does accomplish this goal. However, the references cited by EPA and 

listed in several sections of the Framework are focused on EPA documents and products. EPA 

should provide citations to more than their own Agency documents, so the literature reflects a 

broad perspective and does not seem insular. The other area where the reviewers suggested that 

EPA broaden the literature cited pertains to ecological risk assessment, especially case studies. 

EPA is referred to their own work on watershed assessments (Waquoit Bay, Clinch River, etc.) 

and the peer-reviewed literature associated with those efforts. Other examples were brought up 

during the discussion, including Pacific Northwest forests, Green Bay, WI, Fox River, WI, 

Florida Everglades, Coastal Louisiana, and gypsy moths in eastern forests. 

10
5) EPA should continually answer the query – What is unique to cumulative risk assessment? 

Reviewers raised this point in premeeting comments, in discussion of the whole Framework, and 

in discussion of every section and charge question. The reviewers also frequently asked this 

question of one another. EPA was cautioned to not needlessly repeat text, material, or other 

information that is in existing EPA guidance on risk assessment, but rather to focus on the 

aspects of risk assessment that are unique or special in cumulative risk assessment. If this 

Framework is supposed to serve both human health and ecological assessments, then this effort 

is new, and if implemented in such a way, will be unique. 

6) Public health issues were raised in several contexts. Some reviewers said that public health 

needs more emphasis in the Framework. Others recommended that EPA use more public health 

data and emphasis in framing cumulative risk assessments for communities and populations. 

Public health data are already available to better inform such cumulative risk assessments, such 

as birth weight and biomarker data, which are natural integrators of multiple effects and have the 

potential to be used to assess the separate and combined influences of multiple stressors. 

7) Both qualitative and quantitative aspects of cumulative risk assessment are applicable in many 

if not all areas of the Framework and the document should clearly articulate this fact. The 

reviewers focused on the analysis phase and uncertainty analysis, but this point applies broadly. 

Important qualitative methods and approaches can and must be used in cumulative risk 

assessment. Some qualitative analyses should/could be done before initiating the quantitative 

procedures, particularly in the case of uncertainty analysis. 

8) The role of decision-makers and risk managers in the cumulative risk assessment process is 

important and should be spelled out in somewhat more detail, even for a framework document. 

The reviewers discussed the need for balance between the necessity to inform the process with 

management goals and objectives, and the requirement to prevent management from dictating 

11
the assessment outcome. In regard to cumulative risk uniquely, when these assessments focus on 

a community, a specific population, etc., these groups are the decision-makers and participate 

actively in risk management, thus defining and clarifying their role is key in the initial phases 

(planning, scoping, etc). 

9) Several reviewers raised issues of how this Framework document will fit into the process of 

developing guidelines for cumulative risk assessment. They wanted to know how EPA will deal 

with this document, develop the subsequent issue papers and case studies, and whether EPA will 

make this a living document. The introduction to the Framework seems to be the place to spell 

out to the reader what the next steps are, and how the process will proceed. EPA did address this 

issue on pages 5-6, where the Framework discussed this point, but reviewers felt that more 

information would be useful. Maybe EPA could insert a text box that includes the other major 

steps in the whole process, such as development of issue papers and case studies. 

10) Some aspects of the conduct of a cumulative risk assessment need to be highlighted in the 

initial steps of planning, scoping, and problem formulation. These aspects include population (or 

ecosystem) identification or selection, vulnerability, uncertainty analysis, data collection and 

analysis, statistical analysis, and statutory constraints. These steps are not unique to cumulative 

risk assessment, but are particularly important because the entire risk assessment depends on 

these first steps. Once the risk assessment has started, usually it is not possible to start over 

again with another modified process because of an error or omission. If a community partner has 

been omitted at the outset, this partner may be alienated and no longer trust the assessment no 

matter what transpires. If the statistical and uncertainty analysis are not considered when the 

data acquisition is planned, it may not be possible to conduct the necessary analyses. 

Raising certain issues at the outset also serves to highlight them to the risk assessment team and 

the interested and affected parties. 

12
11) The reviewers recommended addition of examples and references in some key parts of the 

Framework. The additions are intended to accomplish two purposes: (1) add ecological cases 

and perspectives in areas that are now exclusively human health discussions and (2) broaden the 

base, geographic range, or types of studies that are presented. Several examples served to 

illustrate the point. Cumulative risk assessment is most difficult for cases that apply to 

understudied and “hard to reach” populations, such as homeless individuals or some minority 

communities. These same individuals and communities may often be the most vulnerable or 

sensitive. The same trend is true for ecosystems and wildlife populations- Arctic and desert 

ecosystems are notoriously understudied and turn out to be fragile. 

12) The reviewers supported the use of indices or other methods to combine, analyze, and 

present data in a cumulative risk assessment. However, they had some concerns that data and 

qualitative information could be lost or otherwise obscured in moving from conceptual model to 

analysis phase, in analysis, in moving from analysis to characterization, in uncertainty or 

sensitivity analysis, etc. It is important to preserve data throughout the process of assessing risks 

in cumulative risk situations, in no small part because of the number and complexity of stresses 

and agents, endpoints and methods. For example, the method of creating a matrix was selected 

by some previous researchers because it did not collapse data to the extent that other methods 

did, yet there is still distillation of data into a more condensed form. The reviewers cautioned 

that cumulative risk assessment has inherently more information and data than assessments with 

single chemicals, single endpoints, etc. As users seek to collapse and simplify the presentation 

of the results of a cumulative risk assessment, data should not be lost, but preserved. 

13) EPA should acknowledge and incorporate more of the international efforts on cumulative 

risk assessment. Some of these international efforts are noted in the present version of the 

Framework, but more notation would be an important enhancement. Several reviewers pointed 

out that other countries have efforts specifically in cumulative risk assessment or related fields. 

EPA could/should do their best to note those, as in the box on page 63 that uses the European 

report on environmental cases and decisions. 

13
14) The Chair and Versar staff kept a running list of topics that might warrant further 

examination outside the context of the Framework, such as in issue papers. These topics, 

specifically as each applies to the process of conducting cumulative risk assessments, included: 

• Background conditions, especially compared with ambient, control etc. 

• Precautionary Principle 

• Uncertainty analysis 

• Vulnerability (defined and parsed) 

• Biomarkers of exposure and of effects 

• Single metrics in cumulative risk assessment (including combining chemical and nonchemical stressors) 

The reviewers also recommended that EPA prepare and compile materials that would enhance 

the field of cumulative risk assessment or promote further work in the area. Foremost among 

these was the suggestion that EPA post on a web site a database of literature on cumulative risk 

assessment case studies and methods. 

3.1 Charge Questions #1 and #2 

Question #1. Comment on whether the Framework adequately captures, describes, and 

reasonably organizes the key issues for cumulative risk assessment so as to serve as an adequate 

foundation for the development of future guidance. Does the Framework provide adequate 

coverage of: terminology, structure, and methods? 

Question #2. Does the Framework document include any scientific or technical inaccuracies in 

its presentation of terminology, assessment structure, and methods? Please identify any problem 

areas and propose revisions or other actions that will result in a scientifically sound and 

supportable discussion. With respect to methods, comment on whether the Framework 

adequately conveys the state of the science with respect to currently available cumulative risk 

assessment methods/approaches and the areas that are in need of further research and 

14
development. 

In general, most reviewers felt that EPA had done an admirable job at introducing the main 

topics for cumulative risk assessment and that the document had few errors and omissions. The 

Framework provides a broad, flexible assessment structure that is not only appropriate but 

required for the complex scientific and policy needs that call for cumulative risk assessment. 

The main improvements suggested by the reviewers were in: identifying and describing the 

issues that are unique for cumulative risk assessment, clarifying terminology, and providing the 

reader with a clearer sense of what cumulative risk assessment is (and isn’t) as well as when 

such assessments may be required. With respect to methods, reviewers recognized that 

cumulative risk assessment is a rapidly developing field that will benefit greatly from anticipated 

advancements in the supporting scientific fields. Much of the discussion on methods could be 

expanded and improved, particularly on biomarkers, biomonitoring, and drawing analogies 

between human health and ecological risk assessment. Because the Framework is applicable for 

both human health and ecological assessments, it should provide a better balance in the 

discussion of both topics, including examples. 

3.1.1 Terminology 

With respect to terminology, most reviewers felt that the Framework does a good job at 

introducing terms and concepts. The main suggestions were to make sure that terms are clearly 

defined, consistent with the glossary, and reflective of usage in the numerous disciplines related 

to cumulative risk assessment. Several reviewers recommended that the Framework present 

additional examples and illustrations that portray the terms in ways that help the reader to 

understand the relationships among cumulative risk assessment and “traditional” human health 

and ecological risk assessment. Additional references that might be highlighted for this purpose 

are the ILSI (1999) report, as well as international documents, to better reflect what other 

organizations are doing with respect to cumulative risk assessment. 

During the discussion, several terms were raised that were either unclear or their usage might 

15
cause confusion. Some of these terms included: 

• Background (includes: ambient, baseline, control, etc.) 

• Stressor (issues related to its use to encompass toxic chemicals, non-chemical agents, risk 

factors and other types of “stressors”) 

• Adverse 

• Cumulative vs. aggregate risk assessment 

• Mechanism and mode of action 

• Ecological (differentiating between usage in epidemiology and “natural resource” 

contexts) 

• Toxicology (in reference to human health vs. ecological effects) 

The reviewers provided suggestions on ways to improve and clarify use of these terms in the 

Framework. These recommendations included adding text boxes (to distinguish terms and 

provide examples), using the most specific term(s) possible, and making sure the glossary 

defines all terms that might be unclear (vulnerability, susceptibility, etc.). EPA was encouraged 

to look at how different scientific disciplines use these terms and to be aware of the challenges 

inherent in interdisciplinary fields such as cumulative risk assessment. 

3.1.2 Structure for Cumulative Risk Assessment 

On the issue of the Framework providing an adequate assessment structure, most reviewers 

voiced their opinions that the document is very well done and that the structure meets the needs 

for cumulative risk assessment. It was recognized by one reviewer that this structure diverges 

from the one shown in the SAB (2000) report Toward Integrated Environmental DecisionMaking, while another reviewer stated that it does follow the three steps used in the ecological 

risk assessment guidelines. Some discussion should be added to explain the rationale for the 

proposed Framework. Most reviewers supported the idea that having a broad, flexible structure 

is beneficial for responding to the many types of situations that might be encountered and 

anticipated in the future. This is true for EPA’s use of the document as well as the needs of 

16
stakeholder groups. 

Throughout the discussion, the need for clarification on what is unique about cumulative risk 

assessment was voiced many times by the reviewers. Specifically, reviewers suggested that the 

document call attention to the issues that are particularly different for cumulative risk assessment 

and provide references for the reader to consult on more general aspects of those topics (e.g., 

importance of stakeholder involvement, uncertainty analysis, etc.). 

The Framework document should also provide more examples, illustrations, and criteria for 

circumstances where cumulative risk assessment is appropriate/required. These examples should 

include figures that display how cumulative risk assessment differs from traditional risk 

assessments, but may interact with other efforts. It is important that readers and practitioners be 

able to identify the characteristics of a situation that might call for a cumulative risk assessment. 

Similarly, the document should also help stakeholders to understand what such an assessment 

can and cannot do. Reviewers also felt that the document needs to provide information on what 

guidelines will look like, which would help to differentiate the Framework from the guidelines, 

while providing a clearer picture of the “how to” aspects that should be included in the future 

guidelines. 

The focus on populations, identified in the Framework as one of the distinguishing 

characteristics of cumulative risk assessment, should receive even more emphasis (for both 

human health and ecological populations). Rewording of select sections can provide this 

emphasis and readers should also be reminded of the potential difficulties in characterizing 

subpopulations, particularly minority groups in urban areas, which are often undercounted in the 

Census. Similarly, ecological risk assessment uses the population as the focus for assessment. 

Therefore the ecological risk assessment guidelines should be reviewed for applicable text to be 

added in the context of identifying populations. Introducing the need to consider vulnerabilities 

of subpopulations early in the process was suggested by several reviewers. Vulnerability in 

subpopulations should be raised earlier as part of the planning, scoping, and problem formulation 

17
phase (possibly as Section 2.2.4). Similarly, in the analysis phase, mention should be made of 

the needs for data collection and use of existing data (e.g., public health statistics and exposure 

data). 

Several reviewers stated that cumulative risk assessment should be decision-driven, providing 

answers to questions posed by risk managers, communities, and other stakeholders. As such, the 

Framework should provide focus on the types of questions that decision makers need to answer, 

such as those about (1) general human health risks, (2) priority-setting, and (3) policy options. 

Similarly, the Framework should help risk assessors to respond to complex situations where the 

science calls for a cumulative risk analysis. Balancing these needs is difficult and reviewers had 

various suggestions, such as presentation of qualitative information before more quantitative 

analysis and results and planning for public involvement prior to beginning the process. One 

reviewer suggested that two cumulative risk assessment documents be prepared, one for 

stakeholders and another separate document for scientists. Reviewers generally agreed that the 

Framework should point out the benefits of transparency and retaining as much data as possible 

through the cumulative risk assessment phases. 

Finally, many reviewers agreed that the figures depicting the three phases of cumulative risk 

assessment need to be improved. First, figures should distinguish cumulative risk from other 

types of risk assessments. Also, the flow diagrams should be sure to show forward motion, 

while recognizing the need for feedback to accommodate the iterative nature of assessments. 

Suggestions were made to use large arrows going forward and smaller arrows backwards, 

representing feedback/iteration or the need to do screening prior to the actual cumulative risk 

assessment. The overall message to EPA was to be cautious to avoid two headed arrows, which 

might result in “paralysis by analysis.” 

18
3.1.3 Methods 

The reviewers acknowledged EPA’s efforts to introduce some of the major approaches used for 

cumulative risk assessment, recognizing that the Framework would not be the place for 

extensive description of the applicable methods. Many reviewers felt that the document could be 

improved with respect to information on biomarkers, biomonitoring, mixtures risk assessment, 

and alternative methods to address the need to integrate chemical, non-chemical, and other types 

of risks. Suggestions were also made by several reviewers to improve the figures (particularly 

Figure 1-2) to better depict the differences between approaches used in cumulative risk 

assessment and the more linear portrayal of traditional risk assessments. Reviewers felt that the 

Framework can encourage development of alternative methods to improve the cumulative risk 

assessment process. 

The discussion on methods began with several reviewers distinguishing between methods and 

approaches. Approaches are seen to be more general, while methods are more specific. These 

reviewers felt that what the Framework provides are introductions to approaches and that the 

terminology should be distinguished. One reviewer raised the potential to use either “top down” 

or “bottom up” approaches for cumulative risk assessment. The Framework tends to include 

more bottom up examples, such as the use of relative potency factors (RPFs) or toxicity 

equivalency factors (TEFs) to “accumulate” risks from chemical mixtures. Some of the 

strengths and weaknesses of these approaches were reviewed. Top down approaches that 

estimate overall risks and then apportion them among populations, according to different 

measures, may also be useful for cumulative risk assessments. 

Several reviewers advocated the utility of biomarkers in the context of cumulative risk 

assessment and they thought the presentation in the document should be enhanced and expanded, 

with caveats on what they can and cannot do. Discussion on biomarkers focused on their 

potential use as time-based integrators of exposure and effects, which could be used to provide 

an overall picture of environmental health in a community. Selection of biomarkers was 

recognized to be a complicated issue, as there are limitations in their ability to identify specific 

19
exposures or effects. Furthermore, some biomarkers can require invasive studies that call for 

individuals to consent to providing samples, which could lead to environmental justice issues. In 

other cases (such as birth weights and other aspects of birth records) the data are routinely 

collected but precautions need to be taken to prevent inappropriate disclosure of data on specific 

individuals. This topic migrated into a more general discussion of biomonitoring, which, like 

biomarkers, can be very informative. Some reviewers recommended that the Framework discuss 

these issues and provide additional references on existing biomarker and biomonitoring efforts 

and data (e.g., CDC National Exposure Project and NIEHS studies) that could be used in 

cumulative risk assessments. Other valuable public health surveillance data sources include state 

registries on cancer, birth defects, and other diseases. Similarly, other sources of human health 

baseline data, such as Healthy People 2010, can be used in general ways for cumulative risk 

assessments. Reviewers pointed out that the fields of biomarkers and molecular epidemiology 

will be producing many more tools that can be used in the future. The desire to link science on 

the public health side with the ecological side will give the Framework more weight, which will 

result in it being used more extensively. Furthermore, the Framework can help to identify the 

needs and stimulate more research to develop these types of methods. 

Methods for addressing multiple chemical, non-chemical, and other stressors are widely 

recognized as one of the most significant challenges for cumulative risk assessment. 

Aggregating and integrating risks from these disparate types of stressors to human health is not 

as advanced as analogous techniques used for ecosystem-level ecological risk assessments. The 

Framework introduces some methods, such a using a matrix to array data and facilitate 

comparing and contrasting different types of stressors or risks, but the document should try to 

expand the envelope in this area. EPA is encouraged to consult the ecological risk assessment 

literature on single metrics, indices, and other measures of ecological health (e.g., IBI, EMAP 

benthic index, USDA APHIS approach, etc) to determine their applicability to human health 

assessments. A particularly good resource on these issues is the SETAC Press 1999 document 

Multiple Stressors in Ecological Risk Assessment. 

20
3.2 Charge Question #3 

Question #3. Comment on whether the Framework adequately characterizes the importance of 

uncertainty analysis in cumulative risk assessment. What additional discussions of uncertainty 

should be included in the Framework (and in what sections of the document)? 

The reviewers acknowledged that uncertainty analysis, a complicated issue, is even more 

complex in a cumulative risk assessment. The Framework does a good job of conveying the 

general concepts and issues of uncertainty analysis, however, the reviewers suggested that EPA 

focus on those elements that are unique to cumulative risk assessment, rather than try to provide 

an overview on the subject. Uncertainty analysis should be used not only in the risk 

characterization phase, but also as part of the planning of an assessment, to guide data collection 

(value of information analysis), scoping of a study, and communicating to stakeholders the 

possible outcomes of an assessment. For example, use of sensitivity analysis in the early phases 

can be helpful in planning a cumulative assessment. However, it is the communication aspects 

that reviewers felt would be most critical to a cumulative risk assessment and they emphasized 

the importance of communicating both qualitative and quantitative information on the 

confidence of an assessment. These communication challenges exist because uncertainty 

analysis tends to be very complex and quantitative, using language that is not easily conveyed to 

other audiences, which may result in conceptual and language barriers and miscommunication. 

As a result, it is important that qualitative uncertainty analyses be given emphasis in a 

cumulative risk assessment document, which will be useful to decision makers and other 

stakeholders. This type of open and honest communication should clearly state not only the 

uncertainties, but also the unknowns. 

Discussion on uncertainty addressed one of the unique aspects of a cumulative risk assessment, 

evaluating chemical mixtures. One reviewer felt that the Framework could be enhanced by the 

addition of text describing some of the uncertainties that result from approaches for chemical 

mixtures risk assessment, particularly (1) dose-response relationships, (2) relative potency, and 

(3) considering biological activity from other modes of action. This discussion highlighted the 

21
need for the Framework to introduce quantitative techniques to analyze uncertainty resulting 

from these mixtures assessment techniques (e.g., relative potency factors, toxicity equivalency 

factors, etc.). Several reviewers noted that many existing cumulative risk assessments assume 

that chemicals can be grouped together based on the assumption of common modes of action, but 

they pointed out that this assumption does not account for all toxic effects. Similarly, statements 

should be made in the narrative that certain approaches can over- or under-estimate the toxicity 

of actual mixtures and that the different approaches make basic assumptions relative to 

additivity, synergy, and other effects. 

Other suggestions on uncertainty analysis were provided by the reviewers, including the need to 

differentiate between uncertainty and variability (adding definitions in the glossary would be a 

good start) and discussing some of the major tools available to risk assessors. Several reviewers 

stated that the Framework would benefit from introduction of Bayesian statistics tools, but others 

cautioned against overconfidence with such tools. Discussion returned to the communication 

challenges and the recognition that the Framework should not be an uncertainty analysis primer. 

As such, reviewers noted that the Framework should point readers toward additional references 

(e.g., the SETAC document Uncertainty Analysis in Ecological Risk Assessment by WarrenHicks and Moore) for more background and explanatory material on the subject. Additionally, 

reviewers believed that uncertainty analysis is a topic that may be a good candidate for an issue 

paper, where expanded discussion could be presented on the advantages/disadvantages of 

available tools, the balance between qualitative and quantitative presentation, and related topics 

that warrant further development specifically for cumulative risk assessment. 

Finally, a few reviewers acknowledged that even though cumulative risk assessments may have 

large uncertainties, they can be powerful tools for reporting on general environmental health, 

ranking risks for priority setting, and making decisions affecting public health. In cumulative 

risk assessment, a concerted effort should be made to gather and analyze large amounts of data; 

however, it is critical that such assessments do not get “bogged down” because uncertainties 

cannot get resolved. Reference was made to the precautionary principle and the need for an 

adequate margin of safety to be provided, while avoiding long delays to determine cause and 

22
effect and analyze the uncertainties. The precautionary principle, placed in the context of 

cumulative risk assessment, may be a suitable topic for an issue paper. Part of this discussion 

referred back to general public health principles as well as the need to build the capacity among 

stakeholders to consider uncertainty issues as parts of the analysis and decision making 

processes. 

3.3 Charge Question #4 

Question #4. Comment on the adequacy and accuracy of the Framework’s presentation in each 

of these areas: vulnerability, chemical vs non-chemical stressors, and different types of risk. 

The comments and recommendations in response to charge question #4 were provided according 

to the three subparts below. 

3.3.1 Vulnerability 

The issue of vulnerability is critical in cumulative risk assessment and the reviewers were 

pleased that EPA had provided extensive attention to the topic. Reviewers discussed at length 

the following: possible alternate structures/terminology for classifying vulnerability; expanding 

the discussion with additional examples from public health, medical, and ecological fields; and 

the need for vulnerability considerations to be introduced earlier in the document, in the planning 

and scoping phase as part of identifying populations. Like other topics, reviewers suggested that 

the Framework focus the vulnerability discussion on those aspects that are unique to cumulative 

risk assessment. Furthermore, vulnerability is a topic that should receive increased attention, 

possibly via an issue paper. In addition to the issues of classifying types of vulnerability, 

reviewers pointed out the difficulty in characterizing vulnerable populations. In both human 

23
health and ecological contexts, there are “hard to reach” populations that are understudied, and 

therefore, difficult to account for in cumulative risk assessments. As such, it was recognized that 

vulnerability is both a technical and policy challenge for cumulative risk assessment. 

Several reviewers raised questions about the four categories of vulnerability presented in the 

Framework, particularly with respect to terminology and being able to differentiate among the 

types. Reviewers also introduced possible alternate structures for considering vulnerability 

issues. One such structure would consist of two major categories (1) differential susceptibility 

and (2) differential exposure. The differential susceptibility category could encompass issues 

related to ability to withstand initial exposure as well as the ability to recover from effects. 

Another approach is based on dependencies where susceptibility is viewed as the difference in 

the dose required to produce the same effect, essentially a shift in the starting position on the 

dose-response curve. A few reviewers felt that this approach might be less confusing and easier 

to communicate, though the reviewers did not resolve many of the issues introduced. 

Subsequent clarification by EPA indicated that the reviewers’ suggestions were very close to the 

existing scheme presented in the draft Framework. 

Discussion also focused on existing vulnerability terminology used in other disciplines, such as 

those in public health, medicine, and ecological risk assessment fields. It was suggested that 

EPA consult the literature from those (and other) fields to see how vulnerability and related 

terms (e.g., susceptibility, sensitivity, resistance, resilience, dependency, etc.) are used. 

Examples could be added to the Framework to illustrate the similarities and differences. 

However, one reviewer suggested waiting on a vulnerability definition as the field of 

vulnerability is still developing. These conceptual definitions may not be helpful once better 

methods for vulnerability assessment become available. Regardless of the terminology that EPA 

chooses to use, it is important that the way that vulnerability is factored into an assessment be 

transparent and involve stakeholders in the process. 

24
In the context of classifying vulnerability and related issues, reviewers pointed out that one of 

the biggest challenges is in characterizing vulnerabilities of populations, in both human health 

and ecological contexts. One reviewer stated that particularly in urban areas the most vulnerable 

populations may be under represented in census counts and therefore may not be adequately 

addressed in an assessment. This situation is analogous to remote ecosystems (e.g., arctic and 

desert resources) which are not extensively studied. As such, some of the most vulnerable 

populations may be among the ones that are most unknown. This becomes an issue on a 

site-specific analysis where the variability range may become statistically indefensible. The 

Framework’s discussion of differences in race and gender vulnerabilities should be revisited 

because there are references that show that these populations do have differential exposures as 

well as different responses to the same exposures. While most reviewers recognized the 

potential for increased risks among certain populations, some examples were provided where 

select subpopulations often exhibit lower risks from similar exposures, such as the “healthy 

worker.” A related issue was raised by one reviewer that concerns risk perception and the 

different areas of concern of different races and genders, which is important as these perceptions 

can be "drivers" for community participation. The literature (e.g., paper by Paul Slovic) contains 

more information on risk perception factors. These vulnerability considerations should be 

recognized in the scoping process and also in communicating the results of a cumulative risk 

assessment. 

3.3.2 Chemical and Non-Chemical Stressors 

Reviewers stated that EPA deserves credit for the narrative in the current draft of the Framework 

because the issue of combining chemical and non-chemical stressors is one of the most 

challenging aspects of cumulative risk assessment. Several suggestions, as in previous 

discussions, were made for EPA to acknowledge the literature from other countries (e.g., 

Canada, Europe) for information on how they deal with integrating multiple exposures of 

disparate types of stressors. Similarly, public health approaches can be used to evaluate the 

impacts of chemical and non-chemical stressors, though there are limitations in their utility. 

Furthermore, the ecological risk assessment literature should also be consulted for analogues that 

25
might be applicable to human health approaches. It was noted that there is a rich summary of 

what is currently out there; however, discussion should be expanded to figure out what can be 

defined with qualitative and quantitative techniques. As with other complex technical issues in 

cumulative risk assessment, this topic may be worthy of additional development in an issue 

paper. 

Suggestions provided by several reviewers were to use a “top down” approach that relies on 

public health approaches, such as ecological epidemiology, to help to examine risks from 

chemical and non-chemical stressors. Such approaches, advocated by some of the reviewers, use 

biomarkers or other biomonitoring data (e.g., birth weight) as indicators that integrate the effects 

of different exposures, and which can also be used to predict future health outcomes. One 

reviewer, however, voiced reservations that ecological epidemiology studies would not be very 

useful because they will not identify specific causes of problems, they are time and resource 

intensive, and often raise more questions than they answer. Several reviewers felt that EPA 

should expand the discussion of ecological epidemiology to inform the reader of the possible 

advantages and disadvantages of such an approach. Also, a definition for ecological 

epidemiology should be added to the document and glossary (also distinguishing use of the term 

ecological in the context of epidemiology from the more widespread use). The issue of 

interactions (synergy and antagonism) was raised in this context such that approaches to 

combine different types of stressors should be done so with the recognition of such interactions. 

Several reviewers felt that the Framework should clearly state the default assumptions on 

combining stressors relative to additivity and related interactions, as a point of comparison. This 

discussion concluded with the recognition that these types of public health approaches have 

limitations, but can be helpful in developing hypotheses for future focused studies. This is 

similar to approaches used in ecological risk assessment and wildlife experimental science, 

where survey data can identify ecosystem level effects, but not causal relationships. Such 

information is then used to develop hypotheses for more intense monitoring and research, which 

are focused on isolating particular combinations of stressors and responses in resource health. 

Similar to the discussion of vulnerability, one reviewer emphasized the importance of social 

26
sciences in evaluating the significance of non-chemical stressors. Sociocultural aspects, such as 

limited access to health care, risk perception, and other risk factors, should be recognized by risk 

assessors in this context. Several papers are available in the literature on approaches to consider 

these factors in qualitative analyses. Another reviewer revisited one of the major themes from 

the workshop; the need for the Framework to provide more references and possibly a “document 

clearinghouse” for readers to consult for more background information on assessment 

approaches and methods. Such a central repository (or web site) of information on cumulative 

risk assessment will not only help practitioners, but it may promote further research to improve 

available methods. 

3.3.3 Different “Types” of Risks 

Discussion among the reviewers on approaches for assessing different types of risks was similar 

to that on previous topics; EPA had done a good job introducing the subject and providing 

available information, but more could be added to improve the Framework. Reviewers 

recognized that this is among the most complex aspects of a cumulative risk assessment, where 

the goal is to evaluate disparate types of exposures and effects. Suggestions for improving the 

document focused on issues that were similar to those on previous challenging topics, such as the 

importance of acknowledging the literature, the need to retain data and transparency when 

aggregating data, and the utility of ecological risk assessment approaches as possible models for 

human health assessments of different types of risks. This topic, particularly the use of single 

metrics for cumulative risk assessment, might be another good candidate for an issue paper. 

Reviewers discussed approaches such as comparative risk ranking, matrix-type approaches, and 

single metrics, recognizing that all of these options diverge from purely scientific analyses into 

areas of policy, risk management, sociology, economics, and related subjects that rely on 

judgments to be made on the relative weight to be placed on different types of risks. The 

“translation of disparate types of risks into different units” was seen as the core of these 

approaches, with the clearly stated drawbacks that judgments have to be made and information 

can be lost in the process. Reviewers cited the QALY and DALY discussion presented in the 

27
Framework as examples where health effects and other types of impacts (e.g., economics) are 

combined into metrics, with recognized limitations. One reviewer voiced the opinion that 

indices based on economic values or quality of life are useful because communities can relate to 

such topics, often better than environmental indicators. 

Comparative or relative risk assessment was seen by several reviewers as one of the more useful 

tools in the context of cumulative risk assessment, where communities are often trying to 

prioritize risk reduction efforts. Such risk ranking exercises have been conducted for years in 

both human health and ecological risk assessment fields, particularly by the Europeans. The 

difficulty of weighing different endpoints was pointed out as a possible drawback, but reviewers 

felt that tradeoffs are often made in these types of risk ranking procedures. One example was 

provided by a reviewer which had been developed by USDA to evaluate exotic species using a 

high, medium, low scale (see OSTP document Ecological Risk Assessment in the Federal 

Government). Other examples of approaches used in ecological assessments, such as energy 

flow measures, trophic level indices, and other diversity indices, are used in similar ways, but 

with limitations. While reviewers generally felt that such approaches can be useful, they also 

acknowledged the struggle with the types of judgment calls to be made. Several reviewers also 

reiterated the need to retain data (present the disaggregated data too) and provide transparency in 

the process (such as for future re-analysis), as is advised in the draft Framework. 

Reviewers voiced the opinion that the Framework should remain neutral and not advocate any 

particular approach for addressing different types of risks. Rather, the document should present 

available options and criteria for use, and let the risk assessors and stakeholders decide on the 

method that is most appropriate for the particular situation. This discussion again raised the 

importance of involving stakeholders in these types of decisions. Community participation is 

important in considering socioeconomic and sociocultural options and making the types of 

choices involved in deriving or using a metric. The priorities and values of stakeholders should 

be incorporated into the metric(s), which will increase understanding of the metric and 

acceptance of the results. Reviewers recognized overall that in the future, there are scientific 

improvements to be made as well as in the other factors, that should be considered as these 

28
approaches are applied in cumulative risk assessments. 

3.4 Summary List of Suggested Topics for Issue Papers 

During the course of the peer review workshop, reviewers raised issues for discussion, several of 

which are particularly challenging topics in the context of cumulative risk assessment. These 

topics may warrant more attention, possibly in issue papers. Below is a summary list of those 

topics that the reviewers felt were worthy of additional development: 

• Background - issues related to the use of the term and distinguishing among background, 

ambient, baseline, etc. while recognizing the discipline-based meanings. 

• Vulnerability 

• Uncertainty 

• Precautionary Principle 

• Biomarkers and Biomonitoring 

• Single Metrics in Cumulative Risk Assessment (including combining chemical and nonchemical stressors and decision indices) 

29
4.0 DOCUMENT REVIEW 

For the last part of the workshop, the Chair opened the discussion for a section-by-section review 

of the document, to address any issues that had not been raised in either individual written 

comments or during the response to charge questions. He asked the experts to address major 

issues, such as reorganizing sections or clarifying the presentation of technical content, rather 

than providing editorial comments. The lists below summarize the comments and suggestions 

made by individual reviewers during this final document review, organized by the section of the 

Framework document. 

4.1 Section 1 

Incorporate more text in Section 1 on what cumulative risk assessment is as well as what it can 

and cannot do (much like EPA’s introductory presentations). Address issues with definitions 

and consider adding more examples to illustrate definitions. Add a figure that illustrates where 

cumulative risk assessment fits into the larger picture and what inputs are provided to the process 

from other exposure and risk assessment efforts. 

Discuss EPA’s statutory constraints and issues that may need to be overcome when conducting a 

cumulative risk assessment (e.g., integrating across programs and agencies). That discussion 

could be presented in Section 1 or in the planning and scoping phase. It is also important to note 

that despite EPA’s limitations that may result from statutory requirements, communities may 

want to use this approach. 

The examples on environmental justice issues, presented on page 3 (line 23-39), are outdated and 

could be brought up to date. 

Figures 1-1 through 1-3 should be revised and improved. Make Figures 1-1 and 1-2 different, 

perhaps changing Figure 1-1 to a more linear presentation. 

30
The text on page 9, lines 16-24 should be reworded to put the emphasis on populations as the 

starting point for many cumulative risk assessments. This is similar to ecological risk 

assessments, which are often triggered by an impacted resource or the desire to protect a 

resource of value. 

Add a text box on the scientific disciplines that are relevant to cumulative risk assessment and 

how they interact in the process. 

Make sure the glossary definitions are consistent with the text, as well as with accepted usage in 

fields of human health and ecological risk assessment. 

Expand the discussion of when a cumulative risk assessment should be done, on page 11. The 

existing text is a good start, but it could be elaborated upon. Possibly discuss post September 11 

issues as they pertain to cumulative risk assessment. 

4.2 Section 2 

Text could be added discussing the influence of land use decisions and the need for their 

consideration in the planning and scoping phase. 

Add some introductory discussion on consideration of vulnerability and uncertainty in the 

planning and scoping phase. 

Figure 2-1 should show a forward arrow through the phases. 

Wording in the paragraph at the bottom of page 16 on balancing stakeholder participation should 

be checked to make sure the meaning is clear. Several reviewers agreed with the intent of the 

31
text, but had slightly different interpretations. A suggestion was made for EPA to consult 

discussion from the 1996 “Understanding Risk” report by the NRC, which emphasizes that 

every decision does not require the same type and degree of public participation. 

As stated in comments on Section 1, the focus on populations should be emphasized in the 

planning and scoping phase. A specific example was noted, on page 17, line 39, where the 

sentence could be reworded to put identifying the population as the first step in the process. In 

addition, this discussion should be expanded and broadened to emphasize not only human 

populations but also ecosystems, as starting points for planning a cumulative risk assessment 

(reference back to Figure 1-2). 

Text could be added to put community involvement into the larger perspective of the American 

democratic process. 

Some text could be added to Section 2.1 in planning and scoping about risk cascades (e.g., echo 

cancer) and solutions that cause another risk. 

Discussion could be added on data collection and use of existing data, as part of the planning and 

scoping phase. Section 2.1.4 may be a place to present that text. 

Mention the need to retain information in development of conceptual models so that information 

is not lost (transparency) in the move to the analysis phase (on page 26-27, Sections 2.2.3 to 2.3). 

Add more examples on page 27-28 of cumulative risk assessments that have been performed, 

preferably with a broader geographic range, such as examples of projects from the west coast. 

Also, add a text box describing the Baltimore study, similar to the one for Chicago. Clarify the 

discussion of the importance of stakeholder involvement in Chicago, even if the project ended up 

not taking a cumulative risk assessment approach. 

Consider having two documents for cumulative risk assessments, a document that is oriented for 

32
community stakeholders and a technical assessment report. This might also be helpful for 

decision makers who can consult both resources. 

4.3 Section 3 

Section 3 could benefit from a slight reorganization to make the discussion clearer, which would

include renaming section headers and improving transition sentences. The suggested

reorganization was as follows:

3.0 Analysis Phase

3.1 General Process

3.2 Available Methods and Approaches

3.2.1 Estimating Exposures (with a sentence linking to traditional risk assessment terms)

3.2.1.1 Exposure Issues in Cumulative Risk Assessment (move part of vulnerability discussion)

3.2.2 Enhancing the Dose-Response Step (cause and effect)

3.2.3 Decision Indices

One reaction to the above reorganization stated that 3.2.1.1 might not be needed and could be

presented as a text box. Subsequent discussion questioned having decision indices appear in the

analysis section and a suggestion was made that it would be more appropriate to present that in

the risk characterization section. A suggestion made was to change the wording from decision to

synthesis. Further discussion did not resolve the issue on where this material should appear, but

comments were added that it would be desirable to have qualitative information presented before 

the quantitative results from the analysis phase.

Consider the references that are called out in the document and the need to provide a “window to

the literature” to help the reader to obtain key background information on topics related to

cumulative risk assessment (uncertainty analysis being just one example). Emphasis may be

placed on documents published by the NRC or similar groups, which carry influence in the

scientific community. Consider having a list or text boxes that list the most relevant and useful

33
references, organized by topic. 

The sentence on page 35, line 25 was questioned as to its accuracy and a comment was made that 

published data are available on different susceptibilities and sensitivities among races and 

genders. A suggestion was made to strike the beginning of the sentence and add references to 

back up this point. 

A reviewer questioned the accuracy of the statement on page 43, line 26 about “dose addition” 

being the default assumption for mixtures risk assessment. 

A recommendation was made to add a text box on page 50 describing comparative risk 

assessment and possibly providing some examples. 

4.4 Section 4 

On page 57 consider having a different title for the text box. 

In Section 4.2.3 on page 59 consider rewording the paragraph and adding references on the 

caution that should be taken in combining risks and the possibility for underestimation as well as 

overestimation. Several examples were provided, recognizing the frequent overestimation of 

risks, but others pointed out that risk assessments also underestimate risks because of unknown 

contributors. 

Make sure the document states somewhere that while all stakeholder constituents need to be 

heard in the process, “they don’t necessarily get their way.” 

Section 4.3.2 (page 61) the use of public health statistics should be expanded with references and 

possibly reworded to better present the relationships between health endpoints and what answers 

cumulative risk assessment can provide. Specifically, the text on page 61, lines 37-39, could be 

reworded as: Cumulative risk assessments are unlikely to match exactly with community health 

34
statistics, but they should be considered together where possible as a reality check for the risk 

estimates. 

In Section 4.4 on page 62, there are some terms that are not presented that would be expected to 

appear in the discussion on using the results of a risk assessment, such as: risk management, 

public health protection, cleanup, mitigation, siting, etc. Two other issues that could be 

discussed in this context are communication of the results of the assessment and the need for 

long term monitoring and follow up. In addition to these uses of the assessment, communities 

can also negotiate for mitigation options as part of projects that move forward. 

Several reviewers discussed the issue of cause and effect and the ability for cumulative risk 

assessment to answer questions about public health (causes of diseases, cancer clusters, etc). 

The document should mention that while cumulative risk assessment may not be able to identify 

causal links, it can help to yield useful results that can be used for decision making and 

informing communities. 

35
Appendix A

List of Peer Reviewers

A-1
Technical Peer Review Workshop

On The EPA Risk Assessment Forum

Draft Framework For Cumulative Risk Assessment

June 4-5, 2002 

PEER REVIEWERS 

Charnley, Gail HealthRisk Strategies 

Collin, Robert University of Oregon 

deFur, Peter Environmental Stewardship Concepts 

Goldsmith, David George Washington University 

Hattis, Dale Clark University 

Held, Joann New Jersey Department of Environmental Protection 

Locke, Paul Johns Hopkins University 

MacDonell, Margaret Argonne National Laboratory 

Muller, Pavel ToxProbe, Inc. 

Ryan, Barry Emory University 

Sass, Jennifer Natural Resources Defense Council 

Schlosser, Paul CIIT Centers for Health Research 

Travis, Curtis Quest Technologies 

A-2
Appendix B

List of Observers

B-1
Technical Peer Review Workshop

On The EPA Risk Assessment Forum

Draft Framework For Cumulative Risk Assessment

June 4-5, 2002 

OBSERVER ATTENDEES 

Bender, Ed U.S. EPA - Office of Science Policy 

Boa, Elizabeth American Chemistry Council 

Bober, Timothy U.S. General Accounting Office 

Byrd, Daniel LSRO 

Callahan, Michael U.S. EPA - Region VI 

Crawford, David U.S. EPA - OERR 

Dellarco, Vicki U.S. EPA - OPP 

Halper, Marty U.S. EPA - Office of Environmental Justice 

Hoddinott, Keith U.S. Army 

Hofmann, Lee U.S. EPA - OERR 

King, Marva U.S. EPA - Office of Environmental Justice 

Knott, Steve U.S. EPA - Risk Assessment Forum 

Lee, Charles U.S. EPA - Office of Environmental Justice 

Levin, Leonard EPRI 

Margosches, Elizabeth U.S. EPA 

McBride, Alexander U.S. EPA - Office of Solid Wastes 

Murphy, Deirdre U.S. EPA - Office of Air Quality Planning & Standards 

Putzrath, Resha Georgetown Risk Group 

Semeiks, Ilga U.S. General Accounting Office 

Utterback, Dennis U.S. EPA - ORD 

Zahodikin, Phil Pesticide and Toxic Chemical News 

Appendix C 

B-2 
Charge Questions 

C-1
 The Risk Assessment Forum

Draft Framework for Cumulative Risk Assessment

Peer Review Charge 

June 4 and 5, 2002 

The following background and questions are provided to help guide the peer review of 

the draft EPA Framework for Cumulative Risk assessment. The peer review is intended to 

provide input on the technical issues associated with cumulative risk assessment and how to 

capture these issues in a broad, flexible framework that will inform the development of future 

guidance. The focus of the peer review discussions will be on technical issues. Regulatory 

policy issues and specific program management concerns will not be addressed through this 

review. 

Background 

Several recent reports have highlighted the importance of understanding the 

accumulation of risks from multiple environmental stressors via multiple exposure pathways. 

These include the National Research Council’s (NRC) 1994 report Science and Judgment in Risk 

Assessment and the 1997 report by the Presidential/Congressional Commission on Risk 

Assessment and Risk Management entitled Risk Assessment and Risk Management in Regulatory 

Decision-Making. In addition, recent legislation, such as the Food Quality Protection Act of 

1996 (FQPA), directed the Environmental Protection Agency (EPA) to move beyond single 

chemical assessments and to focus, in part, on the cumulative effects of chemical exposures. 

In response to the increasing focus on cumulative risk, several EPA programs have begun 

to explore cumulative approaches to risk assessment. In 1997, The EPA Science Policy Council 

issued a guidance on planning and scoping for cumulative risk assessments. More recently, the 

Office of Pesticide Programs has developed cumulative risk assessment guidance focused on 

implementing certain provisions of FQPA. The Office of Air Quality Planning and Standards 

has recently assessed inhalation health risks associated with cumulative exposure to air toxics in 

its National Scale Assessment. 

The EPA Science Policy Council has asked the Risk Assessment Forum (RAF) to begin 

developing Agency-wide cumulative risk assessment guidance that builds from these ongoing 

activities. The RAF is a standing committee of EPA senior scientists established to promote 

Agency-wide consensus on difficult and controversial risk assessment issues and to ensure that 

this consensus is incorporated into appropriate Agency risk assessment guidance. As a first step, 

a technical panel convened under the RAF has been working to develop a Framework for 

Cumulative Risk Assessment. Building from the Agency’s growing experiences, the framework 

is intended to identify the basic elements of the cumulative risk assessment process. It should 

provide a flexible structure for the technical issues and define key terms associated with 

cumulative risk assessment. Earlier drafts of the framework were presented during meetings 

C-2
with other Federal and State scientists and during a peer consultation workshop with experts 

representing environmental and community groups, academia, and industry. The framework was 

revised based, in part, on discussions during these meetings. A new draft of the Framework for 

Cumulative Risk Assessment is being made available at this time for the purpose of peer review. 

At the completion of the peer review process, the document will be revised and then finalized for 

publication. The final framework document will reflect the peer review comments and will 

require review and approval by the Agency’s Science Policy Council. 

Charge Questions 

The following questions are provided to help guide the peer review and associated 

discussions during the peer review workshop. When considering these topics, keep in mind that 

the purpose of a framework is to identify and “frame” key issues for a broad audience of readers. 

Therefore a balance must be struck between adequately characterizing the issues and providing a 

detailed, comprehensive technical discussion. 

1. Comment on whether the Framework adequately captures, describes, and reasonably 

organizes the key issues for cumulative risk assessment so as to serve as an adequate 

foundation for the development of future guidance. In your comments, please address 

each of the following questions. In answering each question, provide a supporting 

discussion that highlights any areas of the Framework that may need to be clarified and 

relevant topics that may be missing from the current Framework document. Include 

references to any published literature that could help improve the completeness and 

clarity of the Framework. 

a) Does the Framework document capture the relevant terminology? 

b) Does the Framework document provide an adequate assessment structure? 

c) Does the Framework document outline the relevant methods for cumulative risk? 

2. Does the Framework document include any scientific or technical inaccuracies in its 

presentation of terminology, assessment structure, and methods? Please identify any 

problem areas and propose revisions or other actions that will result in a scientifically 

sound and supportable discussion. With respect to methods, comment on whether the 

Framework adequately conveys the state of the science with respect to currently available 

cumulative risk assessment methods/approaches and the areas that are in need of further 

research and development. 

3. Uncertainty analysis is an important aspect of risk assessment (and policy analysis in 

general). However, historically, dealing with uncertainty has been a shortcoming of 

many assessments. Cumulative risk assessments present new challenges for uncertainty 

analysis. For example, assessing cumulative risks will involve combining data of varying 

quality. Perhaps more important, assessing cumulative risks will involve the use of 

C-3
“soft” assumptions. These are assumptions which may have a high degree of uncertainty 

that is difficult (or not possible) to quantify. Comment on whether the Framework 

adequately characterizes the importance of uncertainty analysis in cumulative risk 

assessment. What additional discussions of uncertainty should be included in the 

Framework (and in what sections of the document)? 

4. The following topics have been identified by the RAF technical panel as technically 

difficult areas that will pose challenges to cumulative risk assessment. Comment on the 

adequacy and accuracy of the Framework’s presentation in each of these areas. 

a. Vulnerability 

As applied to cumulative risk assessment, it is useful to think of four components to 

vulnerability: the susceptibility or sensitivity of the human or ecological receptors; the 

differential exposures of the receptors; the differential preparedness of the receptor to 

withstand the insult from exposure; and the differential ability to recover from the effects. 

The issue for cumulative risk assessment is how to consider these aspects of vulnerability 

and their potential impacts on risk. This is highlighted in the Framework as an issue in 

need of further research and development. Comment on the discussion of vulnerability in 

the draft Framework. Has the state of the science been captured in this discussion? How 

can the discussion of this issue be improved? 

b. Cumulative Risk Assessment Involving Chemical and non-Chemical Stressors 

Viewing cumulative risk assessment as an evaluation of the accumulation of stressors 

presents many challenges. These may be seen when attempting to combine, in some 

meaningful way, the risks from multiple chemicals that may act as synergistic, 

antagonistic, or additive doses leading to a single effect. The situation is exacerbated 

when non-chemical stressors (e.g., radiation, biological agents, and psychological stress) 

are considered. Comment on the Framework’s discussion concerning the combining of 

disparate environmental stressors. In commenting, consider the state of the science with 

respect to understanding the effects of different stressors acting together (e.g., chemical 

exposure and viral infection). What can be added to the Framework to adequately 

convey the state of the science in this area? 

c. Cumulative Risk Assessment Involving Different “Types” of Risk 

Conveying the combined risks from multiple chemical and non-chemical stressors, in a 

meaningful way, is the ultimate challenge for cumulative risk assessment. Experience in 

this area is extremely limited. Indices, common metrics (e.g., Disability Adjusted Life 

Years - DALYs) and graphical (e.g., GIS) approaches have been explored but much 

methods development work remains to be completed. Cumulative risk assessment can be 

a valuable part of the decision making process, but only if the results are conveyed in a 

meaningful way. Comment on the Framework’s discussion concerning the combining of 

C-4
disparate measures of risk. Do the example approaches discussed capture the state of the 

science in this area? In particular, consider the role of valuation (i.e., the assignment of 

societal values to disparate health outcomes) implicit in some of the approaches. Suggest 

changes or additions that may improve this discussion. 

C-5
Appendix D 

Agenda 

D-1
United States 

Environmental Protection Agency

Risk Assessment Forum

Technical Peer Review Workshop 

on the EPA Risk Assessment Forum 

Draft Framework for Cumulative Risk Assessment 

Courtyard Crystal City Hotel

2899 Jefferson Davis Highway

Arlington, VA

June 4-5, 2002

Agenda 

Workshop Chair: Peter de Fur 

Environmental Stewardship Concepts 

TUESDAY, June 4, 2002 

8:30AM Registration 

9:00AM Welcome & Introductions David Bottimore, Versar, Inc. 

9:15AM Chair’s Introduction 

Peter deFur, Workshop Chair 

9:30AM Background Steven Knott, Risk Assessment Forum (RAF), 

U.S. Environmental Protection Agency (U.S. EPA) 

9:45AM Presentation on the Draft Framework for Cumulative 

Risk Assessment Michael Callahan, U.S. EPA, Region VI 

10:15AM B r e a k 

10:30AM Summary of Premeeting Comments 

Peter deFur, Workshop Chair 

10:45AM Initial Round Table Discussion 

12:00PM L u n c h 

D-2
TUESDAY, JUNE4, 2002 (continued) 

1:15PM Discussion Session 

# Charge Question #1 - Comment on whether the Framework adequately 

captures, describes, and reasonably organizes the key issues for 

cumulative risk assessment so as to serve as an adequate foundation for 

the development of future guidance. 

a) Does the Framework document capture the relevant terminology? 

b) Does the Framework document provide an adequate assessment 

structure? 

c) Does the Framework document outline the relevant methods for 

cumulative risk? 

2:45PM Break 

3:00PM Discussion Session (cont’d) 

# Charge Question #2 - Does the Framework document include any 

scientific or technical inaccuracies in its presentation of terminology, 

assessment structure, and methods? Please identify any problem areas and 

propose revisions or other actions that will result in a scientifically sound 

and supportable discussion. With respect to methods, comment on 

whether the Framework adequately conveys the state of the science with 

respect to currently available cumulative risk assessment 

methods/approaches and the areas that are in need of further research and 

development. 

4:15PM Observer Comments 

4:45PM Wrap-Up 

5:00PM A d j o u r n 

D-3
WEDNESDAY, June 5, 2002 

8:30AM Discussion Session 

# Charge Question #3 - Uncertainty analysis is an important aspect of risk 

assessment (and policy analysis in general). However, historically, 

dealing with uncertainty has been a shortcoming of many assessments. 

Cumulative risk assessments present new challenges for uncertainty 

analysis. For example, assessing cumulative risks will involve combining 

data of varying quality. Perhaps more important, assessing cumulative 

risks will involve the use of “soft” assumptions. These are assumptions 

which may have a high degree of uncertainty that is difficult (or not 

possible) to quantify. Comment on whether the Framework adequately 

characterizes the importance of uncertainty analysis in cumulative risk 

assessment. What additional discussions of uncertainty should be 

included in the Framework (and in what sections of the document)? 

9:30AM Discussion Session (cont’d) - Charge Question #4 

The following topics have been identified by the RAF technical panel as 

technically difficult areas that will pose challenges to cumulative risk 

assessment. Comment on the adequacy and accuracy of the Framework’s 

presentation in each of these areas. 

#	 Vulnerability - As applied to cumulative risk assessment, it is useful to 

think of four components to vulnerability: the susceptibility or sensitivity 

of the human or ecological receptors; the differential exposures of the 

receptors; the differential preparedness of the receptor to withstand the 

insult from exposure; and the differential ability to recover from the 

effects. The issue for cumulative risk assessment is how to consider these 

aspects of vulnerability and their potential impacts on risk. This is 

highlighted in the Framework as an issue in need of further research and 

development. Comment on the discussion of vulnerability in the draft 

Framework. Has the state of the science been captured in this discussion? 

How can the discussion of this issue be improved? 

10:30AM Break 

D-4
WEDNESDAY, June 5, 2002 

10:45AM Discussion Session - Charge Question #4 (cont’d) 

# Cumulative Risk Assessment Involving Chemical and non-Chemical 

Stressors - Viewing cumulative risk assessment as an evaluation of the 

accumulation of stressors presents many challenges. These may be seen 

when attempting to combine, in some meaningful way, the risks from 

multiple chemicals that may act as synergistic, antagonistic, or additive 

doses leading to a single effect. The situation is exacerbated when nonchemical stressors (e.g., radiation, biological agents, and psychological 

stress) are considered. Comment on the Framework’s discussion 

concerning the combining of disparate environmental stressors. In 

commenting, consider the state of the science with respect to 

understanding the effects of different stressors acting together (e.g., 

chemical exposure and viral infection). What can be added to the 

Framework to adequately convey the state of the science in this area? 

11:45AM Lunch 

1:00PM Discussion Session - Charge Question #4 (cont’d) 

# Cumulative Risk Assessment Involving Different “Types” of Risk -

Conveying the combined risks from multiple chemical and non-chemical 

stressors, in a meaningful way, is the ultimate challenge for cumulative 

risk assessment. Experience in this area is extremely limited. Indices, 

common metrics (e.g., Disability Adjusted Life Years - DALYs) and 

graphical (e.g., GIS) approaches have been explored but much methods 

development work remains to be completed. Cumulative risk assessment 

can be a valuable part of the decision making process, but only if the 

results are conveyed in a meaningful way. Comment on the Framework’s 

discussion concerning the combining of disparate measures of risk. Do 

the example approaches discussed capture the state of the science in this 

area? In particular, consider the role of valuation (i.e., the assignment of 

societal values to disparate health outcomes) implicit in some of the 

approaches. Suggest changes or additions that may improve this 

discussion. 

2:00PM B r e a k 

2:15PM Observer Comments 

2:45PM Wrap-Up, Summary of Comments, and Next Steps 

4:00PM A d j o u r n 

D-5
Appendix E 

Presenter Overheads 

E-1
Technical Peer Review Workshop

on the

Draft Framework for Cumulative Risk Asses

June 4-5, 2002 

David Bottimore 

Versar, Inc. 

Courtyard Crystal City

2899 Jefferson Davis Highway

Arlington, VA 22202 

E-2
Overview of Peer Consultation Workshop 

Welcome Review of Agenda Introduction of Participants EPA Presentations Chair - Peter deFur - groundrules, summary of comments, and dtopics Observer Comments Post Meeting Activities – Workshop Report 

E-3
Technical Peer Review Workshop

on the

Draft Framework for Cumulative Ris

Assessment

June 4-5, 2002 

Peter deFur - Chair 

E-4
Chair’s Opening Remarks 

Goals for Meeting – Provide input to EPA on technical issues relateFramework for Cumulative Risk Assessment Ground Rules – Process issues, do’s and don’ts Peer Review Process 

– Obtaining input on technical issues from experts in divespecialties and from broad perspectives – Focus on technical issues (not regulatory or policy) – Not a consensus building process – Documentation of comments and recommendations – Role of EPA in peer review meeting 

Post Meeting Activities – Workshop report that summarizes discuscomments on Framework 

E-5
Ground Rules and Operating Guidelines 

- We are here as individuals - Consensus is not necessary and will not be actively sought - Everyone participates - Keep to the logistics of time, subject, scope - Keep to the topics and the task (we will keep side lists) - Peer review among ourselves is the activity - not a conversation w- No EPA bashing - or other attacks - Mutual respect - Factually based comments - Distinguish fact and opinion 

E-6
Ground Rules and Operating Guidelines (cont’d) 

- Everyone speaks for themselves - No ad hominum or organizational attacks - Speak up when it is needed - Hold side conversations during breaks etc. - You will be working with these people again in your careers 

E-7
Charge Questions 

1. Comment on whether the Framework adequately captu

describes, and reasonably organizes the key issues for risk assessment so as to serve as an adequate foundatdevelopment of future guidance. In your comments, pladdress each of the following questions. In answering question, provide a supporting discussion that highlighareas of the Framework that may need to be clarified antopics that may be missing from the current Framework Include references to any published literature that coulimprove the completeness and clarity of the Framewora) Does the Framework document capture the relevantterminology? b) Does the Framework document provide an adequateassessment structure? c) Does the Framework document outline the relevant cumulative risk? 

E-8
Charge Questions (cont’d) 

2. Does the Framework document include any scientific o

inaccuracies in its presentation of terminology, assessstructure, and methods? Please identify any problem apropose revisions or other actions that will result in a ssound and supportable discussion. With respect to mecomment on whether the Framework adequately conveof the science with respect to currently available cumuassessment methods/approaches and the areas that arfurther research and development. 

E-9
Charge Questions (cont’d) 

3. Uncertainty analysis is an important aspect of risk asse

(and policy analysis in general). However, historically, uncertainty has been a shortcoming of many assessmeCumulative risk assessments present new challenges funcertainty analysis. For example, assessing cumulatiinvolve combining data of varying quality. Perhaps moimportant, assessing cumulative risks will involve the uassumptions. These are assumptions which may have degree of uncertainty that is difficult (or not possible) tComment on whether the Framework adequately charaimportance of uncertainty analysis in cumulative risk aWhat additional discussions of uncertainty should be inthe Framework (and in what sections of the document)? 

E-10
Charge Questions (cont’d) 

4. The following topics have been identified by the RAF te

panel as technically difficult areas that will pose challecumulative risk assessment. Comment on the adequacaccuracy of the Framework’s presentation in each of tha. Vulnerability As applied to cumulative risk assessment, it is useful to four components to vulnerability: the susceptibility or sthe human or ecological receptors; the differential expothe receptors; the differential preparedness of the recewithstand the insult from exposure; and the differential recover from the effects. The issue for cumulative risk assessment is how to consider these aspects of vulnertheir potential impacts on risk. This is highlighted in thFramework as an issue in need of further research and development. Comment on the discussion of vulnerabdraft Framework. Has the state of the science been cap

E-11
this discussion? How can the discussion of this issue 

improved? Charge Questions (cont’d) 

4b. Cumulative Risk Assessment Involving Chemical and n

Chemical Stressors Viewing cumulative risk assessment as an evaluation oaccumulation of stressors presents many challenges. be seen when attempting to combine, in some meaningrisks from multiple chemicals that may act as synergistantagonistic, or additive doses leading to a single effecsituation is exacerbated when non-chemical stressors radiation, biological agents, and psychological stress) considered. Comment on the Framework’s discussion the combining of disparate environmental stressors. In commenting, consider the state of the science with resunderstanding the effects of different stressors acting (e.g., chemical exposure and viral infection). What can the Framework to adequately convey the state of the scthis area? 

E-12
Charge Questions (cont’d) 

4c. Cumulative Risk Assessment Involving Different “Type

Conveying the combined risks from multiple chemical achemical stressors, in a meaningful way, is the ultimate for cumulative risk assessment. Experience in this areextremely limited. Indices, common metrics (e.g., DisabAdjusted Life Years - DALYs) and graphical (e.g., GIS) ahave been explored but much methods development wto be completed. Cumulative risk assessment can be a part of the decision making process, but only if the resconveyed in a meaningful way. Comment on the Framediscussion concerning the combining of disparate mearisk. Do the example approaches discussed capture ththe science in this area? In particular, consider the role valuation (i.e., the assignment of societal values to disphealth outcomes) implicit in some of the approaches. Schanges or additions that may improve this discussion

E-13
Risk Assessment Forum 

Framework for Cumulative Risk Ass

Steven M. Knott

Exposure Science Coordinator

Risk Assessment Forum Staff
Risk Assessment Forum

Risk Assessment Forum’s Mission: 

To promote Agency-wide consensus on dirisk assessment issues and to ensure that thconsensus is incorporated into appropriate Agency risk assessment guidance. 
Risk Assessment Forum 

Characteristics of Forum Projects: 

They represent difficult or precedent sscientific questions for the Agency They are intended to guide the Agewhole rather than any specific progThey are designed to be regulation ne
Risk Assessment Forum

Forum Products Provide Guidance to EPA Risk A

Agency Guidelines (e.g., Guidelines for EcologicaAssessment) Guidance Documents (e.g., Supplementary GuidanConducting Health Risk Assessment of Chemical MTechnical Papers (e.g., Special Report on EnvironEndocrine Disruption: An Effects Assessment and 

Visit the Forum Web Page at: 

www.epa.gov/ncea/raf 
Risk Assessment Forum

Cumulative Risk Assessment Guidance Development 

SPC Requests that the Forum Begin Developing Cumulative Risk Assessment Guidance Risk Assessment Forum Technical Panel Conve
Risk Assessment Forum

Programs Participating in the Risk Assessmen

Cumulative Risk Assessment Technical P

EPA Region VI, Michael Callahan, Panel Chair Office of Science PolicyOffice of Pesti

cide Programs

Office of Pollution Prevention and ToxicsOffice of Environmental Justi

c

e

Office of Emergency and Remedial ResponseChemical Emergency Preparedness and Prevention OOffice of Solid WasteOffice of Air Quality Planning and StandardsNational Center for Environmental Assessment Regions V, VI, and IX
Risk Assessment Forum

Guidance Development Process

-

Develop Framework 

-

Extract lessons learned/experiences 

from ongoing case studies to illustraspects of the Framework 

-

Develop technical issue papers on 

selected cumulative risk topics

-

Develop Proposed Guidelines for 

Cumulative Risk Assessment 
Risk Assessment Forum

The Framework 

Building from ongoing cumulative risk assessmexperiences within the Agency, the Framework intended to capture the basic elements of the curisk assessment process. The Framework: 

1) provides a flexible structure for 

cumulative risk assessment issues (caevolving with experience) 

2) defines key terms, and basic concept

to promote a common language on cumulative risk assessment 
Risk Assessment Forum

The Framework cont’d 

3) does not provide substantive technical 

guidance; but rather 

4) serves as a basis for the development of 

future issue papers and cumulative risk assessment guidelines; 
Risk Assessment Forum

Milestones in the Development of the DrFramework for Cumulative Risk Assessm

Consultations with Science Advisory Board (FYFY’ 01) Meetings with Federal and State Government S(May 2001) Public Peer Consultation Workshop (August 20External Peer Review (June 4 and 5, 2002) 

Goal:

Complete the Framework for Cumulative Risk 

Assessment by the End of FY’02. 
Framework for 

Cumulative Risk 

Assessment 

Peer Review Meeting 

June 4-5, 2002 
Framework vs. Guidelin

• 

Framework: General description of the t

An information document laying out s

of the subject and how various parts fit together. (This document) 

• 

Guidelines: Description of how it’s done

including boundaries (e.g., limits of “goscience”) not to be exceeded. (To be completed later) 
Framework Definitions 

• Cumulative Risk: The combined risks 

aggregate exposures to multiple agentstressors. 

• Cumulative risk assessment: An ana

characterization, and possible quantificof the combined risks to health or the environment from multiple agents or stressors. 
Key Definition Points 

• 

Multiple stressors or chemic

• 

Combined risks 

• 

Can be qualitative 
Goal of Cumulative Ris

Assessment 

• 

Using the commonly accepted defi

of risk as “probability of harm”, the of a cumulative risk assessment is: 

– 

To address and hopefully answe

questions related to the probabilharm, to human health or the environment, from multiple stressacting together. 
Cumulative Risk Assess

• 

“Traditional” Risk Assessment: 

-

Where we’ve been 

• 

Cumulative Risk Assessment : 

-

Why change? 

• 

Ongoing Agency efforts 

• 

Policy issues 

• 

State of the science 
Chemical, 

Agent, or 

Stressor 

“Traditio
Community, 

Population, or 

Population Segment 

S

Stressor 

Chemi

Stressor 

Chemical 

Chemical 

Chemical 

Stressor 

Stressor 

“Population-bas
Current Agency Effort

• 

OP pesticides 

• 

National Air Toxics Assessment 

• 

Dioxin reassessment 

• 

City-specific studies 
Policy Issues 

• 

Not part of this peer review 

• 

Agency is talking internally about 

various policy issues 
Current State of the 

Science? 

• 

Cumulative risk assessments will be mo

useful in situations where questions neebe addressed concerning the impacts omultiple stressors acting together 

• 

Cumulative risk assessment is a tool 

• It is not appropriate for every task 

• 

Currently, there are methods limitations

research and development is needed 
Organization of Repor

1. Introduction 2. Planning, Scoping, and Problem 

Formulation Phase 

3. Analysis Phase 4. Risk Characterization Phase 5. Glossary 6. References 7. Appendices 
Planning/Scoping, 

Problem Formulation 

• 

Public participation description in 

Chapter 2 reflects recent Agency trend. 

• 

Conceptual model and analysis p

• 

Discussion of possible outcomes 
Analysis Phase 

• 

Can different types of risk be 

combined? 

• 

Toxicologic similarity vs. toxicologic 

independence 

• Influences • 

Common metric approach 

• Index approach 
Analysis Phase: 

Vulnerability 

• 

Susceptibility/Sensitivity 

• 

Differential exposure 

• 

Differential preparedness 

• 

Differential ability to recover 

• 

Sexton (1997): 

important?”, but how much do thesfactors change bottom-line risk? 

Issue is not “are th
Risk Characterization

Uncertainty 

• 

Few good examples of uncertainty 

analysis for Cumulative Risk Assessments 

• 

New GIS-based technology poses 

challenges in uncertainty analysis 

• 

What type of analysis would be use

a decision-maker? 
Schedule 

• Framework currently in peer review • Science Policy Council final signof• Framework final by Fall, 2002? • Case studies developed 2002? • Guidelines development starts 200
Appendix F 

Written Comments 

F-1 
Written Comments from Peer Reviewers 

Gail Charnley - HealthRisk Strategies 

Robert Collin - University of Oregon 

Peter deFur - Environmental Stewardship Concepts 

David Goldsmith - George Washington University 

Dale Hattis - Clark University 

Joann Held - New Jersey Department of Environmental Protection 

Paul Locke - Johns Hopkins University 

Margaret MacDonell – Argonne National Lab 

Pavel Muller - ToxProbe Inc. 

Barry Ryan - Emory University 

Jennifer Sass - Natural Resources Defense Council (NRDC) 

Paul Schlosser – CIIT Centers for Health Research 

Curtis Travis - Quest Technologies

F-2 
Review by 

Gail Charnley - HealthRisk Strategies 

F-3 
PRE-MEETING COMMENTS 

Gail Charnley 

General Impressions 

1. Including risk managers in the problem formulation phase is an important step towards the 

development of risk assessments that are focused on solving problems instead of creating them. 

2. It is notable that the examples of cumulative risk assessments currently underway at EPA 

(executive summary) are confined to individual programs and offices. Risks from cumulative 

exposures are seldom cognizant of statutory constraints. One of the goals of this document 

should be to help move the agency forward in an attempt to, if not overcome, at least start to 

integrate the requirements of its various legislative mandates. Mentioning such a goal and 

providing a couple of examples of possible cross-program assessments would be useful. 

3. The introduction is a useful overview of the evolution of risk-based decision-making at EPA, 

first towards single-chemical-based assessments and now towards cumulative analyses. It also 

provides a clear description of the nature and purpose of the framework 

Charge Questions 

1. The document does a very good job of articulating the relevant terminology and providing an 

adequate assessment structure. However, it does not describe the relevant methods for 

cumulative risk so much as describe the relevant process and approaches, mentioning methods to 

the extent they have been developed while pointing out that the methods and data are still 

challenging and evolving. 

2. I have not yet detected any scientific or technical inaccuracies. 

3. The document does not appear to over-sell its approach to cumulative risk assessment. It 

provides a balanced discussion of what we know and what is still uncertain (much) about 

performing a cumulative risk assessment. It does not over-emphasize quantitative analysis and 

appropriately promotes qualitative analysis. 

Details 

1. Page 27; The Baltimore Community Environmental Partnership Air Committee experience is 

referred to briefly as an example of a community-based cumulative risk attempt that contains 

valuable lessons. I agree and suggest that it be described more fully in an accompanying box 

similar to that for the Cook and Lake Counties Cumulative Risk Initiative on page 29. 

2. Page 61; The discussion of cumulative risk in a public health context could use some 

expansion and possibly relocation. I will bring suggested additional text to the meeting. 

F-4 
Review by 

Robert Collin - University of Oregon 

F-5 
The Risk Assessment Forum May 2002 

Draft Framework for Cumulative Risk Assessment 

Response to “Peer Review Draft Charge” - Robert Collin 

This was prepared in response to Versar’s request. It is a preliminary reaction designed to assist 

Versar in arranging a 2 day meeting. 

1. Framework 

A. Captures- yes, in this context. 

B. Describes - yes generally, but see page 36, line 43 -44; need more on exposure and 

vulnerability. 

C. Reasonably organizes - yes 

key issues in context of foundation for future guidance.[EPA policy I presume?]

A. Capture relevant words -

It seems to capture most of terminology. Possible exception - regional cumulative risk 

management; a broader discussion of the different meanings of the word “inert” in Chemistry 

and in consumer marketing. If the cumulative risk assessment reaches all stressors and close 

pathways, then chemicals under the sink and in the cleaning closet need to be fully disclosed to 

users [like MSDS]. 

B. Provide adequate assessment structure. 

If adequate means a fair start, then yes. If adequate means actually measuring accumulating 

chemicals and their interactions dynamically, then no. If adequate means something that meets 

the minimum legislative and intellectual goals of NEPA, then generally yes. If adequate means 

application to urban areas, then a general, but hopeful, no. 

C. Outline methods of cumulative risk; especially valuation [assignment of societal 

values to disparate health outcomes]. Public health is both a societal value, and presumed aspect 

of citizenship. It is also a Context. P 61, line 21 ; s 4.3.2 Cumulative Risk Assessments in a 

Public Health Context. 

2. Inaccuracies 

Problem areas revised only with “scientifically sound and supportable discussion.” The 

recently formed Canadien Cumulative Risk Association accepts both sciences as part of their 

organizational mission. 

State of science and areas in need of further research and development [ also, how to do 

this research in controlled application] 

3. Uncertainty as “shortcoming”. Additional discussions of “uncertainty” in which section 

of the document. This discussion was interesting and useful. I would add epistemological 

uncertainty - roughly meaning not even knowing the questions to ask, yet. As to where to add 

this term and its explanation, its application to communities will be seen as a need for 

precaution. 

F-6 
4. Adequacy and accuracy of Framework’s presentation of 

a. Vulnerability - did not see much about generation skipping chemical effects. Did not see 

much of the research on “coping” mechanisms; treatment of childhood cancers causing 

secondary cancers; preventative health measures in environmental decisions and their effects on 

cumulative risk assessments; or assessment of migratory and immigrant populations. The 

Demographics can get complicated with dynamic urban issues like ethnic churning and 

inaccurate Census counts of particularly vulnerable populations. 

b. Chemical and non chemical stressors - did not see much discussion of so called “inert” 

chemicals that are part of the everyday exposure paths of households, hospitals, and armies. 

c. Different “types” of risk - did not see much on how perception of industrial hazard risk 

alone increases risk of heart attacks, stroke, and hypertension. In s 3.3.1 Interaction Between 

Stressors and Other Factors, issues like the lack of health care access and the lack of trust of 

Physicians may increase risk, and will increase uncertainty. The biomarker discussion was 

interesting. Metals that bioaccumulate remain metals for a detectable amount of time in tissue. 

They may be adverse, as defined here, or not. It raises a common environmental issue, from an 

Environmental Justice perspective, of some communities being required to scientifically prove 

harm to themselves; and other communities simply preferring not to have the same exposure. 

Here, in the Biomarker discussion, people may be part of your baseline as well as the 

implementing stakeholder. People can create risk to each other from vehicles, waste, etc. and 

would seem to be a necessary component of a good cumulative risk assessment. Also, is there 

any discussion of how these factors can synergize/antagonize other factors? [besides 

nuerotoxicants] 

5. Post Meeting Addendum 

Here is my list of references, in no particular order. I have them if copies are difficult to get. I 

have given pinpoint cites where applicable, and can discuss any of these with you if you wish. I 

have included 3 forthcoming references. 

1. Institute of Medicine, Research, Education and Health Policy Needs, 

ENVIRONMENTAL JUSTICE (National Academy Press, Washington, DC 1999) . ISBN # 

0-309-06407-4 

2. Adam M. Finkel & Domenic Golding, eds. WORST THINGS FIRST:THE DEBATE 

OVER RISK BASED NATIONAL ENVIRONMENTAL PRIORITIES (Resources for the 

Future, Washington, DC1994) isbn # 0-915707-74-8), pp. 237 -324 - The Environmental Justice 

Paradigm. 

3. Sheldon Krimsky & Domenic Golding, eds, SOCIAL THEORIES OF RISK, Praeger 

Publishers, Westport, CT 1992) ISBN # 0-275-94168-X. pp 83 - 196 - Social, Cultural, and 

Psychological Paradigms; pp. 215 - 300 The Role of Science in Risk Assessment. 

4. This is an international journal that EPA should review. Here are 2 recent articles. In the 

F-7 
first they discuss the principles of the Canadien Cumulative Risk organization. Steve Bonnell & 

Keith Story, ADDRESSING CUMULATIVE EFFECTS THROUGH STRATEGIC 

ENVIRONMENTAL ASSESSMENT; A CASE STUDY OF SMALL HYDRO 

DEVELOPMENT IN NEWFOUNDLAND, CANADA, vol. 2, no. 4 12/2000, pp. 477 - 500. 

Same journal volume - Harry Spaling, et al, MANAGING REGIONAL CUMULATIVE 

EFFECTS IN OIL SANDS DEVELOPMENT IN ALBERTA, CANADA, pp. 501 - 528. ISSN 

1464 - 3332, Imperial College Press, London. 

5. Robert Bullard, ed. CONFRONTING ENVIRONMENTAL RACISM: VOICES FROM 

THE GRASSROOTS (South End Press, Boston, MA 1993) ISBN# 0-89608-446-9. 

6. National Academy of Public Administration, MODELS FOR CHANGE: EFFORTS BY 

FOUR STATES TO ADDRESS ENVIRONMENTAL JUSTICE, (June 2002) EQ-82906401-0. 

7. James Flynn, Paul Slovic , and Howard Kunreuther, RISK, MEDIA, AND STIGMA: 

UNDERSTANDING PUBLIC CHALLENGES TO MODERN SCIENCE AND 

TECHNOLOGY (Earthscan Publications WWW.EARTHSCAN.CO.UK, 2001) ISBN3 1 85383 

700 8. This includes a perspective from industry. 

The following are books that are FORTHCOMING. 

1. Robert Riddell, SUSTAINABLE URBAN PLANNING: THE DELIVERY OF 

CONSERVATION WITH DEVELOPMENT (Blackwell Publishers, London 2003). 

2. R & R Collin, FOREVER WILD, FOREVER FREE: ENVIRONMENTAL JUSTICE 

AND SUSTAINABILITY, (Georgia University Press, Athens, GA, 2003). 

3. Robert Bullard, ed. UNEQUAL PROTECTION REVISITED, (Sierra Club 

Publications, San Francisco, CA 2003). 

I have many other International, EJ, Sustainability, and Law references that a Professor would 

think relate to cumulative risk assessment. Footnotes, literature reviews, and research generally 

are some of my favorite things. However, my goal was to give a window to the literature 

approach to this document at this stage of development. This is based on my hope that the issue 

papers develop strong reference and research foundations. This is also based on my hope that 

some type of national clearinghouse, per this meeting, will be developed. 

F-8 
Review by 

Peter deFur - Environmental Stewardship Concepts 

F-9 
Comments on 

"Framework for Cumulative Risk Assessment" 

EPA/630/P-02/001A 

April 23, 2002 Draft 

These comments were prepared in advance of the peer review of the Framework to be held June 

4-5, 2002 in Crystal City, VA and conducted by Versar, Inc. under contract to EPA. These 

comments are arranged as follows: general comments on the Framework; reply to charge 

questions provided by Versar; page specific comments, including editorial comments. 

I. General Comments: 

This Framework document is the initial step in the Agency effort to draft guidelines for conducting 

Cumulative Risk Assessments. The document has been carefully prepared and drafted by staff who 

are familiar with a wide array of Agency risk assessment efforts and reports. The relationships with 

other EPA Frameworks and risk assessments are clear and the intent of this Framework is explained 

by the authors. Some issues warrant a fuller discussion and the peer review should be one context 

in which to hold some of those discussions. 

The Framework has the simultaneous benefit and curse of many authors. The benefit of many 

authors is the greater expertise and experience; the curse is the different styles of each in such a short 

document. Editorial attention is needed to overcome the differences in style. Some sections, not all, 

are written with long sentences (approaching and reaching run-on status); see specific comments, 

e.g. page 49. The passive voice is used too often in my estimation and more so in some chapters 

than others. Complex sentences and dependent clauses are also something that could be evened out 

to improve the readability and flow. Also, EPA needs to be very cautious about writing this for a 

knowledgeable v naive audience; it is now written for the former. 

The Framework uses the term "background" to mean both "background" and "ambient". The word 

ambient is a better term for what is meant in this Framework as the present conditions that exist 

now- to which people and ecological units are exposed. I prefer ambient for these applications and 

actually think background has both explicit and implicit meaning that is not applicable. 

The Framework needs more ecological examples and explanation. My specific comments indicate 

in some places where these might be used, references, and in some cases the text. This document 

seems to be the first EPA effort to integrate human health and ecological risk and as such needs to 

use language and references from both fields. 

Several differences between ecological and human health risk assessment are worth noting and I 

have been thinking about whether or not these warrant a section in the front of the Framework. 

Clearly the authors recognized differences and sought to use the best and most appropriate features 

of each in developing the Framework. 

Some of the major differences between human health and ecological are; 

1) Eco RA uses a problem formulation phase 

2) Ecological systems are not as well known biologically as are human health systems, both 

at the population and the individual level. 

F-10 
3) For this reason, and because biological communities and ecosystems are inherently more 

complex, EcoRA requires more preliminary analysis and deliberation regarding 

endpoints and protective standards 

4) Ecosystems, habitats and ecological communities have traits and properties that 

individuals do not or that are not applicable to individuals or populations 

5) Eco RA has been generally applied to populations, not individuals, a situation reversed 

for human health. 

6) Ecological risk assessment must assess risk at multiple levels of organization, i.e. 

molecule, cell, organism, population, community, ecosystem. 

This Framework follows the form of the Ecological Risk Assessment Framework and it would 

seem EPA is following a similar guideline development process here. This point needs to be 

made more clearly and explicitly in the Framework, with specific references to both substance 

and process similarities. 

The initial step proposed here is planning scoping and problem formulation; the last step coming 

from the Eco Risk Guidelines. EPA should separate these into two steps in order to distinguish 

management and assessment as originally recommended by NRC in 1983 (Risk Assessment in 

the Federal Government). NRC did not say to remove the risk managers from the process, but 

NRC did recommend that the management and assessment process be kept separate B and for 

good reason. Managers have a propensity to become closely involved in the risk assessment 

process unless specifically removed. The NRC was concerned that managers will tell 

assessment teams what sort of outcomes are acceptable or not. EPA heard a similar concern 

when preparing the Eco Risk Guidelines (Peer Review Report from Dec 1995 review). 

The figures all show double headed arrows that more than imply a reversible flow of 

information, results, and data. While there is an important iterative step that must remain a part 

of the approach to conducting risk assessment (RA), this feature creates an endless loop that will 

lead to paralysis by analysis. The Framework needs a change to eliminate this option. The RA 

is a decision-based, outcome driven process, and is not an open-ended investigation. 

Is this intended to be used in combined human health/ecological risk assessments? (see Suter ). 

If so, then some groundwork needs to be inserted here. 

II. Reply to Charge Questions: 

1. key issues, terminology, structure and methods 

The general answer to this question is "Yes, but." The general Framework is good, and builds on 

the Ecological Risk Assessment (RA) Guidelines, with which I am familiar. This Framework 

also indicates that it is the first step in a longer process, but this point needs to be repeated in the 

beginning of each chapter. The big missing element is an explanation up front that the difference 

between single chemical and cumulative risk means that the entire approach is altered and the 

conduct of the resulting assessment will shift accordingly. Compare the TCDD national 

assessment, any of the EPA watershed assessments and some single chemical/single source 

assessment. Basic assumptions differ, data handling, equations differ, etc. 

Terminology is mostly correct except that I do not agree that background is the correct term 

here- it should be ambient. 

F-11 
Methods- See the Warren-Hicks and Moore (1999) book on uncertainty analysis, Stahl et al 

(2001) on Risk Management v RA; Foran and Ferenc provide methods for multiple stressors that 

come from Harris et al (see Foran and Ference) and others. 

2. State of the Science: 

In my opinion, the difference between "ambient" and "background" is a scientific one as well as 

terminology. Ambient really is meant to refer to local current conditions as influenced by 

whatever factors are on-going. "Background" at least implies some untouched, even pristine 

state or condition. Background has the meaning of some reference state or location. 

The double arrows in the figures are important substantive issues for analysis that EPA needs to 

address here and now rather than wait for later. RA must no be an endless loop of analysis and 

measurement. 

Numerous methods exist for uncertainty analysis and I refer to the Warren-Hicks and Moore 

(1999) publication. 

The matrix approach of Harris et al and explained in greater detail in Foran and Ferenc is an 

important method for assessing multiple stressors that can be explained in greater detail here. 

3. Uncertainty analysis: 

The Framework section on uncertainty analysis could be enhanced, but this field is now getting 

more attention than even 5 years ago, so it is going to be hard. But there are methods of 

quantitative analysis that should be noted: Probability bounds, Bayesian analysis, point 

estimation, etc. 

4. Vulnerability, chemical and non-chemical stresses; types of risk: 

The text on vulnerability is quite good in explaining what is meant by vulnerability and why it 

matters. But the context for considering vulnerable targets is not clear to me. It feels awkward 

coming in to the discussion. Vulnerability is an aspect of analysis that has to be considered in 

the problem formulation B what are the vulnerable targets- either humans or ecological units? 

And then the analysis needs to be conducted to account for these vulnerable targets. 

There is not much written in the area of combining chemical and non-chemical stresses in the 

RA literature, but I suggest that the pure literature in pathology and toxicology has some 

information on this matter. EPA can and should get some information on how these combine by 

getting some literature work done in specific areas, such as comparing disease in exposed and 

non-exposed groups, comparing response to second exposures, etc. 

Measures of risk is another area that has not much literature, and I doubt that the non-expert 

reader will really understand the nature of the discussion. I am not especially impressed with the 

discussion on different valuations of risk - unless of course EPA is ready to insert a good 

discussion of the Precautionary Principle into the document. Such text would help a lot, 

especially in the area of valuation of risk. My reading of the literature is that the public is fairly 

F-12 
risk-averse in the environmental area. Citizens do not want to have even low risk in water, air, 

food, soil, which they believe should not be degraded as a matter of course by the fact of humans 

living on the planet. So, the socially acceptable risk business leads down the slippery slope of 

accepting basically unacceptable situations simply because someone ways the situation cannot 

be fixed. 

I am not sure to which sections of the Framework this question applies. And that comment in 

itself is a comment on the labels or structure, or both. Issues of this nature are in section 3.3.3.1, 

and that text is interesting, but I am not sure how necessary it is. I have to read it another time to 

address that question specifically. But one aspect I do not agree with is the appearance of 

economic issues in several places outside the context of management. 

III. Page specific comments: 

Page1 Figure 1-1 does not do the trick. I recommend making it linear, and proceed from left to 

right 

Page 2 This figure does not cover the subject well either, although it is better than fig 1. 

Page 6-7 The definitions of aggregate and cumulative risk need some attention in order to 

address the difference between those two terms. I am not sure that all f the readers and intended 

audience will appreciate the distinctions as they are now expressed. 

In addition, the document should make an explicit comment that cumulative risk refers to time, 

space and method or route of exposure. In single stressor risk assessments for human health, the 

focus on identifying the pathway and the effect drives the entire process. Now, that approach 

has to be replaces with one that essential does that opposite and intentionally includes time, 

space, media exposures. 

Page 13 Figure 1-3 makes a single step out of the two steps of Assessment- Manager discussion 

and problem Formulation that comes from Ecological Risk Assessment. I do think these steps 

are sufficiently different that they should be specifically identified as not one step. 

Page 15 section 2.1 Planning and Scoping – 2nd and 4th paragraphs need language changes to 

require the legal mandates and community participants int eh processes. These are not optional 

elements and if they are not addressed at the beginning will likely bog down the RA. 

Page 18 Line 13- make this “ambient”, not background. See previous comment and discussions. 

Page 19 Section 2.1.2 that starts on p 17 must refer to the analytical and deliberative methods 

and processes. Perhaps the specific equations are unclear until more data are gathered, but the 

expected form of analysis should be spelled out initially. Also, as noted by NRC (1996, 

Understanding Risk), this step needs to specifically indicate both analysis and deliberation are 

required. 

F-13 
Section 2.1.3 needs to include language on how this participation will work for ecological resources 

and evaluations. There are plenty of possible groups, but let's see a list here: outdoors clubs, 

American Indians, community groups, state and local and national resource NGO's, commercial and 

recreational fishers, etc. 

Page 21 Lines 14 et seq. B We need some participant programs like the TAG Superfund 

program. Some states have this idea, but some national leadership would be great. 

Page 23, line 11 Start by identifying all the system components and the relationships among 

them. 

Page 26, line 37 The economics is NOT an RA issue - it is risk management and policy. This 

analysis is later and separate. See Stahl et al 2001 (SETAC book Ecological Risk Management). 

Page 31, line 1 et seq. Somewhere in this section the Framework needs to acknowledge that the 

dose-response and risk characterization are placed in different sections of the RA process in Eco, 

and Human Health RA and why it does or does not matter. 

Line 11, Section 3.1 The whole process should begin with a status assessment of either the 

population or habitat or both. 

Page 42, line 10-11 please explain this in simpler terms- many readers will not understand this as 

written 

In this section, please give an example from an exotic species introduction, and an exotic in 

combination with chemicals B see the excellent risk assessment by USDA for the control of Gypsy 

Moth in the eastern US. 

Page 46, line 6 Remove the economic language or relegate it to indirect or secondary at best. 

Economics is part of the management analysis. 

Page 48, line 9 Explain that "the use of biomarkers is based on the concept that the biological 

unit is an effective and accurate element to integrate the aggregate/cumulative risks or 

exposures." 

Line 25- 31 I don't know the purpose of this language - it is apologetic and dismissive of 

biomarkers. Biomarkers are merely measures of exposure and not effect, and include all that the 

animal has experienced. Furthermore, body burdens are certainly effective estimates that can 

and have been, and are used to estimate total risk to individuals (NB Pb, Hg, TCDD, PCB's, 

etc.), even if these do not provide sufficient information on specific exposures or sources, or 

periods of sensitivity. DDT residues certainly do provide information on total lifetime 

exposures, but not on whether exposure during puberty was greater and caused greater risk for 

one disease or another. 

Lines 39- 42 Please reword to make it more clear- it reads like weasel words as it is now written. 

F-14 
Page 49, lines 9-16 This paragraph is an example of editorial needs in the document B with 

apologies to the original author. This paragraph is two sentences that are both run-on and indirect. 

Page 50, lines 8, 10 Explain how you get HQ here- it is not clear how RfD comes to or from HQ 

unless the reader knows the stuff well. 

Page 52, line 42 See the USDA (APHIS) approach to risk assessment for exotic species (also 

NRC 2001 on Exotic Species and Agricultural Resources). 

Page 53 figure 4.1 has the double arrows that create a do-loop. 

Page 58 Section 4.2.2 See Warren-Hicks and Moore, 1999 (SETAC book- Uncertainty Analysis in 

Ecological Risk Assessment). 

Page 61 line 23 et seq is another run-on. 

Page 62 line23 et seq. This section needs to note that the following are essential in the "use of 

the assessment:" Risk Management; Public Health; Clean-up or mitigation; Siting; 

Communication; Long-term monitoring. 

Page 63, line 33 Somewhere in here the text must acknowledge religious, cultural, aesthetic and 

social values issues. Some decisions are based on the facts that these other issues are the 

overriding reasons for decision-making. We, as a society, once restored and now protect the 

bald eagles because they are an important national symbol. The Corps rejected a project because 

10 years of abuse of American Indian rights could not be undone by new analysis or procedures. 

Page 64 please give the source 

Page 89 please distinguish background and ambient. In this section, give a list of 

stressors/agents for which this issue applies now: TCDD, Hg, Pb, As, E. coli, Salmonella, etc. 

Page 91 None of these have conceptual models. 

F-15 
Review by 

David Goldsmith - George Washington University

F-16 
The draft seems to mash concepts together in order to talk about "Cumulative Risk Assessment" 

I see three areas that we must try to address and make distinct: 

1) Risks from multiple pesticide exposures 

2) Additive mixtures of toxic risks 

3) Ecological risk. 

In order to make things distinct, I believe we need to try to think critically and make the draft 

recognize that threats to health are NOT all stressors. 

A pesticide is a chemical with both beneficial and risky attributes, while stressors are much 

broader concepts of potential change in status, which can result in temporary or permanent 

health. 

For example, a fishery can come under stress when temperature rises, when larger predatory fish 

attack, or when a bolus of pesticide is illegally dumped in it. But when a human is exposed or 

overexpose to either multiple chemicals or multiple OP insecticides, that is a risk to health 

(increase in the likelihood of disease related to that specific exposure), not simply a "stressor." 

We need to be thinking about measuring dose or exposure for use in epidemiology or in 

toxicology, while in ecological risk assessment, we may not be equally able to define when a 

watershed is at risk, simply that it no longer functions to recharge groundwater, or to provide 

aquatic habitat, and thus the 'exposure and response' parameters will need better definition. This 

scenario also means we need new definitions of thresholds. 

F-17 
Review by 

Dale Hattis - Clark University 

F-18 
Dale Hattis 

Clark University 

May 14, 2002 

Premeeting Comments on the 4/23/02 Draft Framework for Cumulative Risk 

Assessment (Risk Assessment Forum, U.S. Environmental Protection Agency) 

General Impressions 

The EPA authors of this document have struggled diligently with a very amorphous problem. In 

consultation with a wide array of participants, they have included a wide variety of analytical 

enterprises that various stakeholders may wish to find shelter and respectability under the rubric 

of ACumulative Risk Assessment@. For example the things that are cumulated in a cumulative 

assessment may share a common biological mode of action (as for organophoshate insecticides) 

or they may not have the slightest commonality in this respect. The object of the exercise may 

be to produce some coherent quantitative assessment of a problem, or no risk quantification may 

be done. The assessment must focus on some set of effects of multiple stressors, but the only 

other thing that evidently needs to be shared in order to be cumulatively assessed, is the 

population exposed or affected (see box on page 9). In seeking this inclusiveness, and in parallel 

striving to avoid giving offense to various stakeholders with knowledge and experience with past 

analyses, the EPA authors have produced a document that is structured in very general terms. 

This unfortunately leads to a relatively small ratio of helpful methodological Ameat@ to 

definitional rhetoric. I think that diligent analysts trying to approach cumulative risk problems in 

the future will be better served by a document that gives them considerably more in the way of 

specific do=s and don=ts, and unobvious insights into ways to structure and illuminate their 

problems. I will elaborate on this general suggestion in my responses to the charge questions 

and the specific comments below. 

Response to Charge Questions 

1. Does the Framework adequately capture, describe and reasonably organize the key 

issues for cumulative risk assessment so as to serve as an adequate foundation for the 

development of future guidance? 

a) Does the Framework document capture the relevant terminology? 

b) Does the Framework document provide an adequate assessment structure? 

c) Does the Framework document outline the relevant methods for cumulative risk? 

Each of the parts a, b, and c above begs the question Arelevant@ to what, or Aadequate@ for what? 

Broadly, the document seems to envision that the cumulative risk assessments covered are 

F-19 
intended to be helpful for some kinds of public policy/decision-making.1 But certainly not all 

kinds of public policy decisions on environmental health and ecological risk issues are well 

served by the kind of cumulative risk assessment covered by the document. The document needs 

to address up front under what circumstances a cumulative risk assessment is in order, and under 

what circumstances the community of stakeholders and governmental actors should undertake 

some different kind of policy analysis. It is in failing to draw the contrast between cumulative 

risk assessments and other policy analysis alternatives that I think the current draft document 

fails to Acapture relevant terminology,@ Aprovide an adequate assessment structure,@ and Aoutline 

the relevant methods.@

The other kinds of policy analysis I think need to be at least briefly covered in a revised 

introduction include traditional cost-benefit analysis (Office of Management and Budget, 2000; 

Freeman, 1997), trade-off analyses (Ashford et al., 1981), priority-setting analyses (Hattis and 

Goble, 1994), or various kinds of cost-effectiveness analyses. In general, these kinds of efforts 

address more directly the needs for information helpful in decision-making whenever it is 

possible to define a coherent set of options for a decision-maker for comparative analysis of 

likely good and bad outcomes. By contrast, I would say that one might prefer to do a cumulative 

risk assessment only as a predecessor to later framing of possible options to make changes in 

emissions/exposures, and only then when the community is really unclear as to which types of 

community exposures are responsible for the greatest portion of risks of concern, and which 

present the most promising opportunities for beneficial interventions. (It is not necessarily true 

that the greatest drivers of risk will provide the most promising opportunities for intervention to 

produce public health and/or ecological benefits.) 

1

 We learn early on in the introduction (p. 2) that the particular virtue of population-based cumulative assessments 

(in contrast to the single-chemical or single-stressor assessments designed to serve the needs of Acommand-andcontrol@ regulatory decision-making in the benighted days before the enlightenment of the 1980s) is that they Awere 

much more useful to decision-makers who were dealing with public health or ecological health questions, rather than 

controlling sources of pollution.@

F-20 
References for Charge Question 1 

Ashford, N. A., D. Hattis, E.M. Zolt, J.I. Katz, G.R. Heaton, and W.C. Priest. 1981. Evaluating 

Chemical Regulations: Trade-Off Analysis and Impact Assessment for Environmental DecisionMaking. Final Report to the Council on Environmental Quality under Contract No. 

EQ4ACA35. Cambridge, Massachusetts: Massachusetts Institute of Technology, Center for 

Policy Alternatives Report No. CPA-80-13, Washington, D.C.: National Technical Information 

Service # PB81-195067. 

Freeman, M. III. 1997. Economics, Incentives, and Environmental Regulation. Chapter 9 In: 

Environmental Policy in the 1990s. Reform or Reaction? Third Edition, Edited by Norman J. 

Vig and Michael E. Kraft, Congressional Quarterly Press, Washington, D.C. 

Hattis, D., and Goble, R. L. 1994. ACurrent Priority-Setting Methodology: Too Little 

Rationality or Too Much?@ Chapter 7 in: Worst Things First? The Debate over Risk-Based 

National Environmental Priorities, A. M. Finkel and D. Golding, eds., Resources for the Future, 

Washington, D.C. pp. 107-131. 

Office of Management and Budget. 2000. Guidelines to Standardize Measures of Costs and 

Benefits and the Format of Accounting Statements. Memo from Jacob J. Lew, Director, OMB, 

for the Heads of Departments and Agencies, March 22, 2000. 

2. Does the Framework document include any scientific or technical inaccuracies in its 

presentation of terminology, assessment structure, and methods? Does it adequate convey 

the state of the science with respect to currently available methods/approaches and the 

areas that are in need of further research and development? 

I think the document can do better in this respect. For example on page 8 the document 

mixes together a discussion of what might be called Abottom up@ assessmentsCin which a 

combined effect is estimated as the predicted aggregation of the effects of several different 

stressorsCwith Aa physician=s use of a model, derived empirically from epidemiological studies, 

to estimate the probability of a woman=s developing breast cancer over the next ten years.@ For 

convenience, I will call the second kind of analysis Atop down@ because it derives the influence 

of various stressors/@risk factors@ from actual observations of a Atop@ effect of concernCoften in 

careful prospective studies with extensive opportunities to statistically control for a variety of 

potential confounding factors in addition to those factors that can be affected by environmental 

health policy interventions. 

F-21 
The methodology for this latter type of cumulative assessment is very different from the 

Abottom up@ assessments, and deserves extensive separate treatment. Indeed, many of the most 

important current analyses of environmental effects take the form of such empirical 

epidemiological studies using multiple regression techniques for analysisCincluding the 

mortality and morbidity effects of airborne particulates, among other air pollutants, to name only 

the most prominent example. The versatility of the technique to tease apart contributions of 

different sources of environmental exposures of interest is illustrated by a recent paper by 

Harvard researchers on the association of particulates from different sources for short term 

mortality changes.2 

There are also some notable pitfalls of the Atop down@ assessments. These are most 

importantly in the form of the equations used to represent the influences of various factors on the 

measured outcome variable(s). These of course need to be causal predictive models, however 

the tradition of the field is simply to derive the most parsimonious Abest fit@ equations that satisfy 

prevailing statistical criteria. One example that I noticed over twenty years ago was that the 

multiple logistic risk equations used to analyze the chronic prospective heart disease studies such 

as Framingham used an equation that made no distinction between possible contributions of risk 

factors (e.g., blood pressure, diabetes, etc.) to the chronic process of atheroscleorsis, vs the acute 

events that are responsible for precipitating observed clinical events such as heart attacks and 

strokes. The foundational paper for these analyses (Truett et al., 1967) goes into extensive detail 

to address statistical issues, but says not one word about why one should believe that their 

generic multiple logistic risk equation is a good causal model of the processes leading to adverse 

cardiovascular outcomes, and the interactions of different risk factors in contributing to the 

adverse events (e.g. contributions of diabetes and smoking to myocardial infarctions), let alone 

give analysts a way to distinguish between contributions to acute precipitating evens vs chronic 

causal processes. 

This subject is worthy of further treatment in the document. This in an important mode of 

analysis for some of the major environmental health hazards known at present. And in addition 

it has considerable potential to be applied in combination with new biomarkers of early response 

to help sort out and quantify the contributions and interactions of multiple stressors. Biomarkers 

represent natural integrators of the influence of multiple environmental and background factors 

on adverse outcomes. Such continuous biomarkers that I think are useful as dependent variables 

for use in future study include birth weights (Hattis, 1998), sperm count and other sperm quality 

parameters (Hattis, 1998), heart rate variability parameters (Pope et al., 1999), measures of lung 

function, and blood coagulation parameters such as fibrinogen (Schwartz, 2001). 

References for the Question 2 Response 

Hattis, D. AStrategies For Assessing Human Variability In Susceptibility, And Using Variability 

To Infer Human Risks@ In Human Variability in Response to Chemical Exposure: Measures, 

2

 They find that a 10 Fg/m3 exposure to mobile source PM2.5 is associated with a 3.4% increase in daily mortality 

(95% confidence interval 1.7-5.2%), in contrast to the smaller 1.1% response indicated for coal combustion PM2.5 

particles particulates (95% confidence interval 0.3% - 2.0%) and no detected response to PM2.5 of crustal origin. If 

confirmed, this is of considerable importance in directing efforts to control the particles from sources with the 

greatest health impacts. 

F-22 
Modeling, and Risk Assessment, D. A. Neumann and C. A. Kimmel, eds., CRC Press, Boca 

Raton, FL, pp. 27-57, 1998. 

Laden, F., Neas, L. M., Dockery, D. W., and Schwartz, J. (2000) Association of fine particulate 

matter from different sources with daily mortality in six U.S. Cities. Environmental Health 

Perspectives 108: 941-947. 

Pope, C. A. 3rd, Verrier, R. L., Lovett, E. G., Larson, A. C., Raizenne, M. E., Kanner, R. E., 

Schwartz, J., Villegas, G. M., Gold, D. R., and Dockery, D. W. (1999). AHeart rate variability 

associated with particulate air pollution. Am Heart J 138:890-899. 

Schwartz, J. (2001) Air pollution and blood markers of cardiovascular risk. Environ Health 

Perspect. 109 Suppl 3: 405-409. 

Truett, J., Cornfield, J., and Kannel, W..1967. A multivariate analysis of the risk of coronary 

heart disease in Framingham. J. Chron. Dis. 20: 511-524. 

3. Does the Framework adequately characterize the importance of uncertainty analysis in 

cumulative risk assessment? What additional discussions of uncertainty should be 

included, and where in the document? 

It seems to me that the document provides only the most cursory treatment of uncertainty 

analysis. There is considerable that could be said about the specific uncertainties that arise in 

cumulative risk analyses, for example the uncertainties that arise in estimating Arelative potency 

factors,@ for organophosphates or TEFs for agents thought to act via the dioxin receptors. Some 

of these uncertainties can be usefully partitioned between (1) the basic uncertainty in the dose 

response relationship for the reference chemical, and (2) the uncertainty in estimating the relative 

potencies of the other chemicals in the common mechanism group in relation to the reference 

chemical, and (3) the uncertainties arising from the fact that members of the common mechanism 

group may have other modes of action that are not fully captured via the common-mechanism 

potency calculation. 

4. Comment on the adequacy and accuracy of the Framework=s presentation in each of 

these technically difficult areas: 

a) Vulnerability 

Of the four components mentioned, I think only two are really neededCsusceptibility or 

sensitivity, and exposure. The third, preparedness to withstand the stressor is I think completely 

redundant with susceptibility, and the fourthCdifferential ability to recoverCI think should be 

treated by having multiple endpoints (some perhaps downstream of each other) to capture the 

duration and secondary consequences of any initially induced effects. 

F-23 
b) Cumulative Risk Assessment Involving Chemical and Non-Chemical Stressors 

The Atop down@ biomarkers methodology discussed in my response to question 2 I think is a 

useful addition to the current discussion of this issue in the document. Biomarkers like birth 

weight naturally integrate the effects of all kinds of conditionsCincluding individual medical 

conditions; differences in exposures and other factors associated with prevailing racial 

categories; socioeconomic factors, etc. I have recently done an analysis of this kind in the 

context of a community military facility (Hattis, 2001): 

Hattis, D. (2001) AHow have Birthweights in Massachusetts Changed from 1990-1997 and 

Why?@ Appendix C in Ryan, L., Schwartz, J., Goble, R., and Hattis, D. Technical Advisors= 3rd 

Report (2000-2001). Research on Health and Environmental Exposures. Massachusetts Military 

Reservation Harvard University and Clark University, 2001. 

c) Cumulative Risk Assessment Involving Different ATypes@ of Risk 

I think uses of common metrics can have some value but only when it is accompanied by (1) 

accommodating those people who may not completely buy the relative tradeoff valuations used 

in the common index by providing the disaggregated effects (e.g., mortality, various kinds of 

morbidity, disability and/or loss work days and days of restricted activity, and (2) good attention 

to equity issues by breaking down the relative burdens on different subpopulations if there are 

appreciable differential effects. 

Specific Observations (other than those covered in the comments above) 

p. ci, 3rd par, 2nd line. The goal is stated here as Areducing@ uncertainty. I think that as a general 

matter, a good analysis will Aclarify@ uncertainties but not generally Areduce@ stated uncertainties 

from naïve statistical calculations. 

p. 8Chere is where a strong distinction needs to be made between Abottom-up@ and Atop down@

analyses of cumulative effects (see above discussion). 

p. 9, last paragraph. Some more detailed description of the Bogen (2001) would be helpful, 

perhaps as an appendix. The document repeatedly refers to this work, but it is not readily 

available, so the reader has very little clue to what exactly is being cited. 

p. 10, line 28CI would refer to risk Aanalysis@ rather than risk Aassessment@ here, in order to 

preserve Aassessment@ as a relatively technical exercise. 

p. 16 (bottom)- p. 17Chere is where I would describe an important goal as to identify options for 

control/abatement of hazards/risks. This also deserves emphasis in the box on page 23 

p. 31 (box) I would say that distributions of hazard index values are not the only option for noncancer effects. Analyses for criteria air pollutants, for example, have done extensive 

quantification of non-cancer effects, largely from epidemiological data. In addition, I have 

recently proposed an alternative to the standard RFD procedure that I think provides a feasible 

F-24 
starting point for quantitative risk assessment for more general non-cancer effects. See, for 

example: 

Hattis, D., Baird, S., and Goble, R. AA Straw Man Proposal for a Quantitative Definition of the 

RfD,@ in Final Technical Report, U.S. Environmental Protection Agency STAR grant # 

R825360, AHuman Variability in Parameters Potentially Related to Susceptibility for Noncancer 

Risks,@ Paper presented 4/24/01 at the U.S. EPA/DoD symposium on Issues and Applications in 

Toxicology And Risk Assessment, Fairborn, Ohio. Full version available on the web at 

http://www2.clarku.edu/faculty/dhattis; shortened version Drug and Chemical Toxicology, in 

press. 

Hattis, D., Banati, P., and Goble, R. ADistributions of Individual Susceptibility Among Humans 

for Toxic Effects--For What Fraction of Which Kinds of Chemicals and Effects Does the 

Traditional 10-Fold Factor Provide How Much Protection?@ Annals of the New York Academy 

of Sciences, Volume 895, pp. 286-316, December, 1999. 

Hattis, D. AStrategies For Assessing Human Variability In Susceptibility, And Using Variability 

To Infer Human Risks@ In Human Variability in Response to Chemical Exposure: Measures, 

Modeling, and Risk Assessment, D. A. Neumann and C. A. Kimmel, eds., CRC Press, Boca 

Raton, FL, pp. 27-57, 1998. 

These references could also be cited and/or discussed at the top of page 36. Also because the 

first two are examples of the simultaneous treatment of uncertainty and variability, they could be 

mentioned on p. 45 lines 14-19. 

p. 38, line 32. The document needs to distinguish the concepts of individual threshold doses 

(does that will not affect a single person in a particular way) and population thresholds (doses 

that will not affect anyone in a large mixed population of people with different individual 

thresholds.) 

p. 45, lines 35-36. I don=t agree that it is generally adequate to characterize only risks to people 

who are Areasonably maximally exposed@. A population approach is generally much better 

because helps managers face the societal aggregate consequences of their actions while also 

illuminating the full distribution of expected risks among diverse people. 

p. 48, line 22. I disagree that the state-of-the science is not yet advanced enough to allow 

biomarkers such as birth weight and some others a place in current risk analyses. 

p. 59, lines 14-18. You should also give other examples of the simultaneous treatment of 

variability and uncertainty, such as my work on cancer risks from genetically acting agents: 

Hattis, D. and Barlow, K. AHuman Interindividual Variability In Cancer Risks--Technical And 

Management Challenges@ Human and Ecological Risk Assessment, Vol. 2, pp. 194-220, 1996. 

p. 59, line 36. Amost probable estimates@ of cancer risks are almost never of primary interest. 

Even if you want to do cost benefit analysis it preferable to use arithmetic mean population 

Aexpected value@ estimates, which can differ from Amost probable@ estimates by many fold. See: 

F-25 
Hattis, D., and Goble, R., "Expected Values for Projected Cancer Risks from Putative 

Genetically-Acting Agents," Risk Analysis, Vol. 11, pp. 359-363, 1991.

F-26 
Review by 

Joann Held - New Jersey Department of Environmental Protection 

F-27 
Written Comments on the Draft Framework for Cumulative Risk Assessment 

Submitted by Joann Held, NJDEP 

General Impressions 

The draft document is well written and thorough (with the exceptions described below). I intend 

to make it required reading for all members of the workgroup that my Agency is presently 

forming to oversee a cumulative risk assessment in South Camden, NJ. 

The introductory section does a good job of saying what the Framework is and is not intended to 

be. There should also be a discussion of the future of the Framework. Is it a living document? 

Will there be future editions and/or updates? 

The example boxes have been well chosen to illustrate the points being made in the text and 

should be especially helpful to nontechnical readers. 

Breaking down the Risk Assessment into 3 phases (Planning, Scoping & Problem Formulation; 

Analysis; Risk Characterization) is a useful way to organize the document. And representing the 

phases with a logo is an interesting approach. However, you never explain why the arrow in the 

logo flows in both directions. Is this to indicate some type of iterative approach, where you 

could get to the Risk Characterization and then find that you need to reformulate the problem? 

Or is it simply to imply that the 3 phases are interrelated and you need to keep the other phases 

in mind as you work on each one? 

Appendix A regarding Research and Development Needs is helpful, but there should be a 

discussion in the Introductory Chapter or at the end of each of the Phase Chapters regarding 

development of these new tools. Who is responsible for doing this? How will we learn about 

them? Will there be a series of follow-up reports after this Framework document that tell us how 

EPA, et al are doing at filling in the gaps? 

Some discussion regarding the following topics should be included in the Analysis Phase 

discussion in the Framework document. 

1. The times when using an iterative approach might be helpful, e.g. screening risk assessment 

to focus on a smaller set of chemicals of concerned followed by a more refined risk 

assessment using the shorter list. 

2. The complimentary role of monitoring and modeling data, especially the complexities of 

using data derived by both means in the same analysis. 

3. Special problems related to ecological risk assessment. 

Response to Charge Questions 

Scoping Phase: The terms “Routes” and “Pathways” are used often in this part of the report. The 

distinction between the two is not completely clear from the way they are used in the text. They 

should be defined more completely somewhere in this section and also added to the Glossary. 

F-28 
Analysis Phase: General Comments 

1. There should be a step built into the early part of this phase for data collection. Where are 

the data coming from? Do they already exist? Will there be literature searches? Ambient air 

monitoring? Blood testing? Adequate time must be set aside in the analysis plan to collect 

this information. 

2. The Analysis Plan should also include a schedule showing how long each step will take and 

what efforts can be carried out concurrently. 

3. Would it be reasonable to add pregnant women to the list of susceptible populations (p. 35 1st 

paragraph)? This group would be especially important when stressors in the assessment are 

likely to result in adverse developmental effects. In my program we often shorten the 

exposure timeframe of interest if we are concerned about the pregnant population. 

Analysis Phase: Available Methods & Approaches 

I have seen an approach suggested for predicting risk of exposure to multiple noncarcinogenic 

chemicals which might be of use here, but I have never seen it used in a final assessment. I 

wonder if any of the other Peer Reviewers have seen it and might recommend that it be described 

in the Framework. The basic idea is that although Reference Concentrations have been set for a 

single endpoint or critical effect, many times the substance in question will have evidence of less 

critical effects at higher concentrations. For these less critical effects, something like a 

Reference Concentration (call it a pseudo RfC) could be developed. Then when Hazard 

Quotients are being added to get a Hazard Index for each target organ or critical endpoint, these 

additional health endpoints could be considered. For example, the RfC for methyl ethyl ketone 

is based on fetotoxocity. However, it also has irritant effects. If it is included in a cumulative 

risk assessment with a number of other irritants, a pseudo RfC for MEK could be developed so 

that this effect would not be lost in the analysis. 

Analysis Phase: Decision Indices 

On p. 44, it is noted that the actual value of HI is not that informative all by itself. It might be 

helpful to elaborate by giving as an example Reference Concentrations (or Reference Doses). 

These health benchmarks have uncertainty factors (UF) that can vary from about 3 to 1000. If 

the UF is only 3, then an exposure prediction that is just a little bit above the RfC is more 

interesting than it would be for a pollutant with a UF of 1000. 

Analysis Phase: Areas of Complexity 

The discussion of Qualitative Approaches for comparing across stressors in Section 3.3.4 is very 

helpful. Without this section, the full scope of cumulative risk assessment envisioned by the 

framework could not be achieved; we would be limited to evaluating only those risks that we 

can quantify. 

Risk Characterization Phase: Uncertainty 

1. It is important not to let the Uncertainty Analysis overwhelm the Risk Characterization 

Phase. Some balance must be maintained in preparing the risk characterization in order to 

keep the uncertainty analysis from paralyzing the project (as per the EEA lesson “Avoid 

paralysis by analysis...”) 

2. Discuss the acceptability of an uncertainty analysis that simply identifies the assumptions 

that would lead to an overestimate of risk and those that would lead to an underestimate, and 

F-29 
tries to draw conclusions about which side of the scale the risk assessment is likely to be on. 

3. The Framework might recommend that some standard statements of uncertainty be 

developed as a starter set for use in cumulative risk assessment. For example, the uncertainty 

associated with Unit Risk Factors (or slope factors) for known human carcinogens vs 

probable carcinogens vs possible carcinogens could be described in general terms in one 

place that we could all access and then it could be elaborated on as necessary within a given 

risk assessment. 

4. For Reference Concentrations (and Reference Doses) is it sufficient for the uncertainty 

analysis to simply review the uncertainty factors that were selected for each RfC? 

Risk Characterization: Risk Description 

Could there be a clearinghouse of studies related to combined risk of exposure to two or more 

substances such as those described by Kodell and Chen (1994) and referenced in section 4.2.3?

Perhaps this could be a recommendation of the Framework or could be added to Appendix A. 

Specific Observations 

Chapter 3 

In addition to the changes to the Section headings that I suggested during the Peer Review 

session, it would be helpful to have some transitional sentences at the beginning of each 

subsection introduces the content of that section. For example, at the beginning of Section 

3.2.1.1, it could be stated that some aspects of estimating risk could be more complex in the 

context of a Cumulative Risk Assessment. These aspects include: variation of exposure with 

time, vulnerability related to differential exposure, and subpopulations with special exposures. 

Page 34, Text Box: Some Examples of Exposure Models Which Consider Time Aspects 

There is a typo in the second to last line. I believe that the acronym for the American Crop 

Protection Association should be ACPA. 

Section 3.2.2.3 Decision Indices 

a) Could you define “high dimensional matrix” and “high dimensional graphical presentation” 

or replace them with a term that is more commonly used? By high dimension I assume that 

you mean more than two. Is that correct? 

b) The discussion of severity categories (p. 44, 3rd paragraph) is not clear. Is this the same as 

ranking exposures as High/Medium/Low and then doing a statistical analysis of all rankings 

for all the stressors in the analysis? 

F-30 
Section 4.2 Uncertainty Analysis 

Three of the Morgan & Henrion “Ten Commandments” are highlighted in this section, but the 

numbering in the text doesn’t seem to match the numbering in the box. As I read it the 

discussion is about: 

#4. Identify all significant assumptions 

#6. Be explicit about uncertainties 

#7. Perform systematic sensitivity... 

The text, however, refers to # 6, 7, and 8. 

Page 60, Line 30: There is a word missing. The line should probably read: 

“it should be transparent...” or “it must be transparent.” 

6. REFERENCES 

USEPA (2001d): The web address is wrong. I think it should be the following. Note that 

watershed is singular. http://www.epa.gov/owow/watershed/lessons/top10.pdf

APPENDIX D: EXAMPLES OF ANALYSIS PLANS 

The Framework should make it clear that these are outlines of analysis plans. 

F-31 
Review by 

Paul Locke - John Hopkins University 

F-32 
EPA’s Draft Framework for Cumulative Risk Assessment 

Pre-Meeting Comments 

Paul A. Locke 

Visiting Scholar 

Johns Hopkins Bloomberg School of Public Health 

Baltimore, Maryland 

30 May 2002

General Impressions 

• The Framework would be strengthened if it contained a comprehensive 

discussion of when and how Cumulative Risk Assessment (CRA) should be 

used. 

• It would be helpful to include additional information about how to integrate 

traditional health risk assessment (as outlined in the 1983 Red Book) and 

ecological risk assessment (as set out in EPA’s guidance documents) with CRA. 

• The population focused approach described in the Framework seems to be 

appropriate for CRA and has the potential to provide focus to the CRA process. 

It would be very useful to include additional guidance about how to choose the 

“right” population(s). 

• The Framework document should more fully explain the differences between the 

CRA called for in the Food Quality Protection Act (FQPA) (with mode of action as 

the trigger) and the CRA outlined in the Framework (which is broader in 

approach and scope). 

• It appears that the conceptual model building exercise is central to the CRA. It 

will define the scope of the RA and drive the questions that are pursued. 

Accordingly, it is probably worthwhile to expand the discussion of this step. 

• Throughout the Framework, but especially in the analysis phase, there is a need 

to spell out when default assumptions could be applied and how to apply them. 

• The section about combining risks should be substantially redrafted. It now 

begins with a focused discussion about some of the methods available for 

cumulating risk metrics and methods to measure the impact of chronic disease. 

Instead, it should first cover the threshold question of whether it is even feasible 

to attempt such a combination. In other words, a preliminary, broader discussion 

should first examine the question of whether risks can even be combined in such 

a fashion. This discussion should be followed by an analysis of ways in which 

qualitative approaches can be applied to combining risk. Finally, quantitative 

approaches (such as HQ and TEF) should be discussed followed by an 

introduction to QALYs and DALYs. Given what precedes this section, it seems 

important to recognize that quantitative approaches may not be available, and 

that qualitative risk combination might be the most appropriate method of thinking 

about risks. 

• Throughout the document it would be useful to emphasize that it is sometimes 

better to lay out the information descriptively in a narrative form and recognize 

that, because of analytical limitations, it may not be amenable to quantification. 

F-33 
• The risk characterization section should emphasize transparency and full 

disclosure. In other words, items should not be dropped from the conceptual 

model merely because they cannot easily be reduced to a summary risk metric. 

The Framework indicates that the CRA document is a decision-making document 

and it is therefore important that all relevant information be provided to the 

decision makers. 

• The Framework should include a fuller discussion about public health 

surveillance, epidemiology, chronic disease endpoints and health tracking, and 

encourage the use of available human disease endpoint data. 

Response to Charge Questions 

1. Comment on whether the Framework adequately captures, describes, and 

reasonably organizes the key issues for cumulative risk assessment so as to 

serve as an adequate foundation for the development of future guidance. In 

your comments, please address each of the following questions. In 

answering each question, provide a supporting discussion that highlights any 

areas of the Framework that may need to be clarified and relevant topics that 

may be missing from the current Framework document. Include references to 

any published literature that could help improve the completeness and clarity 

of the Framework. 

I think the Framework is a good start and incorporates a useful approach to 

cumulating risks. It is heavily slanted to human health risks, and could benefit from a 

fuller discussion of ecological risks and the ecological risk assessment paradigm. 

a. Does the Framework document capture the relevant terminology? 

As far as I can tell, the Framework seems to capture a substantial body of the 

relevant terminology. It could benefit from a fuller discussion of biomarkers (one that 

covered the full range of biomarkers, including biomarkers of susceptibility and markers 

of early disease) and an expanded section on public health surveillance, epidemiology 

and chronic disease tracking. 

b. Does the Framework document capture an adequate assessment 

structure? 

Although there are pieces of the structure that require clarification, I believe that 

the structure presented in the framework is adequate. I think two of the strengths of the 

Framework are its flexibility and its broad scope. 

F-34 
c. Does the Framework document outline the relevant methods for 

cumulative risk? 

The Framework presents a workable context for CRA, and allows flexibility so 

that a number of relevant methodologies can be applied. In some places, additional 

guidance and discussion would be useful. For example, I think the Framework should 

include a fuller discussion regarding the application and use of default assumptions, and 

qualitative methods for combining or cumulating risk. 

2. Does the Framework document include any scientific or technical 

inaccuracies in its presentation of terminology, assessment structure, and 

methods? Please identify any problem areas and propose revisions or other 

actions that will result in a scientifically sound and supportable discussion. 

With respect to methods, comment on whether the Framework adequately 

covers the state of the science with respect to currently available cumulative 

risk assessment methods/approaches and the areas that are in need of 

further research and development. 

As discussed above, there are certain parts of the Framework that I feel would 

benefit from an expanded discussion and sharper focus. For example, it would be 

useful to include a fuller description of biomonitoring and biomarkers, human health 

surveillance data (chronic disease tracking), epidemiology and ecological stressors. 

(Please refer to the specific comments contained in this document). 

3. Uncertainty analysis is an important aspect of risk assessment (and policy 

analysis in general). However, historically, dealing with uncertainty has been 

a shortcoming of many assessments. Cumulative risk assessments present 

new challenges for uncertainty analysis. For example, assessing cumulative 

risks will involve combining data of varying quality. Perhaps more important, 

assessing cumulative risks will involve the use of “soft” assumptions. These 

are assumptions which may have a high degree of uncertainty that is difficult 

(or not possible) to quantify. Comment on whether the Framework 

adequately characterizes the importance of uncertainty analysis in cumulative 

risk assessment. What additional discussions of uncertainty should be 

included in the Framework (and in what sections of the document)? 

I think that the issue of uncertainty is most effectively examined in the context of 

the specific CRA. Methods exist for describing certain uncertainties in traditional, single 

substance risk assessments; it would be useful to reference them in the Framework in a 

text box. (The text box on page 59 provides an example of where uncertainty analysis 

was applied, but it does not talk about uncertainty methodology.) These should be 

applied, if possible. 

In some situations, it is likely that uncertainty will be amenable only to a narrative 

description. I believe that the Framework should explicitly acknowledge that some 

uncertainty in the CRA process is likely to be unquantifiable, but can nevertheless be 

described in narrative form. The Framework should acknowledge that in such cases it 

F-35 
is still appropriate to move forward with the CRA, as long as these uncertainties are 

transparent to the decision makers. Information should not be lost or abandoned 

because a quantitative methodology to explain attendant uncertainty is not available. 

4. The following topics have been identified by the RAF technical panel as 

technically difficult areas that will pose challenges to cumulative risk 

assessment. Comment on the adequacy and accuracy of the Framework’s 

presentation in each of these areas. 

a. Vulnerability – As applied to cumulative risk assessment, it is useful to 

think of four components to vulnerability: the susceptibility or sensitivity of 

the human or ecological receptors; the differential exposures of the 

receptors; the differential preparedness of the receptor to withstand the 

insult from exposure; and the differential ability to recover from these 

effects. The issue for cumulative risk assessment is how to consider 

these aspects of vulnerability and their potential impacts on risk. This is 

highlighted in the Framework as an issue in need of further research and 

development. Comment on the discussion of vulnerability in the draft 

Framework. Has the state of the science been captured in this 

discussion? How can the discussion of the issue be improved? 

I think that the Framework would benefit if at least a portion of the vulnerability 

discussion were put in the context of choosing the target population. For example, 

assume that a CRA was carried out for a target population that was made up of a 

significant portion of children, or of the infirm elderly, or of immune compromised 

individuals. In these cases, the vulnerability question is “built into” the choice of the 

target population. The Framework might benefit from a text box that addresses the 

question “Should populations with specific vulnerabilities be targeted for CRAs?” 

b. Cumulative Risk Assessment Involving Chemical and non-Chemical 

Stressors – Viewing cumulative risk assessment as an evaluation of the 

accumulation of stressors presents many challenges. These may be seen 

when attempting to combine, in some meaningful way, the risks from 

multiple chemicals that may act as synergistic, antagonistic, or additive 

doses leading to a single effect. The situation is exacerbated when nonchemical stressors (e.g., radiation, biological agents, and psychological 

stress) are considered. Comment on the Framework’s discussion 

concerning the combining of disparate environmental stressors. In 

commenting, consider the state of the science with respect to 

understanding the effects of different stressors acting together (e.g., 

chemical exposure and viral infection). What can be added to the 

Framework to adequately convey the state of the science in this area? 

As I mentioned earlier in this document, one of the strengths of the Framework is 

its flexibility. The Framework can also be seen as a “technology forcing” approach, in 

that it will encourage the asking of scientific and policy questions for which we do not 

yet have answers. The aspect of cumulating disparate risks – the topic of this question 

F-36 
and the next one – is one place where the Framework is likely to stimulate the posing of 

hard questions that will push risk assessment methodology (and data collection) 

forward. We must not forget that this could be a very good thing. 

Although the approach taken in the Framework is a good start, I think that the 

Framework should address the issues of cumulating and combining risks in a broader 

fashion, as I explained above. Two threshold questions -- (1) Whether risks should be 

cumulated and (2) How best to combine and cumulate them -- should be addressed 

first. 

c. Cumulative Risk Assessment Involving Different “Types” of Risk – 

Conveying the combined risks from multiple chemical and non-chemical 

stressors, in a meaningful way, is the ultimate challenge for cumulative 

risk assessment. Experience in this area is extremely limited. Indices, 

common metrics (e.g., Disability Adjusted Life Years – DALYs) and 

graphical (e.g., GIS) approaches have been explored but much methods 

development work remains to be completed. Cumulative risk assessment 

can be a valuable part of the decision making process, but only if the 

results are conveyed in a meaningful way. Comment on the Framework’s 

discussion concerning the combining of disparate measures of risk. Do 

the example approaches discussed capture the state of the science in this 

area? In particular, consider the role of valuation (i.e., the assignment of 

societal values to disparate health outcomes) implicit in some of the 

approaches. Suggest changes that may improve this discussion. 

Please refer to my comments set out above under question 4(b). 

Specific Observations 

• The biomarkers and biomonitoring section should be strengthened. A fuller 

discussion of biomarkers and biomonitoring needs to be added (beyond the 

discussion of markers of exposure). In addition, a discussion of CDC’s NHANES 

program and NIEHS efforts in this area should be added. 

• The discussion on page 48, and citations that accompany it, should be updated. 

The Pew Environmental Health Commission reports and materials are now 

available at the website of Trust for America’s Health (healthyamericans.org). In 

addition, this discussion should expand upon the potential uses of chronic 

disease tracking information, epidemiology and public health surveillance in CRA. 

• The Framework should make it clear when it is appropriate to reduce a series of 

risks or risk measures to a common metric. (See §3 generally, especially 

§§3.3.3 and 3.3.3.1). The Framework should add a detailed discussion about 

when combining risks using a quantitative method is appropriate and 

inappropriate. In every case, the underlying data should be provided. 

• In Section 4.3.2, a much broader discussion of the public health context should 

be provided. It could include an examination of the precautionary principle. 

Some users of CRA might also want to adopt the precautionary principle as a 

F-37 
guidepost for decision making. 

• The appendix on background exposures should examine the policy implications 

of how such exposures are addressed. One default approach could be to 

assume that any risks posed by background should be added to, or cumulated 

with, similar risks and that human-made exposure risks should be controlled first. 

For example, if a person is exposed to naturally occurring radiation such as 

radon in her home, and also receives radon exposure from a source in her 

community, such as a uranium mill tailings pile, one approach would be to “add” 

the radiation risks from these two sources and focus on controlling the risks from 

the human-made source first. Similarly, it could be acknowledged that human 

created “background” risks, if they are significant, need to be reduced. 

F-38 
EPA’s Draft Framework for Cumulative Risk Assessment 

Post-Meeting Comments 

Paul A. Locke 

Visiting Scholar 

Johns Hopkins Bloomberg School of Public Health 

Baltimore, Maryland 

2 August 2002 

Overview 

I am submitting these post-meeting comments to supplement the pre-meeting 

comments (dated 30 May 2002) that I submitted previously. 

References 

I have provided an annotated bibliography along with these comments. I hope 

that EPA can consider these sources in re-drafting and amending the Cumulative Risk 

Assessment (CRA) framework. 

General Impressions 

I concur with the definition, approach and scope of the CRA framework that EPA 

introduced at the meeting. I believe that the most important points are: 

• The CRA framework is a document that seeks to learn from on-gong EPA 

experiences and offer a flexible strategy for carrying out a CRA. It is not a 

guidance document in that it does not provide detailed technical guidance. 

Nevertheless, it will serve as the basis for developing future guidelines. 

• The goal of a CRA is to formulate, address and hopefully answer questions about 

multiple harms acting together. 

• The CRA approach is a “community-first” approach in that it starts by identifying 

an impacted or potentially impacted population or community, its public health 

problems and health status. It then looks at the risks the community confronts 

that may be related to its health status. I think that this is one of the most 

important points to emphasize in the CRA framework. 

• CRA has methodologic limitations, and it is important to identify them and 

discuss them transparently. Still, limitations in methodology should not be seen 

as reason for ignoring or not posing questions for which communities seek 

answers. 

F-39 
Additional Specific Comments (organized by CRA Framework section) 

Section 2.2.2 -- Conceptual Model. This section of the report could be expanded to 

include additional information about ecosystems and ecosystem stressors. (I believe 

that other commenters will discuss this issue in more detail.) 

New section 2.4 – From Scoping to Analysis. Consider adding a new section here that 

discusses the transition between these two areas, and focuses in particular on any 

questions posed in the scoping phase that cannot be addressed in the analysis phase. 

(Lack of an appropriate methodology is one reason that certain questions cannot be 

addressed.) 

Section 3.3.3.1 – Creating a Common Metric. In this section, and in other parts of 

chapter three, it would be very useful to describe qualitative as well as quantitative ways 

of cumulating or bundling risks. 

Section 3.3.2 -- Biomonitoring and Biomarkers. This is an area where some additional 

work needs to be done. In particular, the CRA Framework should discuss the ENTIRE 

SPECTRUM of biomarkers, including biomarkers of exposure, effect and susceptibility. 

As it currently stands, the CRA Framework focuses on biomarkers of exposure. As 

discussed at the meeting, the CRA Framework should also discuss the limitations of 

biomarkers and biomonitoring. 

Section 4.2.3 – Uncertainty and Risk Addition. This section is incomplete because it 

only discusses the possible issues associated with over estimating risks. It should 

begin with an explanation of why we use conservative risk estimates (to be public health 

protective) and the possible ways that we can underestimate risk. Risk addition should 

be presented more fully. It is possible that it can overestimate risk; it is also possible 

that it underestimates or mischaracterizes risk. 

Section 4.3.2 – Cumulative Risk Assessment in a Public Health Context. This section 

should be written from the perspective of the population at risk. It should emphasize that 

public health protection is a key feature of cumulative risk. As currently drafted, it is 

overly negative. It should focus on the potential of cumulative risk assessment in 

improving the public health information that will be available to communities. 

New Section 4.5 – Communicating Results of the Assessment. Communities will vary 

substantially, and have different communication needs. As one commenter pointed out, 

it is essential that EPA use culturally appropriate language and methods. This new 

section should describe some of the “basics” of risk communication, and also provide 

information about the need to be sensitive to cultural and ethnic differences. 

F-40 
Annotated Bibliography 

1. CDC NHANES – http://www.cdc.gov/nchs/nhanes.htm. The NHANES data 

and its uses and limitations should be discussed in the CRA Framework. One 

of the purposes of NHANES is to collect appropriate biomonitoring data to 

develop reference ranges of compounds. These reference ranges could be 

useful for communities concerned about their exposure to certain compounds, 

and could help answer the question “Is my exposure ‘normal?’” 

2. Environment and Human Health, Inc. Children’s Exposure to Diesel Exhaust 

on School Buses (2002). This report is available for downloading at 

www.ehhi.org. The report summarizes the results of an ambitious study 

(undertaken in connection with Yale University) that looks at children’s 

exposure to diesel fuel on and from school buses. I think it is a good example 

of an aggregate exposure analysis and the type of study that communities 

interested in asthma and impaired lung function would find compelling. 

3. American Lung Association. Urban Air Pollution and Health Inequities: A 

Workshop Report. Environmental Health Perspectives 109 (Suppl. 3) June 

2001 (available on-line (if you subscribe to ehp on-line) at 

http://ehp.niehs.nih.gov/. This workshop focuses on the disproportionate 

impact of air pollution on certain populations, and is useful in showing why 

and how a population-based approach is important. 

4. Hulka, Barbara S., Wilcosky, Timothy, and Griffin, Jack. Biological Markers in 

Epidemiology. (Oxford University Press, 1990). This is a somewhat old, but 

still useful, fundamental text on biomarkers. It is written from the perspective 

of molecular epidemiology, and explains different types of biomarkers and 

how biomarkers can be used. More recent texts (such as the one written by 

John Groopman) are also useful. 

5. Institute of Medicine. The Future of Public Health (National Academy Press, 

1988). This is an essential text in public health and is useful for the CRA 

framework in at least two ways. First, it discusses the bifurcation of public 

health and environmental protection (and health) (see pages 110 – 111), a 

problem that the CRA framework can directly address. Second, it sets out the 

core functions of the government in public health (pages 43 – 47), which 

should be referenced in the CRA Framework’s discussion of public health. 

6. Michael Siegel and Lynne Doner. Marketing Public Health (Aspen Publishers, 

Inc. 1998). This is a useful text, especially for formulating a strategy for 

communicating public health messages such as the information produced in a 

CRA. 

F-41 
7. Chris F. Wilkinson. Cumulative Risk Assessment: The State of the Science 

(Draft). Paper presented at the ILSI Cumulative Risk Assessment Workshop, 

September, 1998. This paper could be valuable in helping to describe 

common metrics and risk additivity. It goes through a series of examples 

using the TEF, MOE and HI, showing their similarities and differences. (I 

don’t know if this paper was ever finalized. I would be glad to provide a copy 

to interested parties.) 

F-42 
Review by 

Margaret MacDonell – Argonne National Lab 

F-43 
U.S. Environmental Protection Agency, Risk Assessment Forum 

Framework for Cumulative Risk Assessment 

EPA/630/P-02/001A, External Review Draft, April 23, 2002 

Peer Review 

Margaret MacDonell 

Argonne National Laboratory, Environmental Assessment Division

General Impressions 

This document represents a major step forward for the field of environmental risk analysis, 

which until now has been rooted in single-chemical approaches that have ineffectively addressed 

real-world conditions and community concerns. The report provides an excellent overview of 

the key challenges involved in assessing cumulative risks and lays out a sound framework for 

addressing those challenges. It is very well organized and well written, and it answers a critical 

need at a time of increasing public attention on multiple sources, stressors, exposures, and 

effects. It is also noteworthy for the many complex topics discussed and inclusion of illustrative 

examples. The report is impressive for its broad-based flexibility designed to accommodate a 

variety of applications, including those extending beyond the regulatory purview of the agency. 

Through this general umbrella communities will have a means of filling in the gaps created by 

the piecemeal coverage of existing legislation, to incorporate in their risk assessments those 

elements with potentially significant impact that could otherwise fall through the cracks. 

This framework represents a remarkable achievement not just for its advancement of the 

technical risk assessment field, but for the extensive input that has been solicited during its 

development from a wide range of parties with various perspectives. The product has clearly 

been enhanced by this open process. The further value of this framework is that it provides a 

coherent basis for much closer integration among currently distinct (and sometimes inconsistent) 

programs and approaches for evaluating multiple hazards. Such a basis has been sorely needed 

to guide the explosion of new research in this area, made possible by technology developments 

unimaginable until recently that have allowed us to study the human body and impacts of our 

environment with an ever finer eye, moving from the system and organ levels to proteins and 

genes, in pursuing better understanding of the cumulative effects of multiple stressors by 

multiple routes over our lifetimes. It also provides a structure within which we can integrate 

analyses across different risk types, from human health and ecological risks to economic and 

sociocultural risks, as we continue to better understand the interconnections among these 

different areas. 

A comprehensive framework such as this can guide the more realistic assessments and 

predictions of cumulative risks that are essential to effective human health and environmental 

protection programs. The authors are to be strongly commended for the high quality of the 

report and the sustained dedication required by this complex task. I look forward to 

implementing the framework as it is filled in with specific methods and information over the 

coming years. 

F-44 
Charge Questions 

1. Does the Framework adequately capture, describe, and reasonably organize the key issues 

for cumulative risk assessment so as to serve as an adequate foundation for the development 

of future guidance? 

Yes, the key issues have been capably identified and organized, and the framework provides a 

solid foundation upon which future guidance can be developed. In fact, it is precisely because it 

is so good and so relevant that many have wanted it to be extended well beyond this foundation 

phase, asking for the frame to be filled with pieces of the puzzle needed to directly implement a 

cumulative assessment. Recognizing that the aim of this document is to simply establish the 

necessary first step (as clearly stated in a number of places), it is a compliment to its 

effectiveness that it has inspired expectations for so much more, which makes the desire for the 

next phase even stronger. 

a. Does the Framework document capture the relevant terminology? 

Yes, the document captures relevant terminology, and inclusion of a glossary is commendable. 

It might be helpful to revisit this glossary to provide a more internally consistent set of practical 

definitions for terms that directly focus on cumulative risk. Also, the definitions of certain terms 

are confusing (such as stressor and vulnerability), and per various field inputs I consider it 

important to integrate sociocultural and economic risks with health and ecological risks in an 

“integrated multiples” cumulative risk assessment framework. (See related notes under Specific 

Comments.) 

b. Does the Framework document provide an adequate assessment structure? 

Yes, the general assessment structure is quite good and effectively incorporates constructive 

elements and principles of previous work. While a wide range of cumulative risk assessment 

scenarios exists, several core concepts will be common to many applications, and the framework 

provides a good overview of those concepts. The framework is also well organized according to 

the three basic parts – planning, scoping, and problem formulation; analysis; and interpretation – 

which provide a clear backbone for the report. Further, an outstanding effort has been made to 

accommodate a variety of technical backgrounds by walking readers through basic components 

of the assessment process, illustrating key points, explaining terms, and providing a general 

“roadmap figure” to indicate where each section fits. (The communication of this information 

could be even further improved if certain figures were refined for greater readability and 

consistency.) 

In addition, active solicitation of broad stakeholder input is crucial to a balanced and widely 

responsive community-based assessment. In many ways “public involvement” has changed over 

the past 10-15 years, as many community members have stepped away from active engagement 

in the environmental assessment and decision-making process. Without an effort to solicit full 

input, the expectations and desires of a sometimes very large silent majority can go unreported, 

causing the objectives, conduct, and outcome of an assessment to be skewed toward a small but 

vocal subset of an affected or interested population group. Another point for stakeholder 

involvement: as effectively expressed by a Native American storyteller at a State and Tribal 

F-45 
Forum on Risk-Based Decision Making (sponsored by DOE in 1995), consensus means every 

voice that wants to be heard is heard. It does not mean that decisions will be held captive until 

unanimous approval is reached. This framework does an excellent job of identifying the 

importance of stakeholders to the assessment process. 

c. Does the Framework document outline the relevant methods for cumulative risk? 

Yes, the document indeed identifies a number of methods, outlining some more than others, and 

also highlights a variety of very nice examples in text boxes. It would be helpful if the direct 

relevance to cumulative risk (i.e., beyond risk assessment in general) could be emphasized. This 

could be achieved via a summary table, where targeted methods or adaptations of standard 

methods are distinguished per their unique applicability or features relevant to cumulative 

assessments. It would also be helpful if general rules of thumb could be captured in 

“thumbprint” text boxes. (For example, this could include screening secondary effects that are 

>5 to 10 times higher than the critical effect, or where an interaction is indicated, by applying a 

default interaction magnitude of 5 as modified by relative exposure levels of other chemicals – as 

well as general rules of thumb related to contaminant fate and transport and distinction from 

“background” levels). 

Further, it would be helpful if the framework further stressed the importance of jointly 

identifying sources/stressors and pathways/exposure routes, subgroups potentially at risk or 

increased risk, effects, and the potential for interactions early in the planning, scoping, and 

problem formulation phase, as interconnections will affect the selection process. Also very 

important to the timely and efficient completion of cumulative risk assessments is the use of 

phased analyses that begin with screening approaches. This topic could benefit from more 

treatment. 

For example, in screening to focus the cumulative assessment for health risks, contaminants can 

be grouped according to their opportunity for interaction both in terms of environmental fate and 

exposure potential, and in terms of toxicological interactions once taken in by a receptor. It is 

important to emphasize the use of realistic exposure scenarios for predictive estimates versus 

high-end hypothetical scenarios. Current and projected environmental co-location of stressors 

can be evaluated by testing predictive models with hind-casting or history-matching calculations, 

modeling forward from past measurements to determine if predictions match existing data. 

Concentration-toxicity screens and comparisons to benchmarks can also be used to narrow the 

set of stressors to a reasonable core number for the initial phase of the assessment. 

For the toxicity component, different combinations of mode/mechanism of action, target 

system/organ, and effect endpoint can be considered in determining where adverse effects may 

be increased by multiple stressors and exposures (e.g., all the same, different modes and same 

targets and effects, and different modes and different targets) to focus the assessment on those 

stressors most likely to increase adverse effects. And secondary effects can be considered in 

addition to critical effects, also accounting for severity, reversibility, and screening thresholds, as 

well as compensatory responses. 

Further, methods exist to estimate sociocultural and economic risks, which can dominate certain 

cumulative assessments (including place-based assessments such as the Baltimore project and 

F-46 
assessment for federal cleanup sites, such as the Department of Energy Hanford site in 

Washington). Because these can be important assessment issues, the framework should be 

structured to accommodate them as a major components of a comprehensive cumulative risk 

assessment. For sociocultural and economic risks, the nature, severity, and duration of changes 

in public behavior and activities resulting from different types of events can be assessed based 

on historic data and somewhat similar circumstances, with special attention to methods for 

assessing values of nonmarket goods. Economic sectors (such as farms or fisheries) most likely 

to incur impacts based on historic concerns and data relevant to the assessment being conducted 

should be targeted for the screening assessment, recognizing that impacts can readily extend 

from the local to the regional scale. Capacity-building within local community groups can result 

in better information being provided for the sociocultural assessment. Finally, by including 

sociocultural and economic risk components, the framework can lay the groundwork for selfreporting (e.g., by groups that are often underserved), which can ultimately lead to better 

decisions and greater net protection. 

2. Does the Framework document include any scientific or technical inaccuracies in its 

presentation of terminology, assessment structure, and methods? With respect to methods, 

does the Framework adequately convey the state of the science with respect to currently 

available cumulative risk assessment methods/approaches and areas in need of further 

research and development? 

The authors have done an outstanding job of providing sound, accurate information, and it is a 

technically strong report. (A few misinterpretations might result from somewhat unclear 

descriptions or slightly uncommon uses of certain terms, such as the discussions of vulnerability 

or stressors, but these will naturally be reduced as the process continues to move forward.) 

With regard to the second question, I did not understand that this document was to convey the 

state of the science for relevant methods. To do so would in my opinion require a more 

extensive description of existing methods and approaches beyond what is presented here (which 

relies considerably on incorporation by reference and deferment to forthcoming guidance for full 

descriptions). A genuine state-of-the-science presentation would include explanations sufficient 

to readily allow implementation, indications of current “conditions of application” and 

limitations, and future outlooks. If it is desired to fully convey the state of the science for these 

methods and approaches then a companion compendium could be prepared. However, if this 

question is merely asking whether the document imparts a general feel for current methods and 

related research and development needs, then I would say yes. In many places the discussion of 

the current knowledge gaps and research needs is excellent (in others, such as biomarkers of 

exposure and effect, a slightly clearer discussion could even further strengthen the presentation). 

3. Does the Framework adequately characterize the importance of uncertainty analysis in 

cumulative risk assessment? What additional discussions of uncertainty should be included 

in the Framework (in what sections of the document)? 

F-47 
Different types of uncertainty have been nicely described, and the importance of this topic has 

been emphasized. It may help to also include an overview characterization of the unique aspects 

of uncertainty analysis for cumulative risk assessments. It is also important to emphasize that 

many uncertainties will likely remain unknown, and that especially for cumulative risk 

assessments it is essential to identify which uncertainties that we can do something about can 

significantly affect the results (considering sensitivity analyses) so efforts can be focused on 

those. It is useful to involved parties to explain that chasing all uncertainties or waiting until all 

are understood will likely translate to doing nothing (no decisions) for a long time. It is also 

important to note that the full set of nested uncertainties should be considered in evaluating 

dominant contributors to the overall system (including those propagated through individual 

analyses that are then combined) so true drivers can be identified. It is also important to 

distinguish between predictive and protective estimates, for example with regard to incorporating 

uncertainty factors, so these can be limited as appropriate where realistic predictions for practical 

decisions are the objective. It may also be helpful to identify as a research need the development 

of methods to communicate the results and uncertainties of these assessments to decision makers 

and other stakeholders (e.g., Steven Hanna’s work at Harvard, George Mason, and soon back at 

Harvard), given that the aim of this framework is to guide assessments to their ultimate 

objective: better environmental decisions. Maxine Dakins (University of Idaho at Idaho Falls) 

has nicely captured in lay language the bottom line of environmental uncertainty and tools that 

exist to address it, including neural networks (with pattern recognition and feed-forward and 

feedback loops), system dynamic modeling (with non-linear simulations and feedback loops), 

and Bayesian decision networks (with iterative incorporation of experience and judgment). 

Dr. Dakins captures the dilemma in a quote from psychologist Erich Fromm: “The quest for 

certainty blocks the search for meaning.” It is important that cumulative risk assessments focus 

on the decisions that need to be made, and that we identify those parts of the process where we 

can reasonably get better information that will matter for the decision. (Reference: presentation 

at the American Nuclear Society Spectrum 2002 meeting, Reno, Nevada, August 8, 2002.) 

4. What are the adequacy and accuracy of the Framework’s presentation in each of the 

following areas? (These topics have been identified by the Risk Assessment Forum technical 

panel as technically difficult areas that will pose challenges to cumulative risk assessment.) 

a. Vulnerability 

As applied to cumulative risk assessment, it is useful to think of four components of 

vulnerability: the susceptibility or sensitivity of the human or ecological receptors; the 

differential exposures of the receptors; the differential preparedness of the receptor to 

withstand insult from exposure; and the differential ability to recover from effects. The 

issue for cumulative risk assessment is how to consider these aspects of vulnerability and 

their potential impacts on risk. This is highlighted in the framework as an issue in need of 

further research and development. Comment on the discussion of vulnerability on the 

draft Framework. Has the state of the science been captured in the discussion? How can 

the discussion of this issue be improved? 

F-48 
The four components as written seem to confuse rather than clarify the vulnerability issue. 

Three seem to reflect characteristics of the receptor (including individual 

sensitivity/susceptibility), while one relates to the nature of the environmental setting and is 

important in identifying relevant population subgroup(s). Overlaps seem to exist among the 

definitions, and the distinction between individual and population also seems to be muddied in 

related discussions. 

I would prefer a more traditional split into two categories: one dealing with the receptor and the 

other with the source-stressor-exposure setting. For the receptor, sensitivity or susceptibility 

relates to inherent characteristics including past exposures (body burden), as well as age at 

exposure and associated potential for differential effects. As a complement to this “internal” 

piece, characteristics of the “external” exposure setting will help guide the determination of what 

individual or group is most likely to incur exposures and associated effects. Both should be used 

to define the center of the bullseye for a cumulative assessment or to guide screening criteria. 

That is, the question about what makes a group vulnerable per existing stressors can also be 

turned around, to identify contributing stressors working backwards from observed effects. An 

early evaluation of potential target receptors can be useful in identifying sensitive 

subpopulations that can in turn help focus the determination of stressors warranting 

consideration. 

b. Cumulative Risk Assessment Involving Chemical and Non-Chemical Stressors 

Viewing cumulative risk assessment as an evaluation of the accumulation of stressors 

presents many challenges. These may be seen when attempting to combine, in some 

meaningful way, the risks from multiple chemicals that may act as synergistic, 

antagonistic, or additive doses leading to a single effect. The situation is exacerbated 

when non-chemical stressors (e.g., radiation, biological agents, and psychological stress) 

are considered. Comment on the Framework’s discussion concerning the combining of 

disparate environmental stressors. In commenting, consider the state of the science with 

respect to understanding the effects of different stressors acting together (e.g., chemical 

exposure and viral infection). What can be added to the framework to adequately convey 

the state of the science in this area? 

The framework indicates that this research area remains open for development, and I agree. 

Some data and approaches are becoming available through innovative science and technology 

research and development projects, including in the areas of genomics, proteomics, and 

metabolomics. It might be helpful to tabulate progress that has been made and key gaps to be 

addressed (e.g., within the three-element organizational structure of this document, or per the 

four steps of the NRC paradigm). An example of what could be included in a summary text box 

of information needs/ongoing research at the end of the section relating to mixed exposures 

alone follow (summarized from the NIOSH National Occupational Research Agenda, external 

review draft July 2002). 

• A more realistic representation of mixed exposures to improve basic input assumptions for 

the risk assessment models, and more effective approaches for distinguishing between risks 

F-49 
from occupational and ambient exposures and for determining which mixtures drive risk 

concerns. 

• Improved mathematical models to better predict health risks from multiple stressors, 

especially formulas for extrapolating from in vitro to in vivo conditions; across exposure 

routes, durations, and sequences; and across mixture components and dose levels. 

• Approaches for improving how variability is reflected in risk assessments, to replace the use 

of single default parameter estimates in predictive models, and for reducing uncertainties that 

significantly affect the risk outputs, for example by resulting in unrealistic bounding 

assumptions that limit the practical utility of results with regard to guiding interventions. 

• More effective approaches for translating response information from emerging research, 

including molecular- and cellular-level studies (such as genomics and proteomics), into risk 

estimates meaningful to the whole organism, also considering compensatory responses, 

effect severity and reversibility, and recovery to evaluate net or effective health risk. 

• Better methods for identifying deviations from dose and response additivity (such as where 

the severity of an effect exceeds that predicted from simple dose addition), and for 

extrapolating from existing data on specific chemicals or mixtures to develop general rules of 

thumb and default factors that can be applied to estimate risks from exposures to untested 

combinations. 

• Improved approaches for incorporating multiple mixture toxicities into risk models where 

component characteristics vary, building on current methods that consider, for example, 

indicator chemicals, comparative potency, and combined pairwise interactions, and better 

methods for testing the validity of these approaches for estimating occupational risks. 

c. Cumulative Risk Assessment Involving Different “Types” of Risk 

Conveying combined risks from multiple chemical and non-chemical stressors, in a 

meaningful way, is the ultimate challenge for cumulative risk assessment. Experience in 

this area is extremely limited. Indices, common metrics (e.g., Disability Adjusted Life 

Years, DALYs) and graphical (e.g., GIS) approaches have been explored but much 

methods development work remains to be completed. Cumulative risk assessment can be 

a valuable part of the decision-making process, but only if results are conveyed in a 

meaningful way. Comment on the Framework’s discussion concerning the combining of 

separate measures of risk. Do the example approaches discussed capture the state of the 

science in this area? In particular, consider the role of valuation (i.e., the assignment of 

societal values to disparate health outcomes) implicit in some of the approaches. 

Suggest changes or additions that may improve this discussion. 

I strongly agree that results must be presented in a meaningful way in order to be directly useful 

for decisions. However, this discussion seems to describe approaches without giving a clear 

explanation of why and when they should be applied, and without sufficient cautions regarding 

F-50 
lumping distinctly different risk estimates into a potentially meaningless number. It is important 

to retain all key risk results and consider how the discriminating information can be presented 

together, but as combined information captured in matrices or other multi-dimensional forms 

rather than as a single metric. For me, the downsides of a single-metric approach outweigh any 

imagined benefit from having one number. 

Visualization tools including geographic information systems (GIS) are among useful methods 

for assessing cumulative risks, and they can also be effective in communicating results. Other 

methods include trend analyses, matrices, and indices, which can include weight-of-evidence or 

lines-of-evidence considerations as well as relative ranking approaches. 

Specific Comments 

Page xviii, line 26 

Including the absence of a necessity in the definition of a stressor seems a bit odd. Absence of 

something such as an adequate habitat could certainly be considered stressful, but would seem 

not to represent a stressor as traditionally defined but rather a condition more appropriately 

identified as a risk modifier or susceptibility factor. (This comment also applies elsewhere, e.g., 

from page 2 footnote 2 to the glossary.) For example, a fire could result in the loss of a habitat, 

with the habitat being the “stressee” (and that condition can subsequently affect organisms 

aiming to live there) while the fire is the causal stressor. Perhaps it would be useful to consider a 

direct-indirect distinction here. 

Page xviii, line 35 

The definition of cumulative risk assessment as covering qualitative to quantitative 

representations of combined risks to health or the environment from multiple agents or stressors 

is very good, for it also captures economic and sociocultural risks per the definition of 

“environment” (the human environment) under the grandfather of our environmental regulations, 

the National Environmental Policy Act (NEPA). (Also see glossary comments regarding agentstressor use, and additional related comments.) 

Page xix, lines 12-14 

The three phases of the framework are clearly identified and nicely described. It might be 

helpful at this first discussion of the planning, scoping, and problem formulation phases to note 

that the conceptual model would also establish initial population groups to be evaluated 

(including potentially susceptible subgroups, as may be affected by the stressors and effects of 

interest). 

Page xix, lines 23-27 

Given that risk characterization provides the bridge/overlap with risk management and this 

description invokes an evaluation to determine whether original objectives and goals were met 

(with risk managers having been identified as members of that team), it might be useful to 

emphasize here that the framework is intended to guide assessments toward useful decisions, i.e., 

it is outcome-oriented and not a general format for circular (or repetitive) evaluation, as this 

otherwise represents a potential tar pit if clear definitions of scope and timelines are not 

F-51 
developed at the outset. (Iteration is inevitable and appropriate, but up-front clarification of 

what is to be assessed and how the information will be used can be critical to limiting unrealistic 

expectations.) 

Page 1, Figure 1-1 

Figures are used in this document very effectively. One note: inclusion of “chemical” in the 

center circle here may seem redundant as it is included in the definitions of both agent and 

stressor. (Also, the differences between those two have not been clearly described, to explain 

why both are being used. Would agent be considered a subset of stressor?) (Same comment 

applies to Figure 1-2 on page 2). Note that language reflecting stressors as the umbrella 

category within which chemicals and other “agents” reside is nicely captured on page 7, lines 15-

16: “... an assessment which covers a number of chemicals or other stressors ...” 

Page 2, footnote 1 (and elsewhere) 

‘... for traditional, chemical-focused assessments we say we conduct a “risk assessments for a 

certain chemical. In contrast, the essence of a cumulative risk assessment is that the assessment 

is conducted “for a certain population.’ Some might not see the same distinction between these 

two types of assessments as indicated here, as the population or population segment is also the 

basis for source-specific or chemical-specific assessments. That is, for those assessments we 

have generally conducted traditional risk calculations for individuals representing a population 

group (e.g., beginning with the maximally exposed and evolving to the reasonably maximally 

exposed individual and more central tendency estimates) that have simply considered chemicals 

one at a time rather than in combination with their associated potential interactions. This is what 

I see as the contrasting distinction – interaction considerations and combined contributions to 

risk from multiple stressors, sources, and routes. Also, neither are cumulative assessments 

limited to population- or place-based analyses (despite the suggestion that this term does not 

apply to those conducted for chemical classes, such as organophosphate pesticides). And for 

both traditional and cumulative assessments, the individual is still the basis of the non-cancer 

estimate for this threshold-related value (summing hazard quotients to an index for the 

traditional case, calculating an interaction-based hazard index for mixtures), and while the 

cancer risk estimate is based on population statistics it is still commonly reported as the risk for 

an individual (or set of individual receptors) representing whatever subgroup(s) have been 

selected for the given risk assessment situation. Perhaps it would be an option to describe Figure 

1-2 as illustrating a “place-based cumulative assessment” that looks at how multiple stressors 

can act together on a group in a given area, with an indication that cumulative assessments are 

often of this type? 

Page 4, lines 11-23 

Inclusion of NEPA in this very nice historic progression is excellent, especially because that Act 

represents our first formal invocation of cumulative assessment from more than 30 years ago, but 

the description is misleading by explicitly leaving out impacts to human health. (Per CEQ’s 

implementing regulations at 40 CFR 1508.8: “Effects and impacts as used in these regulations 

are synonymous. Effects includes ecological (such as the effects on natural resources and on the 

components, structures, and functioning of affected ecosystems), aesthetic, historic, cultural, 

economic, social, or health, whether direct, indirect, or cumulative.”) Given that human health is 

F-52 
a key concern for many cumulative assessments and this framework emphasizes chemical risks 

to human health (e.g., see page 5, line 8), it would seem appropriate to add something to (e.g., at 

line 18) like “..., and impacts to human health as well economic and social resources.” (This 

comment also applies to the text box on page 97.) 

Page 6, lines 8-14 

This is a nice summary of the context for this framework. It might be helpful to add a text box 

here that captures what this framework does and doesn’t do, to further strike home its scope. 

Page 7, line 12 

Including in the definition of an agent or stressor “... the absence of a necessity such as habitat 

...” is confusing to me (e.g., see related note for page xviii). 

Page 8, line 33 

The terminology here seems a bit non-standard? I have understood certain risk factors identified 

for breast cancer such as age at first childbirth to be not stressors themselves but rather 

modifiers. Taking another example, malnutrition that results in increased susceptibility to disease 

reflects a lifestyle-related risk factor (e.g., associated with poor diet per multiple contributors) 

that can affect one’s subsequent response to a biological stressor, but here again malnutrition 

reflects the condition of the system (compromised or otherwise) when it is subsequently put to 

the test of a given stressor, rather than the stressor itself. Similarly, an increased likelihood of 

cancer death due to limited access to health care (such as is reflected in higher rates of cervical 

cancer mortality in certain areas of the southeastern United States) does not make access to 

health care a stressor. Rather it is a condition (here, societal rather than biological) that affects 

the ultimate outcome of a stressor, but it relates to the management and not the probability of the 

disease. The language being used in my opinion confuses the issue considerably; the fix would 

be straightforward (e.g., distinguishing between actual stressors and stress or risk factors or 

related descriptive indicators of stress or stress levels). 

Page 9, lines 26-32 

Columbia River Basin studies at the Hanford Site have also applied the “bottom up” approach, 

considering multiple stressors present and predicting effects that extend from human health and 

ecological risks to economic and sociocultural impacts (see, for example, http://www.bhierc.com/projects/vadose/sac/sacdocs.htm, Appendix F, May 2001). 

Page 9, line 36 (and elsewhere in the document) 

Consider moving callouts to those reports or articles not prepared by EPA or national peer 

groups (such as NAS/NRC) to a “related scientific literature” text box at the end of each section. 

Otherwise, as currently written citations are imbalanced (seemingly due to individual reviewer 

self-reporting in some cases), and they may or may not carry the full weight of extensive, 

broadly accepted peer review as met by standard Agency references and similarly extensively 

reviewed reports. 

Page 9, text box 

Again, I my view the anchor of cumulative risk assessment needn’t (shouldn’t) be populations, 

F-53 
because calculations are still being made for single individuals (or a very small subset of 

receptors) taken to represent populations/ subgroups, and estimates of the potential for systemic 

effects (hazard indices) are individual- not population-based. (For example, see the first bullet 

under “risk descriptors” in the box on page 54.) Further, many cumulative risk assessments are 

more chemical-based than place-based (e.g., assessments for specific categories or mixtures, 

such as dioxins/furans or jet fuel). An option would be to identify the combined effect of 

multiple sources-stressors-exposures as the heart of a cumulative risk assessment. Also, as a 

note “different sequences” could be inserted in front of “multiple durations, ... ”(line 20) and 

“multiple effects or impacts” (line 22.5) could be followed by “with associated severities and 

durations” (the latter to incorporate the time scale). 

Page 10, lines 6-8 

I don’t consider the definitions of cumulative risk in NEPA and FQPA to be quite adequately 

reflected, as the latter seems to have been somewhat excluded and the former has been 

considerably limited (e.g., see page 4 note). The 1997 CEQ handbook identifies the importance 

of conducting cumulative assessments per natural ecological boundaries or sociocultural 

boundaries, not political or administrative boundaries, while this report at times seems to endorse 

the generic geographic definition (e.g., page 9, lines 20-24; page 18, line 7) that tends to be 

based on the latter. It may help to clarify that the specific population to be studied is not for 

example an urban area as demarcated on a county map, but rather the reasonable potential for a 

given group or groups to be affected by a given set of sources/stressors. Further, it may be 

useful to note that not all impacts are adverse, and that the range from beneficial to adverse 

(including the magnitude and significance of effects) must be considered in determining the net 

effect, consistent with NEPA. 

Page 10, lines 29-31 

The use of “i.e.” here seems too restrictive, so as to undo the earlier excellent identification of 

qualitative analyses as an important feature of cumulative risk assessment (page 7, line 20). For 

me, it would be more helpful to indicate that each phase includes elements of both the analytic 

and deliberative processes, with certain phases typically emphasizing one over the other, as 

dependent on the given assessment (e.g., this could be inserted in line 31 ahead of “Much ...”, 

and less prescriptive words could be used in the following text, such as “For example, much of 

what is discussed ... is typically deliberative ....” 

Page 11, lines 12-13 

Given that this sentence references the text box where economic and social objectives are 

highlighted, consider extending the description of the CBEP approach to: “... encompasses 

ecological and human health assessments as well as economic and social considerations.” 

Page 11, lines 25-31 

It might also help to ask “when should cumulative risk not be done?” and this may be a good 

place to note the utility of screening analyses to focus the assessment on those specific resources, 

population groups, stressors, sources, and exposures for which significant adverse effects are 

relevant. 

F-54 
Page 12, line 25 

Consider adding something like: “Nevertheless, because this framework is outcome-oriented, it 

is important for the specific decisions to be made to be identified up front so the overall 

assessment process will focus on the information needed to address them.” 

Page 13, Figure 1-3; Page 14, Figure 2-1; and others 

Consider changing the double arrows to make the primary point (and limit confusion regarding 

do-loop analyses). 

Page 15, lines 22-24 

Economists and engineers also assess risks (in fact the need for, components of, and results of an 

economic risk assessment may be better understood by many in the lay community than parallel 

information for human or ecological health risk assessment). Further, it is important to include 

an assessment of economic and safety risks (among others) in a cumulative assessment for it to 

genuinely be cumulative. This is also more consistent with NEPA and the 1997 CEQ handbook 

(and referred to per the SAB excerpt on page 12, lines 19-21). A possible reword could be: “... 

along with other technical experts who assess risks and impacts, such as economists and 

engineers) ... 

Page 21, lines 21-22 

Not sure “jurisdiction” applies to academia; may want to consider “purview” or some more 

general term, and add industry (a key source of support for hypothesis-driven research that is 

crucial to progress in cumulative risk assessment). 

Page 22, lines 40-42 

Consider adding the importance of the conceptual model in identifying key data gaps, to guide 

the collection of additional data (as possible for a given assessment) depending on resources, 

timing, and relative importance of those data in providing information necessary for the decision 

at hand. 

Page 23, lines 25-27 

A conceptual model needn’t include both a written description and visual representation of 

actual or predicted relationships, and it seems a framework that aims to accommodate a full 

cumulative assessment should present a conceptual model that includes economic and 

sociocultural components as well as ecological and human health components (as reflected in the 

accompanying Figure 2-2, which calls out environmental justice communities, and also nicely 

referred to on page 26, lines 3-4). 

Section 2 

This section is very well organized, with good text boxes illustrating key points made in the text 

and effective emphasis on the importance of incorporating lessons learned from past assessments 

(both cumulative and traditional). It may be helpful to note the usefulness of value of 

information methods (such as those developed at Harvard) for focusing data collection. 

Page 31, lines 25-33 

F-55 
Given that cumulative risk assessment methods are still being developed, it may be more 

realistic to indicate that the plan will indicate the expected method(s) for generating risk 

estimates or measures, and that this method or set of methods will likely be refined along with 

the exposure, hazard, and dose-response information as more is learned during the assessment 

process. 

Pages 32-44, Section 3.2 

This section describes general issues more than the methods available to address them (referring 

to Appendix B for relevant resources, for example). It would be helpful to include here, if 

possible, a summary table of key methods with unique features (or adaptations) relevant to 

cumulative risk assessments compared with traditional assessments. 

Page 33, subsection on time-related aspects 

It is not clear how a cumulative assessment differs from the traditional assessment in this regard, 

as both should use toxicological data relevant to the exposures being assessed. 

Pages 34-36, subsection on vulnerability 

Similar to the previous comment, it is not clear how a cumulative assessment differs from a 

traditional assessment as both should consider highly exposed or highly susceptible groups. 

Further, the categories identified for vulnerability seem odd (as also noted in the charge 

questions). 

Page 36, subsection on subpopulations with special exposures 

This subsection seems similar to the preceding one; consider combining and streamlining. 

Section 3 

This section contains good information; it may help to review and streamline the organization 

and content, to emphasize those elements truly unique to cumulative risk assessment (rather than 

implying uniqueness for issues also common to traditional assessments). 

Page 46, lines 5-7 

Per earlier notes, I strongly support inclusion of the economic risk context, as this is certainly 

part of the assessment process and of considerable interest to the affected and interested 

communities (for example, as highlighted by the Baltimore example and others). 

Pages 47-48, Section 3.3.2 

The discussion of biomarkers and biomonitoring could be refined and updated. The discussion 

of biomonitoring is limited (suggest expanding). Not all biomarkers are inherently cumulative 

risk measures, although some are available to address cumulative issues. For example, 

cumulative exposure to radiation can be assessed by measuring the frequency of balanced 

reciprocal translocations in human chromosomes using fluorescence in-situ hybridization (FISH) 

to “paint” specific chromosomes with fluorescent chromosome probes that bind to test 

chromosomes along base pair sequences. Chromosomes with translocations can then be readily 

detected and scored by optical scanning. Because evidence indicates that these translocations 

are extremely stable over time, their relative abundance is proportional to the cumulative dose 

F-56 
over the lifespan of the organism. (See, for example, Lucas, J.N., 1997, Chromosome 

translocations: a biomarker for retrospective biodosimetry, Environmental Health Perspectives, 

105 [Supplement 6]:1433-1436, and Ulsh, B., W. Whicker, et al., 2000, Chromosome 

translocations in turtles: A biomarker in a sentinel animal for ecological dosimetry, Radiation 

Research, 153:752-759.) 

Pages 48-51, Section 3.3.3 

The discussion of a single risk metric is cause for some concern, as more can be lost than gained 

by collapsing useful risk estimates into a single lumped number, which typically reflects 

subjective judgments with which not everyone will agree. For this reason, it may be appropriate 

to include more discussion about the pitfalls of pursuing a single metric, and to provide a 

summary table capturing key benefits and limitations.. 

Pages 51, Section 3.3.4 

This section on qualitative approaches is important and deserves emphasis. 

Pages 64-66, Glossary 

It may seem that an inordinate amount of time was spent nitpicking this section, but it is only 

because I consider a clear set of definitions vital to the success of an overarching framework – 

especially given historic problems with inconsistent terminology that have made “accumulating” 

risk estimates across different types, approaches, and studies very difficult. I have commented 

on what is here rather than what is not, but several additional terms are warranted, including 

those unique to cumulative risks (such as mixture [various types], interaction and no interaction, 

inhibition, and potentiation, as well as traditional terms such as synergism and antagonism that 

are often misconstrued). By these comments, I only hope to illustrate how one might give this 

glossary a good scrub before the document is finalized so it can serve as the cornerstone of a 

consistent, broadly useful framework. In the interest of keeping this section brief, an option 

would be to refer to a separate, fuller cumulative risk glossary from which selected terms are 

excerpted for this report – with the main criterion for selection being direct relevance to 

cumulative risk assessment. 

Page 64, line 7: Agent 

Inclusion of “mineralogical” seems a bit odd – I interpret this as chemical components of mineral 

material (such as asbestos) and so do not understand the distinction between this and the 

“chemical entity” already listed in this definition. (Recognizing that radionuclides are simply 

radioactive chemicals, I appreciate that in this case these have commonly been referred to 

separately.) Would “physical” have been an option instead? This would also allow inclusion of 

such entities as heat and noise, which have in certain cases been shown to enhance effects 

beyond those predicted from simple additivity when combined with chemical agents. The 

distinction between agent and stressor is not clear to me (one may cause a deleterious effect, the 

other can induce an adverse response ...) if both terms are to be retained it may help to indicate 

if they are interchangeable or explain why not if not. 

Page 64, lines 21-23: Conceptual model 

I appreciate the need for brevity, but this definition seems to have lost something in the 

F-57 
compression. Conceptual models can be used to identify relationships among stressor sources, 

releases, receptors, associated exposure or impact pathways, and effects. Also, they can extend 

from human health and ecological risks to sociocultural and economic effects. As a minor note, 

conceptual models can be either written or visual (needn’t be both). 

Page 64, lines 30-37: Dose additivity 

The extent of this definition is a bit inconsistent with others. Consider perhaps putting the dioxin 

part in a text box within the document where this topic is discussed. Minor editorial note: the 

first sentence seems incomplete and could be joined to the second by a comma after “every other 

chemical.” 

Page 64, lines 42-43: Endpoint 

It might be useful to also include “physical” manifestations, and to add “for health effects” at the 

end of the current sentence (whether human or ecological). As written, this definition does not 

lend itself to extrapolation to other types of endpoints, so it may be useful to add text that 

extends its relevance to other risk types (such as sector revenue loss, or net regional impact such 

as job loss, for economic effect assessment endpoints). 

Page 65, lines 1-4: LOAEL 

For consistency among related terms, should this also have “some effects may be produced ...” 

as included for NOAEL (within lines 11-16)? 

Page 65, lines 6-9: Model 

“Model” is an umbrella term not limited to mathematical models – i.e., it covers a wide range of 

qualitative to quantitative, conceptual to calculational models. Suggest either renaming this 

“mathematical model” or changing the definition. (Minor note, the second sentence sounds a bit 

buzzy and could be ended after “computer programming.”) An option would also be to fold 

“conceptual model” under here as a subset with various types briefly described, rather than 

having separate entries? 

Page 65, line 18: Ototoxic stressor 

Not sure why this term is included. It is not very commonly used, is not unique to cumulative 

risk, and is at a much lower level (more specific) than the others in this set. I’d recommend 

deleting it here (define it in the text where used, page 42 line 11), although it could certainly be 

retained in a fuller glossary from which these terms are excerpted. In that case it would make 

sense to also present more common, similar terms (such as neurotoxic and fetotoxic stressors) in 

that glossary. 

Page 65, lines 20-23 and 25-27: RfC and RfD 

The order of magnitude parenthetical phrase may elicit questions (as some consider it higher). 

Page 65, lines 29-33: Response additivity 

Consider revisiting this definition to make it a bit more friendly to a non-mathlete. 

Page 65, lines 35-41: Risk 

F-58 
Consider also revisiting this definition. For example, many readers could be familiar with the 

National Oil and Hazardous Substances Pollution Contingency Plan (NCP) target incremental 

risk range of 10-4 to 10-6, and this definition may not seem to offer much insight for that 

“practical implementation” context. 

Page 65, lines 43-45: Stakeholder 

Many have interpreted this term to apply primarily to external groups, beyond the formal project 

decision makers who are often responsible for implementing a compliance program. (I 

personally prefer its application to “all with a stake in the process” which of course includes the 

agency or other implementing party.) Because misconceptions regarding roles and authority 

have been an issue in some cases, you might consider changing “making decisions” to 

“providing input to decisions.” 

Page 66, lines 1-2: Stressor 

See earlier comments (e.g., page xviii and page 7). 

Appendix A 

This Appendix contains good information. Capturing key research needs at the end of each main 

chapter of the report might also be useful. Also, it may be helpful to re-structure this appendix 

into more organized categories (e.g., within the three phases that serve as the backbone of the 

framework), possibly with cross-references to related portions of the main text. 

Appendix B 

Similar to comment for Appendix A. 

Appendix C 

A discussion of background (or ambient) is important, and the authors are to be commended for 

including it. Baseline or background information that accounts for natural variability across the 

appropriate spatial and temporal scales and focuses on characteristics of the environmental 

setting that affect contaminant mobility and ultimate effects (including across human and 

ecological health, and sociocultural and economic risks) is essential for the context it provides 

for an integrated analysis and ultimate interpretation of risks (e.g., per exposures beyond ambient 

or baseline levels). Background is often taken to mean ambient. For example, arsenicals were 

widely used decades ago in vast agricultural areas, and arsenic remains in the soils across the 

midwestern United States. Thus, reference sites should be practically defined – more as a 

reasonable bases for comparison rather than as a rare, pristine location (although the latter also 

provides useful context for absolute risk estimates). 

Appendices D and E 

The inclusion of an example plan and overview of the toxicologic similarity issue is excellent. 

(But it’s interesting that an FQPA example is used and referred to as a cumulative risk 

F-59 
assessment, as portions of the main body of this document seemed to indicate that this was not 

appropriately considered a cumulative assessment?) 

Very Minor Editorial Notes 

Page xvi, List of Abbreviations and Acronyms 

I think CEQ is Council on Environmental Quality, I’ve also seen OP as organophosphate

pesticides (with both organophosphorous and organophosphate found on EPA websites), APCA 

may be ACPA (also on page 67, line 30), Material in MSDS may be singular, and I’ve also seen 

GIS as Geographic (no “al”) Information Systems. 

Page 2, footnote 1 

Excerpt: “... for traditional, chemical-focused assessments we say we conduct a “risk 

assessment (delete s) for a certain chemical. (Here and elsewhere end quotes seem to be 

missing, e.g., page 4, lines 33, 35; through page 64, lines 37 and 39, and beyond – software 

glitch?) 

Page 6, line 1 

The referenced box is on the previous page (rather than “at right”)? 

Page 6, line 34 

Tiny note: are vs. “ar e” 

Page 8, line 14 

Possibly insert “with” before “health”? 

Page 18, line 35 

Consider changing “brainstorming” to “discussions”? (Classic brainstorming involves quickly 

polling a group for input on a topic, then combining those responses that are common and 

deleting duplicates to assemble a final list. The typical process for soliciting community input is 

more measured and not completed within a single meeting.) (In line 43: is it “or” versus “of”?) 

Page 18, text box 

Consider including non-health examples (e.g., Chesapeake Bay for ecological risks, closure and 

realignment of neighboring bases for economic risks). 

Page 19, line 8 

Consider changing “risk” to “overall viability” (the first involves probability, the second simply 

reflects an umbrella “health” status). 

Page 19, lines 33-34 

The phrase “to choose from” seems funny here; possibly consider rewording, e.g.: “... some 

examples of interested or affected parties that could be involved in the deliberative ...” 

F-60 
Page 22, line 20 

Pluralize “Lesson.” (An excellent point is made in this subsection.) 

Glossary 

Page 64, lines 30-37: Dose additivity 

First sentence seems incomplete, could be joined to the second by a comma after “... chemical.” 

F-61 
Review by 

Pavel Muller – ToxProbe Inc.

F-62 
1. Background 

Dr. Pavel Muller of ToxProbe Inc. conducts this peer review at the request of Mr. David 

Bottimore of Versar Inc. ToxProbe Inc. has reviewed the report titled Framework for Cumulative 

Risk Assessment prepared for USEPA by the USEPA’s Risk Assessment Forum and dated April 

23, 2002. 

Versar Inc. has also provided Dr. Muller a copy of The Risk Assessment Forum Draft 

Framework for Cumulative Risk Assessment dated April 26, 2002. Response to the charge 

questions contained within constitutes the main component of the review. Some general 

comments not directly related to the charge questions are also presented. 

2. General comments and recommendations 

USEPA produced a thorough, well researched framework for cumulative risk assessment. The 

report is intended as a first step in the development of a guidance report which would serve the 

Agency and other organizations in conducting cumulative risk assessments. The Framework is 

strong on providing data, discussion of the data and references relevant to cumulative risk 

assessment, and providing definitions of the process. However, the Framework seems weaker in 

explicitly defining how it will be applied in the decision-making process and in recommending 

when it should be used in preference over simpler, faster and less costly alternatives. The actual 

assessment process is defined only in a rudimentary fashion and this makes the report harder to 

follow. Finally, there are other existing processes conceived as frameworks for cumulative risk 

assessment (ILSI, 1999) or which could serve as such frameworks (USEPA 1999, 2000b). Given 

the existence of these frameworks, I believe it would be preferable to build and expand 

particularly on the SAB framework. The SAB framework appears particularly well developed 

and it is recommended that it be used as a starting point for the USEPA’s Framework. 

My vision of the scope of a framework report is wider than that of the authors and this is where 

my main concerns are found. It appears that the authors focused on providing factual, technical 

framework only, and have decided not to provide a framework for performing, communicating 

and acting upon the results of the assessment, relegating these issues to the eventual Guidance 

document. I would have like to see a brief discussion of a complete, balanced framework, which 

include the following issues: 

• Criteria for conducting cumulative risk assessment. Cumulative risk assessment brings 

about considerably higher level of complexity and is likely to require more time and 

resources than more limited traditional risk assessment. Resources committed to these 

cumulative risk assessments will not be available for other worthy projects. It is 

therefore important that the cumulative risk assessment is the appropriate tool for a 

given situation in terms of science and that the net increase in costs is justified by the 

additional benefits from conducting the assessment. 

• The definition of specific roles of stakeholder groups including scientists/assessors and 

managers. 

F-63 
• It is important to give risk managers an opportunity to assure that the output of the 

assessment is in a form which gives risk managers optimal tool to conduct informed and 

effective risk management decisions. I am not comfortable that risk management step is 

not perceived to be an integral part of the envisioned process. In that sense I find the 

approach used by SAB (USEPA 2000b) clearly preferable. 

• Although I agree that a multi-stakeholder participation is a positive and perhaps 

necessary feature of the cumulative risk assessment, I feel that a separate and 

independent technical risk assessment report should be prepared in addition to the 

overall report. For risk managers the separation is important. For example, if the 

technical report concludes that the risk from mobile sources is orders of magnitude 

below what is commonly seen as a level of concern and if the multi-stakeholder report 

identifies mobile sources as an important source of concern, the management solution 

may not be an attempt to further reduce the emissions, as this would in all likelihood not 

diminish public concerns about such emissions. Instead other solutions designed to deal 

with the causes of public concern would be preferable. 

I feel strongly that the framework should contain a “table of contents” for the eventual guidance. 

By that I mean that the framework should indicate what key elements (such as ranking and 

prioritizing, a detailed flowchart describing the cumulative risk process etc.) are expected to 

eventually constitute the guidance on cumulative assessment. It is understood, that some 

elements will not be described in detail within the framework report, but the inclusion of 

statements identifying issues which need to be addressed in the eventual guidance would have 

several benefits: 

• It would reassure readers and peer reviewers that the authors are aware of the issues and 

that these issues will be addressed. 

• It would improve clarity of the report by adding brief presentation of issues that would 

be anticipated by at least some readers. 

• It would help the eventual authors of the guidance report by providing a handy 

“checklist” of issues to address. 

3. Charge questions 

1.a) Does the Framework document capture the relevant terminology? 

I am unaware of any major omission or inconsistency. Some terminology contained in the 

reports cited in the Framework is not specifically discussed in this document (e.g. terminology 

related to complex mixtures – see USEPA, 2000a). I do not see it necessary to include a 

terminology unless it is needed to understand the Framework. 

It would be useful to add a new section 2 (perhaps called Definitions of Cumulative Risk 

Assessment and its Key Features). Include in this new section elements from pages 1-4 and 

Section 3. 

F-64 
1.b) Does the Framework document provide an adequate assessment structure? 

In my view, the assessment structure is not adequate. The current Framework is strong on 

discussion of issues, which could be included in the cumulative risk assessment, as well as the 

knowledge base in support of such an assessment, but the process is outlined in a minimal 

manner. One can argue that the way it is presented has its benefit; it allows the greatest 

flexibility in terms of the problems the process can be applied to and it places minimal 

constraints on future guideline developments. However, I find the assessment structure 

rudimentary to the point of not being entirely clear how the gathered information will be used to 

provide meaningful support in the decision-making process. I recommend adoption of the SAB 

assessment structure, perhaps complemented with extension or elaborations from the USEPA 

assessment structure for cumulative risk assessment. 

It is recommended to add another new section, which would explain in some detail the 

differences between the existing approaches dealing with Cumulative Risk Assessments (such as 

some of those listed on page 5), especially the SAB Framework, and explain why USEPA had to 

develop a new framework, rather than expand on the one previously published by SAB. To me, 

the two frameworks are very compatible and incorporate the use of similar data and methods for 

interpretation and both encourage similar form of stakeholder participation. The apparent 

differences in their application do not seem to be significant and I believe, are reconcilable. I 

strongly believe that the USEPA (and in this context, includes SAB) should create only one 

internally consistent framework. USEPA Risk Assessment Forum needs to present a far stronger 

and more defensible rationale for separate frameworks than the one presented in Section 1.5. 

After examining both frameworks, I personally consider the SAB framework better defined and 

more practical. However, the Framework contained in the USEPA document presented for 

review provides more detail on cumulative risk assessment; and it could be made to complement 

the SAB framework relatively easily. 

I feel strongly, that the Framework should discuss how decision-making, implementation and 

performance will be integrated into cumulative risk assessment. If resources are to be committed 

to complex time- and labour-intensive processes, such as those discussed in this Framework, it 

should be done only after it was ascertained what information (and in what form it takes) is 

needed by the decision-makers to make better decisions. Finally, performance measures need to 

be set to assure (in part) that management activity does not continue needlessly if the 

performance measures are reached or if these measures turn out to be ineffective or 

inappropriate. Explicit linkages with the decision-making component of the process would help 

assure that the resources committed to the analysis will be effectively utilizable by the decisionmakers. 

The Framework states (page 12) that cranking out numbers will not be the sole basis for a 

decision. That is acceptable. However, the Framework should assure that even though the 

process as a whole is analytic-deliberative, the decision-maker still has an ability to determine 

what are scientific/technical conclusions and what are the conclusions based on the overall 

deliberative process. Unless such distinction is made, important information will be lost to the 

decision-maker. It is therefore proposed that the Framework recommends that one of the reports 

F-65 
prepared to describe the outcome of the cumulative risk assessment be a technical report. This is 

consistent with the 1983 National Research Council report Risk Assessment in the Federal 

Government (NRC, 1983), which makes the following statement. 

“We recommend that regulatory agencies take steps to establish and maintain a clear 

conceptual distinction between assessment of risks and consideration of risk 

management alternatives; that is, the scientific findings and policy judgements embodied 

in risk assessments should be explicitly distinguished from political, economic and 

technical considerations that influence the design and choice of regulatory strategies.” 

On page 63, paragraph 3, the Framework states that “the results of the risk assessment will be 

only one of the factors that will need to be considered in making a decision on action to address 

the risk”. On the other hand, the Framework envisages stakeholder and decision-maker input 

throughout the assessment, so that their concerns can be taken into consideration. These 

concerns may include availability of resources, community values, etc. As a result, the final 

report is expected to contain some mix of technical analysis, other concerns and interpretations, 

which decision-makers have an opportunity to influence. It therefore appears that the report 

should contain most, if not all, of the elements a decision-maker needs to make a decision. I 

therefore do not understand why the results of the risk assessment cannot be the primary basis 

for decision-making. 

1.c) Does the Framework document outline the relevant methods for cumulative risk? 

The planning and scoping phase of the Framework is, in my view, the most developed part of the 

process. However, the cumulative risk assessment process, as envisioned in the Framework, has 

the potential of being extremely complex and time- and resource-intensive. Before undertaking 

this type of assessment, it is prudent to determine whether the extra benefits of conducting a 

cumulative risk assessment justify the extra effort involved in using such a process rather than a 

more traditional risk assessment process. I recommend that the Framework incorporate a formal 

feasibility study as the first step of the cumulative risk assessment process. 

Section 3.2 of the Framework (Available Methods and Approaches) gives mostly examples of 

increased complexity with the use of cumulative risk assessment. These examples are described 

and discussed, but the focus is not on specific methodologies of addressing the complexity. It is 

suggested that this section be more appropriately named. 

The actual methods are discussed in Section 3.3 (Areas of Complexity and Current Research). I 

find the section headings unhelpful in identifying the methods of this Framework. Furthermore, 

it is not explicit how these methods will fit into the assessment process; there is a reliance on the 

readers’ insight to make a connection. 

It is recommended that this section be thoroughly rewritten, providing headings, which reflect 

more appropriately the contents of subsections. The methods should be named, and compared. 

The context within which the methods would fit in the assessment process should be clearly 

F-66 
described. This issue is also discussed in Section 2 of this peer review. 

2. Scientific or technical inaccuracies 

None found. 

3. Uncertainty assessment 

Overall, Section 4.2 of the Framework categorizes and describes uncertainty, but it does not tie it 

to the cumulative risk assessment process. I would suggest the section be revised to include the 

following type of discussion. 

• In what way the uncertainty assessment is different for cumulative risk assessment 

relative to other forms of risk assessments. 

• How the uncertainty assessment will be used to refocus the ongoing cumulative risk 

assessment process or to determine whether the use of cumulative risk assessment 

process is appropriate. 

• How the uncertainty assessment will be used to help interpret the cumulative risk 

assessment. 

• Any other intended uses of uncertainty. 

The differences between the three types of uncertainty (page 58) are not clear. Specifically, how 

is uncertainty of type 3 different from type 1 and 2 uncertainties? 

I would incorporate Section 4.2.3 into Section 3.2.2. 

4.a) Vulnerability 

Discussion was easy to follow. 

4.b,c) Cumulative risk assessment involving chemical and non-chemical stressors and 

different Types of risk 

The Framework omitted the discussion of the surrogate approach, where the toxicity of a 

mixture or its fraction is expressed in terms of the concentration of a surrogate. The implied 

assumption is that the concentration of each component of a given mixture rises or falls with 

rising or falling concentration of the surrogate. The surrogate is assigned potency of the entire 

mixture (regardless of what the potency of the surrogate itself may be) and the risk is the product 

of the intake of the surrogate and its assigned potency. This approach has been used, for 

example, for the polycyclic aromatic hydrocarbon fraction of a mixture by World Health 

Organization (Air Guidelines for Europe) and in other guideline reports and assessments. 

The key to assessment involving different stressors and different types of risk is to identify ways 

to express the information using a common metric. Various approaches, which allow for 

expression of diverse stressors using a single metric (in use and under development), are 

F-67 
discussed in Sections 3.2 and 3.3. I find these sections useful and informative; they provided a 

good summary with references to key reports in the area. 

It is clear that some means of dealing with a wide range of risk data with different metric will be 

needed if the cumulative risk assessment is to be a tool accessible to a wider range of 

stakeholders. A section should be dedicated to the discussion of (1) why a single common metric 

or few metrics is desirable and how the metric(s) would be applied, (2) situations where risk 

information is normally presented in a single metric (e.g. oral carcinogens), (3) combination of 

different effects into a single metric currently in use (e.g. hazard index and other indexes), (4) 

other proposed metrics (see Section 3.3). Much of this information is available in Section 3 of 

the Framework, but needs to be isolated from other issues covered in the section. A better focus 

on the purpose and application would aid in understanding. 

F-68 
4. References 

ILSI, 1999. A Framework for Cumulative Risk Assessment. International Life Sciences Institute, 

Washington, DC. ISBN 1-57881-055-8 

NRC, 1983. Risk Assessment in the Federal Government: Managing the Process. Committee on 

the Institutional Means for Assessments of Risk to Public Health, Commission on Life Sciences, 

National Research Council. National Academy Press, Washington, DC. ISBN 0-309-03349-7 

USEPA, 1999. Integrated Environmental Decision-making in the Twenty-first Century: 

Peer Review Draft, U.S. Environmental Protection Agency, Science Advisory Board 

Integrated Risk Project Steering Committee 

USEPA, 2000a. Supplementary Guidance for Conducting Health Risk Assessment of Chemical 

Mixtures. Risk Assessment Forum, Office of Research and Development, U.S. Environmental 

Protection Agency. Washington, DC. EPA/630/R-00/002 

USEPA, 2000b. Toward Integrated Environmental Decision-Making. Science Advisory Board, 

U.S. Environmental Protection Agency. Washington, DC. EPA-SAB-EC-00-011 

F-69 
Review by 

Barry Ryan - Emory University 

F-70 
Review of Framework for Cumulative Risk Assessment 

Pre-Meeting Comments 

P. Barry Ryan 

May 27, 2002 

Overall Impressions 

As I read the Background and Charge questions for this review prior to reading the 

document itself, I began to formulate in my own mind what I believed would be an appropriate 

document fulfilling the intent I had discerned. Upon my reading of the document, I felt that my 

vision for the piece was not being realized even remotely. I encountered a document that was, I 

felt, quite pedagogical in nature and not really descriptive of what must be done in a cumulative 

risk assessment. And I was disappointed. 

But as I continued to read the document, I found myself making mental notes regarding 

the use of the document and the references therein in a second level risk assessment class that I 

teach. Here is a good glossary of terms- there is an excellent reference list- over here is a good 

discussion of what must go into a general risk assessment and a higher-level assessment that 

includes non-chemical components influencing the risk. 

And so I continued to read and my disappointment was assuaged to a large degree. The intent of 

the document is not to give a detailed accounting of the methods by which cumulative risk 

assessments are to be done but to supply, as the title would suggest, a framework upon which to 

develop thinking about doing these types of assessments. With this new mindset, I reevaluated 

my view of the document. While not perfect, I believe this document is an excellent product 

with this intent. The list of authors, contributors, and reviewers speaks to the perceived 

importance of this document within EPA. Further, the fact that at least three stakeholder 

meetings were held to discuss it further suggests its utility. 

I do not, however, think that the document will be well-received by all. The Ateachers@ of risk 

assessment, and I fall into this category, I think will find it a better document than the Adoers@ of 

risk assessment. The former will find the pedagogical approach to their liking and will likely see 

such a document acting as a central piece in a teaching curriculum whether it be in the classroom 

setting as I might implement, or in the private chambers of EPA regulators and other risk 

assessors attempting to design cumulative risk assessments in the future. The later will be 

looking for the meat and details of such an implementation. I think that the concerns of the 

Adoer@ group will be partially allayed by the numerous text boxes throughout the document 

giving specific examples of the way things have been done and by the various appendixes that 

address some Adoer@ issues. I must congratulate EPA on these text boxes. Many describe 

projects that were done that had poor outcomes and are used to illustrate ways of thinking that 

were not appropriate or were incomplete. Of course there are success stories as well, but it is the 

bad outcomes that teach us where we had poor thinking in the past and how to avoid such in the 

future. 

F-71 
The document still has a number of flaws in my opinion. It is quite long and, at times, pedantic. 

There is a significant amount of material that is quite generic applying to simple risk 

assessments, what one might call aggregate risk assessments, and other types of studies that do 

not fit any of the definitions supplied of cumulative risk assessments. While it is certainly EPA=s 

prerogative to include these more generic discussion in this document, it could be made more 

hard-hitting (and considerably shorter) by referencing, rather than repeating, this generic 

material and focusing on the specific components unique to the cumulative risk assessment 

process. However, an alternative point of view would be to include such in this presentation so 

that the relative neophyte would have all of this information at her or his fingertips in a single 

document. The more experienced risk assessor could then choose to skip over the material that 

was well known and focus on the newer components. 

Perhaps the best Anew@ material, in my opinion, is the expansion of the cumulative risk 

assessment concept past the FQPA definition to include all factors- whether chemical, physical, 

or social- that affect risk and risk characterization. This, I believe, represent the single biggest 

Abreakthrough@ in this report. 

Charge Questions 

The following questions are provided to help guide the peer review and associated 

discussions during the peer review workshop. When considering these topics, keep in mind the 

purpose of a framework is to identify and Aframe@ key issues for a broad audience of readers. 

Therefore a balance must be struck between adequately characterizing the issues and providing 

a detailed, comprehensive technical discussion. 

1. Comment on whether the Framework adequately captures, described, and reasonably 

organizes the key issues for cumulative risks assessment so as to serve as an adequate 

foundation for the development of future guidance. In your comments, please address the 

following questions. In answering each question, provide a supporting discussion that 

highlights any areas of the Framework that may need to be clarified and relevant topics that may 

be missing from the current Framework document. Include references to any published 

literature that could help improve the completeness and clarity of the Framework. 

a. Does the Framework document capture the relevant terminology?

Not only does the Framework capture the relevant and essential terminology, it expands on 

certain terms. In particular among these is the expansion of the term Acumulative risk 

assessment@ to go beyond the Acommon mechanism@ definitions of the FQPA to a more 

generalized definition involving other factors such as physical environment and non-chemical 

exposures that may influence the outcome. The Glossary presented on pages 64-65 gives a good 

compendium of definitions, but could be expanded to include more terms and thus be even more 

useful. 

F-72 
b. Does the Framework document provide an adequate assessment structure? 

The Introduction details the assessment structure with Figure 1-3 presenting such in graphical 

form. In later sections, e.g., Figure 2-1, a modified version of Figure 1-3 is used to announce 

and kick off each new section. I think that consolidating the design would be an appropriate 

strategy, i.e., developing a single form for this presentation whereby the new section could be 

highlighted. It is almost done at this point, but there is enough difference between the figures as 

to cause some confusion. 

c. Does the Framework document outline the relevant methods for cumulative risk? 

The document offers examples of cumulative risk assessments, including the expanded 

definition, and offers some details in Section 3. However, if the reader is looking for a cookbook 

method of performing a cumulative risk assessment, this is not the document. Again, the 

concept of a framework for performing cumulative risk assessments is given, but the step-bystep procedure for carrying it out is left to the risk assessor/risk manager to design. 

2. Does the Framework document include any scientific or technical inaccuracies in its 

presentation of terminology, assessment structure, and methods? Please identify any problem 

areas and propose revisions or other actions that will result in a scientifically sound and 

supportable discussion. With respect to methods, comment on whether the Framework 

adequately conveys the state of the science with respect to currently available cumulative risk 

assessment methods/approaches and the areas that are in need of further research and 

development. 

Over the last ten years, there has been a great deal of work, much of it bordering on overlypedantic, describing the differences between exposure, dose, delivered dose, biologicallyrelevant does, toxicologically-relevant does, etc., that tended to bore the practitioner to tears. I 

would not advocate such a rehashing of these definitions, but I would prefer to see some 

discussion of the terminology of exposure assessment and dose-response characterization in this 

document, even if only in the Glossary. 

The assessment structure, I believe, is adequate to convey the framework concept. More could 

be included, but I think that little insight would be gained by such inclusion. 

The Methods component could be increased in detail. However, this is a decision to be made 

based on the intent of the document. To stay with in the rubric of a framework, little or perhaps 

no additional material is needed. However, to appease the Adoers@ as discussed above, more 

detail could be provided regarding the execution of cumulative risk assessments. Again, a 

significant allaying of fears has been accomplished through the use of Text Boxes throughout the 

document. Inclusion of a couple more in the Methods section may make for a document better 

received by those looking for specific guidance on performing such assessments. 

3. Uncertainty analysis is an important part of risk assessment (and policy analysis in 

general). However, historically, dealing with uncertainty has been a shortcoming of many 

assessments. Cumulative risk assessments present new challenges for uncertainty analysis. 

F-73 
Perhaps more important, assessing cumulative risks will involve the use of Asoft@ assumptions. 

These are assumptions which may have a high degree of uncertainty that is difficult (or not 

possible) to quantify. Comment on whether the Framework adequately characterizes the 

importance of uncertainty analysis in cumulative risk assessment. What additional discussions 

of uncertainty should be included in the Framework (and in what sections of the document)? 

Uncertainty is discussed at length in Section 4. The discussion focuses not merely on the 

differences between variability and uncertainty and the effects of parameter uncertainty, but it 

also discusses other forms of uncertainty such as model misspecification uncertainty. Further, 

there has been discussion of the Asoft@ assumptions and their influence on the expected results. 

Little quantitative guidance has been supplied, but the fact that the topic was even discussed is a 

big step in the right direction. 

4. The following topics have been identified by the RAF technical panel as technically 

difficult areas that will pose challenges to cumulative risk assessment. Comment on the 

adequacy and accuracy of the Framework=s presentation in each of these areas. 

a. Vulnerability 

As applied to cumulative risk assessment, it is useful to think of four components of 

vulnerability: the susceptibility or sensitivity of the human or ecological receptors; the 

differential exposures of the receptors; the differential preparedness of the receptor to withstand 

insult form exposure; and the differential ability to recover from effects. The issue for 

cumulative risk assessment is how to consider these aspects of vulnerability and their potential 

impacts on risk. This is highlighted in the framework as an issue in need of further research and 

development. Comment on the discussion of vulnerability on the draft Framework. Has the state 

of the science been captured in the discussion? How can the discussion of this issue be 

improved? 

I do not claim expertise in the assessment of vulnerability in populations, but the discussion 

seemed relatively complete to me. The discussion, presented on pages 34ff covers several 

aspects and supplies a number of references, including EPA-developed reviews. I leave it to 

others to assess the state-of-the-science level of the presentation; to me it appears adequate. 

b. Cumulative Risk Assessment Involving Chemical and non-Chemical Stressors 

Viewing cumulative risk assessment as an evaluation of the accumulation of stressors presents 

many challenges. These may be seen when attempting to combine, in some meaningful way, the 

risks from multiple chemicals that may act as synergistic, antagonistic, or additive doses leading 

to a single effect. The situation is exacerbated when non-chemical stressors (e.g., radiation, 

biological agents, and psychological stress) are considered. Comment on the Framework=s 

discussion concerning the combining of disparate environmental stressors. In commenting, 

consider the state of the science with respect to understanding the effects of different stressors 

acting together (e.g., chemical exposure and viral infection). What can be added to the 

framework to adequately convey the state of the science in this area? 

F-74 
The framework states that little work has been completed in this area that would address the 

simultaneous measurement of various chemical and non-chemical stressors on human or other 

receptors. I am unaware of any such work beyond the most rudimentary discussed in the text or 

Text Boxes. Examples include measurement of temperature and pollutant exposure, measuring 

of two or a small number of chemical exposures at the same time, or epidemiological 

investigations of groups with different sensitivity and exposure. Other experts may be able to 

site a small number of specific studies addressing such issues, but it is clear that no truly 

systematic investigation of these effects has been done. More work is certainly needed in this 

area. 

c. Cumulative Risk Assessment Involving Different ATypes@ of Risk 

Conveying combined risks from multiple chemical and non-chemical stressors, in a meaningful 

way, is the ultimate challenge for cumulative risk assessment. Experience in this area is 

extremely limited. Indices, common metrics (e.g. Disability Adjusted Life Years- DALYs) and 

graphical (e.g., GIS) approaches have been explored by much methods development work 

remains to be completed. Cumulative risk assessment can be a valuable part of the decision 

making process, but only of results are conveyed in a meaningful way. Comment on the 

Framework=s discussion concerning the combining of separate measures of risk. Do the 

examples approaches discussed capture the sate of the science in this area? In particular, 

consider the real of valuation (i.e. the assignment of societal values to disparate health 

outcomes) implicit in some of the approaches. Suggest changes or additions that may improve 

this discussion. 

The framework presents a discussion of attempts to combine different measures of risk with 

wildly different metrics or, perhaps, no metric at all. The authors give several examples and 

thereby attune the reader to the difficulties. I think that this is an excellent introduction to 

guidance, but that real guidance cannot be forthcoming because it is not yet available. However, 

this is not my area of expertise and others more experienced in this area may offer more insight. 

Specific Observations 

I noted a relatively small number of typographical problems and spelling errors that 

would likely be picked up on a final edit. I will supply these at the review workshop. Some of 

the graphics, notably Figures 1-1, 1.2, 2-2, and 2-3 appear to be of low quality and should be 

replaced by high quality art work. Similarly, the lead figures for Sections 2, 3, and 4 should be 

reworked and replaced with high quality art work. As mentioned previously, the art work for 

these figures and Figure 1-3 should be reworked to be consistent. EPA should consider the use 

of color in some of these figures to aid in clarity. 

F-75 
Review by 

Jennifer Sass - Natural Resources Defense Council (NRDC) 

F-76 
June 22, 2002 

Comments on the Technical Peer Review Workshop on the EPA Risk Assessment Forum

Draft Framework for Cumulative Risk Assessment: June 4-5, 2002 

Comments submitted to David Bottimore, Versar 

Cc: Peter de Fur, Workshop Chair 

Comments by Jennifer Sass, Peer Reviewer 

Natural Resources Defense Council 

Washington, DC 

Tel: 202-289-2362 

Email: jsass@nrdc.org 

AREAS OF UNCERTAINTY ARE OFTEN AREAS OF VULNERABILITY 

The identification of uncertainty, susceptibility, and vulnerability issues need to be introduced 

early in the CRA framework, and carried throughout. These concepts are not separate from each 

other, but, rather, are closely related. Uncertainty within the CRA will mainly be in those areas 

in which the least data is available. It is commonly accepted that the areas of least data are often 

the most vulnerable populations and geographic areas. For example, minority ethnic groups are 

under-represented in the government data on eating patterns. Therefore, an area of uncertainty in 

the organophosphorus CRA is the eating patterns, and associated food-pesticide exposures, 

among certain ethnic groups. Likewise, fetuses, infants, and young children are underrepresented in the national eating pattern data; yet, they represent a population with eating 

patterns usually consisting of a very limited selection of foods, substantially different from the 

adult population. For example, apples, which have an unusually high level of the most toxic 

pesticides are also the most popular fruit and juice choice for children. Therefore, compared 

with adults, children are more highly exposed to toxic apple-pesticides, and more at risk of 

permanent damage from the neurotoxic pesticides. These examples are meant to demonstrate the 

widely recognized paucity of data on what are often the most vulnerable subpopulations. 

Geographic areas that are most at risk are often similarly under-represented in the data, and therefore 

represent the areas of greatest uncertainty in an ecological CRA. The unique and highly vulnerable 

ecosystems of deserts, for example, are poorly understood. Yet, they may be just as exposed to 

many air pollutants as more robust and well-understood climate zones, with less ability for recovery 

and repair. Similarly, forests in the temperate climate of the West Coast may take a decade to 

recover from a destructive exposure, whereas forests in the harsher Northern climates of the 

Canadian Shield may take over 50 years to achieve the same regrowth. Recognizing the unique 

vulnerabilities of geographical regions should be part of the vulnerability analysis within a 

cumulative risk assessment. Where these vulnerabilities are poorly understood, the uncertainty must 

be estimated and included in the assessment. 

F-77 
IDENTIFICATION OF UNCERTAINTY 

The identification of uncertainty early in the CRA process must include the identification of 

sensitive subpopulations, sensitive geographic areas, and the “voiceless” or marginalized 

populations who are under-represented in the data available. This should be done in the context of 

CRA. A specific discussion of how uncertainty will be integrated into the cumulative risk 

assessment process would be helpful. Examples of uncertainty unique to CRA, and how it is 

acknowledged (quantitative, qualitative, or both) and integrated into the CRA process would be very 

instructive. The Agency need not invent the wheel; where others have already completed such 

cumulative risk assessments, these examples would be instructive to both the naïve and experienced 

reader. Examples should be drawn from national and international initiatives, from aboriginal 

approaches, from community-input approaches, and from traditional EPA approaches. Both 

successful and unsuccessful examples would be instructive. 

A PROTECTIVE APPROACH SHOULD BE BUILT INTO THE FRAMEWORK 

Since it is recognized that the greatest uncertainty in the CRA will often be areas of greatest 

vulnerability, and since it is also recognized that a public health and environmentally protective 

approach necessarily embodies the principles of precaution and conservativism, this should be 

overtly stated within the framework. The framework should embody the principles of the CRA, like 

a skeleton upon which layers of muscle will be laid. This principle should be guided by precaution 

and protection, even while recognizing the uncertainty in the process. Just as the advisory panel was 

quick to recognize that uncertainty cannot be identified late in the CRA process, but must be noted 

throughout, so, too, must the framework approach to uncertainty be noted throughout. It is rightly 

stated in the framework that the Agency must act, even in the face of uncertainty, to protect the 

environment and the public health. This protective approach should be formed in conjunction with 

the identification of uncertainty, to better ensure that all uncertainty is identified and included, where 

appropriate. Without stating such an approach within the framework, the Agency risks making 

policy-decisions on what uncertainty to include, and what to exclude, which may not be consistent 

with a protective approach. Complete disclosure of all uncertainties is essential, early and 

throughout the process, and therefore must necessarily be built into the framework. 

THE ANALYSIS SHOULD BE ABLE TO IDENTIFY RISK DRIVERS; qualitative, 

quantitative, attributable, avoidable, baseline 

While the Agency is tasked with the difficult process of presenting the cumulative risk, or total load, 

to a person, community, or ecology, the Agency must also be able to work backwards, breaking 

down the cumulative risk assessment to its component parts. This may not be possible in great 

detail, but in general any cumulative risk assessment must be able to not only describe the 

cumulative toxic load, but also to identify the main risk drivers contributing to the toxic load. This 

includes identifying the “background” or baseline contribution, which may be very significant. 

F-78 
Additional exposures above background must also be identified with as much detail as possible, 

including an estimate of the magnitude of exposure each contributes. Likewise, the distinction 

between attributable risk and avoidable risk must be made, with risk drivers identified and 

quantified. Without this ability to move both forwards and backwards through the cumulative risk 

assessment, it will not be helpful in the risk mitigation phase. 

Much information regarding exposure and risk may be qualitative in nature. The collection of 

qualitative information may be very valuable to the cumulative risk assessment, and should be 

encouraged. Qualitative issues and priorities may differ across communities and geographical 

regions, even where measurable endpoints may not. For example, ability to relocate, access to 

health care, ethnic make-up of the community, closeness of the community, perception of risk, 

perception of cause of risk, and other factors may influence the ability of any cumulative risk 

assessment to be successfully accepted by the affected population. Without understanding such 

qualitative contributors, the community and the risk assessor may be speaking at cross purposes. 

Thus, this information should be integrated, as appropriate, in the cumulative risk assessment. And, 

like the measurable contributors, these qualitative contributors should ideally be able to be 

aggregated and disaggregated, as needed, to identify the risk drivers. 

ENVIRONMENTAL JUSTICE ISSUES 

The recognition of environmental justice issues runs throughout these comments; a) in recognizing 

that vulnerable populations are often where the greatest uncertainty lies, b) in recognizing that 

qualitative issues and priorities will differ with socio-economic status, ethnicity, and community 

bonds, 3) and in the call for public health and environmentally protective language to be built into 

the very framework of all cumulative risk assessments. These issues cannot be given meaningful 

discourse without including environmental justice considerations. Similarly then, the language of 

environmental justice should also be built into the framework, so that each and every cumulative risk 

assessment which arises from this framework includes environmental justice considerations at all 

stages. By building this language into the framework, each cumulative risk assessment is 

contextualized within the community and ecosystem it is meant to serve. This will contribute to the 

successful acceptance and implementation of the cumulative risk assessment by the local 

community, and give the assessment its best chance of success. 

F-79 
Review by 

Paul Schlosser – CIIT Centers for Health Research 

F-80 
Review of EPA/630/P-02/001A: "Framework for Cumulative Risk Assessment" (external 

review draft, dated April 23, 2002) 

Prepared by Paul M. Schlosser, CIIT Centers for Health Research 

General comments 

As government regulations now mandate cumulative risk assessments for 

pesticides acting by common mechanisms and there is a generally heightened concern 

that regulation of exposure to individual stressors may not adequately protect against 

cumulative effects, it is appropriate for the EPA to provide guidance on how one should 

go about assessing risks from multiple stressors or agents. Because the number of 

potential stressor combinations that might be considered is enormous and the methods 

for predicting and quantifying combined risks are still at the beginning of development, it 

is appropriate that this guidance be in the form of a framework or overall structure that is 

not proscriptive. The draft document achieves this goal quiet well, with important 

emphasis on the extent to which methods development is needed. 

While the framework seems to lay out very ambitious goals for risk assessments, 

these are likely necessary if cumulative risk is to be properly characterized and the 

results are to meet the needs and expectations of concerned parties. 

A general area where the document could most use further discussion is in the 

combination of different responses into a total measure of "risk", or at least providing a 

level of comparison (section 3.3.3, pp. 48-52). It is suggested that the assessment may 

or may not use a common metric to 'sum up' all effects. But the underlying key issue is 

that the assessment should provide managers a way of weighing and comparing 

different responses, since one would wish to address the most important first and in 

some cases there may be risk-risk trade-offs. Such a weighting is necessary for 

responses to be combined into a single metric, but the discussion of those is focused 

more on a discussion of different types of such metrics rather than the fact that it 

provides such a weighting. Even if a single metric is not used, the input of stakeholders 

and experts could be used to at least rank different responses in level of importance or 

concern, and it would be a shame not to take advantage of their involvement to gather 

this information. 

Charge Questions 

1. I found the document to be well-organized overall, and except for the issue of 

endpoint "weighing" mentioned above, am not aware of any key points that have been 

missed. The issues are described and addressed at appropriate length, with good 

references to supporting materials. Specifically: 

a) The relevant terminology is appropriately defined and used. 

b) The structure defined appears to be comprehensive and broad enough to be 

F-81 
applicable to the wide range of assessments anticipated, and yet specific enough 

in listing the steps involved and what should be achieved to provide meaningful 

guidance. 

c) As the framework correctly points out, methods for predicting and quantifying 

the combined effects of multiple stressors are still in the early stages of 

development. The document describes and references a number of methods 

described in the literature for dealing with combinations of stressors, but some 

other work that might be mentioned is that of Krishnan and colleagues on 

predicting pharmacokinetic interactions of chemicals (Haddad et al., 1999; 

Haddad et al., 2000; Haddad et al., 2001). 

2. There appear to be no significant inaccuracies in the document's description of 

current methods, the structure, or terminology. Some detailed corrections are noted in 

the "Specific Observations" below. As is noted throughout the document, this is an area 

where considerable research and development are still needed, and hence a detailed 

specification of methodology would be inappropriate. 

3. The document does a good job of explaining the importance of uncertainty analysis 

and what types of uncertainty one might wish to understand. Uncertainty analysis is a 

very large area of research, which the document cannot address in great detail. 

However I believe some comments should be made about the magnitude of effort 

potentially required to fully characterize all sources of uncertainty, particularly where 

biologically based modeling is used. 

An example is the fairly recent risk assessment for formaldehyde developed by 

CIIT and collaborators (CIIT, 1999). In the biologically based RA, computational fluid 

dynamics (CFD) models are used as inputs to pharmacokinetic models of formaldehyde 

and DNA-protein-crosslink (DPX) dosimetry, which had adjustable parameters fit to 

data. The output of the PK/DPX model was then used as input to multi-stage clonalgrowth model, which again had adjustable parameters fit to data. To characterize the 

effects of uncertainty in the CFD model on the final output would require re-estimation of 

the PK/DPX parameters (and their upper and lower bounds) for the upper and lower 

bounds of the CFD model. The resultant ranges of PK/DPX predictions would then need 

to be used to estimate the ranges in clonal growth parameters. The clonal-growth 

predictions would then need to be calculated as a function of all of that underlying 

uncertainty. The effort required to do this would be quite substantial – on the order of 

months or perhaps a year, given that the computational effort for fitting the clonal growth 

model, which involved Monte-Carlo simulations. While this is doable, the personnel 

resources required were simply not available. 

In short, quantitative uncertainty analysis involves testing multiple realizations of 

the risk assessment calculation, and when that calculation itself is computationally 

complex, the resources required can become prohibitive. Thus, as for the entire RA 

process, the resources available may proscribe what is achievable in the uncertainty 

analyses. 

F-82 
I firmly support the idea that all underlying assumptions and uncertainties should 

be listed, though. In the case of using quantitative models, the parameter uncertainties 

should be provided and some sensitivity analysis performed to identify those which are 

most significant to the calculations. 

4. RAF technical panel issues 

a. Vulnerability 

I have considerable concerns with the division of vulnerability into the four 

components as described. Differential exposure is clearly defined. But given that doseresponse and/or exposure assessments are basic components of any risk assessment, 

I am not sure of the value of including this as a special definition. Pointing out that past 

exposures may be pertinent is useful, but if those exposures make an individual more 

prone to future insult, that is a component of susceptibility. 

I am having a very hard time distinguishing susceptibility from differential 

preparedness to withstand the insult. I don't see the distinction. 

Differential preparedness to recover is not a factor if in fact it is the acute 

endpoint that occurs: the fact that I might recover from hearing loss doesn't matter if the 

risk assessment is to avoid acute hearing loss in the first place. On the other hand, if it 

is the chronic endpoint that is of concern, then differential ability to recover is another 

component of susceptibility. Thus, depending on the nature of the endpoint, this is 

either irrelevant or not distinct from susceptibility. 

In summary, I believe that vulnerability is really a combination of two 

components: differential exposure and differential susceptibility. Differential exposure is 

handled by properly accounting for the dose-response relationship(s) and individual or 

population exposure assessment. Susceptibility in turn has two components: 

preparedness to withstand and (when a chronic endpoint is evaluated) ability to recover. 

The document can then note that both of these may be impacted by prior exposures. 

I agree completely, though, that by accounting for multiple factors, cumulative 

risk assessments should be uniquely suited to addressing issues of vulnerability. 

F-83 
b. Chemical & Non-Chemical Stressors 

I do not see that determining cumulative risks for chemical and non-chemical 

stressors is inherently more problematic or difficult than working with multiple chemicals 

that act by different mechanisms or modes of action. My awareness of disease 

probability models coming from the field of medicine is that these combine risk factors 

that are diverse in nature as easily as they combine similar factors. When trying to base 

the risk on a mechanistic understanding or model, the real challenge is in combining the 

effects of different stressors which act by different mechanisms, whether those 

stressors are all chemical or not. The long-term approach that will best enable 

predictions of cumulative response from differing agents is the development of 

biological systems models such as Entelos' PhysioLab technology, such that the 

impacts of multiple stressors can be simulated. 

c. Different "Types" of Risk 

Combining risks of different types of responses into a single metric is clearly a 

key issue. Any such combination must implicitly make use of a valuation which 

converts the likelihood or degree of each separate response to a single metric, be that 

QALYs, DALYs, or dollars. For site- and region-specific RAs with a clearly defined and 

manageable number of stakeholders, it is most appropriate that those stakeholders 

define the relative valuation for each response considered to achieve a metric to be 

used in management decisions. In fact, even if this is not explicitly done, any risk 

management decision cannot avoid implicitly weighting or valuing multiple responses. 

But where a relatively small number of responses are being considered, it may 

often be possible to not combine them. In particular, if the objective is to assure that 

each response remains below some 'maximum impact' level which is response-specific, 

then the achievement of that objective can be pursued without a combined metric. 

Specific Observations 

Figures: The figures are quite grainy and the text in them is very hard to read (eg, Fig. 

2-2 and 2-3). 

P. 7, lines 12-17: It should be noted that description of interactions may simply be to 

say that the different stressors act independently (as per toxicological independence 

defined on p. 38), and hence there is no interaction. 

P. 9, lines 40-41: Why is it preferable that the endpoints be independent? And what is 

meant by "independent"? In an aquatic system, water turbidity will effect the health of 

some fish, so these are not independent, but I do not see why that dependence should 

create a bias against using both turbidity and fish health as endpoints. 

Section 2 – Planning, Scoping, and Problem Formulation Phase 

What aspects of this section, if any, are unique to cumulative risk assessment? It 

may be that none are unique – it seems fairly general to me – and that is OK. But then 

F-84 
this should be acknowledged. Alternately it might be worth adding material on what one 

should do in a cumulative RA that differs from that for a single-agent ... such as 

identifying what stressors are to be considered. 

One point that probably can't be over-emphasized: while it is possible to assure 

that all stakeholders have been heard and their opinions given due consideration and 

weight, that doesn't mean that everyone will get what they want. 

P. 17, line 23: The 2nd half of this paragraph, starting with "For example...," seems 

totally disconnected from the first. I was confused by it. 

P. 41, last para. (goes onto p. 42): While it may often be the case that the response 

assessment depends on the timing of multiple exposures, and hence cannot be 

performed independently of the exposure assessment, I do not believe the converse will 

typically be true. In other words, I expect that it will usually be possible to conduct the 

exposure assessment first, rather than together with the response assessment, 

although the response assessment will have to wait for those results. 

3.3.2 – Biomarkers and Biomonitoring (pp. 47-) 

Many biomarkers are not cumulative in the sense defined here because they are 

chemical- or stressor- specific. For example hemoglobin adducts formed from benzene 

oxide are such a biomarker. Thus the statement on p. 48, line 9, is simply incorrect – 

that may be true of biomarkers of effect but not biomarkers in general. A better term 

might be "aggregate". However it should also be recognized that biomarkers do not 

necessarily reflect the entire history of exposure. As red blood cells turn over in the 

body, hemoglobin adducts will be lost: so the current level of adducts will have little 

dependence on benzene exposures that occurred 6 months ago. Thus biomarkers are 

only cumulative over a particular time frame. 

Risk Characterization 

It's important to document how stakeholder input has influenced the process, and also 

where suggestions were not included and why. This documentation provides the 

assurance that individuals have at least been heard and supports their participation. 

Appendix C – Background Exposures 

For the purpose of identifying management options, in particular how well exposures 

can be reduced and the most effective means, identifying background levels vs. those 

from specific sources is critical. 

P. 50, line 35, and p. 51, lines 27-29: While the meaning of common metrics may be 

less clear than specific individual responses, it is not necessarily obvious why this 

should make them less suitable for absolute risk-management decisions. Please 

provide some explanation. 

Post-Meeting Addendum 

To follow up on my comments during the meeting, I mentioned software tools that are 

now being developed to describe integrated physiological systems which have the 

potential to allow for mechanistic modeling of the effects of multiple agents on a given 

F-85 
system. While it is not my intent to promote commercial products, the fact is that 

several of these have been privately developed and are currently only available on that 

basis. Nevertheless, it is worthwhile to be aware of them for the future of cumulative 

risk assessment. 

Particular resources are: Physiome.org (http://physiome.org/), a collaborative effort to 

link a number of models; the Physiome project at the University of Auckland 

(http://www.bioeng.auckland.ac.nz/physiome/physiome.php); PhysioLab by Entelos, Inc. 

(www.entelos.com) (I believe they have models for asthma, cardiovascular disease, and 

kidney disease); and the Immune System Modeling & Simulation tools at Princeton 

University (http://www.cs.princeton.edu/immsim/). This is not a comprehensive list of 

such modeling efforts. Further, these are mostly if not completely still in development 

and likely in need of modification to suite a toxicological analysis. But I believe these 

will become increasingly important to biomedical research in the future and could be 

powerful tools for cumulative risk assessment. So I think they should at least be 

mentioned in the framework document. 

References: 

CIIT (1999). Formaldehyde: Hazard characterization and dose-response assessment 

for carcinogenicity by the route of inhalation (Revised edition): Chemical Industry 

Institute of Toxicology, Research Triangle Park, NC 

Haddad, S., Beliveau, M., Tardif, R. and Krishnan, K. (2001). A PBPK modeling-based 

approach to account for interactions in the health risk assessment of chemical mixtures. 

Toxicol. Sci. 63, 125-131. 

Haddad, S., Charest-Tardif, G. and Krishnan, K. (2000). Physiologically based 

modeling of the maximal effect of metabolic interactions on the kinetics of components 

of complex chemical mixtures. J. Toxicol. Environ. Health, Part A. 61, 209-223. 

Haddad, S., Tardif, R., Charest-Tardif, G. and Krishnan, K. (1999). Physiological 

modeling of the toxicokinetic interactions in a quaternary mixture of aromatic 

hydrocarbons. Toxicol. Appl. Pharmacol. 161, 249-257. 

F-86 
Review by 

Curtis Travis - Quest Technologies 

F-87 
Scientific Peer Review for Framework for Cumulative Risk Assessment 

By 

Dr. Curtis C. Travis 

Quest Technologies 

8112 Bennington Dr 

Knoxville, Tennessee 37909 

5/21/02

In general, I found the document to be well written, easy to understand, and to provide a 

comprehensive framework for evaluation of cumulative risks to humans from exposure to 

multiple stressors. The key elements of the proposed cumulative risk assessment framework are 

traditional and follow accepted risk assessment practice. The document discusses most of the 

major issues associated with the issue of cumulative risk. A strong point of the document is that 

it ties the proposed framework to other risk assessments (like pesticides or quality of life) 

currently being done in various parts of EPA. I think the document could be improved somewhat 

by giving more attention to steps in the process that are unique and critical to cumulative risk 

assessment. In general however, I believe that the current document represents a complete and 

comprehensive framework for analysis of cumulative risks resulting from exposure to multiple 

stressors. 

I first answer the questions posed to the reviewers. 

1. Does the Framework adequately capture, describe, and reasonable organize the key 

issues of cumulative risk assessment so as to serve as an adequate foundation for the 

development of future guidance? 

In general the document does an adequate job of describing the key issues of cumulative risk 

assessment. However, more attention needs to be given to steps in the process that are unique 

and critical to cumulative risk assessment. The document needs to stress the following points: 

C A cumulative risk assessment must identify a unique well, defined assessment endpoint. . 

A general impact like the Ahealth of a community@ or Aoverall human health impact@ is 

not well defined. Assessment endpoints that cannot be linked with measurable attributes 

are not appropriate for a cumulative risk assessment. 

C A cumulative risk assessment must identify a common effect endpoint for all stressors. 

The combined impact of several stressors can only be evaluated when they have a 

common effect endpoint. 

C It should be pointed out that assessment endpoints that cannot be measured directly but 

can be represented by measures that are easily monitored and modeled still provide a 

good foundation for the risk assessment. 

C The document should separate stressors into those that directly cause the effect (direct 

stressors) and those that modify the magnitude of the effect (modifying stressors). 

F-88 
C Direct stressors can be separated into three classes: toxicologic independence (separate 

modes of action), toxicologic similarity (common modes of action), and toxicologic 

interaction (interacting modes of action). 

C Modifying stressors can act upstream of the mode of action of direct stressors, 

downstream of the mode of action, or both. 

C It may sometimes be desirable to have multiple assessment endpoints. However, they 

cannot be combined into single indicator of impact except through definition of some 

index of relative importance. This index definition process is a risk management step, not 

a risk assessment step. 

C The document should note that the Cumulative Risk Assessment Framework follows the 

NRC Risk Assessment Paradigm in all respects except that the toxicity component and 

the exposure component must be evaluated together rather than separately. 

2. Does the Framework document include any scientific or technical inaccuracies in its 

presentation for terminology, assessment structure, and methods? 

No, I did not find any scientific or technical inaccuracies in the document. However, I do not 

think the document gives sufficient attend to the problem of defining assessment endpoints and 

determining the relationship between stressors and assessment endpoints. I realize that detailed 

prescriptions cannot be given, but a general framework is within the scope of this document. 

3. Does the Framework adequately characterize the importance of uncertainty analysis in 

cumulative risk assessment? 

The document discusses the importance of uncertainty in the overall framework of the risk 

analysis process. However, I believe the discussion of uncertainty could be improved. The 

document correctly distinguishes between uncertainty and variability and discusses uncertainty 

in combining cancer risks estimates from exposure to multiple chemicals. However, these are not 

the main types of uncertainty that arise in cumulative risk assessment. There was no discussion 

of uncertainty in determining the combined impact of stressors nor how one must assess such 

uncertainty. 

I would like to see more discussion of the unique uncertainties that may arise in cumulative risk 

assessment. What are the major categories of uncertainties that will arise in cumulative risk 

assessment? Possible answers to this question are: 

C Direct stressor uncertainties (these can be assessed in traditional ways) 

C Specific modifying stress uncertainties (those that directly affect the mode of action of 

direct stressors). These uncertainties can be assessed in traditional ways since they deal 

with mode of action. 

C General modifying stress uncertainties (those that do not affect the mode of action, but 

increase general vulnerability). Maybe these could be handled through an uncertainty 

factor, like a factor of 10. 

F-89 
Are there any uncertainties that are unique to cumulative risk assessment? The answer is 

obviously yes. What can the document say about these? Currently, the document adds little 

insight into the issue of uncertainty in cumulative risk assessment. 

4. Comment on the adequacy and accuracy of the Framework=s presentation in each of 

the following areas: 

a. Vulnerability 

I like the fact that the issue of vulnerability is given a separate section in the document 

and that four components of vulnerability are identified. This approach needs to be taken 

with other important issues in the document. However, a more through discussion of 

vulnerability needs to be added to the document. The definition given of vulnerability is 

not clear. The document lists four components to vulnerability: The susceptibility of the 

human or ecological receptors, the differential exposures of the receptors, the differential 

preparedness of the receptor to withstand the insult from exposure, and the differential 

ability to recover from the effects. It is not clear how susceptibility differs from 

preparedness to withstand the insult or from ability to recover from the insult. It would 

seem that these are two aspects of vulnerability. 

The document needs to point out that vulnerability is an issue that needs to be considered 

while identifying stressors. 

Why is vulnerability a special issue for cumulative risk assessment? How do different 

stressors affect different aspects of vulnerability and then combine to affect overall 

vulnerability? 

As pointed our in the EPA guidance for Ecological Risk assessment: Sensitivity refers to 

how readily a receptor is affected by a particular stressor. Sensitivity is directly related to 

the mode of action of the stressor. For example, individual physiology and metabolic 

pathways influence chemical sensitivity. Sensitivity is influenced by individual and 

community life-history characteristics. Sensitivity may be related to the life stage of a 

receptor when exposed to a stressor. Frequently, the young are more sensitive to stressors 

than adults. Finally, sensitivity may be increased by the presence of other stressors or 

natural disturbances. For example, the presence of insect pests and disease may make 

plants more sensitive to damage from ozone. 

There is not much discussion of these issues in the document. 

b. Cumulative Risk Assessment Involving Chemical and non-Chemical Stressors 

I=m not sure as to the purpose of this question. In some sense, the entire document is 

about this issue. I could not find a section in the report that specifically addressed this 

point, but I would say it is adequately addressed throughout the document. 

F-90 
c. Cumulative Risk Assessment Involving Different Types of Risk 

I believe this issue is adequately addressed. 

Assessment Framework 

A major portion of the guidelines is devoted to describing the interaction among risk assessors, 

risk managers, and interested parties at the beginning (planning and problem formulation) and 

end (risk characterization) of the risk assessment process. The Guidelines emphasize that the 

interface between risk assessors, risk managers, and interested parties is critical for ensuring that 

the results of the assessment can be used to support a management decision. Most of this is 

standard guidance that would apply to any type of risk assessment, and I=m not sure of the value 

of repeating it yet once again, other than in general terms. 

This document does an adequate job of describing the key issues of cumulative risk assessment, 

but more attention needs to be given to steps in the process that are unique and critical to 

cumulative risk assessment. The assessment framework presented in this document follows the 

standard risk assessment paradigm, with three main phases to a cumulative risk assessment: (1) 

planning, scoping and problem formulation, (2) analysis, and (3) interpretation. It is a little 

disappointing that the document discusses little that is unique to cumulative risk assessment. 

Most of the discussion on scooping is standard risk assessment guidance. It focuses on 

stakeholder participation (page 14-16), scope of assessment (page 16-22), conceptual model 

(page 22-30). Not much of this discussion is unique to cumulative risk assessment nor does it 

highlight issues in cumulative risk assessment that need special attention. For example, this 

section does not say that a key step in the scooping process for cumulative risk assessment is to 

identify well-defined assessment endpoints. This is a critical issue for cumulative risk 

assessment. Figures 2-2 and 2-3 do include assessment endpoints as part of the process, but the 

text does not mention them as important issues. (I discuss this issue more below). 

With regard to identification of stressors, the text just says that it needs to be done. No extra 

guidance or discussion is given as to special issues that will arise when conducting a cumulative 

risk assessment. The major difference between a cumulative risk assessment and a regular risk 

assessment is the special attention that must be given to identification of stressors, assessment 

endpoints, and the relationship between them. This document needs to say more on the issue. 

Identification of assessment endpoints and stressors is a circular process. One can start with a list 

of possible stressors and identify possible effects. But then one has to identify those effects of 

primary concern and adjust the list of stressors to those that affect this endpoint. 

How does one go about grouping stressors in a way that will facilitate cumulative risk 

assessment? The EPA pesticide cumulative risk assessment guidance identifies chemicals with a 

common mechanism of toxicity as one possible grouping. This suggests at least two groups: 

stressors that directly cause the toxic effect (the assessment endpoint) and stressors that modify 

the magnitude of the toxic effect. Current risk assessment methodologies suggest methods for 

assessing the affect of single toxic agents (there are basically two methods: those for carcinogens 

and those for non-carcinogens). Progress has been made for assessing the affect of multiple toxic 

F-91 
agents with a common mode of action (the EPA pesticide guidance and the TEQ methodology 

for dioxins are examples). How does one handle modifying stressors? One way might be to add 

another uncertainty factor to account for the presence of modifying stressors. 

Approaches to Cumulative Risk Assessment 

The document lists four approaches to cumulative risk assessment. Attempting to define the 

different approaches to cumulative risk assessment is an excellent idea, but this section is not 

clear and confuses the issue. It purports to present four approaches to cumulative risk 

assessment, but does not do so in a clear manner. After reading this section, I would like to ask 

the reader to tell me what are the four methods? I don=t believe most readers could answer that 

question. I comment on each of these: 

C Using single stressor information to provide information on multi stressor situations. This 

approach uses a single assessment endpoint, but attempts to assess to impact of multiple 

stressors on this endpoint by considering the impact of stressors independently. With the 

current state of knowledge regarding multiple impacts, this is the approach that will be 

used most of the time. 

The document seems to apologize for this approach. It says it can be used for hazard 

identification. It says to go further in terms of quantitative risk assessment requires consideration 

of the potential for joint toxicity. It says transport and environmental transformation of a 

chemical can be influenced by presence of other chemicals. It says exposure to one stressor may 

influence the uptake of a second stressor. 

It is fine to point out the weaknesses of this approach, but the point of this section of the report is 

to list four approaches to cumulative risk assessment. This is one approach and the most likely to 

be used! The document should say something positive about the approach and why it is a sound 

approach. The document seems to say, this is an approach but it is not sound (A statement I do 

not believe). If this approach is not sound, what is the document suggesting be done? This is the 

only approach currently available. 

C Using information of multiple stressor interactions. This approach uses a single 

assessment endpoint, but combines all information on stressor interaction to arrive at an 

assessment of harm or risk. This is a true cumulative risk assessment. This is what one is 

always attempting to accomplish. The other three approaches are approximations to this 

situation. 

C Using decision indices. The document notes that one approach to addressing the 

complexity of cumulative risk assessment is to combine the various measures of harm 

into an index like the smog index or the Hazard Ranking System used in Superfund. This 

is a valid exercise. However, each of the individual components of the index must be 

well-defined and approached in some concrete way. Thus, this method is not so much an 

approach to cumulative risk assessment, as a way of presenting an array of complex 

information. You could say it is an approach, if you define the approach as charactering 

harm to a system by using multiple measures of stress. Thus, rather than using decision 

indices, this approach is really using multiple assessment endpoints. The decision indices 

are just a way of summarizing the result of using multiple assessment endpoints. 

F-92 
C Using a probabilistic approach. The document does not define what it means by a 

probabilistic approach, other than the intuitive notion that it must have something to do 

with probabilities. What is a probabilistic approach to cumulative risk assessment? What 

would be its main components? I=m actually not sure this is really an approach to 

cumulative risk assessment as opposed to a method for presenting risk estimates (its hard 

to tell since this approach was not defined in the document.) 

I believe there is another approach to cumulative risk assessment that is not mentioned in the 

document. That is a combination of analytic and safety factor. This approach would 

quantitatively estimate the risk from the direct stressors and then add a safety factor to account 

for the additional impact from modifying stressors. This approach might be the most currently 

viable and deserves some discussion in the document. 

Definition of Assessment Endpoint 

A major deficiency in the present document is its lack of discussion of assessment endpoints. A 

definition of assessment endpoint needs to be added to the document. The major points that need 

to be made are: 

C Cumulative risk assessments can only be performed on discrete well-defined endpoints. 

A general impact like the Ahealth of a community@ or Aoverall human health impact@ is 

not well defined 

C Assessment endpoints that cannot be linked with measurable attributes are not 

appropriate for a risk assessment 

C The combined impact of several stressors can only be considered when they have a 

common effect endpoint. If stressors have different effect endpoints (and there is a desire 

to include these stressors), then more than one assessment endpoint must be identified. 

It is possible to use a general assessment endpoint like Ahealth of a community@ in a cumulative 

risk assessment if one defines Ahealth of a community@ in terms of a collection of well-defined 

assessment endpoints. For example, in the AQuality-of-life Assessments (Appendix F)@, a set of 

Aquality-of-life@ criteria (assessment endpoints) is used to describe various aspects of quality of 

life. Each of these criteria measures some aspect of overall quality of life. However, each of 

these measures needs to be well defined, which is not the case in the Aquality-of-life@ example 

given in Appendix F. For example, one of the criteria given is Aeconomic well-being@ and 

another is Afairness@. These endpoints are not well defined and will need further refinement to 

obtain concrete, measurable definitions before they can be assessed. 

Stressors Stressors can be separated into the groups: those that directly cause an effect 

(direct stressors) and those that modify the magnitude of the effect (modifier stressors). 

These groups can be further divided. 

Direct stressors can be separated into three classes depending on their modes of action: 

toxicologic independence (separate modes of action to produce the same effect), toxicologic 

similarity (common mode of action to produce the same effect), and toxicologic interaction 

(interacting modes of action to produce a common effect). (Remember that the combined impact 

F-93 
of multiple stressors can only be assessed if they impact they have a common effect endpoint. 

This is a key observation that is missing from the document.) Methodologies for the first two 

have been or are in the process of being worked out. It is the third category that currently 

presents difficulty. 

The example in the text on page 38 of toxicologic independence is somewhat misleading. The 

example given is joint but low exposure to heat (causing minor elevated heart rate) and toluene 

(causing minor hearing loss). A common health effect endpoint has not been selected so this is 

not a cumulative risk assessment in the sense of evaluating the cumulative impact of two 

stressors on a common, well-defined endpoint. It is the impact of two separate stressors on two 

separate endpoints. (I think this type of assessment should be considered to be a component of 

the class of Aaggregate assessments@.) 

The document points out (page 38) that in the case of toxicologic independence, Aresponse 

addition@ can be applied for each adverse effect that the stressors have in common. The text goes 

on to say, AWhen all the single stressor risks are low, the joint risk of a common effect under 

response addition can be approximated by the simple sum of the single stressor risks.@ This is in 

the section on toxicologic independence. In fact, this is a rule that holds for all stressors 

regardless of mode of action. This point needs to be made clear in the document. 

Modifying stressors can act upstream of the mode of action of direct stressors, downstream of 

the mode of action, or both. Unfortunately, exactly where they act is often not known. Even 

more difficult to deal with is the fact that nothing may be known about the mode of action of 

most modifying stressors. 

Additivity of Effect 

The document discusses additivity of effect, but the discussion is disjointed. The document 

should make clear: 

C Additivity for Toxicologic Independence For stressors toxicologic independent 

stressors (separate independent modes of action to produce the same effect), the 

cumulative effect is the simple sum of the single stressor risks. The effect of modifying 

stressors on the toxicologic independent class is to modify the magnitude of impact of 

each direct stressor. Once the impact of modifier stressors on each direct stressor 

independently in known, the cumulative effect is the sum of the single modified stressor 

risks. Thus, in theory the methodology for performing a cumulative risk assessment on 

stressors with independent modes of action is known. This point should be made in the 

document. 

C Additivity for Toxicologic Similarity For stressors in the toxicologic similarity class 

(common mode of action to produce the same effect), the cumulative effect is found by 

dose addition (the relative potency approach). 

Modifying stressors in the case of toxicologic similarity can only act in one of three ways: they 

can act upstream of the common mode of action, downstream of the common mode of action, or 

both. 

F-94 
C If the modifying stressors act upstream of the common mode of action, they can change 

the relative potencies of the direct stressors; that is, increase or decrease the potency (as 

measured in dose equivalents) of the direct stressors themselves or relative to each other. 

In this case, the dose-response relationship between dose (as measured in dose 

equivalent) and effect is not changed. What may be changed in the effective dose of each 

direct stressor. 

C If the modifying stressors act downstream of the common mode of action, they can 

change the dose-response relationship, but not the potency (as measured in dose 

equivalents) of the direct stressors. Thus, is this case the modifying stressors leave 

unchanged the relative ratio of the dose equivalents, but change the overall magnitude of 

effect of the reference stressor. 

C Additivity for Toxicologic Interaction For stressors in the toxicologic interaction class 

(interacting modes of action to produce a common effect), Additivity occurs, as the 

document points out, when single stressor risks are low. This is rarely the case in risk 

assessment. 

A Single Metric of Impact 

The document spends considerable time discussing the possibility of developing a single metric 

for multiple types of hazard. The implication is that this process is somehow part of cumulative 

risk assessment. It is not! A cumulative risk assessment can only be performed on discrete welldefined endpoints. 

The document needs to be clear that there is a difference between the following two processes: 

1. Combining discrete impacts into an assessment of overall health 

2. Defining overall health in terms of discrete impacts. 

The document now intermixes these two concepts. For example, the following statements are 

made: 

C AIdeally a cumulative risk assessment would provide Yan integration of these projections 

into a qualitative characterization of overall potential impact to human health.@ Page 49. 

C ASome cumulative risk assessments may employ some sort of single, common metric to 

describe overall risk.@ Page 49. 

C Some cumulative risk assessments Arequire synthesizing a risk estimate (or risk 

indication) by Aadding up@ risks for different parts of the risk picture.@ Page 49. 

C ACombinations of many types of stressors with different endpoints in a single assessment 

will quickly cause the risk estimation step to become very complex and difficult.@

Page 54. 

All of these statements utilize the first bullet concept that it is possible to ACombine discrete 

impacts into an assessment of overall health.@ I do not believe that this is possible. All of the 

examples given in the document of developing a single index use the second bullet concept of 

defining overall health in term of discrete impacts.@ For example, on page 51, the document says, 

ARecently, EPA has been working on several index-based approaches to dealing with cumulative 

risk issuesY. This index uses a vulnerability index, and gauges the overall well-being of a locale 

F-95 
and various subpopulations.@ This is a proper use of an index to measure well-being since EPA 

has defined Awell-being@ to be that which the index measures. (It may or may not be true that this 

index actually provides a good measure of well-being). 

Combining Impacts 

The document needs to be clear that there is a difference between the following two processes: 

C Combining the impacts of multiple stressors to arrive at a combined impact of the 

stressors on a single, well-defined assessment endpoint (calculating impact of multiple 

stressors on single endpoint and combining them) 

C Combining multiple impacts on a system to arrive at an assessment of overall system 

impact (calculating impact of stressors on multiple endpoints and combining them) 

The document now intermixes these two concepts. For example, In Appendix A, page 84, the 

document says, AAnother key concept in the definition of cumulative risk assessment is that it 

represents the combined risk from multiple stressors. This implies that, in some cases, it may be 

necessary to combine disparate measures of risk (i.e., different types of effects) to simplify the 

expression of cumulative risks.@

The document is clear that an assessment of the effect of combined exposures (from different 

environmental pathways) to a single compound is not to be considered a cumulative risk 

assessment. It is also clear that multiple effects or impacts may be considered (page 9). The 

document says that a cumulative risk assessment would ideally Aprovide projections regarding 

the potential for a particular complex exposure to cause particular effects to different 

physiological systems, and also provide an integration of these projections into a qualitative 

characterization of overall potential impact to human health.@ This statement leads to two 

problems: 

1. It implies that Aoverall impact on human health@ is a proper assessment endpoint for 

cumulative impact assessment 

2. It raises the issue of how does one assess Aoverall impact on human health@ which is not 

addressed in this document. Simply combining individual impacts into an single index 

may not accomplish this. It may be better to leave the impacts separate. 

Specific Comments 

Page 2, footnote 2. The text says, AA stressor is a physical, chemical, biological, or other entity 

that can cause an adverse response in a human or other organism or ecosystem.@ It may be that a 

stressor does not cause harm directly, but only makes a human, organism, or ecosystem more 

susceptible to harm by other stressors. An example would be poor nutrition causing a decrease in 

immune function making one more susceptible to infection. 

Page 6, Definitions. A definition of assessment endpoint needs to be added. The point needs to 

be made that a cumulative risk assessment needs a well-defined endpoint. Overall impact on 

human health is not such an endpoint. A collection of assessment endpoints might be used to 

F-96 
measure various components of Aoverall human health@. An example is AQuality-of-life 

Assessments (Appendix F)@ Here a set of Aquality-of-life@ criteria (assessment endpoints) is used 

to describe various aspects of quality of life, but no assessment is made of overall quality of life. 

Even in this example, the set of Aquality-of-life@ criteria are not well-defined assessment 

endpoints. For example, one is economic well-being and one is fairness. These endpoints need 

measurable definitions or they cannot be assessed. 

Confusion over this issue permeates the document and needs to be straightened out in the 

definition section. 

Page 8, Line 33. The document is clearer with regard to the definition of stressors when 

discussing the risk of breast cancer in women. The document states, AThe Astressors@ in the 

example of the breast cancer model are certain factors known to be correlated with that form of 

cancer, such as the woman=s age at first childbirth, age at menarche, having a previous biopsy 

with atypical hyperplasia, and others.@

Page 9, Table. The table says that cumulative risk is a population-based process. What does this 

mean? How is it different from a regular risk assessment? In any risk assessment, one must 

identify a population being assessed, whether it is a farm family or and an industrial worker. 

How is cumulative risk assessment different? 

Page 11. Line 14. The text says, ACumulative risk assessment, being a population-based or placebased analytic-deliberative process, Y@ The text has not really defined what a population-based 

or place-based process is. 

Page 37, line 12. The text says, Acomponent based mixture assessments are rarely evaluated 

using the strict NRC paradigm, because the exposure and toxicity information must be 

compatible, requiring some iteration to obtain toxicity information that is relevant to the actual 

exposure estimates.@ This is a good point that needs to be made more prominent in the text. You 

cannot separate the toxicity assessment phase and the exposure assessment phase of a cumulative 

risk assessment. 

Page 38, line 40. The text says, AWhen all the single stressor risks are low, the joint risk 

of a common effect under response addition can be approximated by the simple sum of the single 

stressor risks. For example, if reproductive toxicity is the general effect common to the multiple 

chemicals, the cumulative risk of reproductive effects (at low single chemical risk levels) is 

approximately the sum of the single chemical reproductive risks.@ This appears in the section on 

toxicologic independence. Does this only hold for toxicologic independence or for all toxicity?

Page 41, line 29.This section talks about traditionally, the exposure and dose-response steps of a 

risk assessment are separated. It proposes that in a cumulative risk assessment this may not be 

appropriate. It is not clear why this is not appropriate. Nor is it cear how one would combine the 

steps. What point are you trying to make? That different exposure situations require different 

dose response functions? That there is uncertainty in the dose response function, so you need a 

range of dose response functions?

F-97 
Page 41, line 45. The text says, AIf the dose response data do not represent the same conditions as 

the exposure being assessed, Y@ Here is seems like you saying that this is something unique to 

cumulative risk assessment. The same problem arises in regular risk assessment. One often has a 

dose-response for ingestion, but needs one for inhalation. There are methods for making the 

adjustment. However, I do not see how this is related to the issue of separating or combining the 

exposure and dose-response steps in a risk assessment. The document needs to be more explicit 

about what issue it is trying to make. 

Page 42, line 5. The text says, Atoxicologic interactions have been shown to change using the 

same doses but with a reversal of the sequence of exposure (i.e., chemical B then A instead of A 

then B), so that the exposure and dose-response steps must be compatible and performed 

together.@ What does the text mean by Amust be performed together@? It is apparent that one must 

use the correct dose-response function for the situation, but it still looks like the operations 

(exposure and dose-response) are performed in sequence, not together. I am not even sure what it 

would mean to perform exposure and dose-response Aall together@. Again, the text needs to be 

more explicit about what point it is trying to make. 

Page 43, line 5.The text says, AThe issues for these cases are now presented along with their 

main research implications, starting with the simplest case where only chemical interactions are 

considered.@ I do not see where these cases are presented. There are only two paragraphs 

following this sentence. 

Page 43, line 8. This section is a good addition. 

Page 43, line 20. This paragraph is a little confusing, but still contains good information. It 

would help if the paragraph gave some indication of what should be done about the concepts of 

synergism and antagonism for cumulative exposure. Are they to be abandoned completely when 

one has multiple chemical exposures or is there a way to define these concepts that makes sense? 

It would seem that using the Adose additive@ approach to defining no-interaction is the way to 

go? If a mixture of chemicals causes a greater effect that the sum of the effects produced by each 

chemical alone, the situation is called synergistic. What is wrong with this definition in the case 

of cumulative exposure? The document needs to be clearer about this issue. 

Page 44, line 16. The text says, AThe main disadvantage of a simple index is that the 

uncertainties in its calculation are largely hidden.@ I=m not sure this is the main disadvantage. 

Why is this any different than any other risk assessment number? When one presents an estimate 

of cancer as 1.2 E-4, the uncertainties in this calculation are also largely hidden! What is the 

difference? 

Page 44, box. It would seem that a Hazard Index like that presented in the box would defeat the 

purpose of a cumulative risk assessment. This Hazard Index is based on the assumption of 

additively of effects, while the purpose of a cumulative risk assessment is to understand the 

possible impact (additive, non-additive) of multiple exposures. 

F-98 
Page 44, line 25. The text states, AOne alternative for addressing multiple effects is to recast 

these qualitative judgments in terms of severity categories or levels of concern@ It then states, A

The result is not a risk of a particular toxic effect, but rather a risk of exceeding a certain 

minimum toxic severity level, or level of minimal concern.@ This is just the weakness in the 

Hazard Index that you were trying to avoid. 

Page 45, line 14. The text needs to provide a little detail on the approach taken by Bogen. If you 

are citing it as an example, you need to tell why it was good. What did he do that added to the 

field of cumulative risk assessment? 

Page 45, line 21. This paragraph is not clear. The document states, AAny approach to cumulative 

risk assessment needs to carefully define the set of relevant endpoints.@ The document has said 

earlier that a specific endpoint must be defined. But why does it have to be carefully defined? 

What are the problems that may arise from non-careful endpoint definition? The document 

states, Athe risk of inducing a given endpoint may differ among different people in a population 

at risk for some endpoints.@ This is true of any risk assessment. Why does this fact introduce that 

need for careful definition of endpoint? And even with a careful definition, one will always still 

have this problem. What is the point that the document is trying to make here? 

Page 45, line 27. The text says, ADefining the latter risks in terms of individual risk per se will 

thus complicate calculating cumulative risk if a probabilistic approach to cumulative risk 

assessment is used.@ It is not clear why this is true. Even in a probabilistic risk assessment, risk is 

defined on an individual basis. The probabilistic risk assessment gives the probability 

distribution of individual risk. Again, the document needs to be clear on the point it is making. 

Page 45, line 31. The text says, AIn contrast, the probabilistic approach to cumulative risk 

assessment may be facilitated by defining the risk of a given endpoint in terms of population 

risk, i.e., in terms of the predicted number of cases of that endpoint.@ It is not clear how using 

population risk instead of individual risk makes probabilistic risk assessment easier. To obtain 

population risk, one still has to determine how individuals will respond and then take account of 

the number of individuals involved. It seems like population risk and individual risk are just two 

different measures of impact. The risk assessment process to arrive at an estimate of impact is 

basically the same for both. 

Page 45, line 33. The text says, AAlternatively (or additionally), similar simplification can be 

achieved for all heterogeneous endpoints by defining the risk of that endpoint only with respect 

to those persons in the population at risk who are reasonably maximally exposed (e.g., 

individuals adjacent to a proposed source).@ It is true that a risk assessment may want to focus on 

maximally exposed individuals, but I don=t see what this has to do with making probabilistic risk 

assessments better. I think this sentence belongs above in the section defining population-based 

risk assessment, no in the section on probabilistic risk assessment. 

Page 48, Section 3.3.3. In this section the document discusses the issue of developing a single 

index of risk when there are multiple impact endpoints. The document is not sufficiently clear 

that creation of a single index of risk is more of a risk management decision than a risk 

F-99 
assessment decision. Unless the index of risk is a well-defined assessment endpoint (than can 

theoretically be measured), it is just a risk management tool to aid in decision making and risk 

communication. If a cumulative assessment is to use an index of risk, it should state during the 

scoping phase that the assessment end point will be a defined function (the index of risk) of 

multiple assessment endpoints. This is obviously a risk management decision. Page 49, line 1. 

The text says, AIdeally, this evaluation would provide projections regarding the potential for a 

particular complex exposure to cause particular effects to different physiological systems, and 

also provide an integration of these projections into a qualitative characterization of overall 

potential impact to human health.@ This implies that Aoverall impact on human health@ is a welldefined endpoint for cumulative risk assessment. It is not and this needs to be made clear in the 

document. 

Page 49, line 9. This assessment had well defined Aobserved adverse conditions@ and preformed a 

cumulative assessment for the impacts on these conditions. It did not attempt to combine 

individual impacts into an index of overall impact. 

Page 49, line 33. The document says some cumulative risk assessments Arequire synthesizing a 

risk estimate (or risk indication) by Aadding up@ risks for different parts of the risk picture.@ I 

believe this sentence is misleading. Cumulative risk assessment does not require this. 

Cumulative risk assessment can only be performed on discrete well-defined endpoints. One may 

want to present these in matrix form, but combining them into an assessment of overall impact is 

not scientifically possible (unless that overall impact is defined by some single measurable 

quantity). 

Page 54, line 11. The text says, ACombinations of many types of stressors with different 

endpoints in a single assessment will quickly cause the risk estimation step to become very 

complex and difficult.@ Again this implies that it is possible (and the proper function of a 

cumulative risk assessment) to combine endpoints into a single measure of impact. 

Page 54, section 4.1. This section is more about risk assessment in general than about cumulative 

risk assessment. 

Page 61, line 12. The document says, AThe Framework report devotes considerable time to a 

discussion of improving the methods for a single part of the broader picture -- characterizing 

health risks associated with exposures to multiple chemicals via multiple routes.@ This statement 

is not true. The document has discussed that lack of a methodology is a problem, but it has not 

done much to discussing how to improve current methods. 

Page 62, section 4.4. This section is more about risk assessment in general than about cumulative 

risk assessment 

F-100 
Appendix G

Draft Framework for Cumulative Risk Assessment

G-1
EXTERNAL REVIEW DRAFT EPA/630/P-02/001A 

April 23, 2002 

Framework for Cumulative Risk Assessment
 

Risk Assessment Forum
 

U.S. Environmental Protection Agency
 

Washington, DC 20460
 
DISCLAIMER 

This document is in the process of being reviewed in accordance with U.S. 

Environmental Protection Agency’s peer and administrative review policy. It has had extensive 

peer involvement in development (including other Federal, State, and public involvement), but it 

has not yet completed the formal peer review process or administrative review process, so it 

should not be quoted or cited. The mention of commercial products is for illustration only and in 

no way implies EPA endorsement of these products. 

ii 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Table of Contents 

List of Figures ............................................................... vi
 

List of Text Boxes ........................................................... vii
 

Acknowledgments ............................................................ ix
 

Foreword ................................................................... x
 

Preface ..................................................................... xi
 

Authors, Contributors, and Reviewers ............................................ xiii
 

List of Abbreviations and Acronyms ............................................. xvi
 

Executive Summary ......................................................... xviii
 

1. INTRODUCTION	 ........................................................... 1
 

1.1. Purpose and Scope of the Framework Report .............................. 5
 

1.2. Intended Audience ................................................... 6
 

1.3. Key Definitions in Cumulative Risk Assessment ........................... 6
 

1.4. The Cumulative Risk Assessment as a Tool for a Variety of Users and Purposes
 

............................................................... 10
 

1.5. The Broader Decision-Making Context for Cumulative Risk Assessment ....... 11
 

1.6. Organization of this report ........................................... 13
 

2. THE PLANNING, SCOPING, AND PROBLEM FORMULATION PHASE ........... 14
 

2.1. 	Planning and Scoping ............................................... 15
 

2.1.1. Defining the Purpose of the Assessment .......................... 16
 

2.1.2. Defining the Scope of Analysis and Products Needed ............... 17
 

2.1.3. Agreeing on participants, roles and responsibilities ................. 19
 

2.1.4. Agreeing on the Depth of the Assessment and the Analytical Approach
 

......................................................... 21
 

2.1.5. Agreement on the Resources Available and Schedule ............... 21
 

2.1.6. Review of Lessons Learned in Similar Studies ..................... 22
 

2.2. Problem Formulation, Conceptual Model, and Analysis Plan ................. 22
 

2.2.1. Problem Formulation ........................................ 23
 

2.2.2. Developing the Conceptual Model .............................. 23
 

2.2.3. Constructing the Analysis Plan ................................. 26
 

2.3. The Final Step Before the Analysis Phase: Discussion of Possible Outcomes .... 27
 

iii 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

3. 	THE ANALYSIS PHASE ................................................... 30
 

3.1. General Process .................................................... 31
 

3.2. Available Methods and Approaches .................................... 32
 

3.2.1. Examples of Increased Complexity of Cumulative Risk Assessment ... 32
 

3.2.2. Approaches for Predicting Risk of Multiple Stressors ............... 36
 

3.2.2.1. Single Stressor Information ............................ 37
 

3.2.2.2. Information on Stressor Interactions and Multiple Exposures
 

................................................... 40
 

3.2.2.3. Decision Indices ..................................... 42
 

3.2.2.4. Probabilistic Approaches .............................. 44
 

3.3. Areas of Complexity and Current Research ............................... 44
 

3.3.1. Interactions Between Stressors and Other Factors .................. 45
 

3.3.2. The Promise of Biomarkers and Biomonitoring .................... 46
 

3.3.3. A Single Metric for Multiple Types of Hazard ..................... 47
 

3.3.3.1. Creating a Common Metric ............................ 48
 

3.3.3.2. General Issues with a Single Metric ...................... 51
 

3.3.4. Qualitative approaches ....................................... 51
 

4. 	THE RISK CHARACTERIZATION PHASE .................................... 53
 

4.1. Risk Description .................................................... 54
 

4.2. Uncertainty Analysis ................................................ 56
 

4.2.1. Assumptions in the Assessment ................................ 57
 

4.2.2. Uncertainty and Variability .................................... 58
 

4.2.3. Uncertainty and Risk Addition ................................. 58
 

4.3. The Information Provided by Cumulative Risk Assessment .................. 59
 

4.3.1. Making Sense of Multiple Stressor Effects........................ 59
 

4.3.2. Cumulative Risk Assessments in a Public Health Context ............ 60
 

4.3.3. How Scope and Purpose of the Assessment Affect Results ........... 61
 

4.4. Using the Results of the Assessment .................................... 62
 

5. GLOSSARY .............................................................. 64
 

6. REFERENCES ............................................................ 66
 

APPENDIX A: RESEARCH AND DEVELOPMENT NEEDS ......................... 82
 

APPENDIX B: SELECTED RESOURCES FOR EXPOSURE AND RISK ASSESSMENT . 85
 

B.1. Resources Relevant to Chemical Exposures .............................. 85
 

B.2. Resources Relevant to Exposures to Non-Chemical Stressors ............... 86
 

APPENDIX C: SOME THOUGHTS ON BACKGROUND EXPOSURES ................ 89
 

iv 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX D: EXAMPLES OF ANALYSIS PLANS ................................ 91
 

D.1. Human Health Analysis Plan for Pesticides under FQPA ................... 91
 

D.2. Ecological Analysis Plan ............................................ 91
 

APPENDIX E: TOXICOLOGIC SIMILARITY: ORGANOPHOSPHORUS PESTICIDES ... 93
 

APPENDIX F: OTHER TYPES OF CUMULATIVE ASSESSMENTS .................. 96
 

F.1. Quality-of-Life Assessments ......................................... 96
 

F.2. Cumulative Impact Assessments ...................................... 97
 

F.3. Empirically-Derived Medical Models .................................. 98
 

F.4. Risk Surrogates .................................................... 99
 

v 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

List of Figures 

1-1. Chemical or stressor focused assessment ..................................... 1
 

1-2. Population-based assessments .............................................. 2
 

1-3. Framework for cumulative risk assessment ................................... 13
 

2-1. The planning, scoping, and problem formulation phase ......................... 14
 

2-2. Generic conceptual model ................................................ 24
 

2-3. Specific conceptual model for a complex project, NATA ........................ 25
 

3-1. The analysis phase ...................................................... 30
 

4-1. The interpretation phase .................................................. 52
 

vi 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

List of Text Boxes 

Some Examples of Cumulative Risk Assessment Activities in EPA in Late 2001 .......... xx
 

EPA’s Risk Assessment Guidelines ................................................ 5
 

FQPA’s Terminology Interpretations ............................................... 8
 

Cumulative Risk Assessment Features ............................................. 9
 

The Core Principles of Community-Based Environmental Protection .................... 11
 

Possible Management Goals .................................................... 17
 

Examples of Different Cumulative Risk Assessment Scopes ........................... 18
 

Examples of Possible Interested or Affected Parties (Stakeholders) ...................... 19
 

Guidelines for Stakeholder Involvement ........................................... 20
 

Reed Holderman’s Lessons Learned .............................................. 22
 

Example: SAB’s Desired Outputs for Problem Formulation ........................... 23
 

Example: Cumulative Risk Initiative (CRI) for Cook Co., IL, and Lake Co., IN ............ 29
 

Example: Scope of EPA’s National-scale Assessment for Hazardous Air Pollutants ......... 31
 

Some Examples of Models which Consider Time Aspects ............................. 34
 

Example Using Toxicologic Independence: National-scale Air Toxics Assessment ......... 40
 

Example Using Toxicologic Similarity: The Dioxin Reassessment ...................... 42
 

Example Decision Index: The Hazard Index ........................................ 44
 

Risk Characterization Guiding Principles .......................................... 54
 

Some Thoughts on Risk Characterization .......................................... 56
 

Morgan and Henrion’s “Ten Commandments” for Good Policy Analysis ................. 57
 

vii 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Uncertainty Analysis Example: The Cumulative Exposure Project ...................... 59
 

EEA’s Twelve Late Lessons Learned ............................................. 63
 

Vermont’s Quality of Life Criteria ............................................... 96
 

NEPA’s “Cumulative Impact” Definition .......................................... 97
 

viii 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Acknowledgments 

This U.S. Environmental Protection Agency (EPA) report has been developed under the 

auspices of EPA’s Risk Assessment Forum, a standing committee of EPA scientists charged with 

developing risk assessment guidance for Agency-wide use. An interoffice technical panel 

chaired by Michael Callahan (Region 6) was commissioned to write this report. Other members 

of the panel are Edward S. Bender (Office of Science Policy), George L. Bollweg (Region 5), 

Vicki L. Dellarco (Office of Pesticides Programs), Lynn A. Delpire (Office of Pollution 

Prevention and Toxics), Martin P. Halper (Office of Environmental Justice), Richard C. 

Hertzberg (National Center for Environmental Assessment), Elizabeth Lee Hofmann (Office of 

Emergency and Remedial Response), R. Craig Matthiessen (Chemical Emergency Preparedness 

and Prevention Staff), Alexander McBride (Office of Solid Waste), Deirdre L. Murphy (Office of 

Air Quality Planning and Standards,), Henry C. Topper (Office of Pollution Prevention and 

Toxics), and Winona Victery (Region 9). In addition, Carole Braverman (Region 5), Loren Hall 

(Office of Civil Rights), and Denis R. Borum (Office of Water) participated for part of the 

duration of the panel. Steven M. Knott of the Risk Assessment Forum staff coordinated the 

project as well as participated as a member of the panel. The resulting document included peer 

involvement and peer review by scientists from EPA, other federal agencies, state agencies, 

academia, public interest groups, and the private sector. 

ix 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Foreword 

Several reports have highlighted the importance of understanding the accumulation of 

risks from multiple environmental stressors. These include the National Research Council’s 

1994 report Science and Judgment in Risk Assessment and the 1997 report by the 

Presidential/Congressional Commission on Risk Assessment and Risk Management entitled Risk 

Assessment and Risk Management in Regulatory Decision-Making. In addition, legislation such 

as the Food Quality Protection Act of 1996, has directed the Environmental Protection Agency to 

move beyond single chemical assessments and to focus, in part, on the cumulative effects of 

chemical exposures occurring simultaneously. Further emphasizing the need for EPA to develop 

methods to assist consideration of cumulative risks are some of the cases filed with EPA under 

Title VI of the 1964 Civil Rights Act. 

The Superfund program began doing cumulative risk assessments at hazardous waste 

sites as early as the 1980s. More recently, in response to the increasing interest in cumulative 

risk, several other EPA programs have begun to explore approaches to cumulative risk 

assessment. In 1997, The EPA Science Policy Council issued a guidance on planning and 

scoping for cumulative risk assessments. More recently, the Office of Pesticide Programs has 

developed cumulative risk assessment guidance focused on implementing certain provisions of 

FQPA. In addition, the Office of Air Quality Planning and Standards is performing a nationalscale cumulative assessment of human health risks posed by outdoor air exposures to a set of 33 

priority urban air toxics. 

The EPA Science Policy Council has asked the Risk Assessment Forum to begin 

developing Agency-wide cumulative risk assessment guidance that builds from these ongoing 

activities. As a first step, a technical panel convened under the Risk Assessment Forum has been 

working to develop a Framework for Cumulative Risk Assessment. This document is the result 

of that technical panel’s efforts. Building from the Agency’s growing experiences, this 

Framework is intended to identify the basic elements of the cumulative risk assessment process. 

It should provide a flexible structure for the technical issues and define key terms associated with 

cumulative risk assessment. Further efforts and experience in the coming years should advance 

our knowledge beyond the Framework stage to a future set of Agency guidelines for cumulative 

risk assessment. 

William P. Wood, Ph.D.
 

Executive Director
 

Risk Assessment Forum
 

x 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Preface 

In the past several years, cumulative risk assessment, aggregate exposure assessment, and 

research on chemical mixtures has taken on increased importance. This is underscored by reports 

such as the National Research Council’s 1993 report Pesticides in the Diets of Infants and 

Children, (NRC, 1993) the 1994 NRC report Science and Judgment in Risk Assessment, (NRC, 

1994), the 1995 National Academy of Public Administration report Setting Priorities, Getting 

Results (NAPA, 1995), the 1997 report by the Presidential/Congressional Commission on Risk 

Assessment and Risk Management titled Risk Assessment and Risk Management in Regulatory 

Decision-Making (PCCRARM, 1997), and the EPA Science Advisory Board report Toward 

Integrated Environmental Decision-Making (USEPA, 2000a). There also have been several 

recent pieces of legislation that mandate the consideration of cumulative risk and variability 

factors in the risk characterization process. Specifically, the Food Quality Protection Act of 1996 

(FQPA) [PL 104-170, August 3, 1996] directs EPA in its assessments of pesticide safety to 

focus, in part, on the cumulative effects of pesticides that have a common mechanism of toxicity, 

considering aggregate dietary and non-occupational pathways of exposure. 

Assessment of cumulative risk through complex exposures is one of the high priorities of 

the Agency, especially in light of FQPA mandates, and is germane and of great interest to all 

program and regional offices. This area of research is also directly applicable to children’s risk 

issues. This Framework is meant to lay out broad areas where analysis might be conducted if 

needed. It does not suggest that cumulative risk assessment is a tool that should be used with 

every issue, nor does it suggest that when cumulative risk assessment is applied, that all areas of 

analysis outlined or discussed here must or even should be conducted in every assessment. The 

scope of the assessment will define the areas to be analyzed. In some areas discussed in this 

Framework, the methodology for doing the risk analysis may not yet exist. 

According to the expert panel report Safeguarding the Future: Credible Science, Credible 

Decisions (USEPA 1992a), a key role of science at EPA is to reduce uncertainties in 

environmental decision-making. The report points out that while many EPA programs have 

historically focused on chemical-specific impacts, methods to assess or control the effects of 

chemical mixtures and general stressors on human health and ecosystems remained to be 

developed. In Pesticides in the Diets of Infants and Children, (NRC, 1993) the NRC 

recommended that all exposures to pesticides – dietary and nondietary – need to be considered 

when evaluating the potential risks to infants and children. Estimates of total dietary exposure 

should be refined to consider intake of multiple pesticides with a common toxic effect. Further, 

the report identifies important differences in susceptibility with age. NRC in Science and 

Judgment in Risk Assessment (NRC, 1994) states that health risk assessments should generally 

consider all possible routes by which people at risk might be exposed, and recommends this 

approach universally in the assessment of hazardous air pollutants regulated by EPA under the 

Clean Air Act Amendments of 1990 [P.L. 101-549, November 15, 1990]. Regarding variability, 

the NRC Science and Judgment report recommended that EPA assess risks to infants and 

xi 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

children whenever it appears that their risks might be greater than those of adults. Public 

criticisms cited in this report include statements made by some experts that EPA does not appear 

to recognize the possibility of synergistic interactions when multiple chemical exposures occur, 

nor does it consider extreme variability among individuals in their responses to toxic substances. 

A related issue is the problem of how risks associated with multiple chemicals are to be 

combined. Finally, the FQPA [P.L.104-170, August 3, 1996], requires research on the influence 

of complex exposures on non-cancer human health effects of pesticides and other toxic 

substances. 

The issue of cumulative risk is also an important issue with the general public. In public 

meetings of Superfund stakeholders, held in late 1996 in San Francisco and Washington, DC, 

and in early 1998 in Atlanta, the issue of cumulative risk was raised several times in each session 

(USEPA 1996a, USEPA 1998a). 

Cumulative risk assessments will identify the need for many different kinds of data – 

some of them are not the data commonly used now for risk assessment – and often, cumulative 

risk assessment will demand large quantities of such data. Until data bases and data generation 

research can provide such data, for the near term, identification of critical data and research 

needs may be the primary result of many cumulative risk assessment endeavors. 

As of August 1, 2001, there were 19,533 pesticide products on the market (USEPA, 

2001a), and 79,120 existing chemicals on the TSCA inventory (USEPA, 2001b). Each year, an 

additional number of chemicals are added. Assessing the cumulative effect of these chemicals 

will be a great challenge to the Agency and may become the primary issue in the risk assessment 

field in the next ten years. 

xii 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Authors, Contributors, and Reviewers 

Authors: 

EPA Risk Assessment Forum Cumulative Risk Technical Panel: 

Edward S. Bender, Office of Science Policy, ORD
 

George L. Bollweg, Air and Radiation Division, Region 5
 

Michael A. Callahan, Office of the Regional Administrator, Region 6 (Chair)
 

Vicki L. Dellarco, Office of Pesticides Programs, OPPTS
 

Lynn A. Delpire, Office of Pollution Prevention and Toxics, OPPTS
 

Martin P. Halper, Office of Environmental Justice, OECA
 

Richard C. Hertzberg, National Center for Environmental Assessment, ORD
 

Elizabeth Lee Hofmann, Office of Emergency and Remedial Response, OSWER
 

Steven M. Knott, Risk Assessment Forum, ORD
 

R. Craig Matthiessen, Chemical Emergency Preparedness and Prevention Staff, OSWER
 

Alexander McBride, Office of Solid Waste, OSWER
 

Deirdre L. Murphy, Office of Air Quality Planning and Standards, OAR
 

Henry C. Topper, Office of Pollution Prevention and Toxics, OPPTS
 

Winona Victery, Policy and Management Division, Region 9
 

Contributo rs and Rev iewers: 

EPA R eviewers: 

Donald G. Barnes, EPA Science Advisory Board
 

Michael Firestone, Office of Children’s Health Protection
 

Debra L. Forman, Region 3
 

Mark Madd aloni, Region 2
 

Elizabeth H. M argosches, Office of Po llution Prevention and T oxics 
 

Marian Olsen, Region 2
 

James Rowe, Office of Research and Development
 

Scott Schwenk, Office of Research and Development
 

Michae l Shapiro, O ffice of Solid W aste
 

Sally Shaver, Office of Air and Radiation
 

Margaret Stasikowski, Office of Pesticide Programs
 

Jeannette Wiltse, Office of Water
 

Jeffrey Yurk, Region 6
 

The following persons participated in peer involvement workshops held during 2001 to discuss and comment on the
 

draft Framework.
 

Federal Scientists’ Meeting, May 1, 2001: 

Michael A. Babich, Health Sciences Division, U.S. Consumer Product Safety Commission
 

Michael Buchman, Office of Response and Restoration, National Oceanic and Atmospheric Administration
 

Stephen Channe l, Department of the Air Fo rce, U.S. Depa rtment of Defense
 

Christopher T. DeRosa, Division of Toxicology, Agency for Toxic Substances and Disease Registry
 

Dennis Druck, A rmy Center for Health P romotion and P reventive Medicine , U.S. Departme nt of Defense
 

Frank H earl, Division of Respira tory Disease Studies, Na tional Institute for O ccupation al Safety and H ealth
 

xiii 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Keith Hodd inott, Army Center for Hea lth Promotion and P reventive Medicine , U.S. Departme nt of Defense
 

Steve Jones, Office of Regional Operations, Agency for Toxic Substances and Disease Registry
 

Brian Kaplan, Division of Health Assessment and Consultation, Agency for Toxic Substances and Disease Registry
 

Ralph Kodell, National Center for Toxicological Research, U.S. Food and Drug Administration
 

Dan Lew is, Health Effec ts Laborato ry Division, N ational Institute for Occup ational Safety a nd Hea lth
 

Margaret MacD onell, Argonne National Laboratory, U.S. Department of Energy
 

Mark McClanahan, National Center for Environmental Health, Centers for Disease Control and Prevention
 

Anita K. Meyer, H TRW Center of Expertise , U.S. Army Corps of Engineers
 

Moiz G. Mum taz, Division of Toxicology, Agency for Toxic Substances and Disease Registry
 

Leslie G. Rubin, Animal and Plant Health Inspection Service, U.S. Department of Agriculture
 

Antoinette Sebastian, Office of Community Planning and Development, Dept. of Housing and Urban Development
 

Scott D. Sudweeks, Div. of Health Assessment and Consultation, Agency for Toxic Substances and Disease Registry
 

Timothy Taylo r, Army Center for He alth Promotion and Preventive Me dicine, U.S. Depa rtment of Defense
 

John Wheeler, Division of Toxicology, Agency for Toxic Substances and Disease Registry
 

Jody Wirem an, Department o f the Air Force, U.S. De partment of Defense
 

States’ Meeting, May 3, 2001: 

Glen Andersen, National Conference of State Legislators
 

Ligia Mora-Applegate, Florida Department of Environmental Protection
 

Alex M . Barron, V irginia Dep artment of E nvironme ntal Quality
 

Sharon B axter, Office o f Pollution P revention, V irginia Dep artment of E nvironme ntal Quality
 

Joseph P. Brown, Office of Environmental Health Hazard Assessment, California Environmental Protection Agency
 

Michele F. Burgess, Georgia Department of Natural Resources
 

Perry Co hn, Enviro nmental H ealth Service , New Jers ey Depa rtment of H ealth
 

Linda J. Cullen, New Jersey Department of Environmental Protection
 

Susan Dempsey, Nebraska Department of Health and Human Services
 

Michael J. DiBartolomeis, Office of Environmental Health Hazard Assessment, California Env. Protection Agency
 

Michae l Ellenbeck er, Toxic s Use Re duction Institute , University of M assachusetts a t Lowell
 

George Hagevik, National Conference of State Legislators
 

Joann Held, Office of Air Quality, New Jersey Department of Environmental Protection
 

Bruce H ope, Or egon De partment o f Environm ental Quality
 

Tom Hornshaw, Toxicity Assessment Unit, Illinois Environmental Protection Agency
 

Paul B. Jiapizian, Maryland Department of the Environment
 

David R . Johnson, B ureau of E nvironme ntal Epide miology, Flo rida Dep artment of H ealth
 

Michael Juras, Community Right-to-Know, South Carolina Department of Health and Environmental Control
 

Kevin Kessler, Air and Waste Division, Wisconsin Department of Natural Resources
 

Rosanna Kroll, Health and Toxicology Section, Maryland Department of Environment
 

Charles Moore, Office of Environmental Management, South Carolina Department of Natural Resources
 

Rebecca Pottratz, Communication Specialist, Wisconsin Department of Natural Resources
 

John Ridgway, Environmental Justice Program, Washington State Department of Ecology
 

Kirpal S. S idhu, Enviro nmental Ep idemiolog y Division, M ichigan De partment o f Commu nity Health
 

Lawrence P. Sirinek, Division of Emergency and Remedial Response, Ohio Environmental Protection Agency
 

Alan Stern, Bureau for Risk Analysis, New Jersey Department of Environmental Protection
 

Chuck Str oebel, En vironmen tal Researc h Scientist, M innesota D epartmen t of Health
 

Barba ra S. Taylo r, Departm ent of Hea lth and Hu man Serv ices, We st Virginia B ureau for P ublic Hea lth
 

Mark W erner, W isconsin Div ision of Pub lic Health
 

Carol Rowan West, Office of Research and Standards, Massachusetts Department of Environmental Protection
 

Victor R . Wilford , Departm ent of Hea lth and Hu man Serv ices, We st Virginia B ureau for P ublic Hea lth
 

Stephen L. Wust, Drinking Water Bureau, New M exico Environment Department
 

xiv 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Public Meeting, August 22-23, 2001: 

Elizabeth Boa, American Chemistry Council, Inc.
 

Kenneth T. Bogen, Health and Ecological Assessment Division, Lawrence Livermore National Laboratory
 

James P. Butler, Environmental Assessment Division, Argonne National Laboratory
 

Douglas J. Crawford-Brown, Dept of Environmental Sciences and Engineering, University of North Carolina
 

Amy D. Kyle, School of Public Health, University of California-Berkeley
 

Norris McDonald, Center for Environment, Comm erce and Energy
 

Beth E. Mileson, Technology Sciences Group, Inc.
 

Stephen S . Olin, Risk Sc ience Institute, Inte rnational Life S ciences Institute
 

William R. Rhyne, H&R Technical Associates, Inc.
 

Jennifer Sas s, Health P rogram, N atural Reso urces De fense Cou ncil
 

Margo Schwab, S chool of P ublic Hea lth, Johns H opkins U niversity
 

Ken W. Sexton, Center for Environment and Health Policy, University of Minnesota (Chair)
 

Written Public Co mments: 

James M. Solyst, American Chemistry Council, Inc. 

xv 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

List of Abbreviations and Acronyms 

ACGIH - American Conference of Government Industrial Hygienists
 

AFS - AIRS Facility Subsystem
 

AIChE - American Institute of Chemical Engineers
 

AIHA - American Industrial Hygiene Association
 

AIRS - Aerometric Information Retrieval System
 

AMTIC - Ambient Monitoring Technology Information Center
 

APCA - American Crop Protection Association
 

APEX - Air Pollution EXposure model
 

ARE - Acute Reference Exposure
 

ATSDR - Agency for Toxic Substances and Disease Registry
 

CARES - Cumulative and Aggregate Risk Evaluation System
 

CBEP - Community-Based Environmental Protection
 

CEQ - Council for Environmental Quality
 

CFR - Code of Federal Regulations
 

CHIEF - Clearinghouse for Inventories and Emissions Factors
 

COHb - Carboxyhemoglobin
 

CRIA - Cumulative Risk Index Analysis
 

DALY - Disability-Adjusted Life Year
 

DOT - United States Department of Transportation
 

EPA - United States Environmental Protection Agency
 

FIFRA - Federal Insecticide, Fungicide, and Rodenticide Act
 

FQPA - Food Quality Protection Act
 

GAO - United States General Accounting Office
 

GIS - Geographical Information System
 

HAP - Hazardous Air Pollutant
 

HEC - Human Equivalent Concentration
 

HRS - Hazard Ranking System
 

HUD - United States Department of Housing and Urban Development
 

IED - Integrated Environmental Decision-making
 

ILSI - International Life Sciences Institute
 

LADD - Lifetime Average Daily Dose
 

LDP - Locational Data Policy
 

LLE - Loss of Life Expectancy
 

LOAEL - Lowest Observed Adverse Effect Level
 

MOE - Margin of Exposure
 

MSDS - Materials Safety Data Sheet
 

NAAQS - National Ambient Air Quality Standards
 

NAPA - National Academy of Public Administration
 

NATA - National Air Toxics Assessment
 

NEPA - National Environmental Policy Act
 

xvi 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

List of Abbreviations and Acronyms (Continued) 

NHEXAS - National Human Exposure Assessment Survey
 

NIOSH - National Institute for Occupational Safety and Health
 

NOAEL - No Observed Adverse Effect Level
 

NRC - National Research Council
 

OAR - Office of Air and Radiation (EPA)
 

OECA - Office of Enforcement and Compliance Assurance (EPA)
 

OP - Organophosphorous
 

OPP - Office of Pesticide Programs (EPA)
 

OPPTS - Office of Prevention, Pesticides, and Toxic Substances (EPA)
 

ORD - Office of Research and Development (EPA)
 

OSWER - Office of Solid Waste and Emergency Response (EPA)
 

P.L. - Public Law
 

PAH - Polycyclic Aromatic Hydrocarbon
 

PCB - Polychlorinated Biphenyl
 

PCS - Permit Compliance System
 

PM-10 - Particulate Matter with diameter of 10 micrometers or less
 

pNEM - Probabilistic NAAQS Exposure Model
 

QALY - Quality-Adjusted Life Year
 

RfC - Reference Concentration
 

RfD - Reference Dose
 

SAB - Science Advisory Board
 

SAP - Scientific Advisory Panel
 

SAR - Structure-Activity Relationship
 

SCRAM - Support Center for Regulatory Air Models
 

SHEDS - Stochastic Human Exposure and Dose Simulation model
 

SPC - Science Policy Council 
 

TEAM - Total Exposure Assessment Methodology
 

TEMRAP - The European Multi-Hazard Risk Assessment Project
 

TIA - Transient Ischemic Attack
 

TRI - Toxic(s) Release Inventory
 

TRIM.Expo - Total Risk Integrated Methodology, Exposure Module
 

U.S.C. - United States Code
 

UF - Uncertainty Factor
 

USEPA - United States Environmental Protection Agency
 

xvii 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Executive Summary 

This report, “Framework for Cumulative Risk Assessment,” is the first step in a longterm effort to develop cumulative risk assessment guidelines. Its primary purpose is to offer a 

simple, flexible structure for conducting and evaluating cumulative risk assessment within EPA. 

Although this Framework report will serve as a foundation for development of future guidelines, 

it is neither a procedural guide nor a regulatory requirement within EPA and is expected to 

evolve with experience. This Framework report is intended to foster consistent approaches to 

cumulative risk assessment within EPA, identify key issues, and define terms used in these 

assessments. 

This Framework is meant to lay out broad areas where analysis might be conducted if 

needed. It does not suggest that cumulative risk assessment is a tool that should be used with 

every issue, nor does it suggest that when cumulative risk assessment is applied, that all areas of 

analysis outlined or discussed here must or even should be conducted in every assessment. The 

scope of the assessment will define the areas to be analyzed. In some areas discussed in this 

Framework, the methodology for doing the risk analysis may not yet exist. Appendix A includes 

a summary of areas where research is needed. 

In this report, “cumulative risk” means “the combined risks from aggregate exposures to 

multiple agents or stressors.” There are several key points which come from this definition of 

cumulative risk. First, cumulative risk involves multiple agents or stressors, which means that 

assessments involving a single chemical or stressor are not “cumulative risk assessments” under 

this definition. Second, there is no limitation that the “agents or stressors” be only chemicals. 

“Agents or stressors” may be chemicals, but they may also be biological agents, or physical 

agents, or even the absence of a necessity such as habitat. Third, this definition requires that the 

risks from multiple agents or stressors be combined. This does not necessarily mean “added,” 

but it means that some analysis needs to be conducted as to how the risks from the various agents 

or stressors interact. It also means that an assessment which covers a number of chemicals or 

other stressors, but which merely lists each chemical with a corresponding risk without 

consideration of the other chemicals present, is not an assessment of cumulative risk under this 

definition. 

Likewise, “cumulative risk assessment” in this Framework report means “an analysis, 

characterization, and possible quantification of the combined risks to health or the environment 

from multiple agents or stressors.” One key aspect of this definition is that a cumulative risk 

assessment need not necessarily be quantitative, so long as it meets the other requirements. 

The framework itself is conceptually similar to the approach used in both human health 

and ecological assessments, but it is distinctive in several areas. First, its focus on the combined 

effects of more than one agent or stressor makes it different from many assessments conducted 

today (which, if multiple stressors are evaluated, are usually evaluated individually and presented 

xviii 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

as if the others were not present). Second, by the fact that multiple stressors are affecting the 

same population, there is increased focus on the specific populations potentially affected, rather 

than a focus on hypothetical receptors. Third, consideration of cumulative risk may generate 

interest in a wider variety of non-chemical stressors than traditional risk assessments. 

The framework describes three main phases to a cumulative risk assessment: (1) 

planning, scoping and problem formulation, (2) analysis, and (3) interpretation. In the planning, 

scoping and problem formulation phase, a team of risk managers, risk assessors, and other 

stakeholders establishes the goals, breadth, depth, and focus of the assessment. The end products 

of this phase are a conceptual model and an analysis plan. The conceptual model establishes the 

stressors to be evaluated, the health or environmental effects to be evaluated, and the 

relationships among various stressor exposures and potential effects. The analysis plan lays out 

the data needed, the approach to be taken, and the types of results expected during the analysis 

phase. 

The analysis phase includes developing profiles of exposure, considering interactions (if 

any) among stressors, and predicting risks to the population or populations assessed. It is in this 

phase that difficult technical issues are addressed and hopefully resolved, for example, issues 

relating to toxicity of mixtures, vulnerability of populations, or the interactions among stressors 

which may be chemical or non-chemical. The end product of this phase is an analysis of the risks 

associated with the multiple stressors to which the study population or populations are exposed. 

The third phase, interpretation, includes what is usually termed the “risk characterization” 

discussion in risk assessment, where the risk estimates are put into perspective in terms of their 

significance, the reliability of the estimates, and the overall confidence in the assessment. It is 

also in this phase that an evaluation is made of whether the assessment met the objectives and 

goals set forth in phase one. 

The discussion of cumulative risk in this Framework report takes a broad view of the 

topic, including many aspects of an assessment that might conceivably be conducted in the 

future, even though techniques may not currently exist to examine every question. It also 

includes aspects of cumulative risk which may be outside of EPA’s current legislative mandates, 

and where expertise outside of the Agency would be needed to address certain questions if they 

should arise. These aspects of cumulative risk are discussed here for the sake of technical 

completeness and not as a recommendation that EPA perform all possible aspects of a 

cumulative risk assessment in all EPA risk assessments – even all EPA cumulative risk 

assessments. 

EPA is currently engaged in activities which fall under various aspects of the cumulative 

risk assessment umbrella. Some of these activities are listed as illustrations in the box on the next 

page. The broad interpretation of cumulative risk in this Framework report allows these activities 

to be put into perspective relative to one another, and can illustrate how they fit together under 

xix 
1

2

3

4

5

6

7

8

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Some Example Cumulative Risk Assessment Activities within EPA in late 2001 

!	 The Superfund Program has updated its guidance on risk assessment to include planning and scoping for 

cumulative risk assessment and problem formulation for ecological risk assessments. The plan for the Office of 

Solid Waste’s Surface Impoundment Study includes both a conceptual model and an analytical plan, per the 

agency guidance on planning and scoping for cumulative risk. 

!	 The Office of Water is planning a watershed scale risk assessment involving multiple stressors in ecological risk. 

This approach was developed through a collaboration with external scientists and is now being field evaluated. 

!	 Several Regional Offices are evaluating cumulative hazards, exposures, and effects of toxic contaminants in 

urban environments. In Chicago (Region 5), citizens are concerned about the contribution of environmental 

stressors toward endpoints such as asthma and blood lead levels. In Baltimore (Region 3), a 

regional/OPPTS/community partnership tried to address the long term environmental and economic concerns in 

three neighborhoods that are adjacent to industrial facilities and tank farms. Region 6 (Dallas) is developing a 

geographic information system approach for planning and scoping cumulative risks. 

! 	 The Food Quality Protection Act (FQPA) of 1996 requires the EPA to consider the cumulative effects to human 

health that can result from exposure to pesticides and other substances that have a common mechanism of toxicity. 

The Office of Pesticides Programs (OPP) has developed guidance for conducting cumulative risk assessments 

for pesticides, and has prepared a preliminary cumulative risk assessment for Organophosphorous pesticides. 

!	 The Office of Air and Radiation's air toxics program has a cumulative risk focus. Under the Integrated Urban 

Air Toxics Strategy (IUATS), OAR will be considering cumulative risks presented by exposures to air emissions 

of hazardous air pollutants from sources in the aggregate. Assessments will be performed both at the national 

scale - release of a national scale assessment for base year 1996 is planned for later this year - and at the urban or 

neighborhood scale. In partnership with ORD/NERL, the Office of Air Quality, Planning & Standards is 

developing the Total Risk Integrated Methodology (TRIM), a modular, modeling system for use in single or 

multi-media, single or multi-pathway, human health and ecological risk assessments of hazardous and criteria air 

pollutants at the neighborhood or city scale. The Agency’s guidance for planning and scoping of cumulative risk 

was used to develop a conceptual model and analysis plan for the national scale air toxics risk assessment. 

!	 The National Center for Environmental Assessment (ORD) has completed ecological risk assessment 

guidelines which support the cumulative risk assessment guidance. Five watershed case studies are being assessed 

to demonstrate the guidelines approach. Each of these cases deals with cumulative impacts of stressors (chemical, 

biological, and in some cases physical). In addition, NCEA has done a draft reassessment of dioxin and related 

compounds. 

!	 The Risk Assessment Forum has convened a technical panel to develop guidance for conducting cumulative risk 

assessments, of which this Framework is a first step. 

the framework. Individual Program Offices and Regions may have to make decisions affecting 

the scope, types of stressors, or methods used for their programs’ cumulative risk assessments, 

based on legislative mandates or other criteria. Nothing in this Framework report should be 

interpreted to mandate that cumulative risk assessment must be conducted, or must be conducted 

a certain way, for any specific case. Likewise, this Framework report is not an attempt to lay out 

protocols to address all the risks or considerations that are needed to adequately inform 

community decisions. Rather, this Framework report is an information document, focused on 

describing various aspects of cumulative risk, whether or not the methods or data currently exist 

xx 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1 to adequately analyze or evaluate those aspects of the assessment. Because of the limitations of 

2 current science, cumulative risk assessments in the near future will not be able to adequately 

3 answer all questions posed by stakeholders or interested parties. This does not mean, however, 

4 that they can’t answer some of the questions asked; in fact, cumulative risk assessment may be 

5 the best tool available to address certain questions dealing with multiple stressor impacts. 

xxi 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1. INTRODUCTION 

During much of its early history, EPA focused its efforts on cleaning up the overt 

pollution problems of the 1960s and 1970s. Until EPA was established in 1970, relatively 

uncontrolled air emission, water effluents, and dumping of wastes had led to pollution of the 

environment that was easily detected by the five senses. The most effective and efficient way to 

approach these overt problems of the 1970s was to find the entry point of the pollution into the 

environment, and to keep it from entering the environment by controlling it there. Looking back, 

we see a strategy that moved to control stack emission, industrial and municipal effluents, 

pesticide application, land applications, burial of chemical wastes, and other “sources” of 

pollution. In addition, criteria and standards were established as goals for cleanup of the various 

environmental media. By the 1980s, this “command and control” strategy was well established in 

environmental laws and regulations, but was reaching the point of diminishing returns from a 

cost-benefit viewpoint. 

The development of risk assessment methodology during the 1970s and early 1980s 

closely followed the Agency’s strategy for control of pollution, since risk assessments were being 

used as one of the factors in EPA’s decision-making for regulations. The focus on sources led 

naturally to analysis of what types of pollutants were in effluents, air emissions, and waste sites. 

Figure 1-1. Chemical (or stressor) 

focused assessment starts with a source 

and evaluates how the chemical gets to 

various populations or ecological targets. 

Individual assessments may choose to 

pursue some or all pathways, media, or 

population segments. 

These were chemical, biological, and sometimes 

radiological agents. By the 1970s, the links between 

some chemicals and certain diseases such as cancer 

had been established through a series of bioassays, 

or in the cases of chemicals like vinyl chloride and 

asbestos, through epidemiological studies. New 

analytical techniques of the 1970s also made it 

possible to detect very minute concentrations of 

chemicals for the first time. The focus of the EPA 

strategy to control pollution (and the risk 

assessment methodology being used to partially 

support decisions) gradually leaned toward 

assessing and controlling the individual chemicals. 

Congressional legislation tended to underwrite this 

approach by focusing on controlling sources and 

even including lists of individual chemicals to be 

controlled. 

The risk assessment methodology of the 

1970s and early 1980s, for this reason, tended 

towards single chemical assessments (see Figure 1-

1). The 1983 National Research Council report Risk 

1
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Assessment in the Federal Government (NRC, 1983) was largely focused on the single chemical 

risk assessment approach when it spoke of the four parts of a risk assessment: hazard 

identification, dose-response assessment, exposure assessment, and risk characterization. EPA’s 

1986 Risk Assessment Guidelines (USEPA 1986a), with the exception of the mixtures guidelines 

(USEPA, 1986b), were also largely focused on single chemical assessment. 

Research conducted or sponsored by EPA in 

the early 1980s, however, was taking the first steps 

toward investigating a different type of risk 

assessment methodology, one that focused on the 

persons exposed, investigating the chemicals or 

stressors to which they were exposed, and 

consequent risks (Figure 1-2). This is in contrast to a 

focus on either a chemical, to investigate its 

environmental fate, exposed populations, and risks 

(Figure 1-1), or focus on a source to investigate its 

environmental releases, exposed populations, and 

risks. The goals of the population-focused approach1 

were much more useful to decision-makers who 

were dealing with public health or ecological health 

questions, rather than controlling sources of 

pollution. 

The challenges posed by the populationFigure 1-2. Population-based assessments 

start with the receptors, and determine 

what chemicals, stressors, or other risk 

factors are affecting them. 

based assessment can be daunting, even if only a few of the stressors affecting a population are 

evaluated together (i.e., cumulatively). Taken to the extreme, Figure 1-2 represents a concept of 

“total risk” for the population or population segment being evaluated, with each chemical, 

biological, radiological, or other stressor2

 adding some fraction of the total risk. Looking at the 

problem from an individual stressor viewpoint, to do this type of assessment would require not 

1

 A chemical-focused assessment may look at several populations affected by exposure to the chemical, but not at other 

chemicals. A population-focused assessment looks at one population for perhaps many stressors, but not at other populations. 

Consequently, for traditional, chemical-focused assessments, we say we conduct a “risk assessments for a certain chemical.” In 

contrast, the essence of a cumulative risk assessment is that the assessment is conducted “for a certain population.” This 

difference is shown schematically by comparing figures 1-1 and 1-2. How the population is identified for a cumulative 

assessment is not addressed here. 

2 A stressor is a physical, chemical, biological, or other entity that can cause an adverse response in a human or other 

organism or ecosystem. A stressor can be exposure to a chemical, biological, or physical agent (e.g., radon), or it may be the lack 

of, or destruction of, some necessity such as a habitat. A socioeconomic stressor, for example, might be the lack of needed health 

care, which could lead to adverse effects. Harmful events, such as automobile crashes, could also be termed stressors. Obviously, 

calculating risks from different types of stressors can use widely different methods, including probabilistic estimates of disease 

via dose-response relationships, looking up rates in statistical tables of historical events, and other methods. 

2 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

only evaluating each individual stressor, but also developing a way to add up all the risks among 

stressors across a population of individuals with different exposures and susceptibilities. In the 

early 1980s, the state of the science was unready for virtually any part of the methods for doing 

this type of assessment. 

But progress was being made toward developing a population-based methodology. 

Starting in the late 1970s, a group of EPA researchers and contractors began developing what 

would become the Total Exposure Assessment Methodology (TEAM) study (USEPA 1987). 

TEAM measured the concentrations of a number of chemicals simultaneously at the point of 

exposure. This led to a larger study, the National Human Exposure Assessment Survey 

(NHEXAS) in the 1990s (Sexton, et, al. 1995). Both TEAM and NHEXAS were populationbased exposure assessment approaches which developed analytical tools and methodologies to do 

this type of exposure assessment. 

Also in the early 1980s, some progress was being made toward the question of how to 

cumulatively consider the risks from different chemicals or stressors. The 1986 Risk Assessment 

Guidelines (USEPA, 1986a) included a guideline on chemical mixtures (USEPA, 1986b), which 

discussed how the risks from multiple chemicals could be evaluated as a whole. The work on this 

guidance has continued most recently with the Supplementary Guidance for Conducting Health 

Risk Assessment of Chemical Mixtures (USEPA, 2000e) which expands and supplements the 

1986 beginnings. 

About the same time the Agency made some progress on single chemical and chemical 

mixture risk assessment with the 1986 Guidelines, some different kinds of risk assessment 

problems began to catch the Agency’s attention. In 1986, eleven Chicago-area community 

groups joined together to file a petition under Section 21 of the Toxic Substances Control Act 

asking for a community assessment in Southeast Chicago. A series of community-based actions 

which started in 1982 and grew throughout the 1980s focused on disparities of risk among 

various population subgroups, calling specific attention to cumulative effects of pollution on 

minority subgroups (GAO, 1983; United Church of Christ, 1987). This series of communitybased actions, chronicled in the 1990 book Dumping in Dixie: Race, Class and Environmental 

Quality (Bullard, 1990) eventually became known as the Environmental Justice movement. The 

issues raised by the Environmental Justice movement were the basis of a 1994 Presidential 

Executive Order [Executive Order 12898, February 11, 1994] which told Agencies, among other 

things, that “Environmental human health analyses, whenever practicable and appropriate, shall 

identify multiple and cumulative exposures.” In the 1990s, Environmental Justice cases, 

including the cases which have been filed under Title VI of the 1964 Civil Rights Act, [P.L. 88-

352, July 2, 1964] have further emphasized the need for a cumulative human health risk 

assessment methodology. 

Even before Executive Order 12898 was issued, it was apparent that population-focused 

3
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

assessments (like Figure 1-2) were going to be needed, in addition to the chemical- or stressorfocused assessments (like Figure 1-1), if EPA was going to be able to answer the questions and 

issues being raised by the public. Community spokespersons and other “stakeholders,” as well as 

scientific panels, were increasingly coming to the Agency with problems that demanded a multistressor approach (e.g., NRC 1994). Ecological problems, especially, were demanding a “placebased” context (such as the Chesapeake Bay watershed) in which the various populations within 

the area were looked at from a “total system” viewpoint. This place-based focus was a part of the 

1992 Framework for Ecological Risk Assessment (USEPA 1992b) and the 1998 Guidelines for 

Ecological Risk Assessment (USEPA 1998b). 

Although clearly addressing more than cumulative human health or ecological risk 

assessment, the National Environmental Policy Act of 1969 (NEPA) [P.L. 91-190, 42 U.S.C. 

4321-4347, January 1, 1970, as amended by P.L. 94-52, July 3, 1975, P.L. 94-83, August 9, 

1975, and P.L. 97-258, §4(b), Sept. 13, 1982], which was passed at about the same time EPA 

was established, requires assessments on the cumulative impacts of federal or federally-funded 

projects (such as roads, dams, power lines, military projects, and infrastructure development) on 

natural ecosystems, endangered species, habitats, and opportunities for public enjoyment and 

natural resource use. A primary concern for NEPA is “cumulative effects analysis,” defined as 

“the incremental impact of the action when added to other past, present, and reasonably 

foreseeable future actions . . . Cumulative impacts result from individually minor but 

collectively significant actions taking place over a period of time” (CEQ, 1997). Much of the 

NEPA cumulative effects analysis is qualitative, but risk assessments and cause-and-effect 

relationships are key parts of the analysis process for controversial projects. 

In 1997, the Agency issued a policy memo, Guidance on Cumulative Risk Assessment, 

Part 1: Planning and Scoping (USEPA, 1997a), which took the first formal step towards 

developing guidance and guidelines for cumulative risk assessment. 

By the first decade of the twenty-first century, cumulative risk assessment applications 

have become relatively common. These applications are not only for assessments of chemicals 

which operate by the same mode of action, as is mandated for the USEPA Pesticides Program, 

but also community based, population-based, assessments which may include more varied 

stressors than just chemicals alone. Much like the “place-based” ecological assessments, which 

may cover a wide variety of physical, chemical, and biological stressors, some communities have 

added human health and perhaps “quality of life” to the endpoints of interest in their place-based 

assessments. It is the demand for more sophisticated human health risk assessments that has 

driven the need for research into cumulative risk assessment, population-focused assessments, 

aggregate exposure assessment, and risk from chemical mixtures. 

4
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1.1. Purpose and Scope of the Framework Report 

An understanding of the finite purpose and scope of this Framework report is important. 

EPA and other organizations need detailed, comprehensive guidance on methods for evaluating 

cumulative risk. Before such detailed Agency-level guidance is developed on a relatively new 

field of risk assessment, it has been the recent policy of the Agency to first develop a simple 

framework as a foundation for later comprehensive guidance. This Framework for Cumulative 

Risk Assessment will emphasize chemical risks to human health in its discussion, but will do so 

in the context of the effects from a variety of stressors, including non-chemical stressors. Some 

important topics that could be 

characterized as “cumulative risk,” such as 

global climate change, are beyond the 

scope of this Framework report. 

With this background, the 

Framework has two simple purposes, one 

immediate and one longer term. As a 

broad outline of the assessment process, 

the Framework immediately offers a basic 

structure and provides starting principles 

for EPA’s cumulative risk assessments. 

The process described by the Framework 

report provides wide latitude for planning 

and conducting cumulative risk 

assessments in many diverse situations, 

each based on common principles 

discussed in the Framework report. The 

process also will help foster a consistent 

EPA approach for conducting and 

evaluating cumulative risk assessments, for 

identifying key issues, and for providing 

operational definitions for terms used in 

cumulative risk assessments. 

In the longer term, the Framework 

report offers the basic principles around 

which to organize a more definitive set of 

Cumulative Risk Assessment Guidance. 

With this in mind, this report does not 

provide substantive guidance on certain 

issues that are integral to the risk 

EPA’s Risk Assessment Guidelines 

Chemical Mixtures (USEPA 1986b)
 

Mutagenicity Risk Assessment (USEPA 1986c)
 

Carcinogen Risk Assessment (USEPA 1986d)
 

Developmental Toxicity Risk Assessment (USEPA 1991a)
 

Exposure Assessment (USEPA 1992c)
 

Reproductive Toxicity Risk Assessment (USEPA 1996b)
 

Proposed Carcinogen Risk Assessment (USEPA 1996c, 1999a,
 

1999b) 

Ecological Risk Assessment (USEPA 1998b) 

Neurotoxicity Risk Assessment (USEPA 1998c) 

Selected Policy and Guidance Documents 

Risk Assessment Guidance for Superfund (USEPA 1989a)
 

Locational Data Policy (USEPA 1991b)
 

Framework for Ecological Risk Assessment (USEPA 1992b)
 

Application of Refined Dispersion Models (USEPA 1993a)
 

Policy /Guidance for Risk Characterization (USEPA 1995ab)
 

Benchmark Dose (1995c, 2000b)
 

Cumulative Risk Planning and Scoping (USEPA 1997a)
 

Guiding Principles for Monte Carlo Analysis (USEPA 1997b)
 

Acute Inhalation Exposure (USEPA 1998d)
 

Chemical Emergency Risk Management (USEPA 1998e)
 

Draft Comparative Risk Framework (USEPA 1998f)
 

Aggregate Exposure and Risk (USEPA 1999g)
 

Community Involvement in Superfund RA (USEPA 1999c)
 

Guidance for Offsite Consequence Analysis (USEPA 1999d)
 

Guideline on Air Quality Models (USEPA 1999e)
 

Framework for Community Based Env. Prot. (USEPA 1999f)
 

Handbook for Risk Characterization (USEPA 2000c)
 

Handbook for Peer Review (USEPA 2000d)
 

Supplementary Guidance for Conducting Health Risk 
 

Assessment of Chemical Mixtures (USEPA 2000e) 

Cumulative Risk Assessment of Pesticide . . . Common 

Mechanism of Toxicity (USEPA, 2002a) 

5
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

assessment process (see box at right and Appendix B for a listing of useful resources). These 

include specific analytical methods, techniques for analyzing and interpreting data, and guidance 

on issues influencing policy. Rather, on the basis of EPA experience and recommendations of 

peer reviewers, EPA has reserved discussion of these important aspects of cumulative risk 

assessment for future Guidance, which will be based on the risk assessment process described in 

this Framework report. 

This Framework report is meant to lay out broad areas where analysis might be conducted 

if needed. It does not suggest that cumulative risk assessment is a tool that should be used with 

every issue, nor does it suggest that when cumulative risk assessment is applied, that all areas of 

analysis outlined or discussed here must or even should be conducted in every assessment. The 

scope of the assessment should be defined in the planning and scoping stage (see section 2.1), 

and may include or exclude stressors or pathways as relevant to the particular context or 

application. In some areas discussed in this Framework report, the methodology for doing the 

risk analysis currently may not exist. 

1.2. Intended Audience 

This Framework report is primarily intended for EPA risk assessors, EPA risk managers, 

and other persons who either perform work under EPA contract or sponsorship or are subject to 

EPA regulations concerning risk assessments. The terminology and concepts described here also 

may be of assistance to other Federal, State, and local agencies as well as to members of the 

general public, including stakeholders, who are interested in cumulative risk assessment issues. 

The style and language used in this Framework report are chosen to be understood by as wide a 

variety of interested parties as possible, from the policy maker to the risk assessment scientist to 

the concerned non-scientist member of the general public. It is hoped that this Framework report 

will be the first step in developing a broad scientific consensus about cumulative risk assessment, 

and that further guidelines and guidance will build upon this foundation. 

1.3. Key Definitions in Cumulative Risk Assessment3 

In this Framework report, “cumulative risk” and “cumulative risk assessment” are defined 

as follows, assuming a defined population: 

3 In this section, a few basic definitions related to cumulative risk assessment will be discussed. For a glossary of 

terms, the reader is directed to Section 5. 

6 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Cumulative Risk: The combined risks from aggregate exposures4 to multiple agents or 

stressors. 

Cumulative risk assessment: An analysis, characterization, and possible quantification 

of the combined risks to health or the environment from multiple agents or stressors. 

There are several key points which come from this definition of cumulative risk. First, 

cumulative risk involves multiple agents or stressors, which means that assessments involving a 

single chemical or stressor are not “cumulative risks” under this definition. Second, there is no 

limitation that the “agents or stressors” be only chemicals. “Agents or stressors” may be 

chemicals, of course, but they may also be biological agents, or physical agents, or even the 

absence of a necessity such as habitat. Third, this definition requires that the risks from multiple 

agents or stressors be combined. This does not necessarily mean “added,” but it means that some 

analysis needs to be conducted as to if, and how, the effects or risks from the various agents or 

stressors interact. It also means that an assessment which covers a number of chemicals or other 

stressors, but which merely lists each chemical with a corresponding risk without consideration 

of the other chemicals present, is not an assessment of cumulative risk under this definition. 

The definition of cumulative risk assessment follows from the definition of cumulative 

risk, but again, there is a key point: cumulative risk assessments can be qualitative as well as 

quantitative. 

Some examples of types of cumulative risk assessments, and some examples of 

assessments we would not describe as “cumulative risk assessments,” are listed below. Each of 

these presupposes a defined individual or population5

: 

1. Single agent or stressor assessments. Risks can be added or accumulated over time 

for a single agent or stressor across sources, environmental pathways, or exposure routes. This 

is consistent with “aggregate risk” in the FQPA terminology in the box on the next page. 

Although this might conceivably be termed a cumulative risk assessment by some scientists, for 

clarity in this Framework report, such single-stressor assessments will be termed “aggregate risk 

assessments,” rather than “cumulative risk assessments.” Examples of this type of assessment 

might be a multi-source assessment of benzene risk in a community, or an assessment of 

individual risk to a specific pesticide from all uses combined. This type of assessment is not 

4 See the text box on the following page for a definition of aggregate exposure. 

5 Populations can be defined by geophysical boundaries, such as a watershed, geopolitical boundaries, such as city or 

county limits, or by cultural, racial, economic, or other criteria within a certain geographic boundary such as a neighborhood. The 

definition of a population needs to be clear enough so that it can be agreed upon whether any specific individual is included in or 

excluded from the population. 

7 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

discussed in this Framework except to be 

referred to occasionally for clarity and 

contrast to cumulative risk assessments. 

2. Multiple stressor assessments. 

Exposures can be accumulated over time, 

pathways, sources, or routes for a number of 

agents or stressors. These stressors may cause 

the same effects (e.g., a number of 

carcinogenic chemicals or a number of 

threats to habitat loss), or a variety of effects. 

A risk assessment for multiple stressors may 

evaluate the risks of the stressors associated 

health effects or ecological impacts, one 

effect or impact at a time, or it may evaluate 

the combined risk from some or all the 

effects or impacts together. In either case, we 

will call these assessments cumulative risk 

assessments. 

FQPA’s Terminology Interpretations 

The Food Quality Protection Act of 1996 [P.L. 104-170] 

discusses the addition of exposure for a single chemical 

across sources, pathways, routes, and time as aggregate 

exposure. To be consistent with that terminology, the 

Agency has elected to speak of multiple source/pathway/ 

route single stressor exposures and risks as “aggregate 

exposures” and “aggregate risks.” The EPA Science 

Policy Council’s Cumulative Risk Subcommittee has 

developed the following working definitions for singlechemical or single-stressor situations: 

Aggregate exposure: The combined exposure of 

an individual (or defined population) to a specific agent or 

stressor via relevant routes, pathways, and sources. 

Aggregate risk: The risk resulting from aggregate 

exposure to a single agent or stressor. 

A multiple stressor cumulative risk assessment is distinct from a series of aggregate risk 

assessments as it includes consideration of any combined impact of the stressors including the 

potential for interactions among stressors (e.g., synergism or antagonism). One example of a 

multiple stressor, single effect cumulative risk assessment would be the combined risk to an 

individual or population from a series of pesticides all acting by the same mode of action and 

causing the same effect. 

Another example would be a dioxin assessment, where toxic equivalency factors (TEFs) 

are used to combine the toxicities of dozens of different congeners of chlorinated dibenzo-pdioxins and dibenzofurans, resulting in a single estimate of risk for a specific effect from the 

combination of congeners. 

Another example is a physician’s use of a model, derived empirically from 

epidemiological studies, to estimate the probability of a woman’s developing breast cancer over 

the next ten years. The “stressors” in the example of the breast cancer model are certain factors 

known to be correlated with that form of cancer, such as the woman’s age at first childbirth, age 

at menarche, having a previous biopsy with atypical hyperplasia, and others. This example shows 

that stressors may not necessarily be chemical stressors, nor do they all even need to be the same 

types of stressors. 

Another type of cumulative risk assessment that will be discussed in this report is the 

multiple stressor, multiple effects assessment. Again, stressors need not be limited to chemicals, 

8
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

nor do they even have to be the same 

types of stressors to be included in this 

type of assessment. Nor do the effects 

have to be similar. For example, 

chemical, biological, radiological, other 

physical, and even psychological stressors 

can cause a variety of human health or 

ecological health effects. Assessing the 

risk for these situations is considerably 

more complex methodologically and 

computationally than the examples of 

aggregate risk assessments or singleeffect cumulative risk assessments given 

in the above paragraphs. 

As complex as this may sound, 

there are several examples of this type of 

assessment. Although these analytical 

approaches may start with the stressors 

and predict the risk of effects, more 

generally these types of assessments start 

with a defined geographical area or 

defined population and try to determine 

what stressors are important. 

For example, cumulative 

ecological risk assessments such as those 

that have been conducted in the Columbia 

River Basin and the Chesapeake Bay 

focus on a number of observed adverse 

Cumulative Risk Assessment Features 

While many different types of exposures, stressors and other 

factors can be included, the definition of cumulative risk might 

be better understood by contrasting the featured and optional 

considerations. By the definition given above for this Framework 

report, the following features are included: 

• multiple stressors 

•	 consideration of how the stressors act together, rather 

than individually 

•	 population focused assessment. Although this does not 

mean that the assessment must start with a population 

and work “backwards” toward the source, it does mean 

that the population needs to be defined and multiple 

stressors are assessed with regard to impact on that 

population, although not every individual will see the 

same (or all) effects. 

Additional layers of complexity, such as those listed below, may 

or may not be addressed: 

•	 multiple durations, pathways, sources, or routes of 

exposure. 

• multiple effects or impacts. 

•	 nonconventional stressors or risk factors (e.g., lifestyle, 

access to health care). These in general need continued 

research. 

• quantification of risks. 

conditions, then attempt to determine, among all of the possible stressors, which particular 

combination is responsible for the observed adverse conditions (Barnthouse, et al., 2000). 

The National Research Council, in its 1994 book Science and Judgment in Risk 

Assessment (NRC, 1994, appendix I), lays out the general mathematics for a quantitative 

approach to multiple stressor, multiple effect assessments. Recently, Bogen (2001) used this 

approach to quantify combined risk of cancer and noncancer endpoints induced by the chemical 

trichloroethylene (TCE), including quantitative characterization of associated interindividual 

variability and associated uncertainty (including uncertainty regarding mechanism of 

carcinogenic action). Technical hurdles involved in implementing this approach become those of 

defining the set of relevant (preferably independent) endpoints and of quantifying the likelihood 

of inducing each adverse health or ecotoxic response considered unacceptable as a function of the 

9
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

endpoints. 

Another example of a type of multiple stressor, multiple effect assessment would be a 

cumulative community health risk assessment. 

We believe that the definition of cumulative risk used in this Framework report is 

consistent with the sense of most definitions of “cumulative” such as are included in NEPA or 

FQPA. A summary of the features and options of a cumulative risk assessment, by the definition 

used in this report, is given in the box on the previous page. 

1.4. The Cumulative Risk Assessment as a Tool for a Variety of Users and Purposes 

As discussed in the Introduction, the results of the assessment should reflect the purpose 

for doing the assessment. Information from cumulative risk assessments can also serve a variety 

of other purposes, however. Insights gained may also be used to partly meet regulatory 

mandates, to help identify targets for enforcement actions, or be considered when shaping policy 

and regulation. Assessments may also conceivably be used in the long term planning with regard 

to siting new sources of potential pollution in specific areas. Assessments also may be used for 

general educational purposes not directly related to an immediate decision on a course of action. 

Assessment results can also help guide priorities for voluntary or regulatory action, or to 

mobilize community efforts to address concerns. They can be done retrospectively (to determine 

past or current risks), prospectively (to assess the risks of, say, proposed facilities), or even 

creatively (to design a development plan for a community). As helpful as results may be in any of 

these other uses, however, some consideration must be given to the appropriateness of using the 

assessment for these purposes, given the objectives and scope of the assessment. 

Risk assessment, including cumulative risk assessment, is conceptually an analyticdeliberative process (NRC, 1996). It includes both analytic (i.e., rigorous, replicable methods, 

evaluated under the agreed protocols of an expert community) and deliberative (i.e., stakeholdervalue-and-judgment based) parts. Much of what is discussed in Chapter 2, the Planning and 

Problem Formulation Phase, is deliberative in nature, which means it depends on input from 

experts other than those who know how to do risk assessments. These include persons who are 

knowledgeable about a community and its values. Although much of Chapter 3, the Analysis 

Phase, is given over to the analytic process where risk assessment experts apply science to a 

problem, the deliberative aspect returns in Chapter 4, the Interpretation Phase, especially where 

risks of different types are being evaluated and combined. 

Cumulative risk assessment, because of this analytic-deliberative process, can be applied to a 

variety of different problems where analysis of the overall impacts of multiple sources, stressors, 

pathways, or routes is necessary. It can be used as a regulatory analysis tool, such as in reviewing 

the overall impact of several different pesticides that all act by the same mode of action (ILSI, 

10
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1999), or in NEPA analyses (CEQ, 1997). 

It can be used to analyze the overall 

impacts of permit decisions or the results 

of compliance with permits in a given 

community. 

Cumulative risk assessment can 

also be used in a community-based 

assessment approach, such as is outlined 

in EPA’s Framework for CommunityBased Environmental Protection 

(USEPA, 1999f). The CBEP approach 

The Core Principles of Community-Based 

Environmental Protection (CBEP) 

1. Focus on a definable geographic area. 

2. Work collaboratively with stakeholders. 

3. Assess the quality of all resources in a place. 

4. Integrate environmental, economic, and social objectives. 

5. Use the most appropriate tools. 

6. Monitor and redirect efforts through adaptive management. 

Source: U SEPA , 1999f 

(see box) encompasses both ecological and human health assessments. Cumulative risk 

assessment, being a population-based or place-based analytic-deliberative process, is ideal for 

CBEP-type applications. 

Cumulative risk assessment is also applied in ecological assessments. The definition of 

cumulative ecological risk assessment, as given in the EPA’s 1998 Guidelines for Ecological 

Risk Assessment is: A process that involves consideration of the aggregate ecological risk to the 

target entity caused by the accumulation of risk from multiple stressors (USEPA, 1998b). A 

recent Society of Environmental Toxicology and Chemistry publication (Foran and Ferenc, 1999) 

discusses multiple stressors in ecological risk assessment, and gives a good overview of the topic 

of cumulative ecological risk assessment. 

When should a cumulative risk assessment be done? Recognizing that the scope and 

nature of a cumulative risk assessment may range from a very limited qualitative assessment of a 

local situation, to a comprehensive assessment of the cumulative risk patterns for a large 

community, to a national assessment conducted within one of EPA’s programs, the simple 

answer is that one should be conducted whenever the combined impact of multiple stressors 

needs to be considered. Only experience with these assessments over a period of time will 

provide the wisdom needed to develop practical guidelines on this question. 

1.5. The Broader Decision-Making Context for Cumulative Risk Assessment 

Cumulative risk assessments may be used to form hypotheses that could be tested, but it 

is more likely that these assessments will be used as decision-making tools. Decisions can be at 

a wide variety of levels, from a neighborhood group evaluating ways to improve or safeguard 

their health and environment, to a Federal official weighing options for action at a much broader 

geographical level. Although the decision-making method is beyond the scope of this Framework 

report, such decisions usually involve more than the basic science and analysis that make up the 

“scientific” part of risk assessment. Robert T. Clemen, in his book Making Hard Decisions notes 

11
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

that in one type of decision-making approach (called decision analysis): 

Managers and policy makers frequently complain that analytical procedures from 

management science and operations research ignore subjective judgments. Such 

procedures often purport to generate “optimal” actions on the basis of purely objective 

inputs. But the decision-analysis approach allows the inclusion of subjective judgments. 

In fact, decision analysis requires personal judgments: they are important ingredients for 

making good decisions. (Clemen, 1996, page 5) 

Regardless of the type of decision being made or the decision-making approach, a 

cumulative risk assessment’s analytic part is not the decision-making vehicle in itself. That is, 

“cranking out the numbers” will not be the sole basis for a decision. Although in some cases, the 

estimated risks can weigh heavily in the decision, understanding the risk estimate is but one 

factor in a broader decision-making process including risk management components such as 

technical feasibility, economic costs and benefits, political realities, and other factors. The U.S. 

EPA’s Science Advisory Board (SAB) in their August, 2000, publication Toward Integrated 

Environmental Decision-Making (USEPA, 2000a), constructed a framework for what it termed 

Integrated Environmental Decision-making (IED). The SAB noted that “The IED Framework 

recognizes that risks often are experienced simultaneously and are cumulative. . .”. It speaks of 

risk assessments in a very broad way, including human health effects, ecological effects, and 

quality-of-life effects. The first phase and part of the second phase of the IED, “Problem 

Formulation” and “Analysis and Decision-making” essentially correspond to the three phases we 

discuss in this Framework for Cumulative Risk Assessment. Decision-making, and the SAB’s 

third phase, “Implementation and Performance Evaluation,” are beyond the scope of this 

Framework report. 

The SAB’s report (USEPA, 2000a) gives a good insight into the broader context for 

cumulative risk assessment, and some of the aspects of the analytic-deliberative parts of the 

assessment. The analytical-deliberative process will be discussed more in Chapters 2 through 4, 

as these phases of the cumulative risk assessment process are examined. 

The 1996 book Understanding Risk (NRC, 1996) also provided much information on the 

analytic-deliberative aspects of a risk assessment, and devoted a great deal of discussion to risk 

characterization. Needless to say, it is very important to apply cumulative risk assessment in the 

context of the decision or decisions to be made. This is most efficiently done by early and 

continued attention to the “risk characterization” step in the risk assessment process (NRC, 1996; 

USEPA, 2000c). The box in section 4.1 summarizes some of the points made in Understanding 

Risk. 

12
 
1

2

3

4

5

6

7

8

9

10

11

12

13

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Figure 1-3. Framework for Cumulative Risk Assessment 

1.6. Organization of this report 

Figure 1-3 shows the basic structure of this Framework for Cumulative Risk Assessment. 

Each of the three general process steps are described in detail in later chapters. The Framework is 

organized to follow the outline in Figure 1-3, namely (a) a planning, scoping, and problem 

formulation phase (Chapter 2), (b) an analysis phase (Chapter 3), and (c) an interpretation phase, 

where the risk characterization is completed (Chapter 4). Chapter 5 is a glossary of terms, 

followed by References in Chapter 6. Additional information on selected resources and 

cumulative risk related topics are provided in the appendices. 

13
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Figure 2-1. The Planning, Scoping, and Problem Formulation Phase. 

2. THE PLANNING, SCOPING, AND PROBLEM FORMULATION PHASE 

The first step in any risk assessment process is to define the problem to be assessed. This 

step has been called “problem formulation” in the Framework for Ecological Risk Assessment 

(USEPA, 1992b), the NRC book Understanding Risk (NRC, 1996), Toward Integrated 

Environmental Decision-Making (USEPA, 2000a) and elsewhere (e.g., USEPA, 1997a). It is a 

phase where, according to NRC, “public officials, scientists, and interested and affected parties 

clarify the nature of the choices to be considered, the attendant hazards and risks, and the 

knowledge needed to inform the choices” (NRC, 1996) . 

Planning and Scoping of the assessment are often thought of as being part of the Problem 

Formulation phase, although the 1997 Planning and Scoping guidance treats Planning and 

14
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Scoping as a separate activity before problem formulation begins (USEPA, 1997a). Whether it is 

considered a separate phase or not, it takes place at the very start of the process of doing a 

cumulative risk assessment. For convenience, this section incorporates both Planning and 

Scoping and Problem Formulation into a single phase. 

2.1. Planning and Scoping 

Risk assessments are conducted within some context, that is, they are usually conducted 

because of a regulatory requirement, a community need, a health crisis, or some other “driving 

force.” This context generates individuals or groups with interest in having the assessment done, 

and there are several summary articles or books available about the challenges of successful 

participation by these interested parties (e.g., Chess and Purcell, 1999; Frewer, 1999; Thomas, 

1995). They may be public officials, risk experts, community leaders, or any number of others. 

Planning and scoping begins with a dialogue among these interested parties. 

Among these interested parties, there will be a person or a group of people charged with 

making decisions about how a risk may be mitigated, avoided, or reduced. For the sake of 

simplicity, we will call this person or group the “decision maker,” or “risk manager6

,” and for 

ease of discussion, will discuss the risk manager as if it were a single person. 

During planning and scoping, risk experts (including those involved in assessing risk 

such as ecologists, toxicologists, chemists, along with other technical experts such as economists 

and engineers) and decision makers work together as a team, informed by stakeholder input, to 

develop the rationale and scope for the risk assessment and characterization. 

As part of the initial discussions concerning the need for a risk assessment, other 

“interested and affected parties” besides the risk manager and risk assessor may help define 

purpose, scope, and approach. This “risk assessment planning team” seeks agreement through 

extensive dialogue and discussion on what analytical and deliberative steps need to be taken, and 

by whom, by when, and why (USEPA, 2000a). The SAB’s report Toward Integrated 

Environmental Decision-Making explains some of the roles of the various participants on the risk 

assessment planning team during the Planning and Problem Formulation phase: 

“Scientists play an important role in [this phase] by collecting, analyzing, and presenting 

data in such a way that all parties can appreciate the type and magnitude of the problem(s) 

under discussion. This activity will generally involve all four parts of risk assessment, 

including assessment of exposures experienced by special populations and/or ecological 

resources. Planning, scoping, and screening -- including selection of endpoints of 

6

 We will use the term “risk management”to include actions that the risk assessment team recommends or implements 

that are not taken by the risk assessment team, per se. These include actions to address the problems taken by others outside the 

process, who may not be identified until the analysis is underway or complete. 

15 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

concern -- also requires explicit input of societal values and stakeholder participation. 

For instance, while some of the ecological endpoints may be chosen because of their role 

in a valued ecosystem, there may also be ecological endpoints chosen because of their 

direct significance to society. Examples of the latter include both economically important 

species and ‘charismatic’ species. Similarly, in integrated decision-making, judgments 

may have to be made about diverse health endpoints, such as cancer risks in the general 

population and the risk of reproductive/developmental risks in children. While scientists 

can help characterize such risks, they are not uniquely qualified to set priorities among 

them and broader deliberation is essential. Finally, decision-makers also play an 

important role during problem formulation; in addition to bringing the scientific and other 

resources of the Agency to bear on the problem, they also should help to identify the 

range of potential decisions and viable management options, while examining economic, 

political, or other constraints on those options. Decision-makers also serve as managers 

of the overall process.” (USEPA, 2000a) 

Another role of the risk assessment planning team is documentation. The activities of the 

following sections are important, and should be documented by the team for several reasons. 

Written records can be referred to by assessors and people at public meetings. They can also help 

prepare for responding to comments, and begin establishing a peer-review record for any later 

decisions or plans that need to be peer reviewed (USEPA, 2000d). The risk assessment planning 

team should consider whether or not the overall project is to be peer reviewed, and if so, what 

type of peer review will be conducted. The team should plan and execute the peer review at the 

appropriate time. A peer review by an independent review group will not only help establish the 

validity of the science, but can also provide neutral comments on some of the interpretations of 

the assessment. 

In some cases, it may be useful for the stakeholders to appoint a “point person” to serve 

as point of contact for communications. This is not to imply that stakeholders must speak with a 

single voice (which is not likely in any case), but that they have at least one person to help 

facilitate interactions and identify available technical resources and other sources of information. 

The Agency or stakeholders may also consider a public web site for the project. A variety of 

resources can be posted, including cumulative risk tools and databases, project-related news, list 

of experts, glossary, reports, related links, etc. An online discussion forum could also be 

included on the web site as a more interactive way of exchanging information with stakeholders. 

Finally, while including stakeholders in the risk assessment process, a regulatory agency 

like EPA needs to balance stakeholder participation with the Agency’s need to retain the ability 

to carry out its responsibility to protect public health and the environment. For this reason, EPA 

will usually need to set some reasonable boundaries around the process to ensure that progress is 

being made in a timely and efficient fashion. 

2.1.1. Defining the Purpose of the Assessment 

16
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

As discussed in section 1.5 above, the risk assessment should be developed to inform the 

risk management decision by constructing an appropriate, decision-relevant risk characterization. 

After the risk assessment planning team is assembled, the dialogue between the decision maker 

and risk experts begins with a discussion on risk management objectives and information needed 

to manage risks in the particular situation. The manager and assessment planning team must 

discuss any regulatory or legal basis for the risk assessment, and what kind of information is 

needed to satisfy such requirements. If interested and affected parties are part of the risk 

assessment planning team, it is especially important that the entire team agree on the purpose of 

the assessment, since a differing sense of purpose among the team will lead to problems later. 

The purpose and risk management objectives guide the risk assessment strategy (see box for 

some possible management goals from which risk management objectives can be derived, e.g., in 

terms of key participants, data sources, selection of assessment endpoints, approach, and the 

schedule for developing the assessment). 

The previous discussion follows the 

typical situation where the risk manager is 

presented as an independent decision-maker, 

such as a senior official in a regulatory agency 

who is responsible for establishing permit 

conditions for a facility of some type. There 

are situations, however, where the risk 

manager may be one of the interested parties, 

such as a local citizens’ board. For example, 

the risk assessment may indicate that 

mitigation of risks may not be significantly 

affected by any permit decisions but will 

depend instead on local zoning decisions or on 

decisions which affect traffic patterns in a 

community. This is one of the reasons why, in 

the final step in the planning and problem 

formulation phase, the discussion of possible 

outcomes (discussed in section 2.3), is so 

important. 

Possible Management Goals 

The goals of risk management are varied. They may be risk 

related, aiming to: 

• Reduce or eliminate risks from exposure to hazardous 

substances. 

• Reduce the incidence of an adverse effect. 

• Reduce the rate of habitat loss. 

They may be economic, aiming to: 

•Reduce the risk without causing job loss. 

•Reduce the risk without reducing property values. 

They may involve public values, aiming to: 

• Protect the most sensitive population. 

• Protect children. 

• Preserve a species from extinction. 

Source: Presidential/Congressional Commission, 1997 

2.1.2. Defining the Scope of Analysis and Products Needed 

Scoping a cumulative risk assessment effort involves defining the elements that will or 

will not be included in the risk assessment7

 (USEPA, 1997a). These include the stressors, 

sources, pathways, routes, and populations to be evaluated. As illustrated by the examples in the 

text box (next page), the scope of a cumulative risk assessment may be narrow or broad. Initially, 

7 An assessment which looks at all stressors over a period of time for a specific population would be a “total risk” 

assessment, which is difficult to perform given our current methods. 

17 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

the risk assessment planning team needs to 

select the kind of risk information, exposure 

scenarios and assessment issues that need to 

be covered. These should be directly linked 

to the risk-related questions being asked 

when establishing the purpose. Limitations in 

scope can be geographical (such as political 

or ecological boundaries), environmental 

(such as assessing only certain media), 

demographic (such as assessing only risks to 

children or asthmatics), statutory, or by using 

other criteria such as data limitations. The 

issue of “background” exposures to stressors 

should be discussed and agreements reached 

(see Appendix C). An adequate assessment 

scope should make it clear what’s included 

and what’s excluded from the assessment. 

Care must be taken to reconcile the 

Examples of Different 
 

Cumulative Risk Assessment Scopes
 

• Health risks associated with the aggregate exposure (via 

all pathways and routes) to insecticides acting by a 

common mode of action. 

• Human health risks associated with outdoor inhalation 

exposures of the general population to 33 priority air 

pollutants nationwide. 

• Human health risks associated with exposure via all 

routes to all pollutants present or being released from a 

hazardous waste site. 

• Human health risks, for a specific neighborhood, 

associated with exposure via all routes to all pollutants 

present or being released from a set of adjacent sources, 

including several industries, two hazardous waste sites, 

traffic, and a municipal landfill. 

limitations of scope with the list of questions to be answered in the statement of purpose. If, for 

example, data limitations preclude the addressing of certain of the questions outlined in the 

purpose, the list of questions to be addressed should be modified and the risk assessment 

planning team agree to the narrower scope of the assessment. 

Reasons for choosing the particular scope of the assessment, and how it will address the 

questions posed in the purpose statement, should be stated explicitly. Defining the scope of the 

assessment should include details on the limitations of resources, limitations of data, the impact 

of risk elements on the risk estimate (i.e., some pathways may be seen as having negligible 

impact on the risks related to the questions being addressed), and limitations of the methods 

available. In cases where an element of risk is likely to be important, but no valid data are 

available, the assessor must highlight this deficiency or use judgment or assumed values to 

approximate the missing data. Such judgments and approximations should be clearly 

documented, and explained to the manager in the risk characterization. 

Once the elements (sources, stressors, populations, etc.) have been identified through 

brainstorming with all participants, the participants should discuss the need for and availability of 

technical information and how such information may affect the overall uncertainty of the 

assessment. Using input from the risk assessor, the risk assessment planning team must 

determine what elements will and will not (or, can and cannot) be included in the risk 

assessment. Some of the stakeholder concerns may not be suitable for analysis by risk 

assessment, so other expertise and evaluation may be required to provide this additional analysis. 

Information gathered at this stage is preliminary and may be modified during the analysis phase. 

Identification of potential stressors, populations to be assessed, and potential effects are all part 

of the scoping process, and help define the method of approach. 

18
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

As examples of some of these scoping elements, stressors can include physical (including 

radiological) stressors or chemical or biological agents that may cause an adverse effect. The 

sources of the stressors can be human activities in sectors of society (e.g., manufacturing, 

transportation, agriculture, land development), personal human activities (e.g., smoking, diet, and 

other “lifestyle activities”) or natural phenomena (e.g., forest fires, floods). Stressors that are not 

physical, chemical, or biological, such as economic or other quality-of-life stressors may also be 

identified, but good techniques for including the effect these have on risk currently may not exist. 

Possible population elements to be assessed usually focus on the entities that are at risk, 

e.g., populations, communities, ecosystem functions, or vulnerable subpopulations such as 

persons with certain diseases, or persons at vulnerable life stages, such as children. The more 

specifically these can be defined, the more focused the analysis can be. This will be helpful in 

interpreting the results of the assessment. 

2.1.3. Agreeing on participants, roles and responsibilities 

The risk assessment planning team will usually recommend others who should participate 

in the assessment’s planning, scoping, and risk analysis phase. Depending on the schedule, 

approach, and level of effort envisioned for the risk assessment, there may be no additional 

participants, or there may be many. Assessments will usually require substantial technical 

expertise in the analytic portions of the assessment. Some of the fields of science that may be 

necessary or helpful include toxicology, epidemiology, ecology, risk assessment, exposure 

assessment, fate and transport modeling (e.g., indoor and outdoor air, surface and drinking 

water), computer science (including geographical information systems [GIS]), chemistry, 

biology, various engineering fields (e.g., chemical, mechanical, industrial, civil), economics, 

sociology, and others. 

For the deliberative portions of the 

assessment, there can be a number of 

stakeholders and other interested parties that 

should be considered for participation. The 

box at the right lists some examples to choose 

from among interested or affected parties for 

the deliberative portions of the assessment. 

For community-based assessments, in 

particular, it is important that community 

involvement be sought and encouraged. The 

Presidential/Congressional Commission on 

Risk Assessment and Risk Management 

[hereafter, the “Commission”] (1997) suggests 

the following questions to identify potential 

Examples of Possible Interested or 

Affected Parties (Stakeholders) 

(adapted from USEPA 1999b) 

State governments Affected industry 

Tribal governments Civic organizations 

Local governments Business owners 

Community groups Trade associations 

Grassroots organizations Labor unions 

Environmental groups Public health groups 

Consumer rights groups Academic institutions 

Religious groups Impacted citizens 

Civil rights groups Other federal agencies 

19 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

interested or affected parties (stakeholders): 

• “Who might be affected by the risk management decision? (This includes not only 

groups that already know or believe they are affected, but also groups that may be 

affected but as yet do not know it.) 

• “Who has information and expertise that might be helpful? 

• “Who has been involved in similar risk situations before? 

• “Who has expressed interest in being involved in similar decisions before? 

• “Who might be reasonably angered 

if not included?” 

It has become increasingly recognized 

as important that stakeholders be involved in 

risk assessment (e.g., NRC 1996, 

Presidential/Congressional Commission. . . 

1997, USEPA 1996a, 1997a, 1998a, 1999c, 

1999f, 2000a). The Commission suggested 

guidelines for stakeholder involvement (see 

box at right). 

There are several issues concerning 

the stakeholders’ capacity to participate that 

should not be overlooked by the risk 

assessment planning team. First, some 

stakeholders may need training to be able to 

participate in technical and risk management 

discussions. Second, as noted in the box at 

right, some stakeholders may require 

incentives such as travel funds or lodging at 

sites of meetings outside the area where they 

live. The risk assessment planning team, 

along with the potential source of funds for 

such incentives, should decide to what extent, 

if any, such incentives can be provided, based 

on the scope, level of effort, and financial 

constraints of the risk assessment project. 

Guidelines for Stakeholder Involvement 

• Regulatory agencies or other organizations considering 

stakeholder involvement should be clear about the extent 

to which they are willing or able to respond to stakeholder 

involvement before they undertake such efforts. If a 

decision is not negotiable, don’t waste stakeholders’ time. 

• The goals of stakeholder involvement should be 

clarified at the outset and stakeholders should be involved 

early in the decision-making process. Don’t make saving 

money the sole criterion for success or expect stakeholder 

involvement to end controversy. 

• Stakeholder involvement efforts should attempt to 

engage all potentially affected parties and solicit a diversity 

of perspectives. It may be necessary to provide appropriate 

incentives to encourage stakeholder participation. 

• Stakeholders must be willing to negotiate and should be 

flexible. They must be prepared to listen to and learn from 

diverse viewpoints. Where possible, empower 

stakeholders to make decisions, including providing them 

with the opportunity to obtain technical assistance. 

• Stakeholders should be given credit for their roles in a 

decision, and how stakeholder input was used should be 

explained. If stakeholder suggestions were not used, 

explain why. 

• The nature, extent, and complexity of stakeholder 

involvement should be appropriate to the scope and impact 

of a decision and the potential of the decision to generate 

controversy. 

Source: Presidential/Congressional Commission on Risk 

Assessment and Risk Management, 1997 

Roles and responsibilities for technical and non-technical participants (i.e., ground rules 

20
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

for participants) should also be proposed by the planning team, depending upon the schedule, 

approach, and level of effort that is envisioned for the risk assessment. There will be several key 

points in the risk assessment process where stakeholder input will be critical. Some of these are 

the agreements on purpose, scope, and approach. Each project should define and agree upon a list 

of critical points for stakeholder input. The team may even decide to break stakeholders out into 

several subgroups, with specific tasks such as (1) to understand the technical information and 

report back to the larger group; (2) to elevate and clarify stakeholder issues as needed; or (3) to 

provide information and facts to their peers and the analysts. 

Sometimes citizens choose not to participate because they feel they will not influence the 

outcome, the issue is too complex or technical, the effort is too great, or because the decision 

process is unclear (USEPA, 2001c). Moreover, despite increased emphasis on stakeholder 

participation, there are instances where it may not be appropriate for large scale stakeholder 

involvement. EPA (as the decision maker) must determine whether, and to what degree, 

stakeholder involvement in a cumulative risk decision will be useful and what objectives it may 

accomplish. There is a continuum of objectives that may apply to individual cases, from 

exchanging information on one end, through obtaining stakeholder recommendations, to 

developing agreements for joint activities at the other end (USEPA, 1998g). 

Much of the activities and data needed for cumulative risk assessment overlap the 

jurisdiction of EPA, other public health agencies, and academia. The most successful future 

cumulative risk assessments are likely to be those where cooperation among organizations 

(Federal, State, private, environmental, academic, etc.) leads to use of the best data and tools for 

the various parts of the assessment. 

2.1.4. Agreeing on the Depth of the Assessment and the Analytical Approach 

The analysis approach (discussed further in section 2.2.3 and chapter 3) may fall 

anywhere on a continuum from relatively unsophisticated methods which rely heavily on default 

(and often conservative) assumptions, and consequently have greater uncertainty, to increasingly 

refined assessments in which data are substituted for assumptions and uncertainty is reduced. 

Some of the factors that go into deciding on the approach include the level of uncertainty in the 

risk estimates that is acceptable to the participants, the intended use and audience for the 

assessment, the time and money resources available, and the amount, quality and accessibility of 

data. In making the decision on approach, there will need to be an understanding of both the 

level of effort necessary for conducting the assessment selected, with an insight to alternatives, 

and the features and limitations of the selected approach, in comparison to other approaches. 

2.1.5. Agreement on the Resources Available and Schedule 

Schedule and resources are often interrelated. They may also affect whether the work is 

performed in-house by the organization or team desiring the assessment, or by contractor or other 

external source. The need to meet external deadlines or coordinate with schedules of other 

21
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

organizations may become an overriding factor in defining what will be prepared. Assessments 

requiring short-term, low budget efforts, or preliminary screening assessments, may not have the 

scope, time or resources where extensive stakeholder involvement is necessary or beneficial. For 

assessments, especially those where there is extensive stakeholder involvement, a budget and 

time schedule should be developed and known by all participants. 

2.1.6. Review of Lessons Learned in 

Similar Studies 

Much time and effort can be saved by 

taking the advice of those who have been 

through this process – or similar processes – 

before. Risk assessment reports will often have 

a review chapter of “lessons learned” (or, “if I 

had to do this over again, this time I would . . 

.”). We have tried to include some of the 

discussion of recent Agency experiences as 

examples to illustrate parts of this Framework 

report. In addition, the reader is encouraged to 

find similar advice in other reports (e.g., Lesson 

Learned on Planning and Scoping for 

Environmental Risk Assessments, USEPA, 

2002b). EPA’s Office of Water has conducted 

several watershed studies over the past decade 

and has compiled a web page with lessons 

learned (USEPA, 2001d). One of the lists from 

that source is in the box at right, but there are 

many others. Even though the studies were not 

all cumulative risk studies, much of the wisdom 

gained is relevant. 

2.2. Problem Formulation, Conceptual 

Model, and Analysis Plan 

One outcome of the problem 

formulation phase is a conceptual model that is 

intended to identify relevant stressors, sources, 

pathways, exposure routes, receptors, and 

effects, and to identify relationships among 

them. The conceptual model serves as a basis 

for the analysis plan, which is used to focus the 

analysis phase of the assessment. These three 

components are discussed in the sections 

Reed Holderman's Lessons Learned 

(California Coastal Conservancy, Santa Ynez Watershed) 

1. Be sure that [the project] is needed, and if it is, build 

community support for it before proceeding. 

2. Invite everyone into the process and ask political leaders 

to select the steering committee. Otherwise, people will 

ask, “Who appointed you?” 

3. Don't be presumptuous. On the Santa Ynez River, we 

assumed everybody would appreciate a well thought out 

scope of work, budget, and schedule. Wrong. They said it 

only proved that the whole thing was a set-up. Next time, 

let [the whole planning team] figure it out! 

4. When the majority of stakeholders tell you that they 

want to deal with their issue first, believe them. I remain 

convinced that our failure to sustain interest in the Santa 

Ynez River plan was primarily because we were not 

willing to assist the County in carrying out its proposed 

channel clearing activities in the Lompoc valley as a 

separate and distinct project. 

5. Do whatever you can to break down barriers and 

perceptions people have of each other. Be creative. 

Family BBQs, softball games, and parties have done 

wonders to improve relationships among stakeholders and 

build trust. 

6. Maintain constant communication among stakeholders 

throughout the process – and especially in the beginning – 

to pass information along, answer questions, or deal with 

rumors. Whether it's through regular meetings, 

newsletters, web sites, phone trees, or all four, good 

communication is a must. 

7. And finally, line up your money and in-kind services in 

advance of starting your [assessment] project, or else two 

bad things will happen: (a) your stakeholders will buy into 

a process and scope of work only to find out they can't 

afford it; and (b) you will spend more time looking for 

cash than participating in the planning process. Either 

way, you lose. 

[Source: Lessons Learned Web page (USEPA, 2001d)] 

22
 
1

2

3

4

5

6

7

8

9

10

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

below. 

The Science Advisory Board in their report Toward Integrated Environmental DecisionMaking (USEPA, 2000a) suggests a list of desired outputs from the Problem Formulation part of 

an environmental decision-making exercise. Although this is not precisely the same as a risk 

assessment, many of the points they list have applicability to risk assessment, also. The SAB 

suggests these should not only be left to the visual presentation of the Conceptual Model 

Diagram, but should also be explained in narrative form. Some of the SAB’s recommended 

outputs, included here as an example, are listed in the box below. Not all of these would 

necessarily be applicable to a given risk assessment, depending on the scope. 

Example: SAB’s Desired Outputs for Problem Formulation 

! The initial goals for the decision-making exercise, including environmental goals to be achieved
 

! Which environmental problems/stressors/systems will be included and which will not, and the reasons for these decisions
 

! The health, ecological, and quality-of-life effects of concern
 

! The spatial, temporal, and organizational dimensions of the problem
 

! Relevant data and models, and possible approaches to data analysis
 

! Scoping of the uncertainties involved and research needed to significantly reduce critical uncertainties
 

! Initial review of the range of options available to reduce risks, considering likely economic, political, or other constraints
 

! The endpoints upon which the condition of the ecological, human health, or societal systems ultimately will be judged
 

! The types of factors that will be considered when reaching a decision
 

From Toward Integrated Environmental Decision-Making (USEPA, 2000a) 

11
 

12 2.2.1. Problem Formulation.
 

13
 

14 Problem formulation is a systematic planning step that identifies the major factors to be
 

15 considered in a particular assessment. It is linked to the regulatory and policy context of the
 

16 assessment. Problem formulation is an iterative process within which the risk assessor develops
 

17 preliminary hypotheses about why adverse effects might occur or have occurred. It provides the
 

18 foundation for the technical approach of the assessment. The outcome of the problem
 

19 formulation process is a conceptual model that describes the relationship between the stressors,
 

20 the population exposed, and the assessment endpoints that will be addressed in the risk
 

21 assessment.
 

22
 

23 2.2.2. Developing the Conceptual Model
 

24
 

25 A conceptual model includes both a written description and a visual representation of
 

26 actual or predicted relationships between humans (or populations, population segments) or
 

27 ecological entities and the chemicals or other stressors to which they may be exposed. 
 

23
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Figure 2-2. An example of a generic conceptual model (adapted from USEPA, 2002a). 

Conceptual models represent many relationships, and may describe primary, secondary, 

or tertiary exposure pathways. The model is developed by the risk assessor and may include input 

from other experts (including stakeholders). The model narrative needs to distinguish – to the 

extent possible – between what is known or determined, and what is assumed. Also, it needs to 

include a discussion of uncertainties in the formulation of the assessment and state how the 

assessment is cumulative, i.e., for which sources, stressors/agents, pathways/exposure routes, 

receptors/populations, and endpoints. In some cases, conceptual models will be submitted for 

peer review. A general conceptual model is illustrated in Figure 2-2. The conceptual model 

includes factors and endpoints which may not be analyzed in the risk assessment, but may be 

evaluated in the overall decision-making process. 

The conceptual model and the associated narrative show the basic rationale for the 

decisions made in pursuing a particular course of action in a cumulative risk assessment. It 

provides a record of decisions for future reference during risk analysis, characterization, and 

communication of the risk management decision. It is also valuable as a risk communication tool 

both internally within the Agency and externally in interactions with the public. The 

24
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Figure 2-3. Specific conceptual model for a complex project, OAQPS’ National Scale Air Toxics Assessment 

25
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

conceptual model provides a scientific or technical work product that includes: (1) the scientific 

rationale for the selection of the stressors, sources, receptors, exposed populations, exposure or 

environmental pathways, endpoints or effects, (2) the scientific, technical, economic, or 

sociologic basis for the construction of the conceptual model; and (3) the scientific implications 

of additional data gathering. Figure 2-3 is an example of a conceptual model from the National 

Air Toxics Assessment8

. 

It is not inconceivable, given the deliberative nature of the process of developing a 

conceptual model, that more than one model will be considered as alternatives. If the team 

decides to ultimately use more than one model, and to evaluate each as part of hypothesis testing, 

a careful consideration of time and monetary resources needs to be made, as well as a very 

careful consideration of how the results will be interpreted (see section 2.3). 

2.2.3. Constructing the Analysis Plan 

The analysis plan is the final stage of planning and scoping before the risk assessment. 

The analysis plan is discussed in the Ecological Risk Assessment Guidelines (USEPA, 1998b), 

Section 3.5. The analysis plan describes how hypotheses about the relationships among the 

sources, stressors, exposure conditions, populations, and adverse effects, presented in the 

conceptual model and narrative, will be considered during the risk analysis phase of the 

assessment. The plan includes the rationale for which relationships (referred to as “risk 

hypotheses” in the Guidelines for Ecological Risk Assessment) are addressed, methods, models, 

and a discussion of data gaps and uncertainties. It also may include a comparison between the 

level of confidence needed for the management decision with that expected from alternative 

analyses in order to determine data needs and evaluate which analytical approach is best. In 

some cases, a phased, or tiered, risk assessment approach can facilitate management decisions, 

particularly in cases involving minimal data sets. 

The analysis plan provides a synopsis of measures that will be used to evaluate risk 

hypotheses (as shown in Appendix D) . The plan is strongest when it contains explicit 

statements for how measures were selected, what adverse effect (or assessment endpoint) they 

are intended to evaluate, and which analyses they support. Uncertainties associated with selected 

measures and analyses and plans for addressing them should be included in the plan when 

possible. The analysis plan can be a brief summary of what the key components of the risk 

assessment are and how each component will be measured or calculated. 

As in the conceptual model, the economic or societal importance, complexity, data and 

resources available will determine the degree of sophistication and detail needed in the analysis 

plan. Key data gaps should be identified. It should also include thoughts about how to fill the 

information needs in the near-term using existing information, in the mid-term by conducting 

8 NATA is the technical support component of EPA’s National Air Toxics Program [see 64FR38706-38740 (“National 

Air Toxics Program: Integrated Urban Strategy”) or USEPA, 2001e. 

26 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

tests with currently available test methods to provide data on the agent(s) of interest, and over the 

long-term to develop better, more realistic understandings of exposure and effects and more 

realistic test methods to evaluate agents of concern. The plan should explain how measures were 

selected, what they are intended to evaluate, and which analyses they support. Uncertainties 

associated with selected measures and analyses, and plans for addressing them, should also be 

explicitly stated. 

The analysis plan should include (where feasible) milestones for completion of the risk 

assessment. The plan may be revisited and revised periodically. Such revisions may be 

anticipated, if new information is acquired, to refine hypotheses of exposure and toxicity, to 

modify the risk hypotheses addressed, or to compare public concerns with the projected risk 

management options. 

2.3. The Final Step Before the Analysis Phase: Discussion of Possible Outcomes 

It is useful for the entire team to hold some preliminary discussions, before the analytical 

efforts of the assessment are started, about the various possibilities of the cumulative risk 

assessment results and their implications. Given that statutory mandates, regulations, property 

rights, or due process may constrain or define most or all acceptability criteria, what conclusions 

of the team will be associated with various results or risk levels? For example, for a risk 

assessment team with members from the community, industry, and the local and other 

government entities, what would happen if the assessment shows risk levels to be “low”? Would 

members accept this? Conversely, if “unacceptable” risks are determined, will all team members 

accept the results and their possible responsibility to do something about that risk? Do team 

members understand the limitations of the information to be generated? 

Discussions like these will help determine if the assessment can really address the 

questions of the team. If not, the assessment may not be worth doing as planned. If members of 

the team will not accept the possibility of a range of results of the analysis, then it is important to 

reopen the entire planning and scoping discussion before anything is done in the analysis phase, 

since the planning and scoping phase has not been satisfactorily completed. Although it is not 

necessary to have unanimity among stakeholders on the plan before proceeding, knowing where 

some of the potential disagreements may occur after the Analysis and Interpretation phases are 

started allows the stakeholders as a group to plan beforehand for how such disagreements will be 

addressed, should they occur. 

As an example, the Baltimore Community Environmental Partnership Air Committee 

Technical Report (USEPA, 2000f) is a case study where the stakeholders thought they had 

agreement on roles, responsibilities and approach, only to find that the group acrimoniously 

splintered after the analysis results came back. The Baltimore report contains valuable lessons 

learned in the area of stakeholder disagreements and agendas, and can provide some insight for 

planning teams. 

27
 
1

2

3

4

5

6

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Finally, discussions just prior to the analysis phase may lead to an assessment very 

different from the one originally envisioned. The CRI case study (box, next page) is one where 

the original plan was to do a quantitative cumulative risk assessment, but because of the lack of 

some critical information, the scope was changed. This led to an assessment that, while not as 

broad as originally planned – and not even directly calculating risk – had better stakeholder buyin with a better chance of success of providing useful information. 

28
 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Example: Cumulative Risk Initiative (CRI) for Cook Co., IL and Lake Co., IN 

(formerly Chicago Cumulative Risk Initiative, CCRI) 

CRI BACKGROUND AND O VERVIEW 

In 1995 the Chicago Legal Clinic and 11 Chicago-area community advocacy groups filed a petition under the Toxic 

Substances Control Act (TSCA) requesting that the USEPA Administrator prohibit or further regulate the emissions from 

eight proposed or constructed incinerators in the Chicago metropolitan area and Northwest Indiana. The petitioners 

believed that neither current statutes nor local siting laws adequately address cumulative impacts of multiple sources of 

toxic pollutants in a geographic area. They requested that the Administrator restrict emissions of dioxins, furans, mercury, 

lead and cadmium from these sources. In May 1996 the petition was withdrawn in response to a USEPA offer to 

participate in an investigation of multimedia pollutant impacts in Cook County, Illinois and Lake County, Indiana. This 

effort became CRI. CRI is an attempt to investigate cumulative loadings and hazards from pollutant sources, develop 

community-based activities to help address these concerns, and use analytic results to help prioritize use of regulatory 

agency resources. USEPA and the petitioners agreed to a four phase project: (1) Environmental Loadings Profile (EPA 

747-R-1-002); (2) Petitioner Risk Workshop (completed); (3) Hazard Screening Assessment (peer review draft available 

Jan. 2002); and (4) Risk-Hazard Management Response (pending). 

HAZARD SCREENING ASSESSMENT 

The CRI Hazard Screening Assessment was authored primarily by Argonne National Laboratory with input from local, 

state and federal participants. Reflecting stakeholder deliberations, the Report focuses on cumulative hazard (not “risk” as 

typically defined by USEPA) associated with noncriteria air pollutants (“air toxics”) in the two county study area. It relies 

on “off-the-shelf” air pollutant information, including USEPA’s Toxics Release Inventory, Cumulative Exposure Project, 

Regional Air Pollutant Inventory Development System, and outdoor air monitoring data. Emission estimates are “toxicity 

weighted”, while modeled/monitored outdoor air pollutant concentrations are compared with reference values to develop 

hazard index-like ratios. The ratios or toxicity weighted emission estimates are used to derive indicators of cumulative 

hazard, then mapped over study area locations. To identify geographic areas where potentially elevated hazards and 

individuals with potentially greater susceptibility are collocated, another part of the study assembles pollutant hazard 

information and data on existing human disease rates and indicators. 

PRELIMINARY LESSONS LEARNED 

1. A major planning/scoping/problem formulation effort by a broad group of stakeholders narrowed the scope of the CRI 

Hazard Screening Assessment and seemed to increase stakeholder “buy-in” with the process. This was valuable given the 

complexity, expense, effort, time requirement and difficulty encountered in addressing even the narrowed scope. 

2. Large data gaps make risk and hazard assessment of environmentally-relevant chemical exposures highly uncertain, even 

for single agents. Expanded assessments that address cumulative risk considerations (e.g. mixtures; developmental toxicity; 

non-chemical agents) are a better match for real-world circumstances but require acknowledgment of even more 

uncertainty. 

3. Obtaining and managing input from a large group of technical stakeholders is cumbersome and time-consuming, but that 

group’s perspective and expertise greatly improved the CRI assessment. 

4. Given that the NRC’s 1983 four-step “framework” required several years for broad use and acceptance in the U.S., the 

greater complexity of cumulative risk (for CRI, cumulative hazard) assessment suggests an equally long period may be 

needed for terminology standardization, refinement of approaches and development of consensus methods. 

1
 

29
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Figure 3.1. The Analysis Phase 

3. THE ANALYSIS PHASE 

The Analysis Phase is primarily an analytic process where risk experts apply risk 

assessment approaches to evaluating the problem at hand9

. The risk assessment paradigm most 

widely used by risk assessors during the past two decades was first documented by the National 

Research Council (NRC, 1983). It consists of four parts: hazard identification, dose-response 

assessment, exposure assessment, and risk characterization. This paradigm was developed when 

almost all risk assessments were being conducted on single chemicals. Nevertheless, it is a useful 

place to start when considering cumulative risks. As a prerequisite, assessors considering 

cumulative risk assessments should be familiar with the 1983 NRC risk paradigm, as well as the 

various EPA risk assessment guidelines (see text box in section 1.1). 

9 Although the Analysis Phase is primarily an analytic process with heavy emphasis on the role of the scientist, risk 

assessor, or other technical expert, other stakeholders can be involved in various ways as agreed upon before the Analysis Phase 

begins. Some roles stakeholders might have in the Analysis Phase include (1) suggesting sources of data, or providing data for 

the assessment; (2) helping clarify issues identified during Problem Formulation; (3) working alongside the risk assessment 

experts to see what data and assumptions are being used and why, and to better understand how the risk assessment process 

works; and (4) suggesting alternate scenarios that may reflect more realistic exposure conditions in the community. A variety of 

roles for stakeholders in the Analysis Phase can be proposed and adapted for the particular circumstances of the individual case, 

assuming that the roles can be agreed upon by the team. 

30 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

In both single and multiple stressor risk assessments, the analyst will look at hazard and 

dose response relevant to the stressor(s) of interest, and perform an analysis of exposure(s) to 

those stressor(s). This chapter begins with a basic discussion of this general process and its basic 

ingredients (section 3.1). The second part of this chapter (section 3.2) discusses some of the 

situations arising in cumulative risk assessment, methods currently available for addressing them, 

steps in the process, and some limitations to these methods. Finally, section 3.3 identifies areas 

of ongoing work particularly relevant to cumulative risk assessment. 

3.1. General Process 

In developing the conceptual model and analysis plan (see section 2.2), the scope of the 

assessment was specified (see example in box at right). Some of the aspects of scope include 

stressors, sources, pathways and media, exposure routes, populations and subpopulations, 

endpoints, and measures. 

The analysis plan should specify how data, modeling or assumptions will be obtained, 

performed or defined for all of the details 

concerning the characterization of exposure 

for the defined set of stressors, to the 

defined population and subpopulations. 

Additionally, the analysis plan specifies the 

strategy for obtaining and considering 

hazard and dose-response information for 

these stressors. And, the plan will specify 

the method for combining the exposure 

information with the hazard and doseresponse information to generate risk 

estimates or measures. As the risk analysis 

is refined, it may be appropriate to revisit 

and refine the exposure, hazard and doseresponse information in an iterative 

fashion. 

In the integration of exposure, 

hazard and dose-response information for a 

cumulative risk assessment, several aspects 

of the assessment may be particularly 

important. These include multiple stressor 

hazard, dose-response and exposure issues, 

exposure time or duration related issues, 

vulnerability or susceptibility of the study 

Example: Scope of EPA’s national scale assessment 

for hazardous air pollutants (also see Figure 2-3): 

stressors 33 priority urban HAPs 

sources	 major industrial, small “area”, 

mobile (on- and off-road), & extrinsic

“background” in air 

pathways/media	 outdoor air, indoor air 

microenvironments 

routes inhalation 

subpopulations general population only 

endpoints	 cancers, developmental, CNS, kidney,

liver, respiratory effects 

metrics	 for cancer: distribution of high-end

cancer risk estimates, predicted

percent of population within predicted

cancer risk ranges, predicted number

of cancer cases, HAP-specific and 

cumulative 

for other effects: distribution of 

estimated hazard index values and 

estimated percent of population within

specified ranges of index values 

population, along with the influencing factors, and subpopulations with special exposures. These 

items are discussed in the following section, along with the currently recognized methods for 

evaluating the toxicity or risk associated with mixtures. 

The area of identifying and assessing risk to susceptible subpopulations has an increased 

31
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

profile in cumulative risk assessments. A variety of factors may be influential in affecting 

population susceptibility. The extent to which these can be considered will be heavily dependent 

on existing knowledge and available information. Section 3.2 discusses available methods for 

identifying and estimating risk or hazard to susceptible or vulnerable subpopulations. Section 

3.3 discusses areas of complexity and on-going work. 

3.2. Available Methods and Approaches 

There are many aspects of traditional risk assessment methodology which apply to 

cumulative risk assessment. Predicting cumulative risk of multiple stressors, however, has 

required the development of additional specific methods or approaches. Additionally, there are 

some aspects of risk assessment, while common to both single-stressor and multiple-stressor 

assessments, that may increase in complexity or significance, in a cumulative risk assessment. 

Together they frame the methodological issues pertinent to the discussion of cumulative risk 

assessment. 

While these aspects common to single-stressor assessment may be many (e.g., the added 

dimension of multiple stressors influences consideration of stressor sources, routes of exposure, 

environmental media/pathways, and other factors), several examples are raised here. As one 

example, the assessment of the dose-response relationship and corresponding characterization of 

exposures in terms of duration, timing relevant to life stage and exposure history gains an 

additional dimension with the need to consider this in some way cumulatively. The 

consideration of population susceptibility or vulnerability, as recommended in the Agency’s 

policy and guidance on Risk Characterization (USEPA 1995a, 1995b, 2000c), also increases in 

complexity. A third example of a complicating aspect in cumulative risk assessment is the 

consideration of subpopulations with particularly distinctive exposures. These examples are 

further discussed in section 3.2.1. 

Although it is beyond the scope of this Framework report to describe all risk methods in 

detail, Appendix B lists a variety of resources relevant to various exposure assessment methods. 

Relatively speaking, there is a great deal of information on assessing human and environmental 

exposures to chemical stressors, some information on biological and radiological stressors, but 

relatively little information on many other types of stressors. 

The most prominent aspect of cumulative risk assessment is often the prediction of the 

combined effects of multiple stressors. Past and current activities in the development of 

approaches for predicting risk of multiple stressors include the Agency’s Guidelines for the 

Health Risk Assessment of Chemical Mixtures (USEPA, 1986b) and Supplementary Guidance for 

Conducting Health Risk Assessment of Chemical Mixtures (USEPA, 2000e). Concepts, 

approaches, or methods described in these documents or elsewhere are discussed in section 3.2.2, 

with clarification of their applicability, limitations and notable points regarding interpretation of 

the results they produce. 

3.2.1. Examples of Increased Complexity of Cumulative Risk Assessment. 

Three examples of the potential for increased complexity of cumulative risk assessment 

32
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

compared to single stressor risk assessment are described here, and related to: 1) time related 

aspects, 2) vulnerability or susceptibility, and 3) subpopulations with special or particularly 

distinctive exposures. All three of these aspects are relevant in single stressor assessments, but 

have the potential to be more complicated in multiple stressor assessments. 

Time related aspects. The issue of repeated exposures to a single stressor or exposures 

to multiple stressors that may vary in time dimensions may have implications with regard to 

susceptibility, which, consequently, has implications regarding the dose-response relationship. 

Traditionally in dose-response assessment, for many stressors and effects there is an inherent 

presumption that it is the cumulative exposure (combination of intensity and duration) to which 

the organism responds. Thus dose-response assessments based on one pattern of exposure (e.g., 

6 hours per day, 5 days per week over a lifetime) are routinely applied to the assessment of risk 

associated with a variety of patterns of exposure. 

In the case of linear carcinogens, this cumulative exposure assumption has been carried as 

an explicit assumption in the risk assessment step. Regardless of the details of the exposure 

circumstances for the study on which the cancer potency was based, it is assumed that there is a 

linear relationship between amounts of exposure and associated cancer risk. For non-linear 

carcinogens 10, and conceivably for linear carcinogens, if data indicate deviation from the 

assumption that cancer risk is proportional to lifetime dose, the details and sequence of exposure 

may be important, both in developing the dose-response relationship and in predicting risk 

associated with exposures of interest. 

As some chemicals may have the ability to affect an organism’s response to other 

chemicals, consideration of the time sequence of exposure may take on an additional layer of 

complexity in multiple chemical cumulative risk assessments. For example, persons with 

relevant past exposures might have increased susceptibility to the effects of a particular chemical 

due to a previous exposure to the same – or a second – chemical. 

These considerations suggest that for cumulative risk assessment, chemical exposures 

need to be characterized in terms of which other chemicals are present, and when. As noted in the 

ILSI Framework for Cumulative Risk Assessment (ILSI, 1999): “Data collected specifically to 

support a cumulative exposure assessment should conserve the covariance and dependency 

structures associated with the chemicals of concern.” It is important to note, however, that the 

detail to which exposures are characterized should be closely tied to the detail of information 

available in the dose-response assessment, since a lack of corresponding detail in the doseresponse assessment can pose a limitation on the interpretation and usefulness of detailed 

exposure estimates. 

Cumulative risk assessment can present challenges in matching exposure estimates with 

dose-response relationships. Ideally, the dose-response assessment will indicate if the time 

sequence for the chemical(s) or stressors of interest in the assessment is important for risk 

estimation. In cumulative assessments involving chemicals where time sequence of exposure is 

important, it may be necessary to characterize the details and sequence of exposure to the 

10 The draft cancer guidelines (USEPA, 1999l) explicitly recognize the potential for non-linear dose response. It is 

only in the case where non-linear response is modeled that time sequence of exposure can be considered in the risk assessment. 

33 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1 exposed population (see text box, previous page), so there will be a match in not only the form, 

but also the assumptions between the dose-response relationship and the exposure/dose estimate. 

3 

2 

Some Examples of Exposure Models which Consider Time Aspects 

Calendex (Novigen Sciences, Inc), integrates different pathways (e.g., dietary – food and water – and 

residential) and routes (oral, dermal, inhalation) of exposure using a calendar-based probabilistic approach . 

One of the important factors of this approach is it provides estimates of risk which reflect aggregate and 

cumulative exposure to discrete individuals with exposure pathways and routes appropriately linked for the 

scenarios being assessed. Calendex also allows one to estimate exposure pre- and post use of a chemical, as 

well as during degradation periods. Calendar based assessments maintain the integrity of the individual by 

capturing: the location of the exposed individual, the time of year in which he or she was exposed, and the 

patterns of ex posure. C alendex also allows for a va riety of time-brea kout optio ns for the analysis o f exposure . 

APEX - The Air P ollution Exp osure (AP EX) m odel is base d on the pN EM p robabilistic N ational Am bient Air 

Quality Stand ards mod el (pNE M) for ca rbon mo noxide (J ohnson, et al., 2000). T his model m imics the basic 

abilities of the pNEM /CO model; it calculates the distributions of human exposure to selected airborne 

pollutants within a selected study area as a function of time. As a dose model (for CO), it calculates the 

pollutant dose within the body, specifically summarized by the blood carboxyhemoglobin (COHb) 

concentration. APEX is a cohort-m icroenviro nmen t exposure model in tha t it combines d aily activity diaries to 

form a composite year-long activity pattern, which represent specific popula tion coho rts and are tracked as they 

move from one microenvironment to another. A cohort consists of a sub set of the pop ulation that is exp ected to 

have som ewhat similar ac tivity (and hence exposure ) patterns; they ar e formed by comb ining demo graphic 

groups and geographic locations (districts). Once each cohort has been modeled and its relative size determined, 

an exposure distribution for the entire population can be assembled. A microenvironment is a description of 

the immediate surroundings of an individual that serves as an indicator of exposure (e.g., inside a residence, 

school or car, outdo ors, etc.). AP EX has b een deve loped as o ne of the inhala tion expos ure mod els accessible in 

the Exposure Event Module of the Total Risk Integrated Methodology (TRIM.Expo) for assessment of 

exposures to either criteria or hazardous air pollutants (USEPA, 1999j) 

Other models include the LifeLine Model, developed under a cooperative agreement between EPA/O PP and 

Hampshire Research Institute (Hampshire Research Institute, 1999, 2000); the Stochastic Human Exposure and 

Dose Simulation Model (SHE DS), under development by EPA’s Office of Research and Developm ent 

(Zartarian, et al., 2000), and the Cumulative and Aggregate Risk Evaluation System (CARES), under 

development by member companies of the American Crop P rotection Association (APCA, 1999) along with the 

Residential Exposure Year (RExY) model being developed by Infoscientific.com. 

4 Vulnerability. One of the concepts that can be used in risk assessments (both for human
 

5 health and ecological assessments) is that of vulnerability of the population or ecosystem. 
 

6 Vulnerability has been a common topic in socioeconomic and environmental studies. The
 

7 European Commission’s TEMRAP (The European Multi-Hazard Risk Assessment Project),
 

8 studying vulnerability to natural disasters such as floods, windstorms, fires, earthquakes, and
 

9 others, defines “vulnerability” as “the intrinsic predisposition of an exposed element [organism,
 

10 population, or ecologically valuable entity] to be at risk of suffering losses (life, health, cultural
 

11 or economic) upon the occurrence of an event of [a specific] intensity” (European Commission,
 

12 2000, bracketed material added). 
 

13
 

14 Vulnerability of a population places them at increased risk of adverse effect, and may be
 

15 an important factor in deciding which stressors are important in doing a cumulative risk
 

16 assessment. The Agency’s risk characterization policy and guidance (USEPA, 2000c) touches on
 

34
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

this concept by recommending that risk assessments “address or provide descriptions of [risk to] 

... important subgroups of the population, such as highly exposed or highly susceptible groups”. 

Further, the Agency’s guidance on planning and scoping for cumulative risk assessments 

(USEPA, 1995b) recognizes the importance of “defining the characteristics of the population at 

risk, which include individuals or sensitive subgroups which may be highly susceptible to risks 

from stressors or groups of stressors due to their age, gender, disease history, size or 

developmental stage”. That guidance also recognizes the potential importance of other social, 

economic, behavioral or psychological stressors that may contribute to adverse health effects 

(e.g., existing health condition, anxiety, nutritional status, crime and congestion). These same 

concepts may also be discussed as a group in terms of “population vulnerability.” The various 

ways in which a population may be vulnerable are discussed below in four categories: 

susceptibility, differential exposure, differential preparedness, and differential ability to recover. 

The first of these is susceptibility. Susceptible individuals within a population have a 

different or more pronounced dose-response relationship when confronted with a stressor. 

Reasons for susceptibility may be related to any number of sensitivity factors, including life stage 

(e.g., children or the elderly may be more susceptible), genetic polymorphisms (e.g., genetic 

susceptibilities which occur in a small but significant percentage of the population), or existing 

disease state (e.g., asthmatics). In addition, susceptibility may be related to the conditions of 

exposures (e.g., prior exposures leading to the development of sensitization reactions, or having 

had exposures which compromise the immune system). Confronted with equal concentrations of 

a chemical for equal durations, for example, a biologically susceptible individual may show 

effects while the typical individual within the population would not. Although we generally do 

not have a lot of data available on this topic, susceptibilities or sensitivities may also exist among 

races or genders. 

The second category of vulnerability is differential exposure. While it is obvious by 

examining a dose-response curve that two individuals at different exposure levels may have a 

different likelihood of effects, this also extends to differences in historical exposure, body 

burden, and background exposure, which are sometimes overlooked in an assessment. 

The third category of vulnerability is differential preparedness to withstand the insult of 

the stressor, and the fourth is the differential ability to recover from the effects of the stressor. 

These last two are linked to what kind of coping systems and resources an individual, population, 

or community has. Preparedness or recovery is often a crucial factor in ecological assessments. In 

human health assessments, lack of access to health care, income differences, unemployment, or 

lack of insurance, for example, may affect a community’s ability to prepare for or recover from a 

stressor. One aspect of differential ability to recover is illustrated by differing survival rates for 

the same disease (e.g., Lantz, et. al 1998). 

Cumulative risk assessments may be uniquely suited to addressing the issues related to 

vulnerability. In order to do that, however, there needs to be some relationship between the 

factors discussed above and changes in risk. At the current state of the science, many of these 

factors have not been extensively developed beyond correlations between mortality rates and 

several socioeconomic factors such as income (e.g., Lynch, et al. 1998). Susceptibility has had 

much more development than the other factors, and current approaches implemented by EPA and 

others to address risk of noncancer endpoints routinely employ a 10-fold factor to address 

35
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

heterogeneity in sensitivity. Variability with regard to susceptibility was discussed in detail by 

NRC (1994), and the current state of knowledge concerning epidemiologically based (e.g., 

oncogene-specific) risk factors provides empirical data upon which at least crude estimates of the 

magnitude of heterogeneity in susceptibility to toxic response can be based. Much research in 

this area, however, remains to be done. 

Subpopulations with Special Exposures. Certain subpopulations can be highly exposed 

to stressors based on geographic proximity to sources of these stressors, coincident direct or 

indirect occupational exposures, their activity patterns, or a combination of these factors. The 

Agency’s Risk Characterization policy and guidance (USEPA, 2000c) includes recognition of the 

need for risk information to include as available, information on highly-exposed subgroups. 

Accordingly, risk assessments, including those that are cumulative, may need to include special 

emphasis on identifying and evaluating these subpopulations. 

Subpopulations at risk of high exposure due to geographic proximity could include 

workers at a facility which is a source of a stressor or residents near such sources. Specific 

examples might be people living downwind from a coal burning power plant, those near and 

using a polluted water body (for example, for fishing or recreation), or along roadways with high 

levels of vehicular traffic. Occupational exposures may be either direct (occurring in the 

workplace) or indirect (occurring at home). Indirect occupational exposures include those 

experienced by family members of those occupationally exposed, who may be exposed to 

occupational chemicals brought into the house by the worker (e.g., on clothing). Thus, workers 

or family members may be subject to greater exposures than others in the population without this 

additional burden. 

Examples of subpopulations at high exposure due to activity patterns may include people 

who exercise heavily in polluted air, recreational or subsistence fishers or hunters who consume 

large quantities of fish or game, farmers or others who get a large percentage of their food from a 

location near a source of pollution and live in areas with high pesticide use, individuals with long 

commutes in automobiles, or children (because they consume a larger amount of food, drink, and 

air relative to their body weight, and because of additional exposure routes such as incidental soil 

ingestion). Additionally, some subpopulations may be affected by the combined impact of high 

geographic exposure and high exposure activity patterns (e.g., runners who run along heavily 

traveled roadways, and those who fish for food in heavily polluted urban rivers). 

It is important to recognize that some heavily exposed populations may also be 

particularly vulnerable or susceptible to the effects associated with the stressors of concern. 

Examples of those who could be particularly vulnerable to certain stressors include children 

during certain stages of development, people with chronic respiratory problems, the elderly, and 

those economically disadvantaged without access to medical care. A cumulative risk assessment 

may need to take into account potential combinations of high exposure and high vulnerability, 

but few, if any, methods are available and accepted today to address the combined effects of 

exposure and vulnerability. This is an important area for further research and methods 

development. 

3.2.2. Approaches for Predicting Risk of Multiple Stressors. 

36
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Combination toxicology (Carpy, et al., 2000) is the study of the toxicity of mixtures. In 

such studies, one may either measure the mixture toxicity directly (whole mixture toxicity), or 

one may develop an estimate of the combined toxicity from information on the multiple 

component stressors acting in concert with each other. If evaluated using its component 

chemicals, the mixture toxicity data set should only be treated as a snapshot of a 

multidimensional dose-response relationship, because the joint toxicity and interactions can 

change with changes in exposure route, duration, relative proportions of the components, or the 

effect being tracked. The application of such a data set to a specific situation then requires careful 

matching of the test mixture composition and exposure conditions to those of the target situation. 

In whole mixture toxicity, once the mixture toxicity is known, a risk evaluation can be done on 

the mixture using the 1983 NRC risk assessment paradigm. On the other hand, component based 

mixture assessments are rarely evaluated using the strict NRC paradigm, because the exposure 

and toxicity information must be compatible, requiring some iteration to obtain toxicity 

information that is relevant to the actual exposure estimates (USEPA, 2000e). 

To address concerns over health risks from multi-chemical exposures, EPA issued 

Guidelines for Health Risk from Exposure to Chemical Mixtures in 1986 (USEPA, 1986b). 

Those Guidelines described broad concepts related to mixtures exposure and toxicity and 

included few specific procedures. In 1989, EPA published guidance for the Superfund program 

on hazardous waste that gave practical steps for conducting a mixtures risk assessment (USEPA, 

1989a). Also in 1989, EPA published the revised document on the use of Toxicity Equivalence 

Factors for characterizing health risks of the class of toxicologically similar chemicals that 

included the dibenzodioxins and dibenzofurans (USEPA, 1989b). In 1990, EPA published a 

Technical Support Document to provide more detailed information on toxicity of whole mixtures 

and on toxicologic interactions (e.g., synergism) between chemicals in a two-chemical mixture 

(USEPA, 1990a). Whole mixture assessments, toxicologic independence and similarity, and risk 

methods using toxicologic interactions are discussed at length in the recent Supplementary 

Guidance for Conducting Health Risk Assessment of Chemical Mixtures (USEPA, 2000e). 

Risk assessment on mixtures usually involves substantial uncertainty. If the mixture is 

treated as a single complex substance, these uncertainties range from inexact descriptions of 

exposure to inadequate toxicity information. When viewed as a collection of a few component 

chemicals, the uncertainties also include the generally poor understanding of the magnitude and 

nature of toxicologic interactions, especially those interactions involving three or more 

chemicals. Because of these uncertainties, the assessment of health risk from chemical mixtures 

should include a thorough discussion of all assumptions and the identification when possible of 

the major sources of uncertainty. 

3.2.2.1. Single Stressor Information. 

Assessments which evaluate the risk from a single stressor do not fall into the category of 

cumulative risk assessments by the definition given in Section 1.3, whether these single-stressor 

assessments address a single (dominant) endpoint or multiple endpoints, or whether the 

exposures are simple or complex (e.g., multi-source, multi-pathway, multi-route exposure). Some 

of them may be termed “aggregate risk assessments” by extension of the FQPA terminology. 

They can, however, provide useful information for cumulative assessments. 

37
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

A cumulative risk assessment considers the joint impact of multiple stressors. Studies on 

individual stressors can, however, provide informative qualitative information for multi-stressor 

assessments, particularly regarding hazard identification. The collection of single stressor effects 

can indicate the variety of types of adverse effects likely to result from the stressor combination, 

though perhaps not the magnitude or extent of the effects. Factors affecting population 

susceptibility to the individual chemicals are also likely to be important with the combined 

exposure. To go further in terms of quantitative risk assessment requires consideration of the 

potential for joint toxicity. For most exposure situations, hazard and dose-response studies of all 

of the joint effects from the multiple stressors will not be available, so that conclusions will have 

to be based at least partly on the single stressor information. 

Exposure assessments for single stressors also need further consideration before they can 

be used to characterize long term cumulative exposure to all the stressors by all pathways. 

Transport and environmental transformation of a chemical can be influenced by presence of other 

chemicals. Consequently, both the exposure levels and the relative proportions of chemicals at 

future times may not correspond well to present measurements of a combination of chemicals 

unless these influences are taken into account. In addition, exposure to one stressor may 

influence the uptake of a second stressor. For example, a nonchemical stressor that increases 

ventilation rate will increase the inhalation uptake of airborne chemicals. 

Toxicologic independence. Two situations allow plausible approximations of the joint 

exposure-response relationship using only the single stressor information: toxicologic 

independence and toxicologic similarity (USEPA 2000e). In the case of toxicologic 

independence, if the toxicity modes of action are biologically independent, then as long as there 

are no pre-toxicity interactions (e.g., metabolic inhibition, influence on uptake), the single 

stressor information is sufficient to approximate the joint exposure-response relationship. When 

the effects from two or more stressors are different, the cumulative response, if toxicologically 

independent, is merely all the single stressor responses, as if the other stressors were not present. 

For example, joint but low exposure to heat (causing minor elevated heart rate) and toluene 

(causing minor hearing loss) would be expected to cause both the minor heart rate elevation and 

minor hearing loss, but to the same extent as expected for each stressor alone. If each stressor is 

below its toxicity threshold, then for stressors exhibiting toxicologic independence, there will be 

no estimated cumulative response, because the set of individual responses is then a collection of 

zeros. 

When the single stressor and cumulative toxicities are each represented by a frequency or 

probability for affected individuals, also termed a probabilistic risk, then independence means 

that “response addition,” as defined in the Agency’s Supplementary Guidance for Conducting 

Heath Risk Assessment of Chemical Mixtures (USEPA 2000e), can be applied for each adverse 

effect that the stressors have in common. When all the single stressor risks are low, the joint risk 

of a common effect under response addition can be approximated by the simple sum of the single 

stressor risks. For example, if reproductive toxicity is the general effect common to the multiple 

chemicals, the cumulative risk of reproductive effects (at low single chemical risk levels) is 

approximately the sum of the single chemical reproductive risks. Risk addition under 

independence places no constraints on the individual chemical dose-response curves. 

38
 
1

2

3

4

5

6

7

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Toxicologic Similarity. In the second situation, the stressors are grouped according to 

the common mode of action for each effect of concern determined in the planning and scoping 

phase (USEPA, 2002a). For all effects caused by that mode of action, “dose addition” (USEPA, 

2000e) can be applied to the stressor group. Thus far, this approach has only been used with 

combinations of toxicologically similar chemicals, not with combinations of chemicals with 

other kinds of stressors such as radiation, physical factors or health status. With similar 

chemicals, each chemical exposure is converted into the equivalent exposure level of one of the 

39
 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

An Example using Toxicological Independence: National-Scale Air Toxics Assessment 

The National-Scale Air Toxics Assessment, which is based on 1996 emissions data, provides results that are
 

useful in understanding the quality of air and its possible effect on human health nationwide. The assessment
 

includes 32 air toxics (a subset of EPA's list of 188 air toxics) and also diesel particulate matter (which is used
 

as a surrogate measure for diesel exhaust). Specifically, the assessment consists of 4 steps that will produce
 

nationwide estimates of: (1) the release of these pollutants into the air from various sources; (2) the
 

concentra tion of these co mpound s in the air; (3) the e xposure o f populatio ns to this air; and (4 ) the risk of bo th
 

cancer and non-cancer health effects resulting from this exposure.
 

Purpose: The results of the national-scale assessment will provide important information to help EPA continue
 

to develo p and imp lement vario us aspects o f the national air to xics progra m. They w ill not be used directly to
 

regulate sources of air toxics emissions. While regulatory priority setting will be informed by this and future
 

national assessments, risk-based regulations will be based on more refined and source-specific data and
 

assessment tools. More specifically, the assessment results will help to: identify air toxics of greatest potential
 

concern; characterize the relative contributions to air toxics concentrations and population exposures of different
 

types of air toxic s emissions so urces (e.g., ma jor, mob ile) and set prio rities for the collec tion of add itional air
 

toxics data a nd researc h to impro ve estimates o f air toxics conc entrations and their potential p ublic health
 

impacts. Important additional data collection activities will include upgraded emission inventory information,
 

ambient air toxics monitoring, and information on adverse effects to health and the environment; establish a
 

baseline for tracking trends over time in modeled ambient concentrations of air toxics; and establish a baseline
 

for measurin g progress toward m eeting goals fo r inhalation risk re duction from ambient air to xics. 
 

The Four Steps: The national-scale assessment includes the following four major steps for assessing air toxics
 

across the co ntiguous U nited States (a lso Puerto Rico and the Virgin Islan ds). 
 

(1) Compiling a 1996 national emissions inventory of air toxics emissions from outdoor sources. The types of
 

emissions sources in the inventory include major stationary sources (e.g., large waste incinerators and factories),
 

area and o ther source s (e.g.,dry clean ers, small man ufacturers, wildfire s), and both onroad and nonro ad mob ile
 

sources (e.g ., cars, trucks, bo ats). EPA made so me mod ifications to the 1 996 N ational To xics Inventor y to
 

prepare the emissions for computer modeling.
 

(2) Estimating 1996 ambient concentrations based on the 1996 emissions as input to an air dispersion model
 

(the ASP EN mo del). As pa rt of this mode ling exercise, E PA com pared estim ated amb ient concen trations to
 

available ambient air toxics monitoring data to evaluate model performance.
 

(3) Estimating 1996 population exposures based on a screening-level inhalation exposure model (HAPEM4)
 

and the estimated ambient concentrations (from the ASPEN model) as input to the exposure model. Estimating
 

exposure is a key step in determining potential health risk. People move around from one location to another,
 

outside to inside, etc., so exposure isn't the same as concentration at a static site. People also breathe at different
 

rates depending on their activity levels, so the amount of air they take in varies. For these reasons, the average
 

concentration of a pollutant that people breathe (i.e., exposure concentration) may be significantly higher or
 

lower than the concentra tion at a fixed lo cation (i.e., amb ient concen tration). 
 

(4) Characterizing 1996 potential public health risks due to inhalation of air toxics. This includes both cancer
 

and noncancer effects, using available information on air toxics health effects, current EPA risk assessment and
 

risk characterization guidelines, and estimated population exposures. Using the toxicological independence
 

formula and the default assumption of additivity of risks (USEPA 1986b, 2000e), this assessment combines
 

cancer risk e stimates by sum ming them fo r certain weigh t of evidence groupings, a nd also acr oss all group ings. 
 

For non-cancer effects, the assessment assumes dose additivity, and aggregates or sums hazard quotients for
 

individual air to xics that affect the sa me organ or organ syste m (USE PA 20 00e), in this ca se comb ining air
 

toxics that act as respiratory irritants.
 

1 

40 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

chemicals, called the index chemical. The joint toxicity or risk from the combined exposure is 

then estimated by determining the effects or risk for that equivalent exposure level using the 

dose-response information for the index chemical. For example, with the dioxins and furans (see 

text box, next page), each congener exposure level is converted into its equivalent exposure as 

the index chemical, 2,3,7,8-TCDD (USEPA, 1989b). 

Although the assumption itself is not complicated, the decision to assume toxicologic 

similarity can be complicated, depending on the level of assessment decided in the planning and 

scoping phase and described in the analysis plan. The implementation used in Superfund 

assessments (USEPA 1989a, Part D) is a rough approximation to dose addition where a Hazard 

Index is determined whenever chemicals have a common target organ. The implementation by 

the Office of Pesticide Programs in support of FQPA (USEPA, 2002a) is much more extensive 

and requires knowledge of modes of action in order to calculate the Relative Potency Factors 

(RPFs) for the effect of concern (see example in Appendix E). The Toxicity Equivalence Factor 

(TEF) method used for the dioxins is a special case of the RPF method (see Appendix E); it 

requires the most toxicologic similarity because the similarity applies to every toxic effect by any 

type of exposure (USEPA, 2000e). 

Single stressor information can also be used with dissimilar chemicals to gauge the 

potential for toxicologic interaction. For example, chemicals with long whole body half lives, or 

long tissue residence times, have the potential to be present in those tissues at the same time. 

Such overlapping exposures can result in a higher effective tissue dose, altered tissue doses 

caused by toxicokinetic interactions, or altered toxicity from interacting toxic mechanisms. When 

a careful evaluation indicates no internal dose overlap, including metabolites, the single 

exposures might be considered independently. 

3.2.2.2. Information on Stressor Interactions and Multiple Exposures. 

One important simplification that has been common in the assessment of single stressors 

has been the separate evaluation of many of the key steps. That is, simplifying assumptions have 

often been made regarding many characteristics of exposure (e.g., continuous vs. intermittent, 

variations in magnitude) . For a given exposure route, for example, only one dose-response curve 

may be used for the bounding case of setting a cleanup or action level of exposure, and also the 

predictive case of estimating existing risk. These simplifying assumptions allow the doseresponse step to be performed in isolation from the exposure assessment step, with the two steps 

executed in either order. For health-protective action levels, one may use bounds, such as the 

upper bounds on toxic potency and exposure and lower bounds on the resulting acceptable 

exposure level. Such bounds may be much easier to calculate, but may be more difficult to 

interpret in terms of the uncertainties, likelihood and closeness to the best or central estimate. 

The incorporation of multiple chemicals, other stressors, and multiple exposure 

conditions obviously complicates the assessment and the use of simplifying assumptions. In 

cumulative assessments, performing the exposure and dose-response steps of the risk assessment 

paradigm separately is an approximation that obviously invokes a simplifying assumption. If the 

dose response data do not represent the same conditions as the exposure being assessed, an 

extrapolation has to be made, which introduces additional uncertainty that must be clearly stated. 

Joint or cumulative toxicity depends on the total dose or exposure, relative exposure levels, 

41
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

An Example using Toxicologic Similarity: The Dioxin Reassessment 

Scientists from the Environ mental Pro tection Age ncy (EPA ), other Fed eral agencie s and the gen eral scientific 

commu nity have bee n involved in a compr ehensive rea ssessment of d ioxin expo sure and hu man health e ffects 

since 199 1 (USE PA, 20 02c). T he final dioxin re assessment w ill consist of three p arts. Part 1: Estimating 

Exposure to Dioxin-Like Compounds will include four volumes that focus on sources, levels of dioxin-like 

compo unds in enviro nmental me dia, and hum an expos ures. Part 2: Human Health Assessment Document for 

2,3,7,8-Tetrachlorodibenzo-p-Dioxin (TCDD) and Related Compounds will consist of two volumes that include 

information on critical hum an health end points, mod e of action, p harmaco kinetics, dose -response, a nd TE Fs. 

Part 3: Integrated Summary and Risk Characterization for 2,3,7,8-Tetrachlorodibenzo-p-Dioxin (TCDD) and 

Related Compounds will be a stand-alone document. In this summary and characterization, key findings 

pertinent to understanding the potential hazards and risks of dioxins are described and integrated, including a 

discussion o f all important as sumptions a nd uncerta inties. 

2,3,7,8-Tetrachlorodibenzo-p-dioxin (dioxin) is highly toxic to many animal species producing a variety of 

cancer and noncancer effects. Other 2,3,7,8-substituted polychlorinated dibenzo-p-dioxins and dibenzo furans, 

and coplanar polychlorinated biphenyls (PCBs), exhibit similar effects albeit at different doses and with different 

degrees o f confidence in the databa se. The sim ilarities in toxicity betw een specie s and acro ss different diox in 

congene rs stem from a commo n mode o f action via initial bind ing to the aryl hyd rocarbo n (Ah) rece ptor. This 

commo n mode o f action is supp orted by co nsistency in effects ev ident from d ata from mu ltiple congen ers. This 

has led to an international sc ientific consensu s that it is prudent sc ience polic y to use the con cept of toxic 

equivalency factors (TEFs) to sum the contributions of individual PCDD, PCD F, and coplanar PCB congeners 

with dioxin-like activity (van den Berg, et al., 1998). The data supportive of dioxin-like toxicity, both cancer and 

noncancer, are strongest for those congeners that are the major contributors to the risk to human populations. In 

addressing receptor-m ediated re sponses re sulting from co mplex mix tures of diox in-like congen ers, this 

assessment has provided a basis for the use of integrated measures of dose, such as average body burden, as more 

appropriate default metrics than daily intake. The Agency recognizes, however, that the final choice of an 

appropriate dose metric may depend on the endpoint under evaluation. 

In this study, 2,3,7,8-tetrachlorodibenzo-p-dioxin was chosen as the index chemical, and the other dibenzo-pdioxins and dibenzofurans, and coplanar polychlorinated biphenyl doses were adjusted to 2,3,7,8-TCDD 

equivalent toxicities so the doses could be added. 

and the many characteristics of exposure (e.g., duration, continuous vs. intermittent presence, 

route, co-occurrence with other chemicals), and in many cases the complexities introduced by 

multiple stressors will not allow use of some of the common simplifying assumptions of singlestressor assessments. For example, toxicologic interactions have been shown to change using the 

same doses but with a reversal of the sequence of exposure (i.e., chemical B then A instead of A 

then B), so that the exposure and dose-response steps must be compatible and performed 

together. 

Nonchemical stressors can also cause toxicologic interactions. Biological stressors, like 

their chemical counterparts, can interact with chemical exposures and change the overall risk in 

non-additive ways. Ototoxic chemicals, such as toluene, can damage the auditory system and 

have been shown to potentiate the effects of a physical stressor, noise, on hearing loss (Morata, et 

al., 1997; Morata, 2000). 

Toxicity and interaction data for the exposure-response relationship for the mixture of 

interest that covers the full range of exposures is usually impossible because of limits on cost and 

other resources. More feasible approaches to cumulative risk characterization, beyond that with 

42
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

various simplifying assumptions, then require close matching of the exposure and dose-response 

steps to minimize the data requirements. In many cases, screening level ranking may be the only 

practical assessment. In some cases, there will be sufficient information for some quantitative 

evaluation of cumulative health risks that reflect both the complex exposures and toxicologic 

interactions. The issues for these cases are now presented along with their main research 

implications, starting with the simplest case where only chemical interactions are considered. 

“Joint chemical toxicity” means the outcome of exposure to multiple chemicals that 

includes the single chemical effects along with any toxicologic interactions. Chemical 

interactions can be divided into two major categories: those resulting from toxicokinetic and 

those resulting from toxicodynamic modes of action (USEPA, 2000e). Toxicokinetic modes of 

interaction involve alterations in metabolism or disposition of the toxic chemicals, for example, 

by the induction or inhibition of enzymes involved in xenobiotic activation and detoxification. 

Toxicodynamic modes of interaction include those processes that affect a tissue’s response or 

susceptibility to toxic injury. A simplifying observation is that most interactions seem to involve 

pharmacokinetics. Unfortunately, most studies of toxicologic interaction to date have only 

involved two chemicals, and few have quantified the magnitude of the interaction or its 

dependence on exposure conditions. 

Toxicologic interactions are commonly described with terms such as synergism and 

antagonism. These terms are only marginally useful, in part because the underlying toxicological 

concepts are only defined for two-chemical mixtures, while most environmental and 

occupational exposures are to mixtures of many more chemicals. Further, the mathematical 

characterizations of synergism and antagonism are inextricably linked to the prevailing definition 

of "no interaction," instead of some intrinsic toxicological property (Hertzberg and MacDonell, 

2002). The U.S. EPA has selected “dose addition” as the primary no-interaction definition for 

mixture risk assessment, so that synergism would represent observed toxic effects that exceed 

those predicted from dose addition (USEPA, 2000e). The EPA mixture risk guidance also 

describes a modified Hazard Index that incorporates evidence of pairwise toxicologic interactions 

but notes that the pairwise evidence may be specific to the exposure conditions of the study. The 

guidance further encourages development of full biomathematical models for the joint toxicity, 

such as those based on pharmacokinetics, so that qualitative interaction labels such as synergism 

are replaced by quantitative estimates of mixture response that directly reflect the actual 

environmental exposure levels. 

3.2.2.3. Decision Indices. 

The complexities with cumulative risk assessment include the frequent need to combine 

pieces of information that differ widely from each other. Exposure data for some stressors may 

be only as time-weighted averages, while others reflect daily human activity patterns. Toxicity 

data for some chemicals may allow estimation of probabilistic risk for one endpoint, while only 

providing qualitative descriptions of other endpoints. It is possible to develop the risk 

characterization using the original information in a high dimensional matrix, but such a summary 

will be difficult to evaluate and communicate. One approach to diverse multivariate data used 

successfully for weather forecasting is the decision index, with examples such as the smog index, 

the pollen count, and the mold index commonly used to assist in public and personal decisions 

43
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

about environmental exposure. A similar approach can be taken for cumulative risk assessment 

(Hertzberg, 2000). 

The advantage of a decision index is 

the simplicity in converting highly 

multivariate technical information into a 

single number. The most common example 

used for cumulative health risk is the Hazard 

Index (HI) for mixture risk (see box at right). 

Although specific for a single affected target 

organ, each HI reflects multiple studies of 

multiple chemicals, often involving multiple 

test animal species and test exposures, and 

highly varied measures of toxicity. 

The main disadvantage of a simple 

index is that the uncertainties in its 

calculation are largely hidden. Another key 

disadvantage is in quantifying what are often 

Example Decision Index: The Hazard Index 

The Hazard Index for oral exposure is implemented by 

Superfund assessors by the formula: 

HI = sum[ HQj ] = sum[ Ej/RfDj ] 

where Ej

 and RfDj

 are the daily exposure and Reference 

Dose of chemical j. 

The RfD is itself a kind of decision index in that it reflects 

a dose that is selected to be sufficiently low that any toxic 

effects are judged highly unlikely. All available doseresponse data, on all effects, are considered in determining 

each RfD. Uncertainties in the RfD will differ across the 

chemicals, making the uncertainty in HI difficult to 

characterize. 

scientific judgments. For example, the Hazard Index implemented under Superfund (USEPA, 

1989a) is a number whose decision threshold is usually given as 1.0, so that when the HI is 

greater than 1, additional action is indicated. The actual value of HI is not that informative; HI=6 

is not necessarily twice as bad as HI=3. 

One alternative for addressing multiple effects is to recast these qualitative judgments in 

terms of severity categories or levels of concern, and then use statistical methods such as 

categorical regression that use only the ordering of the severity scores, but not their actual values. 

The result is not a risk of a particular toxic effect, but rather a risk of exceeding a certain 

minimum toxic severity level, or level of minimal concern (Hertzberg, 1989; Guth, et al., 1993). 

In the best situations, such as the EPA interaction-based Hazard Index (USEPA, 2000e), the 

decision index formula is modular so that component pieces can be evaluated separately for 

accuracy, and so that improvements in one area can be easily incorporated to give an improved 

index. 

Another example of a decision index with more overt display of its diverse parts is the 

Hazard Ranking System (HRS), a formula developed for characterizing the relative hazards of a 

particular waste site. These hazards were highly diverse, including corrosivity, explosivity, 

toxicity and soil conditions. As with the HI, different uncertainties in the components make the 

uncertainty of the HRS index difficult to describe. Instead of merely presenting the index as a 

number, a high dimensional graphical presentation could be used such as the star plots of 

multivariate data (Chambers, et al., 1983; Hertzberg, 2000), where each arm of the star represents 

one of the sub-indices. While this approach shows the relative contribution of each factor, it 

again hides the uncertainties of the factors as well as of the HRS index itself. 

Hybrid methods also have been used for complex risk assessments that combine 

judgment with numerical descriptions of risk or dose-response. The EPA interaction-based 

Hazard Index (USEPA, 2000e) and the mixture risk approaches of the Agency for Toxic 

44
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Substances and Disease Registry (Hansen, et al., 1998) both include a judgmental weight of 

evidence (WOE) score to reflect the strength of evidence for toxicologic interactions and 

relevance to human health risk. The ATSDR WOE is used in communicating risks and 

intervention options, while the EPA WOE is used to calculate a modified Hazard Index. A 

slightly different approach is the Integral Search System data base program for combinations of 

carcinogens (Woo et al., 1994) by which available studies on pairwise interactions of 

carcinogenicity are used to modify the risk range of the combination from that predicted by 

response addition (USEPA, 2000e). In all these cases, scientific judgment is used to alter the risk 

description or quantitative estimate, but only in terms of an approximate risk interval or a 

decision threshold. 

3.2.2.4. Probabilistic Approaches. 

The recent report by Bogen (2001) illustrates an alternative probabilistic approach to 

noncancer endpoints, in which methods used for integrated quantitative treatment of uncertainty 

and variability are made consistent with those used for probabilistic assessment of cancer risk. 

This report addresses many issues concerning the implementation of probabilistic methods for 

noncancer endpoints, and cites a number of related references (e.g., Lewis, 1993; Dourson et al., 

1994; Slob and Pieters, 1998). 

Any approach to cumulative risk assessment needs to carefully define the set of relevant 

endpoints. Precisely how this is done has important logical and practical implications for how 

the cumulative risk may be calculated and interpreted. For example, the risk of inducing a given 

endpoint may differ among different people in a population at risk for some endpoints, (e.g., 

cancer conditional on all carcinogen exposures), but may be unaffected by interindividual 

variability (e.g., in exposure or susceptibility) for other endpoints (such as ecological or aesthetic 

effects). Defining the latter risks in terms of individual risk per se will thus complicate 

calculating cumulative risk if a probabilistic approach to cumulative risk assessment is used, and 

perhaps if other approaches are used as well. 

In contrast, the probabilistic approach to cumulative risk assessment may be facilitated by 

defining the risk of a given endpoint in terms of population risk, i.e., in terms of the predicted 

number of cases of that endpoint. Alternatively (or additionally), similar simplification can be 

achieved for all heterogeneous endpoints by defining the risk of that endpoint only with respect 

to those persons in the population at risk who are reasonably maximally exposed (e.g., 

individuals adjacent to a proposed source), or to those persons who will incur the greatest 

increased risk (e.g., children or other members of a sensitive subpopulation who might be located 

adjacent to a proposed source). 

3.3. Areas of Complexity and Current Research 

One reason for the somewhat limited availability of cumulative risk assessments may be 

the accompanying complexity that arises in various aspects of the assessment. Some of this is 

discussed in the previous section, along with currently available methods specific to human 

health risk assessment. In this section, some areas where research is ongoing are discussed, and 

some existing methods for quantitatively assessing multiple types of risk or hazard using a single 

metric are described. 

45
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

3.3.1. Interactions Between Stressors and Other Factors. 

In identifying and characterizing susceptible subpopulations, it may be important to 

consider a variety of factors such as current physical and mental health status and past exposure 

histories, which may exacerbate the effects of the stressors of interest. Economic considerations 

such as economic status, community property values, source of income, level of income, and 

standard of living may also affect susceptibility and exposure of subpopulations to certain other 

stressors. Risks associated with chemical or biological stressors may be significantly affected by 

“vulnerability factors” such as lack of health care or genetic predisposition to some diseases and 

effects. Community traditions and beliefs may affect activity patterns and behaviors and therefore 

affect exposure to stressors as well as the risk management options deemed acceptable. 

Depending on the scope of the assessment and the stressors included, “lifestyle factors” such as 

smoking habits, nutritional habits and others may be important to susceptibility. 

In what could be characterized as an exploration of how somewhat abstract factors may 

affect susceptibility, the Agency for Toxic Substances and Disease Registry (ATSDR) held an 

expert panel workshop in 1995 on the subject of psychological responses to hazardous substances 

(ATSDR, 1995). In this report, the panel noted that there is “a significant lack of information” 

about how often communities near hazardous waste sites or spills suffer chronic stress reactions, 

but that psychological stress causes both psychological changes that can be measured by selfreports and objective tests, as well as physical changes such as increased blood pressure, heart 

rate, and biochemical parameters such as changes in stress hormones. Assessing the levels of 

stress, and their potential contribution to risk, is difficult for a variety of reasons. The report notes 

that “unlike the damage and injuries caused by a natural disaster, many toxic substances are 

invisible to the senses.... In the face of no external cues and uncertain circumstances, each person 

affected by a hazardous exposure develops their own beliefs about the nature of the resultant 

harm. These beliefs are based on the facts available to them, pre-existing opinions, cultural 

factors, sensory cues, and the beliefs of leaders and others in the community. . . . Unlike a natural 

disaster, which hits and has a low point after which recovery can begin, the response to a 

hazardous waste site can take 12 to 20 years.” 

Although the ATSDR report indicates that stress related to hazardous chemicals in the 

community can show measurable physical effects, they stopped short of saying that long-term 

health effects from this stress can be converted to risk estimates at this time. One of the questions 

the panel was asked to address was, “Given what is known regarding the psychology of stress, 

are there interactions between chronic stress and exposure to neurotoxicants that could shift the 

dose-response curve for neurotoxins?” The panel concluded: 

“A methodology does not exist that would allow for discrimination between stress or 

neurotoxicant-mediated effects in community-based studies. . . . Experimental animal 

data exist to suggest that stress levels can modulate a toxic response; however, the 

question of specificity remains. Given that stress can induce or unmask a latent effect of 

a toxicant, there is the possibility that chronic stress could alter basal levels of 

neurofunctioning and shift the threshold for neurotoxicity. Indeed, one may find a shift in 

the dose response to a neurotoxicant; however, a specific effect of the neurotoxicant 

needs to be examined in greater detail than the generalized non-specific endpoints. 

46
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Detecting such a shift would require the knowledge of toxicant-specific biological 

mechanisms of actions, which most often are not known.” (ATSDR, 1995, page 30) 

The ATSDR report made many suggestions for research to fill data gaps in this area, and 

scientists may make significant progress in this area in the coming years. 

Another group of factors which may influence the risk to health or the environment, 

whose evaluation may require a different approach from the traditional NRC risk paradigm, is the 

group of “quality-of-life” issues. Although a cumulative human health or ecological health risk 

assessment is not a cumulative impact analysis such as is conducted under NEPA, changes in 

quality-of-life factors may affect the vulnerability of a population to health or ecological risks, 

and consequently may be part of the considerations in a cumulative risk assessment. Since few, if 

any, established and accepted relationships are currently available quantitatively linking qualityof-life factors and health or ecological risk, this is an area in which further research may prove 

valuable. 

To evaluate the effects on human or ecological health from these types of stressors, a 

more deliberative approach (in the analytical-deliberative process) is needed than is used in, say, 

cancer risk analysis. EPA’s Guidebook to Comparing Risks and Setting Environmental Priorities 

(USEPA, 1993b) suggests a six-step process that may help characterize quality-of-life factors, 

some of which may be relevant to the assessment (e.g., in considering population susceptibility). 

An example of a set of quality-of-life criteria, and their descriptions, developed by the State of 

Vermont’s Agency of Natural Resources (State of Vermont, 1991) is provided in Appendix F. 

Quality-of-life issues can encompass much more than the criteria mentioned in Appendix 

F as an example. Some human health or ecological cumulative risk assessments may consider 

quality-of-life factors as having a role in susceptibility to the stressors being assessed. 

3.3.2. The Promise of Biomarkers and Biomonitoring. 

There are a variety of measures that are inherently cumulative. These include biomarkers 

(they give the full effect or full exposure, regardless of source) and measures of the incidence and 

prevalence of disease in a community. The latter give an indication of the total effect of multiple 

sources of exposure. In light of our understanding of the multifactorial basis of disease, a public 

health approach that says “regardless of the cause, a community has x level of disease” can be 

informative. Such statistics can be compared across geographical areas that have different 

sources or different groups that have different levels of vulnerability. The approach is based 

strongly in the field of epidemiology. Indeed, the most often heard critique of epidemiology – 

that it is the prevalence or incidence of disease documented as a function of the combined effect 

of many exposures (over time and/or space), is exactly what makes it so well suited for 

cumulative risk assessment. It is likely that epidemiological concepts will figure prominently in 

cumulative risk assessment, both in identifying the underlying vulnerability of a population and 

by generating hypotheses regarding the determination of relative contributions of multiple 

stressors. 

Sources of data include cross sectional analyses that determine prevalence levels, as well 

as basic surveillance techniques. With respect to the latter, The Pew Environmental Health 

47
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Commission (http://pewenvirohealth.jhsph.edu/html/home/home.html then click on “reports”) 

has recently completed a series of reports that document the extent of national and state level 

resources for chronic disease surveillance. Reports focus on the type of surveillance systems 

needed, as well as the status of registries for birth defects and asthma. Health Track 

(http://health-track.org/ and http://healthyamericans.org/) is the outgrowth of that research, and is 

devoted to tracking and monitoring of chronic disease that would help communities begin to 

identify patterns of health problems. 

Biomarkers are inherently cumulative risk measures. Using biological measurements – 

biomarkers – to determine prior exposures (biomarkers of exposure) or the current health status 

of individuals (biomarkers of effect) holds some promise for cumulative risk assessments of the 

future. Use of biomarkers for a group of chemicals or stressors which act upon individuals in the 

same way can give the assessor a picture of where an individual currently falls on the continuum 

from exposure to effects, making it much easier to predict risks if additional exposure occurs. 

A few biomarkers (or even a single one) can possibly represent exposure to a suite of 

chemicals. Although this reduces the analytical burden and simplifies the process of estimating 

cumulative risk, the approach loses some of the advantages of single-chemical assessment 

(especially being able to quickly discern the importance of different pathways and routes of 

exposure contributing to the risk). 

Biomarkers may be the approach of choice in the future, but the state-of-the-science is not 

developed enough to make this practicable today in an assessment with large numbers of diverse 

stressors (although it may be possible to do this for more simple cases). Currently, biomarker 

development is not at the stage where they can be widely applied. For example, information on 

the cumulative risks in a local population of a group of chemicals that are toxic to the liver might 

be provided by selective liver function tests, but causal inferences would have to take account of 

many other factors that may affect liver function. Likewise, body burden data for chlorinated 

dioxins and related compounds may show that exposure has occurred, but assumptions would 

need to be made as to the pathways, route, and timing of exposures, as well as scenarios 

developed for future exposures if risks are to be estimated. 

One of the benefits of this approach, the development of data which show the actual 

current exposure and risk status of a population, is also its major impediment: it can require 

extensive (or for humans, possibly invasive) monitoring. This can be not only costly, but 

difficult to obtain. This approach uses primarily measurement methods, and also can develop 

statements of probability of adverse effects of additional incremental exposures. This approach 

holds great promise for simplification of a cumulative risk assessment, but few methods exist at 

this time for applying this approach in a cumulative assessment. The main drawback of the 

biomarker approach, at least for a regulatory agency like EPA, is that a decision to act to reduce 

risk is often dependent on separation of contributions from exposure pathways so that effective 

policies can be determined. 

3.3.3. A Single Metric for Multiple Types of Hazard. 

The most complex cumulative risk assessments will evaluate both multiple exposures 

(potentially, multiple sources, stressors, pathways and durations) and multiple effects. Ideally 

48
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

this evaluation would provide projections regarding the potential for a particular complex 

exposure to cause particular effects to different physiological systems, and also provide an 

integration of these projections into a qualitative characterization of overall potential impact to 

human health. Some applications have attempted this via approaches which range from treating 

the assessment as a number of multi-stressor, single effect assessments, where the risks from the 

various effects are combined or characterized at the final step, to those that are more integrated 

throughout. 

For example, cumulative ecological risk assessments such as those that have been 

conducted in the Columbia River Basin and the Chesapeake Bay focused on a number of 

observed adverse conditions, then determined, among all of the possible stressors, which 

particular combination was most influential in creating the observed adverse conditions. 

Stressors such as overharvesting of natural resources, modification of natural hydrology, land use 

change, point-source and non-point-source pollution, including toxic chemicals, and presence of 

exotic species are analyzed, with the goal of the assessment being to design effective restoration 

strategies to eliminate or ameliorate the conditions (Barnthouse, et al., 2000). 

If it is considered desirable to the assessment, an important cumulative risk assessment 

activity may be determining how (if at all possible) to combine risks from different effects – or 

the even more problematic disparate measures of risk – and present them in an integrated 

manner. Depending on the purpose and risk management objectives (see section 2.1.1), some 

cumulative risk assessments may employ some sort of single, common metric to describe overall 

risk. 

One, but certainly not the only, approach to simplifying this problem is to collapse this 

“n-dimensional matrix” of hazards and risks into a few or even a single measure (Murray, 1994). 

However, this requires converting the various measures of risk to a common metric or otherwise 

translating them into a common scale or index. Some methods for combining disparate measures 

of risk are briefly described below. 

3.3.3.1. Creating a Common Metric. 

As discussed earlier in this chapter, there are several different theoretical approaches to 

cumulative risk assessment. Some of these require synthesizing a risk estimate (or risk 

indication) by “adding up” risks for different parts of the risk picture. Actual mathematical 

addition, of course, requires a “common denominator,” or a common metric. Frequently used 

common metrics are risk, money, time, and effort. Finding a common metric for dissimilar risks 

(cancer vs. non-cancer, human vs. ecological, etc.) is not strictly an analytic process, since some 

judgments must be made as to how to link two or more separate scales of risks. These judgments 

often involve subjective values, and because of this, it is a deliberative process. 

As an example of combining different effects into a common metric and the consequent 

judgment needed to achieve a common metric, the EPA Office of Pollution Prevention and 

Toxics in 1999 released its CD-ROM called “Risk-Screening Environmental Indicators Model, 

49
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Version 1.0" (USEPA, 1999i)11. In this model, emissions for both carcinogens and noncarcinogens are weighted by a toxicity factor so that they can be combined in a risk-based 

screening “score” for a particular geographic area. The scale for this weight for carcinogens is 

related to the unit risk factor, and the weight for the non-carcinogens is based on the RfD. 

According to the authors, it is possible to relate these two scales by making a judgment as to how 

they relate. They note that in their case, “when combining cancer and noncancer endpoints, it is 

assumed that exposure at the RfD is equivalent to a 2.5 x 10-4 cancer risk” (Bouwes and Hassur, 

1998; USEPA, 1998h). 

Obviously, as Bouwes and Hassur acknowledge, equating an HQ value of 1.0 (i.e., 

exposure is at the RfD) with a cancer risk of 2.5 x 10-4 is a judgment that is outside the strictly 

analytic part of an assessment; the equating of the two points in the respective scales represents a 

value judgment and as such can be debated. Therefore, this particular part of the assessment is 

deliberative in nature. In most cases, construction of a single scale for different types of 

endpoints will involve comparative risk, a field where different types of risks or endpoints are 

ranked, compared, or converted to a scale based on the judgments and values of the persons 

doing the assessments (USEPA, 1993b, 1998f, 1999j). 

There have been some attempts to allow for transparent and quantitative incorporation of 

values into a common metric. One example flows from the suggestion that “time is the unit of 

measure for the burden of disease”; whether the disease results in disability or premature 

mortality (Murray, 1994). Based on this premise, economic analyses of the costs and benefits of 

disease intervention strategies have used Quality Adjusted Life Years (QALYs) and Disability 

Adjusted Life Years (DALYs) as the metrics for the adverse effects of disease. These metrics are 

intended to reflect the years of life spent in disease states (considering the variation in severity of 

effects) and the years of life lost due to premature mortality resulting from disease as a surrogate 

measure for risk from a variety of different types of effect. Even if this conversion of effects into 

QALYs or DALYs were successful, for diseases that result in periods of morbidity and disability 

(but not death), weighting factors (based on judgments) are used to equate time spent in various 

disease states with years lost to mortality. In this way, dissimilar adverse effects can be 

combined to provide a single measure of disease burden. However, it should be noted that 

aggregation of effects in this manner obscures the meaning of the final measure. QALYs and 

DALYs do not represent an actual shortening of the lifespan but are indicators of the overall 

degradation of well-being that results from various disease states. Therefore, QALYs and 

DALYs may be best suited for ranking and comparative analyses. 

Experience with applying such measures as QALYs and DALYs to environmental risk 

problems is extremely limited. Some very early methods development work has been initiated 

which explores the use of QALYs for combining microbial and disinfection by-product risks 

(USEPA, 1998f). However, some concerns have been raised about the adequacy of such 

measures, especially when integrated with economic information for decision making USEPA, 

2000g). Further methods development work is needed to improve the utility of QALYs and 

DALYs for environmental risk assessments; especially with respect to the incorporation of 

uncertainty (USEPA, 1999j). 

11 As of this writing, EPA has RSEI version 2.0 in beta test. Details are at www.epa.gov/oppt/env_ind/beta_test.htm. 

50 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Categorical regression may provide another tool for combining disparate effects using a 

common metric. In this approach, adverse effects are assigned to severity categories (again, a 

judgment making the process deliberative) and the ordered categories are regressed against 

increasing dose (Teuschler et al., 1999). The use of categorical regression as a tool for 

combining disparate effects has definite limits on interpretation of the results. Since the toxicities 

are only represented by categories, and judgment is used to place the observed response into a 

severity category, the results are rather coarse. But because the analysis is almost totally 

empirical, that is, no low-dose extrapolation is required, the results can still be quite useful. 

EPA has also used decision indices (see section 3.2.2.3) based on dissimilar measures, 

and while they do not produce risk estimates, the indices can still prove useful. The approach 

involves developing a composite score – or index – from measures of various risk dimensions . 

Various environmental risk indices have been developed and applied to ranking and comparative 

analyses. Often, these indices employ surrogate measures for risk rather than using actual 

calculations of the probability of adverse effects. One such index is the Hazard Ranking System 

(HRS) [47 Fed. Reg. 31219, dated July 16, 1982, and amended 55 Fed. Reg. 51532, dated 

December 14, 1990], used to place uncontrolled waste sites on the National Priorities List (NPL) 

for Superfund. This index is based on the likelihood of off-site movement of waste, the toxicity 

of the waste, and the people and sensitive environments that may be affected. It also uses 

corrosivity, toxicity, fire hazard and other factors, all scored and combined into one numerical 

indicator of overall hazard potential. Such an approach for a composite index has been suggested 

for communication of cumulative risk (Hertzberg, 2000). 

Fischhoff et al. (1984) provided an example of this approach as applied to the evaluation 

of energy technologies. In this case, disparate risks are assigned a score from a fixed scale (e.g., 

from 0, representing no risk, to 100, representing the worst risk for that dimension). The scores 

are then weighted to reflect value judgments about the importance of the various risk dimensions 

and the composite score is calculated by summing the individual weighted scores. Again, the 

aggregation of dissimilar adverse effects obscures the meaning of the final score making it more 

appropriate for ranking and comparative analyses. 

Recently, EPA has been working on several index-based approaches to dealing with 

cumulative risk issues. EPA Region III and the Office of Research and Development have been 

jointly working to develop a Potential Risk Indexing System (USEPA, 1993c, 1995d, 1997c). 

This index also uses a vulnerability index, and gauges the overall well-being of a locale and 

various subpopulations. Again, the volume and toxicity of released stressors serve as surrogate 

measures of risk in developing this index. 

Combining diverse effects and risk using either common metrics or indices each have 

pros and cons. A weakness of the index approach is that information is “lost,” and the meaning 

of the final score can be obscured, by aggregating dissimilar information. One strength, however, 

is common to both approaches. Both techniques have the ability to incorporate social values in 

an explicit and quantitative manner in the risk assessment. For example, in the derivation of 

DALYs, weights can be used to reflect the different social roles people play as they age (Murray, 

1994). In the composite scores developed by Fischhoff (1984), public concern was incorporated 

as an adverse effect. This is an important feature for methods that will be applied to cumulative 

risk assessments, especially for communities. Given that cumulative assessments have a 

community/population focus, the ability to incorporate social values in an overall assessment of 

51
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

well-being will be critical. 

3.3.3.2. General Issues with a Single Metric. 

As described above, each approach to portraying the results of a cumulative risk 

assessment has desirable and undesirable features. While common metrics and indices can 

incorporate social values in an explicit and quantitative manner, the meaning of the final measure 

can be obscured by aggregation of dissimilar effects. The abstract meaning of the final measure 

could lead to difficulties when communicating the results of the cumulative risk assessment to 

the public. Graphical and mapping techniques do not necessarily overcome such problems with 

communication. While these techniques may avoid some of the problems associated with the 

mathematical aggregation of dissimilar effects, it can be difficult to accurately describe the 

information that a graphic is intended to convey. 

Because we have relatively little experience in combining different types of risk, a key 

issue is the need for methods development in this area. The approaches described above indicate 

a beginning. Additional exploratory work is needed, however, to further develop existing 

methods and to find additional methods that are flexible, can incorporate social values, are easy 

to communicate, and provide an integrated portrayal of the overall well-being of a community 

and its various subpopulations. 

3.3.4. Qualitative approaches. 

There will be cases where cumulative risk cannot be quantified in any meaningful or 

reliable way. Qualitative approaches can be valuable for cumulative risk assessment and, in the 

near-term, may be the only practical way to address many of the complexities involved. 

Qualitative approaches may be used as a way to overcome the complexity and data deficiencies 

that hinder quantitative approaches. In many assessments, risk may not be a quantifiable 

variable. 

For these cases, there may be qualitative approaches that provide some insight. Broad 

indicators related to exposure in complex ways (e.g., production volumes, emissions inventories, 

environmental concentrations, etc.) and indicators of toxicity can be communicated using 

geographic information systems. Displaying complex multi-dimensional matrices in a map can 

help visualize locations of areas with multiple stressors. Furthermore, geographically based 

measures of hazard are potentially useful cumulative measures – although they do not provide 

information on the risks, the locations of hazards can be used as an indicator of cumulative 

exposures, thus risks from all of the potential chemicals associated with that site. The 

environmental justice literature has used this approach. 

Quantitative results might eventually be reduced to a more qualitative scale (High, 

Medium or Low), or the qualitative results could provide “comments” tacked to the quantitative 

results. The assessment might simply raise “red flags” associated with specific issues (e.g. 

density of emitters in a community; presence of minority populations; special exposure 

pathways; etc); a high number of such flags would indicate unacceptable cumulative risk, even if 

this isn’t quantified. This approach has been used in the European Union, and their experience 

52
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

[ref needed] in using qualitative methods for permitting suggests that “qualitative” is not 

“irrational”. Other relevant tools include expert judgment techniques, focus groups, opinion 

surveys, citizen juries, alternative dispute resolution, and others. 

Figure 4.1. The Interpretation and Risk Characterization Phase. 

4. THE RISK CHARACTERIZATION PHASE 

The last phase of cumulative risk assessment, Risk Characterization, integrates and 

interprets the results of the Analysis phase and addresses the problem(s) formulated in the 

Planning and Scoping phase. It should describe the qualitative and/or quantitative risk 

assessment results; list the important assumptions, limitations and uncertainties associated with 

those results; and discuss the ultimate use of the analytic-deliberative outcomes. Given the 

complexity of cumulative risk issues and the need for clarity and transparency in risk 

characterization, such 'full disclosure' presents a major communication challenge. 

As in the Analysis Phase, there is a substantial analytical component of the Interpretation 

53
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Phase, but there is also a considerable need for deliberation. At a minimum, stakeholders in this 

phase should (1) understand the outcome of the cumulative risk assessment; (2) ask questions 

about how best to frame the interpretation; (3) confirm that the cumulative risk assessment met 

the goals set in the Problem Formulation, or if not, why not. As in the previous phase, the 

stakeholders’ role is only limited by what is proposed and agreed upon in the individual case 

being assessed. 

Risk estimation in a cumulative risk assessment will involve some combination of risks, 

either risks from various stressors causing similar effects, or risks from various stressors causing 

different types of effects. The stressors may be similar or widely different. Combinations of many 

types of stressors with different endpoints in a single assessment will quickly cause the risk 

estimation step to become very complex and difficult. 

Because of its potential complexity, and because in some cases cumulative risk 

assessments will be dealing with “uncharted territory” methodologically, it is very important that 

the planning, conduct, analysis, and characterization of a cumulative risk assessment be 

transparent. As stated by OMB (OMB, 

2002), the "benefit of transparency is that 

the public will be able to assess how much 

an agency's analytic result hinges on the 

specific analytic choices made by the 

agency." The process, methodology, data, 

assumptions, and selection among alternate 

interpretations must be very carefully 

documented and very clearly stated. This is 

noted again in the next section. 

4.1. Risk Description 

The ultimate product in the risk 

assessment process is the risk 

characterization, in which the information 

from all the steps is integrated and an 

overall conclusion about risk is synthesized 

that is complete, informative, and useful for 

decision-makers. The nature of the risk 

characterization will depend on the 

information available, the regulatory 

application of the risk information, and the 

resources (including time) available. It is 

important to identify and discuss all major 

issues associated with determining the 

nature and extent of the risk. Further, the 

EPA Administrator’s March 1995 Policy for 

Risk Characterization (U.S. EPA, 1995a) 

specifies that a risk characterization “be 

prepared in a manner that is clear, 

Risk Characterization Guiding Principles 

Regarding information content and uncertainty aspects: 

�� The risk characterization integrates the information from the 

exposure and dose-response assessments, using a combina tion 

of qualitative information, quantitative information, and 

information regarding uncertainties. 

�� The risk characterization includes a discussion of uncertainty 

and variability. 

�� Well-balanced risk characterizations present risk conclusions 

and information regarding the strengths and limitations of the 

assessment for other risk assessors, EPA decision-makers, and 

the public. 

Regarding risk descriptors: 

�� Information about the distribution of individual exposures is 

important to communicating the results of a risk assessment. 

�� Information abou t population exposure leads to another 

important way to describe risk. 

�� Information about the distribution of exposure an d risk for 

different subgroups of the population are important components 

of a risk assessment. 

�� Situation-specific information adds perspective on possible 

future events or regulatory options. 

�� An evaluation of the uncertainty in the risk descriptors is an 

important component of the uncertainty discussion in the 

assessment. 

Source: USEPA, 1995b. 

54
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

transparent, reasonable, and consistent with other risk characterizations of similar scope prepared 

across programs in the Agency.” In short, estimates of health risk are to be presented in the 

context of uncertainties and limitations in the data and methodology. 

The 1995 Guidance for Risk Characterization (USEPA, 1995b) lists several guiding 

principles for defining risk characterization in the context of risk assessment (see text box), both 

with respect to information content and uncertainty aspects and with respect to descriptions of 

risk. EPA has recently published a handbook on risk characterization (USEPA, 2000c). 

Risk assessments are intended to address or provide descriptions of risk to one or more of 

the following: (1) individuals exposed at average levels and those in the high-end portions of 

the risk distribution; (2) the exposed population as a whole; and (3) important subgroups of the 

population such as highly susceptible groups or individuals (e.g., children), if known. Risk 

predictions for sensitive subpopulations are a subset of population risks. Sensitive 

subpopulations consist of a specific set of individuals who are particularly susceptible to adverse 

health effects because of physiological (e.g., age, gender, pre-existing conditions), 

socioeconomic (e.g., nutrition), or demographic variables, or significantly greater levels of 

exposure (USEPA, 1992a). Subpopulations can be defined using age, race, gender, and other 

factors. If enough information is available, a quantitative risk estimate for a subpopulation can 

be developed. If not, then any qualitative information about subpopulations gathered during 

hazard identification should be summarized as part of the risk characterization. 

The 1996 book Understanding Risk (NRC, 1996) devoted a great deal of discussion to risk 

characterization. Risk characterization is most efficiently conducted by early and continued 

attention to the “risk characterization” step in the risk assessment process (NRC, 1996; USEPA, 

2000c). The box on the following page summarizes some of the points made in Understanding 

Risk. 

55
 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Some Thoughts on Risk Characterization 

The NRC book Understanding Risk (NRC, 1996) has and character of uncertainty. . .
 

risk characterization as its primary focus. In their
 

conclusions, NRC states: Getting the right science. The analysis has addressed
 

the significant risk-related concerns of public officials 

1. Risk characterization should be a decision-driven and the spectrum of interested and affected parties, such 

activity, directed towards informing choices and solving as risks to health, economic well-being, and ecological
 

problems. The view of risk characterization as a and social values, with analytic priorities having been
 

translation or summary is seriously deficient. . .. Risk set so as to emphasize the issues most relevant to the
 

characterization should not be an activity added at the decision.
 

end of risk analysis; rather, its needs should largely
 

determine the scope and nature of risk analysis. Getting the right participation. The analytic-deliberative
 

process has had sufficiently broad participation to 

2. Coping with a risk situation requires a broad ensure that the important, decision-relevant information 

understanding of the relevant losses, harms, or enters the process, that all important perspectives are 

consequences to the interested and affected parties. A considered, and that the parties’ legitimate concerns
 

risk characterization must address what the interested about inclusiveness and openness are met.
 

and affected parties believe to be at risk in the particular
 

situation, and it must incorporate their perspectives and Getting the participation right. The analytic-


specialized knowledge. deliberative process satisfies the decision makers and
 

interested and affected parties that it is responsive to 

3. Risk characterization is the outcome of an analytic- their needs: that their information, viewpoints, and 

deliberative process. . . . Analysis and deliberation can concerns have been adequately represented and taken 

be thought of as two complementary approaches to into account; that they have been adequately consulted; 

gaining knowledge about the world, forming and that their participation has been able to affect the 

understandings on the basis of knowledge, and reaching way risk problems are defined and understood. 

agreement among people. 

Developing an accurate, balanced, and informative 

4. The analytic-deliberative process leading to a risk synthesis. The risk characterization presents the state of 

characterization should include early and explicit knowledge, uncertainty, and disagreement about the risk 

attention to problem formulation. situation to reflect the range of relevant knowledge and 

perspectives and satisfies the parties to a decision that 

5. The analytic-deliberative process should be mutual they have been adequately informed within the limits of 

and recursive. . . . A recurring criticism of risk available knowledge.
 

characterization is that the underlying analysis failed to
 

pay adequate attention to questions of central concern to 6. Those responsible for a risk characterization should
 

some of the interested and affected parties. This is not begin by developing a diagnosis of the decision
 

so much a failure of analysis as a failure to integrate it situation so that they can better match the analytic-


with broadly based deliberation: the analysis was not deliberative process leading to the characterization to
 

framed by adequate understanding about what should be the needs of the decision, particularly in terms of level
 

analyzed. . . . Structuring an effective analytic- and intensity of effort and presentation of parties. . . .
 

deliberative process for informing a risk decision is not a Diagnosis of risk decision situations should follow eight
 

matter for a recipe. Every step involves judgment, and steps: (1) diagnose the kinds of risk and the state of
 

the right choices are situation dependent. Still, it is knowledge, (2) describe the legal mandate, (3) describe
 

possible to identify objectives that also serve as criteria the purpose of the risk decision, (4) describe the affected
 

for judging success: parties and anticipate public reactions, (5) estimate
 

resource needs and timetable, (6) plan for organizational 

Getting the science right. The underlying analysis meets needs, (7) develop a preliminary process design, and (8)
 

high scientific standards in terms of measurement, summarize and discuss the diagnosis with the
 

analytic methods, data bases used, plausibility of responsible organization.
 

assumptions, and respectfulness of both the magnitude
 

1 

2 

3 

4 4.2. Uncertainty Analysis 

56 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

In their 1990 book Uncertainty: A Guide to Dealing with Uncertainty in Quantitative 

Risk and Policy Analysis, Morgan and Henrion (1990) note that historically, the most common 

approach to uncertainty in policy analysis (including in risk assessment) has been to ignore it. In 

a section titled “Why Consider Uncertainty?” they advance three primary reasons, all of which 

are especially relevant to an analytic-deliberative process such as cumulative risk assessment. 

They suggest that it is important to worry about uncertainty: 

•	 “when one is performing an analysis in which people’s attitude toward risk is likely to be 

important, for example, when people display significant risk aversion; 

•	 “when one is performing an analysis in which uncertain information from different 

sources must be combined. The precision of each source should help determine its 

weighting in the combination; and 

•	 “when a decision must be made about whether to expend resources to acquire additional 

information. In general, the greater the uncertainty, the greater the expected value of 

additional information.” 

Morgan and Henrion provide 

“ten commandments” for good policy 

analysis, and although all are 

commendable, and several have been 

discussed elsewhere in this Framework 

report, we should look more closely at 

numbers 6-8 in the box at right for 

some insight into uncertainty analysis. 

There are many resources available 

which talk in detail about how to 

perform uncertainty analysis (e.g., 

USEPA, 1997b, Morgan and Henrion, 

1990). While detailed instruction on 

how to perform uncertainty analysis is 

beyond the scope of this Framework 

Morgan & Henrion’s “Ten Commandments” 

for Good Policy Analysis 

1. Do your homework with literature, experts, and users. 

2. Let the problem drive the analysis. 

3. Make the analysis as simple as possible, but no simpler. 

4. Identify all significant assumptions. 

5. Be explicit about decision criteria and policy strategies. 

6. Be explicit about uncertainties. 

7. Perform systematic sensitivity and uncertainty analysis. 

8. Iteratively refine the problem statement and the analysis. 

9. Document clearly and completely. 

10. Expose the work to peer review. 

Source: Morgan and Henrion, 1990. 

report, we believe that a discussion of some general principles is in order. 

4.2.1. Assumptions in the Assessment 

Cumulative risk assessment will typically be used in a decision-making process to help 

inform the decision-maker(s). For this reason, it is important that the decision makers be made 

explicitly aware of any assumptions that may significantly affect the conclusions of the analysis 

(item #6 in the box above). Morgan and Henrion suggest that these assumptions include: 

• the main policy concerns, issues, or decisions that prompted the assessment; 

• the evaluation criteria to be used to define issues of concern or options; 

•	 the scope and boundaries of the assessment, and ways in which alternate selections might 

influence the conclusions reached; 

• soft or intangible issues that are ignored or inadequately dealt with in the quantitative 

57
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

analysis (e.g., intrinsic value of wilderness, equity of distribution of risks and benefits); 

• approximations introduced by the level of aggregation or by level of detail in models; 

• value judgments and tradeoffs; and 

•	 the objective function used, including methods of combining ratings on multiple criteria 

(or combining risk scales). [adapted from Morgan and Henrion, 1990] 

Identifying significant assumptions can often highlight “soft” uncertainties that are not 

easily quantified, and are therefore often left out of a quantitative uncertainty analysis. 

Nevertheless, these “soft” assumptions can often contribute more to the overall uncertainty of the 

assessment than the factors more easily quantified. 

In item #7 in Morgan and Henrion’s “ten commandments,” they list three types of 

uncertainty that analysts should explicitly include: 

•	 uncertainty about technical, scientific, economic, and political quantities (e.g., quantities 

like rate constants often lend themselves to quantitative uncertainty estimates relatively 

easily); 

•	 uncertainty about the appropriate functional form of technical, scientific, economic, and 

political models (e.g., are the models used, such as dose-response models, biologically 

sound?); 

•	 disagreements among experts about the values of quantities or the functional form of 

models (e.g., different health scientists using different forms of dose-response models). 

In Item #8 in the box on the previous page, Morgan and Henrion suggest that an assessor 

needs to find out which assumptions and uncertainties may significantly alter the conclusions, 

and that process can be conducted using sensitivity and uncertainty analysis. Techniques for these 

include: 

•	 deterministic, one-at-a-time analysis of each factor, holding all others constant at nominal 

values; 

• deterministic joint analysis, changing the values of more than one factor at a time; 

•	 parametric analysis, moving one or a few inputs across reasonably selected ranges to 

observe the shape of the response; and 

•	 probabilistic analysis, using correlation, rank correlation, regression, or other means to 

examine how much of the uncertainty in the conclusions is attributable to which inputs. 

Finally, Morgan and Henrion answer the question of why we should consider uncertainty 

analysis with the following point. “Policy analysts have a professional and ethical responsibility 

to present not just “answers” but also a clear and explicit statement of the implications and 

limitations of their work. Attempts to fully characterize and deal with important associated 

uncertainties help them to execute this responsibility better.” (Morgan and Henrion, 1990) 

4.2.2. Uncertainty and Variability 

58
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

In their 1994 report Science and Judgment in Risk Assessment (NRC, 1994), the National 

Research Council noted a clear difference between uncertainty and variability, and recommended 

that the distinction between these two be maintained: 

“A distinction between uncertainty (i.e., degree of potential error) and inter-individual 

variability (i.e., population heterogeneity) is generally required if the resulting 

quantitative risk characterization is to be optimally useful for regulatory purposes, 

particularly insofar as risk characterizations are treated quantitatively. The distinction 

between uncertainty and individual variability ought to be maintained rigorously at the 

level of separate risk-assessment components (e.g., ambient concentration, uptake, and 

potency) as well as at the level of an integrated risk characterization.” (NRC, 1994, page 

242) 

Variability and uncertainty have been treated separately and distinctly in single-chemical 

assessments such as the assessment of trichloroethylene in ground water at Beale Air Force Base 

in California (Bogen, 2001). The treatment of variability and uncertainty will be an important 

issue in cumulative risk assessments, also, although at the time of this writing there are no good 

examples available of an elegant treatment of this issue for cumulative risk. 

4.2.3. Uncertainty and Risk Addition 

Calculating individual stressor risks, 

and then combining them, presents largely the 

same challenges as combination toxicology, 

but also adds some statistical stumbling 

blocks. Toxicity addition, independence, 

synergism, or antagonism still need to be 

evaluated, but since risk estimates for various 

stressors are often presented as values on the 

same numeric scale (e.g., as cancer 

probabilities), cancer risks are often just 

added together. 

Since most cancer slope factors are 

not “most probable estimates,” but 95% upper 

confidence levels, adding traditional risk 

levels can cause the resulting sum to 

overestimate a 95% upper confidence level 

risk for a mixture. There have been several 

recent papers discussing this problem and 

Uncertainty Analysis Example: 

The Cumulative Exposure Project 

EPA’s Cumulative Exposure Project (CEP), completed in 

1998, modeled 1990 outdoor concentrations of hazardous 

air pollutants (HAPs) across the United States, which were 

combined with unit risk estimates to estimate the potential 

increase in excess cancer risk from multiple HAPs. The 

cancer risks of different HAPs were assumed to be additive 

and were summed across pollutants in each census tract to 

estimate a total cancer risk in each census tract. 

Consideration of some specific uncertainties, including 

underestimation of ambient concentrations, combining 

upper 95% confidence bound potency estimates, and 

changes to potency estimates, found that cancer risk may 

be underestimated by 15% or overestimated by 40-50%. 

Other unanalyzed uncertainties could make these under- or 

overestimates larger. 

Source: Woodruff, et al., 2000 

how it may effect the resulting estimates. Kodell and Chen (1994) looked at several binary 

mixtures and calculated that the summation of individual upper 95% confidence intervals for 

chlorobenzene and hexachlorobenzene would overestimate the upper-bound risk of a binary 

mixture of these compounds by 2-6%, while for chlorobenzene and trichloroethylene, the 

overestimate would be in the range of 12-15%. Seed, et al. (1994) noted that, “in most cases, the 

magnitude of the difference in cancer risk estimates calculated by [Kodell and Chen’s] various 

59
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

methods will be greatest for mixtures of eqipotent compounds. However, even for mixtures of 

equipotent compounds, the differences in joint risk estimated by summing the upper 95% 

confidence levels. . .are not great.” After analyzing four cases, Cogliano (1997) concluded that 

“as the number of risk estimates increases, their sum becomes increasingly improbable, but not 

misleading.” For example, in adding 20 different cancer risk estimates based on a 95% upper 

bound, the resulting sum of the upper bounds was no more than 2.2 times the true upper bound. 

Cogliano goes on to suggest that, for certain cases not involving synergistic or antagonistic 

interactions, “depending on the number of carcinogens and the shape of the underlying risk 

distributions, division by a factor of 2 can be sufficient to convert a sum of upper bounds into a 

plausible upper bound for the overall risk” (Cogliano, 1997). 

The assumption of toxicologic independence (see section 3.2.2) may not be a bad 

assumption if other evidence supports it, but it should be addressed in the assessment if used 

(i.e., if risks are added). Although there are some scientists who believe that toxicologic 

interactions are of minor consequence at concentrations observed in the environment (see 

discussion in USEPA, 2000e), the scientific evidence for such an assumption has not been firmly 

established. 

Notwithstanding the statistical limitations of adding traditional risk estimates, and the 

implicit assumption that the toxicities will be additive12 (i.e., no interactions such as synergism or 

antagonism occur), the numerical ease for combining risks in this way may make it the most 

popular method for approximating cumulative risks in the short term, at least at a screening level 

of assessment. 

4.3. The Information Provided by Cumulative Risk Assessment 

It is important to clarify how cumulative risk assessment and this Framework report relate 

to community assessments and community decision making. Certainly, the Agency’s Risk 

Characterization Handbook (USEPA, 2000c) emphasizes that whatever information is imparted, 

it be transparent, clear, consistent, and reasonable. For example, if it is known that the results of a 

particular cumulative risk assessment will be severely limited because of a lack of data or 

available methods, it may be advisable to start with a screening analysis to set priorities for a 

subsequent more detailed, focused study. In simple terms, what can a cumulative risk assessment 

tell us, and what can’t it tell us? 

4.3.1. Making Sense of Multiple Stressor Effects 

The information provided by cumulative risk assessment is only a portion of the 

information that communities and governments need to make informed decisions about risks. 

There are almost always a multitude of factors that affect health in a community (e.g., crime, 

drugs, health care access, vehicle safety, climate, infectious disease, diet. . .), some of which may 

not have been considered within the scope of any given cumulative risk assessment. Community 

decision-making will typically take risks to the environment into account, as well as 

12 At risk levels often seen with pollutant concentrations observed in the environment, the combined risks calculated assuming 

“response additi vity” (that is, each component acts as if the other were not present) are app roximate ly the same a s with dos e additivity (USEPA, 

2000e). 

60 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

consideration about historical and cultural values, and questions of fairness and distribution of 

risks. The methodology is not currently available to understand how these factors (or stressors) 

may affect cumulative health risk. 

Additionally, benefits that may be associated with chemical or other stressor exposures – 

benefits such as jobs and useful products or services – may be important contexts for decisions 

on the risks considered in cumulative risk assessments. 

This Framework report is not an attempt to lay out protocols to address all the risks or 

considerations that are needed to adequately inform community decisions. Rather, it is focused 

on describing various aspects of cumulative risk, whether or not the methods or data currently 

exist to adequately analyze or evaluate those aspects of the assessment. The Framework report 

devotes considerable time to a discussion of improving the methods for a single part of the 

broader picture -- characterizing health risks associated with exposures to multiple chemicals via 

multiple routes. Because of the limitations of the current state of the science, cumulative risk 

assessments in the near future will not be able to adequately answer all questions posed by 

stakeholders or interested parties. This does not mean, however, that they can’t be useful in 

providing insights to some of the questions asked; in fact, cumulative risk assessment may be the 

best tool available to address certain questions dealing with multiple stressor impacts. 

4.3.2. Cumulative Risk Assessments in a Public Health Context 

The public, in a variety of forms, continually draws attention to health statistics, asking 

for clarification of the relationship between environmental pollution (and risk assessments 

concerning it) and public health. It is important to clarify that to draw relationships between 

environmental pollutant exposures and disease incidence, a body of epidemiological study is 

necessary, and trying to “work backwards” from health statistics to risk factors requires full 

knowledge of the risk factors associated with the relevant disease(s). 

Health statistics, including death rates and incidence of various diseases, illustrate the 

impact of a variety of risk factors (e.g., smoking as well as environmental pollutants) and risk 

reduction factors (e.g., exercise and good nutrition, as well as pollution control measures). 

Indeed, population health statistics are reflective of all risk and risk reduction factors in a 

population’s history-to-date. Even the best cumulative risk assessment given today’s state of the 

science would fall short of being able to include an evaluation of the magnitude and interactions 

of all stressors and effects. At best, the risk estimates of a cumulative risk assessment will reflect 

some of the risks which may be reflected in community health statistics. With rare exceptions13, 

cumulative risk assessment estimates would not be expected to match exactly with community 

health statistics, even for specific health endpoints such as specific cancers. 

4.3.3. How Scope and Purpose of the Assessment Affect Results 

13 It is conceivable that high risks to rare specific effects could be comparable between a risk assessment and 

community health statistics given current state of the art. To be sure this is not coincidental, a substantial effort to match risk 

assessment scenarios with actual histories or exposures would have to be made. 

61 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Historically, the Agency’s risk assessments have focused on assessing the risks from 

environmental pollutants to public health or the environment, usually for the purposes of 

prioritizing risk management activities or triggering regulatory action. Given the need for public 

health protective decisions, traditional risk assessment tools usually focus on predicting high 

ends of the risk distribution. Also, the traditional tools are not designed to predict risk of diseases 

other than cancer. Additionally, the many environmental pollutants comprise only some of the 

categories of risks to public health. While quite adequate for their original purpose, when the 

results of these types of assessments are viewed from another perspective, such as a community 

concerned about the cumulative health impacts of five industrial and commercial facilities within 

a two block area, they may not be useful. 

The Agency is doing more place-based human health and ecological assessments (i.e., 

compared to source- or media-specific assessments) than in the past, but it will be some time 

before place-based assessments become commonplace. Consistent with good practices for 

planning and scoping, these often may be driven by specific risk management needs. To the 

extent there are parties that were outside the process, their desired objectives and purpose may 

differ from those for which the assessment was designed. For this reason, users of cumulative 

risk assessments are advised to carefully study the scope and purpose of the assessment at hand, 

as well as the analysis plan and resulting characterization, in order to determine whether it is 

suitable (or partly suitable) to answer questions outside its stated objectives and purpose. 

4.4. Using the Results of the Assessment 

Once the results of an assessment are in hand, the assessment participants will usually 

focus primarily on the use of those results. The intended use of the assessment was considered at 

the beginning, in the Problem Formulation Phase, both to plan the assessment work and to set the 

stage for what possible actions might be taken at this point.. A detailed discussion of the use of 

the results of a cumulative risk assessment is beyond the scope of this document, but in deciding 

on a course of action, other considerations will need to be taken into account along with the 

results of the cumulative risk assessment. 

If the goals of a cumulative risk analysis are to estimate the risk from multi-chemical and 

multi pathway exposure to individuals living within a geographical area of concern, then an 

important objective in presenting the results is to identify the major risk contributors in order to 

understand the sources, pathways, and stressors which contribute most to that overall risk. The 

results of a cumulative risk assessment provide an additional tool for the risk manager, one that 

permits a more complete accounting and more explicit analysis to target follow-up risk 

mitigation strategies toward those stressors which most contribute to the population’s risk. 

If action to mitigate or prevent risk is the goal of the stakeholders, then options for action 

discussed in the planning of the assessment can be re-evaluated in light of the results of the 

assessment. Some of the issues after re-evaluating the action alternatives might include: “Is 

regulatory authority available to address concerns or are voluntary actions better suited to address 

the risks?” or “Can the concerns be addressed by the stakeholders involved in the assessment or 

are the options for mitigation and prevention beyond the scope of their control?” In the latter 

case, for example, siting issues are usually decided locally and may be within the authority of the 

62
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

participants of a local assessment. In contrast, 

risk from mobile sources or acid rain are likely 

to require action beyond the scope of a single 

local community. In that case, taking action 

will require working with other communities 

and is likely to take more time. Discussion of 

the options available for addressing results of 

a risk assessment will help to keep 

expectations in line with possibilities. 

In taking action – or not taking action – 

after a cumulative risk assessment has been 

interpreted, the team may benefit from lessons 

learned by others, just as in the planning, 

scoping, and problem formulation phase. The 

European Environment Agency (EEA) in early 

2002 released an extensive study of twelve 

classic case studies in human and 

environmental health protection, and the 

lessons learned from them (EEA, 2001). The 

report is available on the internet and should 

be “food for thought” for any group 

contemplating protective actions, but 

particularly for community assessments. 

Twelve of the EEA’s “late lessons learned” are 

reproduced in the box at right. 

Finally, it is important to keep in mind 

that the results of the risk assessment will be 

EEA’s 12 Late Lessons Learned 

- Acknowledge and respond to ignorance, as well as 

uncertainty and risk, in technology appraisal and public 

policy-making. 

- Provide adequate long-term environmental and health 

monitoring and research into early warnings. 

- Identify and work to reduce blind spots and gaps in 

scientific knowledge. 

- Identify and reduce interdisciplinary obstacles to 

learning. 

- Ensure that real world conditions are adequately 

accounted for in regulatory appraisal. 

- Systematically scrutinize the claimed justifications and 

benefits alongside the potential risks. 

- Evaluate a range of alternative options for meeting needs 

alongside the option under appraisal, and promote more 

robust, diverse and adaptable technologies so as to 

minimize the costs of surprises and maximize the benefits 

of innovation. 

- Ensure use of "lay" and local knowledge, as well as 

relevant specialist expertise in the appraisal. 

- Take full account of the assumptions and values of 

different social groups. 

- Maintain regulatory independence from interested parties 

while retaining an inclusive approach to information and 

opinion gathering. 

- Identify and reduce institutional obstacles to learning and 

action. 

- Avoid "paralysis by analysis" by acting to reduce 

potential harm when there are reasonable grounds for 

concern. 

Source: EEA, 2001 

only one of the factors that will need to be considered in making a decision on action to address 

the risk. Risk information can make an important and valued contribution to the decisionmaking process, but risk information, by itself, can not determine the decision. Factors such as 

the availability of resources for change, fairness and other community values, politics, business 

and employment considerations, quality of life issues, or concern for future generations will also 

influence any decision made. In the siting example mentioned above, the assessment may 

determine that the new facility does not significantly increase risk to the community and a 

decision not to site the facility might still be made on the basis of a quality of life issue unrelated 

to risk. Or, in contrast, a community may decide that the economic and employment benefits 

outweigh the risks associated with the siting. Other risk factors not considered in the assessment 

may also enter into the decision-making process. This can include both the environmental risks 

not covered in the cumulative risk assessment as well as the non-environmental risks that may 

affect a community. With limited resources, a community may use all available risk information 

to most effectively target its resources. 

63
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

5. GLOSSARY 

Adverse effect - A biochemical change, functional impairment, or pathological lesion that either 

singly or in combination adversely affects the performance of the whole organism or reduces an 

organism’s ability to respond to an additional environmental challenge. 

Agent - a chemical, radiological, mineralogical, or biological entity that may cause deleterious 

effects in an organism after the organism is exposed to it. 

Aggregate exposure - The combined exposure of an individual (or defined population) to a 

specific agent or stressor via relevant routes, pathways, and sources. 

Aggregate risk - The risk resulting from aggregate exposure to a single agent or stressor. 

Benchmark dose (BMD) - The dose producing a predetermined, altered response for an effect. 

A BMD10, for example, would be calculated based on a benchmark response of 10%. 

Benchmark response (BMR) - A predetermined level of altered response or risk at which the 

benchmark dose is calculated. Typically, the BMRs used are 1%, 5%, or 10%. 

Conceptual model - Both a written description and a visual representation of actual or predicted 

relationships between humans or ecological entities and the chemicals or other stressors to which 

they may be exposed. 

Cumulative risk - The combined risks from aggregate exposures to multiple agents or stressors. 

Cumulative risk assessment - An analysis, characterization, and possible quantification of the 

combined risks to health or the environment from multiple agents or stressors. 

Dose additivity - In a mixture, when each chemical behaves as a concentration or dilution of 

every other chemical. The response of the combination of chemicals is the response expected 

from the equivalent dose of an index chemical (the chemical selected as a basis for 

standardization of toxicity of components in a mixture). The equivalent dose is the sum of 

component doses scaled by their toxic potency relative to the index chemical. For example, for 

chlorinated dibenzodioxins (CDDs), 2,3,7,8-TCDD is selected as the index chemical, and other 

CDD concentrations are adjusted for their potency relative to 2,3,7,8-TCDD, then treated as if 

they were 2,3,7,8-TCDD “equivalents.” 

Dose-response relationship - A relationship between (1) the dose, either “administered dose” or 

absorbed dose, and (2) the extent of toxic injury produced by that chemical or agent. Response 

can be expressed either as the severity of injury or proportion of exposed subjects affected. 

Endpoint - An observable or measurable biological or chemical event used as an index of the 

effect of a stressor on a cell, tissue, organ, organism, etc. 

64
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Lowest observed adverse effect level (LOAEL) - The lowest dose or exposure level in a study 

which there is a statistically or biologically significant increase in the frequency or severity of an 

adverse effect in the exposed population as compared with an appropriate, unexposed control 

group. 

Model - A mathematical representation of a natural system intended to mimic the behavior of the 

real system, allowing description of empirical data and predictions about untested states of the 

system. Use of models is usually facilitated by computer programming of the mathematics and 

construction of a convenient input and output format. 

No observed adverse effect level (NOAEL) - An exposure level at which there are no 

statistically or biologically significant increases in the frequency or severity of adverse effects 

between the exposed population and its appropriate control; some effects may be produced at this 

level, but they are not considered to be adverse or precursors to adverse effects. In an experiment 

with several NOAELs, the common usage of the term NOAEL is the highest exposure without 

adverse effects. 

Ototoxic stressor - A stressor which causes damage to the ear or the sense of hearing. 

Reference Concentration (RfC) - An estimate (with uncertainty spanning perhaps an order of 

magnitude) of a continuous inhalation exposure to the human population (including sensitive 

subgroups) that is likely to be without an appreciable risk of deleterious noncancer effects during 

a lifetime. 

Reference Dose (RfD) - An estimate (with uncertainty spanning perhaps an order of magnitude) 

of a daily exposure to the human population (including sensitive subgroups) that is likely to be 

without an appreciable risk of deleterious noncancer effects during a lifetime. 

Response additivity - In a mixture, when the toxic response (rate, incidence, risk, or probability 

of effects) from the combination is equal to the conditional sum of component responses as 

defined by the formula for the sum of independent event probabilities. For two chemical 

mixtures, for example, the body’s response to the first chemical is the same whether or not the 

second chemical is present. 

Risk - Absolute risk: The probability of injury, disease, or death under specific circumstances. In 

quantitative terms, risk is expressed in values ranging from zero (representing the certainty that 

there is no chance of harm), to one (representing the certainty that harm will occur). Incremental 

risk: The probability of injury, disease, or death under specific circumstances, relative to the 

background probability. In quantitative terms, risk is expressed in values ranging from zero 

(representing the certainty that the probability of harm is no greater than the background 

probability), to one (representing the certainty that harm will occur). 

Stakeholder - An interested or affected party in an ongoing or contemplated project (usually 

involving a group or team planning the project, analyzing one or more problems, and making 

decisions for possible actions based on the interpretation of that analysis). 

65
 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1 Stressor - Any physical, chemical, or biological entity that can induce an adverse response. 

2 Stressors may also be the lack of an essential entity, such as a habitat. 

66
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

6. REFERENCES 

ACGIH, 1998. Threshold Limit Values for Chemical Substances and Physical Agents. 

American Conference of Government Industrial Hygienists, Cincinnati, OH. 

AIChE, 1992. Guidelines for Hazard Evaluation Procedures, 2nd Edition with Worked 

Examples. Center for Chemical Process Safety, American Institute of Chemical Engineers, New 

York, NY. 

AIChE, 1996. Guidelines for Use of Vapor Cloud Dispersion Models, 2nd Ed. Center for 

Chemical Process Safety, American Institute of Chemical Engineers, New York, NY. 

AIHA, 2000. Emergency Response Planning Guidelines Series. Emergency Response Planning 

Guidelines Committee, American Industrial Hygiene Association, Fairfax, VA. 

Albert, Roy E., 1999. Unifying the standard setting process for carcinogens and non-carcinogens. 

Applied Occupational and Environmental Hygiene 14:742-747 

Albert, Roy E., Joellen Lewtas, Stephen C. Nesnow, Todd W. Thorslund, and Elizabeth L. 

Anderson, 1983. Comparative potency method for cancer risk assessment application to diesel 

particulate emissions. Risk Analysis 3:101-117 

American Heart Association, 2000. Stroke Risk Factors. Internet: www.americanheart.org. 

Andersen, Melvin E., M. Elizabeth Meeks, Gary A. Boorman, David J. Brusick, Samuel M. 

Cohen, Yvonne P. Dragan, Clay B. Frederick, Jay I. Goodman, Gordon C. Hard, Ellen J. 

O’Flaherty, and Denise E. Robinson, 2000. Lessons learned in applying the U.S. EPA proposed 

cancer guidelines to specific compounds. Toxicological Sciences 53:159-172 

APCA, 1999. “Cumulative & Aggregate Risk Evaluation System: CARES. Conceptual Model.” 

American Crop Protection Association. Washington, DC. Internet: www.alphacares.org/ 

ATSDR, 1995. Report of the Expert Panel Workshop on the Psychological Responses to 

Hazardous Substances. Agency for Toxic Substances and Disease Registry, U.S. Department of 

Health and Human Services. Atlanta, Georgia. Internet: www.atsdr.cdc.gov/HEC/PRHS/ 

Barnes, Donald G., Ann Alford-Stevens, Linda Birnbaum, Frederick W. Kutz, William Wood, 

and Dorothy Patton, 1991. Toxic equivalency factors for PCBs? Quality Assurance 1:70-81. 

Barnes, Donald G. and Michael L. Dourson, 1988. Reference dose (RfD): description and use in 

health risk assessments. Regulatory Toxicology and Pharmacology 8:471-486. 

Barnthouse, Lawrence W., David R. Marmorek, and Calvin N. Peters, 2000. Assessment of 

multiple stresses at regional scales. In: Multiple Stressors in Ecological Risk and Impact 

Assessment: Approaches to Risk Estimation. (Ferenc and Foran, Eds.) Society of Environmental 

67
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Toxicology and Chemistry, SETAC Press, Pensacola, Florida. ISBN 1-880611-40-6
 

Beck, B.D., R.B. Connolly, M.L. Dourson, D. Guth, D. Hattis, C. Kimmel, S.C. Lewis, eds.
 

Fundam. Appl. Toxicol. 20:9-12.
 

Berenblum, Isaac, and Philippe Shubik, 1947. A new quantitative approach to the study of stages
 

of chemical carcinogenesis in the mouse’s skin. British Journal of Cancer 1:384-391
 

Bogen, K. T., 2001. “Methods for Addressing Uncertainty and Variability to Characterize
 

Potential Health Risk from Trichloroethylene-Contaminated Ground Water at Beale Air Force
 

Base in California: Integration of Uncertainty and Variability in Pharmacokinetics and Dose-


Response.” Lawrence Livermore National Laboratory, U.S. Department of Energy, Livermore,
 

CA. UCRL-ID-135978 Rev 1
 

Bouwes, Nicolaas W. and Steven M. Hassur, 1998. “OPPT’s Risk-Screening Environmental
 

Indicators: Toxic Weights for Toxic Release Inventory (TRI) Chemicals and Chemical
 

Categories.” Office of Pollution Prevention and Toxic Substances, Office of Prevention,
 

Pesticides, and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC.
 

April 28, 1998
 

Bullard, Robert D., 1990. Dumping in Dixie: Race, Class, and Environmental Quality.
 

Westview Press, Boulder, CO. ISBN: 0-8133-7954-7
 

Carpy, Serge A, Werner Kobel, and John Doe, 2000. Health risk of low-dose pesticides mixtures:
 

A review of the 1985-1998 literature on combination toxicology and health risk assessment.
 

Journal of Toxicology and Environmental Health Part B, 3:1-25
 

CEQ, 1997. “Considering Cumulative Effects Under the National Environmental Policy Act.”
 

Council on Environmental Quality, Executive Office of the President, Washington, DC. 
 

Chambers, JM, WS Cleveland, B Kleiner, PA Tukey. 1983. Graphical methods for data analysis.
 

Wadsworth International Group, Belmont, CA, 145-171.
 

Chess, C. and K. Purcell (1999). Public Participation and the Environment: Do We Know What
 

Works? Environmental Science & Technology 33(16): 2685-2692.
 

Clemen, Robert T., 1996. Making Hard Decisions: An Introduction to Decision Analysis. 2nd Ed.
 

Duxbury Press, Wadsworth Publishing Co., Belmont, CA. ISBN 0-534-26034-9
 

Cogliano, Vincent James, 1997. Plausible upper bounds: Are their sums plausible? Risk Analysis
 

17:77-84
 

Cohen, Bernard L., 1991. Catalog of risks extended and updated. Health Physics 61:317-335.
 

DOT, 1998. “High-Speed Ground Transportation Noise and Vibration Impact Assessment.” Final
 

Draft. Office of Railroad Development, Federal Railroad Administration, U.S. Department of
 

68
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Transportation. Washington, DC. Report No. 293630-1 Internet: 

http://project1.parsons.com/ptgnechsr/noise_manual.htm 

Dourson, M.L., S.P. Felter, and D. Robinson. 1996. Evolution of science-based uncertainty 

factors in noncancer risk assessment. Regulatory Toxicol. Pharmacol. 24, 108-120. 

Dourson, Michael L., Linda K. Teuschler, Patrick R. Durkin, and William M. Stiteler, 1997. 

Categorical regression of toxicity data: A case study using aldicarb. Regulatory Toxicology and 

Pharmacology 25:121-129. 

Eadon, George A., Laurence S. Kaminsky, Jay Silkworth, K. Aldous, D. Hilker, P. O'Keefe, R. 

Smith, J. Gierthy, J. Hawley, N. Kim, et al., 1986. Calculation of 2,3,7,8-TCDD equivalent 

concentrations of complex environmental contaminant mixtures. Environmental Health 

Perspectives 70:221-227 

EEA, 2001. “Late Lessons from Early Warnings: The Precautionary Principle 1896-2000.” 

European Environment Agency, Copenhagen. Environmental Issue Report No. 22 (Released 10 

January 2002) Internet: http://reports.eea.eu.int/environmental_issue_report_2001_22/en 

Environmental Defense, 2001. “Scorecard.” Internet: http://www.scorecard.org 

European Commission, 2000. “The European Multi-Hazard Risk Assessment Project 

(T.E.M.R.A.P.)” Directorate General XII for Science Research and Development, Environment 

and Climate - 1994/1998 - Climatology and Natural Hazard, European Commission. Brussels, 

Belgium. Internet: http://phypc9.geo.ulg.ac.be/nouveau/temrap/MainWebPage.htm 

Ferenc, Susan A. and Jeffrey A. Foran (Eds.), 2000. Multiple Stressors in Ecological Risk and 

Impact Assessment: Approaches to Risk Estimation. Society of Environmental Toxicology and 

Chemistry, SETAC Press, Pensacola, Florida. ISBN 1-880611-40-6 

FIFRA SAP, 2000. “Cumulative Risk Assessment Methodology Issues of Pesticide Substances 

that Have a Common Mechanism of Toxicity.” Report from Session II of the FIFRA Scientific 

Advisory Panel Meeting of December 9, 1999 (Report dated February 4, 2000). FIFRA 

Scientific Advisory Panel, Office of Science Coordination and Policy, Office of Prevention, 

Pesticides and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. SAP 

Report 99-06B 

Fischhoff, Baruch, Watson, S., and Hope, C., 1984. Defining risk. Policy Sciences 17:123-139 

Foran, Jeffrey A. and Susan A. Ferenc (Eds.), 1999. Multiple Stressors in Ecological Risk and 

Impact Assessment. Society of Environmental Toxicology and Chemistry, SETAC Press, 

Pensacola, Florida. ISBN 1-880611-32-5 

Frewer, L. (1999). Risk Perception, Social Trust, and Public Participation in Strategic Decision 

Making: Implications for Emerging Technologies. Ambio 28(6): 569-574 

69
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Gandolfi, A.J., I.K. Brendel, R.L. Fisher, and J-P Michaud, 1995. Use of tissue slices in chemical 

mixtures toxicology and interspecies investigations. Toxicology 105:285-290 

GAO, 1983. “Siting of Hazardous Waste Landfills and Their Correlation with Racial and 

Economic Status of Surrounding Communities.” U.S. General Accounting Office. Washington, 

DC. GAO/RCED 83-168 

Guth, D. J., R. C. Hertzberg , and A. M. Jarabek. 1993. Exposure-response analysis: modeling 

severity against concentration and duration. In: “Improvements in Quantitative Noncancer Risk 

Assessment: Symposium Overview,” Society of Toxicology meeting; February 1992; Seattle, 

WA. 

Haber, F., 1924. Zur Geschichte des Gaskrieges [On the History of Gas Warfare], in: Funf 

Vortrage aus den Jahren 1920-1923 (Five Lectures from the years 1920-1923), Springer, Berlin, 

76-92. 

Hampshire Research Institute, 1999. “Review of an Aggregate Exposure Assessment Tool.” 

Presentation to the U.S. EPA FIFRA Scientific Advisory Panel, September, 1999. Hampshire 

Research Institute, Alexandria, VA. Internet: 

www.epa.gov/scipoly/sap/1999/september/aggbkgd.pdf 

Hampshire Research Institute, 2000. “Overview of the Fundamentals of Version 1.0 of LifeLine 

Software for Modeling Aggregate and Cumulative Exposures to Pesticides.” Hampshire 

Research Institute, Alexandria, VA. Internet: 

www.epa.gov/scipoly/sap/2000/september/final_fundamentals.pdf 

Hansen, Hugh, Christopher T. De Rosa, Hana Pohl, Michael Fay, and Moiz M. Mumtaz, 1998. 

Public health challenges posed by chemical mixtures. Environ Health Perspect 106:1271-1280. 

Hertzberg, Richard C. 1989. Fitting a model to categorical response data with application to 

species extrapolation. Health Physics, 57(Supplement 1):405-409. 

Hertzberg, Richard C., 2000. “Communicating cumulative risk: sound bites from chaos.” 

Luncheon address, presented at the conference, Toxicology and Risk Assessment Approaches for 

the 21st Century, April 10-13, 2000, Kings Island, OH. 

Hertzberg, Richard C, and M.M. MacDonell, 2002. Synergy and other ineffective mixture risk 

definitions. The Science of the Total Environment 288:31-42. 

Hertzberg, Richard C., Glenn Rice, and Linda K. Teuschler, 1999. “Methods for health risk 

assessment of combustion mixtures.” In: Hazardous Waste Incineration: Evaluating the Human 

Health and Environmental Risks. S. Roberts, C. Teaf and J. Bean, eds. CRC Press LLC, Boca 

Raton. Pages 105-148 

Hertzberg, Richard C. and Linda K. Teuschler, 2001. “Ideas for Evaluating Quantitative 

70
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Formulas for Dose-Response Assessment of Chemical Mixtures,” presented at the Application of 

Technology to Chemical Mixture Research Conference, January 9-11, 2001, Fort Collins, CO. 

HUD, 1991. “The Noise Guidebook.” Office of Community Planning and Development, U.S. 

Department of Housing and Urban Development. Washington, DC. HUD-953-CPD(1) 

ILSI, 1998. Aggregate Exposure Assessment. International Life Sciences Institute, Washington, 

DC. ISBN 1-57881-040-X 

ILSI, 1999. A Framework for Cumulative Risk Assessment. International Life Sciences Institute, 

Washington, DC. ISBN 1-57881-055-8 

ILSI, 2000. Revised Framework for Microbial Risk Assessment. International Life Sciences 

Institute, Washington, DC. ISBN: 1-57881-081-7 

ILSI, 2001. Aggregate Exposure Assessment: Model Evaluation and Refinement Workshop 

Report. International Life Sciences Institute, Washington, DC. ISBN 1-57881-120-1 

Johnson, Ted, Gary Mihlan, Jacky LaPointe, Kris Fletcher, Jim Capel, Arlene Rosenbaum, 

Jonathan Cohen, and Pat Stiefer, 2000. “Estimation of Carbon Monoxide Exposures and 

Associated Carboxyhemoglobin Levels for Residents of Denver and Los Angeles using 

pNEM/CO (version 2.1).” Draft report prepared for Office of Air Quality Planning and 

Standards, Office of Air and Radiation, U.S. Environmental Protection Agency. Research 

Triangle Park, NC. Contract No. 68-D6-0064. 

Kodell, Ralph L., and James J. Chen, 1994. Reducing conservatism in risk estimation for 

mixtures of carcinogens. Risk Analysis 14:327-332 

Kroschwitz, Jacqueline I. and Mary Howe-Grant (Eds.), 1994. Kirk-Othmer Encyclopedia of 

Chemical Technology. 4th Edn. John Wiley and Sons, New York. ISBN 0-47152-677-0 

Lantz, Paula M, James S. House, James M. Lepkowski, David R. Williams, Richard P. Mero and 

Jieming Chen, 1998. Socioeconomic factors, health behaviors, and mortality: Results from a 

nationally representative prospective study of US adults. Jour. Amer. Medical Assoc. 279:1703-

1709. 

Lewis, S.C. 1993. Reducing uncertainty with adjustment factors: Improvements in quantitative 

noncancer risk assessment. Fund. Appl. Toxicol. 20, 2-4. 

Lewtas, Joellen, 1985. Development of a comparative potency method for cancer risk 

assessment of complex mixtures using short-term in vivo and in vitro bioassays. Toxicol. Ind. 

Health 1:193-203 

Lewtas, Joellen, 1988. Genotoxicity of complex mixtures: Strategies for the identification and 

71
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

comparative assessment of airborne mutagens and carcinogens from combustion sources. 

Fundamentals of Applied Toxicology 10:571-589 

Lynch, John W., George A. Kaplan, Elsie R. Pamuk, Richard D. Cohen, Katherine E. Heck, 

Jennifer L. Balfour, and Irene H. Yen, 1998. Income inequality and mortality in metropolitan 

areas of the United States. Am. J. Public Health 88:1074-1080 

Morata, Thais C., 2000. “Suggested Guidelines to Studying the Combined Effects of 

Occupational Exposure to Noise and Chemicals on Hearing.” National Institute for Occupational 

Safety and Health. Cincinnati, OH. Dated February, 2000. Internet: 

http://www.ucl.ac.uk/noiseandhealth/WP3%20draft%20protocol%20-%20Morata.doc 

Morata, T.C., A.C. Fiorini, F.M. Fischer, S. Colacioppo, K.M. Wallingford, E.F. Kreig, D.E. 

Dunn, L. Gozzoli, M.A. Padrão, and C.L.G. Cesar, 1997. Toluene-induced hearing loss among 

rotogravure printing workers. Scandanavian Journal of Work, Environment & Health 23:289-298 

Morgan, M. Granger, and Max Henrion, 1990. Uncertainty: A Guide to Dealing with Uncertainty 

in Quantitative Risk and Policy Analysis. Cambridge University Press, New York, NY. ISBN 0-

521-36542-2 

Mottram, J.C., 1944. A developing factor in experimental blastogenesis. J. Pathol. Bacteriol. 

56:181-187 

Murray, C., 1994. Quantifying the burden of disease: the technical basis for disability-adjusted 

life years. Bull. World Health Org. 72 (3):429-445 

NAPA, 1995. Setting Priorities, Getting Results: A New Direction for EPA. National Academy 

of Public Administration. Washington, DC. LCCN: 95-68048 

NOAA, 1999. “An Assessment of Lower Snake River Hydrosystem Alternatives on Survival and 

Recovery of Snake River Salmonids.” National Marine Fisheries Service, National Oceanic and 

Atmospheric Administration. Seattle, WA. Dated April 14, 1999 

NRC, 1983. Risk Assessment in the Federal Government: Managing the Process. Committee on 

the Institutional Means for Assessments of Risk to Public Health, Commission on Life Sciences, 

National Research Council. National Academy Press, Washington, DC. ISBN 0-309-03349-7 

NRC, 1993. Pesticides in the Diets of Infants and Children. National Academy Press, 

Washington DC. 

NRC, 1994. Science and Judgment in Risk Assessment. Committee on Risk Assessment of 

Hazardous Air Pollutants, Board on Environmental Sciences and Technology, Commission on 

Life Sciences, National Research Council. National Academy Press, Washington, DC. ISBN 0-

72
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

309-04894-X 

NRC, 1996. Understanding Risk: Informing Decisions in a Democratic Society. Committee on 

Risk Characterization, Commission on Behavioral and Social Sciences and Education, National 

Research Council. National Academy Press, Washington, DC. ISBN 0-309-05396-X 

OMB, 2002. "Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by Federal Agencies" Federal Register 67:369-378 

Osowski, Sharon L., Joseph D. Swick, Jr., Gerald R. Carney, Hector B. Pena, Jeffrey E. 

Danielson, and David A. Parrish, 2001. A watershed-based cumulative risk impact analysis: 

Environmental vulnerability and impact criteria. Environmental Monitoring and Assessment 

66:159-185 

Perry, Robert H., Don W. Green, and James O. Maloney, Eds., 1997. Perry’s Chemical 

Engineers’ Handbook. 7th Edn. McGraw Hill Professional Publishing, New York. ISBN 0-

07049-841-5 

Portier, Christopher J., 2001. “Virtual Human Concept and its Application to Chemical Mixture 

Research,” presented at Application of Technology to Chemical Mixture Research Conference, 

January 9-11, 2001, Fort Collins, CO. 

Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997. Risk 

Assessment and Risk Management in Regulatory Decision-Making. Washington, DC. 

Rouse, P. and J.G. Kidd, 1941. Conditional neoplasms and sub-threshold neoplastic states: A 

study of the tar tumors of rabbits. J. Exp. Med. 73:369-390 

Seed, Jennifer, Ronald P. Brown, Stephen S. Olin, and Jeffery A. Foran, 1995. Chemical 

mixtures: Current risk assessment methodologies and future directions. Regulatory Toxicology 

and Pharmacology 22:76-94 

Sexton, Ken, David E. Kleffman & Michael A. Callahan, 1995. An introduction to the National 

Human Exposure Assessment Survey (NHEXAS) and related phase I field studies. J. Expos. 

Anal. Environ. Epidem. 5, 229-232. Related papers are in the same issue of J. Expos. Anal. 

Environ. Epidem. 5, 233-444. 

Slob, W., and M.N. Pieters. 1998. A probabilistic approach for deriving acceptable human intake 

limits and human health risks from toxicological studies: General framework. Risk Anal. 18, 787-

798. 

State of Vermont, 1991. “Environment 1991: Risks to Vermont and Vermonters.” Agency of 

Natural Resources, State of Vermont. Waterbury, VT. 

Teuschler, Linda K., Michael L. Dourson, William M. Stiteler, Peter McClure, and Heather 

73
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Tully, 1999. Health risks above the reference dose for multiple chemicals. Regulatory 

Toxicology and Pharmacology 30:S19-S26. 

Thomas, J. (1995). Public Participation in Public Decisions: New Skills and Strategies for 

Public Managers. Jossey-Bass, San Francisco 

United Church of Christ, 1987. Toxic Waste and Race in the United States: A National Report on 

the Racial and Socio-Economic Characteristics of Communities with Hazardous Waste Sites. 

United Church of Christ Commission for Racial Justice. New York, NY. 

USEPA, 1986a. “The Risk Assessment Guidelines of 1986.” Risk Assessment Forum, Office of 

Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

EPA/600/8-87/045 

USEPA, 1986b. “Guidelines for the Health Risk Assessment of Chemical Mixtures.” Risk 

Assessment Forum, Office of Research and Development, Environmental Protection Agency. 

Washington, DC. EPA/630/R-98/002 

USEPA, 1986c. “Guidelines for Mutagenicity Risk Assessment.” Risk Assessment Forum, 

Office of Research and Development, Environmental Protection Agency. Washington, DC. 

EPA/630/R-98/003 

USEPA, 1986d. “Guidelines for Carcinogen Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development, Environmental Protection Agency. Washington, DC. EPA/630/R98/001 

USEPA, 1987. “The Total Exposure Assessment Methodology (TEAM) Study.” Office of Acid 

Deposition, Environmental Monitoring and Quality Assurance, Office of Research and 

Development, U.S. Environmental Protection Agency. Washington, DC. EPA/600/6-87/002 

USEPA, 1988. “Federal Guidance Report No. 11: Limiting Values of Radionuclide Intake and 

Air Concentration and Dose Conversion Factors for Inhalation, Submersion, and Ingestion.” 

Office of Radiation Programs, U.S. Environmental Protection Agency. Washington, DC. EPA520/1-88-020 

USEPA, 1989a. “Risk Assessment Guidance for Superfund, Volume 1: Human Health 

Evaluation Manual. Office of Emergency and Remedial Response, Office of Solid Waste and 

Emergency Response, U.S. Environmental Protection Agency. Washington, DC. EPA/540/1-

89/002 

USEPA, 1989b. “Interim Procedures for Estimating Risks Associated with Exposure to 

Chlorinated dibenzo-p-dioxins and -dibenzofurans (CDDs and CDFs) and 1989 update. Risk 

Assessment Forum, Office of Research and Development, U.S. Environmental Protection 

Agency. Washington, DC. EPA 625/3-89/016 

74
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

USEPA, 1990a. “Technical Support Document on Health Assessment of Chemical Mixtures.” 

Office of Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

EPA/600/8-90/064 

USEPA, 1990b. “Reducing Risk: Setting Priorities and Strategies for Environmental Protection.” 

Science Advisory Board, U.S. Environmental Protection Agency. Washington, DC. SAB-EC-90-

021 

USEPA, 1991a. “Guidelines for Developmental Toxicity Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development, U.S. Environmental Protection Agency. 

Washington, DC. EPA/600/FR-91/001 

USEPA, 1991b. “Locational Data Policy.” U.S. Environmental Protection Agency. Washington, 

DC. IRM Policy Manual 2100-CGH2, dated April 8, 1991 

USEPA, 1992a. “Safeguarding the Future: Credible Science, Credible Decisions. A Report of the 

Expert Panel on the Role of Science at EPA.” Washington, DC. EPA/600/9-91/050 

USEPA, 1992b. “Framework for Ecological Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

EPA/630/R-92/001 

USEPA, 1992c. “Guidelines for Exposure Assessment.” Risk Assessment Forum, Office of 

Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

EPA/600/Z-92/001 

USEPA, 1993a. “Guidance on the Application of Refined Dispersion Models for 

Hazardous/Toxic Air Release.” Office of Air Quality Planning and Standards, Office of Air and 

Radiation, U.S. Environmental Protection Agency, Research Triangle Park, NC. EPA 454/R-93-

002 

USEPA, 1993b. “A Guidebook to Comparing Risks and Setting Environmental Priorities.” 

Office of Policy, Planning and Evaluation, U.S. Environmental Protection Agency. Washington, 

DC. EPA 230-B-93-003 

USEPA, 1993c. “Chemical Indexing System for the Toxic Chemical Release Inventory, Part I: 

Chronic Index.” Air, Radiation and Toxics Division, EPA Region III, U.S. Environmental 

Protection Agency. Philadelphia, PA. EPA/903/R-93/002 

USEPA, 1993d. “Federal Guidance Report No. 12: External Exposure to Radionuclides in Air, 

Water, and Soil.” Office of Air and Radiation, U.S. Environmental Protection Agency. 

Washington, DC. EPA-402-R-93-081 

USEPA, 1994. “Methods for Derivation of Inhalation Reference Concentrations and Application 

of Inhalation Dosimetry.” Office of Health and Environmental Assessment, Office of Research 

75
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

and Development, U.S. Environmental Protection Agency, Research Triangle Park, NC. 

EPA/600/8-90/066F 

USEPA, 1995a. “Policy for Risk Characterization.” Memorandum from U.S. Environmental 

Protection Agency Administrator Carol M. Browner, dated March 21, 1995. Washington, DC. 

USEPA, 1995b. “Guidance for Risk Characterization.” Policy paper dated February, 1995. 

Science Policy Council, U.S. Environmental Protection Agency. Washington, DC. 

USEPA, 1995c. “The Use of the Benchmark Dose Approach in Health Risk Assessment.” Risk 

Assessment Forum, Office of Research and Development, U.S. Environmental Protection 

Agency. Washington, DC. EPA/630/R-94/007 

USEPA, 1995d. “Chemical Indexing System for the Toxic Chemical Release Inventory, Part I: 

Chronic Index; Addendum.” Air, Radiation and Toxics Division, EPA Region III, U.S. 

Environmental Protection Agency. Philadelphia, PA. EPA/903/R-93/002a (August, 1995) 

USEPA, 1995e. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” Fifth Edition. Office of Air Quality Planning and Standards, Office of Air and 

Radiation, U.S. Environmental Protection Agency. Research Triangle Park, NC. EPA AP-42 

USEPA, 1996a. “RAGS Reform Stakeholder Forums: Synopsis of Participants’ Comments. San 

Francisco, California, October 30-November, 1, 1996, and Washington, DC, November 6-

November 8, 1996.” Office of Emergency and Remedial Response, U.S. Environmental 

Protection Agency, Washington, DC. 

USEPA, 1996b. “Guidelines for Reproductive Toxicity Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development, U.S. Environmental Protection Agency. 

Washington, DC. EPA/630/R-96/009 

USEPA, 1996c. “Proposed Guidelines for Carcinogen Risk Assessment.” Risk Assessment 

Forum, Office of Research and Development, U.S. Environmental Protection Agency. 

Washington, DC. EPA/600/P-92/003C 

USEPA, 1996d. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” Fifth Edition, Supplements A & B. Office of Air Quality Planning and Standards, 

Office of Air and Radiation, U.S. Environmental Protection Agency. Research Triangle Park, 

NC. EPA AP-42 

USEPA, 1996e. “Radiation Exposure and Risk Assessment Manual (RERAM).” Office of Air 

and Radiation, U.S. Environmental Protection Agency. Washington, DC. EPA 402-R-96-016 

USEPA, 1997a. “Guidance on Cumulative Risk Assessment, Part 1. Planning and Scoping.” 

Science Policy Council, U.S. Environmental Protection Agency, Washington, DC. Attachment to 

memo dated July 3, 1997 from the Administrator, Carol Browner, and Deputy Administrator, 

76
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Fred Hansen, titled “Cumulative Risk Assessment Guidance-Phase I Planning and Scoping.” 

USEPA, 1997b. “Guiding Principles for Monte Carlo Analysis.” Risk Assessment Forum, 

Office of Research and Development, Washington, DC. EPA/630/R-97/001 

USEPA, 1997c. “Chemical Indexing System, Part II: Vulnerability Index.” Waste and Chemicals 

Management Division, EPA Region III, U.S. Environmental Protection Agency. Philadelphia, 

PA. EPA/903/R-97/021 

USEPA, 1997d. “Compilation of Air Pollutant Emission Factors, Volume I: Stationary Point and 

Area Sources.” Fifth Edition, Supplement C. Office of Air Quality Planning and Standards, 

Office of Air and Radiation, U.S. Environmental Protection Agency. Research Triangle Park, 

NC. EPA AP-42 

USEPA, 1998a. “Risk Assessment Guidance for Superfund (RAGS) Stakeholder Forum: 

Synopsis of Participants’ Comments. Atlanta, Georgia, March 2-4, 1998.” Office of Emergency 

and Remedial Response, U.S. Environmental Protection Agency, Washington, DC. 

USEPA, 1998b. “Guidelines for Ecological Risk Assessment.” Risk Assessment Forum, Office 

of Research and Development, Washington, DC. EPA/630/R-95/002F 

USEPA, 1998c. “Guidelines for Neurotoxicity Risk Assessment.” Risk Assessment Forum, 

Office of Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

EPA/630/R-95/001F 

USEPA, 1998d. “Methods for Exposure-Response Analysis for Acute Inhalation Exposure to 

Chemicals: Development of the Acute Reference Exposure.” Review Draft. Office of Research 

and Development, U.S. Environmental Protection Agency. Washington, DC. EPA/600/R-98/051 

USEPA, 1998e. “General Guidance for Risk Management Programs (40 CFR Part 68).” 

Chemical Emergency Preparedness and Prevention Office, Office of Solid Waste and Emergency 

Response, U.S. Environmental Protection Agency, Washington, DC. EPA 550-B-98-003 

USEPA, 1998f. “Comparative Risk Framework: Methodology and Case Study.” SAB Review 

Draft dated November 9, 1998. National Center for Environmental Assessment, Office of 

Research and Development, U.S. Environmental Protection Agency. Cincinnati, OH. 

USEPA, 1998g. “Report of the Common Sense Initiative Council’s Stakeholder Involvement 

Work Group.” Common Sense Initiative Council, U.S. Environmental Protection Agency. 

Washington, DC. 

USEPA, 1998h. “An SAB Report: Review of Disproportionate Impact Methodologies.” Science 

Advisory Board, U.S. Environmental Protection Agency. Washington, DC. EPA-SAB-IHEC-99-

77
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

007 

USEPA, 1998i. “Handbook for Air Toxics Emission Inventory Development, Volume I: 

Stationary Sources.” Office of Air Quality Planning and Standards, Office of Air and Radiation, 

U.S. Environmental Protection Agency. Research Triangle Park, NC. EPA-454/B-98-002 

USEPA, 1998j. “Human Health Risk Assessment Protocol for Hazardous Waste Combustion 

Facilities.” Peer Review Draft. Office of Solid Waste and Emergency Response, U.S. 

Environmental Protection Agency. Washington, DC. EPA530-D-98-001 Internet: 

www.epa.gov/epaoswer/hazwaste/combust/riskvol.htm 

USEPA, 1998k. “A Common Mechanism of Action: The Organophosphate Pesticides.” 

Scientific Advisory Panel, United States Environmental Protection Agency. Washington, DC. 

Internet: http://www.epa.gov/scipoly/sap/1998/march/comec.htm 

USEPA, 1999a. “Guidelines for Carcinogen Risk Assessment.” Review draft dated July, 1999. 

Risk Assessment Forum, Office of Research and Development, U.S. Environmental Protection 

Agency. Washington, DC. NCEA-F-0644 

USEPA, 1999b. “Review of Revised Sections of the Proposed Guidelines for Carcinogen Risk 

Assessment.” Science Advisory Board, U.S. Environmental Protection Agency. Washington, DC. 

EPA-SAB-EC-99-015 

USEPA, 1999c. “Risk Assessment Guidance for Superfund: Volume 1 - Human Health 

Evaluation Manual. Supplement to Part A: Community Involvement in Superfund Risk 

Assessments.” Office of Solid Waste and Emergency Response, United States Environmental 

Protection Agency. Washington, DC. EPA 540-R-98-042/PB99-963303 

USEPA, 1999d. “Risk Management Program Guidance for Offsite Consequence Analysis.” 

Chemical Emergency Preparedness and Prevention Office, Office of Solid Waste and Emergency 

Response, U.S. Environmental Protection Agency, Washington, DC. EPA 550-B-99-009 

USEPA, 1999e. “Guideline on Air Quality Models.” Office of Air Quality Planning and 

Standards, Office of Air and Radiation, U.S. Environmental Protection Agency. Research 

Triangle Park, NC. 40CFR Ch. I (7-1-99 Edition), Appendix W to Part 51, Pages 390-481 

USEPA, 1999f. “EPA’s Framework for Community-Based Environmental Protection.” Office of 

Policy/Office of Reinvention, U.S. Environmental Protection Agency, Washington, DC. EPA 

237-K-00-001 

USEPA, 1999g. “Guidance for Performing Aggregate Exposure and Risk Assessments.” Office 

of Pollution Prevention and Toxic Substances, Office of Pesticide Programs, U.S. Environmental 

Protection Agency. Washington, DC. Item 6043, dated October 29, 1999. Available on the 

internet at: http://www.epa.gov/fedrgstr/EPA-PEST/1999/November/Day-10/6043.pdf 

78
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

USEPA, 1999h. “Total Risk Integrated Methodology, TRIM.Expo.: Technical Support 

Document.” External Review Draft. Office of Air Quality Planning and Standards, Office of Air 

and Radiation, U.S. Environmental Protection Agency. Research Triangle Park, NC. EPA453/D-99-001 

USEPA, 1999i. “Risk-Screening Environmental Indicators: 1988-1997 TRI Data ‘Air-Only’ 

Model.” Office of Pollution Prevention and Toxic Substances, Office of Prevention, Pesticides, 

and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. CD-ROM 

Version 1.0, dated July 6, 1999. (Version 2.0 is now in beta-testing.) 

USEPA, 1999j. “An SAB Report on the National Center for Environmental Assessment’s 

Comparative Risk Framework Methodology.” Science Advisory Board, U.S. Environmental 

Protection Agency. Washington, DC. EPA-SAB-DWC-99-016 

USEPA, 1999k. “Handbook for Criteria Pollutant Inventory Development: A Beginner’s Guide 

for Point and Area Sources.” Office of Air Quality Planning and Standards, Office of Air and 

Radiation, U.S. Environmental Protection Agency. Research Triangle Park, NC. EPA-454/R-99-

USEPA, 1999l. “Federal Guidance Report No. 13: Cancer Risk Coefficients for Environmental 

Exposure to Radionuclides.” Office of Air and Radiation, U.S. Environmental Protection 

Agency. Washington, DC. EPA-402-R-99-001 

USEPA, 1999m. “Guidance for Identifying Pesticide Chemicals and Other Substances that Have 

a Common Mechanism of Toxicity;” Office of Pesticide Programs, Office of Prevention, 

Pesticides, and Toxic Substances, United States Environmental Protection Agency. Washington, 

D.C. Internet: http://www.epa.gov/fedrgstr/EPA_PEST/1999/February/Day_05/ 

USEPA, 2000a. “Toward Integrated Environmental Decision-Making.” Science Advisory Board, 

U.S. Environmental Protection Agency. Washington, DC. EPA-SAB-EC-00-011 

USEPA, 2000b. “Benchmark Dose Technical Guidance Document” Draft report. Risk 

Assessment Forum, Office of Research and Development, U.S. Environmental Protection 

Agency. Washington, DC. EPA/630/R-00/001 

USEPA, 2000c. “Science Policy Council Handbook: Risk Characterization.” Science Policy 

Council, U.S. Environmental Protection Agency. Washington, DC. EPA 100-B-00-002 

USEPA, 2000d. “Science Policy Council Handbook: Peer Review.” 2nd Edition. Science Policy 

Council, U.S. Environmental Protection Agency. Washington, DC. EPA 100-B-00-001 

USEPA, 2000e. “Supplementary Guidance for Conducting Health Risk Assessment of Chemical 

79
 

037 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Mixtures.” Risk Assessment Forum, Office of Research and Development, U.S. Environmental 

Protection Agency. Washington, DC. EPA/630/R-00/002 

USEPA, 2000f. “Baltimore Community Environmental Partnership Air Committee Technical 

Report. Community Risk-Based Air Screening: A Case Study in Baltimore, MD.” Office of 

Pollution Prevention and Toxics, Office of Prevention, Pesticides, and Toxic Substances, U.S. 

Environmental Protection Agency. Washington, DC. EPA 744-R-00-005 

USEPA, 2000g. “Handbook for Non-Cancer Health Effects Valuation.” Non-Cancer Health 

Effects Valuation Subcommittee of the EPA Social Science Discussion Group, Science Policy 

Council, U.S. Environmental Protection Agency. Washington, DC. Dated November, 2000. 

USEPA, 2000h. “AP-42: Compilation of Air Pollutant Emission Factors, Volume II: Mobile 

Sources.” Office of Transportation and Air Quality, Office of Air and Radiation. Washington, 

DC. EPA AP-42, Volume II Internet: www.epa.gov/otaq/ap42.htm 

USEPA, 2000i. “Catalog of Hazardous and Solid Waste Publications.” 13th Edition. Office of 

Solid Waste and Emergency Response, U.S. Environmental Protection Agency. Washington, DC. 

EPA530-B-00-001 Internet: www.epa.gov/epaoswer/osw/catalog.htm 

USEPA, 2000j. “Guide to Field Storage of Biosolids, Appendix A: Odor Characterization, 

Assessment and Sampling.” Office of Wastewater Management, Office of Water, U.S. 

Environmental Protection Agency. Washington, DC. EPA/832-B-00-007 Internet: 

www.epa.gov/owm/bio/fsguide/ 

USEPA, 2001a. Personal communication. Debby Sisco,. Biological and Economic Analysis 

Division, Office of Pesticide Programs, Office of Prevention, Pesticides, and Toxic Substances, 

U.S. Environmental Protection Agency. Washington, DC. August 1, 2001 

USEPA, 2001b. Personal communication. Anna Koutlakis, Office of Prevention, Pesticides, and 

Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. August 1, 2001 

USEPA, 2001c. “Stakeholder Involvement & Public Participation at the U.S. EPA: Lessons 

Learned, Barriers, & Innovative Approaches.” Office of Policy, Economics and Innovation, U.S. 

Environmental Protection Agency. Washington, DC. EPA-100-R-00-040. 

USEPA, 2001d. “Top 10 Watershed Lessons Learned.” Office of Wetlands, Oceans and 

Watersheds, Office of Water, U.S. Environmental Protection Agency. Washington, DC. Internet: 

http://www.epa.gov/owow/watersheds/lessons/index.html 

USEPA, 2001e. “National-Scale Air Toxics Assessment for 1996.” SAB Review Draft. Office of 

Air Quality, Planning and Standards, Office of Air and Radiation, U.S. Environmental Protection 

Agency. Washington, DC. EPA-453-R-01-003 

80
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

USEPA, 2002a. “Guidance on Cumulative Risk Assessment of Pesticide Chemicals that Have a 

Common Mechanism of Toxicity.” Office of Pesticide Programs, Office of Prevention, 

Pesticides, and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. 

January 14, 2002. Internet: http://www.epa.gov/pesticides/trac/science/cumulative_guidance.pdf 

USEPA, 2002b. “Lesson Learned on Planning and Scoping for Environmental Risk 

Assessments.” Science Policy Council, U.S. Environmental Protection Agency. Washington, 

DC. January, 2002. Internet: http://www.epa.gov/ORD/spc/2cumrisk.htm. 

USEPA, 2002c. “Exposure and Human Health Reassessment of 2,3,7,8-Tetrachlorodibenzo 

-p-Dioxin (TCDD) and Related Compounds.” National Center for Environmental Assessment, 

Office of Research and Development, U.S. Environmental Protection Agency. Washington, DC. 

[to be published] 

van den Berg, Martin, Linda Birnbaum, Albertus T.C. Bosveld, Björn Brunström, Philip Cook, 

Mark Feeley, John P. Giesy, Annika Hanberg, Ryuichi Hasegawa, Sean W. Kennedy, Timothy 

Kubiak,John Christian Larsen, F.X. Rolaf van Leeuwen, A.K. Djien Liem, Cynthia Nolt, Richard 

E. Peterson, Lorenz Poellinger, Stephen Safe, Dieter Schrenk, Donald Tillitt, Mats Tysklind, 

Maged Younes, Fredrik Wærn, and Tim Zacharewski, 1998. Toxic equivalency factors (TEFs) 

for PCBs, PCDDs, PCDFs for humans and wildlife. Environmental Health Perspectives 

106:775-792 

WHO, 1983. “Guidelines on Studies in Environmental Epidemiology.” Environmental Health 

Criteria 27. International Programme on Chemical Safety. World Health Organization. Geneva. 

EHC 27 

Woodruff, Tracey J., Jane Caldwell, Vincent J. Cogliano, and Daniel A. Axelrad, 2000. 

Estimating cancer risk from outdoor concentrations of hazardous air pollutants in 1990. 

Environmental Research Section A 82:194-206 

Yang, Raymond S.H., 2001. “Application of Computer Modeling to Simple or Complex 

Mixtures,” presented at Application of Technology to Chemical Mixture Research Conference, 

January 9-11, 2001, Fort Collins, CO. 

Zartarian, Valerie G., Halûk Özkaynak, Janet M. Burke, Maria J. Zufall, Marc L. Rigas, and 

Edwin J. Furtaw, Jr, 2000. “A Modeling Framework for Estimating Children’s Residential 

Exposure and Dose to Chlorpyrifos via Dermal Residue Contact and Non-Dietary Ingestion.” 

Environmental Health Perspectives 108:505-514 

81
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX A: RESEARCH AND DEVELOPMENT NEEDS 

The Framework for Cumulative Risk Assessment is intended to provide a basic structure 

for the issues and define key terms and concepts. In some cases, the concepts introduced in the 

Framework report require the application of knowledge and methods that are not currently 

available. The following is a discussion of the needed areas of research and methods 

development, highlighted within the Framework report, that may be most important to an 

evaluation of cumulative risks. This is not intended to be a comprehensive listing of cumulative 

risk assessment research needs. 

EPA and other scientists are currently investigating the use of similar approaches for 

cancer and noncancer assessments. Although we will not discuss this research need here, it 

would be useful to cumulative risk assessment to have similar approaches, and it is a topic of 

current discussion within scientific circles (e.g., Albert, 1999). 

Understanding the Timing of Exposure and its Relationship to Effects 

A key concept in the definition of cumulative risk is that it represents an accumulation of 

risk over time. However, unlike the traditional approach to risk assessment where exposure 

events are summed and averaged over a period of time, cumulative risk assessment will involve 

developing an understanding of how the sequence and timing of exposures influence the ultimate 

risk of effects. For example, for multiple stressors, it is important to understand how prior 

exposures to one or several stressors influence the risks from subsequent exposures to the same 

or different stressors. In addition, it is important to understand the implications of these 

exposures occurring during critical periods of an individual’s life (e.g., important periods of 

development or periods of disease). Several exposure models are under development which 

recognize the need to understand the timing of various exposure events (e.g., Calendex, APEX, 

Lifeline, SHEDS, and CARES/RExY). 

In addition to gaining a better understanding of the sequence and timing of exposures and 

their relationship to effects, it is important to understand how acute, non-lethal exposures from 

accidents contribute to chronic or long-term effects. 

Understanding the Composition and Toxicity of Mixtures 

Chemical mixtures can change or degrade over time and space making the assessment of 

exposure a particular challenge. For cumulative risk assessment, the composition of the mixture 

at the point of contact with the receptor needs to be well characterized. Both measurement 

techniques (at the receptor) and predictive models are applicable in this characterization. 

EPA’s Guidance for the Health Risk Assessment of Chemical Mixtures (USEPA, 2000e) 

presents approaches for combining the toxicities of multiple chemical stressors. These 

approaches necessarily involve a number of simplifying assumptions when the mixtures are 

complex. Although the current methods provide a valuable resource for assessing cumulative 

risks, future cumulative risk assessment will need a more complete understanding of the 

82
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

interactions among chemicals in complex mixtures. Some current research efforts are seeking to 

identify toxicologic principles of joint action that are applicable to mixtures involving many 

chemicals. 

Applying the Risk Factor Approach to Environmental Health Risks 

The risk factor approach has been used in the medical profession to predict the chances of 

individuals developing various diseases. It has proved to be a useful approach not only in 

assessing certain cumulative risks, but also in communicating with patients. In this approach, 

characteristics of a population (e.g., age, ethnicity, personal habits, genetic polymorphisms, prior 

diseases, etc.) are correlated with the incidence of disease. For some diseases (e.g., breast cancer, 

coronary artery disease, stroke) these correlations are well established. However, there are 

substantial data gaps in terms of the role played by exposures to environmental stressors in the 

development of human disease, and correlations of environmental exposures with disease 

outcomes are generally not available. 

Using Biomarkers and Biomonitoring 

The use of biomarkers of exposure or effect holds a great deal of promise for cumulative 

risk assessment. This approach can provide a method to assess stressors in groups. Currently, 

however, this approach is not practicable when considering a large number of diverse stressors, 

since appropriate biomarkers for many types of stressors have not yet been developed. 

Considering Hazards Presented by Non-Chemical Stressors 

Cumulative risk assessment could encompass the interactions of chemical stressors with 

biological stressors, radiological stressors, other physical stressors, socioeconomic stressors and 

lifestyle conditions. In trying to assess all these different types of stressors, it is helpful to 

determine what types of effects the stressors produce, and then to try to group stressors by like 

effects. Ideally, one would like to know the mechanism or mode of action by which various 

stressors cause effects to allow a more refined grouping. Currently, however, there are few 

methods to understand how these disparate stressors interact to result in risk. 

Considering Psychological Stress as Part of Cumulative Risk 

Psychological stress causes both psychological and physiological changes that can be 

measured. Assessing levels of stress and their potential contribution to risk, however, is difficult 

for a variety of reasons. The Agency for Toxic Substances and Disease Registry (ATSDR) began 

the process of identifying research needs in this area through an expert panel workshop held in 

1995. 

Considering All Aspects of Vulnerability 

The issue of the vulnerability of a population can be thought of as having four 

components: susceptibility of individuals, differential exposures, differential preparedness to 

83
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

withstand the insult, and differential ability to recover from effects. Traditional risk assessment 

may consider one or more of these categories but rarely are all considered. The overall 

consideration of all four categories may be more important in cumulative risk assessment than in 

traditional one-chemical assessments. A cumulative risk assessment, for example, may need to 

consider potential combinations of high exposure and high vulnerability across stressors. 

Methods development work is needed in this area. 

Methods for Combining Different Types of Risk 

Another key concept in the definition of cumulative risk assessment is that it represents 

the combined risk from multiple stressors. This implies that, in some cases, it may be necessary 

to combine disparate measures of risk (i.e., different types of effects) to simplify the expression 

of cumulative risks. There have been some attempts to collapse complex arrays of risk into a few 

or even a single measure. These approaches have involved the use of common metrics (e.g., 

Quality Adjusted Life Years, Disability Adjusted Life Years, Loss of Life Expectancy, etc.), 

indices (e.g., Hazard Ranking System, etc.), and the categorization of effects (e.g., as for 

categorical regression). Alternatively, Geographic Information Systems (GIS) and mapping 

techniques can be used to graphically portray integrated information on risks without 

mathematically combining disparate measures. Much methods development work remains to be 

completed in each of these areas. 

Development of Default Values for Cumulative Risk Assessments 

Just as conventional risk assessments use a series of default values for screening or other 

applications, it may be necessary to investigate whether certain defaults need to be established 

specifically for cumulative risk assessments. 

Development of Case Studies and Issue Papers on Specific Cumulative Risk Topics 

The more detailed technical issues and methodologies should be developed as a series of 

issues papers that would augment the Framework report. The level of detail would, of course, 

vary depending on the topic, and may include the generic material from other guidance 

documents. The issues papers (or white papers) should also include details on additional 

approaches to cumulative risk assessment that are currently being explored (including screeninglevel analyses, place-based assessments, comparative risk assessments, NEPA cumulative effects 

analyses, and hazard assessments). In addition, the issues papers could include summaries of 

case studies of cumulative risk projects that would extend the Framework from theoretical to 

practical approaches and applications. 

84
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX B: SELECTED RESOURCES FOR EXPOSURE AND RISK ASSESSMENT 

B.1. Resources Relevant to Chemical Exposures 

EPA Guidelines: 

Most of EPA’s general guidelines are listed in the text box in section 1.1, page 5. 

Air-related sources and activities: 

EPA’s Clearinghouse for Inventories and Emission Factors (CHIEF) website 

(www.epa.gov/ttn/chief/) is an excellent starting place that has many of the relevant 

documents on methods and data for constructing emissions inventories available for 

download. These include Handbook for Criteria Pollutant Inventory Development: A 

Beginner’s Guide for Point and Area Sources (USEPA, 1999k), Handbook for Air Toxics 

Emission Inventory Development, Volume I: Stationary Sources (USEPA, 1998i),and 

Compilation of Air Pollutant Emission Factors (for both stationary and mobile sources) 

(USEPA, 1995e, 1996d, 1997d, 2000h), as well as many other documents and software. 

EPA’s Support Center for Regulatory Air Models (SCRAM) website 

(www.epa.gov/ttn/scram/) provides extensive information on the models discussed in 

Guideline on Air Quality Models (USEPA, 1999e), including downloadable software and 

users guides for many of the models. 

The Ambient Monitoring Technology Information Center (AMTIC) website 

(www.epa.gov/ttn/amtic/) contains information on monitoring programs, monitoring 

methods, and other monitoring-related information. 

The umbrella website for all three of the above is the Technology Transfer Network 

(www.epa.gov/ttn/), which also has other useful information and links in addition to those 

noted above. 

Sources to land, and waste-related activities: 

The EPA Office of Solid Waste and Emergency Response has published an extensive 

catalog summarizing their publications (USEPA, 2000i). They have also published a 

“peer review draft” document called Human Health Risk Assessment Protocol for 

Hazardous Waste Combustion Facilities (USEPA, 1998j) which deals with how to assess 

risks from hazardous waste incinerators. These reports are available on-line. 

Chemical accidents, transportation-related spills: 

There are several steps in assessing an accidental chemical release exposure. The typical 

analytical steps in an overall accidental chemical release risk assessment are process 

analysis, likelihood or frequency of accidents, source term modeling, dispersion or 

85
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

consequence modeling, and the exposure assessment. 

�	 The process analysis is a formal, systematic analysis of the process where a 

chemical is handled to determine the probabilities and consequences of acute, 

catastrophic failures of engineered systems leading to an accidental release of the 

chemical. This analysis is often called a Process Hazards Analysis (PHA). 

Several formal PHA evaluation techniques are available including “What-If,” 

“Failure Mode and Effect Analysis,” “Event-Tree”, and “Fault-Tree” analysis 

(USEPA 1998e, AIChE, 1992). 

�	 The likelihood or frequency of accidents step is an evaluation of each of the 

scenarios uncovered in the process analysis step for likelihood or frequency of 

occurrence. 

�	 Source term modeling, which estimates the amount or rate of release in case of 

accident, is performed once the failure scenarios are determined. A wide variety of 

published calculation methods or models are available (USEPA 1998e, USEPA 

1999d) to determine the source terms for an accidental chemical release. 

�	 Dispersion or consequence modeling is performed once the source terms (rate and 

duration of the release) are known. A wide variety of dispersion and consequence 

modeling tools, ranging from simple screening models to sophisticated and 

complex computer applications, are available for this step (USEPA 1999d, AIChE 

1996, USEPA 1993a). In addition to the source terms generated above, several 

other data elements are needed, such as physical/chemical properties (e.g., 

whether the vapor cloud is heavier than air or water reactive), meteorological 

conditions (e.g., wind speed and direction, temperature, humidity), and terrain 

surrounding the facility (e.g., buildings or valleys that may channel or disperse a 

vapor cloud). Physical/chemical properties can be found in chemical reference 

texts such as Kirk-Othmer’s Encyclopedia of Chemical Technology (Kroschwitz 

and Howe-Grant, 1994), Perry’s Chemical Engineers’ Handbook (Perry, et al., 

1997), on Material Safety Data Sheets (MSDS)14, or in the Guidance for Offsite 

Consequence Analysis (USEPA 1999d). Meteorological conditions are often 

collected on-site or at local airports. Information about terrain can be collected 

from topological maps or by visual inspection. Guidance on all these parameters is 

available in USEPA 1999d. 

The final step in a chemical accident exposure analysis is the exposure assessment. The 

exposure assessment is related to, and builds from, the dispersion or consequence 

modeling step. The dispersion or consequence modeling depends on a health endpoint 

and the exposure level related to that endpoint. Besides lethality, concentrations for 

certain health effects (e.g., odor thresholds, eye irritation) are available for several 

14 There are many searchable MSDS data bases on-line that can be located with most search engines. 

86 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

common toxic substances (NIOSH 1997, ACGIH 1998, AIHA 2000). 

B.2. Resources Relevant to Exposures to Non-Chemical Stressors 

Biological stressors: 

The ILSI Risk Science Institute recently published a workshop report entitled “Revised 

Framework for Microbial Risk Assessment” (ILSI, 2000), which looks at methods for 

assessing risks to microorganisms such as Cryptosporidium, which has caused disease 

outbreaks when it contaminates drinking water. The methodology is superficially similar 

to a risk assessment conducted for a chemical pollutant, but only at the most general 

level. How exposure is characterized, for example, includes many differences from 

environmental chemical exposure assessment. Under “characterization of exposure,” for 

example, the framework includes (1) pathogen characterization, (2) pathogen occurrence, 

(3) exposure analysis, and finally developing (4) an exposure profile. 

Radiological stressors: 

EPA’s Office of Air and Radiation maintains a web page at 

http://www.epa.gov/radiation/assessment/. This page provides (or cites) much of the 

needed documentation for performing risk assessments for radionuclides. This includes 

the Radiation Exposure and Risk Assessment Manual (RERAM) (USEPA, 1996e) and 

several Federal Guidance Reports (USEPA, 1988, 1993d, 1999l). 

Noise, vibration, and congestion: 

The U.S. Department of Housing and Urban Development has issued The Noise 

Guidebook (HUD, 1991), which implements the existing noise regulations [24 CFR 51-

B] and includes the HUD Noise Assessment Guidelines. (The Guidebook is available in 

hard copy only.) 

The Federal Railroad Administration has developed a manual called High-Speed Ground 

Transportation Noise and Vibration Impact Assessment (DOT, 1998) which provides the 

theory, equations, and applications of noise and vibration analysis for high-speed 

railroads. Much of the theory and information is also applicable to other noise and 

vibration problems. Appendix A of the DOT Guide is a general discussion of noise 

concepts, with references. The Guide is available on-line. 

The National Institute of Occupational Health and Safety has done much research on the 

interaction of noise with chemical exposures (Morata, 2000). 

Odor: 

EPA’s Office of Wastewater Management has issued a report called Guide to Field 

87
 
1

2

3

4

5

6

7

8

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Storage of Biosolids (USEPA, 2000j) which contains an appendix on “Odor 

Characterization, Assessment, and Sampling.” Odor assessment is an analyticdeliberative process, involving both science-based analytical methods and more 

subjective analysis. The appendix of the Guide discusses sensory characterization of 

odors (character, intensity, pervasiveness, quantity), some practical options for assessing 

odors in a community, and the chemistry of odors (including range of odor thresholds). It 

also discusses odor sample collection and analysis, and has several dozen references for 

further information. This report is available on-line. 

88
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX C: SOME THOUGHTS ON BACKGROUND EXPOSURES 

When looking at aggregate exposures or cumulative risks of citizens, “background 

exposures” to specific chemicals are no less “real” exposures than the pollution usually studied 

for regulatory purposes. Whereas in historical single-chemical assessments conducted for 

limiting pollution, background sources of the chemical were often irrelevant to the questions 

being asked of the assessment (or ignored as having negligible effect on risk), background 

sources are rarely irrelevant with cumulative risk assessments15. 

Background concentrations can be categorized as either naturally-occurring, that is, 

chemicals which are naturally present in the environment before it was influenced by humans, or 

anthropogenic, that is, present in the environment due to historical human-made sources. 

Naturally-occurring background chemicals may be either localized or ubiquitous. Anthropogenic 

background sources can be either localized from a point source, or generalized from unidentified 

sources or non-point sources. 

Assessments of morbidity incidence and death rates, market basket surveys, and pesticide 

residue surveys also provide information which can be reflective of background chemical 

concentrations as well as overt pollution. Background issues extend across all media, beyond 

regulated sources, and beyond direct exposure. Many chemicals are naturally present in the 

environment (e.g., soils, water, vegetation and other biota) and are consequently part of dietary, 

dermal and inhalation exposures. In some cases, naturally-occurring substances may occur at 

levels that exceed health-based or risk-based regulatory standards (e.g., drinking water 

standards), or other levels established to protect human health and the environment. Since 

cumulative risk assessments are population based, exposures due to naturally-occurring 

background concentrations should typically be considered to be of importance. 

There are several important issues related to natural or anthropogenic background 

concentrations in cumulative risk assessment. First, if the risks posed by “background” 

concentrations of certain chemicals are significant (and some may approach or exceed health 

reference levels), their exclusion from the cumulative risk estimates and characterization may 

seriously distort the portion of the total estimated risk thought to be posed to the population by a 

specific evaluated source. A second issue is the problem of whether background chemical 

exposures can be clearly distinguished from specific source-related chemicals, and how to 

quantify these exposures. It may be important in a cumulative risk assessment to estimate 

background exposures separately from specific source-related exposures, so that the risk assessor 

15 The word “background” is often used to describe exposures to chemicals or other stressors that derive from sources 

other than the sources being assessed. For example, in the Agency’s assessment of residual risk associated with hazardous air 

pollutant emissions from particular categories of sources that remain after the implementation of technology-based controls, 

“background” is defined as all hazardous air pollutant exposures (via inhalation or other routes) not associated with the source(s) 

being assessed. At a Superfund site, “background contamination” refers to contamination that is not related to the site release of 

chemicals, as defined by Comprehensive, Environmental Response, Compensation and Liability Act (CERCLA).[P.L. 96-510, 

December 11, 1980, as amended by P.L. 98-802, August 23, 1983, and P.L. 99-499, October 17, 1986] Such focusing or 

segregation in a risk assessment can be useful to decisions involving pollution sources covered by particular statutory authorities, 

but it is typical of a chemically-focused assessment rather than a population-focused assessment such as a cumulative risk 

assessment. 

89 
1

2

3

4

5

6

7

8

9

10

11

12

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

can provide the community with a more complete picture of both total and known source-related 

risks. This also provides a clearer, more complete picture for making risk management 

decisions. Finally, there may be problems in identifying representative geographic areas for 

determining “background levels” for comparison. 

Finally, background exposures for a community or population may also include both 

voluntary and involuntary exposures, and subsequent risks. Involuntary exposures are associated 

with the naturally-occurring or anthropogenic background concentrations described above. 

Voluntary exposures, such as are associated with lifestyle decisions, are exposures due to 

activities such as smoking, consuming char-grilled meats with PAHs, or other choice-based 

exposures, and may also sometimes be defined in the assessment as “background” exposures if 

they are not assessed directly in the cumulative risk assessment. 

90
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX D: EXAMPLES OF ANALYSIS PLANS 

D.1. Human Health Analysis Plan for Pesticides under FQPA 

Risk management /regulatory goal: Protection of the general human population and susceptible 

subpopulations to adverse effects from exposure to pesticide “X” under the 1996 Food Quality 

Protection Act (FQPA) 

Assessment endpoints: 

- human or animal health status of exposed versus unexposed populations/cohorts/dose 

groups 

Measures of Effects: 

- general types of toxicological effects grouped according to acute, subchronic, and 

chronic exposure durations 

- organ-specific toxicity such as reproductive effects, developmental effects, 

neurotoxicity, developmental neurotoxicity, immunotoxicity, hepatotoxicity, 

pulmonary effects, cardiovascular effects, etc. 

- general classes of toxic effects such as carcinogenicity, mutagenicity 

Measures of Exposure: 

- monitoring of food, water, residential, occupational exposures, etc. (direct or surrogate) 

- monitoring of biological fluids or biomarkers (blood, urine, DNA or other 

macromolecules) 

What Can and Cannot be Done Based on Planning and Scoping 

- pathways and relationships to be evaluated 

- resource restraints 

- milestones for completion of risk assessment 

Methods for Conducting Risk Analysis 

- RfD 

- Margin of Exposure (MOE) 

- probabilistic risk assessment based on dose-response or exposure parameters 

- quotients (e.g., ratio of exposure level to toxicity threshold) 

- narrative discussions 

- other considerations (e.g., mechanisms of action, toxicokinetic models, timing of dose, 

sensitive population characteristics) 

Data Needs and Uncertainties 

D.2. Ecological Analysis Plan 

Risk management/regulatory goal: Viable, self-sustaining coho salmon population that supports 

a subsistence and sport fishery. 

91
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

Assessment endpoints: Coho salmon breeding success, fry survival, and adult return rates. 

Measures of Effects: 

- egg and fry response to low dissolved oxygen 

- adult behavior in response to obstacles 

- spawning behavior and egg survival with changes in sedimentation 

- population data over time in relation to fish passage 

Measures of Ecosystem and Receptor Characteristics: 

- water temperature, water velocity, and physical obstructions 

- abundance and distributions of suitable breeding substrate 

- abundance and distribution of suitable food sources for fry 

- feeding, resting, and breeding behavior 

- natural reproduction, growth, and mortality rates 

Measures of Exposure: 

- number of hydroelectric dams and associated ease of fish passage 

- toxic chemical concentrations in water, sediment, and fish tissue 

- nutrient and dissolved oxygen levels in ambient waters 

- riparian cover, sediment loading, and water temperature 

What Can and Cannot be Done Based on Planning and Scoping 

- pathways and relationships to be evaluated 

- resource restraints 

- milestones for completion of risk assessment 

Methods for Conducting Risk Analysis 

- quotients 

- narrative discussions 

- stressor-response curves with probabilities 

Data Needs and Uncertainties 

92
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX E: TOXICOLOGIC SIMILARITY: ORGANOPHOSPHORUS PESTICIDES 

The Food Quality Protection Act of 1996 (FQPA) requires that EPA reassess pesticide tolerances 

(legal limits for residues in food) that were in effect as of August 1996. As part of the 

reassessment, EPA must consider available information concerning the cumulative effects on 

human health resulting from exposure to multiple chemicals that have a common mechanism of 

toxicity. In this context, pesticides are determined to have a common mechanism of toxicity if 

they produce the same toxic effect, in the same organ or tissue, and by essentially the same 

sequence of major biochemical events (USEPA, 1999m). 

Shortly after enactment of FQPA, EPA began developing new methods and tools that would 

allow the consideration of combined risks from exposure to several pesticides via several 

pathways and routes of exposure. Actual data sets for organophosphorous pesticides were used 

in pilot analyses to test these methods. The methods and pilot analyses were subjected to peer 

review through the FIFRA Scientific Advisory Panel (SAP) to ensure the use of sound science. 

As part of this ongoing effort, on December 28, 2001 EPA’s Office of Pesticide Programs (OPP) 

announced the availability of the Preliminary Organophosphorus Cumulative Risk Assessment 

[66FR67249-67250]. The risk assessment is available electronically at: 

http://www.epa.gov/pesticides/cumulative. In preparing the cumulative risk assessment for the 

organophosphorous (OP) pesticides, OPP followed 5 major steps. 

1. Selection of the specific pesticides, pesticide uses, pathways and routes of exposure to include 

in the quantitative analysis. 

The selection of the specific OP pesticides began with identifying a “common mechanism 

group.” This was accomplished following the Guidance For Identifying Pesticide 

Chemicals And Other Substances That Have A Common Mechanism Of Toxicity 

(available at http://www.epa.gov/pesticides/trac/science). All 39 registered OP pesticides 

share inhibition of acetylcholinesterase as a common mechanism for causing adverse 

effects (USEPA, 1998k). 

The common mechanism group was further refined to reflect current use patterns and 

information on the detection of residues from USDA’s Pesticide Data Program. This 

resulted in the following recommendations for quantitative analysis: include 22 OP 

pesticides for the food pathway of exposure; 24 OPs for the water pathway and 10 OPs 

for residential exposures were identified based on use patterns and their individual 

assessments. 

2. Dose-response analysis for toxic potencies, relative contribution from each OP, and selection 

of an index chemical to use as the point of reference in the dose-response analysis. 

To determine the combined risk from multiple OP pesticides, EPA used the Relative 

Potency Factor (RPF) approach [for additional examples of comparative potency 

approaches, also see Albert, et al., 1983; Lewtas, 1985, Lewtas, 1988]. The index 

chemical was selected based on the quality of the dose-response data. Then the relative 

93
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

potency of each OP pesticide was estimated by taking the ratio of its toxic potency to that 

of the index chemical. 

In selecting studies for evaluating toxic potencies, EPA used relative potency factors and 

points of departure developed from cholinesterase inhibition in rats exposed to pesticides 

for 21 days or more. This practice was adopted to reflect cholinesterase inhibition at a 

point in the treatment schedule at which a steady state had been achieved. OPP elected to 

use data reflecting a steady state in the interest of producing relative potency factors that 

are reproducible and reflect less uncertainty due to rapidly changing time-sensitive 

measures of cholinesterase. 

Also, EPA considered that people generally have had some level of prior exposure to OP 

pesticides. Further, the effects of exposure can persist for several days to weeks. 

Therefore, people may be more vulnerable to subsequent exposures to OP pesticides than 

might be predicted by not considering these prior exposures. 

3. Estimation of the risks associated with all pertinent pathways of exposure in a manner that is 

both realistic and reflective of variability due to differences in location, time, and demographic 

characteristics of exposed groups. 

Evaluation of the OP pesticide use profiles allowed for the identification of exposure 

scenarios that may overlap, co-occur, or vary between chemicals. In addition, the use 

profiles allowed for the identification of populations of potential concern. Based on this 

analysis, EPA considered exposure to OP pesticides in food to be uniform across the 

nation (i.e., there are no significant differences in food exposure due to time of year or 

geographic location). For the residential and drinking water pathways of exposure, EPA 

divided the nation into 12 regions for assessment. This allowed for the consideration of 

such factors as the location of vulnerable surface watersheds and region specific pest 

pressures. To estimate risks, EPA used a calendar based computer model titled Calendex. 

This model integrates the various pathways of exposure while simultaneously 

incorporating the time dimensions of the data. The model produces a detailed profile of 

the potential exposure to individuals across a calendar year. 

4. Identification of the significant contributors to risk. 

Although interpretation of the preliminary organophosphorous cumulative risk 

assessment is ongoing, there are some early indications concerning contribution to risk. 

The drinking water pathway for exposure does not appear to be a major contributor to the 

total cumulative risk. Residential exposure appears to be a contributor to risk, 

particularly inhalation exposures from certain no-pest strips and crack and crevice 

treatments. Childhood exposure from mouthing hands also appears to be a contributor 

but there is a great deal of uncertainty associated with the estimates. 

5. Characterization of the confidence in the results and the uncertainties encountered. 

94
 
DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1 In addition to some uncertainties noted above, EPA identified many areas for additional 

2 analysis including: sensitivity analyses on input parameters, verification of residential 

3 use patterns, closer examination of the tails of the food consumption distribution, and 

4 evaluation of the effect of assumptions about residue concentrations in baby foods. 

95
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

APPENDIX F: OTHER TYPES OF CUMULATIVE ASSESSMENTS 

There are several other types of cumulative assessments that are related to the types of 

human health and ecological cumulative assessments done by the Agency. It is beyond the scope 

of this Framework to discuss these in detail, but a short explanation of several other types of 

cumulative assessments are given in this appendix. 

F.1. Quality-of-Life Assessments 

One type of assessment which 

resembles a cumulative risk assessment, but 

whose evaluation may require a different 

approach from the traditional NRC risk 

paradigm, is the quality-of-life assessment. 

These assessments define “harm” to an 

individual or community broadly, then 

evaluate the importance of the various threats 

of harm to a set of “quality-of-life” criteria 

(see box at right). These assessments do not 

usually attempt to predict probability that the 

harm will occur (as would a cumulative risk 

assessment), but rather aim to apply the 

community’s values to deal with the most 

important perceived threats. 

Although a quality-of-life assessment 

is not a risk assessment in most cases, changes 

in quality-of-life factors may affect the 

vulnerability of a population to health or 

ecological risks, and consequently may be part 

of the considerations in a cumulative risk 

assessment. Since few, if any, established and 

accepted relationships are currently available 

quantitatively linking quality-of-life factors 

and health or ecological risk, this is an area in 

which further research may prove valuable. 

To evaluate the effects on human or 

ecological health from these types of impacts, 

a more deliberative approach (in the 

analytical-deliberative process) is needed than 

is used in, say, cancer risk analysis. To help 

Vermont’s Quality of Life Criteria 

Impacts on Aesthetics: Reduced visibility, noise, odors, 

dust and other unpleasant sensations, and visual impact 

from degradation of natural or agricultural landscapes. 

Economic Well-Being: Higher out-of-pocket expenses to 

fix, replace, or buy items or services (e.g., higher waste 

disposal fees, cost of replacing a well, higher housing 

costs), lower income or higher taxes paid because of 

environmental problems, and health-care costs and lost 

productivity caused by environmental problems. 

Fairness: Unequal distribution of costs and benefits (e.g., 

costs and benefits may be economic, health, aesthetic). 

Future Generations: Shifting the costs (e.g., economic, 

health risks, environmental damage) of today’s activities to 

people not yet able to vote or not born yet. 

Peace of Mind: Feeling threatened by possible hazards in 

air or drinking water, or potentially risky structures of 

facilities (e.g., waste sites, power lines, nuclear plants), and 

heightened stress caused by urbanization, traffic, etc. 

Recreation: Loss of access to recreational lands (public 

and private), and degraded quality of recreation experience 

(e.g., spoiled wilderness, fished-out streams). 

Sense of Community: Rapid growth in population or 

number of structures, or development that changes the 

appearance and feel of a town; loss of mutual respect, 

cooperation, ability, or willingness to solve problems 

together; individual liberty exercised at the expense of the 

individual; the loss of Vermont’s landscape and the 

connection between the people and the land. 

Source: State of Vermont, 1991 

better characterize these impacts, EPA’s Guidebook to Comparing Risks and Setting 

Environmental Priorities (USEPA, 1993b) suggests a six-step process in Quality-of-Life 

Analysis: 

96
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

1. Identify impacts and determine the values of the community. 

2. Identify and define evaluative criteria. 

3. Collect and analyze data on impacts. 

4. Characterize impacts for all problem areas. 

5. Present findings and rank problem areas for quality-of-life impacts. 

6. Analyze future environmental conditions and risk management considerations. 

Quality-of-Life impacts are determined by analyzing a set of criteria developed for each 

community, depending on what they value. Stressors are those things that threaten to degrade the 

quality-of-life criteria for that community. An example of a set of quality-of-life criteria, and 

their descriptions, is in the box on the previous page. These criteria were developed by the State 

of Vermont’s Agency of Natural Resources (State of Vermont, 1991). Vermont’s experience in 

evaluating these criteria was described as a qualitative description of harm, or in their terms, 

“risk:” 

“Because most of these seven criteria are intangible, they are extremely difficult to 

measure or quantify. The Quality-of-Life Work Group described how each problem area 

affects each criterion and how widespread or intense the effects are. Although these nonquantitative descriptions of risk often lack precision and scientific objectivity, they focus 

attention on specific critical issues and thus are useful tools for comparing the problems 

systematically and consistently.” (State of Vermont, 1991) 

Quality-of-life issues can encompass much more than the criteria used here as an 

example. Ultimately, such an analysis may introduce much additional complexity into the 

analysis. There may, for instance, be feedback loops not easily evaluated (e.g., loss of property 

value, aesthetics, etc., tend to negatively affect the socioeconomic system, which in turn tends to 

increase rates of crime, traffic accidents, and communicable-pathogen transmission, all 

ultimately reflecting on overall community 

health or ecological risk). Some cumulative 

risk assessments may consequently include 

quality-of-life impacts as indirect measures of 

health effects if sufficient links can be 

established between the two. 

F.2. Cumulative Impact Assessments 

The National Environmental Policy 

Act (NEPA) has certain requirements for 

“cumulative impacts” assessment (see box at 

right), which looks at various stressors 

leading to a variety of impacts or effects on 

the environment. Although the Council on 

NEPA’s “Cumulative Impact” Definition 

CEQ Regulation 1508 for Implementing the National 

Environmental Policy Act of 1969 [P.L. 91-190, 42 U.S.C. 

4321-4347, January 1, 1970, as amended by P.L. 94-52, 

July 3, 1975, P.L. 94-83, August 9, 1975, and P.L. 97-258, 

§4(b), Sept. 13, 1982] defines “cumulative impact” as “the 

impact on the environment which results from the 

incremental impact of the action when added to other past, 

present, and reasonably foreseeable future actions 

regardless of what agency (Federal or non-Federal) or 

person undertakes such other actions. Cumulative impacts 

can result from individually minor but collectively 

significant actions taking place over a period of time.” 

Source: CEQ, 1997 

Environmental Quality’s guidelines for cumulative impact analysis (CEQ, 1997) take a primarily 

qualitative approach to the analysis, this is a multiple stressor, multiple effect assessment. 

97
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

The projects or actions that NEPA addresses can be viewed as sources of stressors. 

Environmental impact assessment under NEPA contains a description of the affected 

environment that contains four types of information: (1) data on the status of important natural, 

cultural, social, or economic resources and systems; (2) data that characterize important 

environmental or social stress factors; (3) a description of pertinent regulations, administrative 

standards, and development plans; and (4) data on environmental and socioeconomic trends. 

Health effects on populations and susceptible individuals are part of the affected environment as 

considered by the NEPA cumulative effects analysis, but the NEPA analysis may also consider 

effects on historic and archaeological resources, socioeconomic factors like employment, human 

community structure, and quality of life changes. Although there is not always a clear 

relationship between these NEPA cumulative impacts and effects relevant to human health, the 

NEPA methods and tools for cumulative impact analysis may be useful for cumulative risk 

assessments. For example, cumulative impact analysis begins with an extensive scoping process 

and relies on conceptual models to plan the analysis. NEPA effects data may help risk assessors 

identify susceptible subpopulations, environmental pathways, or exposure patterns. 

EPA’s Region VI has developed a system called the Cumulative Risk Index Analysis 

(CRIA), primarily for NEPA-type assessments (Osowski, et al., 2001). The CRIA contains some 

90 criteria to evaluate the health of an area and its ecosystem/human populations. These criteria 

help evaluate factors as diverse as human health, ecosystem health, and environmental justice 

considerations. Each criterion, which leads to an indexing of 1-5, has been through the 

deliberative process, peer review, and is well documented. 

We also acknowledge that other Federal Agencies have been preparing “cumulative risk 

analyses” for various purposes related to their own mission as part of environmental impact 

statements (e.g., NOAA, 1999). 

F.3. Empirically-Derived Medical Models 

The medical profession has long used empirically-derived models to predict the chances 

of particular health effects in individual patients. In this approach, the characteristics of 

individuals within the population are correlated with the incidence of specific diseases or effects. 

For example, the risk factors for stroke are: increasing age, heredity (family history) and race, 

prior stroke, high blood pressure, cigarette smoking, diabetes mellitus, carotid and other artery 

disease, heart disease, transient ischemic attacks (TIAs), high red blood cell count, sickle cell 

anemia, socioeconomic factors, excessive alcohol consumption, and certain types of drug abuse 

(American Heart Association, 2000). Each of these risk factors can be correlated with stroke 

incidence, and then the risk of stroke from various combinations of these factors can be explored. 

In this way, the analysis is “cumulative,” but “risk factors” are not always synonymous with 

“stressors.” 

Physicians use models containing effect-specific risk factors to advise patients of the 

probabilities of future effects (e.g., stroke, breast cancer) based on their medical history. 

Although the medical data upon which these factors are based have been well developed for 

many effects in humans, there are substantial data gaps remaining in terms of the role played by 

98
 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

DRAFT – External Review Draft – April 23, 2002 – Do Not Quote or Cite 

exposures to many chemicals in the environment in the development of human disease. This 

approach may be built on links between risk factors and effects for better studied stressors, but 

may be limited or nonexistent for less robust health effects data bases. Although this approach 

may some day be applicable to human health and environmental risk assessment such as EPA 

conducts, at present the data and methods are not available. 

F.4. Risk Surrogates 

Geographic Information Systems (GIS) and related mapping techniques (e.g., 

Environmental Defense, 2001) appear to hold some promise as tools for presenting integrated 

information concerning cumulative risks without mathematically combining disparate measures. 

Considerable methods development work remains to be completed. 

Not all statements of probability of harm are expressed as probabilities of specific health 

effects. Bernard Cohen, in his Catalog of Risks Extended and Updated (Cohen, 1991), uses 

mortality ratios to derive “loss of life expectancy” (LLE) estimates for a wide variety of riskrelated activities. For example, workers in all occupations have a 60 day LLE as a result of 

working, but workers in agriculture have a 320 day LLE, construction workers a 227 day LLE, 

etc., as a result of their particular occupation. These types of statements are empirically derived, 

probability-based statements of harm that do not use “probability of adverse health effect” as the 

basis for the risk statement. For estimates such as LLEs, one could theoretically add up the 

various activities and the corresponding LLEs in days to estimate a cumulative risk in terms of 

loss of life expectancy. These “other” types of risk-surrogate probability statements could 

conceivably be used in cumulative risk assessment, although there is apparently no methodology 

currently being used to do so. 

99