EPA/600/R-99/111

February 2000

Workshop Report on Characterizing

Ecological Risk at the Watershed Scale 

July 7-8, 1999

Arlington, VA

National Center for Environmental Assessment–W

Office of Research and Development

U.S. Environmental Protection Agency

Washington, DC
ii

DISCLAIMER

This document has been reviewed in accordance with U.S. Environmental Protection Agency

policy and approved for publication. Mention of trade names or commercial products does not

constitute endorsement or recommendation for use.

ABSTRACT

As ecological risk assessment evolves, it is moving beyond a focus on single species toward

addressing multiple species and their interactions, and from assessing effects of simple chemical

toxicity to the cumulative impacts of multiple interacting chemical, physical, and biological

stressors. While EPA and others have developed guidance and have considerable experience in

applying the ecological risk assessment paradigm in source-based approaches (such as those

focused on particular chemical contaminants), specific guidance for using it in “place-based”

approaches (such as those conducted on a watershed-wide scale) is still limited. 

One of the principal challenges in applying ecological risk assessment to watershed management

and decision making is the need for a framework for characterizing risks that involve numerous

stressors, interconnected pathways, and multiple endpoints. To develop the needed guidance in

this area a workshop was held that gathered 35 participants with extensive experience in relevant

disciplines such as watershed ecological risk assessment, ecological risk assessment, watershed

management, or regional-scale assessment. To focus workshop discussions, several charge

questions addressing the issues of greatest concern for characterizing risk at the watershed scale

were developed prior to the workshop. This report provides the proceedings from the workshop

and includes an introduction, a synthesis of discussion and recommendations, and summaries of

presentations and breakout sessions. 
iii

CONTENTS

LIST OF TABLES AND FIGURES ............................................ v

FOREWORD ............................................................. vii

AUTHORS AND REVIEWERS ............................................. viii

SECTION ONE - INTRODUCTION ......................................... 1-1

1.1 Background and Purpose ........................................ 1-1

1.2 Workshop Organization ......................................... 1-1

1.3 Charge Questions .............................................. 1-2

SECTION TWO - DISCUSSION AND RECOMMENDATIONS .................. 2-1

2.1 Synthesis of Discussion Points and General Recommendations ............ 2-1

SECTION THREE - SUMMARY OF OPENING PRESENTATIONS .............. 3-1

3.1 Welcome and Introductory Comments on Place-Based Ecological 

Assessments .................................................. 3-1

3.2 Workshop Goals and Approach ................................... 3-2

3.3 Risk Characterization Summary ................................... 3-3

3.4 Traditions of Environmental Management and Their Implications .......... 3-7

SECTION FOUR - WATERSHED CASE STUDY PRESENTATIONS ............. 4-1

4.1 Big Darby Creek ............................................... 4-1

4.2 Middle Platte ................................................ 4-10

4.3 Waquoit Bay ................................................ 4-11

4.4 Middle Snake ................................................ 4-12

4.5 Clinch Valley ................................................ 4-14

SECTION FIVE - FACILITATOR PRESENTATIONS .......................... 5-1

5.1 Jack Gentile, Center for Marine and Environmental Analysis, University 

of Miami ..................................................... 5-1

5.2 William Smith, Yale School of Environment and Forestry Studies, Yale University5-2

5.3 Patrick Bourgeron, INSTAAR University of Colorado, Colorado State 

University .................................................... 5-3

SECTION SIX - BREAKOUT SESSION SUMMARIES ......................... 6-1

6.1 Breakout Team 1 .............................................. 6-1

6.2 Breakout Team 2 .............................................. 6-4

6.3 Breakout Team 3 .............................................. 6-9
iv

CONTENTS (continued)

REFERENCES .......................................................... R-1

APPENDIX A: List Of Workshop Participants ................................ A-1

APPENDIX B: Workshop Agenda .......................................... B-1

APPENDIX C: Ecological Risk Assessment Guidelines, Section on Risk 

Characterization ......................................................... C-1
v

LIST OF TABLES

2-1 Middle Snake River Case Study: Sample Table Integrating Stressors, Responses, and

Recovery Potential .................................................. 2-2

LIST OF FIGURES

2-1 Clinch Valley Case Study: Conceptual Model for Clinch Eco-risk and Assessment 

Approaches Used .................................................... 2-5

2-2 Middle Snake River Case Study: Map of Snake River Showing Probability of Life 

Stage Impairment for Rainbow Trout ..................................... 2-7

2-3 Waquoit Bay Case Study: Management Decision Process for Waquoit Bay 

Assessment ....................................................... 2-10

3-1 Ecological Risk Assessment Framework ................................... 3-4

3-2 Proximity of Management to Action on the Ground .......................... 3-9

4-1 Big Darby Creek Case Study: Sample Box Plots of IBI Metrics Showing Source

Signatures ......................................................... 4-2

4-2 Big Darby Creek Case Study: Map Showing IBI Scores by Location ............. 4-4

4-3 Big Darby Creek Case Study: Sample Plots of ICI Versus River Mile ............. 4-5

4-4 Big Darby Creek Case Study: Sample Radar Plots of Trend Analysis of Response

Variables .......................................................... 4-6

4-5 Big Darby Creek Case Study: Map Showing Erosion Potential Based on Model

Predictions ......................................................... 4-7

4-6 Big Darby Creek Case Study: Trends in Stressors Using Radar Plots ............ 4-8

4-7 Big Darby Creek Case Study: Longitudinal Plot of Stressor Versus River Mile ..... 4-9
vi

LIST OF FIGURES (continued)

4-8 Middle Snake River Case Study: Snake River Illustration Linking Stressors to

Recovery Potential for Rainbow Trout ................................... 4-13

6-1 Clinch Valley Case Study: Map Showing Land Use Versus Threatened and

Endangered Species ................................................. 6-5
vii

FOREWORD

Risk assessment is playing an increasingly important role in determining environmental policies

and decisions at the U.S. Environmental Protection Agency (EPA). EPA’s first Agency-wide

guidelines for ecological risk assessment, published in May 1998, provided a broad framework

applicable to a range of environmental problems associated with chemical, physical, and biological

stressors (U.S. EPA, 1998). However, while EPA has considerable experience in applying the

ecological risk assessment paradigm in source-based approaches (such as those focused on

particular chemical contaminants), specific guidance on “place-based” approaches (such as those

conducted on a watershed-wide scale) is still limited. 

As ecological risk assessment evolves, it is moving beyond a focus on single species toward

addressing multiple species and their interactions, and from assessing effects of simple chemical

toxicity to the cumulative impacts of multiple interacting chemical, physical, and biological

stressors. Ecological risk assessment will play a major role in bringing science into place-based

decision making, but its application faces several challenges. Among those challenges is the need

for a framework for characterizing risks that involve numerous stressors, interconnected

pathways, and multiple endpoints. 

The risk characterization phase of ecological risk assessment is the culmination of planning,

problem formulation, and analysis. The objective is to weave together the various lines of

evidence to formulate a clear and compelling summary of the exposure and effects data. Because

it is intended to support scientifically sound decisions by risk managers, the risk characterization

must also address the uncertainties, assumptions, and qualifiers embedded in the risk assessment. 

The recommendations developed from this workshop represent a significant step forward in

meeting the challenges of risk characterization on a watershed scale. 

Michael Slimak

Associate Director of Ecology

EPA, National Center for Environmental Assessment
viii

AUTHORS AND REVIEWERS

The National Center for Environmental Assessment-Washington Office (NCEA–W) of EPA’s

Office of Research and Development was responsible for the preparation of this document. The

first draft was prepared by Tennessee Valley Authority under Interagency Agreement No.

DW64866901 for which Victor B. Serveiss of NCEA–W served as the EPA Project Officer. 

PRIMARY AUTHOR

Victor B. Serveiss

National Center for Environmental Assessment

U.S. Environmental Protection Agency

Washington DC 20460

CONTRIBUTING AUTHORS

Janice P. Cox

Jennifer Moses

Bruce L. Yeager

Tennessee Valley Authority

Knoxville, TN 37901

REVIEWERS

James Andreasen, U.S. EPA, ORD

Ed Bender, U.S. EPA, ORD

Pat Cirone, U.S. EPA, Region 10

Bill Ewald, U.S. EPA, ORD

Jerry Diamond, Tetra Tech

Susan Ferenc, ILSI Risk Science Institute

Bob Fenemore, U.S. EPA, Region 7

Jack Gentile, University of Miami

Ken Jones, Green Mountain Institute for Environmental Democracy

Robert Murphy, Alliance for the Chesapeake Bay

Sue Norton, U.S. EPA, ORD

Don Rodier, U.S. EPA, OPPTS

Anne Sergeant, U.S. EPA, ORD

Michael Slimak, U.S. EPA, ORD

Patti Tyler, U.S. EPA, Region 1

William Smith, Yale University

Barry Tonning, Tetra Tech (at time of workshop-Council of State Governments)

William van der Schalie, U.S. EPA, ORD
ix

ACKNOWLEDGMENTS

This workshop was funded through an interagency agreement between EPA and the Tennessee

Valley Authority (TVA). I would like to thank TVA, EPA, and other staff for their support and

hard work in making this workshop a success. First, thanks to Bill van der Schalie and Sue

Norton for their insight provided in the many discussions we held to plan the workshop. From the

TVA group, Angela Stiles coordinated the vast majority of the logistical activity to get the right

people in one good place at the same time. Jennifer Moses, Janice Cox, and Bruce Yeager took

the notes that formed the basis for the workshop report. Forrest Rich conceived the idea that

EPA and TVA should work together in putting on this workshop because TVA was already

involved in addressing watershed risk communication issues and had experience putting on

workshops. Janice Dockery helped with formatting, typing and distribution of the report.

 

Although many panel members provided input into developing the charge questions, the extensive

assistance of Sue Norton and Bill van der Schalie (EPA National Center for Environmental

Assessment [NCEA]), Maggie Geist (Waquoit Bay National Estuarine Research Reserve), Pat

Cirone (EPA Region I) and Jerry Diamond (Tetra Tech) warrants specific acknowledgment. Jerry

Diamond and Pat Cirone also made a major contribution to the workshop by developing draft

watershed ecological risk characterizations that provided examples for participants to consider in

responding to the charge questions. Jerry Diamond and Pat Cirone’s presentations on Clinch

Valley and Middle Snake assessments, respectively, provided the basis for many of the

recommendations contained in this report. The success of the Clinch Valley assessment would

not have been possible without the dedicated efforts of Don Gowan (The Nature Conservancy)

and Roberta Hylton (U.S. Fish and Wildlife Service).

The initial plenary sessions successfully set the stage for the discussions that followed, and I wish

to thank Mike Slimak, Sue Norton and Bill van der Schalie, all from the EPA NCEA, for their

various perspectives on ecological risk assessment and interactions with risk management. 

Several others presented their work on watershed or regional-scale ecological risk assessments,

including Susan Cormier (EPA National Exposure Research Laboratory, Cincinnati); Bob

Fenemore and Marla Downing (EPA Region 7); Maggie Geist and Patti Tyler (EPA Region 1);

Jack Gentile (University of Miami); William Smith (Yale University); and Patrick Bourgeron

(University of Colorado). Their presentations contributed to the workshop discussion and

recommendations. Drs. Gentile, Smith, and Bourgeron also served as facilitators to guide and

maintain the focus of the break-out discussions in developing responses to the charge questions. 

Finally, while there are too many to list individually here (see Appendix A), I acknowledge with

gratitude everyone who participated in the workshop and contributed to this report. Addressing

the charge questions was not an easy task, and the subject required open and frank

communication and sharing and documenting this collective wisdom. 

Victor B. Serveiss, Workshop Organizer

U.S. EPA, National Center for Environmental Assessment
1-1

SECTION ONE

INTRODUCTION

1.1 BACKGROUND AND PURPOSE

The workshop described in this report was conducted to further develop and document the

process for the risk characterization phase of watershed-scale ecological risk assessment. The

recommendations reflect the responses of workshop participants to charge questions addressing

those aspects of watershed-scale ecological risk characterization deemed most in need of a

procedural framework. 

Participants were invited to the workshop because of their experience in ecological risk

assessment, watershed management, or regional-scale assessment. Many of the attendees have

been active in five EPA-sponsored watershed assessments that were initiated in 1993 to

demonstrate application of the ecological risk assessment process to large scale, place-based

studies. The idea to hold this workshop originated when a few of the chairs of these assessments

noted a lack of specific guidance for developing such watershed risk characterizations. The

workshop was also designed to provide feedback to help complete and improve these five

watershed-scale assessments. 

The recommendations developed through the workshop are intended to supplement the

Guidelines for Ecological Risk Assessment (U.S. EPA, 1998) and the Risk Characterization

Handbook, presently being developed by EPA (U.S. EPA, 1999). While the proceedings from the

workshop include comments that replicate material in the 1998 EPA Guidelines and the

Handbook, this report assumes the reader is already familiar with the principles of risk

characterization as described in the Guidelines (included as Appendix C).

1.2 WORKSHOP ORGANIZATION

The workshop was held on July 7 and 8, 1999, at the Crystal City Marriott in Arlington, Virginia. 

Participants at the workshop represented federal agencies, academia, consulting firms, and

environmental organizations. The workshop was led by three facilitators with extensive

experience in large scale ecological assessment. The other attendees included 13 participants in

one or more of the prototype watershed assessments, five authors of the Guidelines and 22 others

with backgrounds in ecological risk assessment or ecological resource management at various

spatial scales. The program began with individual presentations, then continued as breakout

group discussions that generated recommendations for conducting watershed-scale ecological risk

characterization. A list of workshop participants is included as Appendix A.

The workshop began with opening remarks from four EPA National Center for Environmental

Assessment (NCEA) staff. First, Mike Slimak, Associate Director of Ecology, welcomed

everyone to the workshop and provided a historical background. Next, Vic Serveiss, the

workshop organizer, provided an overview of the workshop plans. Bill van der Schalie then

summarized the major principles of risk characterization as described in the Guidelines and
1-2

Handbook, and Sue Norton discussed different traditions in environmental management and their

implications for watershed ecological risk assessment. The remainder of the day was dedicated to

presentations on the five prototype watershed assessments. 

The second day of the workshop opened with presentations from each of the three facilitators

based on their experience in ecological assessment as related to the charge questions. The group

then divided into three facilitator-led breakout sessions, each with the goal of developing

recommendations for conducting watershed ecological risk characterization. The results from

each breakout session were reported back to the entire group. The workshop agenda is included

as Appendix B.

1.3 CHARGE QUESTIONS

The following charge questions were developed to focus workshop discussions: 

• How can the watershed assessment drafts (or plans) for Clinch Valley, Middle Snake, and

Waquoit Bay be improved, especially in regard to the other charge questions listed below? 

• How should exposure and effects data be integrated in a watershed context to generate a risk

estimate? 

• How should uncertainty be addressed and presented (e.g., incomplete data or analyses,

qualitative estimates, data at different spatial and organizational levels)?

• What are the best ways to address and present the integration of qualitative and quantitative

lines of evidence for a) an individual assessment endpoint and b) drawing overall conclusions?

• How should the degree of adversity of predicted or observed effects in watersheds be

described (e.g., considering nature and intensity of effects, spatial and temporal scale, and the

potential for recovery)?

• How are alternative management options selected to have their risks characterized and how is

management informed of the consequences?

• How should ecological risk be communicated to the manager and the public (journal articles,

computer programs, presentations, town meetings, Internet, fact-sheets)?
2-1

SECTION TWO

DISCUSSION AND RECOMMENDATIONS

2.1 SYNTHESIS OF DISCUSSION POINTS AND GENERAL RECOMMENDATIONS 

This section summarizes recommendations made in response to the charge questions and is based

on the individual presentations, the three breakout team sessions and the report-back

presentations that followed the breakout sessions. More complete documentation of the

individual break-out group discussions is included in Section 6 of this report. Attempts to

characterize the degree of agreement on discussion points or recommendations below are based

on notes taken and observations made during the discussions among the participants who actively

expressed opinions. The group was not polled on any of the points presented. 

CHARGE QUESTION: How should exposure and effects data be integrated in a watershed

context to generate a risk estimate? 

Use the conceptual model to review the purpose

• As an introduction to presenting the data, revisit the purpose of the assessment, the

assessment endpoints, and the pathways described by the conceptual model. This helps

reiterate the value of the findings to the risk manager. 

Use models, associations, and multivariate analysis

• Dose-response curves are very effective in a single stressor - single endpoint scenario. 

However, in watersheds the more common scenario will require analyzing the impacts of

multiple stressors on multiple endpoints. In such situations, models and multivariate analysis

can be used to analyze and present associations between stressors and impacts.

Work with multiple lines of evidence

• Lines of evidence may be brought together to increase confidence in associations between

stressors and responses. This effective approach was taken in the Middle Snake River

assessment and is presented in summary form in Tables 17-20 of the draft project report (see

Table 2-1). References to the literature are supportive. Using multiple lines of evidence from

observations in the field may actually be more useful than dose-response information derived

from laboratory data. 

Look for associations by category

• Risk assessors often strive to present the relationships between stressors and effects in the

form of dose-response curves. However, categorical information can sometimes provide a

clearer picture of a relationship than is evident from continuous data. In examining

relationships between stream habitat quality and biological condition in the Clinch Valley risk

assessment, for example, continuous “dose-response” relationships were not apparent. 

However, by categorizing habitat quality as either poor, fair, good, or excellent (based on

predetermined decision criteria) and relating these habitat categories to either continuous or 
Table 2-1. Middle Snake River Case Study: Sample Table Integrating Stressors, Responses, and Recovery Potential

Factors limiting reproduction, growth, and survival of the rainbow trout population in the Middle Snake River.

Factor Stressor LOE1Risk Uncertainty Assumption Recovery Potential

Number of

spawning

fish

Loss of adult habitat

(e.g., stream side

vegetation,

overhanging banks,

and woody debris)

LIT,

Field,

BPJ

An increase in the

population size is not

possible with low or no

reproduction

Low, field studies show low

numbers of adult fish are

present in the Middle Snake

River

Lack of habitat is a

main factor limiting

the size of the adult

population

Good, if habitat improvements

can be improved, but low if a

stable annual flow regime is

not maintained

Spawning Sedimentation, high

water temperature,

and land use on

tributaries

HSI,

WQS,

LIT,

Field,

BPJ

Population of native fish

can not recover without

successful reproduction

Moderate, historic spawning

areas in the main channel have

not been documented

Poor spawning

success attributed to

poor water quality

conditions

Low, without improving water

quality, reducing

sedimentation, and controlling

land/water use on tributaries

Rearing Unstable stream flow

during the spring and

high water

temperatures

HSI,

WQS,

LIT,

Field,

BPJ

Population can not

recover without

successful recruitment

Rearing areas in the main stem

of the Middle Snake River

identified using habitat

suitability indices

Rearing habitat

important for

maintaining and

increasing adult

populations

Low, the carrying capacity for

native fish was likely

permanently reduced by dam

construction in the Middle

Snake River

Overwinter

ing

Loss of habitat (e.g.,

deep holes and large

woody debris)

LIT,

BPJ

Unknown High, no information available

on the amount of

overwintering habitat in the

main stem

Overwintering habitat

can limit the size of

the adult population

Unknown

Food

supply

Sedimentation and

increased water

temperature

LIT,

Field,

BPJ

Poor growth as

invertebrate fauna do

not support cold water

fish

Moderate, adequate analysis of

sampling information has not

been completed

Food supply can limit

the size of a rainbow

trout population

Low, without improving lotic

conditions, lowering water

temperature, and controlling

sedimentation

Genetic

diversity

Hybridization with

stocked fish

LIT,

BPJ

Poor survival with

mixed genotype

Low, effects of hybridization

on native fish are known

An adequate

population of native

fish remains

Good, provided that existing

native fish are protected from

stocked hatchery fish

Note: 1. LOE - lines of evidence, HSI - habitat suitability indices, WQS - water quality standards, Field - field surveys in the Middle Snake River, LIT - literature,

BPJ - best professional judgement.

2-2
2-3

categorical expressions of the biological condition, the relationships between habitat and biology

were much easier to visualize. 

 Use the right scale and data

• Consider partitioning the spatial scale into smaller units with regard to availability of data,

occurrence of problems, subwatersheds or sub-ecosystem types, and management authorities. 

In some cases, it may be possible to assess relationships within subwatersheds that have

comprehensive data bases, and then extrapolate to the larger area. On the other hand, some

problems may impact only a small subset of the area being assessed. More discussion on the

scale issue is available in a workshop report intended to develop a problem formulation

process for large spatial scales (TN & Associates, 1999).

• Partitioning a watershed into smaller units, especially those with similar attributes such as 

hydrology, structure, function, and habitats, makes the effort more manageable. An example

of this is described in more detail on Page 5-1. 

• The data should be appropriate for the scale being addressed: for example, satellite remote

sensing data might be appropriate for larger scales, but smaller scales may require field

observations.

CHARGE QUESTION: How should uncertainty be addressed and presented (e.g.,

incomplete data or analyses, qualitative estimates, data at different spatial and

organizational levels)?

Focus on confidence

• It is preferable to discuss the “degree of confidence” in the results rather than the “level of

uncertainty,” especially when communicating with a relatively non-technical audience.

• Recognize that most decision makers don’t require the degree of certainty that scientists seek

to attain. 

• In cases where the information is inadequate to define a stressor-response curve or even

document a causal relationship, the discussion should address the strength of association using

standard models or methods for causal inference, such as those presented in the EPA

Guidelines (U.S. EPA, 1998).

Consider the risk manager’s perspective

• The “acceptable” degree of uncertainty (or level of confidence) in a risk assessment may be

influenced by the perspective of the audience (e.g., scientist, risk manager, stakeholder), the

magnitude of the risk, and the cost associated with decreasing the degree of uncertainty. 

Tables can be used to compare accuracy and precision versus the degree of risk or the cost to

reduce uncertainty, although it may be preferable to use such tables for generating estimates

without incorporating them into the actual risk characterization report. 
2-4

• Information may be characterized simply as having a low, medium, or high degree of

confidence. This approach can be applied to the integrated effects and exposure data or to

each set of data individually. 

• Risk characterization should consider the context in which risk management decisions are

being made and the implications of risk management decisions. For example, reducing

uncertainties in data may have a low priority if there are no feasible management options.

• The simplified form of the Clinch Valley conceptual model is a good example of how to

illustrate and communicate the pathways, linkages, associations and locations of uncertainties

(see Figure 2-1).

Focus on the key uncertainties but acknowledge all of them

• Participants were divided on the issue of the scope for the discussion of uncertainties. Some

participants preferred a narrow focus to address only “key” uncertainties (those with the

potential to influence the outcome), thus avoiding an exhaustive “laundry list” of all potential

uncertainties. Others preferred presentation of a comprehensive list of uncertainties, feeling

that omissions were more costly errors than inaccuracies. 

• An intermediate approach mentioned was to assemble a comprehensive inventory of

uncertainties, but then to partition the list based on importance (e.g., ranking as high, medium,

or low). However, one danger identified with using ranking is that it adds another layer of

judgment and may lead the reader to disregard important information. 

• Although it can be useful to identify the factors most likely to affect the outcome, the risk

characterization should strive to make the uncertainties transparent so the reader can judge

which ones are more important. 

• Distinguish between qualitative uncertainties (e.g., is there a cause-and-effect relationship?)

and quantitative uncertainties (e.g., what does the stressor-response curve look like?). 

Address uncertainties quantitatively when possible, but avoid conducting more quantitative

uncertainty analysis than is genuinely necessary. The quantitative uncertainty evaluation

(including references to standard tests, metrics, etc.) should be explicitly presented, but may

be included in an appendix.

CHARGE QUESTION: What are the best ways to address and present the integration of

qualitative and quantitative lines of evidence for a) an individual assessment endpoint and

b) drawing overall conclusions?

Use visuals

• Many participants favored the format used in the Snake River assessment (see Table 2-1). An

advantage of this format is that it can summarize both qualitative and quantitative lines of

evidence. Such a table can be supplemented with a narrative text that refers the reader to 
Conceptual Model for Clinch EcoRisk and Assessment Approaches Used

(Not meant to be an exhaustive model, rather for purposes of the Analysis Plan)

Sources:

Stressors:

Other Factors:

Endpoints/Effects:

– GIS/statistical analyses

of CPRATS data;

– Riparian buffer analyses

Copper Creek

Land Use

Mining Urban/Industrial

* Transportation

Corridors

Spills

*

(historical records, Lone

Mtn, APCO)

Hydrology

Drought

flood

*

(USGS gages)

Native Fish

Recruitment/Diversity

T&E species IBI *

Multi-variate and

other analyses on

CPRATS data

CMCP data and

approximate

habitat

T&E species and

land use

Mussel Recruitment/Diversity

T&E species EPT * CMCP

Habitat Quality (CPRATS) Water/Sediment Quality

embeddedness Cover

loss

*

Riparian

loss

* *

toxics Nutrients (?)

(STORET)

Elevation

*

Physiography (e.g., slope*)

*

* Denotes that aspect has been examined in the risk assessment thus far.

Measurement Endpoints:

*

Agriculture

Riparian Corridor (looked at in Copper Creek*)

Karst topography (looked at in Copper Creek*)

Figure 2-1. Clinch Valley Case Study: Conceptual Model for Clinch Eco-risk and Assessment Approaches Used.

2-5
2-6

other sections of the report for supporting information, and the very detailed data can be

moved to appendices. It was also suggested that such a table include citations to technical

references, or in cases where conclusions are based on best professional judgment, identify the

scientist(s) drawing the conclusion.

• Maps and map overlays may be helpful in illustrating stressors and impacts. 

Use rankings

• Qualitative lines of evidence should be considered and could be ranked as having a high,

medium, or low level of importance or confidence. Risk assessor’s need to use professional

judgement to sort qualitative aspects of issues such as degree of adversity, strength of

association, level of confidence, importance to endpoint, etc. into some coarse categories. 

Addressing the ranked issues in order of importance avoids being overwhelmed because there

is never enough data to address all concerns and avioids getting into a never-ending analysis

of all the potential uncertainties.

• Several participants recommended avoiding techniques that numerically rank or weight

various lines of evidence; others recommended that such techniques only be used with great

caution; and still others considered such techniques to be extremely useful. The concern with

weighting qualitative data, lines of evidence, or measures of strength of association was that it

introduces an additional source of uncertainty.

Presentation style

• As each assessment endpoint is discussed, keep the presentation format relatively constant to

enhance continuity of the text.

• Any contradictions between the various lines of evidence should be explained.

CHARGE QUESTION: How should the degree of adversity of predicted or observed

effects in watersheds be described (e.g., considering nature and intensity of effects, spatial

and temporal scale, and the potential for recovery)?

Use visuals

• The table in the Snake River assessment (reproduced as Table 2-1 for this report) and the map

showing degree of impairment for various life stages along the stream reach (see Figure 2-2)

are a useful format for illustrating observed effects and predictions for recovery. Such visuals

should be accompanied by narrative descriptions. 

Consider the historical context

• Trend assessment and comparison with historical data (including historical maps or photos)

can be useful. However, caution is warranted because historical data may be of poor or

unknown quality, and because historical trends are not always predictive of future trends. 
2-7

Figure 2-2. Middle Snake River Case Study: Map of Snake River Showing

Probability of Life Stage Impairment for Rainbow Trout
2-8

Additional Considerations

• Discussions of recovery should include a clear statement of the recovery goal (including

whether recovery is to be natural or “assisted”) and the potential impact of natural and

anthropogenic factors, including ecosystem interactions.

• If an adverse effect has a substantial probability, the risk assessor should explore the potential

for secondary adverse effects (e.g., loss of mussel species following loss of fish host species).

CHARGE QUESTION: How are alternative management options selected to have their

risks characterized and how is management informed of the consequences?

Interactions with risk managers

• Most participants who expressed an opinion felt that regular consultation with risk managers

and stakeholders is necessary throughout the process, especially when alternative management

options will be evaluated. A few mentioned that this suggestion should be added to the

Guidelines. It was recognized that such ongoing involvement has the potential to change the

direction of a study. 

• One participant noted that only rarely will an individual or distinct committee have complete

responsibility for making a risk management decision. It is more likely (as in the five case

studies in this pilot project) that the “risk managers” will include landowners, local officials,

state officials, environmental groups, news media, and many others. In order to be useful for

decision making in these situations, the risk characterization should reflect close adherence to

a collaboratively designed problem formulation and analysis plan. 

• A few participants and presenters felt it was not appropriate to consider management

alternatives in the risk assessment itself, and view risk assessment and evaluation of

management alternatives as separate activities.

• Ideally, if management alternatives are to be considered in the risk assessment, they should be

introduced by risk managers and discussed in the problem formulation phase. However, it

should be recognized that if management alternatives are crystallized too early in the process,

the risk assessment may eventually show that some of the favored alternatives are not feasible

or that some good alternatives were overlooked. In addition, it was pointed out that some

management options may be “discovered” during the risk assessment process itself. 

• Many believed the risk characterization section should cover both the current situation and

reasonable “what if” scenarios supplied by risk managers. “What to include” and “where to

stop” are affected by the type of management information needed (e.g., to determine if there is

a problem or if action is needed, to identify priority areas for action, or to determine the best

action to take). The risk characterization itself, however, was seen as a summary of the

science that explains the probability of an adverse effect on endpoints.
2-9

• Recognize that many individuals and organizations may need to be involved when a variety of

managers and agencies have authority over subsets of the watershed, stressors, and ecological

endpoints. 

Examples

• The approach being used in Waquoit Bay is to have the managers decide “how much is too

much” (i.e., “How much eelgrass loss is acceptable?”). Researchers are using models to

estimate the stressor load (in this case nitrogen) necessary to meet the goal and to evaluate

the effectiveness of specific management alternatives (see Figure 2-3).

• A “click-on” web site can be developed to allow planners to conduct “what if” scenarios to

evaluate various habitat and landscape feature management options for themselves.

CHARGE QUESTION: How should ecological risk be communicated to the manager and

the public (journal articles, computer programs, presentations, town meetings, Internet,

fact-sheets)?

Packaging the product

• There was considerable discussion during the workshop concerning who constitutes the

audience for the risk characterization. One primary audience is the risk manager(s) who needs

to be informed about the scientific basis for the conclusion in the risk assessment. The

stakeholders also represent an important audience. Stakeholders may be a diverse audience

depending on whether they are directly affected by management decisions or are more

interested in the technical, regulatory, or political aspects of the assessment. Length, detail

and style of the report should be tailored to meet the needs of the intended audience. 

 

• Many participants agreed that the risk characterization should be a technical, stand-alone

document. Yet, many others considered it acceptable to have separate versions of the risk

characterization report targeted for various audiences. One option many participants favored

was to have a single scientific report with “executive summary” and “relevant findings”

sections in “plain language” geared toward managers. An alternative suggestion was for the

risk characterization to be written in straight-forward language understandable by lay persons,

with technical appendices prepared for scientists to explain the studies, lines of evidence, etc.,

mentioned in the risk characterization report. A third alternative that several seemed to favor

was to have three reports: an executive summary for risk managers, a scientific report for staff

and scientists, and a colorful brochure-type product for the public. One participant

recommended that EPA consider field-testing targeted communication products to determine

which report format options were most effective. 

• One participant expressed an opinion that there should only be one risk characterization report

and that multiple versions were undesirable for decision-making on policy issues.
2-10

Figure 2-3. Waquoit Bay Case Study: Management Decision Process for Waquoit Bay

Assessment
2-11

• A few individuals suggested that non-technical stakeholders should be the primary target

audience. One participant pointed out that if the problem formulation incorporates significant

stakeholder involvement and reflects the values of the affected public, then the

characterization should respond to stakeholder information needs. 

• Progress and sample reports are very helpful. For instance, during the Clinch case study,

drafts of the report with preliminary findings were developed as progress reports to determine

how to best present and how technical the writing the final risk characterization should be for

the resource managers.

• Some though not all, thought that the first time a technical term is used in the report, it may be

helpful to highlight the word using a bold font, and then either define the term on that page or

put the term in a glossary. If a term is a central concept and will be used repeatedly, an

expanded definition (i.e., a paragraph or more) may be appropriate.

• Consider other options to paper products (e.g., the Internet, an interactive what-if scenario).

Use visuals

• Tables such as the one in the Snake River report (reproduced as Table 2-1 for this report) are

a good way to present the lines of evidence in one place.

• Color can be effective if used well, but the writer needs to take into consideration that some

readers will be color-blind. A good example of maps created to accommodate color blindness

can be found in Pickle, et al., 1996. 

• Help decision makers see the connection between human activities in a watershed and

achievement of the goals set for the assessment endpoints. This can be accomplished by

referring back to the conceptual model diagram developed during the problem formulation. 

The context in which decisions are being made should also be considered since the linkages

between natural and social systems are so strong that risk assessors cannot ignore them. We

must deal with all the compartments of the natural, land use, and socio-economic systems and

drivers affecting resource endpoints. 

Communicating with the public

• Besides the standard written report information can be disseminated via the Internet, videos,

print media, public meetings, school programs, fair exhibits, and public radio.

• In the Clinch case study, for the public, a more lay-version executive summary of the report

was prepared with input from the entire working group prior to distribution. Feedback from

public meetings was used to evaluate the effectiveness of the Executive Summary and to

determine whether other presentation materials were advisable. 

General communication principles

• Clearly state why the risk assessment was done (e.g., to support a particular management

decision or to explain an observed effect).
2-12

• Communicate early, often, and in a straightforward manner. The vast majority strongly felt

that regular interactions with risk managers were key to the success of the assessment. Some

felt that this should also apply to stakeholders, though others did not. One of the lessons

learned in the Middle Platte study was that it is critical to have the risk assessors and risk

managers in constant touch and that risk assessors need to be aware of local socioeconomic

and political concerns. Recognize that individuals and organizations are more apt to

participate or view the assessment differently when they may be directly affected by

management options under consideration. 

• Consider presenting the assessment as an opportunity to help stakeholders develop tools to

manage their watershed, as opposed to having EPA doing it for them.

C Follow the seven cardinal rules of communication (plan carefully; coordinate and collaborate

with other sources; involve the public; listen to the public; be honest, frank, and open; speak

clearly and with compassion; and meet the needs of the media) and “TCCR” (transparency,

clarity, consistency, and reasonableness) as outlined in Text Box 6-2 and Text Box 5-9,

respectively, in the 1998 EPA Guidelines. 

C Recognize that multiple stressors and large-scale, place-based efforts may make it necessary

to involve multiple resource management authorities.
3-1

SECTION THREE

SUMMARY OF OPENING PRESENTATIONS

This section presents a summary of the opening remarks and background information presented

during the first session of the workshop.

3.1 WELCOME AND INTRODUCTORY COMMENTS ON PLACE-BASED 

ECOLOGICAL ASSESSMENTS

Mike Slimak, Associate Director for Ecology, NCEA

In the early days of the risk assessment process, risk assessment was separate from risk

management. Emphasis then was on human health assessments, and cancer was the main

endpoint of concern. While health assessors never debated the “so what” question for cancer, it

has become a frequently-asked question in the realm of ecological risk assessment. It is therefore

necessary to integrate risk assessment and risk management from the very beginning of an

ecological risk assessment. Dialog during problem formulation is very important to ensure that

risk assessors and risk managers are thinking along the same lines. Selecting assessment

endpoints that the risk manager understands is crucial for successful risk characterization, which

in turn is the key to an effective risk management decision.

The purpose of this workshop is to learn from the experience the participants have in conducting

watershed-scale and place-based ecological risk assessments and to incorporate that knowledge

into improving EPA’s ecological risk assessment guideline products. Ecological risk assessments

are evolving from single species to multiple species, single stressors to multiple stressors,

chemicals to all stressors, and now, place-based assessments. The place-based approach takes

risk assessment to the next level, moving beyond the command-and-control approach of the past.

EPA has a major commitment to place-based assessments. Places represent the real world where

exposure to multiple stressors occurs with effects on multiple species in the context of real

ecosystems. Stakeholder involvement is integral to this approach and adds a sense of ownership

not seen with chemical-based risk assessments. Building and maintaining effective relationships

with stakeholders is crucial when working in these complex situations, and methods for creating

positive relationships with stakeholders is an area worthy of research. The scale issue is another

area worthy of research as the place-based approach is developed. Watersheds range in size from

small (e.g., Waquoit Bay) to mid-sized (e.g., Big Darby Creek) to very large (e.g., Mid-Atlantic

Integrated Assessment, or Environmental Monitoring and Assessment Program western pilot). 

The ecological risk assessment process consists of problem formulation, characterizing exposure

and effects in an analysis phase, and integrating those data in the risk characterization phase. Risk

characterization, which is the focus of this workshop, is the final phase of the risk assessment and

is the culmination of the planning, problem formulation, and analysis. However, risk

characterization actually begins during the characterization of ecological effects in the analysis

phase of the assessment. The characterization of ecological effects consists of three primary

elements that lead to derivation of the stressor-response curve:
3-2

• Plausibility that effects may occur as a result of exposure to stressors

• Relationship between stressor levels and ecological effects

• Linkages between measurable ecological effects and assessment endpoints when the latter

cannot be directly measured

Derivation of a dose-response curve is not an easy task for complex systems with differing

stakeholder values. Derivation of the curve can be a “nested” process, as in the Waquoit Bay case

study:

• risks to estuarine fauna as a function of submerged aquatic plants;

• submerged aquatic plants as a function of water clarity;

• water clarity as a function of algal blooms;

• algal blooms as a function of nutrient levels;

• nutrient levels as a function of nitrogen input; and finally,

• nitrogen input as a function of the density of septic systems.

Interpretation and presentation of the exposure and effects must contain a clear explanation of the

lines of evidence leading to the conclusions, including a description of the uncertainties,

assumptions, and qualifiers in the risk assessment. A compelling, well-communicated risk

characterization will allow risk managers to make a decision instead of asking, “So what?”

3.2 WORKSHOP GOALS AND APPROACH

Vic Serveiss, NCEA Environmental Scientist and Workshop Manager

Participants were invited to the workshop because of their experience in ecological risk

assessment, watershed management, or regional-scale assessment. The goal is to use the

collective wisdom of attendees to further develop and document a process for conducting

watershed ecological risk characterization.

The Framework for Ecological Risk Assessment (EPA/630/R-92/001) was developed in 1992. In

1993, EPA’s Office of Research and Development (ORD) and Office of Water (OW) selected five

demonstration sites to further develop, demonstrate and test the ecological risk assessment

paradigm by applying it to real world situations with multiple stressors. The five sites were

selected based on several factors including availability of data, willingness of organizations to

participate, and the existence of multiple stressors. In addition, the study sites came from a

variety of geographic regions, ecoregions, and EPA regions. The watershed assessment sites

chosen were:

• Big Darby Creek, Ohio; 

• Clinch Valley, Virginia; 

• Waquoit Bay, Massachusetts; 

• the middle segment of the Platte River, Nebraska; and, 

• the middle segment of the Snake River, Idaho. 
3-3

The plan is to complete individual draft assessment reports for each of these five sites between fall

1999 and December 2000, send the assessment reports out for peer review, and then publish

them as individual EPA reports. 

Many of the attendees have been involved in these assessments. As the five assessments

approached the risk characterization phase, several participants, also at this workshop, noted that

there was no specific guidance available on how to develop and present a watershed-scale

ecological risk characterization. The current guidance that is available, though predominantly

focused on single stressors and endpoints, includes: 

• Guidelines for Ecological Risk Assessment (U.S. EPA, 1998)

• EPA’s draft Risk Characterization Handbook (U.S. EPA, 1999)

• Improving Risk Communication (National Research Council, 1989) 

Such guidance is helpful, and participants had some ideas of their own on how to proceed, but

were also enthusiastic to get additional suggestions. Hence, a workshop seemed appropriate to

develop and document additional guidance for watershed and place-based ecological risk

characterization. To focus workshop discussions, several charge questions were developed that

address the issues of greatest concern for characterizing risk at the watershed scale. It is expected

that the workshop proceedings will develop answers to seven selected charge questions. These

“answers” will be used to format recommendations to supplement the existing body of literature

on risk characterization and will provide guidance to place-based risk assessors and managers. 

Draft watershed risk characterizations for the Clinch Valley and Mid-Snake assessments were

provided as background materials to participants. These documents should serve as helpful

references as responses to the charge questions are developed. A second anticipated benefit of

this workshop is to provide information and general guidance to help complete and improve the

five demonstration watershed-scale assessments. A draft report on the workshop will be prepared

with the help of the TVA and will be sent out for review by workshop attendees. Following a

peer review the final workshop products will be an EPA report and a journal article.

3.3 RISK CHARACTERIZATION SUMMARY

Bill van der Schalie, Ecologist and Guidelines Author, NCEA

Ecological risk assessment is a process in which all the elements build on each other. The risk

characterization phase of the process consists of risk estimation and risk description, followed by

presentation of the risk characterization findings, as shown in Figure 3-1. 
3-4

Ecological Risk Assessment

PROBLEM FORMULATION

RISK CHARACTERIZATION

Characterization

of

Exposure

Characterization

of

Ecological

Effects

ANALYSIS

Planning

(Risk

Assessor/

Risk Manager/

Interested

Parties

Dialogue)

As Necessary: Acquire Data, Iterate Process,

Monitor Results

Communicating Results

to the Risk Manager

Risk Management and

Communicating Results to

Interested Parties

Figure 3-1. Framework for Ecological Risk Assessment.
3-5

Risk estimation integrates the exposure and effects information. The Ecological Risk Assessment

Guidelines (U.S. EPA, 1998) reviewed a number of approaches that can be used, depending on

the situation and available data:

• Categories and rankings - Categories and rankings are a qualitative means of estimating risk. 

This method is useful in complex situations with little available data. Expert judgment is

required and should be documented. Categories and rankings are useful for stressor

prioritization, as in the Waquoit Bay problem formulation.

• Single point comparisons - Single point comparisons use the quotient method. This is a

good screening tool that is simple to use; however, it is not quantitative, does not cover

indirect effects, and does not consider uncertainty.

• Methods incorporating variability in exposure and effects - The incorporation of variability

can be accomplished using stressor-response curves or toxicity distributions. Advantages of

these methods are that quantitative measures show the magnitude of change in response to a

stressor and allow uncertainty to be addressed. However, this method does not consider

indirect effects and can address only some of the uncertainties.

• Process models - Mechanistic process models can be used to integrate exposure and effects. 

These models incorporate indirect effects, and are useful for scenario analyses. Disadvantages

are that validation is not always possible, and underlying assumptions may be hidden (i.e., not

transparent).

• Field observational studies - Field observational studies are advantageous because they

include direct measurement of exposure and effects, and they can take into account multiple

stressors and relationships. Disadvantages can include poor statistical power, lack of

replication, and difficulty in establishing causality.

It is clear that each of these methods of risk estimation has both strengths and weaknesses. 

Because no tool is perfect, the best option may be to use a combination of these approaches to

provide different lines of evidence.

Development of the risk description follows risk estimation in the risk characterization process. A

risk description interprets the significance of ecological risk by evaluating the available lines of

evidence and by determining the ecological adversity (the “so what” question). Interpretation and

discussion of lines of evidence should include data quality considerations, uncertainty

considerations, and relationships to assessment endpoints and risk hypotheses. Ecological

adversity considerations include the nature and intensity of effects, spatial scales, temporal scales,

and the potential for recovery.

The risk description should:

• Describe risks to assessment endpoints;

• Revisit critical problem formulation choices;

• Address the risk manager’s needs;
3-6

• Use varying lines of evidence; and 

• Highlight critical assumptions and uncertainties, while avoiding a laundry list of all potential

uncertainties.

Effective presentation of the risk characterization is the key to an effective management decision. 

The principles of ‘TCCR’ described in the EPA draft Risk Characterization Handbook (U.S.

EPA, 1999) are important to a successful risk characterization report and can be summarized as

follows:

• Transparency - Making the process open and frank helps achieve full disclosure of

assumptions and uncertainties and permits ready identification of default or policy-driven

assumptions. Key elements are to highlight the critical assumptions and uncertainties and their

effects on the results; separate science from policy (example - protection of 95 percent of the

organisms is policy); acknowledge any conflicting scientific interpretation of major issues; and

describe the level of confidence in major risk conclusions.

• Clarity - Making the product clear makes the assessment complete and understandable. Key

elements are that the level of writing should be appropriate for the audience; major risk

conclusions should be clearly stated; and graphs and diagrams should be simple and

straightforward.

• Consistency - Consistency provides a context for the reader by ensuring the assessment uses

an approach similar to other assessments (or offers an explanation if the approach or findings

differ). Key elements are to follow Agency-wide ecological risk assessment guidelines, if

appropriate; define and explain the risk assessment purpose and level of effort; and consider

results of other similar risk assessments.

• Reasonableness - The findings should be reasonable within the context of the best-available

science, the default assumptions, and the policy choices made. Key elements include ensuring

the risk characterization is based on the best scientific information available; using generally

accepted scientific knowledge in the assessment; making sure policy judgments use common

sense in light of statutory requirements and agency guidance; and presenting plausible

alternative risk estimates for the various management options under consideration.

In conclusion, risk characterizations should be prepared using principles from the Guidelines 

(U.S. EPA, 1998) and the EPA draft Risk Characterization Handbook, which is scheduled for

release in 2000. The major risk characterization challenges in place-based assessments include

complexity and multiple stressors, lack of data and large uncertainties, and providing clear

information in a usable format to risk managers.
3-7

3.4 TRADITIONS OF ENVIRONMENTAL MANAGEMENT AND THEIR

IMPLICATIONS 

Sue Norton, Ecologist and Guidelines Author, NCEA

Risk characterization is the phase of risk assessment during which the results and conclusions of

the risk assessment are reached and summarized in a way that is useful for managers. Risk

managers are defined as individuals and organizations who have the responsibility or authority to

take action or to require action to be taken (U.S. EPA, 1998). This definition encompasses a

broad spectrum from senior managers in regulatory agencies to the farmer who decides to build a

fence to limit livestock access to a stream. 

The linkage between risk characterization and the decision to be made (i.e., the link between

science and management) varies from one guidance document to another:

• Understanding Risk (NRC, 1996) - National Research Council (NRC) principles for risk

characterization iteratively link the characterization with the decision, beginning in the

problem formulation phase. It requires a broad understanding of harms and consequences and

is an outcome of on-going interactions between the scientific analysis and the societal

deliberation. 

• Risk Assessment and Risk Management in Regulatory Decision-Making (The

Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997) -

The framework diagram in this document shows no management interaction until near the end

of the process at decision-making time.

• Ecological Risk Assessment Guidelines (U.S. EPA, 1998) - Management options are

mentioned in the planning phase, but not in risk characterization.

• draft Risk Characterization Handbook (U.S. EPA, 1999) - Risk characterizations are written

in a generic way to support a variety of decisions. The document makes no mention of the

specific decisions the risk characterization is intended to support. 

There is no consistent guidance on the interface between risk assessment and risk management. 

As a consequence, some characterizations appear to be decision-driven, while others appear to be

a more generic presentation of information. 

Insights into effective ways to link science with decisions can be found by thoughtfully examining

the variety of environmental management processes for place-based ecological assessments,

including:

• Biological assessment

• Watershed approach

• Community-based environmental protection

• Watershed management

• Adaptive environmental assessment

• Adaptive management
3-8

• Regional vulnerability analysis

• Ecosystem management

• Decision analysis

• Natural resource damage assessment

• Risk Assessment for Superfund (includes the no action alternative and comparison of remedial

alternatives)

These approaches all seek to bring science into decision making, and have the common goal of

making a difference that will result in environmental improvement. Most are also geared toward a

specific decision. There are also ways in which the approaches differ. They may have different

starting points (e.g., effects, scenario, source, stressor); they may emphasize prospective or

retrospective points of view; they may build on different academic fields, traditions, and tools; and

they involve varying degrees of interaction with stakeholders. 

However, the two most important ways in which the approaches differ are: 1) the degree of

dispersion of risk management authority, and 2) the proximity of management authority to on-theground action. Decision making or management authority for different types of assessments may

be concentrated in one individual or management chain (e.g., Superfund assessments,

environmental impact assessments, biological assessments, decision analysis). Alternatively,

management authority may be more dispersed, as is the case with the watershed protection

approach, ecosystem management, futures analysis, and community-based environmental

protection. 

The full spectrum in the proximity of management to on-the-ground action can be represented by

the list of considerations and associated activities shown in Figure 3-2.

The implications of this discussion for watershed risk characterizations are that the five ongoing

watershed assessments support all these types of decisions, and may have a dispersed management

audience. As a result, watershed risk characterizations may have different formats. Although

there is currently little written guidance that is directly applicable to watershed risk

characterization, the variety of environmental management approaches discussed above provides a

wide range of styles and formats to build on. 

As we develop recommendations for conducting watershed ecological risk characterization and

for improving the five assessments underway, the following questions should be considered: 

• Where are the case studies on the decision continuum toward taking action on the ground? 

(The five case studies appear to be at the point of prioritizing issues/risks and deciding if

action is required. Waquoit Bay is further along and now involves considering different

management options.)
3-9

Proximity to action on the

ground

• Is there a problem that requires

further work or action?

– Futures analysis

– Ecological/Biological assessment

– Status and trends analysis

• Where are the priority areas or

subjects for action

– Regional vulnerability analysis

– Risk-based scoring schemes

• Is any action required at all?

– Superfund baseline risk assessment

– Most agency chemical assessments

• What is the best action?

– Environmental impact assessment

– Superfund remedial alternatives analysis

– Decision analysis

• Take an action already!

– Adaptive management

Figure 3-2. Proximity of Management to Action on

the Ground
3-10

• How dispersed are the people who will be making decisions based on our analysis?

• What type of decisions are our risk characterizations expected to support?

• How can we present our risk characterizations in a way that facilitates decision making?

One workshop participant suggested that the degree to which risk assessment information is

feeding into and impacting management decisions provides an indirect measure of success in

addressing the last of these four questions. 
4-1

SECTION FOUR

WATERSHED CASE STUDY PRESENTATIONS

This section provides summaries of the presentations made by chairs of the five EPA-sponsored

watershed ecological risk assessments. Each presentation included an overview of the case study

with emphasis on the risk characterization phase. Any discussion related to answering the charge

question, “How can the watershed assessment draft or plan be improved?” is included in this

section. 

4.1 BIG DARBY CREEK

Susan Cormier, EPA NERL, Big Darby Creek Assessment Co-chair

The Big Darby Creek watershed is located west of Columbus, Ohio, in the Eastern Corn Belt

Plains ecoregion. The drainage area of 1440 km2 (557 mi2) is predominately row crop agriculture

with forests along the stream. There are several small towns, but the area is relatively unimpaired

by urban and industrial use. The stream is both a Federal and State Scenic River and still has high

species diversity; however, there has been a 20 to 25 percent decline in aquatic diversity since the

1970s, and the area is in danger of losing additional species.

Based largely on stakeholder input solicited during the first year of the project, the following

management goal was developed for Big Darby Creek: “Protect and maintain native stream

communities of the Big Darby Creek ecosystem.” The approach to achieving this goal has been

to let the stakeholders know the assessment team was there to help them take care of the

watershed themselves, not to do it for them. A primary task of the assessment team has been to

develop new tools for conducting the assessment and identifying interrelationships among

stressors and endpoints.

One of the lessons learned in this case study is that the risk assessor should be able to show

exposure pathways and effects and explain them clearly to stakeholders. This helps the risk

assessor build personal and professional credibility in the eyes of the stakeholders. The

conceptual model has been helpful for this type of communication. 

Risk characterization to date has been mostly through presentations to stakeholders and at

scientific meetings. These presentations identified and explained key risks and protective factors

using some of the following specific methods and techniques:

• Box plots of Index of Biotic Integrity (IBI) metrics showing source signatures (see Figure 

4-1).
4-2

Figure 4-1. Big Darby Creek Case Study: Sample Box Plots of IBI Metrics

Showing Source Signatures
4-3

• IBI scores plotted on a map using different colors for categories of scores (although a

problem was noted with showing temporal variability and measurement uncertainty) (see

Figure 4-2).

• Plot of Invertebrate Community Index (ICI) versus river mile (see Figure 4-3).

• Trend analysis of response variables (IBI and ICI scores from 1970 to the present) using radar

plots (see Figure 4-4).

• Map showing erosion potential based on model predictions (see Figure 4-5).

• Trends in stressors using radar plots (see Figure 4-6).

• Longitudinal plots of stressors versus river mile showing variability and trend of medians (see

Figure 4-7).

The study team identified protective factors (e.g., steep gradient, presence of refugia, forested

streambanks, etc.) and key risks (e.g., livestock, housing developments, etc.). The results

suggested that useful management approaches would focus on the good stream segments by

protecting springs and groundwater, and preventing problem spots from impacting the good

stream segments. 

The presentations have been successful in prompting stakeholders to take action to improve water

quality. Actions taken to date include: removal of lowhead dams (with subsequent improvement

in fish communities); landowner action in response to erosion potential models; a proposal by the

U.S. Fish and Wildlife Service to designate a Darby Prairie National Wildlife Refuge (which

would protect approximately one-tenth of the study area, including the areas with the best aquatic

biological conditions and the recharge area); and use of the risk assessment results by the Natural

Resources Conservation Service to re-target their erosion control efforts. 

Additional technical risk characterization and management options will be considered later as new

tools are developed and better data become available. These actions may include development of 

a “click-on” web site (Ohio Watersheds Modeling Project) with models that allow planners to

conduct “what if?” scenarios to evaluate various habitat and landscape feature 

management options (e.g., if a development of a given size goes in at a location, what impact will

that have on endpoint measures?). The models would be based on empirical relationships that 

link IBI scores to “what if” questions associated with land use, wastewater discharges, etc.
4-4

Figure 4-2. Big Darby Creek Case Study: Map Showing IBI Scores By

Location
4-5

Figure 4-3. Big Darby Creek Case Study: Sample Plots of ICI Versus River Mile
4-6

Figure 4-4. Big Darby Creek Case Study: Sample Radar Plot of Trend

Analysis of Response Variables
4-7

Figure 4-5. Big Darby Creek Case Study: Map Showing Erosion Potential Based

on Model Predictions
4-8

 TSS 

 12:12, p=0.002*

AMMONIA

 2:7, n.s.

 NITRITE

9:10, p=0.017

 NO2+NO3

5:14, n.s.

PHOSPHORUS

12:12, p=0.002

Site Specific Stressor Trends

Relative Median Concentrations for Big Darby Creek

* one-tailed sign tests

79-81

90-93

YEARS

Stressor

n improved, n total, p*

Figure 4-6. Big Darby Creek Case Study Trends in Stressors 

 Using Radar Plots

Figure 4-6. Big Darby Creek Case Study: Trends in Stressors Using Radar Plots
4-9

Figure 4-7. Big Darby Creek Case Study: Longitudinal Plot of Stressor Versus 

 River Mile
4-10

4.2 MIDDLE PLATTE

Bob Fenemore, EPA Region 7, Middle Platte Assessment Chair, and 

Marla Downing, EPA Region 7, Former Middle Platte Assessment Chair

The Middle Platte study area encompasses 5,000 square miles, making it the largest of the five

case studies. The stream reach in the study area is 165 miles long. The watershed is 99 percent

agriculture and in private ownership. There are approximately 200,000 people living in the area,

and there are seven major communities, ranging in population from 1,500 to 42,000. 

The Middle Platte is a highly appropriated river system with intense competition for water.

Stakeholders in the area have a history of competition rather than cooperation, with little

communication. Key issues causing divisiveness included a dam and its operation, water flow

rights, and use impairments identified in a Clean Water Act Section 303(d) report. 

When the 1996 draft of the planning and problem formulation phase of the risk assessment was

issued, the stakeholders were observed to be less concerned with what the risk assessment said

than with who said it. One of the panel members pointed out that this demonstrates how the

personal credibility of the risk assessor is critical to how the message is received by stakeholders.

A key issue for this case study was how best to communicate a risk characterization to such a

diffuse, diverse, divisive, and untrusting group. Among the lessons learned were that it is critical

to have the risk assessors and risk managers in constant touch, and that risk assessors need to be

aware of local socio-political “hot buttons.” It is also important to know stakeholders on a

personal basis and to understand local communication styles. The chair of this assessment team

visited the affected Natural Resource Districts, municipalities, developers, environmental groups,

etc., several times a month to develop a trusting relationship with these stakeholders. 

EPA was particularly disliked by some stakeholders in the region, so it was important to separate

the risk assessment from any regulatory role. Local stakeholders were concerned that information

from the risk assessment could later be used in enforcement actions. However, even if that should

happen at some point in the future, stakeholders will have been involved throughout the process

so that findings will be representative of their concerns and should not come as a shock.

One panelist noted that it might help to clearly separate the management decisions from the

science. The speaker commented that close coordination between risk assessment and risk

management is necessary to be able to present the risk characterization in a palatable manner,

although as a scientist she agrees that they should be separate. It was also suggested that because

EPA is held in such low regard by some stakeholders in the region, it might be helpful to present

the assessment as an opportunity for the stakeholders to help develop the tools to manage their

own watershed, rather than having EPA develop tools to “regulate” them.
4-11

4.3 WAQUOIT BAY

Patti Tyler, EPA Region 1, Waquoit Bay Assessment Co-chair, and

Maggie Geist, Waquoit Bay NERR, Waquoit Bay Assessment Co-chair

The Waquoit Bay watershed covers 21 square miles on the southeast coast of Cape Cod,

Massachusetts. Initially valued for the hunting, farming, and fishing opportunities it provided, 

20 percent of the land is now used for residential purposes. There are seven subwatersheds with

different population densities. With sandy soils and low permeability, the watershed hydrology is

entirely groundwater driven, and groundwater quality is being impacted by the profusion of onsite septic systems and by contaminated groundwater plumes from the Massachusetts Military

Reservation (MMR). Nonetheless, biological resources are many and varied. There are a number

of critical habitats including eelgrass beds, coastal-plain, pond-shore communities, anadromous 

fish runs, salt marshes, shellfish habitats, and barrier beaches. There is a great diversity of

shellfish and finfish species. The area is along the Atlantic coast flyway, and there are numerous

shorebird species. Several federal and state listed birds and plants occur in the area. As a result,

Waquoit Bay has a variety of special considerations and designations, including being a National

Estuarine Research Reserve, a U.S. Fish and Wildlife Refuge, a Land Margin Ecosystem

Research Site, and a state-designated “Area of Critical Environmental Concern.”

Early in the assessment, the watershed stakeholder community was asked what they wanted to

protect and what they thought the stressors were in Waquoit Bay. This stakeholder input was

used to help develop a management goal for the risk assessment. A comparative risk analysis was

conducted to define which stressors and endpoints needed to be examined further. “Fuzzy Set”

decision analysis was used to rank stressors according to their relative contribution to risk to the

endpoints. Results of the decision matrix showed excess nutrient loading was the principal

stressor that was preventing attainment of the management goal for the watershed and that the

effect of nutrient loading on eelgrass beds was the primary stressor/assessment endpoint

relationship that needed to be investigated. 

The team is now developing a model to relate the response of eelgrass coverage to nutrient

loading within Waquoit Bay and eventually to predict future loadings under different development

scenarios (e.g., density of septic systems, which are the primary source of the nutrients). The plan

is to provide resource managers with the information needed to develop guidelines that would

protect estuarine resources. 

Exposure (i.e., nitrogen loading) was estimated using two models. The Nitrogen Loading Model

(NLM) was used to predict the nitrogen load from various sources based on assumptions about

landscape features, atmospheric deposition, groundwater movement, etc. This model tracked

nitrogen delivery to the water’s edge. The Estuarine Loading Model (ELM) was then used to

estimate the load of nitrogen in the estuary that was available to producers (i.e., the effect of

nitrogen load on algal growth).

Risk managers must first decide “how much is too much” (e.g., How much eelgrass loss is

acceptable?). Researchers can then use the models to estimate the N load necessary to meet the

goal selected by the managers and to evaluate the effectiveness of various management

alternatives. A description of how this will be accomplished was shown above in Figure 2-3. 
4-12

4.4 MIDDLE SNAKE

Patricia Cirone, EPA Region 10, Middle Snake Assessment Chair and Guidelines

Author

The Snake River study area is only a portion (a 160 km stretch of the total 1,000 km river) of the

watershed. A travelogue slide presentation along the Middle Snake River pointed out that 85

percent of the Idaho population lives on this stretch of the river, and 90 percent of the trout

served in U.S. restaurants are from aquaculture operations along this stretch of the river.

This assessment began in 1987, and analysis began in 1994. The Snake River Team is made up of

federal and state agencies, county and local governments, private organizations, and academia. 

Major stressors identified by the team include: flow diversion for irrigation and water supply,

flow alteration by impoundments, sediment and nutrient inputs from trout hatcheries, municipal

wastewater discharge, cattle grazing, feedlots, and returns from irrigated land. 

Much of the native fisheries resource has been lost. Five species of snails are endangered and

several others already extinct. Management goals were developed to address these and other

water quality related problems, as well as stakeholder interests related to recreational use of the

river. The goals included attaining state water quality goals, establishing Total Maximum Daily

Loads (TMDLs) for water quality-limited stretches, recovering threatened and endangered

species, sustaining economic activity, and increasing tourism and river use. 

The assessment endpoints drove the decisions for “what are we going to analyze?” In this case

the endpoints selected were growth and survival of cold water biota (Mountain Whitefish, White

Sturgeon, Trout); growth, survival, and diversity of molluscs; and growth of aquatic macrophytes

and algae (which drove most of the public interest). Measures of effect selected for analysis were

water quality criteria (dissolved oxygen, temperature, ammonia, total phosphorus and nitrogen);

Habitat Suitability Indices; and presence, absence, and abundance of flora and fauna. 

Risk estimation was based on four lines of evidence. These were:

• Field observations and studies (empirical models)

• Literature (field and best professional judgment)

• Stochastic or probabilistic estimates for habitat suitability and water quality criteria

• Process models for eutrophication

Estimates of the likelihood of recovery were also developed and presented using both figures and

tables in the case study report (see Figure 4-8 and Table 2-1 of this report). Recovery is an

endpoint that is important to stakeholders and is easily understood. While there was much praise

for these figures and table, one comment to improve the table was to include citations to technical

references or, in the cases where best professional judgment was used, identify the scientist(s)

drawing the conclusion.
4-13

Figure 4-8. Middle Snake River Case Study: Illustration Linking 

 Stressors to Recovery Potential for Rainbow Trout
4-14

It is anticipated that the information in the ecological risk assessment can be used to help establish

total maximum daily loads (TMDLs) for river sections suffering water quality impairment; used in

the review of permits for stream-affecting activities; and used to help evaluate management

options for identification and control of nonpoint source pollution.

4.5 CLINCH VALLEY

Jerry Diamond, Tetra Tech, Inc., Clinch Valley Assessment Principal Investigator

The study area for the Clinch Valley assessment case study is in the unimpounded portion of the

basin in Virginia. There has been an enormous decline in the fish and mussel species diversity. 

Stressor sources include dams, logging, some crops, diminished riparian corridor, mining, spills,

and livestock pastures.

The management goal developed for the Clinch Valley ecological risk assessment was to maintain

or restore the biological diversity of the Clinch-Powell watershed surface and subsurface aquatic

ecosystem. Assessment endpoints selected for evaluation were native mussels, native fish and

cave-dwelling species. (Ultimately, cave-dwelling species were not considered as part of this

assessment due to lack of data.)

The Clinch Valley assessment is analyzing: 

• The upland and riparian land use and its relationship to in-stream habitat quality and mussel

distribution and abundance;

• Riparian vegetation, connectivity, and width in relation to measures of in-stream resource

quality;

• Temporal trends of water quality and mussels; and,

• Water quality, land use, and riparian corridor characteristics in relation to distributions of

mussels and native fish.

Scale issues were paramount. The primary question for land use was, “What scale of information

is related to the biological endpoint measurement?” 

Preliminary conclusions to date include:

• Cumulative impacts appear to be responsible for the decline in species richness and

distributions. A stressor index was developed to indicate cumulative impacts based on

proximity (> or <2 km) to pasture/cropland, urban areas, mining, and major transportation

routes.

• As the number of stressors identified in an area increased (0, 1, 2, or 3), the maximum number

of mussel species present declined. 
4-15

• Percent forest cover is not necessarily an accurate indicator of in-stream conditions due to

effects of other factors, especially close proximity to mining.

• A relatively large riparian area upstream of a site can have substantial effects on in-stream

habitat quality and distribution of mussels and fish. The best fit (predictor) between biological

endpoint measures and riparian zone size was with an area 100 m wide on each side of the

river that also extended 1,000-1,500 m upstream of the site. In narrow floodplains or areas of

higher gradient, a larger upstream reach needs to be considered.

• Relationships which were found to be significant included cropland (over 3 percent) versus

sedimentation, pasture versus riparian vegetation integrity, and urban/barren versus quality of

in-stream cover.

• Fish IBI was a better predictor of mussel populations than was EPT (sum of Ephemeroptera,

Plecoptera, and Trichoptera taxa in a sample of benthic macroinvertebrates).

The major stressors for mussels were found to be proximity to mining (<1km), proximity to

transportation routes (spill sources <1km), percentages of urban area (>10 %), and percentages of

cropland (>19 %). 

Uncertainties:

• Limitations of the Data - Habitat data were qualitative and not always available from the same

sites as biological data; taxonomy for EPT was based on family-level taxonomy (genus may

have yielded more conclusive results); water quality data were sparse; flow data were sparse

and difficult to relate to the risk assessment.

• Sources of Uncertainty - Differences in sampling sites for mussels, fish and EPT; scales used

for calculating land use percentages may not be transferable from stream to stream; and given

that not all land uses co-occur everywhere, apparent correlations with a source may actually

be due to presence or absence of other sources of stress.
5-1

SECTION FIVE

FACILITATOR PRESENTATIONS

5.1 Jack Gentile, Center for Marine and Environmental Analysis, University of

Miami“Large-scale Assessment and Risk Characterization - The South Florida

Regional Assessment as an Example”

The South Florida Regional Assessment and Restoration Strategy for the area of Florida south of

Lake Okeechobee (including The Everglades), is a risk assessment that combines both

retrospective and prospective elements in understanding the future risks associated with specific

restoration actions. 

Restoration goals were developed by the Governor’s Commission for a Sustainable South Florida. 

This commission met quarterly and was tasked with developing the societal and environmental

goals for the restoration. The scale and complexity of this activity have tremendous associated

uncertainty; consequently, a decision was made to construct conceptual models for the whole

system. The first technical problem facing investigators was how to partition an area as large as

this (i.e., approximately 3,000 mi2 ). Ultimately, the study area was characterized as a mosaic of

13 areas with distinct hydrology, habitat assemblages, and habitats. Alternatively, the assessment

could have labeled several habitats, each occurring in one or more areas.

Conceptual models developed for each of these landscape units allowed the development of 

hypotheses that described the causal basis for the current state of the ecosystem as well as for

future recovery. These 13 landscape units became the focus of the South Florida Restoration

Strategy to recover south Florida and the Everglades. 

The assessment was a tool for managers who needed a method for seeing how “what if” scenarios

affected the various endpoints of concern to them. The stress-response relationships developed

were the engines driving the analysis. The risk characterization involved building dynamic

visualization tools. 

In the analysis phase, a 2- by 2-km hydrodynamic model was used to develop exposure

information based on a 30-year record of precipitation data. Model outputs of stage height,

duration, and flow were generated for various climate scenarios. Stressor-response curves were

developed for relationships between percent cover types and hydrologic parameters. 

The effects that optimizing hydrology would have on other endpoints (e.g., wading birds,

panthers, threatened and endangered species) became a major issue. The model indicated that it

was desirable to flood certain areas, but flooding would have had significant social effects (Native

Americans now own the area) and ecological effects (habitat of the endangered Cape sparrow

would have been eliminated, although the species did not occur in this area until the land was

drained by the historical diversion of water). The difficult management decisions have not yet

been made. 
5-2

Information on the assessment has been relayed via area-wide public meetings of the governor’s

commission. The group has also prepared technical reports, had pieces in publications, and

developed brochures and fact sheets several times during the process. 

The South Florida case study illustrates the full application of the ecological risk assessment

paradigm. Problem formulation was used in a retrospective or deductive manner to identify the

sources of the problem as well as to characterize the current state of the system and understand

how it got to be where it is today. Hypotheses were developed for actions that, based upon

several lines of evidence (e.g., historic, baseline monitoring), would lead to recovery. The

Analysis Phase or inductive/prospective component of the guidelines included the examination of

a variety of scenarios with sensitivity analyses to identify the most important variables in the

system. In the Risk Characterization Phase, the outcomes were examined again, using multiple

lines of evidence and often weight of evidence to help optimize the recovery in different landscape

units. While numerical models played an important part in forecasting recovery, expert judgment

was equally important. And, in fact, the latter often had a much higher degree of confidence than

the former. 

5.2 William Smith, Yale School of Environment and Forestry Studies, Yale University

The watershed concept has utility in that it is universal, allows for examining effects on different

scales, allows for hypothesis testing, and is useful as an environmental management unit. Three

goals of regional or watershed-scale risk assessments are to explain observed effects, evaluate

actions with regional implications, and evaluate the status of key endpoints.

Successfully meeting these challenges involves developing good conceptual models with sourcestressor-effects linkages, and comprehensively including what we know to be important. Clarity

on the purpose of the risk assessment (e.g., whether the assessment is being done to answer a

scientific question or to support a management decision) is also critical. 

Ecological risk assessors should move away from the term “uncertainty” and think in terms of

“degree of confidence.” But while risk assessors should strive to be honest and clear about

uncertainties in the assessment, they should also avoid being hamstrung or silenced by those

uncertainties. Most major risk management decisions have a political context, and decision

makers do not usually need the 95 percent confidence level that scientists strive for. In the real

world, risk management decisions will continue to be made even in the face of great uncertainty. 

Several panelists pointed out that the perceived need for certainty increases when risk

management decisions involve the expenditure of large sums of money or an attempt to change

ingrained land management practices. Another panelist commented that the perceived need for

certainty may decrease when it is very expensive to collect and analyze the data to increase the

level of confidence, or when the level of uncertainty ceases to be significant relative to the specific

endpoint of interest.

Focusing specifically on the charge questions for the risk assessment case studies, Dr. Smith

offered the following suggestions:
5-3

• Clearly state why the risk assessment was done (e.g., to support a particular management

decision or to explain an observed effect). This should be repeated in the risk characterization

even though it was stated initially in the problem formulation. 

• Consider the potential benefits of a ranking or weighting system based on relative importance

when dealing with qualitative lines of evidence and focus first on those judged most important. 

A ranking approach would also be useful in discussing the degree of adversity of predicted or

observed effects in a watershed.

• When appropriate, include and develop qualitative lines of evidence. Sometimes both

qualitative and quantitative lines of evidence are required to “tell the story” effectively.

• Keep risk assessment and evaluation of management options as two separate activities.

• Risk assessment reports should be polished, highly readable, professional-looking

communication products. They should include a high quality executive summary targeted to

decision makers.

5.3 Patrick Bourgeron, INSTAAR University of Colorado, Colorado State University

The linkages between natural and social systems are so strong that risk assessors cannot ignore

them. We must deal with all the compartments of the natural, land use and socio-economic

systems and drivers affecting resource endpoints. Too often, there is a disconnect between the

system components considered by landscape ecologists and those considered by land or regional

planners. 

Selecting appropriate data for the scale examined is a crucial decision in assessing risk. In

general, moving toward more conceptual viewpoints of the landscape (i.e., going from the real

landscape, to spatial representation in mapping, to including temporal and dynamic features, to the

strategic level of considering scenarios) necessitates a movement from high levels of precision to

more general probabilities. In addition, as the domain under consideration enlarges (i.e., from

local, to sub-region, to watershed, to regional), the “grain” of data changes from fine, to medium,

to coarse. Use of finer or coarser data than is appropriate for a scale will cause serious problems

in the ability to discern exposure-response relationships, interpret results, and define uncertainty. 

There are typically several spatial areas to be considered in developing large-scale assessments. 

Ideally, the areas are one and the same, but this situation rarely occurs. The areas to consider are: 

• Assessment area (the area over which data are available or will be collected)

• Analysis area (the area over which analyses are conducted)

• Planning area (the area over which community planning will occur) 
5-4

• Response units (the lowest level on which you can expect a response)

• Cumulative impact areas (the area that management or land use activities will affect) 

• Reporting Units (the area on which the risk assessor is reporting)

Tables presenting accuracy/precision versus cost are one method to communicate key information

about the level of uncertainty. The information may be expressed as simply as low, medium, or

high ranges. This information can be used in discussion of the appropriate level of effort in

minimizing uncertainty. 

One panelist commented that it might be useful to include an accuracy/precision versus cost table

in the risk assessment to help communicate the justification for accepting a particular level of

uncertainty. He also noted that there should be consideration of whether the increased cost of

achieving a higher level of confidence produced a comparable value-add in decision quality. 
6-1

SECTION SIX

BREAKOUT SESSION SUMMARIES

This section presents a summary of each team discussion during the breakout session.

6.1 BREAKOUT TEAM 1

Jack Gentile, Facilitator

CHARGE QUESTION: How should exposure and effects data be integrated in a

watershed context to generate a risk estimate? 

• There is value in using a “process diagram” similar to that used in the Clinch Valley

presentation (see Figure 2-1) to summarize what could or could not be addressed with the

available data. To be effective, such a diagram need not be a complete reproduction of the

detailed conceptual model, but rather should provide an “in a nutshell” review of what was

addressed and what the key issues are. The approach would be especially useful in situations

where the risk characterization focuses on a single assessment endpoint within a complex

ecosystem. A similar diagram could also be developed to highlight where the major

uncertainties lie.

• Stakeholders may only be interested in certain endpoints or certain sub-areas of a complete

watershed and the characterization should place greater emphasis on those parameters.

• Although the risk assessor may recognize complex interrelationships between stressors, or

between a single stressor and multiple effect endpoints, models tend to approach such

relationships linearly and one at a time. A potential approach to writing a risk characterization

within this framework of complexity begins with a reiteration of the goals of the analysis and

the linkage between those goals and the assessment endpoints. The second step would be

presentation of a simplified version of the conceptual model and a summary of why the

assessment focuses on selected endpoints within a fabric of cumulative effects. From this

point, the risk characterization would proceed into a summary of the cause-and-effect

relationships or benchmarks that will be used to evaluate exposure data. 

CHARGE QUESTION: How should uncertainty be addressed and presented (e.g.,

incomplete data or analyses, qualitative estimates, data at different spatial and

organizational levels)?

• Discussions of uncertainty should focus on the key uncertainties (i.e., those that could change

the decision) rather than simply reciting a “laundry list” of all possible uncertainties. 

• When uncertainties are due to a gap in the available data, the risk characterization should

make recommendations for filling in the data gaps. The text might preface the
6-2

recommendation with a statement such as: “If ‘X’ data were available, the probability of this

response could be predicted with ‘Y’ degree of confidence.”

• When the data are not adequate to develop a stressor-response curve, the risk characterization

should assess the strength of the association between putative stressor and effect. The

references on biological inference cited in the Guidelines for Ecological Risk Assessment

(e.g., Fox [1991], Hill [1965], Susser [1986], and Suter [1993]) were identified as providing a

useful conceptual framework for this analysis.

• When conclusions are based on comparison of data with benchmarks (such as water quality

criteria), the uncertainties inherent in the development of the benchmarks should be

acknowledged.

• The group discussed the differences in how risk assessors and risk managers might view

uncertainty in the analysis. For example, a risk manager might choose to act based on a

potential for an adverse effect, even if there is no conclusive evidence of causality. 

Discussing the degree of confidence in the results is a critical step in developing a credible risk

characterization even if the uncertainties are only a minor consideration in the risk

management decision making process.

• As scientists, we tend to demand a high degree of confidence before we present results. 

However, much of the uncertainty in ecological risk assessment cannot be captured in terms

of standard statistics, even in those cases where quantitative information on the variability of

exposure or endpoint data is available. Nonetheless, a qualitative, semi-intuitive estimate of

confidence can usually be made based on best professional judgment, and this estimate of

confidence may be adequate for the risk manager. 

CHARGE QUESTION: What are the best ways to address and present the integration of

qualitative and quantitative lines of evidence for a) an individual assessment endpoint and

b) drawing overall conclusions?

• The group liked Table 17 on page 99 (see Table 2-1) of the Middle Snake River report as a

way of displaying and summarizing multiple stressors and lines of evidence. The addition of a

“references cited” column on the right-hand side of the table was identified as a potential 

improvement in the format. The group suggested that a table of this type, if used, should be

supplemented by a narrative treatment of the various lines of evidence.

• Any contradictions between the various lines of evidence should be explored and, if possible,

explained. The text should address whether the contradictory lines of evidence nullify the risk

assessment process or whether it is reasonable to proceed with the risk characterization.

• The agreement or concordance of various lines of evidence, which individually may have a

high degree of uncertainty, can strengthen confidence in the conclusion. 
6-3

• One participant noted that EPA’s Office of Research and Development and Office of Water

are presently developing a document that addresses stressor identification and evaluation

within the context of defining causal relationships.

• The group cautioned against numerically weighting qualitative data.

 

CHARGE QUESTION: How should the degree of adversity of predicted or observed

effects in watersheds be described (e.g., considering nature and intensity of effects, spatial

and temporal scale, and the potential for recovery)?

• A variety of qualifying and modifying factors should be addressed in a description of

adversity, including:

- spatial scale

- temporal scale

- biological scale (i.e., are adverse effects apparent on an individual organism scale

or on a population scale?)

- functional redundancy in the ecosystem that may obscure or limit apparent adverse

ecological effects (e.g., species composition may change, but primary production

rates may not). 

• If the potential for recovery is an important part of determining the degree of adversity,

address the following:

- probability of recovery

- spatial and temporal scale of recovery expected (including alerting the risk

manager to any potential “quick successes”)

- whether recovery will be natural or “assisted” (e.g., species reintroductions,

population supplementation by stocking)

- what other factors (natural and anthropogenic) might limit recovery when the

primary stressor is reduced or eliminated.

CHARGE QUESTION: How are alternative management options selected to have their

risks characterized and how is management informed of the consequences?

• A question was raised about whether it was “heresy” to address management alternatives in a

risk assessment (based on the separation of risk assessment from risk management in the NRC

analytical deliberative paradigm).

• Depending on the purpose of the risk assessment (e.g., characterizing ecosystem dynamics

versus supporting informed decisions by risk managers), evaluation of management

alternatives may or may not be an integral part of the effort. If it is the goal of the risk

assessment to evaluate management alternatives, this should be clearly reflected in the

problem formulation. However, the group noted that even if the ultimate purpose is to

support evaluation of management alternatives, the suite of alternatives may be developed

through an iterative process that involves bringing stakeholders into the process. 
6-4

CHARGE QUESTION: How should ecological risk be communicated to the manager and

the public (journal articles, computer programs, presentations, town meetings, Internet,

fact-sheets)?

• Set the stage for clear communication in the risk characterization by briefly reiterating the

goal(s) of the risk assessment (e.g., characterizing a complex system; providing the

information to support a specific management decision).

• Ecological risk communication should follow the “7 cardinal rules of communication” as listed

in Text Box 6-2 of the 1998 Guidelines for Ecological Risk Assessment.

• The risk characterization should strive for TCCR as discussed in Text Box 5-9 of the 1998

Guidelines for Ecological Risk Assessment.

• The risk characterization presentation style should be tailored for the intended audience. The

potential audiences are diverse and range from other environmental professionals to

congressional representatives to land owners. At a minimum, there should be a scientifically

credible, reasonably detailed and technical version that summarizes and documents the results

of the assessment for the technical reader. Depending on the goal(s) of the assessment, it may

be desirable to prepare a relatively nontechnical executive summary suitable for a broad

audience. The executive summary should be tightly focused, and possibly arranged in a

“bulleted” format. 

6.2 BREAKOUT TEAM 2

William Smith, Facilitator

CHARGE QUESTION: How should exposure and effects data be integrated in a

watershed context to generate a risk estimate? 

• Output from mathematical models comprised a large portion of the case study presentations,

and the breakout group was surprised at the limited use of mapping overlays to summarize

results. Although several presentations used mapping overlays, there was a feeling that there

were many additional opportunities to use the technique. When used effectively, mapping

overlays can make the data more user friendly and aid the scientist in understanding it. For

example, in the Clinch Valley case study, maps of endpoint measurement data and stressor

data were effectively used to portray relationships and linkages (see Figure 6-1). 
Figure 6-1. Clinch Valley Case Study: Map Showing Land Use Versus Threatened & Endangered Species

6-5
• The group felt that examining small subwatersheds was an effective way to decrease

complexity of analysis and presentation, and to focus on the effects of individual stressors. 

Subsequently (as in the Clinch Valley case study), the analysis can be re-expanded (with

assumptions clearly identified) to evaluate the cumulative impacts of stressors. One

participant noted that the Big Darby investigators had used an analogous approach by sorting

out the stressors using Principal Components Analysis (PCA) and then developing a

multivariate model that “reassembled” the effects.

• Ranking techniques and development of stressor indices (but not weighting) for the watershed

were considered reasonable methods of assembling the data into usable information. (There

was no detailed discussion on either of these approaches.) 

• The picture charts and tables relating stressors (see Figure 4-8 and Table 2-1, respectively),

lines of evidence, description of risk, uncertainty, assumptions and recovery potential from the

Snake River case materials in the workshop handout binder were considered to be the best

examples of integrating and presenting information.

CHARGE QUESTION: How should uncertainty be addressed and presented (e.g.,

incomplete data or analyses, qualitative estimates, data at different spatial and

organizational levels)?

• Discussion began with the level of comfort of risk assessors in addressing uncertainty, and the

participants warned against being paralyzed by “uncertainty block.” 

• Risk assessments are likely to be questioned at some point, and one group member remarked

on the fear that a risk assessment might be challenged in court. Another participant noted that

in the NEPA arena courts tend to find for the initial investigator rather than the challenger as

long as the investigator has conducted and documented a “reasonable” scientific approach to

assessing the magnitude and likelihood of impacts. It is generally sufficient to show that the

issue has been considered, the investigator has used a reasonable scientific process to reach

conclusions (conducting studies, developing conceptual linkages and consolidating lines of

evidence), and the degree of uncertainty has been identified. “Sins of omission” (e.g., leaving

out an important stressor or endpoint) are worse than “sins of inaccuracy.”

• Paying due respect to gaps in conceptual models is critical. 

• The levels of confidence needed for scientist and managers differs. Risk managers make many

decisions with levels of uncertainty that are considered high for the risk assessor. The

confidence level needed may be affected by the management decision to be made (e.g., it may

be correlated with capital involved and political context). In public policy decisions, the

uncertainty about effects from economics or cultural issues that influence decisions may be far

greater than the uncertainty associated with factors in the risk assessment. This does not,

however, let the investigator off the hook: uncertainty should be clearly addressed with good

science and judgment.

• Uncertainty can be captured either quantitatively or qualitatively, depending on what is most

appropriate for the data or information being considered. Quantitative data should be used
6-7

whenever possible, but in many cases, uncertainty may simply be characterized as “high,”

“medium” or “low.” The appropriate level of uncertainty analysis may be non-quantitative for

some data or lines of evidence. 

• The objectives should be to lay out the degree of uncertainty with clarity, honesty and

openness; and to be as comprehensive as possible in your "uncertainty" inventory. 

• It is often helpful to reference standards, tests, and accepted metrics (e.g., IBI) in support of

level of uncertainty.

CHARGE QUESTION: What are the best ways to address and present the integration of

qualitative and quantitative lines of evidence for a) an individual assessment endpoint and

b) drawing overall conclusions?

• For the technical audience, use an approach similar to that of the Snake River presentation

was good (see Table 2-1). The advantage of a tabular presentation was that the information

regarding the lines of evidence (e.g., model, field observations, similar sites comparison, etc.)

could be seen in one place. Use of qualitative categories (high, medium or low) was both

appropriate and useful in some instances. The addition of numeric references at the side of the

table was suggested as a useful change in format. 

CHARGE QUESTION: How should the degree of adversity of predicted or observed

effects in watersheds be described (e.g., considering nature and intensity of effects, spatial

and temporal scale, and the potential for recovery)?

• The Snake River table (see Table 2-1 above) in conjunction with the maps showing degree of

impairment for various life stages along the stream reach (see Figure 2-2 above) are good

formats to follow. 

• Temporal scale analyses and presentations should indicate trends for stressors and endpoints. 

Participants noted the need for caution about making subjective judgments. 

CHARGE QUESTION: How are alternative management options selected to have their

risks characterized, and how is management informed of the consequences?

• Management decision making goes on throughout the risk assessment and continues after it is

completed, yet risk assessments need closure.

• Management options and issues should be identified to the extent possible in the

planning/problem formulation stage, but risk managers (whether urban planners or farmers)

should be involved along with the risk assessor in the iterative development of risk scenarios

during the risk assessment process.
6-8

• The risk characterization section should cover both the current situation and reasonable “what

if” scenarios supplied by risk managers. “What to include” and “where to stop” are affected

by the type of management information needed (e.g., to determine if there is a problem or if

action is needed, to identify priority areas for action, or to determine the best action to take). 

The risk characterization itself, however, is seen as a summary of the science that explains the

probability of an adverse effect on an endpoint.

• The risk assessor should not feel obligated to go further than covering the management

options that were identified in the planning/problem formulation stage.

• When considering management options, recognize that many individuals may need to be

involved because different managers and agencies may have authority over subsets of the

spatial scale, stressors, and ecological endpoints.

CHARGE QUESTION: How should ecological risk be communicated to the manager and

the public (journal articles, computer programs, presentations, town meetings, Internet,

fact-sheets)?

• There is a strong need for continued communications to “bring the public along all the way.” 

This is especially true when considering management options. 

• Although somewhat simpler in presentation, executive summaries should have enough “meat”

to give the reader the full understanding of major risks, stressors, and impacts to important

endpoints identified in early stakeholder discussions. 

• Other routes for sharing risk assessment information that were either done in the course of the

demonstration risk assessments or were thought to be good ideas include: exhibits at a county

fair, school audiences, use of print media and development of “slicks” for the public as well as

a technical report for the scientific community. Full text versions of risk assessments should

be made available on web sites.

• Recognition of your audience was a major point of discussion. A key question is, “Do you

know who your audience is?” There were three basic audiences identified: risk managers, the

public, and other scientists. To be maximally effective, correct identification of the

management audience was thought to be critical. Reports should be appropriate for 1)

scientific review; 2) risk management utility; and 3) public access and readability.

• Tables such as the one in the Snake River report (see Table 2-1 above) are a good way to

present the lines of evidence in one place. Using qualitative categories (high, medium, or low)

can be useful sometimes. The addition of numeric references at the side of the table was

suggested as a useful change in format. 

6.3 BREAKOUT TEAM 3

Pat Bourgeron, Facilitator, and
6-9

Patti Tyler, Report-Back Volunteer

CHARGE QUESTION: How should exposure and effects data be integrated in a

watershed context to generate a risk estimate? 

• The dose-response curve format is useful for a single stressor dose-response (e.g., using

eelgrass as an assessment endpoint for nutrients in the Waquoit Bay assessment).

• Multivariate analysis can be used in a system with multiple stressors to help determine which

stressor is having the greatest impact.

• The group liked Table 17 on page 99 of the Snake River report (reproduced as Table 2-1 for

this report) as a way to display and summarize the impact of multiple stressors and lines of

evidence. A table of this type can capture both quantitative and qualitative evaluations. 

• Evidence of exposure, evidence of effects, magnitude of effects, confidence, and risk potential

can all be ranked as high, medium, or low. This qualitative evaluation can be quite useful. 

For example, if there is medium confidence of low risk, it is unlikely that additional data

would change the conclusion that the risk is low.

• Sometimes it may be best to use best professional judgment rather than to try to quantify. For

instance, better information may be derived from the best professional judgment of a local

biologist than from an in-depth study of limited data.

• The group discussed the scale issue and how to extrapolate from subwatersheds to entire

watersheds. Applying specific subwatershed relationships to entire watersheds works better if

the entire watershed is similar in nature. It is possible to develop templates for different

habitats which may then be extrapolated to similar locations rather than the whole watershed. 

In the Clinch Valley Assessment, data from study stations were used to define trends based on

water quality data, land use, habitat, etc., which were then extrapolated to specific sites with

mussels to help explain existing conditions or help predict future recovery.

• It may not always be appropriate to manage a stressor for the whole watershed if only a small

or specific area needs to be protected.

• It is also good to be able to present risk estimates at different scales. Data should be used for

an appropriate area and reported for an appropriate area.

• The relationship between exposure and effects must be communicated so that the manager can

understand all the links between the stressor and the endpoint. Plugging data into the

conceptual model is one way to demonstrate effects, and it is best if it can be shown at

different spatial scales.
6-10

CHARGE QUESTION: How should uncertainty be addressed and presented (e.g.,

incomplete data or analyses, qualitative estimates, data at different spatial and

organizational levels)?

• Be aware that users of the report aren’t as interested in uncertainty as are scientists.

• Nutrient modeling is an example of where a model can demonstrate variability. Confidence in

predictive numbers, limitations of model output, standard deviations, standard errors, etc., can

be determined.

• When it is not possible to describe uncertainty quantitatively, references to the literature may

help support a conclusion.

• One participant commented that variability cannot be reduced, but that the associated

uncertainty can be defined/reduced through use of probabilistic analysis to reach a quantitative

estimate of uncertainty. Uncertainty factors may also be useful in some situations to account

for uncertainties associated with extrapolation.

• Describe how uncertainty is based on available data and analysis. For example, in the Clinch

Valley Assessment a less than optimal taxonomic level was used for the benthic

macroinvertebrate analysis. Investigators questioned whether a benthic macroinvertebrate

index could serve as a surrogate for mussels in their study. The fish IBI had a much stronger

correlation with the mussels than did the EPT index in that study, possibly because taxonomic

data below the family level was unavailable.

• At the end of the process the risk assessor should address which uncertainties have the most

impact on the conclusion. Uncertainties should be ranked for the risk manager. 

• The degree of uncertainty should be addressed as the degree of confidence in discussions with

the public. While it is better to present what we are confident about, uncertainties in some

exposures will still have to be explained. Example: A certain degree of change in an exposure

may not predict with certainty that recovery will occur because of uncertainties in the

exposure.

• Risk characterization should consider the risk management context. For example, reducing all

uncertainties in data may be meaningless if there are no feasible management alternatives.

CHARGE QUESTION: What are the best ways to address and present the integration of

qualitative and quantitative lines of evidence for a) an individual assessment endpoint and

b) drawing overall conclusions?

• The team recommended that conclusions be presented in a summary table with references to

the location of in-depth data. Table 17 on page 99 of the Snake River report (reproduced for

this report as Table 2-1) is a good example. One participant noted that a similar approach is

described in a report from the Society for Environmental Toxicology and Chemistry
6-11

(SETAC) Pellston Workshop on Multiple Stressors in Ecological Risk and Impact Assessment

(Pellston Workshop Report is now available. See Foran and Ferenc, 1999). The Pellston

approach uses a series of matrices to 1) show stressor/risk rankings which provide managers

with relative rankings of stressors within each scenario, including uncertainty information; 2)

show information about the magnitude, duration, and spatial extent of a stressor and the

probability the effect will be expressed, including uncertainty information; and 3) allow

development of a set of management scenarios with associated sets of stressors and

management options.

• The team agreed that maps showing stressors and assessment endpoints are excellent tools for

presentation; however, it was pointed out that people often don’t realize there are

uncertainties involved (accuracy and confidence associated with points on the map) when

viewing map presentations. Web sites with maps were also suggested.

• Multiple stressors and multiple endpoints often make it difficult to concisely state a

conclusion. Use of a matrix, summary table, or the conceptual model may help in presentation

of the results and conclusions. 

CHARGE QUESTION: How should the degree of adversity of predicted or observed

effects in watersheds be described (e.g., considering nature and intensity of effects, spatial

and temporal scale, and the potential for recovery)?

• The Snake River case study is an example where it is easy to illustrate present versus historical

conditions using photographs and maps.

• Maps showing individual stressors help explain the overall situation, especially if there is some

ranking or scoring system.

• Recovery potential is included in Tables 17 and 19 on pages 99 and 101 of the Snake River

report (see Table 2-1 above).

• Refer to the literature on similar situations to give best professional judgment about what

might happen. Sometimes the predicted recovery does not occur. In the Clinch Valley some

improved areas which now provide a suitable habitat still do not have mussels. Another

example is areas where stocking of salmon and habitat improvement have not resulted in

restored populations. 

- Temporal scales for recovery vary for different species (e.g., earthworms versus bear),

thus making recovery difficult to estimate. Spatial considerations should be appropriate

for the assessment.

- Units of adversity, such as percent mortality, may be more useful than rankings if they are

understood by the public and directly relate to something the public cares about.
6-12

CHARGE QUESTION: How are alternative management options selected to have their

risks characterized and how is management informed of the consequences?

• Ideally, management options should be discussed during planning or problem formulation;

however, after some findings are available later in the process, additional discussions with

management may be needed because some options may no longer be realistic and new

alternatives must be considered. At the beginning of an assessment it may not be known

whether recovery is possible, so it may not be clear what alternatives should be considered. If

management options are not discussed throughout the process, the risk assessment may not

produce meaningful findings. 

• Later in a study, the assessor may be able to determine subsections of a study area that need

different management options to achieve management goals.

• A meeting with the stakeholders following the analysis phase was advised to plug in different

scenarios based on stakeholder input. In addition, periodic meetings with stakeholders should

be held to discuss management options and to receive other viewpoints that might help explain

data.

• When stakeholders are included throughout the process, they understand the process and have

buy-in from the beginning. When conclusions are reached, the stakeholders are more likely to 

understand them, and the process does not have to be explained from the beginning. The

stakeholders were a part of the entire assessment.

• A recommendation for more extensive involvement and consultation with risk managers and

stakeholders during the analysis phase should be added to the guidelines. Such involvement

could change the direction of a study. 

- Public advisory committees are a forum to exchange information and meet regularly. 

Public meetings with stakeholders should always be scheduled in advance, and scientists

and investigators should be available for questions and discussions.

CHARGE QUESTION: How should ecological risk be communicated to the manager and

the public (journal articles, computer programs, presentations, town meetings, Internet,

fact-sheets)?

• In risk communication, a few participants felt writers should apply the “teenager rule” (i.e.,

the report should be explainable to a teenager) or use writing at a seventh-grade level. For

example, say “no dissolved oxygen,” rather than “anoxia.” 

• Several members of the team felt the risk characterization should be written at the level of the

technical literacy of the risk manager. Any version of the risk characterization should be

written as appropriate for the target audience.
6-13

C Many felt the risk characterization should be a scientific report, but it should have a “relevant

findings” section for the manager.

C Alternatively a “plain language” watershed plan/risk characterization could be written with

technical appendices. This is in agreement with the approach provided in the Guidelines and

Handbook. 

C Terms not in the dictionary should be briefly defined in the text or in a glossary. If a term is a

central concept that will be used repeatedly, it may need even more definition (possibly

expanded to a paragraph or two). Words that are contained in the glossary should be in bold

type the first time they appear in the text.

C Technical terms should not be used in the executive summary or the relative findings section

of the risk characterization. 

C Encourage stakeholders to participate in presentations. It is advantageous if the

communicator is part of the audience group or is otherwise considered to be credible by the

stakeholders.

C Have open houses for school children.

C Meet the needs of the media.

C Reporting units used for the results should be compatible with the display of other results. 

Consistency in presentation is helpful for the readers.
R-1

REFERENCES

Foran, J., and S. Ferenc (Eds.) (1999) Multiple stressors in ecological risk and impact assessment. 

Proceedings from the Pellston Workshop on Multiple Stressors in Ecological Risk and Impact Assessment:

13-18 September 1997, Pellston, Michigan. Society for Environmental Toxicology and Chemistry,

Pensacola. 

Fox, G.A. (1991) Practical causal inference for ecoepidemiologists. J. Toxicol. Environ. Health 33:359-

373. 

Hill, A.B. (1965) The environment and disease: Association or causation? Proc. R. Soc. Med. 58:295-

300.

National Research Council. (1989) Improving risk communication. Washington, DC: National Academy

Press.

National Research Council. (1996) Understanding risk: informing decisions in a democratic society.

Washington, DC: National Academy Press.

Pickle, L. W., M. Mingle, G. K. Jones, and A. A. White. (1996) Atlas of United States mortality. 

Hyattsville, Maryland: National Center for Health Statistics. 

TN & Associates, Inc. (1999) Workshop Report on Developing a Problem Formulation Process 

Large Spatial Scales. Oak Ridge, TN.

The Presidential/Congressional Commission Risk Assessment and Risk Management. (1997) Risk

assessment and risk management in regulatory decision-making, Volume 2. Washington, DC.

Susser, M. (1986) Rules of inference in epidemiology. Regul. Toxicol. Pharmacol. 6:116-128.

Suter, G. W. (1993) A critique of ecosystem health concepts and indexes. Environ. Toxicol. Chem.

12:1533-1539.

U.S. EPA (U. S. Environmental Protection Agency). (1992) Framework for ecological risk assessment. 

Washington, DC: Risk Assessment Forum, U.S. Environmental Protection Agency. EPA/630/R-92/001.

U.S. EPA. (1998) Guidelines for ecological risk assessment. Washington, DC: Risk Assessment Forum,

U.S. Environmental Protection Agency. EPA/630/R-95/002F.

U. S. EPA. (1999) Risk characterization materials for peer review - March 1999. Washington, DC: 

Office of Research and Development, U.S. Environmental Protection Agency. EPA/600/R-99/025.
APPENDIX A

LIST OF WORKSHOP

PARTICIPANTS
Attendee Name Organization Address Phone e-mail Interest in Meeting

USEPA ORD NCEA

Michael Slimak USEPA ORD NCEA

401 M Street, SW, 8601D

 Washington, DC 20460 202-564-3324 slimak.michael@epa.gov

Associate Director for Ecology, National Center

for Environmental Assessment

Kay Austin USEPA ORD NCEA

401 M Street, SW, 8601D

 Washington, DC 20460 202-564-3328 austin.kay@epa.gov

Senior Science Advisor for Ecology, National

Center for Environmental Assessment

Facilitators:

Jack Gentile University of Miami

Center for Marine and Environmental Analysis

University of Miami

4600 Rickenbacker Causeway

Miami, FL 33149-4152 305-361-4152 jgentile@rsmas.miami.edu Watershed Ecological Risk Assessor

William Smith Yale University

Yale School of Environment & Forestry Studies

285 Prospect

New Haven, CT 06511 203-432-5149 william.smith@yale.edu Ecological Risk Assessor

Patrick Bourgeron Colorado State University

INSTAAR University of Colorado

Campus Box 450, 1560 30th Street

Boulder, CO 80309 303-492-2841 Patrick.Bourgeron@Colorado.EDU Regional Ecological Assessor

Panel Members:

Vic Serveiss USEPA ORD NCEA

Mail Code 8623D

401 M Street, SW

Washington, DC 20460 202-564-3251 serveiss.victor@epa.gov

Watershed Prototype Assessments & Workshop

Manager

Bill van der Schalie USEPA ORD NCEA

National Center for Environmental Assessment

Washington Office

Mail Code 8623D

401 M Street, SW,

Washington, DC 20460 202-564-3371 vanderschalie.william@epamail.epa.gov Guidelines Author 

Sue Norton USEPA ORD NCEA

Mail Code 8623D

401 M Street, SW

Washington, DC 20460 202-564-3246 NORTON.SUSAN@epa.gov Guidelines Author & Big Darby Creek Researcher

Jerry Diamond Tetra Tech

10045 Red Run Blvd., Suite 110

Owings Mills, MD 21117 410-356-8993 jerryd@ccpl.carr.org Watershed Ecological Risk Assessor

Roberta Hylton Southwest Virginia Field Office

988 West Main Street

Abingdon VA, 24210 540-623-1233 roberta_hylton@fws.gov

Fish & Wildlife Service, Southwest Virginia

Field Office

Don Gowan The Nature Conservancy

151 W. Main Street

Abingdon, VA 24210 540-676-2209 dgowan@tnc.org Clinch Valley Assessment Co-chair

Maggie Geist Waquoit Bay NERR

149 Waquoit Highway

Waquoit, MA 02536 508-457-0495 mgeist@capecod.net Waquoit Bay Assessment Co-chair

A-1
Attendee Name Organization Address Phone e-mail Interest in Meeting

Panel Members continued:

Patti Tyler USEPA Region 1

EPA New England Region 1

Office of Ecosystem Assessment

60 Westview Street

Lexington, MA 02421 781-860-4342 tyler.patti@epamail.epa.gov Waquoit Bay Assessment Co-chair

Bob Fenemore USEPA GPCBWWPD

Region 7

901 North Fifth Street

Kansas City, KS 66101 913-551-7745 fenemore.robert@epa.gov Middle Platte Assessment Chair

Marla Downing USEPA EMWCENSV

Region 7

901 North Fifth Street

Kansas City, KS 66101 913-551-7362 Downing.Marla@epamail.epa.gov Former Middle Platte Assessment Chair

Patricia Cirone US EPA MS PEA/095

Region 10

1200 Sixth Avenue

Seattle WA 98101 206-553-1597 CIRONE.PATRICIA@epamail.epa.gov

Middle Snake Assessment Chair & Guidelines

Author

Susan Cormier US EPA NERL

26 West Martin Luther King Drive

Cincinnati OH, 45268 513-569-7995 Cormier.Susan@epa.gov Big Darby Creek Assessment Co-chair

Additional Participants:

John Miller US EPA ORD OSP

401 M Street, SW

Washington, DC 20460 202-564-6489 miller.johne@epa.gov

Former Watershed Prototype Assessment Project

Manager

James Andreasen US EPA ORD NCEA

401 M Street, SW

Mail Code 8623D

Washington, DC 20460 202-564-3293 ANDREASEN.JAMES@epamail.epa.gov

Project Manager of Columbia River Eco Risk

Assessment

Catriona Rogers US EPA ORD NCEA

401 M Street

Mail Code 8601D

Washington, DC 20460 202-564-3391 Rogers.Catriona@epa.gov

Regional Assessment of Ecological Impacts of

Multiple Stressors

Anne Sergeant US EPA ORD NCEA

401 M Street, SW

Mail Code 8623D

Washington, DC 20460 202-564-3249 sergeant.anne@epamail.epa.gov

Developer of Eco Risk Assessment Training

Course and Guidelines Author

Bill Ewald US EPA ORD NCEA

MD52, USEPA Mailroom

Research Triangle Park, NC 27711 919-541-4164 ewald.william@epa.gov Neuse River Risk Assessment

Susan Ferenc ILSI Risk Science Institute

1126 16th Street NW

Washington, DC 20036 202-659-3306 Sferenc@ilsi.org Ecological Risk Assessor

Ed Bender US EPA ORD OSP

Office of Science Policy (ORD-8103R)

401 M Street SW

Washington, DC 20460 202-564-6483 Bender.Ed@epamail.epa.gov Cumulative (Eco and Health Risk) Coordinator

Lawrence Martin USEPA ORD OSP

USEPA Headquarters, 8103R

401 M Street, SW

Washington, DC 20460 202-564-6497 Martin.Lawrence@epamail.epa.gov ORD State and Local Relations Coordinator

Laura Gabanski US EPA OW OWOW

4503F, 401 M Street, SW

Washington, DC 20460 202-260-5868 Gabanski.Laura@epamail.epa.gov

Former Participant Waquoit Bay Assessment,

Developer of Eco Risk & Decision Making

Course

Ken Jones Green Mountain Institute

104 East State Street

 Montpelier, VT 05602 802-229-6070 kjones@gmied.org Green Mountain Institute

A-2
Attendee Name Organization Address Phone e-mail Interest in Meeting

Additional Participants continued:

Barry Tonning

The Council of State

Governments

2760 Research Park Drive

PO Box 11910

Lexington, KY, 40578-1910 606-244-8228 btonning@csg.org

Watershed Monitoring, Assessment,

Management, and Watershed Management

Training

Robert Murphy Alliance for the Chesapeake Bay

6600 York Rd, #100

Baltimore, MD 21212 410-377-6270 bmurphy@acb-online.org Watershed Restoration

Margaret Stewart

Watershed Environment Research

Federation

601 Wythe Street

Alexandria, VA 22314-1994 703-684-2470 mstewart@wef.org Watershed Management

Jennifer Moses Tennessee Valley Authority

Reservation Rd, CEB 3A

Muscle Shoals, AL 35662-1010 256-386-2518 jmoses@tva.gov TVA Technical Writer

Bruce Yeager Tennessee Valley Authority

ABL 1A-N

17 Ridgeway Road (Box 920)

Norris, TN 37828 423-632-1721 blyeager@tva.gov TVA Technical Writer

Angie Stiles Tennessee Valley Authority

400 W. Summit Hill Drive, WT 8C

Knoxville, TN 37902 423-632-8057 adstiles@tva.gov TVA Logistics Coordinator

Ray Alexander Tennessee Valley Authority

1101 Market Street, HB 2A

Chattanooga, TN 37402 423-751-6415 rkalexander@tva.gov Manager, TVA Remote Sensing

Dennis Yankee Tennessee Valley Authority

129 Pine Road

Norris, TN 37828 423-632-1541 dhyankee@tva.gov TVA GIS & Landscape Ecologist

Michelle Witten Green Mountain Institute

104 East State Street

Montpelier, VT 05602 802-229-6081 mwitten@gmied.org Green Mountain Institute

Janice Cox Tennessee Valley Authority

1101 Market Street, CST 17D

Chattanooga, TN 37402 423-751-7337 jpcox@tva.gov TVA Risk Assessment Specialist

Donald Rodier USEPA

Science Support Branch, Risk Assessment Division (7403)

Office of Pollution Prevention and Toxics

401 M Street, SW

Washington, DC 20460 202-260-1276 Rodier.Donald@epa.gov Ecological Risk Assessor and Guidelines Author

A-3
APPENDIX B

WORKSHOP AGENDA
B-1

Watershed Ecological Risk Characterization Workshop 

July 7-8, 1999

I. Purpose of Workshop: To further develop and document the process for conducting

watershed ecological risk characterization. 

II. Desired Output: Develop recommendations for how to conduct a watershed ecological

risk characterization. Discuss the desirable items to include in such a characterization and

how to address common situations.

III. Workshop Approach: We are developing additional guidance for how one should develop

the risk characterization phase of a watershed scale ecological risk assessment. We are

advancing what is documented in the ecological risk assessment guidelines. While we will

attempt to comply with the guidelines, we will also deviate from them if justifiable. Where

applicable we may also develop supplemental and deviating guidelines for the analysis phase

of risk assessment.

The Agency is also developing a risk characterization handbook. These will be revised

based on peer review comments. The peer review comments noted that the document needs

to more fully incorporate ecological risk assessment principles. The latest draft does

conclude that the risk characterization should summarize the earlier phases of the risk

assessment including planning and problem formulation. The current draft also requires the

risk characterization to be written in "plain language" and to be concise.

We need an approach that attempts to comply with the Agency ecological risk assessment

guidelines and the Agency risk characterization guidelines. To achieve scientific credibility

while also using plain language and concise explanations we will develop a scientific risk

characterization report that will also include a relevant findings section in plain language. In

addition, the executive summary for the entire assessment will also be written to be

understandable to an educated non-scientist.

IV. Charge questions to consider when reviewing the provided risk characterizations and other

background materials.

A. How can the three submitted watershed assessment drafts (or plans) be improved,

especially in regards to the other charge questions listed below? 

B. How should exposure and effects data be integrated in a watershed context to generate

a risk estimate? 

C. How should uncertainty be addressed and presented (e.g., incomplete data or analyses,

qualitative estimates, data at different spatial and organizational levels)?
B-2

D. What are the best ways to address and present the integration of qualitative and

quantitative lines of evidence for (a) an individual assessment endpoint and (b) drawing

overall conclusions?

E. How should the degree of adversity of predicted or observed effects in watersheds be

described (e.g., considering nature and intensity of effects, spatial and temporal scale,

and the potential for recovery)?

F. How are alternative management options selected to have their risks characterized and

how is management informed of the consequences?

G. How should ecological risk be communicated to the manager and the public (journal

articles, computer programs, presentations, town meetings, Internet, fact-sheets)?

 

V. Background Documents: 

A. Ecological Risk Assessment Guidelines, Section on Risk Characterization

B. Draft Risk Characterization Handbook, Sections 1, 2, & 3 

C. Waquoit Bay Risk Characterization Plan

D. Review Comments on Waquoit Bay Risk Characterization Document 

E. Draft Middle Snake Watershed Risk Characterization 

F. Draft Clinch Valley Watershed Risk Characterization
B-3

Agenda

Watershed Risk Characterization Workshop

July 7 & 8 1999

July 7:

8:00 Registration and conversation

8:45 Opening remarks, value of this workshop

Mike Slimak, Associate Director for Ecology

National Center for Environmental Assessment (NCEA)

9:00 Workshop goals and approach, progress on five case studies, review agenda

Vic Serveiss, Environmental Scientist, NCEA

9:15 Introductions

9:25 Risk characterization summary 

Bill van der Schalie, Environmental Scientist, NCEA

9:50 Different traditions of environmental management and their implications for

watershed ecological risk assessment 

Sue Norton, Ecologist, NCEA

10:15 Break 

 

10:40 Presentation and discussion of five watershed assessments

Present the risk characterization draft or plan. 

Discuss charge question #1: How can each watershed risk characterization be

improved?

10:45 Big Darby Creek, Susan Cormier

11:15 Middle Platte, Dennis Jelinski

11:45 Lunch

12:45 Waquoit Bay, Maggie Geist; Jack Gentile, facilitator

2:00 Middle Snake, Patricia Cirone; Patrick Bourgeron, facilitator

3:15 Break 

3:30 Clinch Valley, Jerry Diamond; William Smith, facilitator

4:45 Plan for tomorrow, Vic Serveiss

5:00-6:00 Optional conversation time (Crystal Club Room)
B-4

July 8:

8:00 Optional conversation time

Facilitators present responses to charge questions # 2-7 

8:30 Facilitator #1 Jack Gentile

9:00 Facilitator #2 William Smith 

9:30 Facilitator #3 Patrick Bourgeron

10:00 Break

10:15 Break-out groups led by facilitators developed recommendations for conducting

watershed eco-risk characterization by developing replies to charge questions. 

Break-out Session Team 1 - Crystal Club Room

Break-out Session Team 2 - Board Room

Break-out Session Team 3 - Salon F (Main Room)

12:00-1:00 Lunch

2:30 Report back by a volunteer from each group 

 

3:30 Break

4:00 Group synthesis/consensus, led by invited experts

4:45 Closing comments-Serveiss/Norton/van der Schalie 

5:00 Adjourn
B-5

Break out teams:

Crystal Club Room

Team 1

James Andreasen

Kay Austin

Patricia Cirone

Janice Cox

Marla Downing

Bill Ewald

Laura Gabanski

Jack Gentile

Don Gowan

Ken Jones

John Miller

Margaret Stewart

Bill van der Schalie 

Board Room

Team 2 

Ed Bender

Bob Fenemore 

Jack Fowle

Maggie Geist

Roberta Hylton

Lawrence Martin

Robert Murphy

Sue Norton

Anne Sergeant

William Smith 

Molly Whitworth

Dennis Yankee

Bruce Yeager

Salon F

Team 3

Richard Batiuk

Joyce Brooks

Patrick Bourgeron 

Susan Cormier

Jerry Diamond

Susan Ferenc

Jennifer Moses

Donald Rodier

Catriona Rogers

Vic Serveiss

Michael Slimak

Barry Tonning

Patti Tyler

Michelle Witten
APPENDIX C

ECOLOGICAL RISK ASSESSMENT GUIDELINES

SECTION ON RISK CHARACTERIZATION
C-1

5. RISK CHARACTERIZATION

Risk characterization (figure 5-1) is the final phase of ecological risk assessment and is the

culmination of the planning, problem formulation, and analysis of predicted or observed adverse

ecological effects related to the assessment endpoints. Completing risk characterization allows

risk assessors to clarify the relationships between stressors, effects, and ecological entities and to

reach conclusions regarding the occurrence of exposure and the adversity of existing or

anticipated effects. Here, risk assessors first use the results of the analysis phase to develop an

estimate of the risk posed to the ecological entities included in the assessment endpoints identified

in problem formulation (section 5.1). After estimating the risk, the assessor describes the risk

estimate in the context of the significance of any adverse effects and lines of evidence supporting

their likelihood (section 5.2). Finally, the assessor identifies and summarizes the uncertainties,

assumptions, and qualifiers in the risk assessment and reports the conclusions to risk managers

(section 5.3).

Conclusions presented in the risk characterization should provide clear information to risk

managers in order to be useful for environmental decision making (NRC, 1994; see section 6). If

the risks are not sufficiently defined to support a management decision, risk managers may elect

to proceed with another iteration of one or more phases of the risk assessment process. 

Reevaluating the conceptual model (and associated risk hypotheses) or conducting additional

studies may improve the risk estimate. Alternatively, a monitoring program may help managers

evaluate the consequences of a risk management decision.

5.1. RISK ESTIMATION

Risk estimation is the process of integrating exposure and effects data and evaluating any

associated uncertainties. The process uses exposure and stressor-response profiles developed

according to the analysis plan (section 3.5). Risk estimates can be developed using one or more

of the following techniques: (1) field observational studies, (2) categorical rankings, (3)

comparisons of single-point exposure and effects estimates, (4) comparisons incorporating the

entire stressor-response relationship, (5) incorporation of variability in exposure and/or effects

estimates, and (6) process models that rely partially or entirely on theoretical approximations of

exposure and effects. These techniques are described in the following sections.
C-2

Figure 5-1. Risk characterization.
C-3

Text Box 5-1. An Example of Field Methods

Used for Risk Estimation

Along with quotients comparing field measures

of exposure with laboratory acute toxicity data

(see Text Box 5-3), EPA evaluated the risks of

granular carbofuran to birds based on incidents

of bird kills following carbofuran applications. 

More than 40 incidents involving nearly 30

species of birds were documented. Although

reviewers identified problems with individual

field studies (e.g., lack of appropriate control

sites, lack of data on carcass-search efficiencies,

no examination of potential synergistic effects of

other pesticides, and lack of consideration of

other potential receptors such as small

mammals), there was so much evidence of

mortality associated with carbofuran application

that the study deficiencies did not alter the

conclusions of high risk found by the assessment

(Houseknecht, 1993).

5.1.1. Results of Field Observational Studies

Field observational studies (surveys) can serve as risk estimation techniques because they

provide empirical evidence linking exposure to effects. Field surveys measure biological changes

in natural settings through collection of exposure and effects data for ecological entities identified

in problem formulation.

A major advantage of field surveys is that they can be used to evaluate multiple stressors

and complex ecosystem relationships that cannot be replicated in the laboratory. Field surveys are

designed to delineate both exposures and effects (including secondary effects) found

in natural systems, whereas estimates generated

from laboratory studies generally delineate either

exposures or effects under controlled or

prescribed conditions (see text box 5-1).

While field studies may best represent

reality, as with other kinds of studies they can be

limited by (1) a lack of replication, (2) bias in

obtaining representative samples, or (3) failure to

measure critical components of the system or

random variations. Further, a lack of observed

effects in a field survey may occur because the

measurements lack the sensitivity to detect

ecological effects. See section 4.1.1 for additional

discussion of the strengths and limitations of

different types of data.

Several assumptions or qualifications need

to be clearly articulated when describing the results of field surveys. A primary qualification is

whether a causal relationship between stressors and effects (section 4.3.1.2) is supported. Unless

causal relationships are carefully examined, conclusions about effects that are observed may be

inaccurate because the effects are caused by factors unrelated to the stressor(s) of concern. In

addition, field surveys taken at one point in time are usually not predictive; they describe effects

associated only with exposure scenarios associated with past and existing conditions.
C-4

Text Box 5-2. Using Qualitative Categories to

Estimate Risks of an Introduced Species

The importation of logs from Chile required an

assessment of the risks posed by the potential

introduction of the bark beetle, Hylurgus ligniperda

(USDA, 1993). Experts judged the potential for

colonization and spread of the species, and their

opinions were expressed as high, medium, or low as

to the likelihood of establishment (exposure) or

consequential effects of the beetle. Uncertainties were

similarly expressed. A ranking scheme was then used

to sum the individual elements into an overall

estimate of risk (high, medium, or low). Narrative

explanations of risk accompanied the overall

rankings.

5.1.2. Categories and Rankings

In some cases, professional judgment

or other qualitative evaluation techniques

may be used to rank risks using categories,

such as low, medium, and high, or yes and

no. This approach is most frequently used

when exposure and effects data are limited or

are not easily expressed in quantitative terms. 

The U.S. Forest Service risk assessment of

pest introduction from importation of logs

from Chile used qualitative categories owing

to limitations in both the exposure and effects

data for the introduced species of concern as

well as the resources available for the assessment (see text box 5-2).

Ranking techniques can be used to translate qualitative judgment into a mathematical

comparison. These methods are frequently used in comparative risk exercises. For example,

Harris et al. (1994) evaluated risk reduction opportunities in Green Bay (Lake Michigan),

Wisconsin, employing an expert panel to compare the relative risk of several stressors against

their potential effects. Mathematical analysis based on fuzzy set theory was used to rank the risk

from each stressor from a number of perspectives, including degree of immediate risk, duration of

impacts, and prevention and remediation management. The results served to rank potential

environmental risks from stressors based on best professional judgment.

 5.1.3. Single-Point Exposure and Effects Comparisons

When sufficient data are available to quantify exposure and effects estimates, the simplest

approach for comparing the estimates is a ratio (figure 5-2a). Typically, the ratio (or quotient) is

expressed as an exposure concentration divided by an effects concentration. Quotients are

commonly used for chemical stressors, where reference or benchmark toxicity values are widely

available (see text box 5-3).

The principal advantages of the quotient method are that it is simple and quick to use and

risk assessors and managers are familiar with its application. It provides an efficient, inexpensive

means of identifying high- or low-risk situations that can allow risk management decisions to be

made without the need for further information.

Quotients have also been used to integrate the risks of multiple chemical stressors: 

quotients for the individual constituents in a mixture are generated by dividing each exposure 
C-5

a: Comparison of point estimates

Stressor-response

estimate

(e.g., LC10)

Intensity of Stressor (e.g., concentration)

Exposure

estimate

(e.g., mean

concentration)

e.g., uncertainty around

mean concentration

b: Comparison of a point estimate of a stressor-response

relationship with uncertainty associated with an exposure

point estimate

Figure 5-2. Risk estimation techniques. a. Comparison of exposure and stressor-response

point estimates. b. Comparison of a point estimate from the stressor-response relationship

with uncertainty associated with an exposure point estimate.

level by a corresponding toxicity endpoint (e.g., LC50, EC50, NOAEL). Although the toxicity of a

chemical mixture may be greater than or less than predicted from the toxicities of individual 

constituents of the mixture, a quotient addition approach assumes that toxicities are additive or

approximately additive. This assumption may be most applicable when the modes of action of 

chemicals in a mixture are similar, but there is evidence that even with chemicals having dissimilar

modes of action, additive or near-additive interactions are common (Könemann, 1981; Broderius,

1991; Broderius et al., 1995; Hermens et al., 1984a, b; McCarty and Mackay, 1993; Sawyer and
C-6

Text Box 5-3. Applying the Quotient Method

When applying the quotient method to chemical

stressors, the effects concentration or dose (e.g., an

LC50, LD50, EC50, ED50, NOAEL, or LOAEL) is

frequently adjusted by uncertainty factors before

division into the exposure number (U.S. EPA, 1984;

Nabholz, 1991; Urban and Cook, 1986; see section

4.3.1.3), although EPA used a slightly different

approach in estimating the risks to the survival of

birds that forage in agricultural areas where the

pesticide granular carbofuran is applied

(Houseknecht, 1993). In this case, EPA calculated

the quotient by dividing the estimated exposure levels

of carbofuran granules in surface soils (number/ft2) by

the granules/LD50 derived from single-dose avian

toxicity tests. The calculation yields values with units

of LD50/ft2. It was assumed that a higher quotient

value corresponded to an increased likelihood that a

bird would be exposed to lethal levels of granular

carbofuran at the soil surface. Minimum and

maximum values for LD50/ft2 were estimated for

songbirds, upland game birds, and waterfowl that

may forage within or near 10 different agricultural

crops.

Safe, 1985). However, caution should be used when assuming that chemicals in a mixture act

independently of one another, since many of the supporting studies were conducted with aquatic

organisms, and so may not be relevant for other endpoints, exposure scenarios, or

species. When the modes of action for

constituent chemicals are unknown, the

assumptions and rationale concerning

chemical interactions should be clearly stated.

A number of limitations restrict

application of the quotient method (see Smith

and Cairns, 1993; Suter, 1993a). While a

quotient can be useful in answering whether

risks are high or low, it may not be helpful to

a risk manager who needs to make a decision

requiring an incremental quantification of

risks. For example, it is seldom useful to say

that a risk mitigation approach will reduce a

quotient value from 25 to 12, since this

reduction cannot by itself be clearly

interpreted in terms of effects on an

assessment endpoint.

Other limitations of quotients may be

caused by deficiencies in the problem formulation and analysis phases. For example, an LC50

derived from a 96-hour laboratory test using constant exposure levels may not be appropriate for

an assessment of effects on reproduction resulting from short-term, pulsed exposures.

In addition, the quotient method may not be the most appropriate method for predicting

secondary effects (although such effects may be inferred). Interactions and effects beyond what

are predicted from the simple quotient may be critical to characterizing the full extent of impacts

from exposure to the stressors (e.g., bioaccumulation, eutrophication, loss of prey species,

opportunities for invasive species).

Finally, in most cases, the quotient method does not explicitly consider uncertainty (e.g.,

extrapolation from tested species to the species or community of concern). Some uncertainties,

however, can be incorporated into single-point estimates to provide a statement of likelihood that

the effects point estimate exceeds the exposure point estimate (figures 5-2b and 5-3). If exposure

variability is quantified, then the point estimate of effects can be compared with a cumulative
C-7

Figure 5-3. Risk estimation techniques: comparison of point estimates with associated

uncertainties.

exposure distribution as described in text box 5-4. Further discussion of comparisons between

point estimates of effects and distributions of exposure may be found in Suter et al., 1983.

In view of the advantages and limitations of the quotient method, it is important for risk

assessors to consider the points listed below when evaluating quotient method estimates.

• How does the effect concentration relate to the assessment endpoint?

• What extrapolations are involved?

• How does the point estimate of exposure relate to potential spatial and

temporal variability in exposure?

• Are data sufficient to provide confidence intervals on the endpoints?
C-8

Text Box 5-4. Comparing an Exposure

Distribution With a Point Estimate of Effects

The EPA Office of Pollution Prevention and Toxics

uses a Probabilistic Dilution Model (PDM3) to

generate a distribution of daily average chemical

concentrations based on estimated variations in

stream flow in a model system. The PDM3 model

compares this exposure distribution with an aquatic

toxicity test endpoint to estimate how many days in a

1-year period the endpoint concentration is exceeded

(Nabholz et al., 1993; U.S. EPA, 1988b). The

frequency of exceedance is based on the duration of

the toxicity test used to derive the effects endpoint. 

Thus, if the endpoint was an acute toxicity level of

concern, an exceedance would be identified if the

level of concern was exceeded for 4 days or more (not

necessarily consecutive). The exposure estimates are

conservative in that they assume instantaneous

mixing of the chemical in the water column and no

losses due to physical, chemical, or biodegradation

effects.

5.1.4. Comparisons Incorporating the

Entire Stressor-Response Relationship

If a curve relating the stressor level to

the magnitude of response is available, then

risk estimation can examine risks associated

with many different levels of exposure (figure

5-4). These estimates are particularly useful

when the risk assessment outcome is not

based on exceedance of a predetermined

decision rule, such as a toxicity benchmark

level.

There are advantages and limitations

to comparing a stressor-response curve with

an exposure distribution. The slope of the

effects curve shows the magnitude of change

in effects associated with incremental changes

in exposure, and the capability to predict changes in the magnitude and likelihood of effects for

different exposure scenarios can be used to compare different risk management options. Also,

uncertainty can be incorporated by calculating uncertainty bounds on the stressor-response or

exposure estimates. Comparing exposure and stressor-response curves provides a predictive

ability lacking in the quotient method. Like the quotient method, however, limitations from the

problem formulation and analysis phases may limit the utility of the results. These limitations may

include not fully considering secondary effects, assuming the exposure pattern used to derive the

stressor-response curve is comparable to the environmental exposure pattern, and failure to

consider uncertainties, such as extrapolations from tested species to the species or community of

concern.

5.1.5. Comparisons Incorporating Variability in Exposure and/or Effects

If the exposure or stressor-response profiles describe the variability in exposure or effects,

then many different risk estimates can be calculated. Variability in exposure can be used to

estimate risks to moderately or highly exposed members of a population being investigated, while

variability in effects can be used to estimate risks to average or sensitive population
C-9

Cumulative Frequency

0.10

0.50

0.90

cumulative

distribution of

exposures

Intensity of Stressor (e.g., concentration)

comparison of

90th percentile exposure

with EC10

comparison of

50th percentile exposure

with EC50

stressorresponse

curve

0.90

0.50

0.10

Fraction of Test Population Responding

Figure 5-4. Risk estimation techniques: stressor-response curve versus a cumulative distribution of

exposures.

members. A major advantage of this approach is its ability to predict changes in the magnitude

and likelihood of effects for different exposure scenarios and thus provide a means for comparing

different risk management options. As noted above, comparing distributions also allows one to

identify and quantify risks to different segments of the population. Limitations include the

increased data requirements compared with previously described techniques and the implicit

assumption that the full range of variability in the exposure and effects data is adequately
C-10

Text Box 5-5. Comparing Cumulative Exposure

and Effects Distributions for Chemical Stressors

Exposure distributions for chemical stressors can

be compared with effects distributions derived

from point estimates of acute or chronic toxicity

values for different species (e.g., HCN, 1993;

Cardwell et al., 1993; Baker et al., 1994;

Solomon et al., 1996). Figure 5-5 shows a

distribution of exposure concentrations of an

herbicide compared with single-species toxicity

data for algae (and one vascular plant species) for

the same chemical. The degree of overlap of the

curves indicates the likelihood that a certain

percentage of species may be adversely affected. 

For example, figure 5-5 indicates that the 10th

centile of algal species’ EC5 values is exceeded

less than 10% of the time.

The predictive value of this approach is evident. 

The degree of risk reduction that could be

achieved by changes in exposure associated with

proposed risk mitigation options can be readily

determined by comparing modified exposure

distributions with the effects distribution curve.

When using effects distributions derived from

single-species toxicity data, risk assessors should

consider the following questions:

• Does the subset of species for which toxicity

test data are available represent the range of

species present in the environment?

• Are particularly sensitive (or insensitive) groups

of organisms represented in the distribution?

• If a criterion level is selected—e.g., protect 95%

of species—does the 5% of potentially affected

species include organisms of ecological,

commercial, or recreational significance?

represented. As with the quotient method,

secondary effects are not readily evaluated

with this technique. Thus, it is desirable to

corroborate risks estimated by distributional

comparisons with

field studies or other lines of evidence. Text

box 5-5 and figure 5-5 illustrate the use of

cumulative exposure and effects distributions

for estimating risk.

5.1.6. Application of Process Models

Process models are mathematical

expressions that represent our understanding

of the mechanistic operation of a system

under evaluation. They can be useful tools in

both analysis (see section 4.1.2) and risk

characterization. For illustrative purposes, it

is useful to distinguish between analysis

process models, which focus individually on

either exposure or effects evaluations, and

risk estimation process models, which

integrate exposure and effects information

(see text box 5-6). The assessment of risks

associated with long-term changes in

hydrologic conditions in bottomland forest

wetlands in Louisiana using the FORFLO

model (Appendix D) linked the attributes and

placement of levees and corresponding water

level measurements (exposure) with changes

in forest community structure and wildlife

habitat suitability (effects).

A major advantage of using process

models for risk estimation is the ability to

consider “what if” scenarios and to forecast beyond the limits of observed data that constrain

techniques based solely on empirical data. The process model can also consider secondary
C-11

Figure 5-5. Risk estimation techniques: comparison of exposure distribution of an herbicide in

surface waters with freshwater single-species toxicity data. See text box 5-4 for further

discussion. Redrawn from Baker et al., 1994. (Centile ranks for species LC5 data were

obtained using the formula (100 x n/[N+1]), where n is the rank number of the LC5 and N is the

total number of data points in the set; adapted from Parkhurst et al., 1995).

effects, unlike other risk estimation techniques such as the quotient method or comparisons of

exposure and effect distributions. In addition, some process models can forecast the combined

effects of multiple stressors, such as the effects of multiple chemicals on fish population

sustainability (Barnthouse et al., 1990).

Process model outputs may be point estimates, distributions, or correlations; in all cases,

risk assessors should interpret them with care. They may imply a higher level of certainty than is
C-12

Text Box 5-6. Estimating Risk With Process

Models

Models that integrate both exposure and effects

information can be used to estimate risk. During risk

estimation, it is important that both the strengths and

limitations of a process model approach be

highlighted. Brody et al. (1993; see Appendix D)

linked two process models to integrate exposure and

effects information and forecast spatial and temporal

changes in forest communities and their wildlife

habitat value. While the models were useful for

projecting long-term effects based on an

understanding of the underlying mechanisms of

change in forest communities and wildlife habitat,

they could not evaluate all possible stressors of

concern and were limited in the plant and wildlife

species they could consider. Understanding both the

strengths and limitations of models is essential for

accurately representing the overall confidence in the

assessment.

appropriate and are all too often viewed

without sufficient attention to underlying

assumptions. The lack of knowledge on basic

life histories for many species and incomplete

knowledge on the

structure and function of a particular

ecosystem is often lost in the model output. 

Since process models are only as good as the

assumptions on which they are based, they

should be treated as hypothetical

representations of reality until appropriately

tested with empirical data. Comparing model

results to field data provides a check on

whether our understanding of the system was

correct (Johnson, 1995), particularly with

respect to the risk hypotheses presented in

problem formulation.

5.2. RISK DESCRIPTION

Following preparation of the risk estimate, risk assessors need to interpret and discuss the

available information about risks to the assessment endpoints. Risk description includes an

evaluation of the lines of evidence supporting or refuting the risk estimate(s) and an interpretation

of the significance of the adverse effects on the assessment endpoints. During the analysis phase,

the risk assessor may have established the relationship between the assessment endpoints and

measures of effect and associated lines of evidence in quantifiable, easily described terms (section

4.3.1.3). If not, the risk assessor can relate the available lines of evidence to the assessment

endpoints using qualitative links. Regardless of the risk estimation technique, the technical

narrative supporting the risk estimate is as important as the risk estimate itself.

5.2.1. Lines of Evidence

The development of lines of evidence provides both a process and a framework for

reaching a conclusion regarding confidence in the risk estimate. It is not the kind of proof

demanded by experimentalists (Fox, 1991), nor is it a rigorous examination of weights of

evidence. (Note that the term “weight of evidence” is sometimes used in legal discussions or in

other documents, e.g., Urban and Cook, 1986; Menzie et al., 1996.) The phrase lines of evidence

is used to de-emphasize the balancing of opposing factors based on assignment of quantitative
C-13

values to reach a conclusion about a “weight” in favor of a more inclusive approach, which

evaluates all available information, even evidence that may be qualitative in nature. It is important

that risk assessors provide a thorough representation of all lines of evidence developed in the risk

assessment rather than simply reduce their interpretation and description of the ecological effects

that may result from exposure to stressors to a system of numeric calculations and results.

Confidence in the conclusions of a risk assessment may be increased by using several lines

of evidence to interpret and compare risk estimates. These lines of evidence may be derived from

different sources or by different techniques relevant to adverse effects on the assessment

endpoints, such as quotient estimates, modeling results, or field observational studies.

There are three principal categories of factors for risk assessors to consider when

evaluating lines of evidence: (1) adequacy and quality of data, (2) degree and type of uncertainty

associated with the evidence, and (3) relationship of the evidence to the risk assessment questions

(see also sections 3 and 4).

Data quality directly influences how confident risk assessors can be in the results of a

study and conclusions they may draw from it. Specific concerns to consider for individual lines of

evidence include whether the experimental design was appropriate for the questions posed in a

particular study and whether data quality objectives were clear and adhered to. An evaluation of

the scientific understanding of natural variability in the attributes of the ecological entities under

consideration is important in determining whether there were sufficient data to satisfy the analyses

chosen and to determine if the analyses were sufficiently sensitive and robust to identify stressorcaused perturbations.

Directly related to data quality issues is the evaluation of the relative uncertainties of each

line of evidence. One major source of uncertainty comes from extrapolations. The greater the

number of extrapolations, the more uncertainty introduced into a study. For example, were

extrapolations used to infer effects in one species from another, or from one temporal or spatial

scale to another? Were conclusions drawn from extrapolations from laboratory to field effects, or

were field effects inferred from limited information, such as chemical structure-activity

relationships? Were no-effect or low-effect levels used to address likelihood of effects? Risk

assessors should consider these and any other sources of uncertainty when evaluating the relative

importance of particular lines of evidence.

Finally, how directly lines of evidence relate to the questions asked in the risk assessment

may determine their relative importance in terms of the ecological entity and the attributes of the

assessment endpoint. Lines of evidence directly related to the risk hypotheses, and those that

establish a cause-and-effect relationship based on a definitive mechanism rather than associations

alone, are likely to be of greatest importance.
C-14

The evaluation process, however, involves more than just listing the evidence that

supports or refutes the risk estimate. The risk assessor should carefully examine each factor and

evaluate its contribution in the context of the risk assessment. The importance of lines of

evidence is that each and every factor is described and interpreted. Data or study results are often

not reported or carried forward in the risk assessment because they are of insufficient quality. If

such data or results are eliminated from the evaluation process, however, valuable information

may be lost with respect to needed improvements in methodologies or recommendations for

further studies.

As a case in point, consider the two lines of evidence described for the carbofuran example

(see text boxes 5-1 and 5-3), field studies and quotients. Both approaches are relevant to the

assessment endpoint (survival of birds that forage in agricultural areas where carbofuran is

applied), and both are relevant to the exposure scenarios described in the conceptual model (see

figure D-1). The quotients, however, are limited in their ability to express incremental risks (e.g.,

how much greater risk is expressed by a quotient of “2” versus a quotient of “4”), while the field

studies had some design flaws (see text box 5-1). Nevertheless, because of the strong evidence of

causal relationships from the field studies and consistency with the laboratory-derived quotient,

confidence in a conclusion of high risk to the assessment endpoint is supported.

Sometimes lines of evidence do not point toward the same conclusion. It is important to

investigate possible reasons for any disagreement rather than ignore inconvenient evidence. A

starting point is to distinguish between true inconsistencies and those related to differences in

statistical powers of detection. For example, a model may predict adverse effects that were not

observed in a field survey. The risk assessor should ask whether the experimental design of the

field study had sufficient power to detect the predicted difference or whether the endpoints

measured were comparable with those used in the model. Conversely, the model may have been

unrealistic in its predictions. While iteration of the risk assessment process and collection of

additional data may help resolve uncertainties, this option is not always available.

Lines of evidence that are to be evaluated during risk characterization should be defined

early in the risk assessment (during problem formulation) through the development of the

conceptual model and selection of assessment endpoints. Further, the analysis plan should

incorporate measures that will contribute to the interpretation of the lines of evidence, including

methods of reviewing, analyzing, and summarizing the uncertainty in the risk assessment.

Also, risk assessments often rely solely on laboratory or in situ bioassays to assess adverse

effects that may occur as a result of exposure to stressors. Although they may not be manifested

in the field, ecological effects demonstrated in the laboratory should not be discounted as a line of

evidence.
C-15

5.2.2. Determining Ecological Adversity

At this point in risk characterization, the changes expected in the assessment endpoints

have been estimated and the supporting lines of evidence evaluated. The next step is to interpret

whether these changes are considered adverse. Adverse ecological effects, in this context,

represent changes that are undesirable because they alter valued structural or functional attributes

of the ecological entities under consideration. The risk assessor evaluates the degree of adversity,

which is often a difficult task and is frequently based on the risk assessor’s professional judgment.

When the results of the risk assessment are discussed with the risk manager (section 6),

other factors, such as the economic, legal, or social consequences of ecological damage, should be

considered. The risk manager will use all of this information to determine whether a particular

adverse effect is acceptable and may also find it useful when communicating the risk to interested

parties.

The following are criteria for evaluating adverse changes in assessment endpoints:

• Nature of effects and intensity of effects

• Spatial and temporal scale

• Potential for recovery.

The extent to which the criteria are evaluated depends on the scope and complexity of the

risk assessment. Understanding the underlying assumptions and science policy judgments,

however, is important even in simple cases. For example, when exceedance of a previously

established decision rule, such as a benchmark stressor level, is used as evidence of adversity (e.g.,

see Urban and Cook, 1986, or Nabholz, 1991), the reasons why this is considered adverse should

be clearly understood. In addition, any evaluation of adversity should examine all relevant

criteria, since none are considered singularly determinative.

To distinguish adverse ecological changes from those within the normal pattern of

ecosystem variability or those resulting in little or no significant alteration of biota, it is important

to consider the nature and intensity of effects. For example, for an assessment endpoint involving

survival, growth, and reproduction of a species, do predicted effects involve survival and

reproduction or only growth? If survival of offspring will be affected, by what percentage will it

diminish? 

It is important for risk assessors to consider both the ecological and statistical contexts of

an effect when evaluating intensity. For example, a statistically significant 1% decrease in fish

growth (see text box 5-7) may not be relevant to an assessment endpoint of fish population

viability, and a 10% decline in reproduction may be worse for a population of slowly reproducing

trees than for rapidly reproducing planktonic algae.
C-16

Text Box 5-7. What Are Statistically Significant

Effects?

Statistical testing is the “statistical procedure or

decision rule that leads to establishing the truth or

falsity of a hypothesis . . .” (Alder and Roessler,

1972). Statistical significance is based on the number

of data points, the nature of their distribution,

whether intertreatment variance exceeds

intratreatment variance in the data, and the a priori

significance level ("). The types of statistical tests

and the appropriate protocols (e.g., power of test) for

these tests should be established as part of the analysis

plan during problem formulation.

Natural ecosystem variation can make it very difficult to observe (detect) stressor-related

perturbations. For example, natural fluctuations in marine fish populations are often large, with

intra- and interannual variability in population levels covering several orders of magnitude. 

Furthermore, cyclic events of various periods (e.g., bird migration, tides) are very important in

natural systems and may mask or delay stressor-related effects. Predicting the effects of

anthropogenic stressors against this background of variation can be very difficult. Thus, a lack of

statistically significant effects in a field study does not automatically mean that adverse ecological

effects are absent. Rather, risk assessors should then consider other lines of evidence in reaching

their conclusions.

It is also important to consider the location of the effect within the biological hierarchy

and the mechanisms that may result in ecological changes. The risk assessor may rely on

mechanistic explanations to describe complex ecological interactions and the resulting effects that

otherwise may be masked by variability in the ecological components.

The boundaries (global, landscape, ecosystem, organism) of the risk assessment are

initially identified in the analysis plan prepared during problem formulation. These spatial and

temporal scales are further defined in the analysis phase, where specific exposure and effects

scenarios are evaluated. The spatial dimension encompasses both the extent and pattern of effect

as well as the context of the effect within the landscape. Factors to consider include the absolute

area affected, the extent of critical habitats affected compared with a larger area of interest, and

the role or use of the affected area within the landscape.

Adverse effects to assessment endpoints vary with the absolute area of the effect. A larger

affected area may be (1) subject to a greater number of other stressors, increasing the

complications from stressor interactions, (2) more likely to contain sensitive species or habitats, or

(3) more susceptible to landscape-level changes because many ecosystems may be altered by the

stressors.

Nevertheless, a smaller area of effect

is not always associated with lower risk. The

function of an area within the landscape may

be more important than the absolute area. 

Destruction of small but unique areas, such as

critical wetlands, may have important effects

on local and regional wildlife populations. 

Also, in river systems, both riffle and pool

areas provide important microhabitats that

maintain the structure and function of the
C-17

total river ecosystem. Stressors acting on these microhabitats may result in adverse effects to the

entire system.

Spatial factors are important for many species because of the linkages between ecological

landscapes and population dynamics. Linkages between landscapes can provide refuge for

affected populations, and organisms may require corridors between habitat patches for successful

migration.

The temporal scale for ecosystems can vary from seconds (photosynthesis, prokaryotic

reproduction) to centuries (global climate change). Changes within a forest ecosystem can occur

gradually over decades or centuries and may be affected by slowly changing external factors such

as climate. When interpreting adversity, risk assessors should recognize that the time scale of

stressor-induced changes operates within the context of multiple natural time scales. In addition,

temporal responses for ecosystems may involve intrinsic time lags, so responses to a stressor may

be delayed. Thus, it is important to distinguish a stressor’s long-term impacts from its

immediately visible effects. For example, visible changes resulting from eutrophication of aquatic

systems (turbidity, excessive macrophyte growth, population decline) may not become evident for

many years after initial increases in nutrient levels.

Considering the temporal scale of adverse effects leads logically to a consideration of

recovery. Recovery is the rate and extent of return of a population or community to some aspect

of its condition prior to a stressor’s introduction. (While this discussion deals with recovery as a

result of natural processes, risk mitigation options may include restoration activities to facilitate or

speed up the recovery process.) Because ecosystems are dynamic and, even under natural

conditions, constantly changing in response to changes in the physical environment (e.g., weather,

natural disturbances) or other factors, it is unrealistic to expect that a system will remain static at

some level or return to exactly the same state that it was before it was disturbed (Landis et al.,

1993). Thus, the attributes of a “recovered” system should be carefully defined. Examples might

include productivity declines in a eutrophic system, reestablishment of a species at a particular

density, species recolonization of a damaged habitat, or the restoration of health of diseased

organisms. The Agency considered the recovery rate of biological communities in streams and

rivers from disturbances in setting exceedance frequencies for chemical stressors in waste

effluents (U.S. EPA, 1991).

Recovery can be evaluated in spite of the difficulty in predicting events in ecological

systems (e.g., Niemi et al., 1990). For example, it is possible to distinguish changes that are

usually reversible (e.g., stream recovery from sewage effluent discharge), frequently irreversible

(e.g., establishment of introduced species), and always irreversible (e.g., extinction). Risk

assessors should consider the potential irreversibility of significant structural or functional changes

in ecosystems or ecosystem components when evaluating adversity. Physical alterations such as
C-18

deforestation in the coastal hills of Venezuela in recent history and in Britain during the Neolithic

period, for example, changed soil structure and seed sources such that forests cannot easily grow

again (Fisher and Woodmansee, 1994).

The relative rate of recovery can also be estimated. For instance, fish populations in a

stream are likely to recover much faster from exposure to a degradable chemical than from habitat

alterations resulting from stream channelization. Risk assessors can use knowledge of factors,

such as the temporal scales of organisms’ life histories, the availability of adequate stock for

recruitment, and the interspecific and trophic dynamics of the populations, in evaluating the

relative rates of recovery. A fisheries stock or forest might recover in decades, a benthic

invertebrate community in years, and a planktonic community in weeks to months.

Risk assessors should note natural disturbance patterns when evaluating the likelihood of

recovery from anthropogenic stressors. Alternatively, if an ecosystem has become adapted to a

disturbance pattern, it may be affected when the disturbance is removed (e.g., fire-maintained

grasslands). The lack of natural analogs makes it difficult to predict recovery from uniquely

anthropogenic stressors (e.g., synthetic chemicals).

Appendix E illustrates how the criteria for ecological adversity (nature and intensity of

effects, spatial and temporal scales, and recovery) might be used in evaluating two cleanup

options for a marine oil spill. This example also shows that recovery of a system depends not only

on how quickly a stressor is removed, but also on how the cleanup efforts themselves affect the

recovery.

5.3. REPORTING RISKS

When risk characterization is complete, risk assessors should be able to estimate

ecological risks, indicate the overall degree of confidence in the risk estimates, cite lines of

evidence supporting the risk estimates, and interpret the adversity of ecological effects. Usually

this information is included in a risk assessment report (sometimes referred to as a risk

characterization report because of the integrative nature of risk characterization). While the

breadth of ecological risk assessment precludes providing a detailed outline of reporting elements,

the risk assessor should consider the elements listed in text box 5-8 when preparing a risk

assessment report.
C-19

Text Box 5-8. Possible Risk Assessment Report

Elements

C Describe risk assessor/risk manager planning

results.

C Review the conceptual model and the assessment

endpoints.

C Discuss the major data sources and analytical

procedures used.

C Review the stressor-response and exposure

profiles.

C Describe risks to the assessment endpoints,

including risk estimates and adversity evaluations.

C Review and summarize major areas of uncertainty

(as well as their direction) and the approaches

used to address them.

< Discuss the degree of scientific consensus in

key areas of uncertainty.

< Identify major data gaps and, where

appropriate, indicate whether gathering

additional data would add significantly to the

overall confidence in the assessment results.

< Discuss science policy judgments or default

assumptions used to bridge information gaps

and the basis for these assumptions.

< Discuss how the elements of quantitative

uncertainty analysis are embedded in the

estimate of risk.

Like the risk assessment itself, a risk

assessment report may be brief or extensive,

depending on the nature of and the resources

available for the assessment. While it is

important to address the elements described

in text box 5-8, risk assessors should judge

the level of detail required. The report need

not be overly complex or lengthy; it is most

important that the information required to

support a risk management decision be

presented clearly and concisely.

To facilitate mutual understanding, it

is critical that the risk assessment results are

properly presented. Agency policy requires

that risk characterizations be prepared “in a

manner that is clear, transparent, reasonable,

and consistent with other risk

characterizations of similar scope prepared

across programs in the Agency” (U.S. EPA,

1995b). Ways to achieve such characteristics

are described in text box 5-9.

After the risk assessment report is

prepared, the results are discussed with risk

managers. Section 6 provides information on

communication between risk assessors and

risk managers, describes the use of the risk

assessment in a risk management context, and briefly discusses communication of risk assessment

results from risk managers to interested parties and the general public.
C-20

Text Box 5-9. Clear, Transparent, Reasonable, and Consistent Risk Characterizations

For clarity:

• Be brief; avoid jargon.

• Make language and organization understandable to risk managers and the informed lay person.

• Fully discuss and explain unusual issues specific to a particular risk assessment.

For transparency:

• Identify the scientific conclusions separately from policy judgments. 

• Clearly articulate major differing viewpoints of scientific judgments.

• Define and explain the risk assessment purpose (e.g., regulatory purpose, policy analysis, priority setting).

• Fully explain assumptions and biases (scientific and policy).

For reasonableness:

• Integrate all components into an overall conclusion of risk that is complete, informative, and useful in

decision making.

• Acknowledge uncertainties and assumptions in a forthright manner.

• Describe key data as experimental, state-of-the-art, or generally accepted scientific knowledge.

• Identify reasonable alternatives and conclusions that can be derived from the data.

• Define the level of effort (e.g., quick screen, extensive characterization) along with the reason(s) for

selecting this level of effort.

• Explain the status of peer review.

For consistency with other risk characterizations:

• Describe how the risks posed by one set of stressors compare with the risks posed by a similar stressor(s) or

similar environmental conditions.

• Indicate how the strengths and limitations of the assessment compare with past assessments.
C-21

Text Box 6-1. Questions Regarding Risk

Assessment Results (Adapted From U.S. EPA,

1993c)

Questions principally for risk assessors to ask risk

managers:

• Are the risks sufficiently well defined (and data

gaps small enough) to support a risk management

decision?

• Was the right problem analyzed?

• Was the problem adequately characterized?

Questions principally for risk managers to ask risk

assessors:

• What effects might occur?

• How adverse are the effects?

• How likely is it that effects will occur?

• When and where do the effects occur?

• How confident are you in the conclusions of the

risk assessment?

• What are the critical data gaps, and will

information be available in the near future to fill

these gaps?

• Are more ecological risk assessment iterations

required?

• How could monitoring help evaluate the results of

the risk management decision?

6. RELATING ECOLOGICAL INFORMATION TO

RISK MANAGEMENT DECISIONS

After characterizing risks and preparing a risk assessment report (section 5), risk assessors

discuss the results with risk managers (figure 5-1). Risk managers use risk assessment results,

along with other factors (e.g., economic or legal concerns), in making risk management decisions

and as a basis for communicating risks to interested parties and the general public.

Mutual understanding between risk assessors and risk managers regarding risk assessment

results can be facilitated if the questions listed in text box 6-1 are addressed. Risk managers need

to know the major risks to assessment

endpoints and have an idea of whether the

conclusions are supported by a large body of

data or if there are significant data gaps. 

Insufficient resources, lack of consensus, or

other factors may preclude preparation of a

detailed and well-documented risk

characterization. If this is the case, the risk

assessor should clearly articulate any issues,

obstacles, and correctable deficiencies for the

risk manager’s consideration.

In making decisions regarding

ecological risks, risk managers consider other

information, such as social, economic,

political, or legal issues in combination with

risk assessment results. For example, the risk

assessment results may be used as part of an

ecological cost-benefit analysis, which may

require translating resources (identified

through the assessment endpoints) into

monetary values. Traditional economic

considerations may only partially address

changes in ecological resources that are not

considered commodities, intergenerational

resource values, or issues of long-term or

irreversible effects (U.S. EPA, 1995a;

Costanza et al., 1997); however, they may
C-22

Text Box 6-2. Risk Communication Considerations

for Risk Managers (U.S. EPA, 1995b)

• Plan carefully and evaluate the success of your

communication efforts.

• Coordinate and collaborate with other credible

sources.

• Accept and involve the public as a legitimate

partner.

• Listen to the public’s specific concerns.

• Be honest, frank, and open.

• Speak clearly and with compassion.

• Meet the needs of the media.

provide a means of comparing the results of the risk assessment in commensurate units such as

costs. Risk managers may also consider alternative strategies for reducing risks, such as risk

mitigation options or substitutions based on relative risk comparisons. For example, risk

mitigation techniques, such as buffer strips or lower field application rates, can be used to reduce

the exposure (and risk) of a pesticide. Further, by comparing the risk of a new pesticide to other

pesticides currently in use during the registration process, lower overall risk may result. Finally,

risk managers consider and incorporate public opinion and political demands into their decisions. 

Collectively, these other factors may render very high risks acceptable or very low risks

unacceptable.

Risk characterization provides the basis for communicating ecological risks to interested

parties and the general public. This task is usually the responsibility of risk managers, but it may

be shared with risk assessors. Although the final risk assessment document (including its risk

characterization sections) can be made available to the public, the risk communication process is

best served by tailoring information to a particular audience. Irrespective of the specific format, it

is important to clearly describe the ecological resources at risk, their value, and the monetary and

other costs of protecting (and failing to

protect) the resources (U.S. EPA, 1995a).

Managers should clearly describe the

sources and causes of risks and the potential

adversity of the risks (e.g., nature and

intensity, spatial and temporal scale, and

recovery potential). The degree of

confidence in the risk assessment, the

rationale for the risk management decision,

and the options for reducing risk are also

important (U.S. EPA, 1995a). Other risk

communication considerations are provided

in text box 6-2.

Along with discussions of risk and

communications with the public, it is

important for risk managers to consider whether additional follow-on activities are required. 

Depending on the importance of the assessment, confidence in its results, and available resources,

it may be advisable to conduct another iteration of the risk assessment (starting with problem

formulation or analysis) in order to support a final management decision. Another option is to

proceed with the decision, implement the selected management alternative, and develop a

monitoring plan to evaluate the results (see section 1). If the decision is to mitigate risks through
C-23

exposure reduction, for example, monitoring could help determine whether the desired reduction

in exposure (and effects) is achieved.