AN INTRODUCTION TO RISK AND UNCERTAINTY IN THE

EVALUATION OF ENVIRONMENTAL INVESTMENTS

Prepared by 

Charles E. Yoe, Ph.D.

The Greeley-Polhemus Group, Inc.

105 South High Street

West Chester, Pennsylvania 19382-3226

610/692-2224

for

U.S. Army Corps of Engineers

Water Resources Support Center

Institute for Water Resources

Alexandria, VA 22315-3868

U.S. Army Corps of Engineers

Waterways Experiment Station

Vicksburg, Mississippi 39180-6199

Evaluation of Environmental IWR Report 96-R-8

Investments Research Program March 1996

VIEWS, OPINIONS, AND/OR FINDINGS CONTAINED IN THIS REPORT ARE THOSE OF THE

AUTHOR(S) AND SHOULD NOT BE CONSTRUED AS AN OFFICIAL DEPARTMENT OF THE

ARMY POSITION, POLICY, OR DECISION UNLESS SO DESIGNATED BY OTHER OFFICIAL

DOCUMENTATION. CITATIONS OF TRADE NAMES ARE FOR INFORMATIONAL PURPOSES

ONLY AND DO NOT CONSTITUTE AN OFFICIAL ENDORSEMENT OR APPROVAL OF THE USE

OF SUCH COMMERCIAL PRODUCTS. 

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

v

PREFACE

This report was conducted as part of the Evaluation of Environmental Investments Research Program

(EEIRP). The EEIRP is sponsored by Headquarters, U.S. Army Corps of Engineers (HQUSACE). It is jointly

assigned to the U.S. Army Engineer Water Resources Support Center (WRSC), Institute for Water Resources

(IWR), and the U.S. Army Engineer Waterways Experiment Station (WES), Environmental lab (EL). Mr.

William J. Hansen of IWR is the Program Manager, and Mr. H. Roger Hamilton is the WES Manager. Program

Monitors during this study were Mr. John W. Bellinger and Mr. K. Brad Fowler, HQUSACE. The field review

group members that provide complete program direction and their District or Division affiliations are Mr. David

Carney, New Orleans District; Mr. Larry Kilgo, Lower Mississippi Valley Division; Mr. Richard Gorton, Omaha

District; Mr. Bruce D. Carlson, St. Paul District; Mr. Glendon L. Coffee, Mobile District; Ms. Susan E. Durden,

Savannah District; Mr. Scott Miner, San Francisco District; Mr. Robert F. Scott, Fort Worth District; Mr.

Clifford J. Kidd, Baltimore District; Mr. Edwin J. Woodruff, North Pacific Division; and Dr. Michael Passmore,

Walla Walla District. The work was conducted under the Incorporating Risk and Uncertainty Into Environmental

Evaluation Work Unit of the EEIRP. Mr. L. Leigh Skaggs of the Technical Analysis and Research Division

(TARD), IWR and Mr. Richard Kasul of the Natural Resources Division (NRD), WES are the Principal

Investigators.

The work was performed by The Greeley-Polhemus Group, Inc. (GPG) under Task Order No. 1,

Contract No. DACW-72-95-D-0002, managed by Mr. Leigh Skaggs. Dr. Charles Yoe, a principal of GPG, was

the principal author.

Valuable review comments and suggestions were received from Mr. Bob Bass, Galveston District; Mr.

Bob Blama, Baltimore District; Mr. Bruce Carlson, St. Paul District; Ms. Pat Cory, Baltimore District; and Mr.

Steve Garbarino, Baltimore District.

The report was prepared under the general supervision at IWR of Mr. Michael Krouse, Chief, TARD;

and Mr. Kyle E. Schilling, Director, IWR; and at EL of Dr. Robert M. Engler, Chief, NRD and Dr. John W.

Keeley, Director, EL.
vi
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

vii

EXECUTIVE SUMMARY

Incorporating risk and uncertainty analysis into environmental restoration planning studies is seen as a

means of improving the quality of the decision-making process. This report introduces Corps personnel involved

in the planning of environmental restoration projects to the basics of risk and uncertainty analysis. In addition,

it presents a sample incorporation of risk analysis in the application of a habitat evaluation procedure, habitat

suitability index model.

Environmental planning has been done by the Corps for a long time. The growing national interest in

environmental issues has created a new emphasis on environmental values within the Corps’ program. This new

emphasis brings with it new initiatives. Environmental planning, at one level, is nothing more than the

application of good planning techniques to environmental problems and opportunities. Although this report tends

to focus on environmental restoration projects, the principles, tools and methods discussed throughout are equally

applicable to any environmental planning activity.

Environmental planning is future-oriented problem solving and decision-making intended to restore,

preserve, protect and enhance environmental resources or to mitigate unavoidable damages to them. As such,

it is fundamentally decision-making under uncertainty. Incorporating risk and uncertainty analysis into

environmental planning, wherever it is done in the organization, can improve both the planning process and the

quality of decisions made to manage the nation’s environmental resources.

Risk analysis is practiced by analysts from many disciplines. As a result, there is not a single broadly

accepted taxonomy of terms. The terminology used by Corps analysts in environmental restoration is introduced

in a simple manner. For example, risk is defined intuitively as the chance that something bad happens.

Risk and uncertainty analysis is easier to introduce to new applications when there is a structured and

logical approach to the process. There are three broad types of uncertainty: knowledge uncertainty, model

uncertainty and quantity uncertainty. Quantity uncertainty is the most commonly encountered. Understanding

the nature of the uncertain quantity can go a long way toward prescribing a treatment for the uncertainty. Of all

quantities, empirical quantity uncertainty can be the most complex to address, because there are at least seven

different causes for these types of uncertainty. The taxonomy offered in this report provides the new risk analyst

with a way to think about the uncertainty that is present in environmental planning work.

The time may come when risk and uncertainty protocols for environmental restoration studies are

described in detail in Corps’ guidance and the models and tools needed to conduct these analyses are well known

to all. That day has not yet arrived and so the report presents a list of selected tools and broad concepts used by

many risk analysts in Chapter Four. Described in the report are the basic concepts of which environmental

planning risk analysts should be aware, including: acknowledging uncertainty, education and training, EPA’s

ecological risk assessment paradigm, full documentation, jargon and literature, peer review, post facto evaluation,

publicity, risk communication, and software. These tend to address information/communication issues. The more

specific tools described include: analytical solutions, decision criteria, decision trees and influence diagrams,

expert opinion, probability, probability distributions, random sampling, sensitivity analysis, simulations, specialty

models, and statistics.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

viii

Just as there are broad concepts that can be used in risk and uncertainty analysis there are “big picture”

issues of uncertainty encountered in environmental planning studies. These issues are not unique to

environmental planning. In fact, they are more indigenous to the planning process than they are to the

environment. Uncertainties are encountered in each step of the planning process and they can become cumulative

if not addressed, virtually assuring that decision-makers labor in ignorance of potentially important issues.

Nonetheless, the uniqueness and complexity of environmental plans suggests that the uncertainties inherent in

the planning process may become particularly acute in environmental planning. These generic sources of

uncertainty are discussed on a planning step-by-planning-step basis in Chapter Five, where knowledge and use

of the Principles and Guidelines six-step planning process is suggested as a means of reducing uncertainty.

In addition to the big picture uncertainties, there are many uncertainties specific to environmental

planning. Four major sources of uncertainty include: project performance, extrapolation, models and the habitat

evaluation procedures. Common potential sources of uncertainty include: delineation of the study area;

identification of target species; the construction of habitat suitability index models; habitat variable

measurements; calculation of existing and future habitat units; and modeling project performance in the habitat

evaluation procedure.

An example introducing risk-based analysis to the estimation of habitat unit changes is offered to

demonstrate the feasibility of some of the methods presented in the report. A simple example is presented in

Chapter Six illustrating how risk-based analysis can be used to address some of the uncertainty inherent in the

habitat evaluation procedures analysis that is so common to environmental restoration projects. Selected results

from this analysis are presented to demonstrate the types of outputs that can be obtained from a risk-based

analysis.

The example shows it is easy to use spreadsheet-based Monte Carlo/Latin Hypercube simulation to

introduce risk analysis into the estimation of habitat suitability indices and habitat units by simply varying key

data inputs. The results of some habitat suitability index computations are found to be extremely sensitive to the

habitat suitability index model’s structure. In addition, the example suggests that risk-based analysis could be

come a valuable formulation tool insofar as it helps analysts understand the relative importance of the uncertain

variables. The simple example establishes the feasibility of using a risk-based approach to habitat unit estimation

with no significant increase in study costs or schedule and confirms its potential as a tool that can help improve

the quality of environmental restoration planning decisions.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

ix

TABLE OF CONTENTS

PREFACE ............................................................................. v

EXECUTIVE SUMMARY ............................................................... vii

LIST OF FIGURES ..................................................................... xv

LIST OF TABLES ..................................................................... xv

CHAPTER ONE: INTRODUCTION ....................................................... 1

INTRODUCTION ................................................................ 1

THE PURPOSE OF THIS REPORT ................................................. 2

INTENDED AUDIENCE .......................................................... 2

ORGANIZATION OF THE REPORT ................................................ 2

CHAPTER TWO: ENVIRONMENTAL PLANNING .......................................... 5

OVERVIEW .................................................................... 5

PLANNING ..................................................................... 5

ENVIRONMENTAL PLANNING ................................................... 5

THE EEIRP ..................................................................... 7

SUMMARY AND LOOK FORWARD ............................................... 7

CHAPTER THREE: RISK AND UNCERTAINTY ANALYSIS ................................. 9

INTRODUCTION ................................................................ 9

TERMINOLOGY ................................................................ 9

BASIC TERMS ........................................................... 9

THE SPECIALIZED TERMINOLOGY OF RISK AND UNCERTAINTY ............ 9

RISK OR UNCERTAINTY ................................................. 12

WHY BOTHER WITH RISK AND UNCERTAINTY? ................................. 13

TYPES OF UNCERTAINTY ...................................................... 16

KNOWLEDGE UNCERTAINTY ............................................ 16

QUANTITY UNCERTAINTY .............................................. 17

Empirical Quantities ................................................ 17
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

x

TABLE OF CONTENTS (Continued)

Defined Constant .................................................. 18

Decision Variables ................................................. 18

Value Parameters .................................................. 18

Index Variables .................................................... 19

Model Domain Parameters ........................................... 19

Outcome Criteria ................................................... 19

MODEL UNCERTAINTY ................................................. 19

SOURCES OF UNCERTAINTY IN EMPIRICAL QUANTITIES .................. 21

Random Error and Statistical Variation ................................. 21

Subjective Judgments and Systematic Errors ............................. 22

Linguistic Imprecision .............................................. 22

Variability ........................................................ 23

Randomness and Unpredictability ..................................... 23

Disagreement ..................................................... 23

Approximation .................................................... 23

SUMMARY AND LOOK FORWARD .............................................. 24

CHAPTER FOUR: POTENTIAL RISK AND UNCERTAINTY TOOLS ......................... 26

INTRODUCTION ............................................................... 26

BROAD CONCEPTS ............................................................ 26

ACKNOWLEDGING UNCERTAINTY ...................................... 26

EDUCATION AND TRAINING ............................................ 27

EPA’S ECOLOGICAL RISK ANALYSIS PARADIGM ......................... 27

FULL DOCUMENTATION ................................................ 28

JARGON AND LITERATURE ............................................. 28

PEER REVIEW .......................................................... 30

POST FACTO EVALUATION ............................................. 31

PUBLICITY ............................................................. 31

RISK COMMUNICATION ................................................. 31

SOFTWARE ............................................................ 33

SPECIFIC TOOLS .............................................................. 34

ANALYTICAL SOLUTIONS ............................................... 34

DECISION CRITERIA .................................................... 35
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

xi

TABLE OF CONTENTS (Continued)

Laplace Criterion .................................................. 36

Maximin Criterion ................................................. 36

Minimin Criterion .................................................. 36

Maximax Criterion ................................................. 36

Minimax Criterion ................................................. 36

Hurwicz Criterion .................................................. 37

Other Criteria ..................................................... 37

DECISION TREES AND INFLUENCE DIAGRAMS ........................... 37

EXPERT OPINION ....................................................... 39

Professional Judgment .............................................. 39

Elicitation/Opinion Analysis ......................................... 40

Subjective Probability Assessment ..................................... 40

Analytic Hierarchy Process ........................................... 40

PROBABILITY .......................................................... 41

PROBABILITY DISTRIBUTIONS .......................................... 42

Normal Distribution ................................................ 46

Lognormal Distribution ............................................. 47

Exponential Distribution ............................................ 47

Poisson Distribution ................................................ 47

Gamma Distribution ................................................ 48

Weibull Distribution ................................................ 48

Beta Distribution ................................................... 48

Uniform Distribution ............................................... 49

Triangular Distribution .............................................. 49

Binomial Distribution ............................................... 49

Discrete Distribution ................................................ 50

Summary of Distribution Usage ....................................... 50

RANDOM SAMPLING ................................................... 50

SENSITIVITY ANALYSIS ................................................. 50

SIMULATIONS .......................................................... 52

SPECIALTY MODELS .................................................... 55

STATISTICS ............................................................ 55

SUMMARY AND LOOK FORWARD .............................................. 56
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

xii

TABLE OF CONTENTS (Continued)

CHAPTER FIVE: GENERAL SOURCES OF RISK AND UNCERTAINTY IN

ENVIRONMENTAL RESTORATION PLANNING ......................................... 58

INTRODUCTION ............................................................... 58

FOUR GENERAL OBSERVATIONS ............................................... 59

STEP ONE: PROBLEM IDENTIFICATION ......................................... 60

STEP TWO: INVENTORY AND FORECAST ....................................... 61

STEP THREE: FORMULATE ALTERNATIVE PLANS ............................... 63

STEP FOUR: EVALUATE PLANS ................................................ 63

STEP FIVE: COMPARE PLANS .................................................. 64

STEP SIX: SELECT A PLAN ..................................................... 64

WHAT IS TO BE DONE ABOUT GENERAL SOURCES OF RISK AND

UNCERTAINTY? ........................................................ 65

SUMMARY AND LOOK FORWARD .............................................. 65

CHAPTER SIX: SELECTED SPECIFIC SOURCES OF RISK AND UNCERTAINTY IN

ENVIRONMENTAL RESTORATION PLANNING ................................... 66

INTRODUCTION ............................................................... 66

PROJECT PERFORMANCE ...................................................... 66

PROJECT OUTPUTS ..................................................... 66

EFFECTIVENESS OF PLANS .............................................. 67

EXTRAPOLATION ............................................................. 68

MODELS ...................................................................... 69

HABITAT EVALUATION PROCEDURE ........................................... 70

POTENTIAL SOURCES OF UNCERTAINTY IN HEP ANALYSIS ...................... 72

DELINEATION OF THE STUDY AREA ..................................... 72

TARGET SPECIES ....................................................... 73

HSI MODELS ........................................................... 74

HABITAT VARIABLE MEASUREMENTS ................................... 75

EXISTING HABITAT UNIT ESTIMATES .................................... 76

FUTURE HABITAT VARIABLE ESTIMATES ................................ 77

CHANGE IN HU’S ....................................................... 77

PROJECT PERFORMANCE IN HEP ANALYSIS .............................. 78

RISK ANALYSIS EXAMPLE: MOTTLED DUCK HSI MODEL ........................ 79

SUMMARY AND LOOK FORWARD .............................................. 89
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

xiii

TABLE OF CONTENTS (Continued)

CHAPTER SEVEN: SUMMARY AND CONCLUSIONS ...................................... 91

REFERENCES ........................................................................ 93

INDEX ............................................................................... 95

APPENDIX A: MOTTLED DUCK RISK MODEL .......................................... A-1
xiv
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

xv

LIST OF FIGURES

Figure 1: Corps Six-Step Planning Process .............................................. 5

Figure 2: Elements of Risk Analysis .................................................. 12

Figure 3: Risk & Uncertainty Relationship ............................................. 13

Figure 4: Outrage Factors .......................................................... 32

Figure 5: Sample Event Tree ........................................................ 38

Figure 6: Influence Diagram ........................................................ 39

Figure 7: Sample AHP Decision Problem .............................................. 41

Figure 8: Striped Bass Histogram .................................................... 43

Figure 9: Probability Distribution of Fish Lengths ....................................... 43

Figure 10: Truncated Normal Distribution .............................................. 46

Figure 11: Sample SI Functions ....................................................... 71

Figure 12: Sample HSI Model ........................................................ 80

Figure 13: CDF for HU Costs ........................................................ 86

Figure 14: Regression Sensitivity for $/HU ............................................. 88

LIST OF TABLES

Table 1: Types of Empirical Quantities Encountered in the Planning Framework .............. 17

Table 2: Sources of Uncertainty in Empirical Quantities .................................. 21

Table 3: Hypothetical Estimates of White Tail Deer Population ............................ 36

Table 4: Distribution Summary ..................................................... 51

Table 5: HEP Uncertainties ........................................................ 79

Table 6: Without Project Condition Distribution Parameters .............................. 82

Table 7: Selected Simulation Results ................................................. 85

Table 8: Model Sensitivity Based on Choice of HSI Formula .............................. 88
xvi
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

1

CHAPTER ONE: INTRODUCTION

INTRODUCTION

The U.S. Army Corps of Engineers’ Civil Works program has historically evolved and changed to meet

the changing needs and priorities of the Nation. Two relatively recent changes in the Corps’ program are of

particular interest in this report. They are the increased emphasis on environmental outputs of existing and new

projects and the increasing use of risk and uncertainty analysis in the Corps’ decision-making processes.

The trends toward greater emphasis on environmental outputs and more use of risk and uncertainty

analysis began at different times and for different reasons. Now, as environmental activities are routinely

undertaken by planning, operations, engineering and construction divisions throughout all Corps districts and risk

and uncertainty analyses have reached a level of maturity and acceptance, there is a confluence of these two

trends.

The national interest in risk and uncertainty analysis had its genesis in the analysis of environmental risk

analysis in the late 1960s. The National Environmental Policy Act of 1969 (NEPA) is generally credited with

beginning the interest in risk and uncertainty analysis. It stands to reason therefore, that now that the Corps’

involvement in environmental activities is reaching a critical mass, the Corps should begin to introduce techniques

of risk and uncertainty analysis into its decision process in order to improve the quality of decisions.

Introducing risk and uncertainty analysis to the Corps’ environmental activities is a new initiative. It will

require personnel to learn a few new tricks. But the introduction of risk and uncertainty analysis to this area of

endeavor will not impose new significant burdens on analysts or managers. It may be as simple a matter as using

all the information you have, rather than some of it. It may involve playing a few “what if” games during the

decision process. Over time, as the value of risk and uncertainty analysis is proven in the decision process, it may

prove helpful to develop specific tools or adopt more sophisticated analytical techniques.

This report explores the potential for introducing risk and uncertainty analysis into environmental

planning activities in much the same way the Guidelines for Risk and Uncertainty Analysis in Water Resources

Planning, IWR Report 92-R-1 (Principles) and 92-R-2 (Example) did for water resources planning generally.

It begins with the simplest, least effort and most effective ways to improve the quality of decision-making in the

Corps’ growing environmental planning activities. Those who learn to use risk and uncertainty analysis well, and

early, will reap the benefits of improved decision-making through the better informed development of more robust

plans.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

2

THE PURPOSE OF THIS REPORT

The purpose of this report is to introduce Corps planners and other interested parties to the concepts of

risk and uncertainty that already are or may soon become useful in improving the quality of decisions made in

the formulation and evaluation of environmental investments. The reasons for doing this now are straightforward:

I) risk and uncertainty analysis contributes to better decision- making; ii) the Corps is already making extensive

and effective use of these techniques in other programs; iii) there is growing public demand for environmental

risk analysis; and, iv) there is a reasonable likelihood that risk analysis will become a legislative requirement in

the not-too-distant future. The objectives of this report are simple. First, it explains what is meant by

environmental planning. Second, it provides a brief introduction to the “concept” of risk and uncertainty analysis.

 Third, it provides an introduction to some of the basic tools and methods of risk and uncertainty analysis. Fourth,

it integrates the concepts of environmental planning and risk and uncertainty analysis through a generic

identification of potential sources of risk and uncertainty in the Water Resources Council’s six-step planning

process used by the Corps. Finally, it introduces some potential tools and methods that can be used specifically

to do risk and uncertainty analysis in environmental planning, with particular emphasis on the habitat evaluation

procedure (HEP). It is not a procedures manual nor does it prescribe a specific approach for any situation. It is

hoped that the introduction to the concepts, tools, and methods of risk and uncertainty analysis will stimulate

creativity in their application that will ultimately lead to formal analytical approaches and tools.

INTENDED AUDIENCE

The basic audience for this report is “Corps planners.” This includes personnel assigned to planning

divisions and others as well. Environmental activities are being conducted in various locations throughout the

Corps’ organization. Planning divisions conduct Section 1135 and other planning studies. Operations personnel

are looking for beneficial uses for dredged material and conducting their regulatory work. Environmentalists can

be found throughout the organization doing analyses for various elements of the Corps’ program. All of these

people are making decisions that affect the environment and the future. As such, they are doing planning, though

they may not recognize or consider what they do as “planning”. It is to the people who do this kind of work,

regardless of the division, section or branch to which they are assigned, that this report is targeted.

ORGANIZATION OF THE REPORT

The report is organized to achieve the above objectives. In the next two chapters, the basic concepts of

environmental planning and risk and uncertainty analysis are presented. This is done to develop the scope of this

report and to introduce some of the basic terminology it uses. The chapter on environmental planning indicates

that this is not really so new, although the increasing emphasis on environmental restoration may be. The risk

and uncertainty chapter defines basic terminology, discusses the reasons for doing risk and uncertainty analysis

in more detail, and offers a taxonomy of uncertainty that may help the reader, new to this area, to think about

uncertainty in their domain of expertise. 

The fourth chapter presents an introductory overview to a number of broad concepts and specific tools

that can be used to address the sources of risk and uncertainty identified in subsequent chapters. The next two

chapters integrate environmental planning and risk and uncertainty analysis. Chapter 5 presents higher order

sources of uncertainty in environmental planning. They are called “higher order” because they are, to a great
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

3

extent, generic sources of uncertainty that might be found in any planning activities. They are illustrated and

presented with an environmental flavor in the context of the Corps’ six-step planning process. Chapter 6

presents some sources of risk and uncertainty that are specific to environmental planning activities and a sample

application to the use of a habitat suitability index (HSI) model in a HEP analysis. The report concludes with

a summary and some recommendations about which sources of risk and uncertainty can be addressed first in order

to begin to incorporate risk and uncertainty analysis into environmental planning activities. 
4
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 The report is circulating in draft form as IWR Report 95-R-15, December 1995. 1

5

Figure 1: Corps Six-Step Planning Process

1. Identify problems & opportunities

2. Inventory & forecast resources

3. Formulate alternative plans

4. Evaluate plan effects

5. Compare effects of alternative plans

6. Select plan

CHAPTER TWO: ENVIRONMENTAL PLANNING

OVERVIEW

This Chapter defines environmental planning, focussing on restoration as the environmental investment

of primary interest in this report. Despite its focus, all the principles, tools and methods discussed in this report

apply to virtually all types of environmental planning. 

PLANNING

Planning is defined in different ways by different people. It was defined recently in the Corps’ Draft

Planning Manual as “the deliberate social or organizational activity of developing an optimal strategy for

1

solving problems and achieving a desired set of goals.” Thus, planning is goal-oriented problem solving.

Defined in this way, planning is an activity that is not confined to the Corps’ planning offices. Personnel in

operations, engineering and construction are doing planning. These may be different kinds of planning activities,

granted, but they are planning activities nonetheless.

The general approach to planning will vary with the people doing it and the purposes for which it is done.

In some cases, there will be a structured, formal approach to planning as there is in traditional planning studies.

In other cases, the problem solving is more gut-level intuition or ad hoc in nature. In order to be able to discuss

planning we need a common understanding of what we mean. Though there are many models, the one used here

is the six-step planning process defined by the U.S. Water Resource Council’s Economic and Environmental

Principles and Guidelines for Water and Related Land Resources Implementation Studies, commonly called

the “P&G”.

The six planning steps are summarized in Figure 1. Though such a formal framework may not be used

in all the environmental planning activities that are undertaken by the Corps, they easily could be. It is safe to

assume that all the activities should at least have the fundamental elements of identifying problems, formulating

alternative solutions and identifying the best

solution from among the possibilities. Thus,

planning as used in this report implies a more or

less formal process comprising the six steps. 

ENVIRONMENTAL PLANNING

Environmental planning is

differentiated from the more traditional types of

planning the Corps has done, e.g., flood damage

reduction and navigation, only by the specific

goals to be achieved and the problems to be
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

6

New Emphasis on an Old Objective

The roots of the Corps’ tradition of

environmental planning can be found in a long list of

authorization acts, environmental laws and executive

orders. Examples of the Corps’ new emphasis on

environmental planning are evident in a number of

places. These include the Chief of Engineers’

February 1990 memorandum on “Strategic Direction

for Environmental Engineering” and the Assistant

Secretary of the Army’s June 1990 “Statement on

Environmental Approaches”. Policy Guidance Letters

24 (March 1991) and 35 (March 1992) followed up on

the earlier policy pronouncements.

New emphasis has come through legislation

as well. The Water Resources Development Act 

(WRDA) of 1986 in Section 1135 instructed the Corps

to consider how existing projects might be altered to

achieve environmental objectives. Section 1103 of this

act authorized the Upper Mississippi Environmental

Management Plan. Section 306 of WRDA of 1990

authorizes “environmental protection” as a mission for

the Corps. Section 307 calls for a wetlands action plan

with a goal of “no net loss” of wetlands. The Coastal

Wetlands Planning, Protection and Restoration Act of

1990 (Public Law (PL) 101-646) authorized the Corps

to cooperate with other agencies and the State of

Louisiana to identify and construct wetlands projects. 

Section 204 of WRDA of 1992 encourages the Corps

to find beneficial uses of dredged material. Specific

actions like the 1992 commitment of the Corps to the

Kissimmee River, Florida restoration project have also

served to enhance the new environmental emphasis.

solved. Though some consider environmental planning new, it is not. In fact, it is decades old. What is new is

the emphasis it is receiving and the ways in which it is evolving. It might be more accurate to say that increasing

national interest in and importance of environmental values is receiving new emphasis in the water resources

planning process, rather than to suggest that “environmental planning” is something new.

In broad terms, environmental planning could

be considered to be the development of optimal

strategies that preserve, protect, restore or enhance

environmental resources and systems or that mitigate

adverse impacts upon environmental resources and

systems. Although the Corps does not have the

authority to pursue all of these strategies,

environmental planning within the Corps has been

going on for some time.

What does environmental planning mean in

the context of the Corps’ program? Is it the fish and

wildlife work the Corps has been authorized to do

since 1958? Does it mean enhancing hunting and

fishing opportunities? Does environmental planning

mean the evaluation of environmental quality (EQ)

effects of projects? Environmental impact assessment

is nothing new. To some, environmental planning

involves the work of the Corps’ regulatory program.

Clearly, finding beneficial uses for dredged material

is an example of environmental planning coincident

with long-time disposal responsibilities. Some Corps

personnel suggest hazardous, toxic and radioactive

waste issues encountered by the Corps require

environmental planning. Others think of ecosystem or

environmental restoration when they think of

environmental planning.

In this report, environmental planning means

all these things and more. Any Corps’ activities that

involve environmental resources and systems directly

or indirectly can be considered part of environmental

planning. Whether this work is conducted by

Planning, Operations, or other offices, the work is

essentially planning so long as it is involves developing optimal strategies for goal-oriented problem solving. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 These non-traditional planning efforts involve environmental infrastructure, the Corps’ regulatory program, support for the military and 2

others, master planning, regional analysis, drought preparation studies, planning assistance to the States, and other special studies and projects.

7

THE EEIRP

This report is being developed as part of the Evaluation of Environmental Investments Research Program

(EEIRP). As such, it has a narrower focus than what we have described as environmental planning. The focus

of the EEIRP, hence this report, is on the evaluation of environmental investments. In practical terms this has

come to mean the evaluation of environmental restoration projects. This would include projects authorized by

Section 1135 of the Water Resources Development Act (WRDA) of 1986 and Section 313 of WRDA‘92 as well

as those listed in the sidebar on the preceding page. 

Although this report tends to focus more narrowly on environmental restoration projects, its contents are

equally applicable to any area of environmental planning. For example, to the extent that the uncertainty

associated with habitat evaluation analyses are of interest to the restoration of degraded ecosystem structure

function and dynamic processes (the focus of this report) it will also be of interest for mitigation studies,

environmental impact assessments, and any other endeavors that require habitat evaluations. 

Environmental restoration projects are new and they are different. “Traditional” Corps planning is

national economic development (NED) oriented, insofar as the primary objective is to maximize net NED

benefits. Environmental restoration is a prime example of “non-traditional” planning, i.e., planning that is not

driven by the NED objective. Absent the traditional metric of NED benefits and costs, it is necessary to develop 2

new ways to evaluate the impacts of certain kinds of projects. One of the purposes of the EEIRP is to develop

some of these evaluation methods. The purpose of this report is to begin to consider the need for and ways in

which risk and uncertainty analysis may be introduced to the evaluation of environmental infrastructure aimed

primarily at restoring degraded ecosystems. The reason for taking this step is to improve the quality of decisions

made in these planning efforts.

SUMMARY AND LOOK FORWARD

Environmental planning has been done by the Corps for a long time. The growing national interest in

environmental issues has created a new emphasis on environmental values within the Corps’ program. This new

emphasis brings with it new initiatives. Environmental planning, at one level, is nothing more than the application

of good planning techniques to environmental problems and opportunities. Although this report tends to focus

on environmental restoration projects, the principles, tools and methods discussed throughout are equally

applicable to any environmental planning activities.

As this chapter has provided an overview to the notion of environmental planning, so the next chapter

does for risk and uncertainty analysis. It provides the reader with an introduction to the principle language and

concepts of risk and uncertainty analysis that will be used in subsequent chapters when environmental planning

and risk analysis are brought together.
8
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

9

CHAPTER THREE: RISK AND UNCERTAINTY ANALYSIS

INTRODUCTION

This chapter provides the reader with an introduction to the language and concepts of risk and uncertainty

analysis that are needed in order to introduce risk and uncertainty analysis in the evaluation of environmental

investments. There are six main sections including this one. The section that follows is about words. It defines

the most basic terms, considers some of the confusing terminology that has evolved around the concepts of risk

analysis and the environment, and concludes with a return to the distinction between what is risky and what is

uncertain.

The third section of this chapter presents a rationale for environmental planners’ being concerned with

risk and uncertainty analysis. The bulk of the remainder of the chapter comprises two sections that provide the

reader with a taxonomy of uncertainty. One discusses the kinds of things that can be uncertain. The other

discusses the sources of this uncertainty. Although this taxonomy is not unique to environmental planning, it is

very valuable because it gives an expert, untrained in uncertainty analysis, a structured way in which to begin to

think about uncertainty in his or her domain of expertise. The chapter concludes with a summary and a look

forward to the next chapter.

TERMINOLOGY

BASIC TERMS

Uncertainty describes any situation in which we are not absolutely sure. Risk describes a situation in

which there is a chance of something bad happening. Risk analysis is the analysis of risky situations. Risk

analysis is broken into two parts: risk evaluation and risk management. Risk evaluation is the objective analysis

of the situation that answers the question, “How risky is this situation?” Risk management then answers the

question, “What shall we do about it?”

Risk and uncertainty analysis is not a section of a report that is added after the study is complete. It is

not playing “what if” games with a critical variable or two in order to satisfy a requirement. Introducing risk and

uncertainty analysis to environmental restoration planning means changing the way analysts approach the

planning process. At its most basic level, it requires planners to recognize and admit there is really very little of

which they are absolutely sure. This humbling admission can then lead planners to approach their goal-oriented

problem solving tasks in a more informed fashion, because recognizing the limits of our expertise and knowledge

is always one of wisdom’s more difficult tasks. Never is this more the case than in addressing complex

environmental systems. 

THE SPECIALIZED TERMINOLOGY OF RISK AND UNCERTAINTY

The previous section offers some simple definitions of basic terms used in risk and uncertainty analysis.

Despite that apparent simplicity, the language of risk and uncertainty analysis can be very confusing because there

are many disciplines that make use of its tools and methods. Unfortunately, each specialty application has
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

10

developed its own language. For example, what one group calls risk analysis another calls risk assessment.

These discrepancies can’t be cleared up here but the meaning of terms as used in this report can be clarified. If

you read the risk and uncertainty literature, expect to encounter a rather confusing terminology that will not

always match the simple definitions offered here. Following are overviews of the more common specialized

terminology of risk and uncertainty analysis.

The insurance industry first coined the term risk management to describe the tools and methods employed

by businesses to limit and control the economic risks they faced. Loss through accident, fire, disaster, and so on

were among the first hazards businesses faced. In recent decades, the hazards to a firm’s financial health have

mutated to the legal, political, and financial realms as well. Risk management, in this context, is the

identification, measurement, and treatment of exposures to potential accidental losses. Risk management is

essential to the survival and profitability of a business. Risk management is generally not what we mean when

we speak about risk and uncertainty analysis for the Corps of Engineers.

Environmental risk analysis (a.k.a., environmental risk assessment or environmental health risk) sounds

exactly like what this report is talking about. It is not. The term “environmental risk analysis” has most

commonly come to mean the assessment of toxic risk to a species. Most often, the toxic risk is exposure to some

chemical, radioactive or other hazardous material and the species of most interest is, of course, humans. The U.S.

Environmental Protection Agency (EPA) has developed principles of environmental risk analysis that include the

following four steps. 

First, hazard identification is the identification of the adverse effects a substance/agent can cause.

Second, the dose-response assessment is the process of establishing or characterizing the relationship between

the dose of an agent and the incidence of adverse health effects. The Corps uses this concept explicitly in the

bioassays it conducts when identifying contaminants in dredged material. LD50, or lethal dose 50, is that level

of contaminant (i.e., the dose) that kills 50% of the population (i.e., the response). It uses the concept implicitly

in its expected annual damage computations for flood damage reduction studies. The dose is flood stage, the

response is damage. Third, exposure assessment is the process of estimating the intensity, frequency, and duration

of human or animal exposure to an agent in the environment. Fourth, risk characterization is the final step. It

is the synthesis of the dose-response and exposure assessments, often presented in the form of a toxicity/exposure

or T/E ratio. The purpose of this step is to estimate the incidence of health effects under various conditions of

human or animal exposure described in the exposure assessment. Although the Corps makes both explicit and

implicit use of these concepts, environmental risk analysis is, generally, not what we meant by risk and

uncertainty analysis in this report.

Human health risk assessment is split into cancer risks and non-cancer risks. As environmental risk

analysis has more and more come to be identified with risks to human health through exposure to chemicals and

other agents in the environment, it has been necessary to develop new terminology. Ecological risk assessment

is one of these terms. Rather than concentrating on the effects of a toxic substance on a single species, ecological

risk assessments are concerned with the effect of stressors found in the environment that can affect single species,

communities of species, and entire ecosystems.

The risk assessment paradigm under development for ecological risk assessment is essentially an

adaptation of the four-step environmental risk analysis paradigm summarized above. Instead of a dose-response
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 “Compilation and Review of Completed Restoration and Mitigation Studies in Developing an Evaluation Framework for 3

Environmental Resources, Volumes I and II”, IWR Reports 95-R-4 and 95-R-5, April 1995.

11

relationship, a stress-response assessment has been proposed. Ecosystems are subject to a much wider range of

stresses including man-made and natural, chemical and non-chemical. Generally, ecosystem risk assessment is

not what is meant by risk and uncertainty in the Corps’ context. However, the paradigm under development holds

considerable promise for use in analyses related to habitat loss, global climate change and other non-chemical

ecological stresses. Once basic acceptance of risk and uncertainty concepts has been established, adaptation, if

not adoption, of some or all of the ecological risk assessment paradigm under development may warrant serious

consideration.

An early effort at assessing the extent to which risk and uncertainty analysis is currently used by

3

environmental planners indicated that Corps personnel have a rather limited view of what risk and uncertainty

means. The predominant notion of risk analysis is that it applies when there is some danger of loss of human life

or failure of a project. There is no apparent awareness of uncertainty analysis. Though loss of life and project

failure are part of risk and uncertainty analysis, they are only a small part.

To distinguish the Corps’ applications of risk and uncertainty analysis from those addressed above, the

types of analyses of interest to the Corps will be called risk and uncertainty in water resources. Thus, we’re

speaking about risk and uncertainty analysis in a generic sense with specific applications of the general theory

and tools to situations that involve water resources. 

For the most part, Corps projects are concerned with a wide variety of nontoxic risks that are site-specific

and are caused by such processes as dredging a navigation channel, construction of a dam, creation of wetlands,

or natural disaster recovery efforts. The impacts tend to be systematic rather than incremental. That is, a single

event - like the construction of a project, a flood, a volcanic eruption, an earthquake - produces widespread

repercussions. These could include modification of the hydrologic cycle, alteration of natural hydraulics, or

changes in fish and wildlife habitat; i.e., temporary or permanent changes in natural and human systems. The

damages that result from these changes may be economic, environmental or safety related. Incremental effects

are in some sense additive, like the build-up of toxins in the food chain or erosion of a streambank or island. 

A paradigm is an example intended to demonstrate how something is to be done. It provides a pattern

that can be followed by others. The EPA has devoted considerable effort to developing risk analysis paradigms.

The four-step process described above is an example of a paradigm. The Corps does not rely on a single

paradigm. 

EC 1105-2-205, Risk Analysis Framework for Evaluation of Hydrology/Hydraulics and Economics

in Flood Damage Reduction Studies, presents a paradigm for calculating expected annual damages. Risk-based

models have been developed for use in major rehabilitation projects and dredging. Underlying all these risk-based

models is a basic theoretical framework comprising the three steps shown in Figure 2. In each, the hazard

(flooding, component failure, uncertain quantities) are identified. The probability of the hazard causing a problem

is quantified and the consequence of the problem caused by the hazard is quantified and modeled.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

12

Figure 2: Elements of Risk Analysis

! Hazard identification

! Probability estimation

! Consequence modeling

The EPA health risk paradigm can be

adapted for use by the Corps. Flooding is a hazard

that is readily amenable to the hazard, doseresponse, exposure, and risk characterization model.

Contaminant analysis of dredged material requires

no adaptation. There are easily imagined potential

uses for an ecological risk assessment paradigm.

But the Corps’ work will never be reduced to a

single or even a few risk and uncertainty analysis

paradigms. The work is too diverse and too unique in its hazards and their consequences to ever reduce to one

or two established ways of approaching risk and uncertainty. Nonetheless, there are recurring and important

situations that do lend themselves to the development of paradigms and models that could be used over and over

throughout the organization. In time, some may be developed for the Corps’ environmental investment work.

This report is content to speculate about where these paradigms might arise and where they are never likely to

occur. 

RISK OR UNCERTAINTY

The simple definitions offered above may not be enough to help the new reader to distinguish between

risk and uncertainty. It is less important that a situation is properly classified as risk or uncertainty then that

uncertain situations that are important to the decision process are recognized and dealt with forthrightly and as

effectively as possible throughout the planning process. Any situation in which the outcome or final result is not

known with complete certainty is an uncertain one. This covers a lot of ground. We are certain c-a-t spells cat.

We are certain of our names. We are certain the sun will rise again tomorrow, aren’t we? We are sure we set our

keys here a minute ago. We know it’s going to rain tomorrow. When you get right down to it, there is relatively

little in our lives that we can be certain about.

Not all the uncertainty we face is equally important. For all intents and purposes, we can ignore

uncertainty about the sun’s rising and setting. Though not absolutely certain, many things are effectively certain.

Much of our scientific theory falls in this category of knowledge. Other things, like where our keys, glasses,

umbrellas, and the like currently reside are rather trivial in the scheme of things and can be ignored. 

Some uncertain things fall in between these extremes and they need to be addressed. You find yourself

in a strange town at night, is it safe to walk in this neighborhood? You don’t understand the question on a final

exam. Do you drink the water in a foreign country? You have no idea where Faraway Street is.

It’s the same in water resources planning generally and environmental restoration planning specifically.

There are many things that are uncertain and many sources of uncertainty (topics covered later in this chapter).

Not all are created equal. The first step in dealing with this uncertainty in a rational way is going to be to figure

out what uncertainty is most important and what uncertainty can be ignored. The basic criteria for establishing

that importance is to consider whether the uncertain variable could have a significant effect on the decision to be

made. If it could, it’s important. If it’s important, you need to consider the uncertainty. Some tools and methods

that can be used to address various uncertain situations are presented in the next chapter. 
Figure 3: Risk & Uncertainty Relationship

Uncertainty

Risk

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

13

There is a class of uncertain situations that arise frequently enough to earn them the special designation

of risky situations, as shown in Figure 3. Risk was defined above as the chance of something bad happening.

The chance is often unknown as are the nature and extent of the “bad” thing that might happen. Risky situations

are, then, characterized by uncertain likelihood of occurrence and uncertain consequences; clearly, a case of

uncertainty.

A few examples help illustrate these concepts. We may be uncertain about how an ecosystem functions;

we may be uncertain about a habitat suitability index; we may not be comfortable extrapolating from the LD50

to the No Observed Adverse Effect Level (NOAEL) of a contaminant; and, we may be uncertain about the true

output of a mitigation or restoration project. On the other hand, there is a risk of drought or flood and there is

a risk that our project will not perform as expected. The differences can be subtle at times. How we label a

decision problem is less important than that we entertain it. The terms, risk analysis, risk and uncertainty

analysis, and uncertainty analysis are used as virtual synonyms in this report, less because there is no difference

than because the difference is relatively unimportant.

One final note on jargon is necessary. Risk-based analysis is a term invented by the Corps that is

essentially a synonym for risk and uncertainty analysis. Risk-based analysis is defined in EC 1105-2-205 as:

“...an approach to evaluation and decision-making that explicitly, and to the extent practical, analytically

incorporates considerations of risk and uncertainty.”

WHY BOTHER WITH RISK AND UNCERTAINTY?

Each time risk and uncertainty analysis has been introduced to a new area of the Corps’ program the

reaction has always been the same. What is it? Is it just one more thing we have to do? Is it another way for

people to keep us from doing what needs to be done. How do you do this? Who’s going to do this? What’s the

point?
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

14

Four reasons are offered for Corps personnel, involved in environmental activities, to be concerned with

risk and uncertainty analysis. First, planning requires decision-making under uncertainty, so it’s just good

planning. Second, risk and uncertainty analyses are already being used to improve decision- making in other parts

of the Corps’ program without difficulty. Third, there is a growing demand for it. Fourth, there is growing

political interest in providing regulatory reform that focuses national resources on the greatest risks to human

health, safety, and the environment. Any one of these reasons should be sufficient to encourage environmental

resources planners and managers to use risk and uncertainty analysis. Taken together they form a rationale for

beginning to incorporate risk and uncertainty analyses into the Corps environmental planning.

The first reason, that using risk and uncertainty analysis is just good planning, may be the most

compelling. If planning is an attempt to control future consequences through present actions, an appropriate view

of ecosystem restoration planning, it is obvious we are confronted with considerable uncertainty. We don’t fully

understand how complex ecosystems function. We have to work without all the data we’d like to have. We’re

not sure how our alternatives will perform and often don’t know exactly what they will cost to build. There is

also considerable uncertainty over how alternatives can best be evaluated.

Faced with these and other sources of uncertainty it would be unprofessional and irresponsible not to

admit and face the uncertainty with which we are confronted. Given the reality of uncertainty, it only makes sense

to deal with it in a rational and systematic way. This requires the application of the tools and methods of risk and

uncertainty analysis. When a problem exists, it makes sense to recognize it and deal with it. That is simply good

planning. Good planning leads to good plans and good decisions.

The second reason for using risk and uncertainty analysis is that it is already being successfully used by

the Corps. Risk and uncertainty are nothing new. Applying these concepts to environmental planning may be

new, but it’s nothing more than the logical extension of good analysis to a new subject area. Risk analysis began

with the dam safety program more than a decade ago. Flood damage reduction studies are required to use a riskbased analysis to evaluate hydrology/hydraulics and economics (see EC 1105-2-205). Major rehabilitation of

locks and dams has been relying on risk-based analysis for years and the tools and methods in use have grown

increasingly sophisticated. Dredging decisions are now assisted by risk-based analytical tools. Dredged material

disposal programs that use the Evaluation of Dredged Material Proposed for Ocean Disposal - Testing

Manual, 1991, also known as the “Green Book”, have been applying risk-based analytical techniques for years.

Risk and uncertainty analyses have been used for channel width determinations for navigation projects. Pilot

studies have been conducted to explore the use of risk and uncertainty tools and methods in determining

budgeting priorities of operation and maintenance budgets. These techniques are examples of some of the

applications that have been used to improve the quality of decision-making within the Corps of Engineers. 

 Since the inception of the Corps’ Risk Analysis Research Program at the Institute for Water Resources

(IWR), there has been a snowball effect on the application of risk and uncertainty analysis methods to water

resources planning. Now that the feasibility and value of these methods have been proven, efforts to spread them

to new applications within the Corps program continue. Incorporating risk and uncertainty analysis into

environmental investment programs is just part of the natural evolution of improving decision-making processes.

It just makes sense.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Memorandum for Chief, CECW-AO/EINARSEN from CEWRC-IWR-P dated 16 Nov 94, Subject: Draft Administration “Principles 4

for Risk Assessment, Management and Communication.”

 Senate Bills 229 and 333 and House Resolution 96 are additional risk analysis items before the 104th Congress. 5

15

The third reason is that there is a growing demand for risk-based analysis. Growing public awareness

of environmental issues and of risks in general has raised public expectations of the quality of environmental risk

analysis. Informed public debate has resulted in an unwillingness of public officials and the public to leave the

discussion of environmental risks to the so-called experts. “Suppose,” the non-Federal partner to a marsh

creation project asks, “we put up our money and the project doesn’t work, will it be redone? Who pays for that?”

This growing demand for risk analysis leads directly to a fourth argument for incorporating risk and

uncertainty into environmental project planning; the current Administration and Congress have begun to move

in the direction of requiring it. Any teenager recognizes the compelling nature of the “because I said so” rationale

for doing something. There is a good chance that that may soon become a compelling reason for Corps personnel

to conduct risk and uncertainty analysis.

On August 2, 1994 the Clinton Administration published its “Draft Administration Risk Assessment

Principles”. A review of these principles for the Chief of Engineers by IWR said in part:

“Without any reservations, the principles are sound and are virtually the same as those that the

Corps has been promoting in its various planning functions...”4

In addition to these principles, numerous bills were recently introduced in Congress to further the cause

of risk and uncertainty analysis. Two examples were House Resolution (H.R.) 1022 the “Risk Assessment and

Cost-Benefit Act of 1995” and HR 690 the “Risk Assessment and Cost-Benefit Analysis Act of 1995.” Either 5

would have mandated the conduct of risk analyses that would have affected the Corps of Engineers. By beginning

to incorporate risk and uncertainty analysis into environmental planning now, planning and decision-making can

be improved and Corps personnel will be that much further along the learning curve if and when risk analysis

becomes a Federal requirement.

If your goal as a reader is to understand the basic notions of risk and uncertainty, stop here. The

remainder of this chapter goes beyond that modest goal to offer a few ideas that might help the new reader to

think about uncertainty in a systematic manner. It begins with a classification system or taxonomy of the types

of things that can be uncertain. From there it proceeds to a taxonomy of the sources of uncertainty. Much of the

material in the remainder of this chapter is taken from Uncertainty, a Guide to Dealing with Uncertainty in

Quantitative Risk and Policy Analysis by M. Granger Morgan and Max Henrion. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Here and in the remainder of this chapter we use uncertainty in the inclusive sense that includes risky situations as well. 6

16

TYPES OF UNCERTAINTY

How can we break the “uncertain” world down into just a few components so we can think rationally

about them? What kinds of things are uncertain? Planners can think of uncertainty as coming in three classes: 6

uncertainty in knowledge; uncertainty about quantities; and uncertainty about models. Quarrel with this if you

will, it’s a purely arbitrary way to break things down. It does, however, provide a way to think about uncertainty

encountered in environmental restoration planning. By thinking about the knowledge required to approach a

problem, the data needed to analyze it and the models used in the analysis, one has a structured way to begin to

think about the uncertainty inherent in any situation.

In the sub-sections that follow each of these three classifications are addressed. The discussion turns

to the sources of uncertainty in the next section. 

KNOWLEDGE UNCERTAINTY

Some things are uncertain for the most basic of reasons - we just don’t know. That lack of knowledge

can arise from two different sources. You may not know the capital of Belgium. That is an uncertainty that arises

from ignorance. The information is known (Brussels), but not by you. The cure is relatively obvious, obtain the

knowledge. This can be done through education, training, talking to experts, or acquiring experts through hiring,

contract or coordination.

In other cases, some things are unknown at this point in time but they may become known in the future.

The problem may be incomplete theory; we might simply lack a framework for thinking about or understanding

a situation. Even when the theory is complete a lack of data may limit our understanding of a situation.

Examples of this kind of uncertainty include cures for various diseases; knowledge of ungaged stream flow

regimes; and the performance of environmental projects. In time, cures may be discovered, flow data collected

and many projects monitored. As this happens uncertainty is reduced. The solutions here are more research,

more time, more inspiration and more perspiration.

There are other things that are fundamentally unknowable. These include the questions of the great

philosophers like, “Can God make a rock so heavy He can’t lift it?” and “What makes you so sure God is a He?”

It includes many more serious value-based questions about what “should” be done and the “right” thing to do

in various situations. It also includes things that are random and unpredictable, like the year in which the standard

project flood will occur. Many of the risk situations fall into this category. There is no cure for this kind of

uncertainty. Situations like these must often be represented with probabilities.

One of the most common sources of uncertainty is our limited knowledge. Sometimes it is caused by

a lack of expertise, data, or understanding. Other times it’s caused by the fundamental randomness of the

universe. Whatever it’s cause, it’s a problem that may need to be addressed. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

17

QUANTITY UNCERTAINTY

Planning requires information, lots of it. That information is frequently gathered, organized,

summarized, analyzed and expressed in a quantitative fashion. There are 167 acres of emerging wetlands. There

are six eagle nests. The herbaceous cover suitability index is 0.6. The cost of a duck box is $320. The

quantities used in the planning process are frequently a major source of uncertainty. 

Table 1 presents a classification of empirical quantities encountered in planning as well as some methods

for dealing with the uncertainty that may be found in them. Empirical quantities are measurable properties of the

real world systems being analyzed and modeled. Each type of quantity is discussed briefly below.

Table 1: Types of Empirical Quantities Encountered in the Planning Framework

Type of Quantity Example Treatment of Uncertainty

Empirical quantities Mortality rate, stream discharge Probabilistic, parametric

Defined constant Gallons per acre-foot Certain by definition

Decision variable Level of protection, mitigation goal Parametric

Value parameter Discount rate, value of life Parametric

Index variable Acreage, longitude & latitude Certain by definition

Model domain parameter Study area, planning horizon, base year Parametric

Outcome criterion BCR, incremental costs, habitat units Depends on treatment of inputs

Empirical Quantities

Empirical quantities are the most common quantities encountered in a study. To be empirical, variables

must be measurable in principle. Empirical quantities are the measurable properties of the resources and factors

that need to be measured in the planning process. They include quantities such as population counts for

significant species, water temperatures, stream flows, quantity estimates, the percent of unsubmerged substrate,

the probability that a species will become locally extinct in the next ten years, average erosion or shoaling rates,

roughness coefficients, contaminant loads, and so on. Empirical quantities need not be numerical. They can be

nominal as well, like the lists of species present in a study area.

To illustrate the basic idea, suppose we’re trying to estimate the resident population of white tail deer.

There is a single true value for the white tail deer population at a given place and point in time. Whether we can

actually determine that true value is another matter; we may have census data or only sample information. In

either case there is a strong likelihood we do not know the true population with certainty.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

18

Defined Constant

One type of quantity is always certain. These are defined constants. There will always be 43,560 square

feet to an acre and there are 326,000 gallons of water in an acre-foot. These values are certain by definition and

we never have to worry about their uncertainty.

Decision Variables

Decision variables are quantities the planning team exerts direct control over. They are sometimes called

control variables or policy variables. There may be uncertainty about how densely to plant marsh grasses when

trying to establish tidal wetlands or the best level of environmental mitigation. It makes no sense, however, to

be uncertain about the “true” value of grass density or environmental mitigation because unlike empirical

quantities, decision variables have no true values. Their values are selected by the planning team. Representing

them as probabilistic variables detaches them from the decision process. It is better to vary decision variables

systematically using sensitivity analysis, also known as parametric variation.

Value Parameters

Value parameters represent the preferences of the planning team or the people they represent - the general

public, resource agencies, higher authority. The federal discount rate is an example of a value parameter. The

choice of this uncertain variable has been decided for the Corps by Congress. At times, decision-makers assign

weights to various project results to facilitate trade-off analysis. These weights may be explicit or implicit but

they reflect the values of the decision-maker and should not be varied randomly. 

It is not unusual for a value parameter to be treated as an empirical quantity and represented by a

probability distribution. This, however, is usually a mistake, sometimes a serious one. Value parameters are

value judgments. If the decision-maker is not sure what she prefers it is not helpful to leave the value judgment

to chance. It is better to vary the values parametrically, i.e, first try one, then another. This allows the team to

learn how changes in the parameter's value affect the outcomes in which they are interested.

People are often least sure about value parameters and they may contribute significantly to the

uncertainty about the best alternative plan. If these quantities are treated as probabilistic it may mask the effect

of the different value choices on the planning objectives. A systematic sensitivity analysis can be far more

illuminating and can help decision-makers better understand and refine their preferences.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

19

Index Variables

Index variables are used to identify and locate information in a model. For example, the base year,

project year 10, grid cell 230, 90N50' longitude and 30N42' latitude, are all index variables. There is no reason

to be uncertain about the value of an index variable. However, as long as humans are doing the work, human

errors in measurement are always possible. 

Model Domain Parameters

Model domain parameters specify the scope of the study. We identify the Potomac watershed as the

study area and a 50-year planning horizon. Values like these define the extent of the study. The planning horizon

delimits the temporal domain of the study, the study area delimits the spatial domain. Considering levee heights

in two-foot increments and defined project increments in the incremental cost analysis are additional examples

of model domain parameters.

There are no true model domain parameter values and it would be inappropriate to represent these model

domain parameters with probability distributions. The choice of the domain values is up to the planning team.

Outcome Criteria

The outcome criteria are the variables we use to rank or measure the desirability of possible outcomes.

The Corps has traditionally used net National Economic Development (NED) benefits in its planning studies.

Habitat suitability indices, habitat units and incremental costs are other outcome criteria in common usage for

mitigation and restoration planning. 

New decision criteria will evolve as this branch of planning develops. These outcome criteria will be

deterministic if there is no uncertainty in the inputs used to develop them and there is an unambiguous analytical

method to develop the criteria. However, if the inputs are uncertain or there are no analytical methods for

estimating the criteria, the outcome criteria will be uncertain. For example, if the habitat suitability index is

uncertain and the acreage is certain, habitat units will be an uncertain outcome criteria because it is the product

of an uncertain variable times a certain constant.

MODEL UNCERTAINTY

Models are simplifications of reality used in order to understand or represent complex situations and

systems. Models can be physical or mathematical, the latter being far more common. The ideal model is a simple

one that closely replicates the functioning of a system. Model uncertainty is a type of uncertainty that can be

more important than knowledge or quantity uncertainty in some cases. 

The issue with model uncertainty is closely related to knowledge uncertainty in that it poses the question,

“are we thinking about this system correctly?” In other words, do the models used truly represent what is going

on with the phenomena of interest? Model uncertainty includes the limits of our knowledge and theory as well

as the specific forms of the models we use to represent complex relationships.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

20

Model Uncertainty In An HSI Model

It may be easier to understand the nature of

model uncertainty with a brief example. Consider the

U.S. Department of the Interior Fish and Wildlife

Service’s HSI model for the mottled duck. The preface

describes the model as “. . . a hypothesis of specieshabitat relationships, not a statement of proven cause

and effect relationships.” This is a clear admission of

model uncertainty.

The model uses a variety of literature to

identify eight specific habitat variables. Might there be

nine or more? Are these the eight most important

variables? These are all legitimate questions that point

up the nature of model uncertainty. The suitability

graphs linking the percentage of a habitat variable

present and the suitability of such a percentage

expressed as a zero (low) to one (high) index are linear

or piecewise linear relationships. Many scientists

would agree that few biological relationships can be

accurately described by linear relationships. This is

another potential source of model uncertainty.

Equations used to combine suitability indices

for groupings of habitat variables into cover values

involve forms like cube roots of three factor products

and other presumably arguable weights of the relative

importance of the various habitat variables. Likewise,

the calculations of life requisite values and ultimately

the HSI are all contestable formulations. These

questions raised about the manner in which we think

about the mottled duck are examples of potential

sources of model uncertainty.

Many kinds of model uncertainty can be envisioned in a HEP analysis. There will be considerable

uncertainty about the choice of target species. There is some question about whether the life requisites are truly

understood. The methods by which suitability indices are combined to form a habitat suitability index are open

to vigorous debate. The major appeal of the process may be more its acceptance than its accuracy. 

It is not difficult to envision uncertainty

about the values of the quantities that have to be

entered into the HEP model. It is much more

difficult to try to think about the uncertainty in the

model itself. How much uncertainty is there in the

form of the model? How well developed is the

theory behind the HEP model? Are the assumptions

underlying the model reasonable? Unassailable?

How much uncertainty is there in the validity of

identifying an indicator species to represent the

functioning of an ecosystem? How much uncertainty

is there in tying the future viability of a species to a

habitat suitability index? Are these even proper

ways to think about the problems? 

These kinds of uncertainties are far more

difficult to think about and confront and they are

often overlooked as matter of convenience if not fear

of what a critical analysis would reveal about our

favorite models. Uncertainty about model form

frequently reflects a lack of theory or data, or

perhaps a disagreement among experts about the

underlying scientific or technical mechanisms at

work in the system we are trying to model. Given

the complexity and fledgling state of ecosystem

restoration planning there would appear to be

potential model uncertainty present in many

environmental investment planning activities. 

The biological and habitat models currently

in use are major sources of uncertainty in the

evaluation of environmental resources investment

plans. Lack of theory and knowledge about how

even simple ecosystems function is another common

source of model uncertainty. The danger of model uncertainty often lies in the fact that it is not recognized. Once

the model has been used and accepted there is little incentive for its users to ask critical questions about its

fundamental “appropriateness”.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Most of the discussion of quantity and model uncertainty as well as the discussion that follows are adapted from Uncertainty A Guide 7

to Dealing with Uncertainty in Quantitative Risk and Policy Analysis by M. Granger Morgan and Max Henrion. 

21

SOURCES OF UNCERTAINTY IN EMPIRICAL QUANTITIES

Empirical quantities are the most common source of quantity uncertainty in planning studies. What

causes the uncertainty? It’s not enough to identify the type of uncertainty; we need to know something about its

cause so that the uncertainty can be addressed in an appropriate manner. Some common sources of uncertainty7

in empirical quantities are shown in Table 2. Each is discussed in turn below. 

Table 2: Sources of Uncertainty in Empirical Quantities

Random error and statistical variation

Systematic error and subjective judgment

Linguistic imprecision

Variability

Randomness and unpredictability

Disagreement

Approximation

Random Error and Statistical Variation

One of the best understood sources of uncertainty is the random error or statistical variation that arises

from errors in the direct measurement of a quantity. Few measurements are exactly right. Our instruments are

not properly calibrated, our powers of observation are imperfect, we are flawed human beings. Our tools and

ourselves make mistakes. These are random errors. Examples of this type of uncertainty in environmental

investment will include estimates of the extent of different ground covers, infiltration of water through soil types,

or fish and wildlife counts.

In addition, there is statistical variation that results from random processes like sampling error. Suppose

the average depth of water is estimated by measuring depths at randomly selected locations in a study area.

Imagine the locations were selected by laying a grid over a map of the water body and each grid cell was uniquely

numbered from 1 to 500. If 40 cells are randomly selected and the average depth is computed we would not

expect the same average depth to be obtained if a different sample was taken. In fact, if we could take every

conceivable combination of 40 cells from the population of 500 we would have a distribution of sample means.

The mean of this distribution of sample means would be the true average water depth. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

22

The problem is that when we take one sample and calculate one sample mean we do not know where in

that theoretical distribution of sample means our estimate comes from. The “laws” of chance suggest our

estimate will be reasonably close to the true mean when our sample is large, but there is always a “dumb luck”

chance we’ll have a sample mean quite different from the true mean. Knowing this potential for sample error

exists, it must be dealt with. Inferential statistics provides well known methods for doing just that.

Subjective Judgments and Systematic Errors

Subjective judgments are common sources of systematic errors. So-called experts are frequently ignorant

of their own biases in estimating or assessing quantitative measures. Experts are sometimes the last to recognize

the limits of their experience and knowledge. Even when experts know their limitations, their opinions are often

little more than educated guesses. Guesses are uncertain by their very nature.

Systematic errors arise due to biases in the measurement instrument or the individual taking the

measurement. Attempts to estimate populations from animal excrement counts can lead to errors if the ratio of

excrement to organisms is incorrect. Estimating values with inappropriate or poorly calibrated tools yields

systematic errors. Systematic error is defined as the difference between the true value of the quantity being

measured (the actual number of animals) and the value to which the mean of the measurements converge as more

observations are collected (the estimated population based on our counting method).

Systematic errors can occur in any values that have to be physically measured with equipment that needs

periodic calibration (e.g., hand-levels, the length of a person's pace, flow meters) or that are estimated by people

unaware of their imprecision (e.g., habitat suitability indices). The solutions to these types of uncertainty are

well-calibrated equipment in sufficient quantity for all analysts and rigorous training in data collection techniques.

New analysts, contractors with minimally trained field personnel, and inflexible experts are common sources of

such errors.

Linguistic Imprecision

Language is frequently an imprecise vehicle for communication. Filling a lake shoreline to “minus one

foot” with dredge material sounds very precise until we consider that one agency uses mean low water, another

mean sea level, and a third national geodetic vertical datum. Though everyone is hearing the same words, they

are all thinking something quite different. Worse, they may be unaware of the uncertainty because they

understand exactly what the words mean to them.

In many cases, a bit of clear thinking and precise language can clear up most uncertainty stemming from

imprecise communication. Unfortunately, when a new jargon is developing, as it is with environmental

investments, ecosystem restoration programs, and risk analysis, there is going to be considerable imprecision in

the language and considerable uncertainty about its meaning. There is a great deal of variation in the way

different people interpret words and phrases and their interpretation is very context-dependent. 

Even words in common usage like ecosystems, resources, and functioning, have clouded meanings in new

contexts. What is a functioning ecosystem? Would lumbermen give the same answers an Audubon Society

spokesperson would when describing the functions of the forests of the Pacific Northwest? The linguistic
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

23

imprecision that permits two people to give widely varying and honest definitions to the same terminology is a

significant source of uncertainty. Fortunately, linguistic imprecision can be relatively easily avoided once its

potential existence is recognized.

Variability

Many quantities vary from time-to-time or from place-to-place. The flow or a stream through a project

area during one year is expected to be different from the flow in another year. The birth weight of deer in one

area may differ from the birth weight in another area. Conceptually we might have a high degree of certainty

about a frequency or probability distribution. However, if a single organism is randomly selected from that

population it may differ from the mean because of the variability inherent in the population.

Environmental variables of interest might include size of fish, weight of deer, flow in a stream, salinity,

temperature, tidal fluctuations, or other such variables. An element from one of these populations might be a fish,

a deer, a peak annual flow, an average annual flow, a salinity or temperature reading, and so on. Anyone of these

can be expected to differ from another because of the variability that exists within the population. Uncertainty

due to variability can best be reduced by disaggregation.

Randomness and Unpredictability

Inherent randomness is often defined as uncertainty that is impossible to reduce. The practical issue for

the analyst is, is the uncertainty reducible in practice? A quantity that is random to one person who does not

understand the systems at work may be deterministic for another person who knows the underlying process that

generated the quantity. The planner should try to distinguish uncertainty that can be reduced by further study

from that which is inherently random, for the latter cannot be reduced. This source is clearly related to the type

of uncertainty due to lack of knowledge.

Taking full advantage of the expertise present in the interdisciplinary study team can help reduce this

type of uncertainty to the maximum practical extent. Uncertainty about the cost of constructing structures can

often be reduced by more detailed estimating techniques. Uncertainty about the years in which future floods will

occur cannot be reduced by any practical means and it is not worth trying to do so.

Disagreement

Different people can view the same problem from very different perspectives. Two experts can look at

the same evidence and draw different conclusions. The levees have prevented millions of dollars in damages.

The levees have destroyed thousands of acres of wetlands. Which side is right? When experts disagree about

a quantity, a project’s effect, an ecosystem’s function, or any other critical factor, the nature of the disagreement

should be carefully explained and evaluated.

Approximation

Models are simplifications of complex real systems. Tradeoffs between model complexity and

computation costs dictate that most model results yield mere approximations of the actual values. The only
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

24

reliable way to estimate the uncertainty introduced by the approximation is to increase the resolution of the model

and measure the difference between the two models. In other words, approximation uncertainty can be difficult

to ascertain. It is most often addressed by sensitivity analysis.

Average erosion rates based on comparison of aerial photos taken 25 years apart are approximations.

Depending on the methods used many quantities can be approximations. For example, analysts who base their

estimate of the percent of unsubmerged substrate on visual inspection are using approximations. Calculations

of habitat units are an approximation. Project cost estimates are approximated early in the study process. How

good are the approximations? The only way to tell is to compute the habitat units or costs with a better model.

This is rarely possible. If a better model were available it would have been used.

Occasionally, the best models are not used because they are too expensive. A reconnaissance report may

quantify project outputs in acres of wetland or acres of a restored island. The feasibility report may use a HEP

analysis to better quantify the project outputs. In a case like this, the uncertainty due to approximation can be

estimated in subsequent studies by using the more sophisticated HEP evaluation. Likewise, the uncertainty in

cost estimates is eliminated as projects move to implementation.

SUMMARY AND LOOK FORWARD

Environmental planning is future-oriented problem solving and decision-making intended to restore,

preserve, protect and enhance environmental resources or to mitigate unavoidable damages to them. As such,

it is fundamentally decision-making under uncertainty. Incorporating risk and uncertainty analysis into

environmental planning, wherever it is done in the organization, will improve both the planning process and the

quality of decisions made to manage the nation’s environmental resources.

Risk and uncertainty analysis is easier to introduce to new applications when there is a structured and

logical approach to the process. There are three broad types of uncertainty: knowledge, model and quantity

uncertainty. Quantity uncertainty is the most commonly encountered. Understanding the nature of the uncertain

quantity can go a long way toward prescribing a treatment for the uncertainty. Of all quantities, empirical

quantity uncertainty can be the most complex to deal with, because there are at least seven different causes for

these types of uncertainty. The taxonomy offered in this chapter provides the new risk analyst with a way to think

about the uncertainty that is present in environmental planning work.

Although the taxonomy offers a way for analysts to think about the uncertainty encountered in

environmental planning it does not suggest specific methods for dealing with that uncertainty. That is done

generically in the next chapter which introduces some of the concepts and tools used by risk analysts. Subsequent

chapters will suggest specific applications of some of these tools to environmental planning situations, both

generally (Chapter Five) and specifically (Chapter Six).
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

25
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

26

CHAPTER FOUR: POTENTIAL RISK AND

UNCERTAINTY TOOLS

INTRODUCTION

Once important sources of risk and uncertainty are identified in environmental restoration studies, what

can be done about them? Fortunately, there are a wide variety of tools and methods available for addressing risk

and uncertainty. This section introduces a number of them. The chapter is divided into two major sections,

following this introduction. The first section presents some broad concepts that can be used to introduce risk and

uncertainty to environmental planning. The second section concentrates on more specific tools. 

The concepts and tools presented in this chapter are general methods that can be applied by anyone, such

as hydrologists, economists, environmental scientists, engineers, and others, who are part of an interdisciplinary

team. There is a great deal of range and diversity in the uncertainty analysts will encounter in their work. There

are also a large number of tools available for considering much of this uncertainty. This chapter presents some

of the concepts and tools in common usage in order to give the reader a feel for the nature of what is available

without imparting any detailed knowledge about their usage. This means at times there will be too much detail

for some readers and not enough for others. Additional details on these techniques are readily available in the

literature or form standard college-level courses. It is, perhaps, most appropriate to think of this chapter as a

partial catalogue of tools that are available for future use and adaptation by the Corps of Engineers as it integrates

the use of risk and uncertainty analysis into the environmental planning decision process.

BROAD CONCEPTS

Some of the most obvious places to start with risk and uncertainty analysis are, in retrospect, rather broad

and obvious. The first step might be to acknowledge that uncertainty exists and then to begin to educate people

about the need to do something about it. This education process will require communication, but communication

about risky situations has some unique elements to it. Acknowledgment, education and risk communication are

examples of some of the broad concepts described in this section of the report. These broad concepts are,

perhaps, the types of things common sense would lead us to, given the luxury of time to think about the

application of risk and uncertainty analysis to environmental planning. A number of these broad concepts are

presented here to give the reader the benefit of the common sense thinking others involved in risk analysis have

already done. The concepts are presented in alphabetical order. The order of significance will depend on the

circumstances of each planning activity.

ACKNOWLEDGING UNCERTAINTY

The simplest place to start in incorporating risk and uncertainty analysis into environmental planning

is to explicitly acknowledge the existence of uncertainty in the analysis. Far too often, precision is substituted

for accuracy. Analysts, failing to recognize the assumptions and limitations of their data and models, often

present results as if they are far more certain that they in fact are.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

27

It may be far more honest to say the percent of unsubmerged substrate is estimated, with 95% confidence,

to be between 26% and 34% than to say it is 30%. The latter value, though precise, is likely to imply an

accuracy/certainty, that does not likely exist in reality. It is not a weakness to be uncertain, it is essential honesty

in many cases. Acknowledging our uncertainty is the cornerstone of any risk and uncertainty analysis.

EDUCATION AND TRAINING

The first and most important tool for the risk and uncertainty toolbox is knowledge. Risk and

uncertainty-based analysis uses selected methods and tools that may or may not be known to Corps analysts. You

can’t do water resources planning without knowledge of the hydrologic cycle and you can’t do risk-based analysis

without some knowledge of probability, statistics, and the emerging literature on risk and uncertainty.

People are not born knowing these things and if Corps analysts are going to be expected to apply these

techniques they must first be taught them. The necessary knowledge can be obtained in a variety of ways. There

is a vast and growing literature treating probability, statistics and risk and uncertainty analysis. Incorporating

Risk and Uncertainty into Environmental Evaluation: An Annotated Bibliography (IWR Report 96-R-9 March

1996) provides an excellent introduction to some of this literature. These are topics that many people find

difficult to pick up on their own, however.

The standard college-level introduction to statistics courses will provide the rudimentary knowledge of

probability and statistics that is required. These courses frequently emphasize descriptive statistics, interval

estimation, and hypothesis testing while glossing over or ignoring distribution fitting and other topics more useful

to Corps risk analysts. Application-specific training such as the Corps currently provides in a number of areas,

including a risk analysis course, may be the most effective means of providing Corps personnel with the requisite

skills for conducting risk-based analyses. Once trained, Corps field personnel will be perfectly capable of

applying the tools and techniques, extending their usage and finding new and innovative applications in the field

of environmental planning. It’s not reasonable to expect that each Corps analyst will develop knowledge and

expertise in every aspect of risk and uncertainty analysis. The necessary expertise can frequently be acquired

through other people. This can mean consultation with other Corps experts, interpersonnel agreements with

agencies/organizations outside the Corps, or contracting for the services of a consultant. Regardless of the

preferred method for acquiring the necessary knowledge, acquiring it is an essential step in dealing with risk and

uncertainty analysis. 

EPA’S ECOLOGICAL RISK ANALYSIS PARADIGM

At this writing there is not yet a final ecological risk paradigm. It is under development. Nonetheless,

the development of this paradigm bears watching. The existing environmental risk assessment paradigm is

already finding useful applications where the concern is the effect of toxic substances on one or more species.

As noted earlier, the Corps dredged material disposal agreements with EPA make some use of that paradigm. 

The appeal of the ecological risk assessment paradigm is that it will employ a stress-response concept.

That is, substances or events that introduce stress to an ecosystem will be identified and the ecosystem’s response

to those stressors will be quantified. The reliance on stressors will make this paradigm much more amenable to

the evaluation of Corps projects because changes in stream flow velocity, disruption of sediment budgets, and
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

28

the many perturbations of the natural hydrologic cycle that can result from Corps projects do stress ecosystems.

The ecosystems orientation is also more appealing than a single species approach.

It is suggested that Corps analysts keep abreast of what EPA is doing in this area. Both the Corps and

EPA are struggling with many of the same issues of target species selection, extrapolation of analytical results,

habitat loss and higher level systems like ecosystems and the global environment.

FULL DOCUMENTATION

Full documentation of the risk and uncertainty analyses is very important, particularly during the early

stages when these techniques are first being applied. Describing the initial uncertainty, the steps taken to reduce

it, and the uncertainty that remained are essential steps in a full documentation. Describing the hazards,

probabilities and consequences of risky situations are equally important. 

A complete documentation is important to the related tasks of risk communication and peer review.

There are other reasons for documenting the analysis. Positive examples of how to document analysis are always

welcome. Although many analyses are case specific there may be components of the analysis or results that can

be used elsewhere.

A well-documented analysis aids the decision process. The report should not banish documentation of

the risk and uncertainty analysis to a brief section at the end of the report. If the problems are uncertain or risky,

the analysis should be presented when the problems are described. Descriptions of sampling results, subjectively

estimated data and model elements, rationales for assumptions and leaps of faith, justifications of decisions and

the like should be documented throughout the study report. In some cases it may be appropriate to summarize

the results of more formal risk and uncertainty analyses, for example, simulations or sensitivity analyses that

could affect the decision, in a separate section of the report. Careful documentation can be an effective tool for

the internal and external two-way flows of information and feedback that are so essential to a successful project.

JARGON AND LITERATURE

Risk has been described as a subset of uncertainty. Risk analysis, however, is a term in general usage

that includes the analysis of uncertain situations. Hence, we find the cumbersome situation in which the jargon

is quite confusing. Risk is a subset of uncertainty, but risk analysis includes uncertainty analysis? The way the

language is used, that is correct! Be forewarned, the language of risk and uncertainty can be uncertain. When

the distinctions are important it is best to take the time needed to clarify meanings precisely. Fortunately, there

will not likely be many situations in which the distinctions are that important.

There is a dichotomous nature to risk analysis that is important and that bears repeating. There are the

technical/scientific aspects of risk and uncertainty and there are the policy issues of risk and uncertainty. Often

these two worlds are not well-suited to mutual communication. The rational world of science is ill-prepared to

address the irrational fears and emotions of humans. Decision-makers are often unable to absorb the real meaning

of a technical analysis. Risk analysis is a process that includes the assessment and management of risk

information for an individual, group, society or environment. It is the combination and, hopefully, the synthesis

of these two tasks.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

29

One part of a risk analysis is the technical or scientific part. This is presumably an objective assessment

of the hazards, their probabilities and consequences handled by qualified experts. This is a detailed investigation

performed to understand the nature of unwanted, negative consequences. It’s an analytical process that provides

information regarding undesirable events by quantifying, to the extent possible, their probabilities and their

consequences. 

Once risks have been objectively assessed the limits of science have been reached and now social and

political values are required to decide what to do about the assessed risks. Risk management is the decisionmaking or management process used to protect human life, health, property or the environment from unwanted

adverse consequences. The management step of the risk analysis expressly weighs ethical, social and political

values.

The risk assessment may identify a 10% to 30% chance that beaver will become locally extinct in the

next 20 years. The risk management must determine whether or not that is an acceptable risk. If it is not, it must

decide what, if anything, to do about it. 

There is not a well-defined discipline called risk and uncertainty analysis. Risk analysis comprises a set

of tools and methods developed in and borrowed from a variety of disciplines. Nonetheless, this collection of

tools and methods have enough common elements that risk analysis has evolved into a specialty subfield of many

disciplines. Risk analysis has its own language, its own theory, and its own paradigms. 

Corps personnel doing risk analysis need to understand some of the basic language of this subfield,

because it is not always precise. Different authors and agencies use the same terms to convey different meanings.

Analysts need to be familiar with terms like expected utility, acceptable risk, risk assessment, environmental risk,

ecological risk, the EPA paradigm, expected values, partitioned risk, comparative risk analysis, risk-cost and

benefit-risk trade-offs, risk communication and the like.

There is a great deal of existing and emerging literature on risk analysis. This report refers the reader

to the following books as a good place to start. 

1) Managerial Decisions Under Uncertainty: An Introduction to the Analysis of DecisionMaking. By Bruce F. Baird. New York: John Wiley & Sons, 1989.

2) Making Hard Decisions: An Introduction to Decision Analysis. By Robert T. Clemen.

Boston, MA: PWS-Kent Publishing Company, 1990.

3) Readings in Risk. Edited by Theodore S. Glockman and Michael Gough.

Washington, D.C.: Resources for the Future, 1993.

4) Uncertainty: A Guide to Dealing with Uncertainty in Quantitative Risk and Policy

Analysis. By M. Granger Morgan and Max Henrion with a chapter by Mitchell Small.

New York and Cambridge: Cambridge University Press, 1992.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

30

5) Simulation and Modeling and Analysis. By Averil M. Law and W. David Kelton.

New York, NY: McGraw-Hill, Inc., 1991.

6) “Reducing Risk: Setting Priorities and Strategies for Environmental Protection.” By

Science Advisory Board (A-101), SAB-EC-90-021. The Report of the Science

Advisory Board: Relative Risk Reduction Strategies Committee to the U.S.

Environmental Protection Agency. Washington, D.C.: USEPA, September 25, 1990.

Basic awareness of the jargon and developments in the risk and uncertainty literature is essential.

PEER REVIEW

An independent review of any risk analysis can be an effective check on the integrity and credibility of

a risk analysis. Peer review of risk analyses is a central part of many of the risk analysis legislation initiatives

now before Congress. Legislative requirements aside, whenever risk analysis is an important part of a project

with potential outrage factors (see section on Risk Communication that follows) that could affect the viability

and/or performance of a plan, peer review can be an effective tool. This will be particularly so for studies with

international or significant intercultural effects. 

Consider a recent example. It had been proposed that the channel catfish be introduced to New Zealand

via fish farming, not in the wild. An environmental impact assessment considered the consequences of these fish

escaping to the wild and concluded the risks would be tolerable. Public concern resulted in a peer review that

disclosed documented cases of catfish escapes from farms to the wild. Its rapid spread to various habitats and

the resulting damage through predation and competition with indigenous species were enough to halt the

introduction of the species.

More points of view and fuller information are obvious benefits of a peer review. The environmental

impact statement (EIS) review by the public has been designed to assure these benefits. Corps studies have

always been subjected to a peer review of sorts in recent decades. That review has, traditionally, been an in-house

review by other agency personnel. As the Corps’ review process continues to evolve, independent peer review

as appropriate may prove a valuable feature of controversial risk and uncertainty analyses, especially in

environmental restoration projects. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

31

POST FACTO EVALUATION

Environmental restoration projects are relatively new in concept. There is much that is not known about

the environments’ we seek to restore and the methods by which this might be best accomplished. Monitoring

would ideally be part of every project design. There is a tremendous need for information about the results of

these projects. Were objectives achieved? Can errors in project design be corrected to avoid catastrophe?

One of the most effective ways to reduce uncertainty is through information. Monitoring and observing

the performance of projects after they are built and disseminating that information broadly is one of the most

efficient uses of project funds that could conceivably be made. Monitoring might include visual inspection,

mapping, bathymetry, aerial photographs, water quality monitoring, fish and other species monitoring, field

inspection, GIS mapping, and other methods. 

Lessons learned with environmental restoration projects will be worth their weight in gold. Newsletters,

conferences, seminars, training programs, world wide web site home pages, mailing lists, bulletin boards, news

groups, Internet relay chat (IRC), teleconferences, video cassettes, multi-media presentations, photo libraries, and

repository libraries for restoration projects are but some of the means by which valuable information can be

disseminated and uncertainty reduced. 

PUBLICITY

Publicity can be an important tool in risk and uncertainty analysis. Take something as basic as the recent

conditions at a project site as an example. A typical analysis most often involves either a literature search or a

comparison of old aerial photographs to the current condition to determine what changes have been taking place.

Publicity can be very helpful in uncovering the maximum possible information. There may be obscure local

journals or publications with valuable information of which Corps personnel are unaware. Personal histories from

long-time residents may be important sources of information. The mere act of publicizing the need for

information about a place where many interested people live can provide valuable information.

In the case of risky situations, publicity can provide an opportunity for all sides to come out and be heard.

Publicity can be provided through formal means like meetings, newsletters and the like or through informal means

like talk radio appearances, conversations with historical societies, community groups and the like. Personal

communications can be a very effective part of a publicity campaign to reduce uncertainty and deal with risk.

RISK COMMUNICATION

Communication is always a critical element in any planning activity. In its absence, lack of information

and uncertainty is the default condition. Risk communication, however, refers to something more specific. It is

communication about situations in which risk is important. That communication may be among team members,

between Federal and non-Federal partners, between risk assessors and risk evaluators, communication with the

media, or communication with the public. The planner needs to consider the four elements of the risk

communication process:

! Objective(s): Why is the communication being undertaken?
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

32

Figure 4: Outrage Factors

! Inadequate or unclear benefits

! Imposed rather than voluntary risks

! Outside personal control

! Unethical or unfair distribution of risk burden

! Managed by untrustworthy information sources

! Artificial rather than natural risks

! Insidious damages

! Unknown duration

! Unfamiliar

! Associated with memorable events

! Content of the message: What information is to be conveyed in order to accomplish the

objectives?

! Form of communication: How should the message be transmitted from the source to

the receiver?

! Feedback from the audience: What is being received?

There has been much debate about what defines an acceptable risk. In fact, it may be that risk is not

accepted or acceptable in the sense that it’s consciously balanced against a perceived good as much as it is

tolerated where necessary. Generally, people are risk adverse. Risks are perceived as intolerable when specific

“outrage factors” are present. Risks tend to be less tolerable if they include the characteristics shown in Figure

4.

Environmental projects can be

controversial and subject to strong

emotional reactions. Environmental risks

are subject to outrage factors. This makes

risk communication all the more important.

The Corps of Engineers is confronted with

past projects, now taken out of their

historical context by some interest groups

and stakeholders. Even when a restoration

project succeeds there will be some who

feel the Corps has failed by not doing more.

It is important that the perceptions

of these and other groups, particularly as

they concern the risks of a project, be

considered part of the planning process and

not a problem to be overcome. For example, if some people are concerned that a species may become locally

extinct, or that a project will fail to result in the habitat quality expected, these are not things that people should

be talked out of, i.e. problems to overcome. They are legitimate concerns of the public that need to be adequately

addressed in the investigation. The results of the analysis must, likewise, be effectively communicated. As a

general rule, when discussing uncertain situations it’s wise not to use untested communications. Take care when

discussing risk in the public sphere.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Available from Palisade Corporation at 1-800-432-7475. 8

 Available from SciTech International, Inc. at 312-486-9191. 9

33

SOFTWARE

Risk analysis software comes from four different sources. First, there is the commercial software

developed for sale in the retail market. These include such things as @RISK and Crystal Ball. Second, there are

special application proprietary tools and models developed privately or commercially for very specific

applications. These include efforts like HEC’s new risk-based expected annual damage estimating software.

Third, there is shareware that is available free on the Internet and through other on-line sources. Fourth, there

is the do-it-yourself software that are individual models, developed for specific studies, by individuals, that are

not intended for multiple uses. 

The first class of software tools has become quite large. They include Monte Carlo and Latin Hypercube

simulation modeling techniques (e.g., @RISK, Crystal Ball, FuzziCalc, Predict!), analytical hierarchy process

multi-criteria decision models (e.g., Expert Choice), decision tree and influence diagram packages (e.g., DPL,

DATA), visual spreadsheets (e.g., DS Lab), neural networks (Neuralyst, Braincel), systems dynamics (e.g.,

Powersim, ithink, Optima, Logical Decisions, Stella), distribution fitting (e.g., BestFit), linear and nonlinear

programming (e.g., What’s Best!, Evolver, Lindo) and traditional statistics packages (e.g., SPSS for Windows,

Statgraphics, eHOP). Two sources of information are industry catalogues produced for and by the makers of this

software: “Analytical PowerTools”, and “SciTech Software for Science”. 8 9

The second type of software has been developed for specific applications but has found a limited niche

market. At times this software has been developed for a government agency and its availability is subject to

agency regulation. In other cases the developer maintained control of the software and it is available for sale.

This occasionally includes adaptation for the buyer. This type of software is sometimes substantially more

expensive than software available on the retail market. 

Examples of this software include Systems Analysis Programs for Hands-On Integrated Reliability

Evaluations (SAPHIRE5.12), a fault tree package originally developed for the U.S. Nuclear Regulatory

Commission. Much of this software has been developed for the health and safety aspects of risk analysis. 

There is shareware and free software available from the Internet. These are of varying quality. One

example is the risk/exposure assessment model developed for the EPA by the University of Nebraska available

from the World Wide Web (at: http://eeyore.lv-hrc.nevada.edu/therdbase.html) at no cost. There are numerous

mailing lists and home page web sites detailing these opportunities. A great number of these are software for

processing graphical belief functions, e.g., BAYES, BELIEF 1.2, IDEAL, + MacEvidence, + Pulcinella, + SPI,

+ tresBel, and XBaies.

The final category of risk software includes models developed for specific project evaluations. These

are normally developed by individuals working for the Corps in order to meet the specific needs of a project.

Currently, there is no systematic cataloging of these models. Creating a Corps news group or mailing list where
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

34

such information can be exchanged would be a valuable early step in facilitating the spread of risk analysis

throughout the Corps’ environmental planning program. 

One particularly useful source of information about software and other risk-related topics is a risk

analysis mailing list maintained on the Internet by Batelle Corporation. The e-mail address for this mailing list

is riskanal@bbs.pnl.gov. This mailing list can put any risk analyst in instant contact with hundreds or thousands

of other risk analysts around the world.

SPECIFIC TOOLS

In addition to the broad concepts of the preceding section there are a great variety of specific tools used

by risk analysts. A basic familiarity with the existence of these methods and tools can be an invaluable aid to an

analyst grappling with the introduction of risk and uncertainty analysis to a new domain area. It is not necessary

to be an expert in all or even any of these techniques, as that expertise can be acquired by training or contract.

It is, however, important to have some awareness of the range of tools used by risk analysts. What follows is

another alphabetized list of selected tools in common usage among risk analysts. Several of them will be used

in the example in Chapter Six.

ANALYTICAL SOLUTIONS

Linear programming, integer programming, nonlinear programming, dynamic programming, calculus,

algebra, and game theory are some tools frequently used to solve problems. These tools yield analytical solutions

to problems. Analytical solutions are solutions that meet all the criteria of a problem specification. The analytical

solution to 2 + x = 4 is x = 2. Some problems have more than one analytical solution, for example, 2x + y = 10

has both (2,6) and (0,10) among its solution set. In more complex problems we are often interested in the

solution that optimizes a value. We may be interested in the analytical solution that makes net benefits as large

as possible or stream velocity as small as possible.

Some complex systems can be formulated as models that can be solved analytically. The goal of most

of those models is to determine an optimal solution through the use of algorithms. For example, linear

programming problems can be solved analytically. Not all problems can be represented adequately by such

models. Some systems are too complex and they defy analytical solutions.

Where analytical models represent reality, simulation simply imitates it. Simulation, discussed below,

imitates the operation of a real world system over time. It is a numerical technique that can be used to estimate

the analytical solution to a problem. Simulation is not an optimization technique. The answers obtained from

simulations are not analytical solutions. Solve the equation 2 + x = 4 a thousand times and the answer will always

be the same, x = 2. Simulations, on the other hand could yield many different answers to a question.

Analytical solutions are preferred whenever they can be obtained so long as the models from which they

are derived adequately represent the real world system being analyzed. Risk analysis is a new concept to many

Corps analysts. The first exposure for many people is a Monte Carlo process such as is used for the estimation

of expected annual damages for flood damage reduction studies. A limited exposure to risk analysis tools and

methods leads some to believe that risk analysis is synonymous with simulation. It is not.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 For more information about these criteria see “Guidelines for Risk and Uncertainty Analysis in Water Resources Planning Volume I 10

Principles with Technical Appendices, Appendix I”, March 1992, IWR Report 92-R-1.

35

Even when risk and uncertainty is involved, there is often an analytical solution to a problem. Suppose

you are a gambler and you face a game based on the toss of a fair coin. If it comes up heads you lose $20 and

if it comes up tails you win $10. Is this a game you want to play?

We don’t need a simulation or any other fancy tools and methods to answer this question. The expected

value of this game’s result can be calculated. It is 0.5(-$20) + 0.5($10) = -$5. On average you expect to lose

$5 per game if you play. Clearly it is not worth playing.

Many risky or uncertain situations are far more complex than this simple example. The important point,

however, is that some problems have solutions that can be determined through analytical solutions. “Analytical

Solutions” is included here to assure that Corps personnel do not make the mistake of thinking that simulation

and Monte Carlo processes are always the best approach to a problem. An analytical solution is always to be

preferred when one is available. Unfortunately, they are not often available or they are too complex to solve.

Another limitation of analytical solutions is that they often require simplifying assumptions in order to make them

solvable. These simplifying assumptions can often limit the usefulness of the resulting analytical solution.

DECISION CRITERIA

Under uncertainty there might be several different estimates of a value of interest. For example, a

sensitivity analysis could lead to three or more estimates of the number of habitat units (HU’s) that would result

from a project. If we elicit subjective probability estimates (see section on Expert Opinion) from three experts

we can expect three different answers. What do you do when you have multiple estimates of an important value?

How do you make a decision?

There are a number of well known decision criteria that can be used in decision-making under

uncertainty. At this point the Corps has not taken a position on any of these criteria, so presumably the criterion

that best suits the needs of the planning partners can be used. To facilitate the discussion assume three experts

have identified the most likely, the minimum, and maximum number of white tail deer that would remain in the

study area 50 years from now if no action is taken. These hypothetical estimates are presented in Table 3 and

are used to illustrate the use of the selected criteria that follow.10
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

36

Table 3: Hypothetical Estimates of White Tail Deer Population

Expert 1 Expert 2 Expert 3

Minimum 0 50 30

Most Likely 21 75 300

Maximum 45 100 350

Laplace Criterion

The probability of Expert 1's outcome being realized is unknown, as is the probability of any other

expert's outcome being realized. When the probabilities of the possible outcomes (i.e., the “partial probabilities”)

are not known, equal probabilities can be assigned to all the outcomes. The Laplace criterion assigns each of the

outcomes an equal probability. With three estimates of the most likely population each outcome receives a weight

of a. The result of the Laplace criterion is an estimated population of a(21) + a(75) + a(300) or 132 deer.

This is a generally accepted default criterion.

Maximin Criterion

The maximin criterion yields the “best” of the worst potential outcomes. The worst outcomes are the

minimum population estimates. The best of the worst outcomes is the one that has the largest minimum

population estimate, in this case it is Expert 2's estimate of 50 deer. This emphasizes the worst case set of

outcomes but takes an optimistic view of these outcomes.

Minimin Criterion

Under this criterion we choose the smallest of the minimum values. It is the most pessimistic estimate

of all the population estimates offered here. The so-called “minimin” criterion is zero deer. This is the most

severe criterion and may be favored when the worst case outcome is unacceptable.

Maximax Criterion

This is the opposite of the minimin criterion. It considers only the maximum population estimates and

it uses the highest of these estimates. The maximax criteria yields a population estimate of 350 deer. This

criterion is often preferred by those who want to stress the most optimistic outcome.

Minimax Criterion

The minimax criterion selects the estimate that is the lowest of all the maximum estimates. In this case,

Expert 1's maximum estimate of 45 deer is the most pessimistic of all maximum population estimates.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

37

Hurwicz Criterion

The maximin and maximax criteria are frequently used when alternative actions are compared. The

maximin (best of the worst) and maximax (best of the best) criteria represent pessimistic and optimistic

outcomes. The Hurwicz criterion represents a compromise between these two criteria. If we define a coefficient

of optimism as " that ranges from 0 to 1, then 1-" is a coefficient of pessimism. By weighting the maximum and

minimum outcomes by the coefficients of optimism and pessimism, respectively, we can obtain results such as

the following: Let "=0.7 reflect a view that is more optimistic than pessimistic. The Hurwicz Criterion yields

0.3(50) + 0.7(350) = 260 deer.

Other Criteria

There are other criteria. The minimum, most likely, and maximum values can be combined with any

weights that reflect the decision-makers belief in the likelihood of the respective outcomes. The dominance and

regret criteria are two other common criteria found in the literature. There are additional criteria that are more

complex. 

The example presented here has been kept simple to make it easy to follow. The true appeal of these

criteria gets lost in the simplicity of the example however. These criteria are most useful when we have multiple

estimates of values for numerous alternatives. Imagine, if you will, a set of estimates of deer population for say

eight alternative plans. In cases like this, these criteria become more useful decision criteria. 

DECISION TREES AND INFLUENCE DIAGRAMS

There are many ways to represent a decision problem. Two of them are decision trees, sometimes called

event trees, and influence diagrams. An event tree or decision tree is a modeling approach used to structure a

decision problem. Construction of the tree helps to focus the analysts' attention on those elements of the decision

problem that are most important. The completed event tree provides a blueprint for construction of the computer

model needed to analyze the problem. 

The event tree is not intended to capture every element of the decision problem. Its purpose is to capture

the most important elements of the problem so they can be addressed in the analysis. Thus, the goal in developing

a model is to make the analysis as simple as possible, but no simpler. Let’s illustrate this with a simple example

of an event tree for a gated spillway that when opened can cause manatees to be sucked through the gates under

certain conditions. Suppose changes in the maximum allowable gate opening (MAGO) rules are being

investigated. 

Figure 5 shows the basic structure of the event tree used to model the situation. The analysis begins with

a random discharge event that produces a head differential in excess of that which can be handled by the normal

spillway opening. The “bell curves” indicate a chance event. If the gate is not opened there will be no manatee

casualties. There may be flood damages but that is not the concern of this particular analysis. If the gate is

opened there is some chance that manatees will be affected. If they are not there will be no
Figure 5: Sample Event Tree

Year's 

Discharge Events

Do not open

structure

Open

structure

No manatees

affected

Manatees

affected

No casualties

No casualties

Injury

Fatality

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

38

casualties. If they are affected there could be injuries or death to one or more animals. There is uncertainty

associated with each branching in the event tree.

By assigning probability distributions to the uncertain values and constants to the known values a

simulation model can be built to estimate the expected casualties or the probability of a casualty given the size

of a discharge event. A similar model could be built for conditions with the new MAGO rules and if reductions

in manatee casualties result these project outputs can be taken into account in the decision process.

Decision/event trees can be a valuable tool for helping the analyst think through the major elements of a problem,

even if a model is not subsequently built and run.

An influence diagram is used to represent the decisions, uncertainties, values and relationships among

these elements in a decision problem. It is usually less detailed than a decision tree. Figure 6 shows an influence

diagram for the above problem. The decision process begins with an uncertain flow (ovals represent chance

values) that effects the structure’s operation. Structure operation influences manatee casualties. The basic

elements of the problem are here and the influences are clear. There are no feedback loops or other values in this

simple model.
Figure 6: Influence Diagram

Flow

Operation of

Structure

Manatee

Casualties

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

39

EXPERT OPINION

When uncertainty results because things are unknowable there are few options. You either ignore the

uncertainty or you make some best guesses about its nature. When analysis comes down to guessing, we at least

want the best guessers available. These are usually experts in the domain of that which we are guessing. Expert

opinion is often the best option for estimating that which is unknown or unknowable.

Professional Judgment

There are a lot of variables, relationships, and situations encountered in environmental infrastructure

planning that are not only uncertain but that are either unknown or unknowable. How does an ecosystem

function? Will a species become locally extinct? When will the next 100-year flow occur? Where do you get

information about those things that are unknown or unknowable? From experts. It is the professional judgment

of the best informed people on these subjects that we generally must rely on.

Professional judgment has always been an important part of the Corps’ decision process. Often

professional judgment has been a mask for that blind leap of faith that must be made in many investigations.

When data, information, theory and analysis have taken you as far as they can, it’s time to drag out the hunches

and gut feelings, dressing them up as professional judgment. To dispense with professional judgment in such

a flippant way would be both extremely unfair and wrong. The knowledge gained from years of experience and

observation have imbued people with an inherent understanding of systems and situations that is not easily

reduced to a checklist, theory, or standard operating procedures. Professional judgment is one of the most

important tools available in a risk-based analysis.

The essential difficulty with professional judgment often lies in the identification of the experts whose

judgments will be sought. There is considerable professional literature that suggests that so-called experts often

do not know the limits of their own knowledge and expertise. It is common procedure for experts to overstate

their confidence. In other words, the experts aren’t right as often as they think they are.

It is important to systematically search for the expertise that is required. The most knowledgeable person

in the office may not be sufficiently expert on a subject to provide a full range of values or impacts that could

result from an uncertain situation. The consultants with whom you have a contract may be willing to do the work,

but they may lack true expertise in an area. It’s a good general rule to search for and verify the expertise of your

experts. Using multiple experts can help assure a more complete picture of the uncertainty involved in a situation.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

40

Elicitation/Opinion Analysis

Experts have considerable knowledge. Obtaining it from them in a useful and reliable manner is not

always as easy as it might seem. Suppose for example there is concern that some species of interest will become

locally extinct. Take something as simple as the beaver. How would you go about getting an expert to estimate

this event? How do we define local? Extinct? Over what time frame? 

The process of discovering and quantifying the knowledge and opinions of experts so they can be

communicated to others is called elicitation or opinion analysis. In risk-based analysis both the likelihood and

the consequences of uncertain events can be unknown or unknowable. Environmental planning, because of its

complexity and its relative newness, is full of unknown and unknowable events and situations. Eliciting

information from experts will likely be an extremely important analytical method for risk analysts involved in

this area.

Subjective Probability Assessment

Estimating the likelihood of events is a special subfield of elicitation that has received considerable

attention in the human behavior and decision-making literature. Subjective probability assessments are

techniques used to elicit from experts the probabilities of various events. Subjective probabilities (see section

on Probability) are not based on analytical or frequency-based probability estimates. They are opinions based

on the expert’s strength of belief. 

Experts assess and evaluate the available evidence and then use their professional judgment to estimate

the likelihood of various events. These likelihoods can be point estimates (e.g., there is a 0.2 chance beaver will

be extinct within 30 years) or estimates of entire distributions (there is a 0.9 chance that there will be 100 or less

beavers per acre, a 0.6 chance there will be 45 or fewer beavers, a 0.2 chance of 15 or fewer beavers and 0.01

chance of 3 or fewer beavers). There is no right or wrong answer, only opinions that are more or less accurate.

Analytic Hierarchy Process

The analytic hierarchy process (AHP) was initially developed as a multi-criteria decision tool.

Alternatives are chosen on the basis of their contributions to various objectives. AHP is a tool that can be used

to make decisions in situations involving multiple objectives. Consider a decision problem like that shown in

Figure 7 where we have a goal (the top square), criteria for reaching that goal (the second row of squares), and

alternatives for meeting each criterion (the tic marks beneath the criteria). The number of levels is purely

arbitrary with this technique; there can be as many levels of sub-criteria as desired. 

Using pairwise comparisons, first of the criteria (and sub-criteria), then of the alternatives for reaching

each criterion, this technique produces index numbers for each alternative plan that reflect the
Alternative 1 Alternative 1 Alternative 1 Alternative 1

Alternative 2

Alternative 4

Alternative 2

Alternative 3

Alternative 4

Alternative 2

Alternative 3

Alternative 4

Alternative 2

Alternative 3

Alternative 4

Goal

Food Cover Disturbance Reproduce

Figure 7: Sample AHP Decision Problem

Alternative 3

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

41

collective judgments of the decision-maker/expert. These index numbers identify the best alternative consistent

with the experts judgment. 

The pairwise comparisons proceed at each level of the hierarchical structure. For example, food source

is compared first to cover, then food source is compared to the disturbance factor, and finally it is compared to

reproduction potential. At each comparison the expert may assign a numerical or qualitative rating of the two

factors, e.g., food = 1, cover = 3 or cover is moderately more important than food. Each criterion is eventually

compared to every other criterion. Then each alternative is compared to every other alternative for each criterion.

For example, with regard to food sources alternative 1 is strongly more important than alternative 2. With each

such judgment the index number is constructed reflecting the expert’s relative weighting of each criteria and

alternative. 

This technique can be readily adapted to situations involving uncertainty. In this case, the weights of subcriteria and criteria are not the relative importance of these items but their relative likelihoods. AHP has proven

to be a useful tool in quantifying the subjective judgments of experts in a variety of decisions under uncertainty.

One of the more recent applications of this technique was the development of structure condition indices for the

80 largest locks and dams in the Corps’ inland waterway system.

PROBABILITY

Many situations of uncertainty are represented by probabilities. An event such as a drought or extreme

flow, for example, may or may not happen. What is the probability it will occur this year? A sample survey may

indicate an indigenous population of white tail deer of 850, with 95% certainty the true
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

42

Three Approaches to Determining A Probability

The three approaches to determining a

probability are classical probability, empirical

probability, and subjective probability. In classical

probabilities all outcomes are equally likely. Examples

include the probability of rolling a six with a die, or

drawing a king from a deck of cards. Empirical

probabilities are based on past experience. Most

hydrology is based on empirical probabilities, i.e., the

observed frequencies of events. Probabilities based on

available evidence and an individual’s own judgment

are called subjective probabilities. 

population is between 700 and 1,000 deer. We might

be interested in the likelihood that beavers will

become locally extinct in the absence of a project.

Experts would be relied upon to estimate this

probability subjectively. There are endless possible

uses for probabilities in environmental planning.

Corps personnel doing risk analysis have to

be extremely comfortable with the basic notions of

probability and probability distributions. They need

to understand the differences among analytical,

empirical/frequentist, and subjective notions of

probabilities. The laws and language of probability

are minimum requirements for risk analysts.

A basic knowledge of the most common discrete and continuous distributions and when to use each is

essential. This knowledge should include a knowledge of the distribution parameters, their shapes and locations,

as well as distribution-fitting methods. Analysts should be able to move back and forth between probability

density functions and the cumulative distribution functions. 

Armed with these skills, Corps personnel will have the probability background necessary for

understanding the chance dimension of uncertain situations. Furthermore, this knowledge is a necessary precursor

to the use of more sophisticated models, methods, and techniques.

PROBABILITY DISTRIBUTIONS

When the Corps first began to incorporate risk analysis into its planning studies a fundamental goal was

to get analysts to represent the value of critical variables as distributions rather than as determinant single values.

Issues of which distribution to use were clearly secondary to taking the first step and using a distribution. Now

that risk analysis is gaining more widespread acceptance analysts readily embrace the use of probability

distributions. The question has now become, which distribution should I use? For example, what distribution

should we use to represent the uncertainty attending the HSI index? 

Suppose we could measure every striped bass in the bay. What would a graph of these data look like?

A histogram is a bar graph that depicts the frequency with which values occur in a data set. Figure 8 presents

a histogram of fish sizes that range from about 2 cm to over 180 cm. Most of the fish fall in the range of 90 cm.

Each bar is 10 cm wide. If each bar were only 1 cm wide the graph would take on the smooth look of a normal

distribution like the one shown in Figure 9. Histograms are pictures of frequency distributions. They show how

values in a data set are distributed. 
Figure 8: Striped Bass Histogram

0 20 40 60 80 100 120 140 160 180

0

20

40

60

80

100

120

140

Number of Fish

Size (cm)

Figure 9: Probability Distribution of Fish Lengths

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

43

Histograms need not have the neat symmetry of Figure 9. They can assume any shape at all. A histogram is a

picture of how the elements of a data set are distributed. A probability distribution is a picture of how all the

elements of a population is distributed. The distinction between the two is that a histogram is usually a picture

of some of the data and a probability distribution is a picture of all the data. Because having “all the data” is

often a practical impossibility, probability distributions are often theoretical rather than empirically observed the
f(x)'

1

(2B)

.5F

e

(x&μ)

2

2F

2

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Although the normal distribution is unique in that any normal distribution can be converted to the standard normal distribution, also 11

known as the Z distribution.

44

way a histogram is.

For example, the histogram above was presented as a picture of the length of all the fish in the bay.

Analysts never have that kind of information. The histogram above would more likely be a picture of the

distribution of lengths for 1,000 fish sampled from the bay. Nonetheless there is a theoretical distribution of the

length of all striped bass living, dead or yet to be born and it’s called the probability distribution. That

distribution yields information like the probability that a fish will be 76.4 cm or more in length and that

probability can be estimated from the theoretical probability distribution of fish lengths. 

The histogram yields information about the frequency with which fish 76.4 cm or longer are observed

in the sample of 1,000 fish. As the sample gets bigger and bigger the frequency with which fish 76.4 cm or more

are observed gets closer and closer to the true probability with which that occurs. Frequency distributions or

histograms, then, are approximations of probability distributions.

Analysts are interested in more than fish lengths. They may be interested in the distribution of stream

discharges, tidal fluctuations, salinities, weights of animals, heights of plants, depths of water, speed of winds,

and so on. Each of these characteristics has a parent population that has some probability distribution.

Experience and theory have revealed that these distributions tend to assume certain basic shapes; or, at least, they

have certain basic characteristics. 

The shapes of many of these distributions can be described by a mathematical formula. The normal

distribution, for example, is described by the following formula:

where f(x) is the probability density function, what we have been calling the probability distribution. The values

of B and e are known. Unknown are the values of μ and F. These are the parameters or constants which, once

known, give the distribution its location along a real number line and its shape above that line. The population

mean is μ, the Greek mu, and F, sigma, is the standard deviation of the population. Hence, given the population

mean and standard deviation any normal distribution can be described. The normal distribution in Figure 9 is

centered about a mean of 90 and its spread is dictated by its standard deviation of 30.

Other distribution shapes are described by other formulas. Frequently, the formulas describe families

of distributions. That is, when the parameters of the distribution change the shape and location of the distribution

can change as well. For example, there is not one normal distribution but an entire family of distributions. Even 11

though each has that distinctive single-peaked symmetrical bell shape each distribution can vary in its location
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 RISKview is available from Palisades Corporation and eHOP, The Electronic Handbook of Probability, is available from the Crunch 12

Software Corporation at 415-584-0122.

45

on the normal line and the extent to which it is tall and thin or short and fat. Other families of distributions can

vary in far more dramatic ways.

What are we to assume about the nature of the probability distributions that are used to describe the

uncertainty in, for example, environmental restoration studies? If empirical data are available, visual assessments

of the frequency distribution may be used to approximate the underlying probability distribution. Statistical

distribution fitting tests using goodness-of-fit measures can be used to more effectively test how well the data

conform to any given distribution. The chi-square and Kolmogorov-Smirnov statistics are goodness-of-fit

measures most often used to determine if one’s data are consistent with a normal distribution or any other

distribution. These tests are available from a number of commercially available statistics packages. BestFit from

Palisades Corporation is especially designed for distribution fitting tests. 

When there are no empirical data to examine, theory or understanding the process that determines the

data of interest may suggest the distribution to use. Prior studies and the literature may offer insights into the

nature of the distributions that are appropriate to use. The books A Guide to Probability Theory and Application

by C. Derman, L. Glaser and I. Olkin (1973) and Uncertainty: A Guide to Dealing with Uncertainty in

Quantitative Risk and Policy Analysis by M. Granger Morgan and Max Henrion (1990) are two sources of such

insight. The users’ manuals for @RISK and Crystal Ball are two examples of commercial software products that

provide some useful information about probability distributions.

In the absence of better information, a systematic process of selecting a distribution may be the best

approach in many circumstances. Following these few simple steps may help the analyst make a reasonable

choice of a distribution for representing the uncertainty when there is no better guidance on distribution choice.

First, clearly identify the variable of interest. Second, list everything you know about the variable being sure to

include quantitative and qualitative aspects of the variable. Third, identify the type(s) of uncertainty with which

you are dealing. Fourth, review the approaches to the different types of uncertainty. Fifth, list the description

and characteristics of the candidate probability distributions. Sixth, identify the distribution(s) that best

characterizes the variable in question. If none of these methods identify a single best distribution use sensitivity

analysis to determine the significance of choosing one distribution over another.

The remainder of this section attempts to acquaint the reader with some of the most commonly

encountered families of distributions used by risk analysts. It is impossible to do justice to this subject matter

in a few sentences. Undoubtedly, it would be interesting to see the formulas, discuss the parameters, and look

at sample distributions. The subject matter is too complex to do that here. Instead the briefest of introductions

to the distributions is offered. Reasonably accessible additional detail can be obtained from the books cited

above. A more concise and mathematical description can be found in Statistical Distributions by Merran Evans,

Nicholas Hastings, and Brian Peacock or in many advanced statistics texts. In addition, there are several software

packages that can be used to explore these distributions. RISKview and eHOP are two packages that allow the 12

user to explore the characteristics of a variety of distributions. 
Figure 10: Truncated Normal Distribution

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

46

Normal Distribution

The normal distribution is probably the most common distribution encountered. Variables formed by

the addition of other uncertain quantities are, generally, normally distributed. A total cost that is the sum of

several uncertain line item costs will tend to be normally distributed. Uncertainty that results from unbiased

measurement errors can be represented by a normal distribution. Thus, if the chief concern about the size of a

local population of a species is the potential measurement error, the uncertainty may be represented by a normal

distribution. Many other distributions approach or reduce to a normal distribution under circumstances (i.e., for

certain parameter distributions) that vary from distribution-to-distribution. The Central Limit Theorem

contributes to the utility of the normal distribution in assuring that the distribution of some sample statistics is

always normal, regardless of the parent distribution from which the sample was drawn, as long as the sample size

is large enough.

The normal distribution can be described by two parameters, the mean and standard deviation. For

quantities where negative values are not meaningful (populations, costs, flows) it may be inappropriate to use

the normal distribution unless it is truncated to eliminate the illogical values. Distributions can be truncated at

any values required by the analysis. Figure 10 shows Figure 9 truncated at values of 20 and 150.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 A purely random process of events is called a Poisson process. The exponential, Poisson, and gamma distributions deal with Poisson 13

processes.

 Suppose the average time between customers is ten minutes. The parameter is 0.1, and 0.1 is the number of customers per minute. 14

47

Lognormal Distribution

If the logarithm of the random variable of interest is normally distributed the random variable has a

lognormal distribution. Variables formed by the multiplication of other uncertain quantities tend to have a

lognormal distribution because the sum of the logs approaches the normal distribution. For example, an uncertain

area estimate times an uncertain habitat suitability index will yield a lognormal distributed estimate of the number

of habitat units.

If we know a value within an order-of-magnitude, say a factor of two or ten and when uncertainties are

large, the lognormal is appropriate to use. The lognormal distribution cannot be used with non-positive values

so it’s useful when negative values of the variable of interest are not permitted. It is a useful distribution to use

when values are positively skewed, i.e., most values occur near the minimum value. The distribution is described

by the mean of the logs of the random variable and the standard deviation of the log of the variable.

Exponential Distribution

The time between events that are purely random often has an exponential distribution as does the size 13

of random events. The exponential distribution can be used to describe the duration and size of events, like

storms and oil spills. It is commonly used in reliability theory, waiting times and queuing theory. It is described

by a single parameter, 8, that is equal to one divided by the average time between events. Thus, the single

parameter is the rate at which the events occur .

14

The exponential distribution has a mode of zero and is constrained to non-negative numbers. Because

the distribution has a fixed shape the single parameter determines only the scale of the distribution. It is not a

very flexible distribution and is useful for a very limited type of phenomena.

Poisson Distribution

Unlike the continuous distributions above, the Poisson distribution is a discrete probability distribution

that is constrained to non-negative numbers. The Poisson is useful when we're dealing with the number of

occurrences of an event over a specific interval of time or space. For example, the number of plants per acre or

the number of floods per 100 years may have Poisson distributions. If a single trial can be defined as some fixed

area of opportunity, like a fixed length of time (minute, day, year, planning period) or space (per foot, acre, cubic

meter) the Poisson may be useful. The number of events that occur in the fixed time period or space may have

a Poisson distribution.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Alpha, ", is defined as the average number of events per time (or space) interval, 8, times the length (size) of the time (space) interval, 15

T. Thus we define, " = 8T.

 The additional parameters translate the distribution and change its scale, so it is possible to model variation over any scale. 16

48

The single parameter of the distribution, ", is the average number of events per area of opportunity, e.g.,

the average number of floods per 100 years or the average number of plants per acre. Like other single

15

parameter distributions, the Poisson lacks flexibility. When the parameter is large the discrete Poisson

distribution may be approximated by a continuous normal distribution.

Gamma Distribution

Poisson processes confined to non-negative values that, for various reasons, yield distributions skewed

to the right can be described by a gamma distribution. The bulk of the area under a gamma density function is

located near the origin and the density function area drops gradually as the random variable increases in value.

The gamma distribution describes the time required for k events to occur. 

The gamma distribution is a two parameter distribution that describes a Poisson process. The shape of

the distribution is determined by k, while 8 is the scale parameter. This distribution is similar to the lognormal

but it is less positively skewed and less tail-heavy. Thus, it may be used in situations similar to those appropriate

to the lognormal. The gamma is useful for describing times (or spaces) between events (items) that are not pure

random processes. Some quantities frequently exhibiting skewed distributions like the gamma include physical

quantities such as precipitation quantities and pollutant concentrations as well as the time between malfunctions

of aircraft engines, time between customers at a checkout counter, and the time required to complete maintenance

on an engine. 

Weibull Distribution

The Weibull distribution is actually a family of distributions that can assume the properties of several

other distributions. With location, scale, and shape parameters the Weibull can resemble the exponential and

normal distributions as well as assume a number of other shapes.

The performance of many complex systems depends upon the reliability of the components making up

the system, perhaps an ecosystem. The time to failure or length of life of a component from a specified time to

its failure can often be described by a Weibull distribution. It has also been used to describe some physical

quantities like wind speeds. In its more common form it is similar in shape to the gamma distribution though it

is less skewed and tail heavy. Use it only with non-negative values. 

Beta Distribution

The beta distribution yields a two parameter distribution that is defined over the interval from zero to

one. There is a four-parameter form of the beta distribution where the additional parameters define the range

endpoints. It is a very flexible distribution for representing variability over a fixed range. Defining the way a 16
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 If the bus always comes between 7:00 and 7:10, the probability that it will come between 7:05 and 7:07 is exactly the same as the 17

probability that it will come between 7:02 and 7:04.

 For a random variable X create Y = lnX. The loguniform is a uniform distribution of Y. 18

49

variable varies over a range makes the beta a valuable distribution for modeling proportions, fractions and

percentages. It can be used to represent the probability of occurrence of an event or the proportion of time that

a structure or system functions as designed.

Uniform Distribution

The simplest of all probability distributions is the uniform distribution, where the random variable

assumes all its values with equal probability. The probability that a value from a given sub-interval will be

realized only depends on the size of the sub-interval relative to the overall interval. 

17

Use the uniform distribution when you can identify a range of possible values but are unable to say which

values in that range are more likely to occur. In other words, the uniform can be used when any one value is as

likely as any other. It is a two-parameter distribution where the parameters are the minimum and maximum

values that can occur. Subjective reasoning can often be used to estimate the parameters. When the uncertainties

are large and somewhat asymmetric you may use a loguniform distribution to make them more orderly.18

Triangular Distribution

Most likely values are frequently identified in the planning process. If it is believed that values close to

the most likely value will occur more often than values near the extreme ends of the range of possible values a

triangular distribution may be used. The arbitrary shape of the triangular distribution and its sharp corners

telegraph the fact that the precise nature of the distribution are unknown. A default distribution of sorts, the

triangular should not be used too quickly or too frequently. Its three parameters are the most likely, minimum

and maximum values the variable can assume. If the uncertainties are large, a logtriangular distribution can be

used.

Binomial Distribution

The binomial distribution gives the probability of obtaining x events, each with probability p of

occurring, from n independent trials where each trial has only two outcomes, frequently called success and failure.

We might want to know the probability of observing two or more terns with toxic levels of cadmium from a group

of 100 terns when the probability that a tern has a toxic level is 0.005. Its parameters are the number of events,

x, and the probability of the event occurring on a given trial, p. Use the binomial distribution when the

probability of success, p, or failure, 1 - p, are known. Do not use the binomial distribution when the event

probability is unknown.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

50

Discrete Distribution

A discrete distribution allows for a finite number of discrete outcomes. Each possible outcome is

assigned a probability. The probabilities must be mutually exclusive and sum to one. For example, when

describing the performance of a tidal wetland restoration project we would identify three discrete outcomes: 1)

tidal wetlands, 2) mudflats, and 3) daisies. We might estimate the probabilities of these events as 0.6, 0.2, and

0.2, respectively. 

Summary of Distribution Usage

The proper distribution to use is an empirical issue. Ideally, we'd like to fit a distribution to some

existing data to determine the distribution to use. Lacking data, theory and similar studies can be used for

suggestions. Undoubtedly, this will frustrate many people because it doesn’t provide specific guidance on which

distributions to use and when to use them. The truth of the matter is there is much yet to be learned about the

uncertainty confronted in environmental infrastructure studies. Table 4 summarizes the distributions described

above.

RANDOM SAMPLING

Information from a sample used to make generalizations about the population from which it came must

be representative of that population. In order for a sample to be representative of the population, the elements

of the sample should be randomly selected from the population. The basic idea here is that if every member of

the population has the same possibility of being selected as part of the sample and if the sample size is large

enough, pure chance will generally assure a representative sample. 

Simple random sampling is the most common method for obtaining such a sample. Stratified random

sampling, cluster sampling, two-stage cluster sampling, sequential sampling, and systematic sampling are

sampling techniques with which those in environmental planning should be familiar. Quadrat sampling involves

estimating the number or percentage of elements (deer, fish, trees, submerged substrate, people, etc.) in a defined

area or quadrat (per acre, per hectare). Quadrat sampling is useful in estimating population densities and size.

It’s not practical to expect all Corps analysts to be expert in sampling techniques. The techniques are

quite straightforward and not at all difficult to implement. Many analysts will be able to master them with modest

training and support. Inevitably, complex sampling situations will arise in which it will be sufficient for Corps

analysts to be familiar with the existence of other tools and methods so they can supervise and review the work

of others.

SENSITIVITY ANALYSIS

Thisis one of the more practical methods for introducing risk and uncertainty analysis to environmental

restoration projects. Completing an analysis of an uncertain situation requires numerous assumptions. It might

be important to know if the results of the analysis are sensitive to the assumptions that have been made. An

investigation that is intended to determine the sensitivity of the analysis’ results to different assumptions is called
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

51

a sensitivity analysis. It is also called parametric variation. A number or parameter that is treated like a constant

in the analysis is varied systematically over a range of values to examine its impact on the decision. 

Table 4: Distribution Summary

Normal distribution C Variable has a most likely value (mean) 

C Uncertain value as likely above as below mean

C Uncertain value near mean more likely than far away

Lognormal distribution C Variable is non-negative 

C Uncertain value more likely near lower limit 

C Natural log of variable yields normal distribution

Exponential distribution C Describes duration and size of events

C Variable is non-negative

Poisson distribution C Single parameter discrete distribution confined to non-negative values

C Number of events over interval of time or space

Gamma distribution CTime/space between events/items

C Two parameter distribution of non-negative variable

Weibull distribution C Three parameter distribution of non-negative variable

C Can resemble many other distributions

C Time-to-failure, life of component

Beta distribution C Two or four parameter distribution

C Defines variation over a range

CProportions, fractions, and percentages 

Uniform distribution C Minimum value is fixed

C Maximum value is fixed

C All values between min and max are equally likely

Triangular distribution C Minimum value is fixed

C Maximum value is fixed

C Most likely value falls between min and max, with values near min and max less 

 likely than values near most likely

Binomial distribution C Only two outcomes possible

C Trials are independent

C Probability of “x” events in “n” tries given probability on any trial is “p”

Discrete distribution C Finite integer number of outcomes possible

C Each outcome has independent probability of occurrence

Consider a very simple sensitivity analysis/parametric variation. Suppose you have 100 acres of land

and you estimate the HSI to be 0.5. There are 50 HU’s in this example. But, suppose the HSI, normally treated
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 In some instances it may be appropriate to treat these critical variables in a probabilistic fashion. In others, they will be varied in a 19

systematic fashion. The determining factors are the types of quantities that are uncertain and the nature of the uncertainty. See Chapter Four for

additional discussion of this point. 

52

like a constant, was a matter of professional judgment that was uncertain. Suppose the HSI could be 0.75. Then

we have 75 HU’s. The number of HU’s is very sensitive to the HSI estimate. The HSI was the parameter in this

example that was allowed to vary.

The purpose of a sensitivity analysis is at least two-fold. Sensitivity analyses conducted early in the

investigation can help you identify the key parameters/variables/factors. If you vary one of these values and the

result changes significantly, you know you have identified an important source of uncertainty or a key parameter.

In this case you do one of two things. First, you should try to reduce the uncertainty if at all possible. Second,

if the uncertainty cannot be reduced it must be addressed and its potential importance in the decision-making 19

process to the decision-makers needs to be explained.

A second purpose of sensitivity analysis may be to examine the cumulative effects of various uncertain

factors. For example, suppose the area estimate and the HSI are both uncertain. Though neither alone may have

a significant impact on the HU’s, the two together might.

Sensitivity analysis is a controlled and deliberate process. Allowing things to vary randomly over a range

of values, as one does in a Monte Carlo process, is not sensitivity analysis. It would not be proper, for example,

to do a simulation in which the period of analysis is allowed to vary according to some probability distribution.

We may be unsure how long a project will last but that is not a value one allows to vary randomly. The proper

approach is to vary that parameter systematically. Consider the plan impacts at 25, 30, 50 or whatever number

of years is reasonable and see if the decision would vary with the period of analysis.

Sensitivity analysis is a valuable tool. It can be a reasonable compromise between a detailed and

sophisticated risk-based analysis and completely ignoring the uncertainty that exists in your investigations.

Identifying the important variables in an investigation and varying the key parameters and noting the effect on

the best decision can be an effective way to begin to address risk and uncertainty in environmental projects.

SIMULATIONS

Simulation is a technique used to answer what-if types of questions. It is used most appropriately when

a decision problem under analysis is too complex to be solved by analytical models. Simulation is a quantitative

procedure that describes a process by constructing a model of the process and then observing how the model

behaves over a series of iterations in order to learn how the process itself might behave. 

The simulations that Corps personnel use are most often mathematical models that cannot be solved

analytically so they are run on trial data to simulate the behavior of a system. As a general rule, simulation should

be used only as a last resort and never as the first option. There may be theory or analytical techniques available

to solve a problem. When there are they should be used.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

53

Simulation can be useful in situations that are commonly encountered in environmental planning studies,

however. Some of the reasons for using simulation include the following:

1) Simulation is the only method available because the actual environment is too difficult

to observe. (The fate of a contaminant introduced to a new environment may be

impossible to track.)

2) It is not possible to develop an analytical solution.

3) Actual observation of a system is too expensive. (Planting tidal wetlands with different

numbers and densities of plants to determine the optimal scheme may be too expensive

to be feasible.)

4) There is insufficient time to allow the system to operate extensively. (We cannot wait

to see the results of a study of long-term trends in the population of a certain species.)

5) Actual operation and observation of a system is too disruptive. (If we would like to

know the impacts of a hiatus from maintenance dredging on submerged aquatic

vegetation (SAV) it is not feasible to actually cease dredging.)

6) Simulation reveals new facts about the problem. (Building a model requires careful

and logical thought about the process under investigation.) 

7) Simulation can be an effective training tool. (It allows controlled experimentation with

a process.)

8) Simulation allows testing of a greater range of situations than have

been observed.

If you use simulation in place of another technique there are trade-offs. Analysts should bear in mind

some of the disadvantages of simulations. A simulation is not an optimization process. It is not precise because

you do not get an “answer”. What you get from a simulation is a set (or distribution) of a system’s responses to

different operating conditions.

Suppose, for example, we have a simulation model that estimates project benefits and project costs in

order to estimate the project’s net benefits. Though we might examine 10,000 different sets of operating

conditions and estimate the resulting costs 10,000 times, the resulting benefits 10,000 times, and the estimated

net benefits 10,000 times we will not know what the actual net benefits are. We might know the average net

benefits for the 10,000 iterations and we might know the minimum and maximum observed values, but we will

not have a precise and accurate answer.

Simulation models can take a long time to build and they can be very expensive. Not all situations can

be evaluated using simulation. There must be uncertainty and the uncertainty must be the kind that is amenable
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 In a Latin Hypercube process the cumulative probability distribution of an input variable is stratified into n equal intervals, where n is 20

the number of iterations in the simulation, along its probability axis. A sample is drawn from each interval. Sampling is “forced” to represent values

in each interval, thus recreating the input’s probability distribution. Latin Hypercube is generally regarded as a more efficient sampling technique.

54

to probabilistic modeling. The method for treating the uncertainty depends in part on the type of quantity that

is uncertain and the source of the uncertainty as indicated in Chapter Three. For example, uncertainty that results

from management’s inability to resolve questions is not a proper subject for simulation in many cases. Such

instances are more properly handled with sensitivity analysis.

Simulations vary greatly in complexity but all generally consist of the following steps:

1) Define the problem or system to be simulated.

2) Formulate the model to be used.

3) Identify and collect the data needed to test the model.

4) Test the model and compare its behavior with the behavior of the actual problem

environment.

5) Run the simulation.

6) Analyze the simulation results and make changes to the situation/scenario under

investigation.

Flowcharts are commonly used in simulations. They are graphic aids (like decision trees discussed

above) that help one think logically about a process and to keep track of it. Process generators are usually needed

to generate the data needed for the simulation. The Monte Carlo process is one of the most commonly used

process generators. 

The Monte Carlo process is a procedure that generates values of a random variable based on one or more

specific probability distributions, such as those described above. The Monte Carlo process is not a simulation

method or model per se, although it has become almost synonymous with stochastic simulation. The Monte Carlo

process begins with the generation of a random (or pseudo-random) number from a uniform distribution. This

number is then transformed into a value from the specific probability distribution used by the model builder. 

In essence, the process works as if you specify a type of probability distribution and its parameters and

a number is randomly selected from that distribution. Numbers that are common in the specified distribution are

selected more often than the numbers that are less common in the parent distribution. If you select enough

numbers from the parent distribution, you would recreate the original distribution with your sampled values. The

random numbers generated by a Monte Carlo process can represent a sample of potential results in the process

under consideration. By contrast, Latin Hypercube is an alternative process for generating random numbers. 20
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

55

SPECIALTY MODELS

Risk-based models have been developed for a wide variety of applications. The insurance industry has

its own set of models, the chemical industry has developed numerous models, as has the EPA. The Corps of

Engineers is not without its own specialty models for expected annual damage estimation and major rehabilitation

of locks and dams. 

Special models have been developed for use by the Corps in many of their areas of endeavor. The

Hydrologic Engineering Center has developed numerous hydrologic and hydraulic models for use by Corps

personnel. HEP, developed by the U.S. Fish and Wildlife Service, is but one of many special models developed

for use in the evaluation of habitat. It may well prove useful to explore ways in which risk and uncertainty tools

and methods can be incorporated into a redesign of existing tools that are serving their users well. This could

include HSI models as the example in Chapter Six illustrates.

STATISTICS

Statistics are used to make sense of data. Specifically, they are used to obtain information from data.

There are two main bodies of statistics that Corps risk analysts need to master: descriptive and inferential

statistics. Descriptive statistics are used to organize, describe, summarize, present and understand data. This

includes measures of central tendency (means, medians, modes), measures of dispersion (range, percentiles,

variation, skewness, etc.), and tabular and graphical techniques (contingency tables/ cross tabulations, stem-andleaf displays, frequency distributions, histograms, pie and bar charts, line graphs, and so on) used to present the

information embodied in the data.

Frequently, we do not have or cannot get information about the entire population in which we are

interested. For example, we will never know the true flow regime. At best we will have a sample of several years

of data. It may be impractical to measure the true percentage of each type of ground cover found in the study area

for an HSI model. Instead, we may inventory a few sample sites and use that information to make generalizations

about the entire study area. Inferential statistics use information from a sample to generalize to the population

from which it was drawn. 

Anytime a sample is used to make some generalizations about the population there is risk involved. We

are using available and incomplete information to make judgments about the population. There is always a

chance the sample is unrepresentative of the population from which it came. An important part of any risk

analysis is properly quantifying the risks associated with sampling. This is, in part, the purpose of inferential

statistics.

Inferential statistics involves the use of sample statistic distributions and the calculation of interval

estimates that can be used in sensitivity analyses or Monte Carlo simulations. Interval estimates can be calculated

for various confidence levels. These same interval estimates can be used in hypothesis testing. Hypothesis

testing has many applications, one of which is testing the goodness-of-fit of a data set to a probability

distribution.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

56

It will often be useful for analysts to understand and be able to estimate correlations between and among

variables that may interrelate in models or calculations used in the planning process. The need for nonparametric

statistical techniques would be more likely to arise in environmental planning than in many other professional

settings. Although these techniques may not be the analyst’s highest priority, neither should they be considered

exotic techniques.

Many of us sat in our statistics classes and wondered, “When am I ever going to need this again?” One

answer is, “When you begin to incorporate risk analysis into your planning in order to improve the decisionmaking process.” A good command of the basics of inferential statistics is essential for every risk analyst.

SUMMARY AND LOOK FORWARD

The time may come when risk and uncertainty protocols for environmental restoration studies are

described in detail in Corps’ guidance and the models and tools needed to conduct these analyses are well known

to all. That day has not yet arrived and so this chapter has presented a list of selected tools and concepts used

by many risk analysts. The chapter began with some basic concepts of which environmental planning risk

analysts will need to be aware. These tend to deal with information/communication issues. The latter half of the

chapter addressed more specific types of knowledge or tools that are generally encountered by risk analysts.

The purpose of this chapter is not to say these are the techniques and here’s how you use them. Rather,

its intent is to say here are some techniques you might want to learn more about. The reasons for learning more

about them begin to become apparent in the next two chapters. Chapter Five presents some general sources of

risk and uncertainty in environmental planning. This discussion tends to use the broad concepts found in this

chapter. Chapter Six, by contrast, discusses more specific examples of risk and uncertainty encountered in

environmental planning. These examples make more use of the specific tools presented here.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

57
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Compilation and Review of Completed Restoration and Mitigation Studies in Developing an Evaluation Framework for 21

Environmental Resources, Volumes I and II, IWR Reports #95-R-4 and 5. 

58

CHAPTER FIVE: GENERAL SOURCES OF RISK AND

UNCERTAINTY IN ENVIRONMENTAL

RESTORATION PLANNING 

INTRODUCTION

“. . . it is impossible to predict with certainty how animal populations will respond to restored

or improved hydrological conditions.” Central and South Florida Project Reconnaissance Report

Comprehensive Review Study, November 1994, p. 108, U.S. Army Corps of Engineers,

Jacksonville District.

The above quotation sums up the frustration of the environmental planner’s job quite well. Their task

is a difficult one. Many things are uncertain in their studies. The best way to proceed is to acknowledge those

uncertainties, identify the most important among them, and address them in a rational fashion, subject to study

constraints. This chapter identifies sources of risk and uncertainty that could be generic to virtually any water

resources planning project. They are also the broad concepts of risk and uncertainty encountered in

environmental planning.

An earlier research report reviewed a number of Corps environmental project reports. Very little

21

evidence of risk and uncertainty analysis was found in them. A number of environmental project reports were

reviewed for this effort with a similar result. There is considerable evidence of uncertainty in the project reports.

Frequently, the limits of the analysts’ knowledge are straightforwardly acknowledged. For example, comments

like “. . . insufficient information is available to make definitive statements about the habitat requirements of

hickory shad. . .” are not uncommon. This acknowledgment of the existence of uncertainty is an important first

step in doing something about it.

Despite some acknowledgment of uncertainty it is still more common to see very precise values presented

as if they are absolute certainties. For example, in one study 17,281 habitat units were reportedly affected in the

past. Though an incredibly precise number, it is pure folly to believe that number is anything more than an

approximation of conditions that used to exist. There is still a great deal to be done in acknowledging the limits

of our knowledge.

This report’s review of project documents made it clear that there were different categories of sources

of uncertainty and different places where risk and uncertainty analyses could be introduced. Ignorance of the

planning process is one of the greatest sources of uncertainty evident in many reports. Many reports did not have

clearly identified problems and relatively few had any planning objectives. Absent these fundamentals, it was

often difficult to know what objectives the plans were formulated to meet and what problems they might solve.

These shortcomings are not in any way unique to environmental planning. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

59

There are, however, uncertainties that are unique to the environmental restoration process. One of these

is the entire habitat evaluation process. Another is the, “If we build it, will they come?” question, i.e., will

projects actually result in environmental improvements? With issues running from the general to the very specific

it is necessary to provide some order to the manner in which these issues are to be discussed. Some general

sources of risk and uncertainty are presented in this chapter. They are presented here because they may be

expected to occur with greater regularity, and perhaps with more impact, in environmental restoration studies.

To organize the presentation, the P&G-mandated six-step planning process used by the Corps in its

traditional planning work provides the organizational structure for the chapter. Those steps are presented in

Figure 1. The chapter is divided into ten sections: this introduction, some general observations, one section for

each planning step, and two summary sections. Discussion related to specific environmental restoration issues

are covered in Chapter Six.

FOUR GENERAL OBSERVATIONS

Before some broad concepts of risk and uncertainty are discussed, a few observations, too general to fit

elsewhere, are introduced here. First, risk analysis is not intended to increase the work requirements for anyone.

Any risk analysis must be purposeful, i.e., it must contribute to improved decision-making. In order to be

purposeful, we need to examine and understand what is already being done by environmental planners. That

process only begins with this report. 

The second observation follows from the first. There is a need to identify critical variables and key

relationships in what planners are already doing in environmental restoration studies. This will take more time

and focussed effort than were possible in this initial step in the process of introducing risk and uncertainty

analysis to environmental planning.

A third observation is not specifically related to risk and uncertainty. Many planners do not know about

or understand the P&G planning process. When planners don’t know how to go about planning there is a good

chance there will be considerable uncertainty about what they have done. 

The last general observation is a simple one. Assumptions must constantly be made by planners in order

for work to proceed. Professional judgment, intuition and gut feelings are important tools. Assumptions should

be avoided where answers exist. The need for and rationale of assumptions that are made should always be clear,

as the assumptions themselves need to be clear. 

These are “big picture” uncertainties as is most of what follows in this chapter. Much of what follows

considers uncertainty about the decision-making process itself. If the process by which decisions are made is

uncertain or poorly understood, the decisions that stem from it are questionable. Fundamental uncertainties such

as, “Are we addressing the right problems?” or “Do we have the best solution?” are indeed broad in concept.

But they are far from trivial.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

60

STEP ONE: PROBLEM IDENTIFICATION

There should be no uncertainty about the purpose and objectives of an environmental restoration study.

The first step in the planing process is the most critical one. In this step, planners identify the problems that have

motivated the study. Problems may be identified by the public or by technical experts. In addition to problems,

planners should identify opportunities for improving conditions now and into the future. Building on the

identification of problems and opportunities, the planning team develops specific objectives they want to achieve

in the planning process. At the same time, they identify constraints, i.e., those things they want their plans to

avoid doing. 

The planning objectives and constraints effectively become the mission statement for the Federal/nonFederal planning partnership. They are a concise and unique statement of the reasons these two parties have

come together in this study effort. Plans are formulated, evaluated, compared and selected based on their

contributions to the planning objectives.

A review of 15 environmental restoration project studies in preparation of this report revealed an uneven

record in terms of their faithfulness to this planning process. Some studies used a study team. Other studies were

done almost entirely by an individual. Problem identification was the part of this first planning step that was

generally handled best. Although there was rarely a clear and concise statement of the problems with the kind

of directness found in statements like, “The problem is. . .”; one usually understood the problem after reading

the report.

Cause and effect linkages were frequently missing from problem discussions. A somewhat typical

problem identification follows: 

“There is a deficiency of at least 15 acres of barrier island habitat at or above the 10-year flood

elevation, with stabilizing vegetation.”

Statements like these do a good job of specifically identifying problems - lack of a specific habitat type. They

are less effective in explaining why 15 acres is singled out and they do not clearly establish the cause of the

problem. Frequently, these additional details can be ferreted out from the surrounding text. That is not always

the case. When it’s not, problems remain uncertain. When problems are uncertain so are the solutions.

The manner in which problems are discussed and described, largely absent clear and concise descriptive

problem statements, can contribute to uncertainty over what the problems are perceived to be. We might all agree

there is a lack of a certain habitat type, but we might each have a different idea about why that is so. This

potential uncertainty can be limited by a careful problem statement that includes a cause and effect linkage.

This is not a trivial matter. It may be easy to note a decrease in duck habitat that accompanies a decline

in migratory duck populations. The temptation to link the two is strong. However, understanding the problem

is essential. If the decline in ducks is due to overhunting over a multi-state region, then no amount of habitat

restoration will solve the problem. In order for plans to be effective, studies must begin with clear and definitive

problem statements, devoid of ambiguity or other uncertainty.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

61

The reports reviewed tended to do less well with other important aspects of this first planning step.

Solving problems is clearly one way to make things better. Seizing opportunities is another. Opportunities are

rarely addressed. And yet, altering projects to seize opportunities seems very much in concert with the spirit of

environmental restoration. As a result, there can be uncertainty about the planners’ responsiveness to potential

opportunities.

Another common shortcoming is the manner in which planning objectives and constraints are handled.

Too often, there are no clearly stated planning objectives. Too many reports lean on discussions of the NED

objective which, though a Federal objective, is not a planning objective. In other cases the objectives remain

vague, for example, “. . . to preserve and enhance the existing aquatic plant beds for fish and wildlife habitat,”

to be truly useful in plan formulation. When this happens there is substantial uncertainty about the objectives

of the entire planning process.

At the other extreme, objectives sometimes include plan formulation and selection. For example, an

objective that said, “Re-establish 15 acres of stabilized islands from river miles 300 to 300.8”, is too specific.

Objectives should never include solutions. 

One of the most glaring offenses of the planning reports is that once planning objectives are identified

there is no apparent further use of them in the planning process in the majority of the reports reviewed. This is

one of the more troubling indications of a potential general ignorance of the Corps’ planning process. The work

done in this first step is essential to the success of the planning process. Identifying problems and opportunities

and specifying objectives and constraints are essential steps in limiting the uncertainty of purpose that can be

encountered in a planning study. 

There are, on occasions, studies in which it’s never really clear what is being accomplished, despite the

copious details on project design and increased habitat units. This may be due to ignorance of the planning

process. Some planning steps may not have been taken or were taken incorrectly. In other cases, the report

writers have simply failed to tell their story effectively. In either case, it is in everyone’s interests that planners

eliminate all uncertainty about the fundamental purposes of the study effort. Knowledge of the planning process

and a clear “story telling” style of documentation are the best ways to eliminate or reduce this potential

uncertainty of purpose.

STEP TWO: INVENTORY AND FORECAST

In the second step of the planning process, planners identify the data they’ll need to solve problems, seize

opportunities and formulate plans to meet their planning objectives. At this step, information is gathered in

sufficient detail to adequately define the existing conditions and to define a most likely future condition without

a project.

Uncertainty is removed by information. So, it should be evident that this information gathering step is

an essential one in any attempt to limit uncertainty. It should be equally evident that any attempt to describe a

future condition is fundamentally an exercise in uncertainty.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

62

At times, environmental restoration studies may rely too much on “cut and paste” data. This may seem

a contradiction when Section 1135 studies are expected to make use of existing data sources. The problem is not

so much using existing data as much as it is using data because it exists. In some cases, the recitation of existing

condition data have no material bearing on the problems, opportunities, objectives, plans or their effects. When

that’s the case, the data are extraneous and should not be used. 

If these data are used in lieu of the data required to enlighten the decision process, uncertainty will

persist. There are countless potential sources of uncertainty. They include professional judgments, professional

disagreements, lack of information, inaccessibility of private property, interpolations and extrapolations of

existing information, failure to recognize limiting factors, and on and on. For example, measurements of habitat

variables critical to the estimation of habitat unit changes attributable to a plan frequently appear in precise point

estimates that cover the data in a mantle of false precision and certainty. This issue is taken up in Chapter Six.

To a very great extent, efforts to quantify past conditions are, at best, guesswork. Rarely will planners

have sufficient information to describe past environmental conditions with certainty. What would the current

state of the environment have been in the absence of the Corps project? Is that what is motivating the restoration

study? Would it have been different without the project? If so, would it have been better or worse? When we

consider the cumulative impact of all the stresses on an ecosystem, it is not always evident that such a discussion

has any real meaning for a complex, dynamic ecosystem. Thus, it may be better to acknowledge the uncertainty

inherent in this backward looking analysis and to concentrate resources more on solving problems and seizing

opportunities. 

One of the more common situations in which uncertain values are left to stand unchallenged is when

planners must rely on the judgments of professionals to describe past, present, and future conditions. These

judgments are used in the absence of better information but they are rarely varied or tested. Planners infrequently

say why this judgment is best and they never say why it is the only possibility. The accuracy of subjective data

and judgments is always improved by providing an interval estimate. We’re always more likely to be right when

we can say an uncertain thing is between “this and that” rather than by stating precisely what it will be. 

A complicating factor in identifying existing and future conditions for environmental restoration studies

is that a static condition is inadequate for describing a dynamic system. What does it mean to describe the natural

conditions of a dynamic ecosystem when conditions are constantly changing in nature? The point of these and

other such questions is not to bog analysis down in esoteric questions as much as it is to get planners who work

in the field of environmental restoration to begin to develop some introspection and insight into what is uncertain

in the work they do so that they can identify those uncertainties that are most important; then find ways to

address them. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

63

STEP THREE: FORMULATE ALTERNATIVE PLANS

How can we have the best plan unless it’s chosen from the set of best alternatives? If there is not a clear

set of planning objectives, what guides the plan formulation process? How can there be a certain link between

problems and solutions if we do not have, or have but don’t use, planning objectives? From where do plans

come? How do you know you have a good set of alternatives? Having the best plan is a “big picture”

uncertainty, but it is most certainly not a trivial issue.

The greatest source of uncertainty in this step is, once again, ignorance of the planning process. If there

is not a principled and structured approach to solving problems and seizing opportunities there is no way to know

if an implementable project is really the best one or even a good one. Most project reports offered little evidence

of a thorough formulation process. The choice of a plan is the most important decision made in the planning

process. If risk and uncertainty analysis is intended to improve the quality of the decisions as well as the decision

process, it is a tautological fact that the best choice can only come from the best set of alternatives. Planners can

reduce the uncertainty about whether they have identified the best plan by eliminating the uncertainty attending

their planning objectives and by providing as broad a range of alternatives as possible.

STEP FOUR: EVALUATE PLANS

The evaluation of plans requires well defined planning objectives and well formulated plans. This step

is subject to many of the generic uncertainties already mentioned. A with project future condition must be

identified for each alternative plan. This will require data, judgment and guesswork because it, like identification

of the without project future condition, is a fundamentally uncertain exercise. 

The decision process becomes even more uncertain in this step, however, because it requires assumptions

about project performance. The performance of environmental restoration projects may be subject to more

uncertainty than most projects, a topic taken up again in Chapter Six.

Any project is subject to uncertainties about costs and design. A common problem encountered in

measuring plan effects is the tendency to substitute precision for accuracy. It may be comforting to see the cost

of earth moving precisely estimated to be $2.45 per cubic yard (CY). The accuracy of this cost is another

question. Do we really expect the cost per CY to be $2.45? How often are costs as accurate as they are precise?

Generally, estimators work with costs from a number of similar jobs. These costs tend to cover a range

of values. The estimator’s expertise comes into play in how he develops a point estimate from all the information

gathered. The quality of the decision might actually be improved, however, if costs were estimated as a range.

Then, decision-makers could see the range that costs could eventually span. Such information could also be used

by analysts in combination with other information to provide a better estimate of the possible true range in costs

per habitat unit produced by the project.

 Environmental restoration projects may be more uncertain than other studies because there has been

relatively less experience in building and maintaining them. Many things being done may be done for the first

time. Even when planners are experienced it is not uncommon to run into substantial design and cost uncertainties
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

64

because of a lack of foundation or hydrologic and hydraulic data, and so on. Planners might be wise to expect

these common sources of uncertainty to be exacerbated in unique environmental restoration studies. 

STEP FIVE: COMPARE PLANS

Once plan effects have been evaluated it is necessary to compare the effects of one plan to the effects

of another plan as well as to the option of taking no action. A comparison of plan effects requires some criteria

upon which the comparison will be based. Once again, those who are ignorant of the planning process may be

laboring in the shadows of uncertainty. If the criteria upon which the comparison is based are not known to

decision-makers there is the potential for considerable misunderstanding and error in the decision process. 

The plans’ relative contributions to the planning objectives provide the basis for the most objective

comparison of plans. What is not known are the weights that stakeholders will place on the different plans’

effects and their contributions to the various planning objectives. It may be easy to say that more of one thing

beats less of that thing but it becomes very difficult to trade-off differences in the mix of contributions plans make

to planning objectives. Thus, clarity and certainty of purpose in the study as well as in the explanation and

display of results are essential to the reduction of common sources of uncertainty in many planning studies.

There is an additional source of uncertainty in environmental restoration planning. Not all of the benefits

of these projects can be measured in dollar terms. This lack of a common metric means there is less reliance on

the national economic development criterion of net benefits and more reliance on other criteria that may be less

familiar to planners and stakeholders.

STEP SIX: SELECT A PLAN

In the final step of the planning process a course of action is chosen. This usually means the selection

of a plan, although no action is always an option. At this step in the planning process, all the uncertainties

encountered in the study become cumulatively and hopelessly hidden from the view of unsuspecting decisionmakers unless there has been a systematic attempt to deal with risk and uncertainty throughout the process. 

If significant issues have been obscured from the view of decision-makers there is a real possibility that

an error will be made in plan selection. Bad investments may be made. Good investments may be overlooked.

The sole purpose of introducing risk and uncertainty analysis to environmental restoration planning is to improve

decisions. That happens in this step, but only if it has taken place throughout the planning process.

None of the sources of risk and uncertainty discussed in this section are unique to environmental

restoration projects. Inasmuch as the newness of environmental restoration challenges planners at many levels,

it would seem all the more important to engage in a structured, rational planning process as devoid of uncertainty

as possible. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

65

WHAT IS TO BE DONE ABOUT GENERAL SOURCES OF

RISK AND UNCERTAINTY?

This chapter is not an indictment of the state of planning. It is instead an endorsement of the Corps’

planning process. Virtually all of the sources of uncertainty identified in the preceding sections can be eliminated

through the knowledge and use of the six-step planning process promulgated by the Principles and Guidelines.

A structured, rational framework for approaching environmental restoration problems can eliminate

confusion about the purpose and the process of planning for planners and stakeholders alike. This would free

planners to concentrate on those specific elements of a study in which risk and uncertainty analysis can be

systematically applied in order to improve the quality of decision-making.

SUMMARY AND LOOK FORWARD

Just as there are broad concepts that can be used in risk and uncertainty analysis there are “big picture”

issues of uncertainty encountered in environmental planning studies. These issues are not unique to

environmental planning. In fact, they are more indigenous to the planning process than they are to the

environment. Uncertainties are encountered in each step of the planning process and they can become cumulative

if not addressed, virtually assuring that decision-makers labor in ignorance of potentially important issues.

Nonetheless, the uniqueness and complexity of environmental plans suggests that the uncertainties inherent in

the planning process may become particularly acute in environmental planning.

In addition to the big picture uncertainties, there are many uncertainties specific to environmental

planning. Some of these are discussed in the chapter that follows. An example introducing risk-based analysis

to the estimation of habitat unit changes is also offered to demonstrate the feasibility of some of the methods

presented in the last chapter to addressing some of the problems of the next.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

66

CHAPTER SIX: SELECTED SPECIFIC SOURCES OF RISK

AND UNCERTAINTY IN ENVIRONMENTAL

RESTORATION PLANNING

INTRODUCTION

This chapter considers potential sources of risk and uncertainty that are specific, if not necessarily

unique, to environmental restoration projects. These sources of risk and uncertainty have been grouped for

convenience into four topics for discussion. These are project performance, models, extrapolation, and, the

habitat evaluation procedures (HEP) process. The HEP process is the primary focus of this chapter because it

is one of the primary tools of environmental restoration analyses and, as such, is a logical point at which to begin

looking for ways to introduce risk and uncertainty analysis. 

PROJECT PERFORMANCE

Project performance involves some fundamentally uncertain issues. They range from questions about

what environmental restoration projects produce to whether the management measures will result in what is

wanted. This is the most basic environmental restoration uncertainty. What are environmental projects supposed

to do and do they work? There was a surprising candor about the relevance of these questions encountered among

Corps personnel interviewed during the preparation of this report. 

PROJECT OUTPUTS

Many environmental restoration projects increase habitat units. But, what does that mean? The actual

outputs of environmental projects were not often identified in the reports reviewed for this document despite the

fact that EC 1105-2-206 Project Modifications for Improvement of the Environment requires them to be. When

they are identified, they are frequently poorly specified. For example, one Section 1135 report’s sole description

of project outputs was “. . . aquatic insects would benefit. . .”. Without judging the worthiness of such an output,

most would agree that it may have been useful to identify the types of insects that would benefit, the extent of

their benefit and the significance of that benefit. 

There is no reason for project outputs to remain uncertain to either planners, decision-makers, or the

readers of project reports. Although the outputs of flood damage reduction or navigation projects may be more

or less taken for granted, outputs of environmental restoration projects need to be specified especially because

they are new. Although habitat units may be one measure of project outputs, it should be clear that what a project

produces is far more specific than simple HU’s. They should be identified as completely as possible.

To do this well requires good planning. If problems and opportunities are carefully identified in the

planning process, planning objectives can be specified. Achieving the planning objectives is the purpose of any

plan. Hence, careful identification of planning objectives is an essential step in the specification of project

outputs. Estimating habitat units is simply one means of evaluating and comparing the effects of plans developed

during the planning process. Outputs can be measured in any number of ways, including habitat units, population

counts, biological integrity, environmental values, energy flow, physical and chemical characteristics of water,
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

67

and other ecosystem components. The Corps’ Evaluation of Environmental Investments Research Program has

an ongoing work unit, “Determining Objectives and Measuring Outputs”, that addresses some of these concerns.

Too often, the vagueness or uncertainty surrounding project outputs is due to incomplete planning. It

is impossible to develop good plans unless the objectives of those plans are clearly identified. Project outputs

are contributions to the planning objectives. These should never be uncertain and they should always be clearly

identified in the study documentation.

EFFECTIVENESS OF PLANS

A number of environmentalists and planners interviewed during the preparation of this report identified

project performance as a fundamental source of uncertainty. One planner, paraphrased below, described the

uncertainty rather succinctly. “We’re fiddling with landscape - the spacing of plants, height of fill, number of

duck boxes - and we’re using our hydraulics and hydrology and water quality data to manipulate water. Will this

result in what we want? If we build it, will they come?”

Can we manage salinity and water levels to increase diversity in the aquatic plant community for wildlife?

Dozens of such questions can be posed. In some cases we can pinpoint projects that are remarkable successes.

In other instances there have been abject failures. In all honesty, planners may not always know how well, if at

all, their projects will succeed in achieving the planning objectives and producing the desired project outputs.

The uncertainty of project performance can be addressed in a number of ways. In some cases, planners

have adjacent areas or similar projects they can use to gage project performance and place more reasonable

bounds on the uncertainty accompanying project performance. When such projects are not available, sensitivity

analysis can be used to test project performance. 

For example, once the most likely future scenario has been used to evaluate plan effects, reductions in

critical outputs can be considered. One might ask, if water quality improvements are only 90 percent of what we

expect them to be, would we continue with the project? How about if they are 80, 70 or 60 percent of what we

expect? Simple consideration of alternative outputs like this can help identify thresholds for the plan’s success

or for project support. Then the likelihood of those thresholds can be subjectively estimated to determine if this

presents a significant possibility. See the sidebar on the next page for an example. 

Sensitivity analysis can be more sophisticated. Critical variables in the evaluation and comparison of plan

effects can be systematically varied or simulations can be used to model potential outcomes. An example

illustrating how estimates of habitat units can vary is provided later in this chapter.

In some instances, the ecosystems are so complex and the solutions so experimental that monitoring of

project performance after the fact is the only way to reasonably gage project performance. In these instances, the

uncertainty of project performance would best be acknowledged during the study and used to
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

68

Simple Sensitivity

Suppose a project to produce a freshwater marsh

has a salinity goal of 0-0.5 parts per thousand (ppt). If the

project results in 1 ppt would it still be constructed? How

about 1.5 ppt? Let’s further suppose that such a thought

process leads the study team to conclude that salinities

could reach as high as 4 ppt before the project would be

considered undesirable, based on the plan’s contributions

to the full set of planning objectives. 

The next step in the sensitivity analysis would be

to try and gage the likelihood of the events that would

have to transpire to result in a 4 ppt salinity level. If these

are judged to be unlikely, the plan can proceed with

relative confidence that the project will at least achieve

acceptable salinity levels. If the likelihood is greater than

decision- makers would like then perhaps the plan

requires some fine tuning or a more detailed risk analysis. 

design a satisfactory monitoring program. As more

projects are constructed and monitored the bounds

of uncertainty about project performance will be

narrowed. 

Some uncertainty over project performance

can be addressed through simple sensitivity

analysis. In other cases more sophisticated and

systematic analyses may be helpful. In still other

cases, monitoring of experimental projects may be

the most appropriate way to handle uncertain

project performance. In any event, one or more of

these techniques are within easy grasp of Corps

planners. 

EXTRAPOLATION

Extrapolation is extending judgments or

conclusions into an area that is not known or

experienced. It is conjectural knowledge and it’s

everywhere in environmental restoration projects.

Under the best circumstances, extrapolation is scientific guesswork. It is often the best we can do given the

present limits to our knowledge. Planners are forced to extrapolate because the systems they work with are so

complex and our knowledge of them is so incomplete. Under the worst circumstances, extrapolation can result

from lack of knowledge or a lack of resources required to complete a study.

The very notion of identifying a target species and then using the condition of the habitat for this species

as an indicator of the ecosystem’s condition involves considerable conjecture. The assumed linear relationship

between an HSI and carrying capacity, that is built into this approach, is also conjecture. Extrapolating the results

of a project at one location to expected results of a project at another location is redolent with uncertainty,

especially when there are significant differences between the locations.

Even fundamental cause and effect relationships are frequently extrapolated. For example, a decrease

in habitat accompanied by a decrease in migratory ducks would often be considered prima facie evidence that

habitat loss is causing duck losses. More careful study might indicate duck losses are due to overhunting and

duck habitat is being lost because, as the ducks in this part of the country have disappeared over time, the

opportunities (and hence the demand) for hunting, have fallen off. As a result, land that was once preserved for

hunting the ducks that populated it may now be put to alternative uses. To some, these alternative uses could

appear to be the cause of the decline in ducks when they are in fact the effect. 

In more traditional environmental risk analysis, i.e., the analysis of human health risks, extrapolation is

a major problem. Analysts must extrapolate the effects of low dosage exposure to a contaminant from data on

high dose exposures. To complicate matters, the extrapolations must be made from laboratory animals to

humans. Dredging studies consider similar issues when estimating the lethal doses that will kill 50% (LD50) of
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

69

the fish exposed to a contaminant. To the extent that the EPA risk analysis paradigm may be adapted for Corps

studies, for either environmental or ecological risk analyses, extrapolation may become an even greater problem.

Identifying extrapolation from the known to the unknown as a source of uncertainty is, in some respects,

a cheap shot. When planners are forced to confront and deal with the unknown what else can they do but

extrapolate? Given they have no choices, is it fair to criticize this method? Certainly not, if the sole intent is

criticism. 

Extrapolation, under the best circumstances, is stepping tentatively from the known to the unknown.

Recognition of the uses of extrapolation is the most important first step in addressing the uncertainty that

accompanies extrapolation. Frequently, planners extrapolate without recognizing they have passed from the

known to the unknown. Little if any distinction is made between scientifically based facts and conjecture or

speculation. Planners should recognize when they are extrapolating and say that is how they got from point A

to point B.

Sensitivity analysis is, again, one of the most useful ways of resolving this problem. If we are forced to

speculate and conjecture as we step from the known to the unknown, surely we can speculate on alternative

outcomes of this step. When extrapolation is critical to study results it is advisable to first extend the limits of

what is known. If additional data or analysis can reduce the realm of the unknown and, consequently, the

uncertainty attending our extrapolation, getting the data and doing the analysis should always be the first step.

When extrapolation is the best approach to a problem, it should be done openly and accompanied by sensitivity

analysis when the results of the extrapolation could affect project performance or plan selection.

MODELS

Reality is complicated. Ecosystems are too complex to consider comprehensively in an effective manner.

Models are used to get around these problems. Models are simplified representations of real systems that capture

the most important elements of the system we want to understand. 

Consider a map. If you want to tell someone how to get from one place to another you do not need a life

size map that details every landmark and street encountered along the way. State maps show interstate highways

and major roads only. If you draw a map for a friend it generally shows only the route you want her to follow,

with occasional cross streets for reference. When giving directions you want people to have only the information

they’ll need to arrive at their destination. All else is extraneous detail. A map is a model of a much more

complex and detailed real world.

Just as maps leave out most of the details and ignore a great deal of information, so do the models used

by planners. Model uncertainty has been discussed previously and will be discussed in the next section as well.

Nonetheless, model uncertainty presents a potentially significant source of uncertainty. 

Models come in many forms. The Hydrologic Engineering Center’s (HEC) models are well known by

the Corps and around the world. The habitat evaluation process has spawned hundreds of HSI models. The

Corps has developed a cost effectiveness and incremental cost analysis model for use in evaluating environmental

projects. Dozens of spreadsheet models may be developed to perform specific calculations during any one study.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

70

Expert opinions may comprise models. The six-step planning process used by the Corps is itself a model.

Models abound in the environmental restoration planning process. 

The responsibility of the planner who wants to deal effectively with uncertainty is to be aware of the

assumptions underlying the models he uses and their limits. If important details are omitted from the model you

use you’re not likely to arrive at the proper destination. It makes no difference whether the model is an HSI

model, HEC-2, or a simple spreadsheet computation you developed; if it lacks important details it could lead you

to the wrong conclusion.

Many environmental restoration projects are conducted with existing data on limited budgets using

available tools. These are circumstances that can lead to the use of models that are not ideally suited for the

analysis. When this is the case, and it may often have to be, risk analysis may help compensate for the lack of

detailed information. 

Most HSI models are predicated upon an extensive review of the relevant literature. Conflicting views

are sometimes averaged, other times ignored. Some suitability graphs are based on scientific evidence, many

others are straight lines running from an SI of 0 and percentage of the habitat variable of 100 through an SI of

1 and a percentage of the variable of 0, or vice versa, as shown in Figure 11. The quality of the input data is

usually an important constraint in any habitat evaluation. The following section takes a closer look at the

potential uncertainty in the HEP analysis. 

HABITAT EVALUATION PROCEDURE

Planners are interested in the restoration and protection of ecological resources. The viability of plant

and animal populations is, however, difficult to quantify. Populations are generally assumed to be a function of

the site’s carrying capacity. Carrying capacity, in turn, is generally assumed to be a function of the quantity and

quality of available habitat. Biological models linking levels of habitat quality and observable characteristics of

the ecosystem of interest are a popular tool for estimating the effectiveness of a variety of management measures.
Figure 11: Sample SI Functions

0 10 20 30 40 50 60 70 80 90 100

Percentage

0

0.2

0.4

0.6

0.8

1

Suitability Index

0 10 20 30 40 50 60 70 80 90 100

Percentage

0

0.2

0.4

0.6

0.8

1

Suitability Index

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 These models are identified in Appendix C of EC 1105-2-210 Ecosystem Restoration in the Civil Works Program,

22

1 June 1995.

71

At least 14 different techniques and models have been developed to assist planners in the task of

22

measuring plan effects on ecological resources. In general, these models rely on quantitative relationships among

habitat variables such as depth of water, height of vegetation, percentage of submerged substrate and the like.

Environmental restoration plans consist of management measures, either features or activities, that are expected

to produce changes in the habitat variables. These changes in variables, in turn, translate into changes in habitat

quality.

This section uses the U.S. Fish and Wildlife Service’s (USFWS) habitat evaluation procedures, known

as HEP, as the basis for its discussion. The HEP is one of the more widespread modeling procedures. The

USFWS has developed a series of Ecological Services Manuals (ESM’s) that provide considerable detail in the

background and development of the habitat evaluation procedures. 101 ESM Habitat as a Basis for

Environmental Assessment, 102 ESM Habitat Evaluation Procedures (HEP), 103 ESM Standards for the

Development of Habitat Suitability Index Models, and 104 ESM Human Use and Economic Evaluation

(HUEE) provide the best available discussion of the HEP process. This discussion approaches the HEP process

from the user’s perspective rather than from the perspective of an analyst developing a new HSI model. 

In order to make the linkage between plan effects and changes in ecological resources, the typical HEP

analysis identifies a single species to serve as the target species. The underlying presumption is that if this target

species is doing well, most other species in that ecosystem will likewise do well. In order to quantify existing and

future habitat quality for the target species, a habitat suitability index (HSI) model is frequently used. These

models may be developed specifically for a study or they may be USFWS models developed for general use for

specific target species.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

72

The HSI models identify the habitat variables that are most critical to determining the quality of the target

species’ habitat. In addition, they describe the nature of the mathematical relationships among those variables

based on informed professional judgments, a potentially significant source of model uncertainties. The models

provide a framework for describing the overall habitat quality through the estimation and measurement of habitat

variables and the calculation of a variety of model components. 

There is considerable uncertainty inherent in this evaluation process. The section that follows identifies

some of the potentially more significant sources of uncertainty. The subsequent section presents an example risk

analysis of a hypothetical HEP analysis to demonstrate the feasibility of incorporating basic principles of risk

and uncertainty analysis into the habitat evaluation procedures. 

POTENTIAL SOURCES OF UNCERTAINTY IN HEP ANALYSIS

The HSI model is one of the principal determinants of the change in habitat units that results from an

environmental restoration plan, the other being the area of habitat affected by the plan. The HSI model is the

focus of the discussion that follows. 

There are four categories of uncertainty encountered in a typical HSI model. First, there are theoretical

uncertainties. These include the basic premise of the HEP procedure and the model uncertainties discussed in

previous sections. The existence of such uncertainties are acknowledged but not further addressed in this report.

Second, there is uncertainty inherent in the “hard wiring” of the model, i.e., the relationships field analysts use

without question, based on the literature review and professional judgments of the model’s creators. Third, there

are uncertainties attending measurements of model inputs by field personnel. Finally, there are uncertainties

about project performance and future variable values without and with a project. 

This section concentrates primarily on the uncertainty accompanying habitat variables for existing

conditions as well as future conditions without and with a project , the third and fourth categories of uncertainty.

Using the Morgan and Henrion approach to uncertainty analysis, the following subsections identify the type of

quantity that is uncertain as well as the source of uncertainty and potential methods for addressing this type of

uncertainty. This information is summarized in a table at the end of the section. 

DELINEATION OF THE STUDY AREA

The HEP analysis study area is defined by the USFWS as “. . . those areas where biological changes

related to the land or water use proposal under study are expected to occur.” This would include areas affected

directly or indirectly. Uncertainties can arise because project effects can involve complex biological linkages.

Alternatively, privately owned lands that may be affected by a project are sometimes inaccessible to planners and

may be excluded from the project area simply because they cannot be adequately investigated. 

 Simply identifying the area affected by a project is often an exercise in uncertainty. The planning area

is a domain variable. The source of the uncertainty is approximation. Because this is a model domain parameter

to be determined by project analysts it would be most appropriate to address the uncertainty, if it is judged

significant, through sensitivity analysis. That is, alternative definitions of the study area can be analyzed to

determine whether or not the recommended plan or its impacts are significantly affected by the definition of the
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 See “Guidelines for the Development of Community-level Habitat Evaluation Models” by Richard L. Schroeder and Sandra L. 23

Haire, Biological Report 8 dated February 1993 for the USFWS.

73

study area. A more comprehensive definition of the study area could result in larger project-related changes in

HU’s and, consequently, lower incremental costs per HU. This is not likely to be a serious source of uncertainty

in most instances. The default strategy would be to select the more encompassing area as the study area.

TARGET SPECIES

Formulation of plans intended to benefit a single target species could result in unintended adverse effects

on other species. An HEP assessment is directly applicable only to the evaluation species. The degree to which

results of an HEP analysis can be extrapolated to the larger wildlife community depends on the care taken in

selecting the evaluation species. No one target species can adequately represent the welfare of all ecological

resources in any but the simplest of ecosystems. Even species in the same guild can have very different responses

to certain changes in the habitat variables.

During the early years of habitat evaluation, identification of a single target species was considered a

giant step forward. As experience in developing and applying models has been gained, there is a growing

realization that what was learned about target species modeling can be applied to communities as well. Many

biologists contend the best HSI models would be community-level habitat evaluation models. If that is not

23

feasible, using a suite of evaluation/target species may better balance the diverse habitat needs of a community.

In truth, evaluation species are often selected with far less analysis and for far more pragmatic reasons.

For example, advisory groups may have identified key species for previous studies or other purposes. Adopting

one of them gains instant credibility from these groups even if the selections were not optimal for the specific

study area. In other cases, non-Federal partners or other powerful stakeholders may identify the evaluation

species. The less scientific the selection process the more important it becomes to do a sensitivity analysis, using

a species that more effectively indicates the ecosystem function.

The target species is a decision variable. The associated uncertainty arises primarily from subjective

judgment but there is potential for professional disagreement. Given the complexity of ecosystems it may be

impossible to identify a single target species that represents the full range of potential ecological effects. Yet,

study budgets may make community-level models or suites of species impractical choices. In this case the analyst

picks the model that is most practical after weighing the analytical requirements of the study, its budget and other

constraints. 

A common source of disagreement can arise from the fact that there are at least two approaches to

selecting the evaluation species. One is to pick a species with high public interest. Another is to pick a species

that provides a broad ecological perspective of the area. A species of interest to hunters or fishers may be a

pragmatic choice but it may not be a good indicator of the ecosystem’s well-being. Disagreement over the

objectives of the study could lead to disagreement over the evaluation species as well.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

74

The most appropriate treatment of uncertain decision variables from either of these two causes is to use

a sensitivity analysis. That is, try other species and see if the plans would change. Given the potential sensitivity

of habitat quality to key habitat variables and the sensitivity of these variables to management measures it is not

difficult to imagine that plans could change significantly in some cases if different target species are used. 

This analysis of different species can be done as a pure sensitivity analysis or it could be more

appropriately done in the evaluation of a suite of target species. It is not likely that such a step would be possible

for most small budget studies. However, in larger studies suites and sensitivity analyses may be appropriate. 

HSI MODELS

Model uncertainty is not the focus of this discussion, but it is important enough that to ignore it would

be negligent. The HEP analysis and the HSI models that are used in the HEP are all the result of informed

professional judgment. Nevertheless, they are subjective. They are scientific guesswork.

There are some who would challenge the assumption that habitat quality is a good indicator of the

viability of ecological resources, opting instead for condition or diversity indices, studies of energy flow and so

on. There may be a tenuous relationship between carrying capacity and habitat quality as measured by the

variables. The problem of extrapolating assumed impacts on the habitat of a target species to the well being of

an entire ecosystem has already been touched upon. 

There are still more specific sources of uncertainty. Habitat models in the HEP analysis must be in index

form. The index runs from 0 to 1.0 and is dimensionless. An index of 0 indicates a complete lack of suitability

of any area as habitat for the indicator species. An HSI of 1 identifies the area as an ideal habitat, in a practical

sense. The HSI is defined as:

Study Area Habitat Conditions

 HSI = 

Optimum Habitat Conditions

Furthermore, the use of an HSI model in an HEP process requires the HSI to be a linear index. That is, an

improvement from 0.1 to 0.2 is the same magnitude as an improvement from 0.8 to 0.9. It requires a heroically

optimistic leap of faith to expect the relationship between HSI and carrying capacity to be linear. Linearity may

not be a realistic assumption but it does allow us to proceed.

The HSI is an index variable that purports to relate habitat and carrying capacity. Many users of the HEP

may not understand its assumptions and limitations. Uncertainty may arise from imprecise language that results

in less than a clear understanding of what an HEP or an HSI is to do. The most effective treatments for this are

education and training.

The structure of the models, i.e. the “hard wiring”, is another important source of potential model

uncertainty. Not all habitat variables are of equal importance to the quality of the target species’ habitat.

Assumptions about the relative importance, the additivity, and compensation among variables are built into the
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

75

mathematical assumptions of the model. For example, the relationships among variables that determine a

particular life requisite might be modeled as follows:

HSI = lesser of V or [(V V ) + .5(V V ) ] 1 2 3 4 5

.5 .5

This formula yields an analytical result. It is nothing more than an attempt to reflect the relative

importance of different variables in determining the quality of a habitat. It gives a predominate role to the

limiting factor V . The weights assigned to the other four variables by the formula are very subjective. Suppose 1

we used a cube root instead of a square root? Suppose the variables were additive rather than multiplicative? 

Should the weight for the (V4 V5) factor be .25 or .75 instead of .5? Would these or any other changes have an .5

effect, significant or otherwise, on the evaluated effects of the plan or on plan formulation?

In addition to concerns about the extent to which the models reflect the workings of ecosystems, we need

to be concerned about how people learn the HSI procedures. Are they all trained and certified by USFWS or do

they learn it by looking at previous reports, talking to people who have done them before, or winging it? Expert

opinion, the development of special models, and peer review may be methods for improving the structure of HSI

models. 

As good as these new biological models are, they are still only models and are subject to potential model

uncertainty. As noted earlier, model uncertainties may be the most significant and the most difficult to deal with.

Model uncertainties, as important as they may be, are not the uncertainties that Corps planners should deal with

first. They are mentioned here, however, because they may, in the long-run, prove to be among the more

important sources of uncertainty and they should not be forgotten. 

HABITAT VARIABLE MEASUREMENTS

Estimating the existing values of habitat variables is perhaps the most basic source of uncertainty in the

HEP analysis. Many environmental restoration projects are small budget planning studies. Although the HSI

models usually provide recommendations on the collection of data, Section 1135 studies, for example, are

intended to be based as much as possible on existing data. The HEP analyses for these projects are often

subjective. 

Typically, data must be collected on variables that define the species’ life requisites. These include cover

type, food, and other factors. Baseline habitat data will generally be based on subjective estimates (opinion),

primary data (field measurement) and secondary data (e.g., remote sensing). For example, estimates of the

percentage of unsubmerged substrate, the percentage of land in the affected area, the height of vegetation or the

depth of water may be based on conversations with professionals familiar with the study area or on very cursory

eyeball estimates. These estimates carry the weight of informed professional opinion. It is a matter of

convenience that we not challenge these numbers because to do so would increase study budgets and analytical

requirements. Or would they?

As the example in the section that follows shows, it is a relatively easy matter to increase the accuracy

of one’s estimate of the value of habitat variables by using interval estimates rather than point estimates.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

76

Although point estimates are more precise, they are never more accurate. If an analyst’s job depended on being

right, who would prefer to say the average depth of water is 30 cm to saying it is between 25 and 35 cm? 

When an analyst is not sure of the exact value of a variable it is just as easy to have him estimate it as

an interval as a point. That is no extra work. In fact, going from an uncertain interval over which he feels fairly

comfortable in his accuracy to a seemingly certain single point, is often the more difficult task.

When study budgets are sufficient or when the problems are significant enough, planners may actually

take measurements of variables. This will invariably be a sampling process. It may be done through remote

sensing techniques, using geographic information system databases or it may be direct field measurements. When

sampling is used, inferential statistical techniques can be used to estimate the parameters of the sampling

distribution. The sample data are used to estimate a sample mean and a standard error. These values can then

be used to define a normally distributed sampling distribution or confidence intervals. These results can be used

for subsequent risk analyses.

The habitat variable quantities are empirical quantities. They are uncertain because of measurement error,

statistical variation, subjective judgement, disagreement, variability, inherent randomness or approximation. The

treatments of such uncertainties span the range of analytical techniques discussed earlier in the manual. 

EXISTING HABITAT UNIT ESTIMATES

An HEP analysis is structured around the calculation of habitat units for each evaluation or target species

in the study area. HU’s are defined as the product of the Habitat Suitability Index (quality) and the total area of

habitat (quantity). 

The HSI for available habitat is a function of the suitability of all the evaluation species’ life requisites

in the study area. Every model has specific data requirements and a unique mathematical relationship for

combining the habitat variables into an HSI. The HSI for available habitat can embody many different and

cumulative sources of uncertainty. These include uncertainties about the representativeness of the target species

selection, uncertainty about the habitat variables as well as model uncertainty about the manner in which an HSI

is calculated. 

Ideally, it may be desirable to develop different HSI’s for sections of the study area over which the

quality or quantity of the habitat variables vary significantly. It is common practice, however, to calculate a single

HSI. Although such an HSI should represent average conditions over the study area, it is rarely clear from the

study reports whether that is the case or not. Thus, there is a fundamental question about what the calculated HSI

represents. Is it the average HSI? Is it representative of the entire study area or a small part of it?

In addition, there may be some uncertainty about the amount of land that will be affected by the proposed

project. This uncertainty could arise from measurement errors (e.g., planimetering inexact areas on maps of

questionable accuracy) or basic uncertainty about the spatial extent of project effects. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

77

The estimate of existing HU’s is an outcome variable and its uncertainty is derived from the various

sources of uncertainty that affect the quantities that determine HU’s. The treatments of these uncertainties span

the gamut of known methods.

FUTURE HABITAT VARIABLE ESTIMATES

Measurements of existing habitat variables are theoretically determinate. Given enough time and money,

analysts could go out into the field and take careful measurements that would enable them to quantify habitat

variables with a high degree of certainty. Estimates of future habitat variable values are by their nature uncertain.

There is no way these values can be certain no matter how much time or money the investigators have.

Forecasting the future is fundamentally uncertain. 

At least two estimates of future habitat variable values must be prepared. One describes what will most

likely happen to the habitat variables if no project is implemented. The other describes what will most likely

happen if a project is implemented. There must be a with project future forecast for every alternative plan

evaluated. It seems a matter of basic honesty in the analysis to acknowledge the uncertainty inherent in this

aspect of the HEP analysis. 

These values are chance variable quantities (i.e., empirical quantities) and their sources of uncertainty

include subjective judgment, professional disagreement, statistical variation, variability, and inherent randomness.

Treatments of the uncertainty could include most known methods. 

CHANGE IN HU’S

Ostensibly, the purpose of the HEP analysis is to calculate changes in habitat units that are attributable

to project effects. An increase in HU’s is one measure of the presumed outputs of the project. The change in

HU’s is, in Morgan and Henrion’s language of uncertainty, an outcome variable. It is an uncertain value if any

of the models or values required to estimate it are uncertain. If there is model uncertainty, uncertainty about the

spatial extent of project effects, uncertainty about the choice of evaluation species, uncertainty in the

measurement of variables, and so on, then there is also uncertainty in the measurement of project outputs as

evidenced through the change in habitat units from the without project to the with project condition.

The timing of changes in habitat variables can ultimately have significant effects on the change in HU’s

for a study area. That is, do the forecast changes in habitat variables take place instantaneously or over many

years? Devoting more attention and detail to estimating the temporal sequence of changes in habitat units can

be another effective way of dealing with a potentially significant source of uncertainty. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

78

PROJECT PERFORMANCE IN HEP ANALYSIS

A Corps biologist consulted during the preparation of this manual pointed out that the public often knows

what it wants but translating that into terms an engineer can deal with is difficult. He aptly described the

challenge when he said, “Land managers talk function and feathers while engineers talk feet and inches.”

Nonetheless, he pointed out that when a community has the resources and facilities of the Corps of Engineers to

do good things for the environment, that is an advantage most environmentalists will never have.

Two uncertain issues arise regarding environmental projects. One is, do we have the right project? The

other is, will it work? The first issue requires us to ask the question, can this project contribute to our planning

objectives? The follow-up question is, will it? During the preparation of this manual a surprising degree of

frankness was encountered among Corps personnel concerning the basic uncertainty about the performance of

environmental projects.

The environmental restoration programs are new. The ecosystems are complex. And, although the

science is often state-of-the-art, it can only take the analysts so far, then they must guess and hope. How long

will the geotubes last in the water? Will the coconut fiber grass mats take root? Will the plants that grow in the

restored wetlands provide a diversity of food and cover for species or will invasive plants with little or no food

and cover values grow? Will the closure structure allow the marsh to drain in time to prevent disruption of

salinities and nests in the normally unsubmerged substrate? The performance questions go on and on.

Even though virtually every analyst consulted could raise questions about the performance of their

projects, none of the project reports reviewed reflected this uncertainty. That is not surprising. Admitting a

project could have an outcome other than the one the partners hope for does nothing to build support for the

project. Until data on project performance begin to become available through monitoring programs, there is no

objective way to know how serious a problem this may be, if it is a problem at all. 

At this point, we simply note that project performance is a potentially significant source of uncertainty.

 It is, in fact, an empirical quantity, a chance variable, but it is estimated as an outcome variable. Defining

alternative with project conditions that reflect different outcomes and evaluating the changes in habitat units under

each is one feasible way to address this problem. 

Table 5 summarizes the uncertainties described in this section. They are precisely the kinds of

uncertainty that would be addressed in the example that follows, although emphasis in the example is on the last

two items in the table.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

79

Table 5: HEP Uncertainties

Item Type of Quantity Source of Uncertainty Treatment of Uncertainty

Study area Model domain Approximation Sensitivity analysis

Evaluation species Decision Disagreement, statistical Sensitivity analysis

variation, subjective judgment

Total area Empirical Approximation, statistical Sensitivity analysis, probability

variation, subjective judgment, distribution, sampling

variability, disagreement distribution, uncertainty criteria,

expert opinion

Habitat variable Empirical, decision, Approximation, statistical Education & training, sensitivity

measurements index variation, subjective judgment, analysis, probability distribution,

variability, linguistic sampling distribution, uncertainty

imprecision, disagreement criteria, expert opinion

HU’s Outcome Approximation, statistical Education & training, sensitivity

variation, subjective judgment, analysis, probability distribution,

variability, linguistic sampling distribution, uncertainty

imprecision, disagreement criteria, expert opinion

RISK ANALYSIS EXAMPLE: MOTTLED DUCK HSI MODEL

Is risk-based analysis of environmental restoration projects feasible? What will it take to do such

analyses? This section presents a simple example of how risk-based analysis can be used to estimate the expected

change in HU’s. The example uses the mottled duck HSI model to demonstrate the feasibility of risk analysis

with commonly available basic information. Data from a 1992 Section 1135 study were modified and used for

this analysis. The subsections that follows describe the analysis and its results.

The basic elements of the mottled duck model are described below. This example focuses on how the

uncertainty in input data can be reflected. No attempt is made to address model uncertainty, though some

opportunities for doing so are discussed. The modification of the data was arbitrary. The results of the example

analysis have no significance beyond the demonstration. They have absolutely no bearing on the original study.

Mottled Duck HSI Model

The description that follows is taken from the September 1983 Habitat Suitability Index Models:

Mottled Duck, FWS/OBS-82/10.52. The model was prepared by James C. Rorabaugh and Phillip J. Zwank of

Louisiana State University for the Fish and Wildlife Service of the U.S. Department of the Interior. The

relationship of the habitat variables and life requisites to the habitat suitability index are shown in Figure 12. The

HSI is a function of three life requisites: cover, food and other (disturbance level). There are eight habitat
Figure 12: Sam p le HSI Mode l

Habitat Variables Life Requisites

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

80

variables defined as shown below. Suitability graphs transform habitat variable measurements into suitability

indices on a 0 to 1 scale. The functional form of their suitability graph is given in parenthesis. 

V = Percentage of unsubmerged substrate covered by rushes, bullrushes, or cattails. (Linear 1

suitability graph.)

V = Percentage canopy cover of trees and shrubs on unsubmerged substrate. (Piecewise 2

linear suitability graph.)

V3 = Structure of herbaceous vegetation on unsubmerged substrate. (Step function suitability

graph with four classes/steps.)

V = Percentage of continually submerged substrate covered by woody or herbaceous 4

emergent vegetation. (Piecewise linear suitability graph.)

V5 = Structure of woody or herbaceous emergent vegetation growing in continually

submerged substrate. (Step function suitability graph with four classes/steps.)

V = Percentage of study area that is land. (Piecewise linear suitability graph.) 6

V = Percentage of continually submerged substrates with water depth less than 30 cm at 7
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Excel v. 5.0 with the @RISK add-in v. 3.1 was used to develop the model. A more detailed explanation of the model is presented at 24

Appendix A.

 Uniform distributions were used. The minimum and maximum values were obtained by adding and subtracting 10 percent to/from the 25

value used by the District, except where a logical constraint of 0 or 100 prevent this. These arbitrary changes were made for the sake of simplicity.

81

low mean tide. (Linear suitability graph.)

V8 = Disturbance level: extreme; moderate; minimal; and, none. (Step function suitability

graph.)

Four model components and three life requisites are calculated using these input variables. Nesting hen

cover (NHC) is the cube root of the product of the SI’s for V , V , and V . Hen with brood cover (HBC) is the 1 2 3

square root of the product of the SI’s for V4 and V5. Cover structure (CS) is defined as the minimum of NHC

or HBC. The cover ratio (CR) is the SI for V . The reproductive cover life requisite (C) is the cube root of the 6

product of CS squared and CR. The food life requisite (F) is the SI for V . The other life requisite (O) is the SI 7

for V . The mottled duck HSI is the minimum of C, F or O. 8

To demonstrate the feasibility of introducing simple risk analysis to the estimation of an HSI the mottled

duck model was built in a spreadsheet. Appendix A explains the spreadsheet model in greater detail. The

24

original Corps analysis estimated the existing condition HSI to be 0.31 and the improved condition HSI was 0.79.

To simplify this demonstration a simple comparison of with and without condition HSI’s is used. There is no

discounting or multi-year analysis. In order to allow the investigation of the effect of a risk analysis on potential

incremental costs, habitat units were computed using the same 1,000 acres of land used in the original Section

1135 report. Project costs of $1,824,000 were used. Because a single increment of 1,000 acres was used in this

calculation for demonstration purposes, the example should not be considered a true incremental cost analysis.

 Although it does reflect the approach of the original report a true incremental cost analysis requires different

increments of output. 

Using the original without and with project condition input data from the 1992 report, habitat variable

values were allowed to vary according to a simple probability distribution. A value for each variable was

randomly sampled from this distribution using a Latin Hypercube process and these values were then used to

generate other model components, the HSI’s without and with the project, and the change in HU’s and the

incremental costs of those changes.

For example, V , the percentage of unsubmerged substrate, was estimated in the report to be 10 percent 1

without or with the project. In the example, this value was arbitrarily allowed to vary from 0 to 20 percent . A 25

uniform distribution was used for simplicity. 

It would not be uncommon for an analyst working under budget and time constraints to be uncertain of

the actual percentages and classes of habitat variable values. Although it may be more precise for the analyst to

say the percentage of unsubmerged vegetation is 10 percent that is not likely to be the actual value. It would most

likely be far more accurate to estimate this variable with an interval, like 0 to 20 percent. Though far more
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

82

sophisticated analyses could be done, this simple change serves the purpose of demonstrating the ease with which

interval input data can be obtained and used. 

The actual range of values could be estimated from confidence intervals obtained from the sampling

distribution of a proportion (or mean) if sampling from aerial photographs or field measurements are used. Or,

it could be based upon the opinion of knowledgeable personnel of the resource management agency.

Alternatively, it might be based on the “eyeball” estimate of the Corps biologist. In any case, it is a relatively

simple matter to prepare interval estimates of input variable values.

Rather than enter V as 10 percent, the unsubmerged substrate was estimated to range from 0 percent 1

to 20 percent. A uniform distribution was assumed for all input variables in this example. Table 6 summarizes

the without project condition distributions used for the input variables. With condition distributions were

changed for the variables the project was expected to improve. The minimums and maximums of the uniform

distributions are presented as percentages in the table. Percentages of each of the four classes for the step

function habitat variables are provided. Classes that are not listed are not present in the study area and are

assumed to equal zero percent. Thus, for V3 there is 0 to 10 percent in Class 3 with the remainder (90 to 100

percent) in Class 4.

Table 6: Without Project Condition Distribution Parameters

Variable Uniform Distribution Min and Max

Class 3% for V3 0, 10

Class 4% for V3 90, 100

Class 3% for V5 0, 10

Class 4% for V5 90, 100

Class 3% for V8 0, 10

Class 4% for V8 90, 100

V1 0, 20

V2 0, 10

V4 0, 20

V6 10, 30

V7 70, 90
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

83
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

84

Using the Terminology

Early sections of this report developed a taxonomy of uncertainty that can be used to help an analyst to

think about how to approach the analysis. Table 5 presented a somewhat generic example. Here you’ll find a more

specific application of the concepts to the mottled duck model. The example is a hypothetical one, loosely based on

an actual study.

Item Type of Quantity Source of Uncertainty Treatment of Uncertainty

Study Area Model domain parameter Approximation Assumed certain

Mottled Duck Decision variable Subjective judgment Assumed certain

1,000 Acres Empirical parameter Approximation Assumed certain

V1 Empirical parameter Variability Probability distribution

V2 Empirical parameter Variability Probability distribution

V3 Empirical parameter Variability Probability distribution

V4 Empirical parameter Variability Probability distribution

V5 Empirical parameter Variability Probability distribution

V6 Empirical parameter Approximation Probability distribution

V7 Empirical parameter Approximation Probability distribution

V8 Empirical parameter Subjective judgment Probability distribution

HSI Outcome criterion All the above Determined by inputs

HU’s Outcome criterion All the above Determined by inputs

$ per HU Outcome criterion All the above Determined by inputs

The first three quantities have been assumed to be determinate values, i.e., the analyst considers them to be

certain values. These items are not treated as uncertain in this analysis. This was done because of the “politics” of

the study. The first five habitat variable values are based on the results of sample measurements. They are

uncertain because of the variability in the sample results. The sixth and seventh habitat values were based on

approximations made from aerial photographs. The eighth variable value is based on the subjective judgment of the

park rangers who are most familiar with the project area.

The treatment for the habitat variable uncertainty is to represent the potential range of values as a

probability distribution. The last three items are outputs of the model. The uncertainty in them is a result of the

uncertainty in the variables from which they are calculated.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Despite the tendency to want to reproduce table results from table values, that generally cannot be done. For example, the minimum 26

change in HU’s is 282 but the minimum number of HU’s with the project less the maximum number of HU’s without the project do not yield this

value. That is because these minimum and maximum values were not observed during the same iteration of the simulation.

85

A Latin Hypercube procedure was used to transform the HSI model from a simple calculation to a

simulation model. The model calculated a without project HSI, a with project HSI, HU’s without and with the

project, as well as the change in the number of HU’s attributable to the project during each iteration of the model.

The cost per each additional HU was also calculated. One thousand iterations of the model were run in which

a value was randomly selected from each input distribution and “plugged” into the HSI computation. Thus, 1,000

pairs of HSI values, HU’s and their costs were separately computed based on the varying inputs shown in Table

6.

The results of the simulation are summarized in Table 7. Minimum, maximum and mean values are

presented for selected model outputs. For example, the minimum observed HSI without the project was 0 while

the maximum observed value was 0.42. The average without project HSI for the 1,000 calculations was 0.23 A

26

simple variation of the input variables results in a considerable range in potential project outputs. The project

could produce as few as 282 HU’s at an incremental cost of $6,479 each (costs are present values, not average

annual equivalents) or as many as 750 HU’s at $2,432 each. Figure 13 shows a cumulative distribution of costs

per habitat unit. There is a 30% chance HU’s will cost less than $4,000 each and nearly a 70% chance they’ll

cost less than $5,000 each. The mean number of HU’s obtained in this 1,000 iteration simulation was 456 HU’s

at an incremental cost of $4,350 each.

Table 7: Selected Simulation Results

Item Minimum Mean Maximum

HSI Without 0 .23 .42

HSI With .58 .69 .75

HU’s Without 0 232 417

HU’s With 580 688 750

Change in HU’s 282 456 750

Cost per HU $2,432 $4,350 $6,479

These results make the uncertainty in project performance very apparent. The number of HU’s that result

are estimated to be between 282 and 750 HU’s. The incremental cost of these HU’s is likewise uncertain.

However, if the maximum cost of $6,479 per HU is reasonable there is little reason to question the wisdom of

implementing the project. Now, suppose for the sake of this example that $6,000 per HU is considered the

maximum feasible incremental cost for political, institutional, economic or other reasons. 
Figure 13: Cumulative Distribution Function for HU Costs

0

0.2

0.4

0.6

0.8

1

1.2

0 1000 2000 3000 4000 5000 6000 7000

Incremental Cost per Hu

Probability Cost < or = X Value

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 A preliminary test of the risk-based HSI model used data provided in the USFWS description of the mottled duck model. Varying the 27

input variable values in this document as described above led to quite a different result. Because the HSI in this model is simply the minimum of three

life requisite SI’s, a single SI always determined the HSI. In an extreme case like this an analyst need gather data only on a that single constraining

input variable, the one that determined the controlling SI in this preliminary case. The point to be drawn from this is that there is a great deal that can

be learned about HSI models from risk analysis. Some of this may be helpful in directing data collection and other strategies.

86

The cumulative distribution function (CDF) indicates that taking all the considered uncertainty into account there

is only a 10 percent chance costs will exceed this level. Thus, we’re about 90 percent sure the project will result

in acceptable costs per unit of output. These are a few examples of the results of a risk analysis that could lead

to a better informed decision process.

The range in the HSI, HU and cost values is due largely to the structure of the HSI model . In this case, 27

once life requisite indices are calculated for cover, food, and other (i.e., disturbance), the minimum of the three

becomes the HSI. Although model uncertainty is not the focus of this example, a simple sensitivity analysis of

this HSI formula was conducted to demonstrate the feasibility of such analyses.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

87

In addition to examining the decision rule of selecting the minimum life requisite SI as the HSI, two

alternative decision formulas were investigated to determine the sensitivity of project outputs to the HSI formula.

The first was a simple average of the three SI’s. The second was the cube root of the product of the three SI’s.

There are substantial differences among the means, as well as the extreme values that result from such a

sensitivity analysis as Table 8 shows. “Minimum” means the HSI is based on the minimum of the three life

requisite SI’s. “Average” means the HSI is the average of the three life requisite HSI’s. “Cube Root” means the

HSI is based on the cube root of the product of the three life requisite SI’s. Using an average of the SI’s leads

to substantially higher HU costs. The cube root formula could lead to cheaper HU’s but, on average, HU costs

would be expected to increase.

This is a single example and generalizations cannot be made from it. It’s a hypothetical example that

offers only anecdotal evidence about the mottled duck HSI and no evidence about other HSI models. The

example is sufficient, however, to make the point that model uncertainty can be a significant concern in the

estimation of HSI’s. 

There are many points in an HSI model where somewhat arbitrary looking formulas are used to weight

variable SI’s or life requisites SI’s. These formulas ultimately determine the HSI’s and the change in HU’s. 

Table 8 indicates that estimates of project outputs are very sensitive to how life requisite SI’s are

combined into an HSI. These alternatives were used as an arbitrary test of the sensitivity of the HSI to the

decision rule. They were not used to suggest the model is incorrect, only that the results are sensitive to the

model’s structure. Investigating the cumulative sensitivities of HSI’s to these various formulas as well as to the

suitability graphs and input data would seem to be a fruitful area of future risk analysis research. 
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

 Using cost per HU as the dependent variable, single variable linear regressions were run using each input variable as the independent 28

variable. The graph reports those input variables with the largest significant coefficients. 

88

Table 8: Model Sensitivity Based on Choice of HSI Formula

Rule Minimum Mean Maximum

HU’s: Minimum 281 456 750

HU’s: Average 95 168 291

HU’s: Cube Root 156 360 842

$/HU: Minimum $2,432 $4,350 $6,479

$/HU: Average $6,278 $11,758 $19,257

$/HU: Cube Root $2,168 $7,330 $11,711

Risk analysis has the ability to indicate which variables are most important in the determination of the

HSI’s, the HU’s, or the incremental cost per HU. Figure 14 shows the relative importance of variables

influencing the cost of HU’s . The most important variable was the percentage of submerged substrate under 28

the without project condition, which has a strong positive influence on costs. The second most influential variable

was the percentage of submerged substrate under with project conditions. Variable V5 without the project, V2

with the project, and V without the project, are the other influential variables shown in the tornado graph. 7
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

89

Results like these may be helpful in providing insight into the workings of HSI models. It would surely

be helpful to know which variables are most important before data are collected. Such information could be very

useful in developing data collection strategies. Even after the fact this information is useful. It suggests that if

we want to improve our estimates of HU costs it would be best to get careful estimates of the percentage of

submerged substrate.

The example presented here establishes at least three facts. First, it is relatively easy to use spreadsheetbased Monte Carlo/Latin Hypercube simulation to introduce risk analysis into the estimation of HSI’s and HU’s

based on varying data inputs. Second, the results of at least some HSI computations will be extremely sensitive

to the HSI model’s structure. This implies that HSI model uncertainty may be important enough to warrant future

investigation. Third, the results of risk analysis can help analysts understand the relative importance of the

various uncertain factors. This information may be helpful in data collection strategies, plan formulation, and

incremental cost estimation. 

The possibilities for introducing risk analysis to environmental restoration projects does not end with

this brief discussion of HSI models. For example, at this writing a beta version (2.6) of ECO-EASY: Cost

Effectiveness and Incremental Cost Analysis software is available to Corps personnel. It marks a significant

contribution to the evaluation of environmental restoration projects. It is, however, a deterministic evaluation

that could possibly be modified to incorporate variation in the expected number of habitat units produced by a

project as well as to reflect uncertainty in project costs.

SUMMARY AND LOOK FORWARD

Four major sources of uncertainty in environmental restoration projects have been tentatively identified.

They are project performance, extrapolation, models, and the habitat evaluation procedure. A simple example

was presented illustrating how risk-based analysis could be used to address some of the uncertainty inherent in

the HEP analysis that is so common to environmental restoration projects. Selected results from this analysis

were presented to demonstrate the types of outputs that can result from a risk-based analysis. A few suggestions

were offered to show how these outputs might enlighten and improve the decision process.

The report concludes with the summary found in the next chapter.
90
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

91

CHAPTER SEVEN: SUMMARY AND CONCLUSIONS

First steps are always frustrating. It takes so long to work up the courage to take one, then, once taken,

there are so many places we’d like to go, and so quickly. Introducing risk and uncertainty analysis into the

evaluation of environmental restoration projects now stands where introducing risk and uncertainty analysis in

general stood with the Corps a decade ago. The concepts are vague. Illustrative examples using specific tools

and methods field personnel can use are yet to be developed. The required expertise does not yet exist in many

districts. The costs to the study are unknown. The impacts on plan formulation and project construction are

speculative but largely feared. The motivation for the change is poorly understood and widely suspected. The

people who will have to embrace the changes may be unconvinced of the need to do so and appear resistant. And

few can imagine how any of this could possibly improve decisions and decision-making.

To combat these, and other attitudes, two things are absolutely clear. First, any risk and uncertainty

analysis must be purposeful. Second, it must be scaled to the study.

A purposeful risk and uncertainty analysis will improve decision-making. This has always been the

primary purpose of risk and uncertainty analysis. Risk analysis, once implemented, will help us to avoid errors.

In a world of scarce program resources it is essential that bad investments be avoided. It is equally essential that

good investments not be overlooked. Simple analyses, such as the example presented in the preceding chapter,

present a full range of potential results that provide decision-makers with a much more complete picture of the

range of potential project outcomes. The true value of such analysis will not be evident until the tools are used.

Risk and uncertainty analysis will subject studies to more scrutiny. They will require new analysis and

possibly some more data, but these are reasonable costs to incur to improve decision-making if the analysis is

scaled to the study. Environmental restoration analyses, like other Corps studies, vary in size, significance,

priority and budget. It would be inappropriate, for example, to require analysts to address all the potential

sources of uncertainty identified in this report for any environmental restoration study. For example, although

model uncertainty may be a significant source of uncertainty in the HEP analysis, the Corps analyst does not

always have influence over the development of these models. To address these concerns could require interagency cooperation that is well beyond the scope of agency needs at this point. When Corps analysts develop

the HSI models it would be prudent to consider model uncertainty. 

It is more important to look for things that can be done easily to improve decision-making. Section 1135

studies were to be done based largely on existing data. Habitat variable measurements in these studies may be

along the lines of, “It looks like 40 percent to me, how about you?” In such cases, it would be inappropriate to

require detailed field measurements to limit the uncertainty attending these “eyeball” estimates, but it would not

be unreasonable to make that subjective estimate as an interval. “That’s somewhere between 30 and 45 percent,

don’t you think?” As the study’s scope, budget, and significance increase there may be compelling reasons to

increase the sophistication of the analyses, but only if those changes can be demonstrated to improve the quality

and timing of decision-making.

This report has attempted to advance the discussion of introducing risk and uncertainty analysis into the

evaluation of environmental restoration projects. It has done that by providing some overview discussion of
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

92

environmental planning and risk and uncertainty analysis. It has considered generic and specific potential sources

of uncertainty likely to be encountered in the environmental planning process. A simple example has shown the

feasibility of developing low cost methods for using some risk-based analysis.

At least two subsequent steps appear warranted. The first and most important is to expand the

discussion of introducing risk and uncertainty into environmental restoration analysis. The attitudes mentioned

at the opening of this section can only be addressed as the education and debate processes proceed. This report

may find some role in furthering that education and debate.

The second step would appear to be to demonstrate the feasibility and purpose of risk and uncertainty

analysis by applying it to an on-going study effort. Nothing succeeds like success, it has been said. And nothing

will advance the cause of risk-based decision-making in environmental restoration projects as well as a successful

demonstration of its ability to contribute positively to the decision-making process.
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

93

REFERENCES

Memorandum for Chief, CECW-AO/EINARSEN from CEWRC-IWR-P dated 16 November 94. Subject: 

Draft Administration “Principles for Risk Assessment, Management and Communication.”

Morgan, M. Granger and Henrion, Max. 1990. Uncertainty A Guide to Dealing with Uncertainty in

Quantitative Risk and Policy Analysis. New York: Cambridge University Press.

Rorabaugh, James C. And Zwank, Phillip J. 1983. Habitat Suitability Index Models: Mottled Duck,

Performed for Fish and Wildlife Department, U.S. Department of the Interior, Washington, D.C.

Schroeder, Richard L. and Haire, Sandra L. 8 February 1993. Guidelines for the Development of CommunityLevel Habitat Evaluation Models. Prepared for the U.S. Fish & Wildlife Service.

U.S. Army Corps of Engineers, Institute for Water Resources. 1992. Guidelines for Risk and Uncertainty

Analysis in Water Resources Planning Volume I Principles with Technical Appendices, Appendix

I. IWR Report 92-R-1. Alexandria, VA.

U.S. Army Corps of Engineers, Institute for Water Resources. 1995. A Plan Formulation Manual (Draft),

prepared for the U.S. Army Corps of Engineers, Institute for Water Resources. Alexandria, VA. 

U.S. Army Corps of Engineers, Institute for Water Resources. 1995. Compilation and Review of Completed

Restoration and Mitigation Studies in Developing an Evaluation Framework for Environmental

Resources, Volumes I and II, IWR Report #95-R-4 and 5. Alexandria, VA. 

U.S. Environmental Protection Agency and U.S. Army Corps of Engineers. 1995. Ecosystem Restoration

in the Civil Works Program, Appendix C. EC 1105-2-210. 
94
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

95

INDEX

AHP .......................................................................... xiv, 41, 42

analytic hierarchy process ................................................................ 41

analytical solutions .............................................................. viii, 35, 36

beta distribution ..................................................................... 49, 52

binomial distribution ................................................................. 50, 52

carrying capacity .............................................................. 69, 71, 75, 76

chi-square ............................................................................. 46

classical probability ..................................................................... 43

cluster sampling ........................................................................ 51

critical variables ............................................................... 43, 53, 60, 68

decision criteria .............................................................. viii, 19, 36, 38

decision problem ....................................................... xiv, 13, 38, 39, 41, 53

decision trees ................................................................... viii, 38, 55

decision variables .................................................................... 18, 75

defined constants ....................................................................... 18

descriptive statistics .................................................................. 28, 56

discrete distribution .................................................................. 51, 52

dose-response assessment ................................................................ 10

ecological risk assessment ....................................................... vii, 10-12, 29

ecological risk paradigm ................................................................. 28

ECO-EASY ....................................................................... 89, A-1

EEIRP .............................................................................. v, 7

EIS .................................................................................. 31

elicitation ............................................................................. 41

empirical probability .................................................................... 43

empirical quantities ..................................................... xiv, 17, 18, 21, 77, 78

environmental health risk ................................................................. 10

environmental impact statement ........................................................... 31

environmental investments ................................................ 1, i, v, 2, 7, 9, 22, 68

environmental outputs .................................................................... 1

environmental planning ............ vii, viii, 1-3, 5-7, 9, 14, 15, 24, 25, 27, 28, 35, 41, 43, 51, 54, 57, 59,

60, 66, 92

environmental quality .................................................................... 6

environmental restoration ...... vii, viii, 2, 6, 7, 9, 12, 16, 27, 31, 32, 46, 51, 57, 60-67, 69, 71-73, 76, 79,

80, 85, 89, 91, 92

environmental risk analysis .................................................. 1, 2, 10, 11, 15, 70

environmental risk assessment ......................................................... 10, 28

EPA ............................................................... 10-12, 29, 30, 34, 56, 70

EQ ................................................................................... 6

Evaluation of Environmental Investments Research Program ................................ v, 7, 68

evaluation species ............................................................. 74, 75, 78, 80
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

96

event trees ......................................................................... 38, 39

expert opinion ............................................................ viii, 36, 40, 76, 80

exponential distribution ............................................................... 48, 52

exposure assessment ................................................................. 10, 34

extrapolation .......................................................... viii, 29, 67, 69, 70, 89

frequency distributions ............................................................ 43, 45, 56

full documentation ...................................................................... 29

gamma distribution .................................................................. 49, 52

goodness-of-fit measures ................................................................. 46

Green Book ........................................................................... 14

habitat evaluation procedures ...................................................... viii, 72, 73

habitat suitability index ................................. vii, viii, 3, 13, 19, 20, 48, 72, 77, 80, 81, 93

habitat units .............................. viii, 17, 19, 24, 48, 59, 62, 67, 68, 73, 77-79, 82, 89, A-3

habitat variables ................................................. 20, 63, 72-79, 81, 83, A-1, A-3

hazard identification ................................................................. 10, 12

HEP .............................................. xiv, 2, 3, 20, 24, 45, 45, 56, 67, 71-80, 89, 91

HSI ............................. xiv, 3, 20, 43, 53, 56, 69, 71-78, 80-82, 84, 85, 87-89, 91, A-1, A-3

human health risk assessment ............................................................. 10

Hurwicz criterion ....................................................................... 38

index variables ......................................................................... 19

inferential statistics ............................................................... 22, 56, 57

influence diagrams .................................................................. viii, 38

inherent randomness .............................................................. 23, 77, 78

Kolmogorov-Smirnov ................................................................... 46

laplace criterion ........................................................................ 37

Latin Hypercube ....................................................... viii, 34, 56, 82, 85, 89

LD50 .......................................................................... 10, 13, 70

life requisites ...................................................... 20, 76, 77, 81, 82, 87, A-3

lognormal distribution ................................................................ 48, 52

logtriangular distribution ................................................................. 50

maximax criteria .................................................................... 37, 38

maximin criterion ....................................................................... 37

minimax criterion ....................................................................... 37

minimin criterion ....................................................................... 37

model components ............................................................... 73, 82, A-3

model domain .............................................................. 17, 19, 74, 80, 84

model uncertainty ............................................ vii, 19-21, 71, 75-78, 80, 87, 89, 91

monitoring ...................................................................... 32, 69, 79

Monte Carlo ........................................................ viii, 34, 36, 53, 55-57, 89

multi-criteria decision ................................................................ 34, 41

national economic development ....................................................... 7, 19, 65

National Environmental Policy Act of 1969 ................................................... 1

NED ............................................................................ 7, 19, 62

NED objective ....................................................................... 7, 62
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

97

NEPA ................................................................................. 1

No Observed Adverse Effect Level ......................................................... 13

NOAEL .............................................................................. 13

nontoxic risks .......................................................................... 11

normal distribution ......................................................... xiv, 43, 45-49, 52

opinion analysis ........................................................................ 41

outcome criteria ........................................................................ 19

outrage factors .................................................................. xiv, 31, 33

P&G ................................................................................. 60

pairwise comparisons ................................................................ 41, 42

parametric variation ............................................................... 18, 52, 53

peer review ................................................................... vii, 29, 31, 76

planning ......... vii, viii, xiv, 1-3, 5-7, 9, 12, 14-21, 24, 25, 27, 28, 32, 33, 35, 36, 40, 41, 43, 48, 50, 51,

54, 57, 59-62, 64-69, 71, 74, 76, 79, 85, 92, 93

planning steps ....................................................................... 5, 62

poisson distribution ............................................................... 48, 49, 52

probability ............. viii, xiv, 12, 17-19, 23, 28, 36, 37, 39, 41-46, 48, 50-53, 55-57, 80, 82, 84, A-1

probability distribution ................................ xiv, 18, 23, 44-46, 48, 53, 55-57, 80, 82, 84

professional judgment ....................................................... 40, 41, 53, 60, 75

publicity ........................................................................... vii, 32

quadrat sampling ....................................................................... 51

random error .......................................................................... 21

regret criteria .......................................................................... 38

risk .......... i, v, vii, viii, xiii, xiv, 1-3, 7, 9-17, 21, 22, 24, 25, 27-36, 40, 41, 43, 46, 51, 53, 56, 57, 59,

60, 64-67, 69-71, 73, 77, 80, 82, 87-89, 91-93, 98, A-1

risk analysis ......... vii, viii, xiv, 1, 2, 7, 9-15, 22, 27-31, 34-36, 43, 57, 60, 69-71, 73, 80, 82, 87-89, 91

risk and uncertainty analysis ..... vii, viii, 1-3, 7, 9-15, 24, 27-30, 32, 35, 36, 51, 59, 60, 64-67, 73, 91, 92

risk characterization .................................................................. 10, 12

risk communication ............................................................ vii, 27, 29-33

risk evaluation .......................................................................... 9

risk management .................................................................. 9, 10, 30

risk-based analysis ................................... viii, 13-15, 28, 40, 41, 53, 66, 80, 89, 92, A-1

sampling error ......................................................................... 21

Section 1135 .................................................... 2, 6, 7, 63, 67, 76, 80, 82, 91

sensitivity of the analysis ................................................................. 52

sequential sampling ..................................................................... 51

simple random sampling ................................................................. 51

simulation ............................................. viii, xiv, 31, 34-36, 39, 53-56, 85, 89, A-1

statistical variation ............................................................. 21, 77, 78, 80

statistics ............................................................ 22, 28, 34, 46, 47, 56, 57

stratified random sampling ............................................................... 51

stress-response assessment ............................................................... 11

study area ............................................ viii, 17-19, 21, 36, 56, 73-75, 77-80, 82-84

subjective judgments ................................................................. 22, 42
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

98

subjective probabilities ............................................................... 41, 43

subjective probability ............................................................. 36, 41, 43

subjective probability assessments ......................................................... 41

suitability graphs .......................................................... 20, 71, 81, 87, A-1

suitability index ................................... vii, viii, 3, 13, 17, 19, 20, 48, 72, 77, 80, 81, 93

systematic errors ....................................................................... 22

systematic sampling ..................................................................... 51

T/E ratio .............................................................................. 10

target species ....................................................... viii, 20, 29, 69, 72-75, 77

triangular distribution ................................................................ 50, 52

two-stage cluster sampling ............................................................... 51

U.S. Environmental Protection Agency ................................................ 10, 31, 93

U.S. Fish and Wildlife Service ........................................................ 56, A-1

uncertainty .......... 1, i, v, vii, viii, xiv, 1-3, 7, 9-24, 27-32, 35, 36, 39-43, 46, 47, 51, 53, 55-57, 59-71,

73-80,84, 85, 87, 89, 91-93, A-1, A-3

uniform distribution ..................................................... 50, 52, 55, 82, 83, A-1

value parameters ....................................................................... 18

Weibull distribution .................................................................. 49, 52
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

APPENDIX A:

MOTTLED DUCK RISK MODEL
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

A-1

INTRODUCTION

The risk-based analysis presented in Chapter Six is based on the spreadsheet model described in this

appendix and shown at Figure A-1. The model was built using Excel v. 5.0 and @RISK v. 3.1. It was built to

demonstrate the feasibility of doing such an analysis. As such, there are redundancies and inefficiencies that

could be eliminated in future versions. Or, the basic rationale could be adapted to existing HEP or ECO-EASY

software. The explanations that follow assumes a basic familiarity with the structure of an HSI model.

The example discussed below is a single iteration of a simulation. The simulations used for this report

consisted of 1,000 iterations. Each iteration begins with a new set of randomly generated values for the habitat

variables. These, in turn, result in a different set of output values.

THE MODEL

Rows 3 through 24 reproduce the suitability graphs found in the U.S. Fish and Wildlife Service (FWS)

HSI model for the mottled duck. Column A presents the percentage values that correspond to the SI’s for each

of the five habitat variables contained in columns B through H. A more efficient model would simply reproduce

the function. For example, all of the simple linear functions could be replaced by a simple function of the form:

y = mx + b. The table format is easier to follow, so it was used for demonstration purposes.

Rows 27 through 33 present values for classes that comprise habitat variables V , V , and V under the 3 5 8

without project condition. The SIs for these variables are weighted sums of the four classes of conditions.

Definitions of the classes and details of the weighting function can be found in the mottled duck HSI model. The

classes are defined as random variables. For example, cell B32, class 4 for V3, is a uniform distribution ranging

from 0.9 to 1. Class 3 for V3 is 1 minus the class 4 variable. In this iteration there was 94 percent Class 4 and

6 percent Class 3 for the V3. There were no observed class 1 or 2 conditions for V3 in the Corps study upon

which this analysis was built.

Rows 34 through 40 repeat the process for with project conditions. The difference this time is that the

percent of each class observed in the project area will vary. For example, consider the column C values for V .5

Without the project this variable comprises classes 2 and 3 only. With the project it comprises classes 3 and 4

only. Changes like these reflect project impacts.

Rows 42 through 51 present measurements of the habitat variables displaying the without project

condition values in column B and the with project values in column D. These values are all subject to uncertainty

and are estimated using probability distributions that reflect this uncertainty. In this example uniform

distributions were used for simplicity for all variables. Variables V and V must logically sum to one so once 6 7

one variable is randomly determined the other is known. 
1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

A B C D E F G H

Figure A-1: Mottled Duck HSI Model with Risk Analysis

Rows 3 through 24 display linear suitability index graphs

Percent SI for V1 SI for V2 SI for V4 SI for V6 SI for V7

0 1 1 0 0 0

5 0.95 0.8333 0.1176 0.125 0.05

10 0.9 0.6666 0.2353 0.25 0.1

15 0.85 0.5 0.3529 0.375 0.15

20 0.8 0.3333 0.4706 0.5 0.2

25 0.75 0.1667 0.5882 0.625 0.25

30 0.7 0 0.7059 0.75 0.3

35 0.65 0 0.8235 0.875 0.35

40 0.6 0 0.9412 1 0.4

45 0.55 0 1 1 0.45

50 0.5 0 1 1 0.5

55 0.45 0 1 1 0.55

60 0.4 0 0.9412 1 0.6

65 0.35 0 0.8235 0.875 0.65

70 0.3 0 0.7059 0.75 0.7

75 0.25 0 0.5882 0.625 0.75

80 0.2 0 0.4706 0.5 0.8

85 0.15 0 0.3529 0.375 0.85

90 0.1 0 0.2353 0.25 0.9

95 0.05 0 0.1176 0.125 0.95

100 0 0 0 0 1

Rows 27 through 40 allow for input of without and with project condition values for habitat

values based on class/step functions rather than linear suitability graphs.

W ithout Project Enter % of each class as decimal

% for V3 % for V5 % for V8

class 1= 0 0 0

class 2 = 0 0.92988066 0

class 3 = 0.061555 0.07011934 0.040817

class 4 = 0.938445 0 0.959183

Sum 1 1 1

W ith Project Enter % of each class as decimal

% for V3 % for V5 % for V8

class 1= 0 0 0

class 2 = 0 0 0

class 3 = 0.0341965 0.08156815 0.0315256

class 4 = 0.9658035 0.91843185 0.9684744

Sum 1 1 1

Rows 42 through 51 present measurements of the habitat variables and their corresponding SI's.

Model W ithout Project W ith Project

Component Data SI Data SI

V1:% Unsubmerged substrate 4.8841023 1 1.5245874 1

V2: % Canopy cover 8.4760538 0.8333 7.8370752 0.8333

V3: Unsubmerged substrate vegetation 0.975378 0.98632142

V4: % Submerged substrate 5.4579538 0.1176 41.333654 0.9412

V5: Submerged substrate vegetation 0.3210358 0.96737274

V6: % Land 25.410497 0.625 25.410497 0.625

V7: % Water 74.589503 0.7 74.589503 0.7

V8: Disturbance level 0.98367319 0.98738976

Rows 53 through 56 calculate model components, (W/ and W/O). Life requisites are calculated in rows 57 through 59.

Nesting Hen Cover ((NHC) 0.9332359 0.9367131

Hen with Brood Cover (HBC) 0.1943034 0.9541966

Cover Structure 0.1943034 0.9367131

Cover Ratio 0.625 0.625

Cover Life Requisite 0.2868228 0.8185234

Food Life Requisite 0.7 0.7

Other Life Requisite 0.9836732 0.9873898

Model outputs, including HSI values, HU's, costs, and sensitivity analysis results are presented below.

Habitat Suitability Index: Minimum 0.2868228 HSI W/Min 0.7 HSI Change 0.4131772 Incremental

HSI: Average 0.656832 HSI W/Ave 0.8353044 0.1784724 Cost per

HSI: Cube Root 0.5823546 HSI W/CR 0.8270642 0.2447097 H U

Habitat Units W/O: Minimum 286.82275 HU W/Min 700 CH HU Min 413.17725 $/HU Min 4414.5703

H U 's W /O: Average 656.83198 HU W/Ave 835.30437 Ch HU Ave 178.47239 $/HU Ave 10220.068

H U 's W /O: Cube Root 582.35458 HU W/CR 827.06425 CH HU CR 244.70967 $/HU CR 7453.7308

Cost 1824000

An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

A-2
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

A-3
An Introduction to Risk and Uncertainty

in the Evaluation of Environmental Investments

A-4

Columns C and E present the suitability indices for the habitat variable values presented. The SI values

for variables V , V , and V are based on the values presented in rows 29 through 40 above. Notice that most 3 5 8

SI values differ between the without and with project conditions.

Model components and life requisites are calculated in rows 53 through 59. The components in rows

53 through 56 are functions of the habitat variables. The specific details of these functions can be found in the

HSI model and are discussed briefly in Chapter Six. The life requisites are functions of the components and

habitat variables. These are all straightforward calculations or equalities, e.g., the food life requisite is simply

the SI of V . Life requisites are shown in rows 57 through 59. 7

The HSI is defined as the minimum of the three life requisite SIs. Under the without project condition

in the example this would be the cover life requisite of 0.29 (cell B57). Under the with project condition it is the

food life requisite of 0.70 (cell D58). Thus, in this example the HSI improves from 0.29 to 0.70 as a result of

the project, a change of 0.41 as shown in cell F61.

Row 62 tests the sensitivity of the HSI to the decision rule, i.e., choosing the minimum life requisite, by

taking the average of the three HSI’s. The with project HSI improves from 0.66 to 0.84 a change of 0.18. A third

decision rule, the cube root of the product of the three life requisites, yields values of 0.58 and 0.83 for a change

of 0.24, as shown in row 63.

The habitat units corresponding to each of these three decision rules are presented in rows 64, 65 and

66. For simplicity it was assumed the mitigation area consisted of 1,000 acres. Thus, the HSI of cell B61 times

1,000 acres yields 287 HU’s (cell B64) without the project and 700 HU’s (cell D64) with the project for a change

in HU’s (cell F64) of 413 HU’s. Row 64 presents results using the model decision rule while rows 65 and 66

present alternative results.

The Corps project used for this example did not use a true incremental cost analysis. It basically used

a single increment of 1,000 acres. Costs were assumed to be determinate so this demonstration could focus on

environmental input data. They could have easily been allowed to vary to reflect the uncertainty inherent in them.

Incremental costs for this single increment are presented in cells H64 through H66. Using the model’s decision

rule of the minimum life requisite as the HSI, costs are $4,415 per HU. In this iteration the other rules yield

incremental costs of $10,220 and $7,454.