United States Department of Agriculture

The Science of Decisionmaking:

Applications for Sustainable Forest and Grassland

Management in the National Forest System

Forest 

Service

General Technical

Report WO-88

July 2013
The U.S. Department of Agriculture (USDA) prohibits discrimination in all its programs and activities on the basis of race, color, national 

origin, age, disability, and where applicable, sex, marital status, familial status, parental status, religion, sexual orientation, genetic information, political beliefs, reprisal, or because all or part of an individual’s income is derived from any public assistance program. (Not all 

prohibited bases apply to all programs.) Persons with disabilities who require alternative means for communication of program information (Braille, large print, audiotape, etc.) should contact USDA’s TARGET Center at (202) 720-2600 (voice and TDD). To file a complaint 

of discrimination, write USDA, Director, Office of Civil Rights, 1400 Independence Avenue, S.W., Washington, D.C. 20250-9410, or call 

(800) 795-3272 (voice) or (202) 720-6382 (TDD). USDA is an equal opportunity provider and employer.

The use of trade, firm, or corporation names in this document is for the information and convenience of the reader, and does not 

constitute an endorsement by the Department of any product or service to the exclusion of others that may be suitable.

Cover photographs from http://www.thinkstockphotos.com.

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

Stages of the structured decisionmaking (SDM) process (Marcot et al. 2012a).
The Science of Decisionmaking:

Applications for Sustainable Forest and Grassland

Management in the National Forest System

General Technical Report WO-88

July 2013

Matthew P. Thompson, Bruce G. Marcot, Frank R. Thompson, III, Steven McNulty, 

Larry A. Fisher, Michael C. Runge, David Cleaves, and Monica Tomosy 
Who Should Read This Report? 

1. Decisionmakers (including line officers, staff officers, and other managers) and 

policymakers (including agency leadership, station directors, etc.), and

2. Analysts (including planning specialists, modelers, interdisciplinary teams, etc.).

Section 1.2 “A Road Map for Reading This Report” reviews the contents and intended audience 

of each chapter. 

What Information Does the Report Contain?

Briefly, this report is divided into five main sections, each color coded as follows:

The first section of the report—Chapters 1 and 2—provides an overview of structured decisionmaking in the 

context of forest and grassland management. This section will be most useful for decisionmakers.

The second section—Chapter 3—provides greater detail on the structured decisionmaking framework and 

related decision support approaches. This section will be most useful for analysts.

The third section—Chapters 4 and 5—reviews applications of structured decisionmaking to forest and 

grassland management, including opportunities, challenges, and case studies. This section will be most useful 

for decisionmakers and policymakers.

The fourth section—Chapter 6—summarizes the major conclusions and implications of the report, 

useful for all audiences.

Lastly, appendixes provide additional information to assist decisionmakers and analysts in tackling 

tough problems.
Contents 

1. Introduction

1.1. Why Focus on Decisions and Decision Science?...............................................................................3

1.2. A Road Map for Reading This Report.................................................................................................3

2. The WHY of Structured Decisionmaking: A Guide for Decisionmakers

2.1. Decisionmaking Is at the Core of Sustainable Forest and Grassland Management...........................5

2.2. Science-Based Decisionmaking Includes the Science of Decisionmaking.........................................5

2.3. The Structured Decisionmaking Framework Is a Template for Understanding, Analyzing, 

 and Implementing Decisions ...........................................................................................................7

2.4. Why Use the Structured Decisionmaking Framework?.......................................................................9

3. The HOW of Structured Decisionmaking: A Guide for Decisionmakers and Decision Analysts

3.1. The Structured Decisionmaking Framework—A Closer Look...........................................................10

3.2. Stage One: Problem Structuring .......................................................................................................11

3.2.1. Problem Framing...............................................................................................................................11

3.2.2. Defining Objectives...........................................................................................................................12

3.2.3. Defining Evaluation Criteria ..............................................................................................................14

3.3. Stage Two: Problem Analysis ............................................................................................................15

3.3.1. Defining Alternatives .........................................................................................................................17

3.3.2. Evaluating Consequences ................................................................................................................17

3.3.3. Identifying Key Uncertainties ............................................................................................................19

3.3.4. Tradeoff Analysis...............................................................................................................................20

3.4. Stage Three: Decision Point..............................................................................................................22

3.5. Stage Four: Implementation and Monitoring.....................................................................................25

3.5.1. Implement Preferred Alternative........................................................................................................25

3.5.2. Monitoring Results of Implementing the Decision.............................................................................26

3.5.3. Adaptive Management......................................................................................................................26

3.6. A Guide to Identifying Appropriate Decision-Support Tools and Approaches ...................................27

4. Opportunities and Challenges for Adopting Decision Science Principles in the National Forest System

4.1. A Survey of the National Forest System on Use of Structured Decisionmaking...............................29

4.2. Survey Results on Decision-Support Methods and Impediments to Their Use ................................32

4.3. Survey Results on Needs for Structured Decisionmaking Training...................................................33

4.4. Institutionalizing Structured Decisionmaking: Recommendations for Policymakers .........................34
5. Applications of Decision Science Principles in Forest and Grassland Management

5.1. Case Study 1: Problem Structuring: The National Cohesive Wildland Fire Management Strategy 

 (Cohesive Strategy).......................................................................................................................35

5.2. Case Study 2: Problem Structuring and Analysis: Hoosier National Forest, 

 2006 Land Management Plan .......................................................................................................36

5.3. Case Study 3: Decision Point: Deschutes National Forest, Five Buttes Project ................................36

5.4. Case Study 4: Implementation and Monitoring: Tongass National Forest, 

 Implementation and Monitoring Plan.............................................................................................38

5.5. Case Study 5: Monitoring and Adaptive Management: The Survey and Manage 

 Program of the Northwest Forest Plan...........................................................................................39

5.6. Case Study 6: The Full Picture: Nonnative Fish Control Below Glen Canyon Dam...........................40

6. Adopting Structured Decisionmaking: Conclusions and Implications

6.1. A Change of Paradigm......................................................................................................................42

6.2. Contemplating the Role of Uncertainty.............................................................................................42

6.3. The Challenge and the Promise Ahead ............................................................................................42

6.4. Summary ..........................................................................................................................................42

Acknowledgments.....................................................................................................................................43

References .................................................................................................................................................44

Appendix A: Guidance for Tough Choices..................................................................................................47

Appendix B: Tools and Approaches Useful in Stages of Structured Decisionmaking, 

 With Examples of Applications ......................................................................................................49

References-Appendix B: .........................................................................................................................52
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System v

Foreword

At the start of the 21st century, as land management options 

have narrowed, as issues have become more complex, and as these 

decisions have become more difficult, many managers are turning to 

processes that examine and evaluate a problem in a more structured 

way. Managers and administrators are seeking objective, replicable, 

and explicit ways to assess choices and the probable outcomes of 

those choices to make the most optimal choice. The scientific 

concepts and techniques of risk management and structured 

decisionmaking have grown dramatically, making it challenging for anyone who does not specialize in this 

field to keep up with the pace of this advancement.

This General Technical Report (GTR) presents highlights from the existing body of work, illustrations of 

the relevance to land management decisionmaking, and tips on how to enhance the effectiveness of each 

aspect within the entire process. As a result of increased natural resource demands and expectations from 

the public, agencies, and natural resource managers for “sound decisions based in science,” the demand for 

decision science expertise has grown. Hence, we are excited about this synthesis of decision science for land 

management.

Decision science principles and their component decision-support approaches and tools have become 

bridges between the “process” of decisionmaking and the “content” of environmental sciences. The U.S. Department of Agriculture (USDA), Forest Service and other organizations have intensified efforts to develop 

structured decision processes; we believe those processes provide value in codifying and providing access to 

science and in enhancing problem-solving in complex and uncertain environments more effectively. We are 

pleased that this report has explored the intersection between “process” and “content” sciences in the design 

and use of decision support and in the context of National Forest System (NFS) decisionmaking.

When we asked the authors to take stock of our body of decision-support systems and tools, they organized 

their response under an anatomical framework for generalized decision process that is well established and 

documented in the decision sciences. Their work has opened a whole new perspective on our investments in 

decision science and science for decisions. The authors took the charge further by surveying users to better 

understand how they put these tools to use. The result has been an integrative approach to better understanding how decision-support systems and tools supplement different phases of decision processes, how we 

can design them to better incorporate science, and how we can better frame and meet land managers’ needs 

for scientific knowledge and expertise.
vi The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

The approach in this report provides a conceptual meeting place for research and management partnerships. 

If scientists and tool developers can better understand the nature, limitations, and potentials of human decisionmaking—individual, group, and organizational—they can better interpret present findings and frame 

new science questions around decision-relevant knowledge gaps. Managers likewise can more realistically 

judge the applicability of science and science-based tools to the choices they confront and gauge how their 

own judgment and choice processes influence the interpretation and use of that science.

We hope that this report will encourage future efforts to build decision frameworks and support systems 

that combine “smarts” about decisions and environmental science. To maintain and improve high-quality 

attributes of natural resource decisionmaking—consistency, adaptability, transparency, and others—we must 

become more sophisticated with models of our decisions and the environmental functions we hope to sustain. In the era known as the Anthropocene, in which the environment is pervasively influenced by human 

decisions and actions at many scales, we must strive to blend knowledge of decision and environmental 

process toward innovative solutions to challenges.

We offer this report as a window on practice and a stimulus to create new value for science in and of decisions. We hope managers, scientists, researchers, and analysts will use it as a resource and as a centerpiece 

for bringing these important members of the natural resource decision process closer together.

In summary, Forest Service managers saw a need and asked Forest Service Research and Development to 

help fill this need. Our many thanks to the authors who have sewn together a very thoughtful and readable 

piece of work, one integrative stitch at a time. We know their enthusiasm and collaborative spirit will draw 

others to efforts that follow from this report.

JIMMY L. REAVES

Deputy Chief, Research and Development

DAVID A. CLEAVES

Climate Change Advisor, Office of the Chief
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 1

Summary

Sustainable management of national forests and grasslands within the National Forest System (NFS) 

often requires managers to make tough decisions under considerable uncertainty, complexity, and potential conflict. Resource decisionmakers must weigh a variety of risks, stressors, and challenges to sustainable 

management, including climate change, wildland fire, invasive species, insects, pests, diseases, demographic 

shifts, economic conditions, and changing societal values. The craft of natural resource decisionmaking 

will demand more adaptive qualities in the future and a more flexible toolkit. In light of these current and 

emerging challenges and the need for science delivery to keep pace with advancements in the disciplines of 

decision science and risk management, now is a good time to refocus on supporting and enhancing natural 

resource management decisionmaking. The science of decisionmaking can provide managers with useful 

concepts and tools to address risks, stressors, and challenges and to achieve desirable outcomes.

Improving natural resource decisionmaking requires a firm understanding of how decisions are currently 

made and where opportunities for enhancement might exist. To that end, we surveyed U.S. Department of 

Agriculture (USDA), Forest Service line officers, planners, and specialists to gain an understanding of how 

decision-support tools and processes are used, under what circumstances they are used, and what improvements may be needed. Although responses varied, line officers principally expressed a desire for a structured 

decisionmaking framework that is risk informed, transparent, and adaptable; can facilitate evaluation of 

consequences and tradeoffs; and can be readily explained to stakeholders. More than 90 percent of respondents who had used some form of a decision-support approach stated they were satisfied with the outcome, 

and their primary reasons for adopting decision-support approaches were to improve the decision process 

and to mitigate potential social conflict. Their responses suggested that lack of information is the principal 

impediment to broader adoption of structured decisionmaking and decision-support approaches.

We have therefore focused on reviewing the core principles of decision science and how they can be 

brought to bear on issues of sustainable forest and grassland management. This report outlines the rationale 

and pathway for incorporating decision science into the natural resource decisionmaking process. The approach we outline in this report—structured decisionmaking—is a flexible, well-established framework that 

helps decisionmakers better define the problem context, identify and evaluate management options, and 

make informed choices under complex and uncertain conditions. In particular, the initial stage of framing 

the problem is critical for effective problem-solving and decisionmaking. We highlight examples in which 

principles of decision science were incorporated into forest and grassland management. We also provide an 

extensive list of appropriate decision-support tools and approaches as they relate to various stages of decisionmaking processes, as well as guidance for identifying those tools and approaches.

Adopting the structured decision framework can streamline decision processes, help ensure the right 

problem is being solved, lead to high-quality decisions that are defensible and durable under scrutiny, and 

ultimately lead to improved sustainable forest and grassland management. We highlight key knowledge 

gaps and identify training needs that could facilitate broader adoption of decision science principles within 

the NFS. We suggest structured decisionmaking can be more effectively used in forest and grassland 

management if analysts and planners are equipped to provide guidance to leadership and management on 

decision support; scientists clearly explain the use of predictive models in decision contexts and participate 

in decision processes; and decisionmakers and managers foster a collaborative, transparent, and defensible 

basis for decisions, using structured decision frameworks.
2 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 3

1. Introduction

“Conservation is the foresighted utilization, preservation and/or renewal of forests, waters, lands and 

minerals, for the greatest good of the greatest number 

for the longest time.”

Gifford Pinchot

1.1 Why Focus on Decisions and Decision Science?

Decisions—made by humans and rooted in science—are the 

fundamental elements of natural resource management. Often 

neglected in discussions about natural resource decisionmaking is 

the science of the human decision process itself. When faced with 

complex, uncertain decisions, humans can be susceptible to biases, 

shortcuts, and other limitations that can degrade decision processes—but these pitfalls can be avoided. Much has been learned 

in the last 30 years in the field of decision science combining 

contributions from psychology, management science, operations 

research, economics, statistics, and other fields to provide ways to 

improve the quality of decisions and problem-solving processes. 

Decision science provides a sound theoretical basis, and a specific 

framework and method, for making sound decisions under uncertainty by using formal decision analysis techniques and methods 

of risk analysis and risk management. Decision analysis is “a formalization of common sense for decision problems which are too 

complex for informal use of common sense” (Keeney 1982: 806). 

Decision science is applied increasingly in management of natural 

resources, including fisheries, wildlife, forestry, rangeland, and fire 

(Marcot et al. 2012a).

In this report, we present the principles of decision science and 

a well-established, structured decisionmaking (SDM) framework to help natural resource managers and analysts navigate the 

decision process (Gregory and Long 2009, Gregory and Keeney 

2002, Hammond et al. 1999). SDM is not a new tool or model 

in and of itself, but rather an integrated system of principles and 

concepts that link to tools and models and that can improve 

decision processes and outcomes. We review the wide variety of 

decision-support tools and approaches useful for each stage in the 

natural resource decisionmaking process. We perform this review 

through the lens of insights gained from an extensive survey of 

U.S. Department of Agriculture (USDA), Forest Service line 

officers, planners, and specialists describing their experiences 

and information needs. We also conducted a series of indepth 

interviews with selected decisionmakers who have considerable 

experience with these tools and approaches.

The primary objective of this report is to present useful information to help decisionmakers and analysts adhere to decision 

science principles and select appropriate decision-support 

approaches. Managing forests and grasslands of the Forest 

Service’s National Forest System (NFS) lands provides a compelling context for the role of SDM under the complex, multiple-use mandates of the National Forest Management Act and 

its implementing regulations and under mandates of the National 

Environmental Policy Act (NEPA). We do not set out to comprehensively review and critique past agency decisions in light 

of the current state of decision science principles. We do hope to 

make natural resource managers, planners, and scientists more 

aware of the availability of SDM approaches and tools, how they 

have been used successfully in the NFS, and how they can be used 

more effectively for future decisions.

1.2 A Road Map for Reading This Report

This report consists of five chapters, all of which are related but 

can be read largely independent of each other. The chapters vary 

in terms of content, scope, and intended audience, as described 

below. It is our hope that this report will serve as a useful guide 

that managers and analysts can repeatedly turn to for assistance in 

tackling tough natural resource management decisions.

The WHY of Structured Decisionmaking: 

A Guide for Decisionmakers

This chapter addresses the question, “Why should I care about 

structured decisionmaking frameworks?” and is intended largely 

for resource managers and decisionmakers. The section provides 

a broad overview of the structured decision framework and 

describes the potential benefits from formally incorporating decision science principles into decision processes. We describe how 

the decision framework aligns with the NFS Land Management 

Planning Rule promulgated in 2012, as well as NEPA.

The HOW of Structured Decisionmaking: 

A Guide for Decisionmakers and Decision Analysts

This chapter addresses the question, “How should I implement 

structured decisionmaking frameworks?” We describe in more detail the various decision stages and their relation to fundamental 

decision science principles. Although useful for decisionmakers, 

this chapter is primarily targeted to planners, resource specialists, 

and other analysts supporting the decision process. We provide concrete examples as they relate to the stages of SDM and 

describe some major recommended decision-support approaches, 

including objectives hierarchies, influence diagrams, and multicriteria decision analysis. We also provide broader guidance 

for identifying appropriate decision-support tools and methods, 

jointly based on the decision context, the SDM stage, and the 

nature of uncertainties faced.
4 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Opportunities and Challenges for Adopting Structured 

Decisionmaking in the NFS

This chapter addresses the question, “How are we currently making decisions, and how can we improve on decision processes?” It 

is directed toward decisionmakers and policymakers. We present 

results from a survey we recently conducted to better understand 

how and under what circumstances decision-support tools and 

methods are currently being used. We identify opportunities for 

improvement as well as potential challenges to a broader adoption 

of SDM within the NFS. We suggest a need to strengthen the 

agency’s commitment to training, communication, and support 

for structured decision frameworks.

Applications of Decision Science Principles in Forest and 

Grassland Management

This chapter addresses the question, “How has decision science 

been applied to date in addressing natural resource management 

problems?” We highlight selected instances in which principles of 

SDM have been incorporated into forest and grassland management. These case studies are organized primarily according to the 

stage of SDM to illustrate how different decision-support tools 

and approaches can be brought to bear according to the context. 

We present a more comprehensive application of the 

SDM framework to a complex natural resource management 

problem, nonnative fish control, in a much more expansive case 

study from outside the NFS. Decisionmakers will find this 

chapter most useful.

Adopting and Implementing Structured Decisionmaking: 

Conclusions and Implications

We target managers and policymakers with a discussion of the 

major implications of adopting SDM within agency decision 

processes. The weight of evidence from the scientific literature we 

cite describing real-world applications of SDM provides strong 

support for the value of decisions reached through adoption of 

structured decision frameworks. These applications come from 

detailed case studies within the NFS and elsewhere, from survey 

results regarding NFS personnel use of SDM and decision-support tools, from interviews with individuals who have experience 

applying SDM, and from the authors’ collective experience. 

Photo by Bruce G. Marcot

Photo by Larry A. Fisher
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 5

2.1 Decisionmaking Is at the Core of Sustainable Forest 

and Grassland Management

Natural resource decisionmaking is challenged 

by considerable complexity, uncertainty, and 

potential conflict.

Natural resource management is ultimately about making tough 

decisions in the face of uncertainty, complexity, and potential 

conflict. Often the outcomes extend beyond the lifetime of the 

decision process participants, requiring a long-term perspective 

when making these decisions. Decision environments also may 

be time-sensitive and require dynamic responses to changing 

conditions. Decisions sometimes touch values that are deeply felt, 

yet elusive to measure. They involve complex, interacting processes 

that make prediction difficult at best. The problem-solving process is subject to the reasoning ability of individuals and groups, 

human fatigue, competing issues, busy schedules, and the influences of political and organizational power. Not only must natural 

resource decisions be made, but they must also be communicated 

to stakeholders, supporters, detractors, employees, experts, the 

media, legislators, the courts, and others to ensure credibility and 

trust for sustainable outcomes. Providing for sustainable forests 

and grasslands and their ecosystem services requires efficient, 

effective problem solving through structured, transparent human 

decision processes.

Structured decisionmaking: A template for 

high-quality decisionmaking

2.2 Science-Based Decisionmaking Includes the 

Science of Decisionmaking

Natural resource decisionmaking is a broad problem-solving process, not just the choice of an alternative management action (i.e., 

the “decision point”). Natural resource managers and decisionmakers know that much goes on before and after the point of decision in any problem. Science-based decisionmaking recognizes 

a need not only to include science in decisionmaking but also to 

include the science of decisionmaking. Combining the science of 

sustainability (ecological, social, economic) as decision “content” 

with the science of decisionmaking as “process” is a fundamental 

premise of truly science-based decisionmaking.

Process and Content: Science of decisionmaking 

and science in decisionmaking

2. The WHY of Structured Decisionmaking: A Guide for Decisionmakers Photo by Matthew P. Thompson
6 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Natural Resource Decisions

What is a decision?

A decision and its implementation constitute an “irrevocable allocation of resources...not a mental commitment to follow 

a course of action but rather the actual pursuit of the course of action” (Howard 1966). A decision can be the choice of 

a strategic direction, such as a land and resource management plan (LRMP) under the National Forest System Planning 

Rule, or it can be a project-level decision, such as one that entails specific management practices and resources consistent with an LRMP. Throughout the decision process, a number of “small” decisions will be made—for instance choosing 

which sources of data, analysis techniques, and decision-support approaches to use. Similarly, many other circumstances 

exist in which resource managers and specialists face choices regarding research designs, monitoring strategies, etc., 

that may not be in service of a specific “big” decision. Using structured decisionmaking (SDM) principles might help in 

thinking through these choices, and such principles could guide thinking toward related decisions in the future; we principally present SDM as a framework to guide “big” decisions.

Who are decisionmakers?

For the purposes of this report, the term decisionmaker can apply broadly to those who make a wide variety of decisions 

about management of natural resources, including line officers, staff officers, and others, at all administrative levels of the 

agency. The identity of the decisionmaker will depend on the context—for “big” decisions; this decisionmaker will typically 

be a line officer. Resource staff and others, although not serving as the decisionmaker per se, nevertheless have important 

contributions to make at various stages throughout the overall decision process.

How do objectives and desired outcomes differ?

Decisionmakers ultimately choose an alternative based on the choice that best satisfies their established set of objectives. 

Objectives relate to the decisionmaker’s overarching aspirations and desired outcomes. Outcomes, by contrast, are the 

situations that result after the decision is implemented. Outcomes are influenced by management actions but also by 

factors outside of management control, such as the actions of other landowners, weather, and other natural variation. A 

good decision, we hope, will lead to a good outcome, but this outcome is not guaranteed. To quote Howard (1966: page 

56)—”We may be disappointed to find that a good decision has produced a bad outcome or dismayed to learn that someone who has made what we consider to be a bad decision has enjoyed a good outcome. Yet, pending the invention of the 

true clairvoyant, we find no better alternative in the pursuit of good outcomes than to make good decisions.”

So then, what are good decisions?

A good decision is a logical decision that is based on available knowledge, uncertainties, and the values and preferences 

of the decisionmaker. How does one make a good decision? To borrow from the idea expressed above, we find no better 

alternative in the pursuit of good decisions than to adopt good decisionmaking principles.

Principles of high-quality decisionmaking

Good decisionmaking is defined in large part by the decision process and decision content, which are controllable, and not 

necessarily by the ultimate outcome, which may not be controllable. Adopting the principles of decision science helps lead 

to decisions with process and content favorable for a positive outcome and decisions that are robust to scrutiny.

High-quality decisionmaking has the following attributes (Berg et al. 1999):

•	 Clearly and accurately describes the problem.

•	 Establishes objectives and measurable criteria with which to evaluate attainment of objectives.

•	 Effectively uses available information and collects new information wisely.

•	 Generates a range of alternatives and evaluates the consequences of proposed alternatives in terms of 

evaluation criteria.

•	 Examines sensitivities and assumptions and identifies key uncertainties and impediments.

•	 Integrates, but does not conflate, value-based and science-based information.

•	 Clearly and logically documents the decision rationale.

•	 Ensures transparency and accountability.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 7

2.3 The Structured Decisionmaking Framework Is 

a Template for Understanding, Analyzing, and 

Implementing Decisions

Structured decisionmaking is a well-established, organized framework by which decisionmakers better understand and frame their 

decision context, clearly and comprehensively identify and evaluate management options, and make informed choices under conditions of complexity and uncertainty. Structured decisionmaking 

(SDM) is a generic plan for developing customized approaches to 

problem solving, and it is a guide for identifying where decision-support tools and approaches might best fit. The components of this generic plan emphasize the importance of knowing 

one’s location in a decision process, and knowing which tools and 

approaches are available at each stage. SDM also provides a solid 

platform with which to assess and manage risks, with an explicit 

focus on identification of uncertainties and outcome probabilities 

to inform decisionmaking (Haynes and Cleaves 1999).

Application of structured decisionmaking can 

help managers and decisionmakers navigate 

complex decision processes and directly address 

potential sources of conflict and uncertainty.

The SDM framework consists of four primary stages: problem 

structuring, problem analysis, decision point, and implementation and monitoring, and includes several substages and 

feedback loops (figure 1). This framework helps clearly identify 

the appropriate roles of science and values in the decision process 

and provides a template for making and implementing high-quality decisions. Each decision stage can be characterized by a series 

of questions.

Problem Structuring

•	 What is the problem or issue to be solved?

•	 What are the management objectives and desired outcomes to 

be achieved?

•	 What are the spatial and temporal scales of analysis?

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

Figure 1.— Stages of the structured decisionmaking (SDM) process (Marcot et al. 2012a).
8 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

•	 Who is(are) the decisionmaker(s)?

•	 Who should be involved in the decision process, and is 

conflict likely? 

•	 How can progress toward objectives and desired outcomes 

be evaluated?

Problem Analysis

•	 What management options and alternatives are possible 

and available?

•	 What are the predicted consequences of each alternative?

•	 What key uncertainties are presenting an impediment 

to decisionmaking?

•	 To what extent are uncertainties controllable?

•	 What are the tradeoffs across alternatives?

•	 Is it possible to identify and remove underperforming 

alternatives from consideration, and to recraft others to 

provide better performance?

Decision Point

•	 Are there multiple objectives to balance?

•	 If so, what is the relative importance of each objective?

•	 What are managerial attitudes toward risk?

•	 Which alternative(s) best satisfies overall objectives, manages 

risk, and approaches desired outcomes?

•	 Is the selected alternative consistent with the science and with 

stated objectives?

•	 Can the selected alternative be feasibly and cost-effectively 

implemented?

Implementation and Monitoring

•	 What resources are required to successfully implement the 

selected alternative?

•	 What adaptive management principles and procedures are 

appropriate and feasible?

•	 What is the primary purpose for monitoring (achievement 

of objectives, determining system state, reducing future 

uncertainty)?

•	 What is the appropriate monitoring protocol and how will it 

be implemented?

•	 What happens if implementation does not occur?

Two Hallmarks of SDM: Problem Decomposition and 

Values-Focused Thinking

A structured approach to making decisions helps resource managers by decomposing tough decisions into logical and 

manageable elements. These elements—the four stages of structured decisionmaking (SDM)—can then be addressed 

by the right people with the right tools. A key benefit of the problem decomposition embodied in SDM is the ability to 

separately analyze the values-based and science-based aspects of the decisions.

Values-focused thinking stresses that values are the driving force behind decisions and that evaluation of alternatives is 

ultimately tied to the decisionmakers’ ability to achieve outcomes consistent with decisionmakers’ values (Keeney 1996a, 

1996b). That is, science-based information is not the sole basis for making decisions, and scientists’ involvement in 

decision processes does not extend to setting policies or articulating values (Wilhere et al. 2012). Discussion of decisionmakers’ values and preferences, therefore, precedes other components of the decision process.

Whose values?

In this report, we specifically focus on decisions made by Forest Service employees regarding National Forest System 

lands. Thus, when we speak of values, we do not mean the personal values and preferences of the individual resource 

manager or line officer. Rather, a Federal decisionmaker’s job is to reflect the values of society at large, as expressed 

through the mandate of the agency, the legislation that enables it, the regulations that have been promulgated, the court 

decisions that have clarified the agency’s roles and responsibilities, and, as applicable, the stakeholders’ viewpoints. It is 

recognized that balancing multiple objectives and reflecting societal preferences are daunting tasks, especially where nonmarket resources and ecosystem processes are concerned. (We provide additional guidance for addressing these issues 

in section 3.4 and in appendix B.)
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 9

2.4 Why Use the Structured Decisionmaking 

Framework?

Adopting the SDM framework and its associated principles is 

not likely to come without some measure of difficulty or cost. 

A significant upfront investment might be present in structuring the problem, especially to ensure that the right questions are being asked, the right problem is being solved, and 

the right players are involved. Clearly identifying the roles 

of each player (decisionmakers, decision analysts, stakeholders, etc.) is necessary, as is initiating communication among 

these roles throughout all stages of SDM. Decisionmakers need to commit to collaboration, inclusiveness, clarity, 

and transparency throughout the entire decision process.

Despite these potential difficulties and costs, the benefits of 

adopting these decision science principles are likely to prove 

worthwhile. SDM helps improve decision processes by making 

them more comprehensive, clear, and consistent with stated 

values and preferences. These properties of SDM lead to a suite of 

benefits for sustainable forest and grassland management. These 

potential benefits include the following attributes.

“I think the real benefit of SDM is that it’s an organized approach 

to making any kind of a decision, and I think as folks who are 

familiar with it know, you can use it at just about any level—an 

individual can use it, a family, an agency, or it can be used by an 

organized multi-stakeholder group, like the Glen Canyon Dam 

Adaptive Management Work Group.

“So it has a lot of utility, as a structured, organized way to make 

as informed a decision as you can. For our purposes, it’s served 

us well—in terms of organizing data, identifying gaps, and identifying uncertainty associated with different aspects of making a 

decision. It’s also served us well in group settings—in bringing 

stakeholders into processes, in getting their input and their views 

in how a question or a decision should be structured, and in 

getting a range of information you can use to evaluate different 

potential decisions.

“But I’d also say it’s been helpful in encouraging participation 

and buy in from a number of stakeholders. It provides a process 

through which people can participate in the decisionmaking 

and have a real sense of benefit from having participated in that 

process, having a sense of buy-in to the decision because they 

were directly involved.”

Glen Knowles

Chief, Adaptive Management Work Group

Bureau of Reclamation

Greater Defensibility and Durability of Decisions

The use of best available science (for process and content), explicit 

inclusion of stakeholders, and commitment to transparency could 

reduce the prospect for appeals, and more importantly, could increase the prospect for successfully withstanding legal challenges.

Clearer Alignment With the Planning Rule and NEPA

The decision framework presented here is well aligned with the 

NFS Land Management Planning Rule (Planning Rule) and 

the National Environmental Policy Act (NEPA). The Planning 

Rule requires that the best available science be used to inform the 

planning process, and that the use of science in the development 

of these plans be documented. These mandates are consistent 

with the approach and concepts of SDM. Similarly, NEPA is the 

process that the agency uses to analyze and disclose the environmental effects associated with Federal decisions, and the structure 

of NEPA mirrors all stages of the SDM framework. The purpose 

and need statement in an environmental impact statement (EIS) 

serves to frame the problem and to articulate objectives (problem 

structuring stage). An EIS includes multiple alternatives and their 

possible consequences (problem analysis stage). Finally, a record 

of decision documents the decision rationale (decision point 

stage) and outlines plans for implementation and monitoring 

(implementation and monitoring stages). SDM methods can 

provide a rich structure, protocol, and set of tools for decisionmaking reported under NEPA. Photo by Bruce G. Marcot
10 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Time and Cost Savings

Adherence to SDM is likely to result in time and cost savings by 

reducing prospects for “dead ends” due to ill-formed problems or 

ill-defined evaluation criteria. In addition, SDM will likely enhance synergies with existing decision processes. Thus, SDM may 

also reduce the time spent addressing legal challenges.

Clearer and More Efficient Decisionmaking

SDM methods can improve clarity in properly defining the right 

problem to be solved and can help ensure that alternatives and 

analysis best align with objectives. This approach can engender 

trust among participants and lead to improved resource management decisions.

Facilitated Adaptive Management

Monitoring and assessment of decision outcomes will facilitate 

evaluation of success and provide guidance useful to inform 

future decisions.

3. The HOW of Structured Decisionmaking: A Guide for Decisionmakers

and Decision Analysts

3.1 The Structured Decisionmaking Framework

—A Closer Look

Each of the four primary stages in structured decisionmaking 

(SDM) (figure 1) consists of multiple steps; some are linked with 

feedback loops to denote opportunities for learning and adaptive 

management. In the problem structuring stage, the key aspects 

and drivers of the problem are identified, providing a framework 

for subsequent steps. Problem structuring includes defining the 

management objectives and evaluation criteria for achieving 

the objectives. The problem analysis stage entails predicting and 

evaluating the consequences of alternatives. The influence of uncertainties and multiple objectives can complicate the evaluation 

of how well alternatives satisfy overall management objectives. 

The decision point stage is where the decisionmaker identifies 

a preferred alternative, integrating science- and value-based 

information into the selection process and decision rationale. 

Finally, the implementation and monitoring stage entails putting 

the preferred alternative into action. Frequently, there are reasons 

to monitor the outcomes of the decision. Monitoring can help 

reduce critical uncertainty and provide feedback that can support 

adaptive management, that is, monitoring can help improve subsequent decisions and actions. We describe each of these stages in 

detail in the sections that follow.

“The big issue in public land decisionmaking is ‘purpose and need.’ 

So, for example, we have a piece of land out there and we want to 

do something with it, because we recognize, through our eyes and 

filters, that—for example—there are bugs out there and we need 

to do some thinning.

“So we’ve got a watershed management issue here—what’s the 

purpose and need? Well, to control the bugs, and healthy forests, 

and restoration, and that sort of thing. And we start developing that 

on our own. But if we can get everyone together to say in the beginning that we all need to get something done here, and get them 

to help us develop the purpose and need, it may come out a little 

bit different. I can almost guarantee you it’ll come out a little bit 

different, if not a lot different, than if we’d have done it on our own.

“And my contention is that it’s going to be better, (a) because it 

has the social license, and (b) because there will probably be six 

things that hadn’t fit into our original view of the problem. It could 

be, initially, that only our timber people are seeing this, but now our 

recreation people are seeing it, and homeowners are seeing it, and 

everybody’s seeing it.”

Rick Brazell

Forest Supervisor

Clearwater—Nez Perce National Forests

Photo by Bruce G. Marcot
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 11

Each decision stage in the framework entails interaction and 

collaboration among different groups in the process (i.e., decisionmakers, stakeholders, scientists, and analysts). Decisionmakers have primary responsibility for the problem-structuring and 

in particular the decision point stages, but need to be involved 

throughout all stages. Stakeholders’ engagement, in general, is 

present across all stages (Ascough et al. 2008), with primary 

interaction in the problem-structuring and decision point stages. 

Scientists and analysts, by contrast, are responsible for objectively 

evaluating consequences of proposed alternatives. They can also 

identify the presence and implications of key uncertainties (in 

the problem analysis stage) and help design monitoring strategies 

(Kiker et al. 2005).

Managing national forests and grasslands provides a compelling 

context for the role of decision science and risk management. Although adoption of the SDM framework may not solve all conflicts involving social perceptions and political interests, it does 

help provide a structure for transparency and rigor to decision 

processes, especially within the context of NEPA and the NFS 

Land Management Planning Rule (Planning Rule). One of the 

powerful advantages of SDM is the deliberate process of structuring the decision process, in particular, identifying the problem 

to be solved, the objectives, the alternatives, the consequences, 

the tradeoffs, and the analytical logic that leads to a preferred 

alternative.

“If you don’t know where you’re going, you might not 

get there.”

Yogi Berra

3.2 Stage One: Problem Structuring

A critical first step to making decisions is to answer the question, 

“Does a decision need to be made?” If yes, this step leads the 

decisionmaker to the question of “What decision needs to be 

made?” or alternatively, “What is the problem being solved?” This 

question will necessarily be informed by management objectives 

and desired outcomes; i.e., by values-focused thinking. An incomplete or unclear answer to this question will unnecessarily complicate subsequent decision stages. Beyond providing clarity to 

the decision objectives, problem structuring also helps guide the 

decision process toward the appropriate tools and information. 

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

SDM Flowchart

“...the most important conversation I had in the whole project, from 

start to finish, was the 45 minutes to an hour I spent with the ID [interdisciplinary] team leader, just talking about how we wanted that to 

go, and how we would build a process to start off on, and knowing 

that we would have to be pretty adaptive as we went through it.”

Gina Owens

Former District Ranger

Green Mountain National Forest

Problem structuring also helps determine the appropriate levels 

of investment in the problem analysis stage and ensures that the 

right problem is being solved. Three primary components are used 

to structure a problem statement—framing the problem, defining 

the objectives, and defining criteria by which alternative solutions 

can be evaluated. These early steps are specifically based on the 

management context and the values set by decisionmakers.

3.2.1 Problem Framing

Clearly understanding the context in which a decision is 

required can provide important information about the nature 

of the problem and about which decision-support tools may be 

appropriate. Four broad pieces of information can help guide 

problem framing:

1. Identify Governance: One component to identifying governance structure is determining who the decisionmaker(s) is (are). 

In the simplest of cases, one fully authorized decisionmaker is 

needed to make the decision. Problem complexity will increase 

when multiple decisionmakers are involved and will depend on 

whether they are acting together, independently, cooperatively, or 

competitively. Another component to identifying the governance 

structure is to establish accountability, which could, in certain 

circumstances, include staff and external stakeholders.
12 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

2. Involve Stakeholders: In agency decision processes, stakeholders do not have the legal authority to make decisions; however, 

they often exert considerable influence over decision processes 

and outcomes. Engaging and communicating with the interested 

public and others can help identify issues, better frame problems 

that consider managing resources for the greater good, and build 

ownership and investment for the decision ahead. It is important 

to note here that the Forest Service is required by the Planning 

Rule to involve stakeholders in all stages of land management 

planning.

3. Identify Timing of the Decision: It is important to clarify 

the timing and frequency of the decision(s) to be made. Some 

decisions are made only once, such as large land acquisitions or 

installation of permanent infrastructure. Other decisions may 

be repeated periodically through time, however, such as annual 

decisions about the amount and method of timber harvest in a 

particular forest.

4. Know the Background: Legal, regulatory, policy, ecological, 

social, and economic drivers all influence how decisions are 

made. Various statutes and regulations provide legal direction, 

constraints, and mandates for decisions made on National Forest 

System land. Collectively these factors circumscribe the decision 

space of resource managers.

3.2.2 Defining Objectives

Ultimately, a decision is an expression of the values of the decisionmaker(s). Clear and upfront identification of objectives can 

help ensure that the decision process works toward achieving the 

decisionmaker’s aims. Objectives are the long-range aspirations 

of the decisionmakers and stakeholders and can include ecological, economic, recreational, cultural, and aesthetic dimensions. In 

cases with multiple objectives, it will ultimately be necessary for 

decisionmakers to articulate the relative importance of each objective, and doing so early on can help direct future stages of the 

decision process. Often, however, this articulation is not possible 

nor is it necessarily desirable, because how multiple objectives are 

balanced may depend on the set of real tradeoffs presented by the 

alternatives (see section 3.4).

Forest and grassland management often involves multiple, possibly competing, objectives. In such cases, an informed decision 

requires careful attention to defining and structuring objectives. 

One important element is to differentiate types of objectives.

1.Fundamental objectives are the ends that the decisionmaker desires to achieve through the decision. For example, in a large-scale 

forest plan, the fundamental objectives might be to maximize sustainable timber yield; persistence of threatened, endangered, and 

sensitive species; and consumptive and nonconsumptive recreational uses of the forest. Note that these fundamental objectives 

might compete with one another, and the optimal solution might 

involve balancing the tradeoffs among them. For an example at 

the project level, a fundamental objective for a fuel treatment 

could be minimizing the likelihood of wildfire-related mortality 

to old-growth trees.

2. Means objectives are ways of achieving fundamental objectives 

and, as such, are valued only insofar as they lead to achievement 

of the desired outcomes. Means objectives could include things 

such as increasing the number of recreational access points, improving forest access roads, and preserving a portion of the forest 

as a permanent no-harvest zone. The means objective for the 

fuel-treatment project example could be reducing stand density 

and fuel loading through mechanical treatment.

3. Process objectives pertain to how the decision will be made, but 

not what the decision will be. An example of a process objective 

for a forest plan might involve the level of engagement the public 

will have in the forest planning process; e.g., through National 

Environmental Policy Act (NEPA) consultation.

Objectives, which are the long-range aspirations 

of the decisionmakers and stakeholders, can 

include ecological, economic, recreational, 

cultural, and aesthetic dimensions.

“The big issue in public land decisionmaking is ‘purpose and need.’ 

So, for example, we have a piece of land out there and we want to 

do something with it, because we recognize, through our eyes and 

filters, that—for example—there are bugs out there and we need 

to do some thinning.

“So we’ve got a watershed management issue here—what’s the 

purpose and need? Well, to control the bugs, and healthy forests, 

and restoration, and that sort of thing. And we start developing that 

on our own. But if we can get everyone together to say in the beginning that we all need to get something done here, and get them 

to help us develop the purpose and need, it may come out a little 

bit different. I can almost guarantee you it’ll come out a little bit 

different, if not a lot different, than if we’d have done it on our own.

“And my contention is that it’s going to be better, (a) because it 

has the social license, and (b) because there will probably be six 

things that hadn’t fit into our original view of the problem. It could 

be, initially, that only our timber people are seeing this, but now our 

recreation people are seeing it, and homeowners are seeing it, and 

everybody’s seeing it.”

Rick Brazell

Forest Supervisor

Clearwater—Nez Perce National Forests
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 13

Fundamental and Means Objectives

The distinction between ends and means can be difficult to track in complex decision problems, especially in cases with 

multiple and possibly conflicting stakeholders’ values and viewpoints. Ends and means are not necessarily absolutes but 

rather depend on the decision context. The process of indicating whether an objective relates more to a “why” or a “how” 

helps make the transition from an intuitive picture of what is desired to a structured and formal representation of objectives—which is critical for the future stages of the decision process. Using top-down and bottom-up approaches can help 

clarify fundamental and means objectives and can even help expand, reframe, or redefine a decision context.

Note that the process of identifying fundamental objectives is value driven, whereas the process of identifying causal 

relationships and connecting means objectives to fundamental objectives is knowledge driven.

Fundamental and means objectives can often be organized into 

hierarchies. (An example of an objectives hierarchy is presented 

in section 5.1.) Structuring objectives in a hierarchy can provide 

considerable clarity for the decisionmaker, stakeholders, and public, and it might help identify tradeoffs, complementarities, and 

priorities, where needed. Well-constructed objectives hierarchies 

portray relationships among objectives, ensure relevance to an 

inclusive mix of stakeholders, and provide the focus necessary to 

analyze tradeoffs and assess risks. Fundamental objectives can be 

stratified according to high-level, general objectives (sometimes 

referred to as “goals”) and low-level, specific objectives and can 

further incorporate means objectives that describe how to achieve 

fundamental objective(s). This “top-down” approach is in fact 

similar to SDM, decomposing a fundamental objective into more 

manageable elements by asking what the key components of the 

objective are and how can they be achieved. By contrast, a “bottom-up” approach asks why a given action or activity is important 

(i.e., to which objective might it contribute), and why that objective is important. That is, moving down in the hierarchy focuses 

on execution, and moving up the hierarchy focuses on purpose. 

In some cases multiple linkages and interrelationships may exist 

“That was the crux of the problem, I think [framing the problem and decision]. It goes back to a class I took in college, 

called Creative Problem Solving, where we spent most of the semester focusing on how to define the problem. That’s 

where we struggle the most... What problem are we trying to define? And that’s why it took so long [on decisions over 

threatened and endangered species issues]—we couldn’t agree. We couldn’t come to agreement on what exactly it was 

we were trying to resolve.

You spend most of your time defining the problem... that’s what’s going to pay off the most... that’s where the focus is, 

and that’s where the improvement needs to come from.”

Jim Smalls

National Environmental Policy Act Specialist

Washington, DC

between fundamental and means objectives (e.g., a given silvicultural action may help achieve multiple fundamental objectives 

such as improving forest health and reducing wildfire hazard), in 

which case, an objective network rather than a hierarchy may be a 

more appropriate way to structure objectives (Keeney 1996a; see 

also “means-ends objective network” in appendix B). Photo by Bruce G. Marcot
14 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Figure 2.— The Watershed Condition Assessment Framework (Potyondy 

and Geler 2011) as an example of an evaluation criteria hierarchy.

Aquatic

Physical Aquatic

Biological TerrestrialPhysical TerrestrialBiological

Watershed

Condition

Water

Quality

Water

Quantity

Aquatic

Habitat

Aquatic

Biota

Riparian

Wetland

Vegetation

Roads and Trails

Soils

Fire Regime

or Wildfire

Forest Cover

Rangeland

Vegetation

Terrestrial

Invasive 

Species

Forest Health

3.2.3 Defining Evaluation Criteria

Evaluation criteria are necessary to measure how well alternatives may achieve objectives. Developing evaluation criteria can 

be challenging, especially when objectives might seem largely 

intangible or when quantification methods do not yet exist (for 

example, to improve landscape resiliency). In many cases, evaluation criteria may be structured in a hierarchy to match an 

objective hierarchy. For instance, the Forest Service’s Watershed 

Condition Class model is comprised of a hierarchy of multiple 

evaluation criteria and subcriteria, which are grouped according 

to four major process categories (figure 2). Defining evaluation 

criteria consists of the following components:

1. Measurement scale: The measurement scale, typically quantitative, is the explicit gauge of how a given alternative performs with 

respect to fundamental objectives. There are three major types of 

measurement scales (Keeney 1996b):

a. Natural: These evaluation criteria are direct measures that 

do not require additional interpretation or assumptions. 

Timber board-feet production is an example of a natural 

measurement scale. While natural criteria are preferable, for 

many natural resource management objectives they may not 

exist or may be difficult to measure directly.

b. Proxy: Proxy evaluation criteria are measures of quantities 

(often of means objectives) that are indirectly associated with 

the objective of interest. For example, if the objective was to 

increase recreational usage, the number of miles of managed 

trails might be a useful proxy attribute.

c. Constructed: For complex objectives, measurement scales 

can be constructed to gauge performance. These scales cannot 

be observed or measured on the ground, but rather represent 

the overall intent of an objective. In some cases constructed 

criteria can be composites of multiple pieces of information 

and could consist of multiple natural and proxy measurement 

scales combined in ways to specifically reflect the objectives. 

For example, habitat suitability indices are constructed scales 

that represent the quality of habitat for a particular species.

“...we probably should have taken a more expansive and 

more deliberate approach to framing the opportunity and the 

problem. If you don’t do things right up front, then you’re going 

to have problems throughout the whole process. In the past few 

years, that’s kind of been my professional conclusion, when 

it comes to monitoring decisions that get made—for example, reviewing appeals on particular decisions that rangers or 

supervisors make.

“You can almost always—practically 90 percent of the time—

trace just about any error you find in the review back to the 

beginning of the process. Something wasn’t done well in the 

beginning, or it was misguided conceptually. They didn’t think 

it was important...and it just manifests itself in the question—‘Why didn’t they think about this in the beginning?’ They 

didn’t think about it...or they tried to bandage a bleeding artery.

“You get to the point where you’re actually implementing, and 

you’re looking back and saying, ‘I think that maybe I should’ve 

started somewhere differently’.”

Tony Erba

Director, Planning, Appeals, Litigation, and 

Landscape-Scale Conservation

Eastern Region, Forest Service
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 15

The development of alternative actions has a value component 

and can involve input from stakeholders. Other components of 

problem analysis, however, are conducted as a “clinical” set of 

tasks in which the alternatives are tested against the objectives 

and the decision evaluation criteria developed in the problem structuring stage. By clinical, we mean that these tasks are 

conducted unbiased by and separate from the subjective components of desired outcomes and risk attitudes. A wealth of tools is 

available for analyzing consequences of alternatives and tradeoffs 

in natural resource management decisionmaking (see section 3.6 

and appendix B). Examples of tools include modeling approaches 

such as simulation, Bayesian networks, fuzzy logic, and optimization analysis, and approaches to analyzing potential and untried 

management options such as scenario analysis and comparative 

risk assessment.

2. Desired direction: This component of evaluation criteria indicates whether higher or lower values on the measurement scale 

are preferred. For instance, using the example above, timber 

board-feet production might constitute a natural measure of an 

evaluation criterion. Higher levels of timber production might be 

the desired direction.

3. Aspiration: Aspirations are expressions related to performance 

of the objective. The aspiration might be to maximize performance of the objective or to achieve some minimum level of 

satisfactory performance. Threshold aspirations for evaluation 

criteria are common in natural resource management, such as 

minimum population levels for recovering a threatened species. 

In the example above, exceeding a particular level of board-feet 

production per year might be the desired threshold. Thresholds 

could be based on avoiding adverse or undesirable conditions. 

For example, not wanting to exceed a specific concentration of 

particulates in the air shed caused by controlled burning could 

constitute a desired maximum threshold.

Table 1 presents some examples of evaluation criteria that could 

apply for a multiobjective land management project involving 

timber harvest. Note that aspirations might conflict across criteria. For example, truly maximizing carbon storage might be infeasible or undesirable due to severe impacts on biodiversity, water 

yield, and fire danger. Figure 3 illustrates fundamental objectives 

and evaluation criteria developed for a case study applying SDM 

to respond to hemlock wooly adelgid (Adeleges tsugae) on the 

Cumberland Plateau of northern Tennessee.

3.3 Stage Two: Problem Analysis

The problem analysis stage entails defining the range of possible 

alternative actions, evaluating their potential consequences, analyzing potential complementarities and tradeoffs among objectives, and identifying key uncertainties and their implications. 

Problem analysis also involves identifying and quantifying risks 

(i.e., the likelihood and severity, or the probability and magnitude, 

of potential outcomes and their consequences).

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

SDM Flowchart

Influence Diagram

An influence diagram can be a particularly useful and powerful tool for framing the problem and for developing a conceptual model that 

can feed into the problem analysis stage. It can be used to collectively understand the major relationships and important causal pathways and (most importantly) to determine how natural variation and management actions may influence or drive outcomes. An influence 

diagram can help form a bridge between qualitative descriptions and more concrete quantitative specifications, including relationships 

(e.g., profit is a function of revenue and cost), functions (e.g., profit = revenue – cost), and, where appropriate, actual numbers (e.g., 

revenue = $20; cost = $10; therefore, profit = $10) (Howard and Matheson 2005). An influence diagram is relatively easy to construct 

and comprehend, can be used to represent a wide variety of variables and relationships, and can be built off expert judgment rather than 

relying on availability of data (Marcot 2006). An influence diagram is similar to the means-ends objective networks discussed earlier in 

this report, but it can include other causal factors with no direct relation to objectives, such as the weather or other uncontrollable factors 

(see appendix B). An example of an influence diagram is provided in section 5.1.
16 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Table 1.— Examples of evaluation criteria and their associated measurement scale type, desired direction, and aspiration. In addition, each criterion 

could have a specific threshold value to achieve (or avoid), such as shown with the soils objective.

Objective Category Evaluation Criteria Measurement Scale Type Desired Direction Aspiration

Silviculture Board-feet of timber Natural Increase Maximize

Carbon Storage Tons of CO2 Natural Increase Maximize

Soils Percentage of area potentially disturbed 

from ground-based harvesting

Proxy Decrease Threshold; no 

more than 35%

Recreation Miles of maintained trails Proxy Increase Maximize

Wildlife Habitat suitability index Constructed Increase Maximize

Fire and Fuels Torching index Constructed Decrease Minimize

Figure 3.— Subset of fundamental objectives and evaluation criteria for a case study applying SDM to respond to hemlock wooly adelgid (Adeleges 

tsugae) on the Cumberland Plateau of northern Tennessee. Fundamental objectives included conservation of two target species, blackside dace 

(Phoxinus cumberlandensis) and Swainson’s warbler (Limnothlypis swainsonii), and protection of hemlocks (Tsuga canadensis and 

Tsuga caroliniana). Additional objectives were to minimize treatment and mitigation costs, and to minimize concerns raised by the public. Modified 

from Blomquist et al. (2010).

Conserve target

species

Protect and 

preserve hemlock

Minimize

Percent loss

of speciesspecific

habitat

changes for

blackside

dace

Minimize

Percent loss

of speciesspecific

habitat

changes for

Swanson’s

warbler

Minimize

Percent 

motality

and

defoliation

Maximize

Stand size

Maximize

Percent of

hemlock

in 

overstory

Minimize

Percent of

understory 

or

rhododendron

“On the Colville, when we did travel management planning, we had all our internal folks and the public, and we said, ‘Let’s 

sit down in a room together and figure this out.’ We’ve got to figure out the social issues. We need to know where all the 

trails are, first of all, because there are trails not on our system that you folks use, and you have to trust us when you tell 

us where they are that we’re not just going to run out and close them.

“So we gathered the data, and of course it takes a lot longer to gather the data because you’re gathering it externally and 

internally. We talked about the need to get from Point A to Point B—there may be five ways to get there, but maybe you 

only need one or two.

“...you go through the analytical process of collecting data, and discuss the need for getting from here to there. It may be 

a timber sale, or treating for bugs, or whatever the need is. You talk about the need to do it, you get everybody together 

on the same page on the need, and then you get ready to make the decision because you’ve gathered up the information 

together, and you’ve got everybody in the room face to face. So you’re mixing the collaborative process with the data 

gathering—all at the same time.”

Rick Brazell

Forest Supervisor

Clearwater—Nez Perce National Forests
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 17

3.3.1 Defining Alternatives

Alternatives are specifications of potential options for moving 

from current to desired conditions (i.e., for achieving management objectives). Defining alternatives involves identifying specific decision variables (those items we can control) and acceptable 

ranges for those variables (e.g., levels of timber harvest or range 

allotments). Decision variables and their acceptable ranges are 

informed by the decision evaluation criteria previously articulated 

by the decisionmaker in the problem structuring stage of SDM. 

Alternatives can be generated from the decision variables and 

their acceptable ranges in various ways, such as with modeling 

(Stage 2003).

In the simplest cases, the decision alternatives are a small set 

of discrete options, such as the choice to mow, burn, or leave a 

grassland undisturbed. In some cases, the decision variable(s) 

might be continuous, so the alternatives (at least in theory) are 

infinite, such as the specific rate at which to apply an herbicide. 

Continuous decision variables are often treated as a small set of 

discrete choices in practice. For example, a decisionmaker might 

evaluate herbicide application rates of 0, 1, 2, 3, 4, 5, or 6 pounds 

per acre, although models or formulae could be used to identify 

more precise rates, given a desired outcome.

Decision Variable

What is a decision variable?

A decision variable is a description of a single element in a decision. 

Natural resource management contexts typically include multiple 

decision variables, and, as a result, alternatives are defined as sets 

of given values and levels of decision variables. Formally defining 

decision variables leads to improved clarity in problem evaluation, 

and it facilitates generation of alternatives.

Types of decision variables

Decision variables come in many forms, broadly defined as quantitative or qualitative. These variables will necessarily be tied in some 

way to evaluation criteria. Quantitative decision variables can be one 

of the following three:

• Binary (0/1); for instance, a yes or no decision on whether to gate

 a road.

• Integer (1, 2, 3, etc.); for instance, the number of stands to treat.

• Continuous; for instance, the miles of trail to maintain.

Qualitative decision variables may be characterized categorically 

(e.g., low, medium, and high) where quantitative depictions are 

difficult or have not yet been developed. Qualitative values can also 

indicate general directions of change (e.g., more and less).

In many natural resource management problems, the alternative actions are complex combinations or sequences of multiple 

decision variables. For instance, in designing a forest management prescription, a manager might build in not only the level of 

annual timber harvest, but also the method of harvest, the spatial 

arrangement and timing (scheduling) of harvest, the method of 

treatment of the residual material, and the rate and method of 

regeneration. A very large set of combinations is possible. Practically, the analysis often focuses on a smaller set of alternatives that 

span the range of all the decision variables, but represent contrasting strategies for achieving the objectives. (These alternatives will 

likely vary in the tradeoffs across objectives. See section 3.3.4.)

Individual decision components (variables) may be eliminated 

and sets of possible alternatives are often` reduced early in the 

analysis process by screening out alternatives that exceed set 

constraints. For example, if staying within set costs is an objective 

with a threshold aspiration, then the portfolios of management 

projects might be screened so that those options that are above a 

given cost are rejected or amended.

3.3.2 Evaluating Consequences

Evaluating consequences is the second step in problem analysis. It 

involves predicting the likely outcomes of each alternative action 

in terms of the fundamental objectives. This stage is based on an 

understanding of the ecological, economic, and social systems 

affected, and thus relies heavily on the integration of science in 

decisionmaking. Often models, whether conceptual, field-derived, 

or expert-based, are helpful in predicting outcomes. Models serve 

to link the alternative actions to the objectives, using the evaluation criteria as the quantitative scales for prediction.

Consequences can be evaluated qualitatively by using conceptual models such as influence diagrams (Howard and Matheson 

2005) that graphically link possible actions through intermediate 

variables to the outcomes desired by management. Influence diagrams provide a way for the decisionmakers and analysts to clarify 

their understanding of how the ecological, social, or economic 

system responds to management actions. Influence diagrams can 

then be developed into quantitative models (Marcot 2006) for 

more specific evaluation of consequences.

In natural resource management, perhaps the most common, and 

often the most reliable, predictive tools for evaluating consequences are quantitative models built on empirical field data. An 

enormous variety of predictive empirical models is available, each 

with its own purpose and context, from forest-stand models to 

Bayesian network models and linked wildlife habitat-population 

models. Such models are based on empirical data that describe 

and quantify the linkages between potential management actions 

and desired outcomes. These models can help reveal the importance and implications of random variability and uncertainty, if 

appropriately parameterized. Empirical models are limited to 

situations in which historic conditions are projected to continue 

into the future, however. If this projection is not the case (e.g., 

examining climate change effects on forest sustainability and 
18 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

ecosystem services), more mechanistic models may be used. The 

uncertainties associated with any mechanistic models used, however, must be assessed and understood. 

For contexts in which multiple objectives are important, projections of how any potential action will affect each of the objectives 

are necessary. This approach might require a different predictive 

model for each objective. Thus, in a multiple-objective tradeoff 

analysis, the expected consequences of each potential action are 

compared among all objectives and with respect to their ultimate 

benefits and costs on multiple scales.

“...as a decision support tool, [SDM] forces you to think about all aspects of the problem. You also engage in identifying 

all the 

resources that are going to be affected by your decision, and what are all the sources of information that are going to be 

affecting your decision. It causes you initially to focus specifically on what the problem is that you’re trying to address, and 

in doing that, a lot of times the initial question you think you have will change, just based on looking at the whole suite of 

resources that are going to be affected by your decision.

“And then in thinking about that a little bit, you’ll often change your problem statement a bit to be more accurate. Another 

aspect of this is that you think about how you’re going to evaluate alternative decisions you might make, and the way you 

do that is to define resources and define quantitative metrics (as best you can). Now, of course, for some aspects it’s very 

difficult, if not impossible. But you can qualify at least and get some information, some feedback, in doing that.

“So going through the process of trying to quantify the different things that are going to be affected by your decision: and 

in this case it included cultural resources, recreation, endangered fish, a trout sport fishery, water deliveries, the production of hydro-power at the dam (so an economic impact), and we also looked at cost for any particular alternative. And just 

in trying to think about all those aspects of a decision and in trying to define a measurable way of looking at performance 

of a given alternative or decision, caused us to think more critically about the problem statement at the beginning, and 

caused us to reframe the problem.

“And it also provided us with a way to assess how well our decision would perform, and in that way SDM guides you to 

make an informed decision in terms of which is the best decision.”

Glen Knowles

Chief, Adaptive Management Work Group

Bureau of Reclamation

If empirical or mechanistic models are not available or appropriate, decision analysts can use formal methods of expert elicitation 

(Martin et al. 2012) including expert paneling to develop quantitative predictions (Marcot et al. 2012b). In these methods, expert 

knowledge is elicited in a rigorous manner to help analyze consequences, particularly when major areas of uncertainty and gaps 

are in empirical research. As with other consequence evaluation, 

the purpose of using expert knowledge is to predict consequences 

(in terms of the evaluation criteria) of alternative actions.

Does Uncertainty Matter?

Is uncertainty an impediment to decisionmaking?

Some measure of uncertainty is likely present in every natural resource decision. In many instances, after uncertainties are identified, they 

can be managed with little disruption to the broader decision process. For instance, when random variability is present, a decision analyst 

might turn to probabilistic techniques such as simulation or decision trees, without significantly affecting the decisionmaker’s ability to 

choose a preferred alternative. In other cases, however, the uncertainties may be quite substantial, requiring alternative techniques and 

additional expertise to analyze consequences, or perhaps even invest in additional research or monitoring to improve the quality of available 

information.

Is the uncertainty controllable?

Some sources of uncertainty are inherently uncontrollable; for instance, weather patterns. In such cases, the question is whether available 

models and scientific understanding afford a reasonable depiction of the underlying uncertainty. For cases in which uncertainty can be 

controlled (for instance, through additional monitoring), the question is whether the cost of obtaining improved information is worth the value 

of that information.

Does uncertainty change the ultimate decision?

The critical question for the decisionmaker is whether the uncertainty makes it difficult to select a preferred alternative. Does the evaluation 

of the alternatives against the objective depend on information that is highly uncertain? Would reduction of uncertainty lead to different 

courses of action depending on what was discovered?
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 19

Table 2.—The four types of uncertainty that may be encountered in a complex natural resource decision (Ascough et al. 2008).

Uncertainty Type Definitions Examples

Linguistic Uncertainty Issues of vagueness, ambiguity, the contextual 

dependency of words, evolving definitions, and 

difficulty in explaining results

Definition of “ecosystem resiliency”; distinct notions of 

the “problem” by different stakeholder groups

Knowledge Uncertainty Limitations of scientific understanding and 

observation

Unknown model parameters; limited understanding of 

ecosystem processes

Variability Uncertainty Inherent variability that manifests itself in natural 

and human systems

Climate and weather patterns; political cycles

Decision Uncertainty Ambiguity or conflict over social objectives 

and values

Conflicting values of old-growth forest resource use 

or preservation

3.3.3 Identifying Key Uncertainties

All four stages of SDM may be impeded by uncertainty, leading 

to difficulty in clearly framing a problem, defining evaluation criteria, evaluating the consequences of alternatives, selecting a preferred alternative, or designing appropriate monitoring protocols. 

The best approaches for grappling with uncertainty will depend 

on which stage is most impeded. In identifying uncertainties, it 

can be useful to distinguish their various types and implications. 

Table 2 describes four primary types of uncertainty commonly 

faced in natural resource decisionmaking (Ascough et al. 2008). 

For instance, uncertainty regarding the definition or evaluation 

of “landscape resiliency” might be handled very differently than 

uncertainty regarding the frequency and severity of disturbances 

that may affect landscape resiliency (see section 3.6). Aiming 

to reduce uncertainty over time can include decisions related to 

monitoring and adaptive management (see section 3.5.2)

Identifying key uncertainties is particularly critical with regards to 

problem analysis, as this stage provides the information to distinguish alternatives and analyze tradeoffs. Uncertainties can pertain 

to parameter values, overall model structure, definition of terms, 

and functional relations among variables. Uncertainties can arise 

from sampling error, limited knowledge of the system, imprecise 

language, and variable expert judgment (Benke et al. 2007, Brugnach et al. 2010, Janssen et al. 2010, Regan et al. 2002).

Often uncertainty is encoded with probabilities, particularly when 

attempting to capture variability inherent to natural processes. 

The use of probabilistic information can be a powerful tool for 

characterizing processes with random or poorly understood components, and for quantifying the likely consequences of various 

alternatives. The “expected value,” or the probability-weighted average, is a common way to characterize probabilistic information, 

although it can be of limited use when capturing the range of 

possible outcomes or values is important. A variety of tools exist 

to help quantify and analyze probabilistic information (e.g., Monte Carlo simulation, logistic regression, and Bayesian networks). 

A number of other valid approaches, however, can characterize 

and manage uncertainty, the applicability of which will depend on 

the nature of the uncertainty and the stage of the decision process 

(see appendix B).

Uncertainty can be of prime importance in the decisionmaking 

stage, particularly when managers deal with linked or sequential 

decisions. Diverse and conflicting goals and interests, and changing and unpredictable environmental conditions reduce confidence in projecting the consequences of alternatives (Brugnach 

et al. 2008). Uncertainty analysis can help identify how partial 

knowledge might affect the projected outcome of each alternative.

The analysis of uncertainty in a decision context is quite different than its analysis in a scientific context. What matters to a 

decisionmaker is whether the uncertainty would lead to choice 

of a different course of action. There are cases in which considerable uncertainty exists about the predicted outcomes, yet the 

best course of action is unaffected. In these cases, the reduction 

of scientific uncertainty is not important to the decisionmaker’s 

selection of an alternative. Estimating the “expected value of 

information” is a powerful decision analysis method for evaluating 

whether uncertainty is relevant in a choice (Runge et al. 2011a, 

Williams et al. 2011). When the expected value of additional 

information is high, or when the power to reduce uncertainty is 
20 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

high, it may be appropriate to implement monitoring or research 

to reduce uncertainty before committing to a course of action, 

or as part of the action itself, thereby establishing a proactive 

adaptive management strategy. But when the expected value of 

additional information is low, or when the power to reduce uncertainty is low, there is no advantage in gathering more information, 

and thus little to no justification for delaying a decision.

3.3.4 Tradeoff Analysis

In some cases, particularly with single-objective decisions that are 

not greatly affected by uncertainty, the evaluation of the consequences can lead transparently to identification of a preferred alternative. In multiple objective decisions, the consequences could 

display a complex mix of tradeoffs so that no one alternative is 

obviously best for meeting all objectives. Before proceeding to the 

decision point stage, it may be helpful to carefully examine the 

pattern of these tradeoffs and simplify the set of choices.

A multiobjective land management and fuel-treatment planning 

project on the Bitterroot National Forest in Montana is an illustrative example of a tradeoff analysis (table 3; Stockmann et al. 

2010). The example presents a (reduced) mix of quantitative and 

qualitative evaluation criteria that relate to fundamental objectives 

and the desired direction or aspiration for each criterion (table 3). 

A first step for simplifying the set of choices is to identify criteria 

for which alternatives do not differ. None of the alternatives are 

anticipated to have direct effects on wilderness characteristics, 

and therefore this criterion can be removed from the table (not 

because adverse effects on wilderness characteristics are not important, but because this objective does not help distinguish the 

alternatives).

Table 3.— Example tradeoff analysis for a multiobjective land management and fuel-treatment planning project (Stockmann et al. 2010). Management 

alternatives vary in terms of number and area of treatment units, treatment types, and degree of related restoration work. Various management objectives and details of evaluation criteria are reported for each management alternative. The scores for each alternative are reported in comparison to the 

no action alternative. The best performing alternative for each objective is shaded in green.

Objective Category Evaluation Criterion Desired Direction Alternative

1 (no action) 2 3 4

Fire and Fuels Area of reduced fireline 

intensity (hectares)

Max. 0 41 818 375

Watershed Rate of sediment delivery 

(% reduction)

Max. 0 19% 15% 19%

Wilderness Adverse effects on wilderness characteristicsMin. No effects

Silviculture Timber volume harvested 

(m3)

Max. 0 23,786 32,176 21,185

Soils Area of potential detrimental soil disturbance 

from ground-based 

harvesting (hectares)

Min. 0 32 45 38
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 21

Another step in considering tradeoffs is to identify and remove 

dominated alternatives, that is, alternatives that do not outperform other alternatives on at least one criterion. Consider alternative 4 with respect to alternative 2—it reduces fireline intensity 

over a smaller area, has the same sediment delivery reduction, 

harvests less volume, and has greater potential soil disturbance. 

Therefore, alternative 4 is “dominated” by alternative 2 and can 

be removed from consideration. Table 4 presents the reduced set 

of alternatives and criteria that analyze tradeoffs. Note that the 

remaining table displays the tradeoffs; a choice of alternative is 

not obvious, but requires value judgments to determine which 

objectives are more important to achieve.

Decisionmakers will be faced with alternatives that vary across a 

range of remaining criteria, which can introduce complexity and 

uncertainty at the decision point stage. Formal methods for quantifying and visualizing the performance of alternatives (e.g., spider 

plots; Gareau et al. 2010) can help decisionmakers better identify 

potential tradeoffs, better distinguish across alternatives, and to 

better communicate these results of problem analysis. Articulation of the relative importance of objectives with respect to each 

other is required to determine the preferred alternative and will 

be addressed in the next section.

“I’ve long operated under the premise that people support what they help create. And so, giving people a way to join into the 

development of something, typically results in them being supportive...when somebody from the public comes up and says 

‘You’re clear cutting my backyard, and that’s not going to work for me,’ we have to be open to not rationalizing why we want to 

do that, but ask them what they would like to see. What would work for you? And then we can balance out the science and the 

silviculture with the public perceptions and desires.

“So [on the Nordic vegetation management project] ...we began to devise a way to provide the maximum amount of inclusion, 

not just about what we wanted to do, but in having the people who lived in that area participate in developing what the project 

looked like. We spent the bulk of our time on the left side of NEPA. We were asking, ‘What do you think should happen in this 

area? How do we return a forest landscape in Vermont, and integrate it into the lifestyle you have here—whether you live on 

the land, whether you’re a second home owner, or whether you only like to come up and ski and look at the pretty fall colors.’

“And so it was interesting to build a public engagement strategy that was not around ‘Here’s our proposal. What do 

you think?’ But ‘Here’s a chunk of land ripe for forest restoration, what do you think we ought to do?’ And then building open 

houses and public engagement opportunities where people could see what we had done with what they wanted us to do....

“What we heard over time—the effect of that when we rolled out our proposed action, was that people could really correlate 

between what they asked for, and what they got. And it was one to one, most of the time. It was pretty cool.”

Gina Owens

Former District Ranger

Green Mountain National Forest

Table 4.— Reduced tradeoff analysis for a multiobjective land management and fuel-treatment planning project. Management alternatives vary in terms 

of number and area of treatment units, treatment types, and degree of related restoration work. Various management objectives and details of evaluation 

criteria are reported for each management alternative. This table presents a simplified tradeoff analysis from table 3 with nondistinguishing evaluation 

criterion (direct effects on wilderness) and dominated alternative (#4) removed. The best performing alternative for each objective is shaded in blue.

Objective Category Evaluation Criterion Desired Direction Alternative

1 (no action) 2 3

Fire and Fuels Area of reduced fireline intensity (hectares)Max. 0 414 818

Watershed Rate of sediment delivery (% reduction)Max. 0 19% 15%

Silviculture Timber volume harvested (m3

)

Max. 0 23,786 32,176

Soils Area of potential detrimental soil disturbance 

from ground-based 

harvesting (hectares)

Min. 0 32 45
22 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

SDM Flowchart

Figure 4.— Flowchart for guiding the decision point. If evaluation criteria can be quantified monetarily, cost-benefit analysis is appropriate. If all nonmonetary effects can be quantified in a single criterion, then cost-effectiveness analysis is appropriate. Otherwise, multicriteria decision analysis (MCDA) 

techniques will likely be necessary to help identify the relative importance of management objectives.

Can all evaluation be 

quantified in dollars?

Cost-Benefit

Analysis

Can nonmonetary impacts be

quantified in a single criterion?

Cost-effectiveness

Analysis Multicriteria

Decision Analysis

3.4 Stage Three: Decision Point

The decision point stage is ultimately one in which a decisionmaker chooses a preferred alternative. This selection 

could relate to a policy, plan, or management option. In contrast to problem analysis, which focuses on answering the 

question “what do we know,” the decision point stage focuses on answering the questions “what matters most to us?” 

and “which alternative best achieves our objectives?” When 

there is a single objective, the decision point stage is fairly 

straightforward: select the feasible alternative that best performs with respect to the single evaluation criterion. When 

multiple objectives and a complex set of evaluation criteria 

exist, however, it can be difficult to find clear solutions that 

effectively balance tradeoffs across competing objectives.

Fortunately, many tools exist to help decisionmakers navigate 

through such difficult processes (figure 4). If monetary values can 

be assigned to all evaluation criteria, the economic impacts of a 

proposed alternative can be compared against the costs of the 

alternative with cost-benefit analysis, presuming that the decision 

is to be based on economic consequences. If benefits exceed costs, 

an alternative is considered economically efficient. As an example, 

one appropriate context for cost-benefit analysis might be evaluation of alternative commercial thinning strategies. As another 

example, suppose a resource economist has generated estimates 

of the monetary value of various recreational trails. It might be 

possible in such a case to evaluate alternative trail management 

strategies with cost-benefit analysis. Methods are available to 

handle more complicated cost-benefit analyses, such as the 

comparison of future benefits against current costs (e.g., the cost 

of precommercial thinning to provide future improved timber 

quality and volume), or the use of willingness-to-pay surveys to 

estimate proxies to costs and benefits (e.g., Festa-Bianchet 2012, 

Waldhardt et al. 2010).

Yes No

Yes No
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 23

It is often difficult to assign monetary values to all criteria and 

objectives, even using such approaches as willingness-to-pay, 

especially when nonmarket resources (e.g., habitat) are considered. One possible approach, based on cost-benefit analysis, is to 

ask “what are the required benefits associated with known costs 

such that benefits equal costs?” This type of approach is known 

as break-even analysis and seeks to determine the minimal value 

of nonmarket benefits to justify the costs of a management 

alternative. If a given treatment is selected and implemented, the 

implication is that the minimum value of the potential resource 

value change exceeds the cost of the treatment (augmented as 

appropriate by considering the likelihood of treatment success). 

Break-even analysis is more limited in situations in which effects 

to multiple nonmarket resources must be considered and the decisionmaker is uncertain about the extent of possible benefits (see 

section 3.3.3 regarding uncertainty).

Quantification of effects by a single, nonmonetary evaluation 

criterion may be possible when cost-benefit analysis is infeasible. Cost-effectiveness is a common alternative to cost-benefit 

analysis that compares alternatives on the basis of their costs with 

respect to a nonmonetized performance measure (figure 3). An 

alternative with a lower ratio of costs to effectiveness is considered more efficient than an alternative with a higher ratio. For instance, an alternative that would restore 10,000 acres of dry ponderosa pine forest at a cost of $5 million (C/E = $500/acre) would 

be considered more cost effective than an alternative achieving 

restoration on 20,000 acres at a cost of $11 million (C/E = $550/

acre). Limitations to cost-effective analysis include an inability to 

account for differences in project scale (e.g., 10,000 acres treated 

versus 20,000 acres treated, although C/E ratios could be weighted by scale), and, more importantly, it is rare that in multiresource, 

multiobjective problem contexts, all nonmonetary effects can be 

quantified with a single effectiveness index.

Multicriteria decision analysis (MCDA) approaches can be used 

when all effects cannot be quantified with a single measure of effectiveness (figure 4). MCDA is a family of tools and approaches 

that facilitates the systematic evaluation and selection of management alternatives by identifying potential tradeoffs, conflicts, and 

complementarities across objectives. The most common MCDA 

approaches in natural resource management rely on what are 

known as value measurement models. These models seek to quantitatively establish decisionmaker preferences so that the relative 

importance of objectives can be distinguished. Decisionmakers 

can transparently and systematically evaluate and rank alternatives when the preference structure is clearly articulated.

The land management and fuel-treatment planning project on 

the Bitterroot National Forest in Montana (table 4) can be used 

to illustrate the application of MCDA for the decision point 

stage. Although a variety of approaches are possible (see appendix 

B), we demonstrate a relatively simple weighted-sum approach 

(Ananda and Herath 2009). The first step is to rescale the criteria 

in table 4 (predicted values for evaluation criteria for each alternative) so that values fall in the range of 0 to 1 (a value of 1 is most 

desirable) to enable comparisons with a common currency (table 

5). Where more is better (i.e., the desired direction is to increase), 

Equation 1 is used to assign rescaled scores. Where less is better 

(i.e., the desired direction is to decrease), Equation 2 is used.

(criterion level - minimum level)

(maximum level - minimum level)

(Equation 1)

(maximum level - criterion level)

(maximum level - minimum level)

(Equation 2)

As an example, for areas of reduced fireline intensity, where more 

is better, Alternative 1 is scored (0 – 0) / (818 – 0) = 0.00, Alternative 2 is scored (414 – 0) / (818 – 0) = 0.51, and Alternative 3 

is scored (818 – 0) / (818 – 0) = 1.00. The rescaled criteria (table 

5) can be used to help understand the tradeoffs across alternatives in terms of relative achievement of objectives, and to better 

understand how much importance one would have to place on 

each objective for a given alternative to be preferred. For instance, 

Alternative 1 performs the worst on criteria relating to Fire and 

Fuels, Watersheds, and Silviculture Objective Categories. Thus 

minimizing soil disturbance would have to be very important to 

the decisionmaker for Alternative 1 to be preferred.

A set of weights establishing the relative importance of objectives is also required for the weighted sum approach. As with the 

rescaled criteria, a higher score is considered better. For example, 

suppose the decisionmaker made the value judgment to place 

70-percent weight on the Fire and Fuels objective and 10 percent 

on each of the remaining objectives. In that case, Alternative 1 is 

scored (0.70 * 0.00) + (0.10 * 0.00) + (0.10 * 0.00) + (0.10 * 1.00) 

= 0.10; Alternative 2 is scored 0.56; and Alternative 3 is scored 

0.879. Thus, Alternative 3 is the best alternative for collectively 

achieving the importance-weighted desired outcomes. An advantage to this technique is that the effects of a range of possible 

weights can be explored to determine how the preferred alternative would change under different weighting schemes (table 6). 

We strongly suggest that this exploration not be used to justify a 

predetermined decision, however.

A variety of MCDA techniques are used for different settings. 

Whichever technique is adopted, it is important to handle the 

decision point stage in a systematic and documentable manner. 

One approach is to identify target levels for satisfactory performance for evaluation criteria and use a technique called goal 

programming to identify the preferred alternative. When weights 

are difficult to assign directly and satisfaction thresholds are not 

appropriate, decisionmakers can rely on pair-wise comparisons 
24 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Table 6.— Example of sensitivity analysis paired with multicriteria decision analysis for a multiobjective land management and fuel-treatment planning 

project. The preferred alternative is identified under various weighting schemes using the weighted sum approach, based on the alternative criteria 

scores presented in table 5 and with the additional step of applying importance weights to scores.

Example Weighting 

Scheme

Weights Preferred 

Alternative

Fire and Fuels Watershed Silviculture Soils

Favor Fire and Fuels 0.70 0.10 0.10 0.10 3

Favor Watershed 0.10 0.70 0.10 0.10 2

Favor Silviculture 0.10 0.10 0.70 0.10 3

Favor Soils 0.10 0.10 0.10 0.70 1

Emphasis on Watershed and Soils0.15 0.35 0.15 0.35 2

Table 5.— Normalized multicriteria decision analysis applied to the multiobjective land management and fuel-treatment planning project from tables 

3 and 4. Here the evaluation criteria for management alternatives presented in table 4 are rescaled, which “equalizes” or “normalizes” scoring across 

variables, for use with the weighted sum approach. The best performing alternative for each objective is shaded in blue.

Objective Category Evaluation Criterion Desired Direction Alternative

1 (no action) 2 3

Fire and Fuels Area of reduced fireline intensity (hectares)Max. 0.00 0.51 1.00

Watershed Rate of sediment 

delivery (% reduction)

Max. 0.00 1.00 0.79

Silviculture Timber volume 

harvested (m3)

Max. 0.00 0.74 1.00

Soils Area of potential detrimental soil disturbance 

from ground-based 

harvesting (hectares)

Min. 1.00 0.29 0.00

of each objective to establish a ranked priority. In fact in cases 

with ill-defined values, starting by ranking objectives can provide 

opportunities to develop and reveal values and can provide a basis 

for establishing more quantitative weights. In difficult group decision settings, decisionmakers could turn to techniques premised 

on the principles of voting to identify the degree to which one 

alternative or objective may be favored more than another (Mendoza and Martins 2006).

In addition to using the techniques presented in figure 4 to help 

with the decision point stage, it can be helpful to revisit previous 

stages of the decision process. Doing so can help ensure that the 

right questions were asked and the correct answers provided. 

One of these questions is to address the potential costs of doing 

nothing (often a “no action” alternative in the NEPA context). 

Appendix A provides some additional guidance for addressing 

particularly tough decisions. Ultimately, if a decision is made, the 

principles of good decisionmaking call for clear and logical decision documentation, including a description of how value-based 

information was considered and how objectives were balanced.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 25

3.5 Stage Four: Implementation and Monitoring

The implementation and monitoring stage follows the completion 

of the decision point stage. Implementation and monitoring are 

separate components in the decision process, but for the Forest 

Service, monitoring is an integral part of land management. 

Therefore, we discuss implementation and monitoring together.

3.5.1 Implement Preferred Alternative

The implementation stage is the point when the preferred alternative is enacted. Several aspects may be considered for successfully implementing a decision, including the amount of time and 

cost associated with the implementation, the effect of the implementation, risks and benefits associated with the implementation, 

and the operational structure needed to implement a complex 

decision intended to guide management of large land areas with 

diverse administrative units. Not considering these aspects could 

result in a decision not being implemented as desired. Indeed, 

these aspects of implementation should be considered well before 

the implementation process is undertaken, preferably as part 

of the evaluation criteria in the problem structuring stage. We 

address these issues here to stress the importance of considering 

implementation feasibility and likelihood of success.

The Interior Columbia Basin Ecosystem Management Project 

(ICBEMP) is an example of how implementation can fail. The 

ICBEMP intended to produce one set of ecosystem management 

guidelines in one record of decision for Forest Service and Bureau 

of Land Management (BLM) lands in the interior West of the 

United States, (USDA and U.S. Department of the Interior 2000) 

for a project area larger than France. The diversity of regional administrative units, the contrasts in policy and management goals 

between the two agencies, and the complexity needed to interpret 

and enact the tangle of ecological, economic, and social management guidelines proposed, however, led to the development 

of two parallel draft environmental impact statements (EISs) 

for contiguous geographic portions of the interior Columbia 

River Basin rather than one EIS for the entire area. Additional 

Two Common Multicriteria Decision Analysis Methods 

for Establishing Weights

One key step in any multicriteria decision analysis is developing weights to apply to each objective. This value judgment 

reflects the decisionmaker’s evaluation of the importance of the various objectives. Two of the most common varieties of 

methods for eliciting these judgments are described below.

Simple Multiattribute Rating Technique (SMART)

SMART begins by asking decisionmakers to identify the most important objective, based on its importance and the degree

to which it differs across the alternatives; this objective is then assigned a score of 100. All other objectives are assigned

scores from 0 to 100, relative to the most important one. The weights are found by normalizing the scores so they sum to

100 (or 1 on a 0-1 scale). Weighting can proceed hierarchically for subcriteria. “Swing weighting” can be used to account

for the range of variability across objectives; a score of 100 is assigned to the objective for which the swing from worst

performance to best performance is most preferable.

Analytic Hierarchy Process (AHP)

AHP relies on a series of pairwise comparisons such as, “How important is objective X relative to objective Y?” Comparisons are scored from 1 for objectives of equal importance to 9 for absolute importance of one objective over another. As

with SMART, AHP can proceed hierarchically through criteria and subcriteria. The pair-wise comparisons can be analyzed

with statistical techniques to derive importance weights across objectives.

Sources: Ananda and Herath (2009); Diaz-Balteiro and Romero (2008).

Stage One: Problem Structuring

Stage Three: Decision Point

Stage Four: Implementation & Monitoring

Problem

Framing

Define

Objectives

Define

Evaluation

Criteria

Define

Alternatives

Evaluate

Consequences

Identify Key

Uncertainties

Tradeoff

Analysis

Identify

Preferred

Alternative

Implement 

Preferred 

Alternative

Monitor

Stage Two: Problem Analysis

SDM Flowchart
26 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

disagreement among agencies ultimately led to no final EIS and 

record of decision being created for either portion and, despite 

the wealth of analysis and scientific data produced (Haynes et al. 

2001), the project was abandoned. A key lesson learned here was 

that the goal and geographic scope of the project were simply 

too ambitious and complex to satisfy the immense diversity of 

affected decisionmakers and stakeholders and to provide feasible 

and cost-effective implementation.

3.5.2 Monitoring Results of Implementing the Decision

Three purposes for monitoring in the context of decisionmaking 

are (1) to evaluate achievement of the objectives, (2) to determine 

the state of the system (for State-dependent decisionmaking), and 

(3) to reduce uncertainty to improve future decisions (Nichols 

and Williams 2006). First, monitoring for evaluation serves as a 

way of documenting the outcomes of management. In the case of 

one-time decisions, monitoring determines whether implementing the decision achieved what was intended. In the case of recurrent decisions, monitoring provides a way to alter management 

practices if unforeseen outcomes occur or underlying conditions 

change. Second, for some decisions the preferred action depends 

on the state of the system, and the system needs to be monitored 

to determine the appropriate course of action. For example, the 

basal area needs to be monitored if a treatment prescription 

calls for thinning whenever the basal area exceeds a specified 

maximum threshold value. Third, when uncertainty impedes a 

recurrent decision, monitoring (and perhaps research) provides 

feedback that reduces uncertainty over time and allows adaptation 

of future actions. For example, a recurrent decision may pertain 

to successive thinning entries into a forest stand, timed to provide 

for at least a minimal economic return but also to provide enough 

understory space for development of a multilayered canopy 

structure for wildlife that use mature forests. Uncertainty in this 

case may pertain to the nature and degree of canopy layering 

that would provide habitat for the wildlife species of interest. 

Monitoring would provide information on when the forest stand 

achieves desired wood volume growth (determining the state of 

the system), and research and monitoring together would provide 

information on understory and wildlife response to thinning 

activities (reducing uncertainty to improve future decisions).

Monitoring design arises from the decision context. The monitoring metrics, the monitoring methods, the geographic scope, and 

the sampling design (including sampling locations and rate) are 

all determined based on the information needs of the decisionmaker. The cost of monitoring is an important consideration; to 

be warranted, cost needs to be offset by the benefit (monetary or 

otherwise) that accrues from the monitoring information. Collectively these factors help determine the type, extent, and frequency 

of monitoring required.

3.5.3 Adaptive Management

Monitoring the outcomes of early decisions can be used to 

improve later decisionmaking. Scientists and decisionmakers 

can use monitoring to evaluate predictions that were used in 

the decisionmaking process. Broadly, two potential outcomes of 

monitoring and assessment are possible. Monitoring can suggest 

that the current course of action is largely correct and that it is 

not necessary to change the implemented action. Alternatively, 

monitoring might provide novel insights that lead to a different preferred alternative, if the original objectives are to be met. 

Revisiting the management decision may be warranted if it is 

determined that the present course of action is not producing the 

desired outcome. Importantly, adaptive management is needed 

when there are critical uncertainties impeding the decision (e.g., 

when great uncertainty in predictions results in a high value for 

additional information).

“...the basic premise of adaptive management is learning by doing, so you’re constantly trying to evaluate the decisions 

you’ve made, monitor the effects of those decisions, and re-evaluate based on what you’ve learned. And what SDM really 

does for you is help you marshal that information.

“So for us, in the Glen Canyon Dam Adaptive Management program... it’s a big program; it’s a well-organized program 

that’s been in place for about 15 years; we have a mountain of information. We have some pretty good modeling tools in 

place that we can use. We have a pretty good ecological model; we have a model for the rainbow trout fishery in Lee’s 

Ferry, and we have some really good models on sediment transport and on water temperature in relation to flow and 

dam releases.

“But SDM makes all that information so much more useful, because it helps you marshal that information in a decisionmaking process and make your best use of it.”

Glen Knowles

Adaptive Management Work Group

Bureau of Reclamation
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 27

Adaptive management can be described in terms of two feedback 

loops (Williams et al. 2007). Monitoring results can be used to 

revisit the problem analysis stage (so-called “inner-loop learning,” 

figure 1, inner dashed line) or, more fundamentally, the problem 

structuring stage (“double-loop learning,” figure 1, outer dashed 

line). The knowledge gained through this learning may lead the 

decisionmaker to include new objectives or, indeed, to identify 

a need to change the very governance structure of the decision 

(Pahl-Wostl 2009).

3.6. A Guide To Identifying Appropriate 

Decision-Support Tools and Approaches

Making decisions in the world of multiresource objectives, such 

as under the National Forest Management Act and the current 

Planning Rule, is an enormously complex process. Decisionmaking in this context often entails balancing needs and desires for 

potentially conflicting land uses, addressing conflicting interests 

of multiple stakeholders, trading off short-term use for maintaining long-term options for future use, and coordinating among 

multiple land holders and decisionmakers, within the context of 

uncertainty. The potential for litigation is constantly present.

Structured decisionmaking is one option for addressing these 

complexities and challenges by posing a set of questions that can 

help determine the most appropriate decision-support tools. Thus, 

the broader decision process itself entails decisions about which 

tools and approaches to use.

•	 What is the main decision(s) to be made?

•	 Will a single decisionmaker or multiple decisionmakers make 

the decision? If multiple, will they cooperate or compete?

•	 Who is involved in and affected by the decision and its implementation? Are they government agencies, industries, local 

communities, NGOs, and other stakeholders?

•	 How complex is the decision? Is the decision about single or 

multiple resources, single or multiple objectives, and single or 

multiple decision variables and evaluation criteria? What are 

the anticipated important secondary effects of the decision?

•	 How much contention exists regarding the decision? What 

potential conflicts may need to be resolved?

•	 What is the timeframe for the decision? Does the decisionmaking involve a single decision to be made at a specific point 

in time, concurrent decisions to be made at a single time, or 

multiple decisions to be made over a span of time?

•	 To what extent does uncertainty influence the decision? 

What exactly is most uncertain, how does the uncertainty affect the decision, and how might that uncertainty be addressed 

in the decision?

We identified six typical combinations of answers to these questions and identified examples of appropriate SDM approaches 

(table 7). Note the importance of clarity in problem structuring 

to answer these questions, as well as the connection between 

decision context and appropriate decision-support approach. 

Considering if the decision problem fits one of these six types of 

problems can help determine which general set of SDM approaches might be appropriate for addressing the problem. After 

the problem type and SDM approach are narrowed, one can then 

move on to selecting specific decision-support tools and methods. Since the universe of possible decision contexts is broader 

than these few examples, it is likely that decisionmakers will find 

themselves in a different situation. The important thing is to 

clearly answer the above list of questions to enable identification 

of SDM tools and methods that are appropriate for the specific 

decision context. Appendix B provides a more comprehensive list 

of tools and approaches that could support structured decisionmaking.

One key step to sorting through decision-support options is to 

identify whether uncertainty is a major impediment, and if so, 

what type of uncertainty (or types of uncertainties) are most 

prominent. Identifying the type of uncertainty can then lead to 

families of appropriate decision-support approaches. For instance, 

substantial variability may be best managed with probabilistic 

approaches, whereas substantial knowledge gaps may be best 

managed with expert paneling methods. Decision uncertainty can 

often be best managed with multicriteria decision analysis techniques that help decisionmakers articulate preferences, visualize 

tradeoffs, and rank alternatives.

Decision-support systems and approaches DO 

NOT make decisions—they inform the decisionmaker and support the decisionmaking process.

Photo by Bruce G. Marcot
28 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Table 7.— Six typical resource decision problems and examples of appropriate decision-support approaches that would best help solve them, roughly 

listed in increasing order of problem complexity.

Type of Resource Decision Problem Best General Structured Decisionmaking 

Approach for Solving the Problem

Examples

1. Single resource but with other considerations 

or implications; single decisionmaker, single 

objective, single decision variable

If uncertainty is not a major impediment, use 

management science, optimization, singleobjective decision analysis without uncertainty.

If uncertainty is a major impediment, use 

single-objective risk analysis; if uncertainty is 

uncontrollable, use risk analysis; if controllable, 

use value information analysis (expected value 

of information, cost of acquiring additional 

information).

Restoration of a natural community; 

anthropogenic fire control

2. Multiple objectives; multiple stakeholders; 

single decision, single time period

Use multicriteria decision analysis (MCDA) to 

address tradeoffs among objectives.

Project-level decisions; e.g., wildfire control, 

fuels management

3. Single objective; multiple decisions over time 

with substantial uncertainty

Use classic active adaptive management; 

consider stochastic dynamic programming.

Forest or grassland ecosystem restoration

4. Multiple objectives; multiple stakeholders; 

multiple decisions recurrent over time; substantial uncertainty (often for secondarily affected 

resources)

Use MCDA and adaptive management in a 

multiple-objective approach; break the problem 

into a series of single-objective problems with 

the use of ad hoc approaches for considering 

multiple objectives or treat as a multiple objective problem with optimization solution.

Project or regional, strategic-level decisions, 

such as setting allowable annual timber harvest 

levels concordant with other objectives for water 

quality, wildlife habitat, range use, etc.

5. Multiple resources, objectives, and decision 

variables; potentially contentious stakeholder 

involvement; single initial decision, single time 

period; also sequential decisions later on; moderate to high uncertainty

Use MCDA and adaptive management in a 

multiple-objective approach; break into a series 

of single-objective problems and consider use of 

ad hoc approaches for considering multiple objectives or treat as a multiple objective problem 

with optimization.

Forest plan direction; strategic plan with monitoring and adaptive management component

6. Multiple objectives; multiple decisionmakers 

who do not necessarily cooperate

If decisionmakers cooperate, use conflict resolution and negotiation approaches.

If decisionmakers do not cooperate, each decisionmaker could use a game theory approach 

and consider his or her decision independent of 

the others.

Managing wildfire and fuels that straddle 

jurisdictional boundaries; fuels and fire 

management in wildland-urban interface areas 

with a mix of landownerships
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 29

4.1 A Survey of the National Forest System on Use of 

Structured Decisionmaking

As a foundation for developing this report, we conducted an 

anonymous, online survey in June 2012 to determine if, how, 

and under what circumstances structured decisionmaking 

(SDM) tools and methods are being used and what improvements may be needed. We surveyed 4,819 planners and specialists 

and 1,738 decisionmakers including line officers in the Forest 

Service’s National Forest System (NFS), who have been recently involved in projects related to the National Environmental 

Policy Act (NEPA), including planning at the project, forest, and 

regional levels.

We received 1,562 responses (23.8-percent response rate) of 

which 14 percent were decisionmakers and line officers, 20 

percent were planners and interdisciplinary team leaders, 39 

percent were specialists or researchers, and 27 percent were others 

involved in NEPA activities. Responses were from all nine NFS 

regions and the Washington Office, and they were roughly in 

proportion to the total numbers of employees and to numbers in 

line and staff. The survey revealed that respondents were responsible for a wide range of resource management issues, including 

recreation, planning and plan revision, wildlife management, 

water and watershed management, fire and fuels treatment, silviculture, timber scheduling, wilderness management, public affairs, 

and other issues. Most respondents (58 to 84 percent) noted that 

they are dealing with a number of high priority issues, including 

forest ecosystem health, soil and water conservation, biodiversity 

conservation, legal and institutional issues, socioeconomic effects, 

and forest ecosystem productivity.

4. Opportunities and Challenges for Adopting Decision Science Principles in the 

National Forest System

When presented with a list of 10 decision-support processes or 

methods for risk analysis or risk management, most respondents 

replied that they have used 6 of them—(1) modeling to generate 

alternatives, (2) modeling to compare alternatives, (3) comparative 

risk assessment, (4) scenario planning and analysis, (5) tradeoff 

analysis of alternatives, and (6) participatory or collaborative 

group decisionmaking methods. Respondents were familiar with, 

but have never used, expert paneling and were unfamiliar with 

optimization, objectives hierarchy assessment, and social choice 

theory. Importantly, fewer respondents noted that they use any of 

the 10 processes or methods frequently, and very few noted that 

they were not relevant to their work.

Most respondents had used 4 of 10 decision-support models and 

protocols— (1) models based mostly on expert judgment, (2) 

simulation models, (3) statistical models based on field data, and 

(4) decision trees. Most respondents were unfamiliar with fuzzy 

logic models, agent-based modeling, influence diagrams, Bayesian tools, multiattribute utility theory, and multicriteria decision 

analysis. Fewer respondents used any of the models and protocols 

frequently, and even fewer noted that they are not relevant to 

their work.

Photo by Bruce G. Marcot
30 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

We asked decisionmakers why they used any of the 10 SDM approaches and allowed them to indicate multiple reasons (figure 5). 

Most respondents (69 percent) said SDM approaches provided 

greater clarity or transparency to the decision process, 43 percent 

said they helped mitigate controversy and conflict, 43 percent said 

they provided protection against litigation, and 24 percent said 

they simplified risk management. Only 12 percent of respondents 

said they did not use any SDM approaches; however, only 17 

percent of respondents noted that using the SDM approaches saved time. Clearly, the main reasons for using such SDM 

approaches pertained to improvement of the decision process 

and bolstering against informal or formal conflicts, rather than 

saving time in the decision process. As stated previously in this 

report, however, increased time in the decision process may still 

lead to overall time savings due to reduced time spent in appeals 

and litigation. In addition, of those respondents that had used 

some SDM approach, a full 91 percent noted that they were very 

satisfied, satisfied, or somewhat satisfied with the outcome (figure 

6). This result is strong testimony of the value and utility of SDM 

approaches.

When asked about impediments to using SDM approaches (figure 7), most (73 percent) respondents did not know enough about 

the SDM approaches, with fewer (20 to 30 percent) stating that it 

is difficult for the public to understand them, that colleagues are 

not aware of or are resistant to using them, that it does not apply 

to their work, and that it appears to take too much time (figure 7). 

Only 15 percent of the respondents noted that using SDM approaches is not better than what is done now, and only 2 percent 

noted that using SDM approaches makes the decision process too 

transparent and vulnerable to appeals and litigation. Thus, most 

of the reasons given for not using SDM approaches pertained to 

knowledge gaps, education, and application of those approaches, 

with very scant concern about additional burdens or making the 

decision process more open.

Respondents noted a wide array of management problems for 

which SDM approaches would help. These approaches included 

the following line items.

•	 Evaluating consequences and analyzing tradeoffs.

•	 Monitoring and adapting to results.

•	 Addressing controversy and conflict with stakeholders.

•	 Providing greater credibility to the decision process to reduce 

the likelihood of litigation and appeals.

•	 Developing evaluation criteria and defining alternatives.

•	 Identifying and framing problems and articulating objectives.

•	 Identifying uncertainties.

The least frequent problem noted was choosing and implementing an alternative. This statistic suggests that respondents viewed 

SDM approaches as potentially more helpful for the early stages 

of the SDM process than for the decision point and implementation stages.

Figure 5.— Survey results asking respondents why they used listed decision-support procedures or tools, ranked in order of most to least 

prominent factor.

Provided greater clarity/transparency to the decision process

Helps mitigate controversy/conflict

Provided protection against litigation

Simplifies risk management

My colleagues are advocates for its use

Saved time

Did not use any of them

Its better than what I used to do

0% 10% 20% 30% 40% 50% 60% 70% 80%
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 31

Figure 6.— Survey results asking respondents how satisfied they were with outcomes after having used listed decision-support procedures or tools

Very Satisfied

Satisfied

Somewhat satisfied

Not very Satisfied

Not at all satisfied

Did not use any of them

0% 10% 20% 30% 40% 50%

Figure 7.— Survey results asking respondents what impeded them from using listed decision-support procedures or tools, ranked in order of most to 

least prominent factor.

Don’t know enough about them

Difficult for the public to understand

Colleagues are not aware of or are resistant to using them

Does not apply to my work

Appears to take too much time

Based on my experience, it takes too much time

Not better than what I do now

Makes decision processes too transparent and vulnerable

to appeals and litigation

0% 10% 20% 30% 40% 50% 60% 70% 80%
32 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

4.2 Survey Results on Decision-Support Methods and 

Impediments to Their Use

Our survey also provided insights into knowledge gaps, needs, 

desires, and concerns over risk analysis and SDM tools and methods. When asked what decision-support processes or methods for 

risk analysis or risk management they use in addition to the lists 

provided in the survey, many respondents suggested that decisions 

are made without recourse to formal procedures. Some sample 

comments we received in the survey follow:

•	 [I use] common sense and 40-plus years experience.

•	 Most decisions are made without formal use of a “method or 

process” and involve the need to meet targets for accomplishments or respond to some agency or public demand and are 

based on either common sense or the will of the decisionmaker 

or person pushing the decision.

•	 Engaging in conversation, reading materials, and determining 

a course of action based on that information and what I believe 

is “right” for the resource.

•	 In my experience, risk analysis and management is nearly never 

a formal process for line officers, with the exception of documentation for fires in WFDSS [Wildland Fire DecisionSupport System]. It is usually a well-informed assessment 

made without a formal process after talking with staff.

•	 [I use] learned experiences and follow what is in the 

agency handbook.

•	 In more than a decade as a line officer, I have never heard of 

a line officer using a decisionmaking model.

Some respondents noted that they use additional, formal 

procedures, however, such as in the following comments:

•	 Comparison of alternative effects to soil, water, vegetation, 

and plans.

•	 Decision-support tools—frequently (when they are required).

•	 Really it has been the NEPA process—purpose, need, public 

input, comparison of alternatives, decision—that I have used. 

Sometimes I go with my professional opinion (i.e., gut) on 

some issues that cannot be analyzed in perpetuity, and, 

of course, I consult my staff and partners on direction 

and decision.

•	 Use multicriteria decision-support and ecosystem management 

decision-support for programmatic planning.

•	 We do a LOT of budget modeling of forecasts, alternatives, 

scenarios, and trade-offs—I would say nearly monthly.

The SDM survey results also provided insights into what many 

respondents noted as high work load and work-related stress that 

impede interest in and use of SDM approaches. For example, 

regarding the general kinds of resource management issues they 

are dealing with, the respondents made the following comments: 

•	 “High level of recreational use on the District—no recreation 

staff—keeping it going with public pressure and concern is [a] 

continuous effort.”

•	 “High Priority: Explaining to the public what restoration is 

and how that is different than how we use to manage 

the forest.”

•	 “I work on a national grassland. Our major management issues 

revolve around multiple use and resource conflict management, 

Forest Service Sensitive and MIS [management indicator 

species] population and habitat management, and interactions 

with adjacent landowners within a very fragmented landownership pattern.”

•	 “The topics analyzed depend much less on environmental 

concerns and much more on risk analysis ... I think most FS 

[Forest Service] analysis these days is all about litigation and 

political sell and very little about true environmental concerns!”

•	 “Watershed health and T&E [threatened and endangered species] protection are critical to my mission work. Helping the 

forested landscapes heal after years of aggressive harvest and 

gypo logging methods while still providing commodities and 

supporting forest health ... A sound approach to forest management allows us to protect our resources while still providing 

multiple use values.”

When queried about what might be impediments to use of SDM 

approaches, respondents made the following statements.

•	 “That is not the way we do it.”

•	 “Do not have enough time to learn a new process.”

•	 “I participate mostly at project level planning, seems these tools 

would be more useful at forest level or above.”

•	 “Another thing about NEPA analysis is that we need to resist 

appearing to know more than we actually do. NEPA simply 

requires a comparison of alternatives, not accurate predictions 

or perfect knowledge. The time factor is one that could be 

managed more effectively if leadership set clear priorities, but 

this has not happened in the 11 years I have worked on this 

forest and I do not expect it to change. Given people’s huge 

work load, time is a real constraint.”

•	 “Currently all of the risk management done here is on the 

political ability to implement a decision. No consideration is 

made about environmental risks.”

•	 “I need more open and positive relationships with collaborators 

and stakeholders to get past differences in ideology, find areas 

of agreement and develop solutions.”

Additional reasons cited as impediments to the use of SDM 

approaches often included lack of time and funding, lack of 

supervisory interest or support, that they seem unnecessarily complex, and lack of access to the tools and knowledge to use them. 

A number of respondents also worried that spending time with 

computers and models takes away from valuable and irreplaceable 

field time. Others cited the value of SDM approaches, however, 

although many more highlighted lack of funding, staffing, and 

time as major impediments to their use.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 33

Figure 8.— Survey results asking respondents how they currently learn about decision-support approaches (Current Learning) and how they would 

prefer to learn about decision-support approaches (Preferred Learning), ranked in order of most to least prominent areas for preferred learning.

Work-supported training/workshop

On the job use

Webinars

Personal interactions with scientists

Websites

Not familiar with them (Not interested in learning more)

College Course

0% 10% 20% 30% 40% 50% 60% 70% 80%

4.3 Survey Results on Needs for Structured 

Decisionmaking Training

We found NFS personnel have learned about risk management 

and decision-support procedures and tools from a wide variety of sources: 68 percent from on-the-job experience dealing 

with appeals, litigation, and NEPA procedures; 40 percent 

from work-supported training and workshops; 24 percent from 

personal interactions with scientists; 15 percent from webinars; 

13 percent from Web sites; and 11 percent from college courses 

(figure 8). Some 18 percent of respondents noted that they are 

not familiar with such procedures and tools. Additional potential 

sources of learning included: conferences and workshops held by 

various professional societies, particularly on fire issues; leadership 

training; colleagues and networking; and incident management 

teams and fire management leadership.

When asked how they would like to learn more about SDM 

approaches, most asked for work-supported training and workshops (noted by 70 percent of respondents, figure 8), followed 

by on-the-job use (52 percent), webinars (37 percent), and Web 

sites (20 percent). In addition, some would like to learn through 

personal interactions with scientists (28 percent) and a few by 

taking college courses (9 percent). Some 10 percent noted that 

they were not interested in learning more about these topics. 

Additional suggestions for ways to learn more included how-to 

guides, guidance from a knowledgeable mentor, consultation with 

NEPA coordinators, publications, and terse case studies.

Many respondents also expressed concerns suggesting that SDM 

approaches may be overly complex and inappropriately applied to 

project-scale decisions; that risk analysis is equated with undue 

constraints on operations because of mandates on safety; and 

that models in general are inaccurate, take away from field time, 

are too complex to explain to the public, and are used to justify 

premade decisions.
34 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

4.4. Institutionalizing Structured Decisionmaking: 

Recommendations for Policymakers

Decision science is solidly grounded in theory and practice. As 

decision analysts and decisionmakers apply the concepts and 

tools of SDM, they are rapidly developing sets of best practices 

for high-quality decisionmaking. With training of analysts and 

decisionmakers alike, SDM can become an overarching framework and formalized decision structure for decisions made under 

the NFS Land Management Planning Rule, and a defensible and 

rigorous means of meeting regulations under NEPA and related 

directives. The SDM approach can address the complexities of 

sustainable natural resource management in many venues. Ultimately, application of SDM can help the Forest Service effectively and efficiently achieve sustainable management of forest 

and grasslands. SDM can be a powerful tool if embedded in the 

broader context of social decisions that guide management of 

public natural resources.

The real challenge in any agency or institution is how to bring 

these useful tools, approaches, and processes into daily implementation. We offer four general recommendations pertaining to 

more effective use of SDM tools and procedures.

•	 Analysts and planners can be equipped to provide guidance to 

management on decision support.

•	 Scientists and researchers can clearly explain the underlying 

logic of predictive models so they can be used appropriately in 

decision contexts, and can clearly articulate to help prioritize 

studies for adaptive management programs.

•	 Decisionmakers and managers can foster a transparent and 

defensible basis of their decisions, big and small, and work 

efficiently and closely with their support staff and stakeholders 

to identify important values in decision criteria.

•	 The agency can develop mechanisms that reward and support 

communication and interaction between scientists and users 

of scientific knowledge in the decision process (Pouyat et al. 

2010). More specific suggestions for areas of training are provided in table 8.

Table 8.— Some suggested themes and purposes for training in the area of structured decisionmaking.

Theme Purpose

Types of uncertainties and their characteristics For building a common lexicon

Use of influence diagrams For encouraging stakeholder involvement in problem definition

Role of uncertainty analysis, sensitivity analysis, and scenario analysis For analyzing alternative actions

Structured incorporation of expert knowledge and judgment For dealing with incomplete data and using available expertise

Application of multicriteria decision analysis and related techniques For identifying and incorporating diverse preference attitudes across 

stakeholders, and for transparently documenting decision rationales

Comparative risk assessment For evaluating and comparing and contrasting consequences of various 

management alternatives
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 35

5.1 Case Study 1: Problem Structuring: 

The National Cohesive Wildland Fire Management 

Strategy (Cohesive Strategy)

The Cohesive Strategy is an ongoing collaborative process with 

active involvement of all levels of government and nongovernmental organizations, as well as the public, to seek national, 

all-lands solutions to wildland fire management issues (http://

forestsandrangelands.gov). To ensure a shared vision, participation, and commitment and support for strategy implementation, 

problem structuring is a critical step for this multistakeholder, 

complex issue. We briefly review the use of two decision-support 

approaches that facilitated problem structuring.

Decision-Support Approach #1: Objectives Hierarchy

Regional Strategy Committees are tasked to develop regional 

goals and objectives in wildland fire management through a 

collaborative process under the Cohesive Strategy. Regional teams 

comprised of a variety of stakeholders representing government 

and nongovernmental organizations, as well as the public, were 

able to collaboratively create objectives hierarchies by adhering 

to a structured process of problem framing and objective definition. All hierarchies are tiered to the three overarching goals: (1) 

Restore and Maintain Landscapes—landscapes across all jurisdictions are resilient to fire-related disturbances in accordance with 

management objectives; (2) Fire Adapted Communities—human 

populations and infrastructure can withstand a wildfire without 

loss of property; and (3) Wildfire Response—All jurisdictions 

participate in making and implementing safe, effective, and 

efficient risk-based wildfire management decisions. We present an 

abbreviated objective hierarchy related to the third goal—Wildfire Response. The hierarchy contains high-level and low-level 

fundamental objectives, supported by means objectives.

All jurisdictions participate in making and implementing 

safe, effective, and efficient risk-based wildfire 

management decisions.

•	 Protect the health and safety of wildfire responders

•	 Improve safety awareness and avoid unnecessary 

risktaking by all firefighters.

•	 Use technology or specialized equipment to minimize 

firefighter exposure in difficult situations.

•	 Minimize firefighters’ exposure to smoke or other toxic 

substances in the short and long term (chronic).

•	 Provide adequate personal protective equipment and 

training across jurisdictions.

•	 Ensure effective communication among all responders.

•	 Maintain fire response effectiveness.

•	 Strategically align resources (personnel and equipment) 

across jurisdictions.

•	 Improve cost-share and grant programs to leverage 

resources.

•	 Ensure that wildfire suppression response reflects strategic 

landscape objectives or landowner values.

•	 Tactically integrate wildfire use with prescribed fire or 

mechanical treatments, when appropriate.

•	 Ensure that suppression effectiveness is balanced 

against long-term objectives and landowner or 

management priorities.

Decision-Support Approach #2: Influence Diagrams

Another critical component in structuring the problem was 

establishing a shared understanding of the impacts of wildfire, 

and what management actions and alternatives might be available 

to ultimately mitigate wildfire risk. Iteratively developing and 

revising influence diagrams helped provide a shared conceptual 

model of wildfire, and a common understanding of how specific 

management actions can affect specific components and processes. Figure 9 illustrates one such influence diagram developed 

through this process.

5. Applications of Decision Science Principles in Forest and Grassland Management

Figure 9.— Influence diagram examining how various management 

activities (land cover and land use change, fuel treatments, and prescribed 

fire) may influence factors driving wildfire extent and intensity. (Danny Lee, 

Eastern Forest Environmental Threat Assessment Center).

Wildfire Extent

and Intensity

Wildfire

Ignitions

Management Actions, Policies, and

Activities (Alternatives)

Fuel

Treatments

Mechanical

Chemical

Biological

Land Cover and

Land Use

Biophysical

Drivers

Topography

Climate

Substrate

Prescribed

Fire

Vegetation

Composition and

Structure
36 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

5.2 Case Study 2: Problem Structuring and Analysis: 

Hoosier National Forest, 2006 Land Management Plan

Problem framing for the Hoosier National Forest (HNF) plan 

began in 1999 when the HNF completed an assessment with 

public input on the need for change in the forest plan and then 

issued a notice of intent to revise the 1985 forest plan. Based on 

the need-for-change assessment and public input on the notice of 

intent, the HNF framed the problem around three issues: maintenance of watershed health; ecosystem sustainability and viability 

of plant and animal populations; and recreation management. 

The HNF then identified objective criteria that would be used 

to compare alternatives. Based on species viability assessments 

conducted by the HNF with species experts and an ecological 

assessment of the region (Thompson 2004), the HNF identified 

19 focal species that would serve as indicators of the ability of 

alternatives to maintain viable populations of native and desired 

nonnative species. Additional criteria reflecting watershed health, 

ecological sustainability, and recreation opportunities were the 

spatial and temporal distribution of forest age classes and dominant tree composition.

Problem analysis consisted of defining five forest plan alternatives, evaluating the consequence of these alternatives through 

simulation analysis, and considering tradeoffs among alternatives 

(Rittenhouse et al. 2010). The five plan alternatives were developed considering the issues raised in the need-for-change assessment and public input. They differed primarily in the amounts 

and types of forest management and ranged from no timber 

harvest to different amounts and spatial distribution of even- and 

uneven-aged forest management and prescribed burning. The 

forest plan alternatives were simulated with the spatially explicit 

landscape model LANDIS (He 2009), which modeled management activities, succession, and natural disturbance for each alternative more than 150 years and resulting effects on the spatial and 

temporal distribution of forest age classes and tree composition 

(figure 10). Predictions from LANDIS were then assessed with 

wildlife habitat suitability models to determine the effects of plan 

alternatives on indicator species (figure 10).

The tradeoffs among forest plan alternatives in wildlife species 

habitat suitability and forest composition and age class distribution were qualitatively assessed by the planning team and discussed at public meetings using graphical and tabular summaries 

of model outputs. While formal analytical methods were not used 

to evaluate tradeoffs and make a decision, the team did weight 

species differently based on their conservation status when qualitatively assessing tradeoffs. A decision was reached by a consensus 

recommendation from the planning team to the regional forester, 

and the regional forester selected the alternative recommended 

by the planning team (USDA Forest Service 2006). Recognizing 

that perfect information is impossible and anticipating that new 

scientific information may become available, the plan proposes 

an adaptive management approach involving monitoring and a 

process for amending the plan when needed.

5.3 Case Study 3: Decision Point: Deschutes National 

Forest, Five Buttes Project

In the National Forest System, the publication of a Record of 

Decision (ROD) is the culmination of a process incorporating 

the science supporting the environmental impact statement and 

the values of agency officers and stakeholders. A good example 

of this process is the ROD describing the rationale behind a 

vegetation and fuel-treatment project implemented on the Deschutes National Forest in Oregon, called the Five Buttes Project 

(USDA Forest Service 2007). The broad goals of the Five Buttes 

Project were to increase resistance to widescale fire and related 

disturbance events and to retain large trees, while also providing 

forest products and supporting local and regional economies. In 

particular, the project focused on reducing the risk of catastrophic 

wildfire and associated loss of old-growth forest habitat for the 

northern spotted owl (Strix occidentalis caurina). The project 

considered a no-action alternative (A), as well as two alternatives 

(B and C) that differed in extent and intensity of timber harvesting (commercial and noncommercial) and prescribed burning 

activities. Implementation of alternative B would result in a larger 

commercial harvest and associated mill activity, whereas alternative C emphasized modification of fire behavior and retention of 

spotted owl habitat, treating a larger area but yielding less commercial forest products. Alternative C also included commercial 

harvest of trees more than 21 inches in diameter and the modification of spotted owl nesting, roosting, and foraging (NRF) 

habitat, proposals which were the subject of public controversy.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 37

Figure 10.— Predicted patterns in tree size-class and species distribution on a management compartment and predicted area in habitat suitability 

classes for three bird species on the Hoosier National Forest at year 50, fewer than five land management alternatives. (Rittenhouse et al. 2010)
38 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

The project was one of the first to use spatially explicit burn 

probability modeling techniques for analysis of consequences 

(Ager et al. 2007). This approach enabled improved estimation of 

fuel-treatment effects on wildfire behavior, and refined analysis on 

the basis of comparative wildfire risk assessment. Results indicated that the no-action alternative increased risk of widescale 

disturbance, while providing no economic benefit. By contrast, 

results indicated that Alternative C best interrupted wildfire 

travel routes across the landscape and best provided for overall 

disturbance risk reduction and long-term maintenance of spotted 

owl habitat.

The forest supervisor ultimately selected the third alternative (C), 

stating that it provided the “best combination of commercial and 

noncommercial activities to reduce risk and improve forest health 

on the landscape while maximizing the retention of desirable 

habitat features, including late- and old-structured forest for 

wildlife species that are dependent upon those habitats” (USDA 

Forest Service 2007). Notably, the ROD explicitly identified 

tradeoffs, especially “risk-risk tradeoffs,” that is, risks of inaction 

compared with risks of action. The supervisor stated that thinning within late- and old-structured forest stands was a necessary 

tradeoff to effectively reduce landscape-scale disturbance risk. 

The ROD also directly addressed stakeholders’ concerns over 

commercial removal of large trees, acknowledging that ecological 

objectives ultimately outweighed economic objectives.

5.4 Case Study 4: Implementation and Monitoring: 

Tongass National Forest, Implementation and 

Monitoring Plan

The Tongass National Forest completed a review and amendment 

of its land and resource management plan (LRMP) (USDA 

2008a, 2008b) that directed plan implementation to include monitoring and evaluation under an adaptive management strategy. 

The design and sampling methods are stipulated in the LRMP 

monitoring protocol guidebook (USDA 2005). Three kinds of 

monitoring are specified in the LRMP: (1) implementation 

monitoring, to determine if the plan management standards and 

guidelines are being fully and correctly implemented; (2) effectiveness monitoring, to determine if the management standards 

and guidelines actually help achieve the plan objectives; and (3) 

validation monitoring, to determine if the assumptions and predictions underlying the plan are accurate and valid. Monitoring 

results are evaluated and, in an adaptive management framework, 

used to revisit management standards and guidelines, budgets, 

and work plans, and to determine if new courses of action are 

needed to respond to changing conditions. Monitoring reports on 

the Tongass National Forest are completed at 1- and 5-year increments, the former providing time-critical reviews and the latter 

providing more comprehensive evaluations of plan implementation progress and results.

The 2010 annual monitoring and evaluation report tracks a 

number of metrics and conditions broadly grouped into three 

themes: physical and biological environment, human uses and 

land management, and economic and social environment. For 

each theme and metric to be tracked, the LRMP monitoring 

protocol guidebook: provides a clear summary question and more 

detailed goals and objectives; identifies, by name, the responsible staff, authors, and specialists; and specifies data collection 

procedures, evaluation criteria, guidelines on desired precision 

and reliability of monitoring results, and general analysis methods 

to be used. As an example, one part of biodiversity effectiveness 

monitoring pertains to the question, “Are the effects on biodiversity consistent with those estimated in the Forest Plan?” The 

LRMP monitoring protocol guidebook specifies that Geographic 

Information Systems be used to measure the cumulative harvest 

of old-growth forest by biogeographical province. As part of the 

adaptive management process, some of the monitoring questions 

were changed in the recent LRMP amendment from those in the 

previous plan to better focus on more appropriate or recent topics 

of scientific and social interest, and some monitoring protocols 

are still being developed. Some of the results from the annual 

monitoring report of 2010 have been used, however, for reevaluating or reaffirming management direction. For example, results 

of effectiveness monitoring of old-growth forests protected under 

the LRMP to support viable and well-distributed populations 

of old-growth-associated species and subspecies suggested that 

current guidelines are adequate for this objective. In this case, a 

decision was made to not change the spatial distribution, size, and 

composition of protected old-growth forest reserves and other 

Photo by Bruce G. Marcot
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 39

nondevelopment land use designations as currently denoted in 

the LRMP implementation guidelines. Use of other monitoring 

results awaits completion of the next comprehensive 5-year monitoring report due in 2013, and during the next LRMP revision 

which, by mandate of the National Forest Management Act, is to 

occur every decade.

5.5 Case Study 5: Monitoring and Adaptive 

Management: The Survey and Manage Program of 

the Northwest Forest Plan

The Northwest Forest Plan (NWFP) was instituted in 1994 and 

is a set of guidelines for conservation of old-forest biodiversity 

on 9.7 million hectares (24 million acres) of Federal public lands 

of the Pacific Northwest United States, particularly for ensuring 

the persistence of hundreds of species closely associated with 

late-successional and old-growth (LSOG) forests. One part of 

the NWFP is the Survey and Manage (SM) Program, which 

is a highly structured procedure for a mixed panel of biologists 

and managers to evaluate recent monitoring data on rare and 

little-known LSOG-associated species and to assign the species 

to appropriate conservation categories under the NWFP (USDA 

Forest Service and BLM 1994, 2001). As the evaluation panels have been held annually, the SM Program thus serves as an 

adaptive management framework for incorporating new scientific 

monitoring information and adjusting species conservation goals 

and activities accordingly.

The suite of species addressed by the SM Program includes bryophytes, fungi, lichens, mollusks, selected vertebrates, and functional groups of arthropods, which may be associated with late 

successional forests, which are 80 to 200 years old, or old-growth 

forests, which are more than 200 years old. Under the NWFP, 

each SM species is put into one of six conservation categories 

that vary by the type and intensity of surveys required, or into 

a seventh category in which the species is no longer deemed 

to require special considerations under the SM Program and is 

thereby removed from the SM species list (but is still presumed 

to be conserved under the overall NWFP forest management 

guidelines) (Molina et al. 2006).

In the SM Program procedures, first, taxa experts compile data on 

the natural history, occurrence, and ecology of each SM species 

for which substantial new information is available that might 

change scientific understanding of species and of their conservation needs. Second, the taxa experts present the new information 

to the evaluation panel, which consists of four natural resource 

managers and four natural resource specialists, including biologists. The evaluation panel then deliberates over the information 

and uses a strict Delphi paneling approach, involving a structured 

series of steps by which panelists vote, discuss, and revote on their 

recommended conservation category for each species. The panel’s 

final recommendations are then sent to an interagency decisionmaking board that provides the final choices on the disposition of 

each species.

To aid the evaluation panel’s deliberations, a decision-support 

tool was developed by which the SM species evaluation guidelines were codified, in a set of Bayesian network models (Marcot 

et al. 2006). The models were run using the taxa experts’ new 

information on each species, and provided an initial suggestion of 

which species conservation category might be consistent with the 

new information. The models were not prescriptive but aided the 

evaluation panelists in dealing with the complicated and potentially conflicting conservation guidelines. The models also served 

to reduce bias and uncertainty in evaluating the new information 

on each species.

Eventually, the complexity and scope of the SM Program led to 

its temporary termination after lawsuits were filed in 2001 by 

timber industry and environmental groups. The SM Program 

was later reinstated under court order and, as of this writing, is 

being revised in a more streamlined structure, but is still to serve 

as a key monitoring and adaptive management component of the 

NWFP guidelines.

Photo by Bruce G. Marcot
40 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

5.6 Case Study 6: The Full Picture: Nonnative Fish 

Control Below Glen Canyon Dam

The Glen Canyon Dam is located on the Colorado River upstream of Glen Canyon National Recreation Area and Grand 

Canyon National Park and is managed by the Bureau of Reclamation (Reclamation) for water storage, hydroelectric generation, and other benefits. In a 2008 Biological Opinion under 

the Endangered Species Act, the U.S. Fish and Wildlife Service 

found that the dam’s operations may affect the humpback chub 

(Gila cypha)—a federally listed endangered native Colorado 

River fish—and included nonnative fish control as a conservation 

measure. Rainbow and brown trout had been introduced into the 

Colorado River and may pose a threat to humpback chub through 

predation on juvenile chub, habitat exclusion, and competition for 

food resources. Trout removal (through electrofishing) had been 

experimentally implemented in the early to mid-2000s, and Reclamation was considering making this program a part of regular 

operations. Several North American tribes raised serious concerns 

and objections about the lethal removal of thousands of fish from 

their sacred area. Besides the location of the proposed removal, a 

number of tribes consider all life sacred and object to the killing 

of nonnative fish. In response, Reclamation deferred undertaking 

plans for nonnative fish removal and initiated development of an 

Environmental Assessment (EA) under NEPA to evaluate alternative methods for nonnative fish control in 2010.

A formal structured decisionmaking (SDM) process was convened to help in development of the EA. In the problem structuring phase, the Reclamation’s Upper Colorado River Regional 

office was clearly identified as the sole decisionmaker for the 

EA. Representatives from six Federal agencies, one State agency, 

and five tribes were invited to participate in the SDM process as 

stakeholders to provide scientific and policy input to Reclamation 

for its consideration as part of its decisionmaking process. Assessment teams were assembled to evaluate the nonnative fish control 

alternatives against the array of objectives. Whereas Reclamation 

and other stakeholders provided value-based information regarding framing the problem and defining and weighting objectives, 

the assessment team was to be responsible for science-based 

information, primarily throughout the problem analysis stage. 

The decision was viewed as a multiple-objective tradeoff problem under uncertainty; formal methods of multicriteria decision 

analysis (MCDA) and value-of-information were used to analyze 

the alternatives (Runge et al. 2011b).

At the heart of the conflict among the stakeholders was a rich set 

of 22 fundamental objectives, hierarchically structured around 5 

main themes: (1) protecting tribal sacred sites and spiritual values, 

including the sanctity of life and the sacredness of the Canyon; 

(2) promoting ecological and native species integrity, including 

humpback chub recovery; (3) preserving and enhancing recreational values and uses, including enhancing the rainbow trout 

fishery and providing a wilderness experience for visitors to the 

Colorado River and Glen and Grand Canyons; (4) maintaining 

and promoting local economies and public services, including 

hydropower generation and water delivery in the arid Southwest; 

and (5) operating within the authority, capabilities, and legal 

responsibility of the Bureau of Reclamation. Stakeholders’ input 

was sought to articulate and define these objectives and to develop evaluation criteria for measuring achievement of the objectives. While conservation of humpback chub and preservation of 

tribal cultural heritage were the central concerns that motivated 

the SDM process, it became clear that the full set of fundamental 

objectives needed to be considered and weighed to more comprehensively analyze alternatives and understand tradeoffs inherent 

in the problem.

In the problem analysis phase, Reclamation and stakeholders’ 

groups used insight about objectives to generate alternatives for 

consideration in the EA. These alternatives were complex portfolios that considered the degree and frequency of nonnative fish 

removal, the location of removal, the methods (lethal and live) 

of removal, and the disposition of the fish removed, as well as 

other strategies meant to reduce the effect of trout on humpback 

chub (such as flow management from the dam and trout fishery 

management). Again, stakeholders’ input was used to craft the 

alternative portfolios—the representatives worked in small, mixed 

groups to develop alternatives with various themes (e.g., “No 

Action,” “Status quo,” “Culturally sensitive removal,” “Removal 

curtain,” “Kitchen sink”). In all, 27 alternatives were considered.

“From my perspective, one of the unique values of SDM, if you 

use it with stakeholders, is that it helps with the transparency of 

the decisionmaking process and their buy-in.”

Glen Knowles

Adaptive Management Work Group

Bureau of Reclamation
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 41

Each of the 27 alternatives was evaluated against the 22 objectives. The evaluation took a number of forms, depending on the 

evaluation criteria associated with the objective. For some of the 

objectives, such as persistence of humpback chub, predictive models based on empirical data were available. For others, such as the 

expected catch rate in the trout fishery, some simple models could 

be constructed based on available data. For still others, expert 

judgment was used for the evaluation. One of the challenges in 

the evaluation was critical uncertainty that affected the ranking of the alternatives. For example, the causal relationship and 

interaction between rainbow trout predation and humpback chub 

population is disputed and uncertain. To handle this uncertainty, 

two competing models for that interaction were posited, and an 

expert panel was asked to weigh the evidence for the two models. 

Evaluation of the outcomes was made under both models and included in a value-of-information analysis to determine how much 

that uncertainty affected the choice of a preferred outcome.

Moving towards the decision point stage required an additional piece of information—preference weights for the multiple 

objectives. These weights were elicited individually from the 

stakeholder agencies and tribes using MCDA, and an integrated 

assessment of the alternatives against all the objectives was computed for each stakeholder group. Reclamation was not asking the 

stakeholder agencies for a consensus recommendation; instead, 

individual stakeholders’ viewpoints were being sought, as a way 

for Reclamation to understand the differing perspectives. As it 

happened, the analysis resulted in identifying nearly the same 

preferred alternative for all agencies: a trout removal strategy that 

was focused about 60 miles upstream from the most sacred part 

of the Canyon (in the Paria-Badger Reach, PBR), which avoided, 

if possible, lethal removal and sought substantial beneficial use for 

the trout removed. This strategy was not one that any agency had 

promoted at the beginning, but neither was it one that was deeply 

objectionable.

Reclamation published a draft EA in January 2011, identified the 

PBR strategy as the preferred alternative, and included a detailed 

decision analysis as an appendix (Runge et al. 2011b). The SDM 

process was readily integrated with the NEPA process; indeed, 

the SDM report provided much of the raw material for the EA. 

Subsequent public comment, government-to-government consultation with the tribes, and additional scientific evaluation and 

analysis led to important modifications to the preferred alternative, including the inclusion of monitoring and adaptive management to address several key uncertainties. This modified PBR 

strategy was identified as the preferred alternative in the final EA 

and Finding of No Significant Impact, released in May 2012.

Now Reclamation is in a position to move forward with implementation and monitoring, having incorporated best available 

science for process and content. The SDM process provided a 

framework and tools for stakeholders’ engagement, clear articulation of objectives, development of creative alternatives, formal 

evaluation of alternatives, deliberation over tradeoffs, and evaluation of uncertainty. In addition, it provided a detailed administrative record of how Reclamation approached, analyzed, and arrived 

at the decision, and it integrated well with the agency’s NEPA 

compliance efforts.

Photo by Amy S. Martin
42 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

“...the existence of decision analysis concepts as a 

language for communication may be its most 

important advantage.”

Ronald A. Howard

6.1 A Change of Paradigm

Adoption of structured decisionmaking (SDM) may change the 

way resource managers think about decisionmaking, because it 

entails making values and preferences explicit while making decisions informed by science. Therefore, the decision context drives 

the science needs, not the reverse. SDM separates the clinical 

problem analysis process from the value-based decision process. 

This structure provides for a full consideration of science in all 

phases of the decision process. SDM requires commitment from 

decisionmakers and analysts to communicate and stay involved 

from the start. In the setting of natural resource management 

agencies, SDM calls for objective participation by conservation 

scientists, and transparent exposition and early articulation of decision criteria by decisionmakers. The independence of Research 

and Development and the National Forest System in the Forest 

Service provides a firm foundation for the objective delivery and 

application of science in the context of SDM, and more broadly 

in the sustainable management of national forests and grasslands.

The decision context drives the science needs, 

not the other way around.

6.2 Contemplating the Role of Uncertainty

For complex environmental and natural resource management 

problems, a need exists for explicit identification and analysis of 

uncertainties, clear articulation and separation of subjective and 

objective components, and a systematic framework for approaching decision analysis that includes explaining how uncertainties 

are used in developing evaluation and decision criteria. Different 

uncertainties and challenges present themselves at different stages 

of the decisionmaking process, and a wide variety of tools exist to 

address particular manifestations and aspects of decisionmaking 

under uncertainty (appendix B). Using SDM as an overarching 

framework can help to identify, critique, and discuss sources and 

implications of uncertainty, to support decisionmaking in the 

context of natural resource management.

6. Adopting Structured Decisionmaking: Conclusions and Implications

6.3 The Challenge and the Promise Ahead

SDM is a template for considering data, knowledge, values, and 

uncertainty transparently in the decision process. Indeed, uncertainty—appropriately explained and displayed—is a form of 

useful information. Analysts and planners can present uncertainty 

in a useful light by evaluating its implications in tradeoffs among 

alternative actions, and by estimating the incremental value of 

additional knowledge. The decisionmaker’s risk attitude will determine the implications of such uncertainties in practice (Cussen 

2010).

 Never has a better time existed to focus on enhancing decisionmaking. The coming years will demand closer attention to 

achieving and demonstrating tighter alignment with stated goals, 

despite increasing financial constraints and increasing social conflicts over natural resources. Decision processes will increasingly 

weigh environmental costs and benefits against those of economic 

development, social equity, and contribution to financial solvency.

SDM will not solve every problem, but it can improve transparency and clarity in decisionmaking processes. This review of 

SDM principles and concepts suggests that SDM can be helpful 

in a number of ways. Notable ways include deciphering, decomposing, and understanding complex problems that create the need 

for decisions; maintaining the sequences and internal consistency 

of the various stages of decisionmaking; articulating and quantifying values that guide the design and selection of alternatives; 

guiding the input from scientific, experiential, and traditional 

forms of knowledge; and organizing and documenting the logic 

of choice and tradeoff.

6.4 Summary

The craft of natural resource decisionmaking will be more demanding in the future, and it will require a more flexible toolkit 

to support decisions that address complex problems. Choosing 

decision processes and support tools in the face of new challenges 

may require multiple returns to the basics of decision support 

that have been discussed in this synthesis. Large-scale drivers of 

change—climate, demographics, global economic patterns, and 

changing social values—will increasingly provide surprises and 

uncertainties that will further shape decision spaces and prompt 

rethinking decisions already made. Resource managers will have 

to learn from the past without being shackled by it and use evolving tools to deliver the agency’s mission. Adoption of decision 

science principles can help resource managers address these challenges more effectively and, ideally, will lead to improved decisions and effective actions supporting sustainable management of 

forests and grasslands.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 43

Acknowledgments

This work—prompted by the Forest Service Deputy Chief for 

Research and Development Dr. Jim Reaves and his team of 

research station managers—is part of a larger effort by the Forest 

Service to synthesize major contemporary research themes. The 

authors thank Joe Carbone of the Forest Service’s National Forest 

System Washington Office for his expertise and sage advice on 

the application and delivery of these principles and techniques 

to those who manage decision processes and those who make 

decisions on land management. They extend thanks to all survey 

respondents and especially to Rick Brazell, Tony Erba, Glen 

Knowles, Gina Owens, and Jim Smalls for their interviews. They 

also thank the U.S. Geological Survey for partnering with the 

Forest Service on this effort. The authors also offer thanks to Dr. 

Deborah Finch for providing operational support through her 

partnership with the University of Arizona and to the University 

of Arizona’s School of Natural Resources and the Environment 

for their contributions to the project. They also thank Danny Lee 

for earlier reviews and for providing material from the Cohesive 

Strategy. In particular, the authors thank Andrew Bliss and Scott 

Wilbor for their tireless efforts and excellent staff support and 

Larry Fisher for his leadership and management. They are grateful for the reviews provided by Kerry Britton, Marilyn Buford, 

Bob Dow, Tony Erba, Don MacGregor, Chris Miller, and Jim 

Smalls. 

Photo by Larry A. Fisher
44 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Ager, A.A.; Finney, M.A.; Kerns, B.K.; Maffei, H. 2007. Modeling 

wildfire risk to northern spotted owl (Strix occidentalis caurina) 

habitat in central Oregon, USA. Forest Ecology and Management. 246: 45–56.

Ananda, J.; Herath, G. 2009. A critical review of multi-criteria 

decision making methods with special reference to forest management and planning. Ecological Economics. 68: 2535–2548.

Ascough, J.C. II; Maier, H.R.; Ravalico, J.K.; Strudley, M.W. 

2008. Future research challenges for incorporation of uncertainty 

in environmental and ecological decision-making. Ecological 

Modelling. 219(3–4): 383–399.

Benke, K.K.; Lowell, K.E.; Hamilton, A.J. 2007. Uncertainty 

analysis and risk assessment in the management of environmental 

resources. Australasian Journal of Environmental Management. 

14(4): 243–249.

Berg, J.; Bradshaw, B.; Carbone, J.; Chojnacky, C.; Conroy, 

S.; Cleaves, D.; Solomon, R.; Yonts-Shepard, S. 1999. Decision 

Protocol 2.0. FS-634. U.S. Department of Agriculture, Forest 

Service. 126 p.

Blomquist, S.M.; Johnson, T.D.; Smith, D.R.; Call, G.P.; Miller, 

B.N.; Thurman, W.M.; McFadden, J.E.; Parkin, M.J.; Bommer, 

G.S. 2010. Structured Decision-Making and Rapid Prototyping 

to Plan a Management Response to an Invasive Species. Journal 

of Fish and Wildlife Management. 1(1): 19–32.

Brugnach, M.; Dewulf, A.; Pahl-Wostl, C.; Taillieu, T. 2008. Toward a relational concept of uncertainty: about knowing too little, 

knowing too differently, and accepting not to know. Ecology and 

Society. 13(2): 30. http://www.ecologyandsociety.org/vol13/iss2/

art30/. (2012 November 20).

Brugnach, M.; Dewulf, A.; Henriksen, H.J.; van der Keur, P. 

2010. More is not always better: Coping with ambiguity in natural resources management. Journal of Environmental Management. 92(1): 78–84.

Cussen, K. 2010. Handle with care: Assessing the risks of the 

precautionary principle. Australasian Journal of Environmental 

Management. 16(2): 66–69.

Diaz Balteiro, L.; Romero, C. 2008. Making forestry decisions 

with multiple criteria: A review and an assessment. Forest Ecology and Management. 255: 3222–3241.

Festa-Bianchet, M. 2012. Rarity, willingness to pay and conservation. Animal Conservation. 15(1): 12–13.

References

Gareau, T.P.; Smith, R.G.; Barbercheck, M.E.; Mortensen, D.A. 

2010. Spider plots: A tool for participatory extension learning. 

Journal of Extension. 48(5): 8 p.

Gregory, R.; Long, G. 2009. Using structured decision making to 

help implement a precautionary approach to endangered species 

management. Risk Analysis. 29(4): 518–532.

Gregory, R.W.; Keeney, R.L. 2002. Making smarter environmental decisions. Journal of the American Water Resources Association. 38(6): 1601–1612.

Hammond, J.; Keeney, R.; Raiffa, H. 1999. Smart choices: A 

practical guide to making better decisions. Boston: Harvard Business School Press. 256 p.

Haynes, R.W.; Quigley, T.M.; Clifford, J.L.; Gravenmier, R.A. 

2001. Science and ecosystem management in the interior Columbia basin. Forest Ecology and Management. 153(1–3): 3–14.

Haynes, R.; Cleaves, D. 1999. Uncertainty, risk, and ecosystem 

management. In: Johnson, N.C.; Malk, A.J.; Szaro, R.C.; Sexton, 

W.T., eds. Ecological stewardship: A common reference for ecosystem management. Oxford, U.K.: Elsevier Science: 413–429.

He, H.S. 2009. A review of LANDIS and other forest landscape 

models for integration with wildlife models. In: Millspaugh, J.J.; 

Thompson, F.R. III, eds. Models for planning wildlife conservation in large landscapes. San Diego, CA: Elsevier Science: 

321–338.

Howard, R.A. 1966. Decision analysis: applied decision theory. 

In: Hertz, D.B.; Melese, J., eds. Proceedings of the Fourth International Conference on Operational Research. New York: Wiley: 

55–71.

Howard, R.A.; Matheson, J.E. 2005. Influence diagrams. Decision Analysis. 2: 127–143.

Janssen, J.A.E.B.; Krol, M.S.; Schielen, R.M.J.; Hoekstra, A.Y.; 

de Kok, J.L. 2010. Assessment of uncertainties in expert knowledge, illustrated in fuzzy rule-based models. Ecological Modelling. 221(9): 1245–1251.

Keeney, R.L. 1982. Decision analysis: an overview. Operations 

Research. 30(5): 803–838.

Keeney, R.L. 1996a. Value-focused thinking: a path to creative 

decisionmaking. Harvard Cambridge, MA: University Press.

416 p.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 45

Keeney, R.L. 1996b. Value-focused thinking: identifying decision opportunities and creating alternatives. European Journal of 

Operational Research. 92: 537–549.

Kiker, G.A.; Bridges, T.S.; Varghese, A.; Seager, T.P.; Linkov, I. 

2005. Application of multicriteria decision analysis in environmental decision making. Integrated Environmental Assessment 

and Management. 1(2): 95–108.

Marcot, B.G. 2006. Habitat modeling for biodiversity conservation. Northwestern Naturalist. 87(1): 56–65.

Marcot, B.G.; Hohenlohe, P.A.; Morey, S.; Holmes, R.; Molina, R.; Turley, M.; Huff, M.; Laurence, J. 2006. Characterizing 

species at risk II: using Bayesian belief networks as decision 

support tools to determine species conservation categories under 

the Northwest Forest Plan. Ecology and Society. 11(2): 12. http://

www.ecologyandsociety.org/vol11/iss2/art12/. (2012 November 

20).

Marcot, B.G.; Thompson, M.P.; Runge, M.C.; Thompson, F.R.; 

McNulty, S.; Cleaves, D.; Tomosy, M.; Fisher, L.A.; Bliss, A. 

2012a. Recent advances in applying decision science for managing national forests. Forest Ecology and Management. 285(1): 

123–132.

Marcot, B.G.; Allen, C.; Morey, S.; Shively, D.; White, R. 2012b. 

An expert panel approach to assessing potential effects of bull 

trout reintroduction on federally listed salmonids in the Clackamas River, Oregon. North American Journal of Fisheries Management. 32: 450–465.

Martin, T.G.; Burgman, M.A.; Fidler, F.; Kuhnert, P.M.; LowChoy, S.; McBride, M.; Mengersen, K. 2012. Eliciting expert 

knowledge in conservation science. Conservation Biology. 26: 

29–38.

Mendoza, G.A.; Martins, H. 2006. Multi-criteria decision analysis in natural resource management: A critical review of methods 

and new modelling paradigms. Forest Ecology and Management. 

230(1–3): 1–22.

Molina, R.; Marcot, B.G.; Lesher, R. 2006. Protecting rare, oldgrowth forest associated species under the survey and manage 

guidelines of the Northwest Forest Plan. Conservation Biology. 

20(2): 306–318.

Nichols, J.D.; Williams, B.K. 2006. Monitoring for conservation. 

Trends in Ecology & Evolution. 21: 668–673.

Pahl-Wostl, C. 2009. A conceptual framework for analyzing 

adaptive capacity and multi-level learning processes in resource 

governance regimes. Global Environmental Change. 19: 354–365.

Polya, G. 1973. How to solve it. Princeton, NJ: Princeton University Press. 253 p.

Potyondy, J.P.; Geler, T.W. 2011. Watershed condition classification technical guide. FS-978. Washington, DC: USDA Forest 

Service. 42 p.

Pouyat, R.V.; Weathers, K.C.; Hauber, R.; Lovett, G.M.; Bartuska, A.; Christenson, L.; Davis, J.L.D.; Findlay, S.E.G.; Menninger, H.; Rosi-Marshall, E.; Stine, P.; Lymn, N. 2010. The role 

of Federal agencies in the application of scientific knowledge. 

Frontiers in Ecology and the Environment. 8(6): 322–328.

Regan, H.M.; Colyvan, M.; Burgman, M.A. 2002. A taxonomy 

and treatment of uncertainty for ecology and conservation biology. Ecological Applications. 12(2): 618–628.

Rittenhouse, C.D.; Shifley, S.R.; Dijak, W.D; Fan, Z.F.; Thompson, F.R., III.; Millspaugh, J.J. 2011. Application of landscape 

and habitat suitability models to conservation: the Hoosier 

National Forest land-management plan. In: Li, C.; Lafortezza, 

R.; Chen, J., eds. Landscape ecology in forest management. Challenges and solutions for global change. Beijing and Springer-Verlag, Berlin Heidelberg: Higher Education Press: 299–328.

Runge, M.C.; Converse, S.J.; Lyons, J.E. 2011a. Which uncertainty? Using expert elicitation and expected value of information 

to design an adaptive program. Biological Conservation. 144: 

1214–1223.

Runge, M.C.; Bean, E.; Smith, D.R.; Kokos, S. 2011b. Non-native fish control below Glen Canyon Dam—Report from a structured decision making project. U.S. Geological Survey Open-File 

Report 2011–1012: 1–74.

Stage, A.R. 2003. How forest models are connected to reality: 

evaluation criteria for their use in decision support. Canadian 

Journal of Forest Research. 33: 410–421.

Stockmann, K.D.; Hyde, K.D.; Jones, G.J.; Loeffler, D.R, 

Silverstein, R.P. 2010. Integrating fuel treatment into ecosystem 

management: A proposed project planning process. International 

Journal of Wildland Fire. 19: 725–736.
46 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Thompson, F.R., III. 2004. The Hoosier-Shawnee ecological 

assessment: objectives, approach, and major findings. In: Thompson, F.R., III, editor. The Hoosier-Shawnee ecological assessment. 

Gen.Tech. Rep. NC-244. St. Paul, MN: U.S. Department of 

Agriculture, Forest Service, North Central Forest Experiment 

Station: 1–8.

U.S. Department of Agriculture (USDA). 2005. Monitoring 

and evaluation guidebook for the Tongass Land and Resource 

Management Plan. Version 05.0. Juneau, AK: Tongass National 

Forest. 109 p.

USDA. 2008a. Land and resource management plan, Tongass 

National Forest. R10-MB-603b. Juneau, AK: U.S. Department of 

Agriculture, Forest Service, Tongass National Forest. 468 p.

USDA. 2008b. Record of Decision. 2008 environmental impact 

statement. Tongass National Forest Land and Resource Management Plan Amendment. R10-MB-603a. Juneau, AK: U.S. 

Department of Agriculture, Forest Service, Tongass National 

Forest. 72 p.

USDA Forest Service. 2007. Record of decision. Five Buttes 

project environmental impact statement. Crescent, OR: U.S. 

Department of Agriculture, Forest Service, Deschutes National 

Forest. 43 p. 

USDA Forest Service. 2012. National Forest System land management planning, final rule. 36-CFR-219. Federal Register. 77: 

21162–21276.

USDA Forest Service and Bureau of Land Management (BLM).

1994. Record of decision on management of habitat for late-successional and old-growth forest related species within the range 

of the northern spotted owl. Northwest Forest Plan. Portland, 

OR: U.S. Department of Agriculture, Forest Service; Moscow, 

ID: Bureau of Land Management. 

USDA Forest Service and BLM. 2001. Record of decision and 

standards and guidelines for amendments to the survey and manage, protection buffer, and other mitigation measures standards 

and guidelines. Portland, OR: U.S. Department of Agriculture, 

Forest Service; Moscow, ID: Bureau of Land Management. 

USDA and U.S. Department of the Interior. 2000. Interior 

Columbia Basin final environmental impact statement proposed 

decision. Interior Columbia Basin ecosystem management project. Washington, DC: U.S. Department of Agriculture, Forest 

Service and U.S. Department of the Interior, Bureau of Land 

Management. 126 p.

Waldhardt, R.; Bach, M.; Borresch, R.; Breuer, L.; Diekötter, 

T.; Frede, H.G.; Gäth, S.; Ginzler, O.; Gottschalk, T.; Julich, S.; 

Krumpholz, M.; Kuhlmann, F.; Otte, A.; Reger, B.; Reiher, W.; 

Schmitz, K.; Schmitz, P.M.; Sheridan, P.; Simmering, D.; Weist, 

C.; Wolters, V.; Zörner, D. 2010. Evaluating today’s landscape 

multifunctionality and providing an alternative future: a normative scenario approach. Ecology and Society 15(3): 30. http://

www.ecologyandsociety.org/vol15/iss3/art30/. (2012 November 

20).

Wilhere, G.F.; Maguire, L.A.; Scott, J.M.; Rachlow, J.L.; 

Goble, D.D.; Svancara, L.K. 2012. Conflation of Values and 

Science: Response to Noss et al. Conservation Biology. doi: 

10.1111/j.1523-1739.2012.01900.x.

Williams, B.K.; Eaton, M.J.; Breininger, D.R. 2011. Adaptive 

resource management and the value of information. Ecological 

Modelling. 222(18): 3429–3426.

Williams, B.K.; Szaro, R.C.; Shapiro, C.D. 2007. Adaptive management: the U.S. Department of the Interior technical guide. 

Washington, DC: U.S. Department of the Interior. 72 p.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 47

Appendix A: Guidance for Tough Choices

What Are the Practical Ways To Deal With Complex Decisions?

Below are some practical approaches and tactics (not necessarily mutually exclusive) that could help if you 

find yourself stuck at a crossroads when facing a tough decision.

1. How Did I Get Here?

First, you might ask—“how did I get here?” It may help to go back through the previous stages of the structured 

decisionmaking (SDM) process, recounted below, and ensure that the work performed up to the decision point 

stage is sufficiently clear and comprehensive and provides enough information to make a fully informed decision.

Problem Structuring Stage

Objectives and evaluation criteria

•	 Be sure that all key terms in your objectives and decision criteria are fully defined.

•	 In simple English—articulating terms brings far greater clarity and may suggest further, 

guiding ideas, as words often have multiple meanings and connotations (linguistic uncertainty).

•	 If appropriate, with threshold or acceptable values and clear units of measure (evaluation metrics).

•	 Simplify, combine, or exclude some of your objectives and decision criteria.

•	 If some are redundant or highly correlated with others.

•	 If some are illogical or clearly contradictory with others.

Problem Analysis Stage

Creating alternatives and evaluating their consequences

•	 Break the problem down into smaller, simpler problems that already have solutions or methods for solutions.

•	 Separate out sequences of decisions and their effects into individual units.

•	 Separate out type of effects or geographic areas affected.

•	 Simplify the range of potential alternatives.

•	 Exclude outright alternatives that are infeasible or illegal.

•	 Combine the remaining alternatives into a smaller set that are feasible and practicable, and that all meet 

the overall, initial decision objectives.

•	 Identify those alternatives that are truly infeasible, illegal, or intractable, and the elements of decisions 

that would lead to them, in which case these elements may be ones to avoid in the final solution; call this the 

“painting the negative space first” approach.

•	 Work backwards from desired outcomes.

•	 Clearly state your decision criteria.

•	 For each outcome, write down several possible means by which it could be met.

•	 Combine all these means into a single table with columns for decision criteria and rows for possible means; 

compare the performance of each alternative to this table and identify those alternatives that best meet the 

most decision criteria; and consider amending some alternatives if new solutions can be identified.

Tradeoff analysis—or, characterizing and distinguishing alternatives

•	 Title each decision alternative with a catchy name that best characterizes its main attributes.

•	 Use this catchy name to help identify how alternatives differ in key ways, rather than how they are similar.

•	 Restate each alternative and its potential consequences as a story, avoiding the use of math, probabilities, 

and numbers.

•	 Group the alternatives into a few, well-named categories.

•	 Ensure that the first choice, if it exists, is among the general categories.

•	 The next choice is among the (fewer) alternatives within that category(s).
48 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

2. Use Methods to Balance Alternative and Objectives

Second, you might turn to decision-support approaches that directly relate to the decision point stage itself and that 

provide guidance for balancing alternatives and objectives.

Decision Point Stage

Comparing performance of alternatives and examining sensitivities

• Consider your objectives and decision criteria one at a time.

• Add them on incrementally and list which alternatives still meet the set.

• Help narrow down the set of alternatives for fuller analysis by using this evaluation.

• Consider the effects of each of your decision criteria.

• Think about how alternatives might rank differently if you were to drop each of the decision criteria

individually (sensitivity testing of the criteria).

• Consider alternative ways the most influential decision criteria could be met, perhaps outside the specific set of

decision alternatives presented.

Think “outside the box”—beyond the specific decision context

• Consider the outcome of some similar problem already solved.

• Use metaphor and analogy to reconsider the problem at hand. (This evaluation is a good heuristic to help discover a solution.)

• Look at the uncertainties—think of a problem that has similar unknowns.

• Focus on those key unknowns that are most uncertain and that might most influence the outcome (your decision) if known.

• Consult with fellow decisionmakers on similar problems they may have addressed.

• Find another situation in which the objectives have successfully been met, even if the problem statement per se was different, and

see how they got there.

• If you cannot find another similar problem, find one that has similar key unknowns and look at how the unknowns were handled

in that solution.

• Based on any of the above actions, find out if the problem, objectives, and perhaps even the decision criteria can be restated for

greater tractability, clarity, and feasibility.

• Exploit your success! (Polya 1973)

• Determine if you can apply a successful approach, and solution, to another problem.

Turning to more formalized approaches (e.g., multicriteria decision analysis)

• Use an objective hierarchy approach.

• Prioritize those objectives you initially laid out.

• Rank each alternative according to how well the alternatives meet each objective.

	 • Note that “How well” can be denoted by a probability or on a 1-to-10 scale.

• Sum all the rankings, weighting them by the priority level of the objectives.

	 • May establish priorities with ordinal scale (e.g., 1st, 2nd), or

	 • May establish priorities with subjective weights.

• Filter out the alternatives that fall below an acceptable threshold.

	 • Or simply order the alternatives according to their rankings.

	 • And then consider the top-ranked alternatives as best possible choices.

• And consider that perhaps one decision criterion may pertain to the degree of reversibility of a decision.

Still Can’t Make a Decision?

Ask yourself if not making a decision is more detrimental to the suite of objectives than is making a decision 

under uncertainty. Do not forget the key role of monitoring and adaptive management, although these actions 

do not substitute for a poorly made or ill-informed decision.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 49

Many tools or approaches can serve more than one stage. Types of uncertainty (see table 2) addressed by each tool or approach: 

LU = linguistic uncertainty, KU = knowledge uncertainty, VU = variability uncertainty, DU = decision or preference uncertainty.

Appendix B: Tools and Approaches Useful in Stages of Structured Decisionmaking, 

With Examples of Applications. 

Name of Tool or Approach Use and Type of Uncertainty Addressed Examples of Application

Problem Structuring Stage

Cognitive mapping and modeling Organizes and synthesizes system components 

and dynamics. LU, KU

Mendoza and Prabhu 2005 (participatory 

modeling and sustainable forest management), 

Wolfslehner and Vacik 2011 (forest management sustainability evaluation)

Influence diagrams Represents key system parameters, decision 

points, and outcomes in a qualitative graph. 

Can be further developed into other quantitative 

model types (e.g., state and transition models, 

Bayesian network). LU, KU

Bashari et al. 2009 (rangeland management)

Decision tree Diagrams choices, outcomes (utilities), and 

probabilities to evaluate expected values of alternatives; evaluates knowledge and preference 

(risk attitude) uncertainties; useful for all SDM 

stages. KU, VU, DU

Coops et al. 2011 (tree growth modeling), 

Failing et al. 2004 (adaptive management), Wan 

et al. 2009 (vegetation modeling)

Objectives hierarchy analysis Helps resolve conflicting objectives, social 

values, and preferences. VU, DU

Maguire et al. 2004 (invasive species 

management)

Means-end objective network Helps to identify causal relationships between 

actions and outcomes, and to connect means 

objectives to fundamental objectives. KU

Keeney 1996b (hydroelectric management)

Problem Analysis Stage

Simulation modeling A broad set of tools useful for modeling system 

dynamics and response to management; e.g., 

timber growth and yield, wildfire, hydrology, 

climate change, weather. VU, DU

Krawchuk and Cumming 2011 (forest fire 

under climate change), Vuilleumier et al. 2011 

(invasive species control), Rittenhouse et al. 

2010 (forest management, succession, and 

wildlife habitat)

Bayesian networks Models the conditional dependence between 

variables accounting for prior knowledge. KU, VU

Aalders 2008 (land-use decisions), Dlamini 

2010 (fire risk analysis), Aguilera et al. 2010 

(species distribution), Galan et al. 2009 

(reforestation)

Data mining Analyzes relationships between numerous data 

fields in an existing database, gaining new 

knowledge. KU

Dlamini 2011 (vegetation mapping)

Fuzzy logic, fuzzy set theory models Allows computation of vague and uncertain data 

using a membership function for data inputs. 

LU, KU, VU

Glenz et al. 2008 (flooding impact on woody 

species growth), Reeves et al. 2006 (evaluating watershed condition and aquatic habitat), 

Andreucci et al. 2000 (environmental responses 

of plant associations)

Rough set theory Unknown values for data are represented by 

their approximated lower and upper bounds. 

KU, VU

Xie et al. 2011 (land cover data retrieval), Tan 

2005 (life-cycle environmental burdens of 

products or processes)
50 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Analytic hierarchy process (AHP) Hybrid approach allowing for imprecise and 

vague definitions embedded within a hierarchy. 

Allows for joint consideration of objective and 

subjective information with expert judgment. 

KU, VU

Coulter et al. 2006 (forest roads), Hessburg 

et al. 2007 (wildfire danger, fuels treatments), 

Vadrevu et al. 2009 (fire risk evaluation)

Analytic Network Process (ANP) Better for expert judgment use and capturing 

feedbacks and interdependencies, relative to 

AHP. KU, VU

Bottero et al. 2011 (wastewater treatment assessment), Wolfslehner et al. 2005, Wolfslehner 

and Vacik 2007 (sustainable forest management)

Rule and network induction Results in rules or networks based on the 

relationship between a given set of attributes; 

networks can then be further developed as 

probability transition models or Bayesian networks. LU, KU, VU

Berger 2004 (crop suitability)

Neural networks Allows modeling of nonlinear and unknown 

relationships. KU, VU

Ejrnaes et al. 2002 (habitat quality), Scrinzi et 

al. 2007 (forest distribution data), Lippitt et al. 

2008 (species distribution model), Özçelik et al. 

2010 (tree bole volume)

Reliability analysis Assessment of the potential failures (probabilities and timing) of a system and their effects. VUChowdhury et al. 2009 (drinking water 

contaminants)

Scenario analysis Considers multiple outcomes providing a range 

of alternatives and their likelihoods. KU

Bohnet et al. 2011 (sustainable landscape 

development), Dougill 2008 (grassland response 

to climate change)

Comparative risk assessment Extends traditional risk assessment to include 

decision space available to managers and 

stakeholders to allow them to explore tradeoffs 

between alternative courses of action. VU, KU

Ager et al. 2007 (fuel-treatment strategies), 

Calkin et al. 2011 (wildland fire management)

Decision Point Stage

Valuation and cost-benefit analysis A family of stated and revealed preference models for establishing the value of nonmarketable 

goods and services. VU, DU

Champ et al. 2010 (hedonic pricing model and 

homebuyer wildfire risk perceptions), Holmes 

et al. 2004 (contingent valuation and riparian 

restoration), Rolfe et al. 2000 (choice modeling 

and tropical rainforest preservation)

Exact optimization methods A variety of mathematical techniques that can 

identify a set of nondominated alternatives or 

a single best answer. Includes linear programming, nonlinear programming, integer programming, others. VU, DU

Thompson et al. 2010 (forest road erosion control), Toth et al. 2009 (spatial harvest scheduling 

with habitat objectives), Snyder et al. 2004 

(reserve site selection)

Heuristic optimization methods Iteratively update solution(s) through process 

of information exchange, self adaptation, and 

competition; entails perturbing decision vector, 

accepting and retain new solution(s) according 

to various criteria. Includes evolutionary algorithms, genetic algorithms, simulated annealing, 

tabu search, others. VU, DU

Icaga 2005 (water quality monitoring), Kennedy 

et al. 2008 (fuel-treatment planning), Ducheyne 

et al. 2004 (even-flow forest management), 

Madej et al. 2006 (sediment reduction and 

road removal)

Multiattribute utility theory (MAUT) Defines a cardinal utility function according to 

all criteria, typically by defining performance of 

each. VU, DU

Merkhofer et al. 1997 (siting hazardous waste 

management facility), Moffett et al. 2005 (conservation planning)

Analytic hierarchy process (AHP) Pairwise comparisons of attributes and criteria 

based on linguistic scale then converted to 

ratio-scale weights, which can be aggregated up 

through an objective hierarchy. LU, VU, DU

Darin et al. 2010 (invasive plant management), 

Wolfslehner et al. 2005 (sustainable forest 

management)
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 51

Simple multiattribute ranking technique (SMART) Assigns criteria weights on 0–100 scale, by 

scaling weights for all attributes relative to the 

most important attribute, assigned 100. DU

Kajanus et al. 2004 (tourism management 

and sustainable development), Reynolds 2001 

(salmon habitat restoration)

Stochastic multicriteria acceptability analysis 

(SMAA)

Family of methods designed to facilitate decisionmaking for contexts in which both criteria 

and preferences may be subject to uncertainty; 

based on exploring the weight space to describe 

the preferences that would make each alternative the most preferred one, or that would give a 

certain rank for a specific alternative. DU, VU

Kangas et al. 2003 (forest management plan 

development), Kangas et al. 2005 (natural 

resource management by Finnish Forest and 

Park Service)

Implementation and Monitoring Stage

Multimodel analysis Multiple models for evaluating various representations of expert knowledge, scales, and variable 

interactions. KU, VU

Rehme et al. 2011 (general application)

Multiagent systems Models multiple interacting “agents” (programs, 

humans, or human teams), representing diverse 

interests, in role-playing scenarios. VU

Lynam et al. 2002 (rangeland management)
52 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Aalders, I. 2008. Modeling land-use decision behavior with 

Bayesian belief networks. Ecology and Society. 13(1): 16. http://

www.ecologyandsociety.org/vol13/iss1/art16/. (2012 November 

20).

Ager, A.A.; Finney, M.A.; Kerns, B.K.; Maffei, H. 2007. Modeling wildfire risk to northern spotted owl (Strix occidentalis 

caurina) habitat in central Oregon, USA. Forest Ecology and 

Management. 246: 45–56.

Ascough, J.C.; II, Maier, H.R.; Ravalico, J.K.; Strudley, M.W. 

2008. Future research challenges for incorporation of uncertainty 

in environmental and ecological decision-making. Ecological 

Modelling. 219(3–4): 383–399.

Bashari, H.; Smith, C.; Bosch, O.J.H. 2009. Developing decision support tools for rangeland management by combining state 

and transition models and Bayesian belief networks. Agricultural 

Systems. 99: 23–34.

Berger, P.A. 2004. Rough set rule induction for suitability assessment. Environmental Management. 34(4): 546–558.

Bohnet, I.C.; Roebeling, P.C.; Williams, K.J.; Holzworth, D.; 

van Grieken, M.E.; Pert, P.L.; Kroon, F.J.; Westcott, D.A.; Brodie, J. 2011. Landscapes toolkit: an integrated modelling framework to assist stakeholders in exploring options for sustainable 

landscape development. Landscape Ecology. 26(8): 1179–1198.

Bottero, M.; Comino, E.; Riggio, V. 2011. Application of the analytic hierarchy process and the analytic network process for the 

assessment of different wastewater treatment systems. Environmental Modelling & Software. 26: 1211–1224.

Calkin, D.; Ager, A.; Thompson, M. 2011. A comparative risk assessment framework for wildland fire management: the 2010 cohesive strategy science report. Gen. Tech. Rep. RMRS-GTR-262. 

Fort Collins, CO: U.S. Department of Agriculture, Forest Service, 

Rocky Mountain Research Station. 63 p.

Champ, P.A.; Donovan, G.H.; C.M. Barth. 2010. Homebuyers 

and wildfire risk: a Colorado Springs case study. Society and 

Natural Resources. 23: 58–70.

Chowdhury, S.; Champagne, P.; McLellan, P.J. 2009. Uncertainty characterization approaches for risk assessment of DBPs in 

drinking water: a review. Journal of Environmental Management. 

90(5): 1680–1691.

References—Appendix B

Coulter, E.D.; Sessions, J.; Wing, M.G. 2006. Scheduling forest 

road maintenance using the analytic hierarchy process and heuristics. Silva Fennica. 40: 143–160.

Darin, G.M.S.; Schoenig, S.; Barney, J.N.; Panetta, F.D.; 

DiTomaso, J.M. 2010. WHIPPET: A novel tool for prioritizing 

invasive plant populations for regional eradication. Journal of 

Environmental Management. 92(1): 131–139.

Dlamini, W.M. 2010. A Bayesian belief network analysis of factors influencing wildfire occurrence in Swaziland Environmental 

Modelling & Software. 25(2): 199–208.

Dlamini, W.M. 2011. A data mining approach to predictive vegetation mapping using probabilistic graphical models. Ecological 

Informatics. 6: 111–124.

Ejrnaes, R.; Aude, E.; Nygaard, B.; Munier, B. 2002. Prediction 

of habitat quality using ordination and neural networks. Ecological Applications. 12(4): 1180–1187.

Failing, L.; Horn, G.; Higgins, P. 2004. Using expert judgment 

and stakeholder values to evaluate adaptive management options. 

Ecology and Society. 9(1): 13. http://www.ecologyandsociety.org/

vol9/iss1/art13. (2012 November 20).

Glenz, C.; Iorgulescu, I.; Kienast, F.; Schlaepfer, R. 2008. Modelling the impact of flooding stress on the growth performance of 

woody species using fuzzy logic. Ecological Modelling. 218(1–2): 

18–28.

Hessburg, P.F.; Reynolds, K.M.; Keane, R.E.; James, K.M.; Salter, R.B. 2007. Evaluating wildland fire danger and prioritizing 

vegetation and fuels treatments. Forest Ecology and Management. 247(1–3): 1–17.

Holmes, T.P.; Bergstrom, J.C.; Huszar, E.; Kask, S.B.; Orr, F., III. 

2004. Contingent valuation, net marginal benefits, and the scale 

of riparian ecosystem restoration. Ecological Economics. 49(1): 

19–30.

Icaga, Y. 2005. Genetic algorithm usage in water quality monitoring networks optimization in Gediz (Turkey) River Basin. 

Environmental Monitoring and Assessment. 108(1–3): 261–277.

Kajanus, M.; Kangas, J.; Kurtilla, M. 2004. The use of value 

focused thinking and the A’WOT hybrid method in tourism 

management. Tourism Management. 25(4): 499–506.
The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System 53

Kangas, J.; Hokkanen, J.; Kangas, A.A.; Lahdelma, R.; Salminen, P. 2003. Applying stochastic multicriteria acceptability 

analysis to forest ecosystem management with both cardinal and 

ordinal criteria. Forest Science. 49(6): 928–937.

Kangas, J.; Store, R.; Kangas, A. 2005. Socioecological landscape 

planning approach and multicriteria acceptability analysis in multiple-purpose forest management. Forest Policy and Economics. 

7(4): 603–614.

Kennedy, M.C.; Ford, E.D.; Singleton, P.; Finney, M.; Agee, J.K. 

2008. Informed multi-objective decision-making in environmental management using Pareto optimality. Journal of Applied 

Ecology. 45(1): 181–192.

Krawchuk, M.A.; Cumming, S.G. 2011. Effects of biotic feedback and harvest management on boreal forest fire activity under 

climate change. Ecological Applications. 21: 122–136.

Lynam, T.; Bousquet, F.; Le Page, C.; d’Aquino, P.; Barreteau, 

O.; Chinembiri, F.; Mombeshora, B. 2002. Adapting science to 

adaptive managers: spidergrams, belief models, and multi-agent 

systems modeling. Conservation Ecology. 5(2): 24. http://www.

consecol.org/vol5/iss2/art24. (2012 November 20).

Maguire, L.A. 2004. What can decision analysis do for invasive 

species management? Risk Analysis. 24(4): 859–868.

Mendoza, G.A.; Prabhu, R. 2005. Combining participatory modeling and multi-criteria analysis for community-based 

forest management. Forest Ecology and Management. 207(1–2): 

145–156.

Merkhofer, M.W.; Conway, R.; Anderson, R.G. 1997. Multiattribute utility analysis as a framework for public participation 

in siting a hazardous waste management facility. Environmental 

Management. 21: 831–839.

Moffett, A.; Garson, J.; Sarkar, S. 2005. MultCSync: a software 

package for incorporating multiple criteria in conservation planning. Environmental Modelling & Software. 20(10): 1315–1322.

Reeves, G.H.; Williams, J.E.; Burnet, K.M.; K. Gallo. 2006. The 

Aquatic Conservation Strategy of the Northwest Forest Plan. 

Conservation Biology. 20(2): 319–329.

Rehme, S.E.; Powell, L.A.; Allen, C.R. 2011. Multimodel 

inference and adaptive management. Journal of Environmental 

Management. 92(5): 1360–1364.

Reynolds, K.M. 2001. Prioritizing salmon habitat restoration 

with the AHP, SMART, and uncertain data. In: Schmoldt, D.; 

Kangas, J.; Mendoza, G.M.; Pesonen, M.; eds. The Analytic Hierarchy Process in natural resources and environmental decision 

making. Dordrecht, The Netherlands: Kluwer Academic Publishers: 199–218 

Scrinzi, G.; Marzullo, L.; Galvagni, D. 2007. Development of a 

neural network model to update forest distribution data for managed alpine stands. Ecological Modelling. 206(3–4): 331–346.

Thompson, M.P.; Sessions, J.; Boston, K.; Skaugset, A.; Tomberlin, D. 2010. Forest road erosion control using multiobjective 

optimization. Journal of the American Water Resources Association. 46(4): 712–723.
54 The Science of Decisionmaking: Applications for Sustainable Forest and Grassland Management in the National Forest System

Toth, S.F.; Haight, R.G.; Snyder, S.A.; George, S.; Miller, J.R.; 

Gregory, M.S.; Skibbe, A.M. 2009. Reserve selection with minimum contiguous area restrictions: An application to open space 

protection planning in suburban Chicago. Biological Conservation. 142(8): 1617–1627.

Vuilleumier, S.; Buttler, A.; Perrin, N.; Yearsley, J.M. 2011. Invasion and eradication of a competitively superior species in heterogeneous landscapes. Ecological Modelling. 222(3): 398–406.

Wan, L.; Zhang, B.; Kemp, P.; Li, X. 2009. Modelling the 

abundance of three key plant species in New Zealand hill-pasture 

using a decision tree approach. Ecological Modelling. 220(15): 

1819–1825.

Wolfslehner, B.; Vacik, H. 2011. Mapping indicator models: 

From intuitive problem structuring to quantified decision-making 

in sustainable forest management. Ecological Indicators. 11(2): 

274–283.

Wolfslehner, B.; Vacik, H.; Lexer, M.J. 2005. Application of the 

analytic network process in multi-criteria analysis of sustainable 

forest management. Forest Ecology and Management. 207(1–2): 

157–170.

Xie, F.; Lin, Y.; Ren, W. 2011. Optimizing model for land use/

land cover retrieval from remote sensing imagery based on variable precision rough sets. Ecological Modelling. 222(2): 232–240.