U.S. Department of the Interior

U.S. Geological Survey

Scientific Investigations Report 2015–5176

Prepared in cooperation with the Bureau of Reclamation, National Park Service, and Argonne 

National Laboratory

Decision Analysis to Support Development of 

the Glen Canyon Dam Long-Term Experimental 

and Management Plan
Covers. Front: An oar boat with university students from Northern Arizona University’s Grand 

Canyon Semester passes river mile 61 near the confluence of the Little Colorado River and the 

Colorado River. This area is known for its biological and cultural significance, as the stronghold of 

the endangered humpback chub, and a place of importance in many tribal histories. Photograph 

taken by Amy S. Martin, Northern Arizona University, November 2014.

Back: Lower Beaver Falls lies approximately 4 miles up Havasu Creek from its confluence with the 

Colorado River at river mile 157 and is the site of ongoing endangered humpback chub translocations. 

Photograph taken by Amy S. Martin, National Park Service, October 2013.
Decision Analysis to Support 

Development of the Glen Canyon 

Dam Long-Term Experimental and 

Management Plan

By Michael C. Runge, Kirk E. LaGory, Kendra Russell, Janet R. Balsom, R. Alan 

Butler, Lewis G. Coggins, Jr., Katrina A. Grantz, John Hayse, Ihor Hlohowskyj, 

Josh Korman, James E. May, Daniel J. O’Rourke, Leslie A. Poch, James R. 

Prairie, Jack C. VanKuiken, Robert A. Van Lonkhuyzen, David R. Varyu, Bruce T. 

Verhaaren, Thomas D. Veselka, Nicholas T. Williams, Kelsey K. Wuthrich, Charles 

B. Yackulic, Robert P. Billerbeck, and Glen W. Knowles

Prepared in cooperation with the Bureau of Reclamation, National Park Service, 

and Argonne National Laboratory

Scientific Investigations Report 2015–5176

U.S. Department of the Interior

U.S. Geological Survey
U.S. Department of the Interior

SALLY JEWELL, Secretary

U.S. Geological Survey

Suzette M. Kimball, Acting Director

U.S. Geological Survey, Reston, Virginia: 2015

For more information on the USGS—the Federal source for science about the Earth, its natural and living 

resources, natural hazards, and the environment—visit http://www.usgs.gov or call 1–888–ASK–USGS.

For an overview of USGS information products, including maps, imagery, and publications, 

visit http://www.usgs.gov/pubprod/.

Any use of trade, firm, or product names is for descriptive purposes only and does not imply endorsement by the 

U.S. Government.

Although this information product, for the most part, is in the public domain, it also may contain copyrighted materials 

as noted in the text. Permission to reproduce copyrighted items must be secured from the copyright owner.

Suggested citation:

Runge, M.C., LaGory, K.E., Russell, Kendra, Balsom, J.R., Butler, R.A., Coggins, L.G., Jr., Grantz, K.A., Hayse, John, Hlohowskyj, Ihor, Korman, Josh, May, J.E., O’Rourke, D.J., Poch, L.A., Prairie, J.R., VanKuiken, J.C., Van Lonkhuyzen, R.A., 

Varyu, D.R., Verhaaren, B.T., Vesekla, T.D., Williams, N.T., Wuthrich, K.K., Yackulic, C.B., Billerbeck, R.P., and Knowles, 

G.W., 2015, Decision analysis to support development of the Glen Canyon Dam Long-Term Experimental and Management Plan: U.S. Geological Survey Scientific Investigations Report 2015–5176, 64 p., http://dx.doi.org/10.3133/

sir20155176.

ISSN 2328-0328 (online)
iii

Acknowledgments

A project this large owes a debt of gratitude to many people who generously offered their 

support, insights, and expertise. The authors thank Anne Castle and Lori Caramanian from the 

Department of the Interior for their vision and support for the inclusion of decision analysis 

methods in the development of the Environmental Impact Statement. Ted Melis (U.S. Geological 

Survey Grand Canyon Monitoring and Research Center) provided extensive insights during the 

first 2 years of this project, helping shape the design of this analysis. Jack Schmidt and Scott 

VanderKooi (U.S. Geological Survey Grand Canyon Monitoring and Research Center) shared their 

expertise and leadership, and provided access to the scientific experts at their Center.

This project required expertise from many agencies outside of the U.S. Geological Survey. Too 

numerous to list here, the many members of the subject-matter expert teams provided critical 

input to the development of performance metrics and the quantitative evaluation of alternatives. 

The efforts to articulate and attempt to quantify tribal viewpoints would have been impossible 

without the dedication and expertise of many representatives, including Kurt Dongoske (Pueblo 

of Zuni), Mike Yeatts (Hopi Tribe), Loretta Jackson-Kelly and Peter Bungart (Hualapai Tribe), 

Charley Bulletts (Southern Paiute Consortium), Robert Kirk and Tony Joe (Navajo Nation), and 

Sarah Rinkevich (U.S. Fish and Wildlife Service). Terry Walshe and David Smith provided insightful comments on an earlier draft of this report. Rich Valdez and Carl Walters shared their depth 

of knowledge and astute perspectives on many aspects of the Grand Canyon ecosystem. The 

authors are deeply grateful to the representatives from the stakeholder agencies who participated in the swing-weighting exercise; and worked diligently, enthusiastically, and with integrity 

to share their perspectives. 

Funding for this project was provided by the Bureau of Reclamation.

v

Contents

Acknowledgments........................................................................................................................................iii

1 Abstract.....................................................................................................................................................1

2 Introduction..............................................................................................................................................2

3 Decision Analysis....................................................................................................................................4

3.1 Structured Decision Making .....................................................................................................4

3.2 Multicriteria Decision Analysis.................................................................................................4

3.2.1 A Full Articulation of Resource Goals..........................................................................5

3.2.2 Performance Metrics .....................................................................................................5

3.2.3 Transparent and Quantitative Evaluation of Alternatives.........................................6

3.2.4 Tradeoff Analysis.............................................................................................................6

3.3 Uncertainty, Value of Information, and Adaptive Management..........................................6

3.3.1 Adaptive Management Versus Experimental Management....................................6

3.3.2 Articulating Uncertainty as Competing Hypotheses .................................................7

3.3.3 Deconstructing Complex Alternatives.........................................................................7

3.3.4 Value of Information........................................................................................................8

3.3.5 Experimental and Adaptive Design ..............................................................................8

4 Resource Goals and Performance Metrics ........................................................................................9

4.1 Aquatic Ecology...........................................................................................................................9

4.1.1 Humpback Chub ..............................................................................................................9

4.1.2 Other Native Fish.............................................................................................................9

4.1.3 Rainbow Trout Fishery ....................................................................................................9

4.1.4 Nonnative Aquatic Species...........................................................................................9

4.2 Archaeological and Cultural Resources ...............................................................................10

4.3 Hydropower and Energy ..........................................................................................................11

4.4 Natural Processes.....................................................................................................................11

4.5 Recreational Experience..........................................................................................................11

4.6 Riparian Vegetation...................................................................................................................11

4.7 Sediment.....................................................................................................................................12

4.8 Tribal Resources........................................................................................................................12

4.8.1 Health of the Ecosystem ..............................................................................................12

4.8.2 Sites of Cultural Importance........................................................................................12

4.8.3 Respect for Life..............................................................................................................13

4.8.4 Sacred Integrity of Grand, Marble, and Glen Canyons...........................................13

4.8.5 Stewardship ...................................................................................................................13

4.8.6 Tribal Connections to the Canyons.............................................................................14

4.8.7 Economic Opportunity ..................................................................................................14

4.8.8 Tribal Water Rights and Supply...................................................................................14

4.8.9 Process Objectives .......................................................................................................14

4.9 Water Delivery ...........................................................................................................................14

5 Alternatives ............................................................................................................................................15

5.1 Action Elements.........................................................................................................................15

5.1.1 Monthly Release Pattern and Other Base Operations............................................15
vi

5.1.2 High-Flow Experiments ................................................................................................15

5.1.3 Trout Management Flows ............................................................................................15

5.1.4 Other Flow Manipulations............................................................................................15

5.1.5 Nonflow Actions............................................................................................................17

5.1.6 Long-Term Strategies ...................................................................................................17

5.2 Alternative A (No Action).........................................................................................................18

5.3 Alternative B...............................................................................................................................18

5.3.1 Alternative Long-Term Strategies...............................................................................18

5.4 Alternative C...............................................................................................................................18

5.4.1 Alternative Long-Term Strategies...............................................................................18

5.5 Alternative D...............................................................................................................................19

5.5.1 Alternative Long-Term Strategies...............................................................................19

5.6 Alternative E...............................................................................................................................19

5.6.1 Alternative Long-Term Strategies...............................................................................20

5.7 Alternative F ...............................................................................................................................20

5.8 Alternative G...............................................................................................................................20

6 Quantitative Methods ...........................................................................................................................20

6.1 Modeling Overview...................................................................................................................20

6.2 Hydrology....................................................................................................................................22

6.2.1 Critical Uncertainty .......................................................................................................22

6.3 Sediment Dynamics ..................................................................................................................22

6.3.1 Critical Uncertainty .......................................................................................................22

6.4 Temperature ...............................................................................................................................23

6.5 Coupled Rainbow Trout-Humpback Chub Dynamics...........................................................23

6.5.1 Critical Uncertainties....................................................................................................23

6.6 Fish Habitat Suitability..............................................................................................................23

6.7 Archaeological and Cultural Resources ...............................................................................24

6.8 Hydropower................................................................................................................................24

6.9 Recreation ..................................................................................................................................25

6.10 Riparian Vegetation ..................................................................................................................25

7 Consequence Analysis Results...........................................................................................................25

7.1 Humpback Chub Results ..........................................................................................................25

7.2 Rainbow Trout Fishery Results................................................................................................26

7.3 Archeological and Cultural Resources Results ...................................................................32

7.4 Hydropower Generation and Capacity Results....................................................................32

7.5 Recreational Experience Results ...........................................................................................32

7.6 Riparian Vegetation Results ....................................................................................................32

7.7 Sediment Results.......................................................................................................................37

7.8 Tribal Resources Results..........................................................................................................37

7.9 Full Consequence Table ...........................................................................................................37

8 Decision Analysis Results....................................................................................................................42

8.1 Swing-Weighting Method and Results..................................................................................42

8.2 Multicriteria Decision Analysis ..............................................................................................42

8.3 Expected Value of Information................................................................................................48

8.4 Effects of Climate Change........................................................................................................51
vii

9 Discussion ..............................................................................................................................................52

9.1 Evaluation of Alternatives........................................................................................................52

9.2 Motivation for Adaptive Management...................................................................................53

9.3 Decision Analysis and the National Environmental Policy Act .........................................54

10 Summary...............................................................................................................................................54

11 References Cited.................................................................................................................................54

12 Appendix 1. Disclaimers .................................................................................................................59

12.1 Standard Disclaimer...............................................................................................................59

12.2 Arizona Department of Water Resources Disclaimer.......................................................59

12.3 Arizona Game and Fish Department Disclaimer................................................................60

12.4 Salt River Project Explanatory Letter...................................................................................60

12.5 Utah Associated Municipal Power Systems Explanatory Letter ....................................61

12.6 Colorado River Energy Distributors Association Submittal Letter..................................61

Figures

1. Map showing the Colorado River ecosystem below Glen Canyon Dam, depicting

the Glen Canyon Dam Adaptive Management Program project area.................................3

2. Diagram showing model flow for the analysis of the Long-Term Experimental and

Management Plan long-term strategies against the performance metrics .....................21

3. Graphs showing uncertainty in the humpback chub population dynamics as

represented by four alternative sets of parameters.............................................................24

4. Box plots showing minimum number of adult humpback chub (performance 

metric 1) forecast for 19 long-term strategies with equal weighting of traces................26

5. Box plots showing temperature suitability index for humpback chub at river 

mile 157 and 213 (performance metric 2) forecast for 19 long-term strategies 

with equal weighting of traces.................................................................................................30

6. Box plots showing rainbow trout catch rate (performance metric 3) forecast for 

19 long-term strategies with equal weighting of traces ......................................................30

7. Box plots showing rainbow trout emigration rate from Glen Canyon (performance

metric 4) forecast for 19 long-term strategies with equal weighting of traces................31

8. Box plots showing abundance of high-quality rainbow trout (performance metric 5)

forecast for 19 long-term strategies with equal weighting of traces ................................31

9. Box plots showing wind transport of sediment index (performance metric 6)

forecast for 19 long-term strategies with equal weighting of traces ................................33

10. Box plots showing Glen Canyon flow index (performance metric 7) forecast for 

19 long-term strategies with equal weighting of traces ......................................................33

11. Box plots showing cultural resources time-off-river index (performance metric 8)

forecast for 19 long-term strategies with equal weighting of traces ................................34

12. Box plots showing annualized net present value of hydropower generation

(performance metric 9) forecast for 19 long-term strategies with equal weighting 

of traces .......................................................................................................................................34

13. Box plots showing annualized net present value of hydropower capacity

(performance metric 10) forecast for 19 long-term strategies with equal weighting

of traces .......................................................................................................................................35
viii

14. Box plots showing camping area index (performance metric 11) forecast for 19

long-term strategies with equal weighting of traces ...........................................................35

15. Box plots showing fluctuation index (performance metric 12) forecast for 19

long-term strategies with equal weighting of traces ...........................................................36

16. Box plots showing Glen Canyon rafting use metric (performance metric 13)

forecast for 19 long-term strategies with equal weighting of traces ................................36

17. Box plots showing riparian vegetation index (performance metric 14) forecast 

for 19 long-term strategies with equal weighting of traces ................................................38

18. Box plots showing sand load index (performance metric 15) forecast for 19

long-term strategies with equal weighting of traces ...........................................................38

19. Box plots showing marsh vegetation ratio (ratio of average marsh cover over

20-year period to current marsh cover, performance metric 16) forecast for 19

long-term strategies with equal weighting of traces ...........................................................39

20. Box plots showing frequency of mechanical removal (performance metric 17)

forecast for 19 long-term strategies with equal weighting of traces ................................39

21. Box plots showing frequency of trout management flows (performance metric 18)

forecast for 19 long-term strategies with equal weighting of traces ................................40

22. Graphs showing principal components analysis of the weights on the performance

metrics across stakeholder agencies.....................................................................................45

23. Box plots showing joint-lead agency weighted performance across 18 metrics

forecast for 19 long-term strategies using equal weighting of hydrological traces .......46

24. Box plots showing joint-lead agency weighted performance across 18 metrics,

sorted by mean performance, forecast for 19 long-term strategies using equal

weighting of hydrological traces .............................................................................................48

25. Box plots showing weighted performance across 18 metrics, for four stakeholder

agencies, sorted in descending order of mean performance.............................................49

26. Box plots showing mean annual inflow to Lake Powell during the 20-year period of

the Long-Term Experimental and Management Plan for 21 historical hydrological

traces and 112 climate-change-influenced hydrological traces........................................51

27. Box plots showing joint-lead agency weighted performance across 18 metrics

forecast for 19 long-term strategies using the climate-change weighting of

hydrological traces.....................................................................................................................52
ix

Tables

1. Summary of the 18 performance metrics used in the decision analysis ..........................10

2. Operational characteristics of Long-Term Experimental and Management Plan

alternatives ..................................................................................................................................16

3. Triggered and experimental elements of the Long-Term Experimental and

Management Plan long-term strategies.................................................................................17

4. Effect of critical uncertainty on minimum adult humpback chub population size

(performance metric 1)..............................................................................................................27

5. Effect of hydrological trace on minimum adult humpback chub population size

(performance metric 1)..............................................................................................................28

6. Expected value of information, by performance metric, for resolving critical

uncertainty...................................................................................................................................29

7. Full consequence table showing the mean value of each performance metric for

each long-term strategy ............................................................................................................41

8. Swing-weighting response form..............................................................................................43

9. Agencies participating in the swing-weighting exercise....................................................44

10. Weighted performance for 19 long-term strategies using the weights on the

performance metrics expressed by 14 participating agencies ..........................................47

11. Expected value of perfect information for resolving various uncertainties, as a

function of the agency-specific weights ................................................................................50

Conversion Factors

[Inch/Pound to International System of Units]

Multiply By To obtain

Length

inch (in.) 2.54 centimeter (cm)

inch (in.) 25.4 millimeter (mm)

foot (ft) 0.3048 meter (m)

mile (mi) 1.609 kilometer (km)

Area

acre 4,047 square meter (m2

)

acre 0.4047 hectare (ha)

Volume

acre-foot (acre-ft) 1,233 cubic meter (m3

)

acre-foot (acrea-ft) 0.001233 cubic hectometer (hm3

) 

Flow rate

cubic foot per second (ft3

/s) 0.02832 cubic meter per second (m3/s)

Vertical coordinate information is referenced to the North American Vertical Datum of 1988 

(NAVD 88).

Horizontal coordinate information is referenced to the North American Datum of 1983 (NAD 83).

Altitude, as used in this report, refers to distance above the vertical datum.
x

Abbreviations

ADWR Arizona Department of Water Resources

AGFD Arizona Game and Fish Department

AMWG Adaptive Management Working Group

Argonne Argonne National Laboratory

Aurora AURORAxmp

AZGFD Arizona Game and Fish Department

BIA Bureau of Indian Affairs

CMIP3 Coupled Model Intercomparison Project Phase 3

CREDA Colorado River Energy Distributors Association

CRSP Colorado River Storage Project

CRSS Colorado River Simulation System

DEIS Draft Environmental Impact Statement

DFC desired future condition

DOI U.S. Department of the Interior

DT Decision-Theoretic school of adaptive management

EIS Environmental Impact Statement

ESA Endangered Species Act of 1973

EVPI expected value of perfect information

EVXI expected value of partial information

FWS U.S. Fish and Wildlife Service

GCDAMP Glen Canyon Dam Adaptive Management Program

GCMRC Grand Canyon Monitoring and Research Center

GCPA Grand Canyon Protection Act of 1992

GCRG Grand Canyon River Guides

GTMax-Lite Generation and Transmission Maximization model, spreadsheet version

HBC humpback chub (Gila cypha)

HFE high-flow experiment

IFFF International Federation of Fly Fishers

LTEMP Long-Term Experimental and Management Plan

MCDA multicriteria decision analysis

MLFF Modified Low Fluctuating Flow

NEPA National Environmental Policy Act

NPCA National Parks Conservation Association

NPS National Park Service
xi

NPV net present value

O&M operating and maintenance

RBT rainbow trout (Onchorhynchus mykiss) 

RE Resilience-Experimentalist school of adaptive management

Reclamation Bureau of Reclamation

RM river mile

ROD Record of Decision

SDA structured decision analysis

SDM structured decision making

SLI sand load index

SMART Simple Multi-attribute Rating Technique

SME subject-matter expert

SRP Salt River Project

TMF trout management flow

UAMPS Utah Associated Municipal Power Systems

WAPA Western Area Power Administration

WTSI wind transport of sediment index 

Decision Analysis to Support Development of the Glen 

Canyon Dam Long-Term Experimental and Management 

Plan

By Michael C. Runge,1 Kirk E. LaGory,2 Kendra Russell,3 Janet R. Balsom,4 R. Alan Butler,3 Lewis G. Coggins, 

Jr.,5 Katrina A. Grantz,3 John Hayse,2 Ihor Hlohowskyj,2 Josh Korman,6 James E. May,2 Daniel J. O’Rourke,2

Leslie A. Poch,2 James R. Prairie,3 Jack C. VanKuiken,2 Robert A. Van Lonkhuyzen,2 David R. Varyu,3 Bruce T. 

Verhaaren,2 Thomas D. Veselka,2 Nicholas T. Williams,3 Kelsey K. Wuthrich,2 Charles B. Yackulic,1 Robert P. 

Billerbeck,4 and Glen W. Knowles3

1

U.S. Geological Survey

2

Argonne National Laboratory

3

Bureau of Reclamation

4

National Park Service

5

National Oceanic and Atmospheric Administration

6

Ecometric Research, Inc.

1 Abstract

The U.S. Geological Survey, in cooperation with the 

Bureau of Reclamation, National Park Service, and Argonne 

National Laboratory, completed a decision analysis to use in 

the evaluation of alternatives in the Environmental Impact 

Statement concerning the long-term management of water 

releases from Glen Canyon Dam and associated management 

activities. Two primary decision analysis methods, multicriteria decision analysis and the expected value of information, 

were used to evaluate the alternative strategies against the 

resource goals and to evaluate the influence of uncertainty.

A total of 18 performance metrics associated with 8 out 

of 12 resource goals (fundamental objectives) were developed 

by the Bureau of Reclamation and National Park Service in 

partnership with subject-matter teams composed of Federal, 

State, tribal, and private experts. A total of 19 long-term 

strategies associated with 7 alternatives were developed by 

the Bureau of Reclamation, National Park Service, Argonne 

National Laboratory, U.S. Geological Survey, and Cooperating 

Agencies. The 19 long-term strategies were evaluated against 

the 18 performance metrics using a series of coupled simulation models, taking into account the effects of several important sources of uncertainty. A total of 27 Federal, State, tribal, 

and nongovernmental agencies were invited by the Assistant 

Secretary of Interior to participate in a swing-weighting 

exercise to understand the range of perspectives about how 

to place relative value on the resource goals and performance 

metrics; 14 of the 27 chose to participate. The results of the 

swing-weighting exercise were combined with the evaluation 

of the alternatives to complete a multicriteria decision analysis. The effects of uncertainty on the ranking of long-term 

strategies were evaluated through calculation of the value of 

information.

The alternatives and their long-term strategies differed 

across performance metrics, producing unavoidable tradeoffs; 

thus, there was no long-term strategy that was dominated by 

another across all performance metrics. When the performance 

of each alternative was weighted across performance metrics, 

three alternatives (B, D, and G) were top-ranked depending 

on the set of weights proposed: Alternative B was favored by 

those stakeholders that placed a high value on hydropower; 

Alternative G was favored by those stakeholders that placed a 

high value on the restoration of natural processes, like beachbuilding and natural vegetation; and Alternative D was favored 

by the remaining stakeholders. Surprisingly, these rankings 

were not sensitive to the critical uncertainties that were evaluated; that is, the choice of a preferred long-term strategy was 

sensitive to the value-based judgment about how to place 

relative weight on the resource goals but was not sensitive to 

the uncertainties in the system dynamics that were evaluated 

in this analysis. The one area of uncertainty that did slightly 

affect the ranking of alternatives was the long-term pattern of 

hydrological input; because of this sensitivity, some attention 

to the possible effects of climate change is warranted.

The results of the decision analysis are meant to serve 

as only one of many sources of information that can be used 

to evaluate the alternatives proposed in the Environmental 

Impact Statement. These results only focus on those resource 

goals for which quantitative performance metrics could be 

formulated and evaluated; there are other important aspects of 

the resource goals that also need to be considered. Not all the 

stakeholders who were invited to participate in the decision 
2 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

analysis chose to do so; thus, the Bureau of Reclamation, 

National Park Service, and U.S. Department of Interior may 

want to consider other input. 

2 Introduction

The Glen Canyon Dam is on the Colorado River in 

Arizona, United States, within the boundaries of Glen Canyon National Recreation Area and upstream from Grand 

Canyon National Park (fig. 1) and is managed by the Bureau 

of Reclamation (hereinafter referred to as “Reclamation”). 

The Glen Canyon Dam Adaptive Management Program 

(GCDAMP) was established in 1997 to provide research and 

monitoring of downstream resources to Reclamation and the 

U.S. Department of the Interior (DOI). The GCDAMP project 

area stretches along the Colorado River from the forebay of 

Glen Canyon Dam to the westernmost boundary of Grand 

Canyon National Park. Locations along the Colorado River 

are indexed by river miles (RM) with a reference point at Lees 

Ferry (RM 0). The Glen Canyon Dam is at RM -15.5 (15.5 mi 

upstream from Lees Ferry). Other important locations that are 

referenced in this report include the following: Paria River 

(RM 1.0), Little Colorado River (RM 61.4), and Bright Angel 

Creek (RM 87.8) (fig. 1). The reach from Glen Canyon Dam 

to Lees Ferry is known as Glen Canyon, the reach from Lees 

Ferry to the Little Colorado River is known as Marble Canyon, and Grand Canyon proper begins at the Little Colorado 

River.

In October 1996, the Secretary of the Interior signed a 

Record of Decision (ROD) documenting the selection of operating criteria for Glen Canyon Dam (Bureau of Reclamation, 

1996) as analyzed in the 1995 Final Environmental Impact 

Statement (Bureau of Reclamation, 1995). The preferred alternative, known as Modified Low Fluctuating Flow (MLFF), has 

governed the operation of Glen Canyon Dam for the last 19 

years (1996 to present) with important modifications described 

in “Final Environmental Impact Statement—Colorado River 

interim guidelines for lower basin shortages and the coordinated operations for Lake Powell and Lake Mead” (Bureau of 

Reclamation, 2007), “Environmental Assessment—Development and implementation of a protocol for high-flow experimental releases from Glen Canyon Dam” (Bureau of Reclamation, 2011a), “Environmental Assessment—Non-native 

fish control downstream from Glen Canyon Dam” (Bureau 

of Reclamation, 2011b), and many other regulatory documents. The Bureau of Reclamation (1996) also established 

the GCDAMP and Grand Canyon Monitoring and Research 

Center (GCMRC), which has led extensive monitoring and 

research aimed to improve the management of the Colorado 

River and its environs below Glen Canyon Dam.

In July 2011, the Secretary of the Interior announced the 

intent to develop a Long-Term Experimental and Management 

Plan (LTEMP) and Environmental Impact Statement (EIS) 

for Glen Canyon Dam as the first comprehensive review of 

dam operations in 15 years and as an opportunity to integrate 

the considerable scientific information collected since the 

GCDAMP began in 1996. The Bureau of Reclamation and 

National Park Service (NPS) are serving as joint-lead agencies for the EIS. The following agencies are participating as 

Cooperating Agencies in development of the EIS: Arizona 

Game and Fish Department (AGFD), Bureau of Indian Affairs 

(BIA), Colorado River Board of California, Colorado River 

Commission of Nevada, the Havasupai Tribe, the Hopi Tribe, 

the Hualapai Tribe, the Kaibab Band of Paiute Indians, the 

Navajo Nation, the Pueblo of Zuni, Salt River Project (SRP), 

U.S. Fish and Wildlife Service (FWS), Upper Colorado River 

Commission, Utah Associated Municipal Power Systems 

(UAMPS), and Western Area Power Administration (WAPA).

The purpose of the LTEMP is to provide a comprehensive 

framework for adaptively managing Glen Canyon Dam over 

the next 20 years consistent with the Grand Canyon Protection Act of 1992 (GCPA) and other provisions of federal law 

(Bureau of Reclamation and National Park Service 2015); 

thus, the preferred alternative ultimately selected for the 

LTEMP will govern the management of water releases at Glen 

Canyon Dam for the next 20 years, specifying conditiondependent seasonal, weekly, and daily patterns of release, as 

well as nonflow actions, including vegetation management 

and the potential for mechanical removal of nonnative fish. 

The goals for the LTEMP are to meet the requirements of 

the GCPA; and to minimize, consistent with the law, adverse 

effects on the downstream natural, recreational, and cultural 

resources in Glen Canyon National Recreation Area and Grand 

Canyon National Park, including resources of importance to 

American Indian Tribes, while ensuring water delivery and 

maintaining or increasing hydroelectric capacity and generation. The need for the LTEMP arises from scientific information developed since the 1996 record of decision (Bureau 

of Reclamation, 1996), the use of which will better inform 

DOI decisions on dam operations and other management and 

experimental actions so that the Secretary of the Interior may 

continue to meet statutory obligations to protect resources 

downstream from Glen Canyon Dam for future generations, 

conserve species listed under the Endangered Species Act of 

1973 (ESA), avoid or mitigate effects on National Register 

eligible properties, and protect tribal interests, while meeting water delivery obligations and providing hydropower 

generation. The list of resources of concern in the analysis 

of alternatives includes the following: tribal resources and 

interests, sediment deposition and retention, riparian vegetation, humpback chub (HBC) (Gila cypha) and other native 

fish, historic properties, recreation, the rainbow trout fishery in 

Glen Canyon, water delivery, and hydropower. There is uncertainty about how management actions in this system affect the 

resources of concern, which complicates the analysis of alternatives. There is an acknowledged need for adaptive management (Walters, 1986), perhaps even for experimental actions 

chosen to accelerate learning for the benefit of selecting future 

management actions. The decision problem, therefore, can be 

characterized as one of multiple objective tradeoffs in the face 
2 Introduction 3

GRAND

CANYON

NATIONAL

PARK

HAVASUPAI

INDIAN 

RESERVATION

PAIUTE

INDIAN

RESERVATION ZION 

NATIONAL 

PARK

UTAH

NEVADA

ARIZONA

GRAND CANYON-PARASHANT 

NATIONAL MONUMENT GRAND

CANYON

NATIONAL

PARK

ESCALANTE 

NATIONAL 

MONUMENT

KAIBAB

INDIAN 

RESERVATION

HAVASUPAI

INDIAN 

RESERVATION

NAVAJO 

INDIAN 

RESERVATION

HOPI

INDIAN 

RESERVATION

WUPATKI

NATIONAL

MOMUMENT

GLEN CANYON

NATIONAL

RECREATIONAL

AREA

HUALAPAI 

INDIAN 

RESERVATION

Lake Powell

Glen Canyon Dam

Lees Ferry

Pearce

Ferry

Lake Mead

LAKE MEAD

NATIONAL 

RECREATION 

AREA

Colorado River

Havasu Creek

Diamond

Cr

Tapeats

Cr

Shinumo

Cr

Bright Angel Cr

Little Colorado River

Paria River

Kanab Creek

Colorado R

NEVADA UTAH

ARIZONA

Little Colorado R

Map

area

CALIFORNIA

0 10 20 30 40 MILES

0 20 40 60 80 KILOMETERS

114°30' 114° 113°30' 113° 112°30' 112° 111°30' 111°

37°

36°

30'

36°

35°

30'

Base map modified from National Atlas, National Hydrographic Dataset 

and other digital data, various scales. Coordinate reference system: 

NAD 1983 NSRS2007 State Plane Arizona Central FIPS 0202. 

Horizontal datum is North American Datum of 1983 

RM

30

RM

150

RM

150

Pumpkin

Spring

EXPLANATION

National Park land

Other National Park Service land

Bureau of Land Management land

Indian reservation

River mile 

Figure 1. The Colorado River ecosystem below Glen Canyon Dam, depicting the Glen Canyon Dam Adaptive Management Program 

project area.
4 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

of uncertainty, where the management actions themselves are 

condition-dependent, information-dependent, and complex; 

and where there is the opportunity to reduce uncertainty early 

on to improve later actions through adaptive implementation.

In July 2012, the Assistant Secretary of the Interior for 

Water and Science, Reclamation, and NPS embraced the 

idea of using structured and formal decision analysis as one 

of several tools to evaluate alternatives in this EIS. Formal 

decision analysis was seen as a way to address multiple objectives, engage stakeholders, and evaluate the degree to which 

uncertainty is an impediment to the selection of management 

actions. 

The purpose of this report is to document the formal 

decision analysis completed by the U.S. Geological Survey, 

in cooperation with the Bureau of Reclamation, National Park 

Service, and Argonne National Laboratory, as one component 

of the evaluation of alternatives for the LTEMP EIS. This 

report is a stand-alone document and an appendix to the Draft 

EIS (Bureau of Reclamation and National Park Service, 2015); 

it is also anticipated to be an appendix to the Final EIS. The 

emphasis in this report is on the decision analysis; the reader is 

referred to the Draft EIS (DEIS) and its appendixes for many 

other details, including the following: the legal, ecological, 

and cultural context of the LTEMP; an indepth description of 

the alternatives; the details of the quantitative models used to 

evaluate the alternatives; and a comprehensive comparison 

of the alternatives and their effects on resources of concern, 

including qualitative assessments that were not included in the 

decision analysis.

3 Decision Analysis

The DOI is including formal decision analysis tools to 

accompany more traditional qualitative tools to evaluate alternatives in the Glen Canyon Dam LTEMP EIS. The LTEMP 

EIS concerns the management of a very complex system with 

many, possibly competing, resources of interest and considerable uncertainty about the relations between management 

strategies and the responses of resources of interest to management strategies; furthermore, there are multiple stakeholder 

viewpoints to consider, and the DOI wants to use a structured 

process as one of the tools for better evaluating and understanding stakeholder viewpoints. This section provides an 

overview of decision analysis and describes the quantitative 

methods the DOI used to analyze the alternatives.

3.1 Structured Decision Making

“Structured decision making” (SDM) is a term of art, 

used by a community of practitioners in the United States, 

Canada, and Australia, to refer to the application of a broad 

array of decision analysis tools to natural resource management (Gregory and others, 2012). The analytical tools used 

in any application will depend on the specific needs of that 

decision setting, so SDM can look quite different from case 

to case. The set of analytical tools that might be used include 

the following: multicriteria decision analysis (MCDA), decision trees, expert elicitation, objectives hierarchies, value of 

information, stakeholder involvement, predictive modeling, 

utility theory, dynamic optimization, portfolio analysis, and 

many others. The common features in any application of decision analysis are (1) an attention to value-focused thinking 

(Keeney, 1996), recognizing that any decision is an attempt 

to achieve something of value to the decision maker; and 

(2) decomposing the decision problem into basic elements 

(objectives, alternatives, consequences, and tradeoffs) (Runge, 

2011). The goal of SDM is to provide a transparent process for 

articulating objectives, developing alternatives, and evaluating 

those alternatives against the objectives. Note, however, that 

SDM does not substitute for a decision maker, and the application of the SDM process does not make a decision; rather, 

SDM only serves to aid the decision maker in understanding 

and organizing the complexities of the problem. The DOI 

prefers to use the term “structured decision analysis” or simply 

“decision analysis” for the LTEMP EIS process because the 

Secretary of the Interior retains the responsibility and authority 

to make a decision. In the context of the LTEMP EIS, two key 

decision analysis tools are used to evaluate the alternatives: 

MCDA and the expected value of perfect information (EVPI).

3.2 Multicriteria Decision Analysis

Many objectives associated with management of Glen 

Canyon Dam are important to stakeholders, including tribal 

cultural and spiritual values, endangered species, hydropower 

generation, sediment conservation, and recreation. It is possible that some of these objectives compete; if so, no single 

strategy will be best at achieving all the objectives. One of the 

difficulties the decision maker faces, then, is the value judgment regarding how to weight this host of objectives based 

on the many statutes governing Glen Canyon Dam operation 

and resource protection. Multicriteria decision analysis is a 

formal decision analysis tool designed to help evaluate the 

competing objectives and explore how to weight them within 

the context of the statutes (Figueira and others, 2005), and has 

been applied to a wide variety of natural resource management 

problems (Herath and Prato, 2006). Considerable literature 

advocates the use of MCDA in National Environmental Policy 

Act (NEPA) processes (Gregory and others, 1992; Kulkarni 

and others, 1993; Prato, 1999; Sheehy and Vik, 2002; Kiker 

and others, 2005; Linkov and others, 2006a, 2006b, 2006c; 

Stich and Holland, 2011; Marcot and others, 2012). Although 

not common, a few examples of NEPA documents explicitly 

incorporate MCDA methods (National Oceanic and Atmospheric Administration, 1995a, 1995b; Kimbrough and others, 

2008; Nobrega and others, 2009; Bureau of Reclamation, 

2011b; Runge and others, 2011a). Notably, the value of using 

MCDA for evaluation of management alternatives for Glen 

Canyon Dam has been argued by Flug and others (2000).
3 Decision Analysis 5

One of the key advantages of MCDA in the context of 

the LTEMP EIS is that it provides a structured and transparent method for receiving detailed stakeholder input about the 

resource goals, ways to evaluate performance of the alternatives against the resource goals, and the value of resource 

goals relative to each other. Stakeholders have legitimate 

differences in viewpoint about the relative importance of the 

objectives affected by the LTEMP alternatives, differences that 

stem from policy judgments rather than scientific judgments. 

Multicriteria decision analysis provides a way for stakeholders to articulate those judgments. By clearly understanding 

those different viewpoints, Reclamation and NPS (hereinafter 

referred to as the “joint-lead agencies”) were better able to 

analyze and compare the alternatives and ultimately advise the 

Secretary of the Interior regarding her choice of a preferred 

alternative. In selecting a preferred alternative, the Secretary 

of the Interior needed to consider the appropriate suite of laws, 

regulations, agency guidance, and policies; the language in the 

purpose and need statement; stakeholder input; and the public 

input at various points in the process. The MCDA helped organize at least part of that complex input.

3.2.1 A Full Articulation of Resource Goals

The first step in a MCDA is a full articulation of the 

resource goals important to the decision maker, stakeholders, 

and the public. The set of resource goals should be (1) complete because it should cover the full range of concerns relevant to the decision; (2) concise because it should not contain 

redundant or irrelevant resource goals; (3) sensitive so that the 

resource goals are able to distinguish the performance of the 

alternatives under consideration; (4) understandable so that 

the resource goals directly communicate what matters; and, if 

possible, (5) independent so that the resource goals describe 

unique aspects of the problem (Gregory and others, 2012). 

The completeness of the set of resource goals is often very 

important because it provides objectivity and transparency to 

otherwise invisible values (Turner and others, 2008).

In an MCDA, the focus of the resource goals should be 

should be on objectives that are fundamental (called “fundamental objectives”) to the decision maker, stakeholders, 

and public. Fundamental objectives, as distinguished from 

means objectives, are objectives that are important in their 

own right—they are the desired outcomes of the decision, 

not because they lead to something else of importance but 

because of their inherent value. Means objectives are pursued 

as a pathway to fundamental objectives but are not themselves of inherent value to the decision maker; for example, 

a high juvenile survival rate of HBC is important as a means 

to achieving a sustainable population of HBC in the Colorado 

River below Glen Canyon Dam, but juvenile survival is not 

itself the objective that the decision maker fundamentally 

cares about. Achieving population sustainability with low or 

moderate juvenile survival rates of HBC, if possible, would be 

acceptable; thus, the fundamental objective concerns population sustainability.

The joint-lead agencies, in consultation with the tribes 

and Cooperating Agencies and with input from public scoping 

comments, developed a set of resource goals, which represent 

the fundamental objectives to be pursued in the LTEMP EIS. 

Some of these resource goals are closely aligned with the 

desired future conditions (DFCs) developed by the Adaptive 

Management Working Group (AMWG); and are grounded in 

the laws, regulations, and policies relevant to the joint-lead 

agencies. In a few instances, the resource goals differ from the 

DFCs to more clearly identify a set of fundamental objectives for the specific context of the LTEMP EIS that meet the 

characteristics described previously in this section (complete, 

concise, sensitive, understandable, and independent). In a 

parallel effort, the joint-lead agencies worked directly with 

interested tribes to develop resource goals that are specific to 

tribal perspectives.

3.2.2 Performance Metrics

Performance metrics are scales of measurement on which 

the fundamental objectives (resource goals) can be evaluated. 

By developing performance metrics that are closely tied to 

the resource goals, the assessment of alternatives can include 

a quantitative, rather than solely narrative, analysis. Also, 

articulation of performance metrics forces considerable clarity about the resource goals; thus, using performance metrics 

encourages a high degree of transparency in the analysis of 

alternatives. It is difficult, however, to express all the resource 

goals in quantitative form. The use of performance metrics, 

therefore, does not preclude use of narrative analyses to evaluate additional resource goals in a NEPA process.

Ideally, performance metrics should directly reflect 

the resource goals, but this is often very difficult to achieve 

because the resource goals can be subtle, nuanced, complex, 

and difficult to quantify. There are also cases where a desired 

performance metric can be articulated, but the scientific 

tools do not exist to predict the performance of alternatives 

on that scale; thus, development of performance metrics is 

a very important science-policy interface. For the management agency, the desire is to have performance metrics that 

closely track the corresponding resource goals, but for the 

scientists, the desire is to have performance metrics that can be 

predicted with high confidence. When the difference between 

these desires is large, the question is how far the quantitative 

assessment can stretch toward the desired performance metric 

while maintaining a robust scientific foundation. There is, 

of course, a tendency to want to use proxy metrics that can 

be reliably predicted (for example, temperature as a proxy 

for HBC recruitment), but this only shifts a difficult burden 

to the decision maker who has to then make invisible judgments about how closely the proxy aligns with the underlying 

resource goal. The performance metrics developed for this 

decision analysis span the gamut from direct, natural measures 

of resource goals to distant proxies, depending on the science 

available to support their assessment. An effort has been made 
6 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

to transparently articulate the relation between each performance metric used and the resource goal it represents.

The joint-lead agencies in conjunction with subject-matter expert (SME) teams from all relevant disciplines developed 

performance metrics corresponding to the resource goals that 

were used as the scales on which alternatives were quantitatively evaluated. During the course of development, the jointlead agencies sought feedback on draft performance metrics 

from the Cooperating Agencies, tribes, and other stakeholders.

3.2.3 Transparent and Quantitative Evaluation 

of Alternatives

As part of the effects analysis for the LTEMP EIS, and 

as a supplement to the traditional narrative analysis, Argonne 

National Laboratory (hereinafter referred to as “Argonne”) 

and collaborators completed quantitative analyses of the 

alternatives against the performance metrics. These quantitative analyses are an exercise in forecasting—predicting the 

relative performance of the alternatives against the resource 

goals using the best available science. In some cases, existing 

models were available for this task; in other cases, new predictive models were developed; and in still other cases, formal 

methods of expert elicitation (Martin and others, 2012) were 

needed. But, in all cases, the methods for assessing the alternatives provide transparency to the evaluation.

The ecological, economic, and social systems being 

evaluated in the LTEMP EIS are complex, and even though 

the systems are some of the best studied systems in the 

world, they are nevertheless incompletely understood; thus, 

the models used to predict performance of the alternatives 

were necessarily simplifications of the real world and had to 

account for uncertainty. Specific details about how uncertainty 

was handled in the decision analysis are described in section 3.3, “Uncertainty, Value of Information, and Adaptive 

Management.”

A traditional, narrative evaluation of alternatives has to 

make all the same assumptions as a quantitative evaluation 

does. The use of a quantitative approach raises the degree 

of transparency about these assumptions and allows better 

inspection by interested parties. Also, quantitative analysis of 

the alternatives provides the raw material for later steps in a 

decision analysis.

3.2.4 Tradeoff Analysis

The outcome of the assessment phase can be viewed as a 

“consequence table”—a summary of how each alternative is 

expected to perform against each resource goal (as expressed 

by the corresponding performance metric). That assessment 

phase is a scientific endeavor—an evaluation of the current 

knowledge of the system to forecast how the system will 

respond to any proposed alternative. The consequence table 

provides a visual way to compare the alternatives and is an 

important analytical and communication step. Selection of a 

preferred alternative involves a policy choice based on sound 

science, which requires a values judgment about how to 

comply with the laws applicable to the proposed action while 

achieving a multitude of fundamental objectives, particularly 

if there are tradeoffs among the objectives. Some fundamental objectives, because of legal and policy considerations, 

may take precedence over other objectives; the remaining 

objectives need to be balanced appropriately. The joint-lead 

agencies, in evaluating the alternatives and providing a recommendation to the Secretary of the Interior, wanted to understand how Cooperating Agencies and other stakeholders would 

individually value the range of resource goals. This structured 

input was not the only consideration; in the NEPA process, 

decisions are also informed by the legal and policy framework, including relevant laws, regulations, agency policies 

and guidance, court cases, consultation with tribes, and public 

comment at various points in the process.

Multicriteria decision analysis provides tools for eliciting and investigating the values judgments associated with 

balancing tradeoffs. There are a variety of MCDA methods; we used the Simple Multi-Attribute Rating Technique 

(SMART) (Edwards, 1971; Goodwin and Wright, 2004) with 

swing weighting (Winterfeldt and Edwards, 1986). Stakeholders were asked to individually complete a swing-weighting 

exercise to express how they valued the resource goals relative 

to one another, while accounting for the range in performance 

across alternatives. The weights derived were combined 

with the consequence table to rank the alternatives from the 

viewpoint of each participating stakeholder. These individual 

viewpoints were provided as input to the joint-lead agencies 

and are documented in this report. 

3.3 Uncertainty, Value of Information, and 

Adaptive Management

The second major set of tools from decision analysis 

that were used in the development and assessment of alternatives for the LTEMP EIS concern how to articulate, evaluate, 

and address uncertainty in an adaptive design. The primary 

analytical tool that informed this process is the expected value 

of information (Raiffa and Schlaifer, 1961; Howard, 1966). In 

a decision analysis context, the value of information is the best 

method for sensitivity analysis (Felli and Hazen, 1998). Runge 

and others (2011b) describe how the value of information 

can be used to design an adaptive management program that 

focuses on finding the best management strategy (rather than 

reducing the most uncertainty).

3.3.1 Adaptive Management Versus 

Experimental Management

Several different schools of adaptive management, which 

differ in their emphasis on various decision analysis tools and 

approach to experimental design, exist (McFadden and others, 2011). All the schools trace their lineage back to Walters 
3 Decision Analysis 7

(1986), but their current approaches can look very different. 

In the Resilience-Experimentalist (RE) school of adaptive 

management, emphasis is placed on experimental design 

focused on policy choices (management alternatives) as the 

elements of uncertainty. To some extent, the RE school is most 

focused on “unknown unknowns”—uncertainty that cannot 

be anticipated in advance—and, therefore, advocates robust 

experimental designs composed of management alternatives 

to provide accelerated learning about the system dynamics 

in response to management. This accelerated learning might 

not be formally linked to subsequent long-term management 

planning because of a sense that there is too much hubris in 

making long-term plans when so much surprise is expected. 

The RE school perhaps draws more inspiration from Walters 

and Holling (1990) than from the seminal text (Walters, 1986). 

Examples of adaptive management that are often associated 

with the RE school include management in the Columbia 

River Basin, water management in the Everglades, and past 

management in the GCDAMP (McFadden and others, 2011).

In the Decision-Theoretic (DT) school of adaptive 

management, the emphasis is placed on a priori articulation 

of uncertainty through alternative hypotheses about system 

response to management and derivation of optimal strategies 

that solve the “dual-control problem” of achieving long-term 

management objectives by balancing the benefits and costs 

of learning in the short term. The DT school is most focused 

on known unknowns—uncertainty that can be explicitly 

articulated—and, therefore, advocates adaptive design that 

seeks resolution of this uncertainty but only to the degree that 

such resolution will improve future management. Learning is 

explicitly linked to future management decisions by associating particular courses of action with the degree of evidence in 

support of the alternative hypotheses; in this way, long-term 

management is articulated as part of the adaptive program, so 

the natural transition from experimentation to implementation 

is specified up front. The tenets of the DT school are arguably 

a more direct expression of Walters (1986) than other schools 

of adaptive management. The example most commonly associated with the DT school is the adaptive harvest management 

of mallards in North America (Johnson and others, 1997).

It seemed appropriate that the 1995 Glen Canyon Dam 

Final EIS and the GCDAMP embraced an RE school approach 

to adaptive management. At the time, there was very significant uncertainty about the dynamics of the Colorado River 

system below Glen Canyon Dam, so an approach that emphasized learning about unknown unknowns made sense. Several 

policy experiments have been completed, and a great deal 

has been invested in research; together these have greatly 

advanced the state of knowledge. It would have been difficult 

in 1995 to anticipate all the discoveries and specify clearly 

how and when management of Glen Canyon Dam would 

move from an experimental phase to a long-term implementation phase. The current question for the joint-lead agencies 

is if the last two decades of research and experimentation 

have generated enough knowledge to allow the articulation 

of uncertainty as known unknowns and pursue a DT school 

approach to adaptive management. The joint-lead agencies 

believe this is the case, and approached the development of the 

LTEMP EIS with this philosophy in mind.

3.3.2 Articulating Uncertainty as Competing 

Hypotheses

For many, if not all, of the resource goals, the performance of the alternatives is uncertain. Some of that uncertainty can be captured simply by including estimates of variance around the forecasts, but some uncertainty is pervasive 

enough that it could change the ranking of the alternatives 

relative to a particular resource goal and, thus, could impede 

the identification of a preferred alternative. For these critical 

uncertainties, the SME teams characterized the uncertainty in 

the predictive models as a set of competing hypotheses. The 

focus of the uncertainty was on the mechanisms or parameters in the models, not on a holistic statement of uncertainty 

about whether a particular management intervention will work 

or not; thus, for example, rather than state the uncertainty 

as whether or not a particular management alternative will 

build and sustain beaches, the uncertainty was expressed as 

competing hypotheses about the long-term rate of sediment 

input from the Paria River. Likewise, whether trout management flows (TMFs) work or not was expressed as competing 

hypotheses about the rate of recruitment of juvenile rainbow 

trout when TMFs are used.

Development of competing hypotheses to capture critical 

uncertainty is challenging. It requires very deliberative thought 

about the limits of knowledge and the nature of uncertainty 

that would affect the ranking of management alternatives. To 

do so with a small number of discrete hypotheses requires a 

simplification of the full degree of uncertainty. But careful 

thought along these lines allows a transparent and explicit 

statement of the uncertainty that is the focus of experimental 

and adaptive design. In a NEPA context, such explicit articulation of critical uncertainty allows full disclosure of the motivation for experimental design and explanation of how subsequent management will respond to newfound knowledge.

3.3.3 Deconstructing Complex Alternatives

Several of the alternatives under consideration in the 

DEIS (especially Alternatives C, D, and E; see explanations of 

each in the “Alternatives” sections 5.4, 5.5, and 5.6) include, 

either explicitly or implicitly, complex experimental or adaptive designs (see, for example, figs. 2–13 and 2–14 in Bureau 

of Reclamation and National Park Service, 2015). These strategies prescribe different management interventions depending 

on state and information conditions. It is particularly important 

to distinguish state-dependent triggers from informationdependent triggers. State-dependent triggers are conditions of 

the state of the system that would lead to implementation of a 

different management action, primarily in recognition that the 

Colorado River is a stochastic ecosystem, and it is important 
8 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

to be able to respond to natural variation. Most obviously, 

all the alternatives contain state-dependent triggers under 

the Bureau of Reclamation (2007) guidelines that specify, 

among other parameters, the annual volume for water release 

from Glen Canyon Dam. Other state-dependent triggers are 

designed to guard against chance reductions in HBC or other 

resources of concern. State-dependent triggers are in contrast 

to information-dependent triggers, which are indications that 

the state of knowledge has changed substantially and a new 

approach to management is warranted; for example, Alternatives C and E propose experimental implementation of TMFs, 

and it is at least implicit in both that if those TMFs are ineffective in reducing downstream migration of rainbow trout from 

Lees Ferry, they would be discontinued. This is an information-dependent trigger. To the extent possible, informationdependent triggers should be explicit in that they specify the 

weight of evidence that would be needed on one hypothesis or 

another to induce a change in management strategy. 

A complex experimental or adaptive strategy can be 

viewed as a portfolio of simpler strategies combined and 

implemented in a manner to reduce critical uncertainty, with 

the ultimate intention to implement one of the simpler strategies as the operational standard in the long term. The information-dependent triggers govern the switch between these 

simpler strategies. To evaluate such a complex design, the first 

step is to understand what the simpler strategies are; thus, the 

complex strategies were deconstructed into simpler long-term 

strategies, each of which might have state-dependent triggers 

but not information-dependent triggers. In other words, an 

experimental or adaptive strategy can be viewed as a set of 

simpler, operational strategies arrayed against a set of competing hypotheses. Resolution of the competing hypotheses 

would lead to identification of a long-term strategy, but of 

course, in the short term, an experimental design is needed to 

achieve that resolution. The first step in development of such a 

complex strategy is the articulation of the simpler component 

strategies and the attendant competing hypotheses; the development of the experimental design is a later step.

3.3.4 Value of Information

The essence of the DT school of adaptive management 

is a focus on uncertainty that is an impediment to the decision 

maker; that is, resolution of the uncertainty could affect the 

choice of the long-term management strategy. To evaluate the 

importance of a source of uncertainty, we used a technique 

known as the value of information (Raiffa and Schlaifer, 1961; 

Howard, 1966; Runge and others, 2011b), which was also 

advocated by Walters (1986) as the motivation for adaptive 

management. The idea behind the value of information is to 

compare the expected performance if a decision has to be 

made in the face of uncertainty with the expected performance 

if uncertainty can be resolved before committing to a decision. 

This contrast, known as the EVPI, sets an upper bound on the 

value of experimental or adaptive management measured on 

the scale of the management objective(s). To make a decision 

in the face of uncertainty, a decision maker chooses to implement an action that achieves the highest expected performance 

against the uncertainties, but that decision carries the risk of 

being wrong (because some other action could have performed 

better); the value of information measures how important it is 

to eliminate that risk. There are two related methods: (1) the 

expected value of partial information (EVXI) allows calculation of the value of reducing some component of uncertainty, 

while remaining uncertain about the rest; and (2) the expected 

value of sample information takes into account the noise in the 

monitoring.

To calculate the value of information, uncertainty must be 

explicitly articulated (as competing hypotheses with evidentiary weights or as a probability distribution on a set of parameters). With a small set of competing hypotheses, the calculations involve several steps: (1) forecasting the performance of 

each long-term strategy under each hypothesis, (2) calculating 

a weighted performance across hypotheses to identify an 

optimal decision in the face of uncertainty, (3) calculating the 

expected performance if uncertainty could be fully resolved 

before the management alternative was chosen, and then (4) 

comparing the expected values in steps 2 and 3. 

In a single-objective setting, the EVPI is measured on the 

scale of that objective; thus, it could be expressed, for example, as the expected increase in the probability of persistence 

of HBC if uncertainty could be resolved before committing 

to a long-term management strategy. In a multiple-objective 

setting, EVPI is calculated on the composite utility scale (in 

which the full array of objectives is weighted); thus, an evaluation of the importance of uncertainty cannot be calculated 

without first completing a MCDA to understand the value 

weights on the resource goals. The sequence of assessment 

matters; the LTEMP EIS process was crafted to accommodate 

this series of steps. First, the long-term strategies (the elements of the full alternatives broken down into their constituent parts) were analyzed against the array of performance 

metrics under all the competing hypotheses. Second, using the 

expected value across hypotheses, MCDA (with input from 

stakeholders) was used to generate a preliminary weighting of 

fundamental objectives. Third, with those weights, the individual competing hypotheses were investigated, and the expected 

value of information (perfect and partial) was calculated.

3.3.5 Experimental and Adaptive Design

The motivation for the EVPI analysis was to provide 

guidance useful for developing an experimental or adaptive 

design for implementing management actions, taking into 

account their value as long-term strategies and the value of 

reducing uncertainty. This report only describes the results 

from the decision analysis, with a brief discussion of the 

implications of the results for experimental design (section 

9.2, “Motivation for Adaptive Management”); the development of the experimental design is described in the DEIS.
4 Resource Goals and Performance Metrics 9

4 Resource Goals and Performance 

Metrics

In February 2010, the Assistant Secretary of the Interior asked an ad hoc group of the AMWG to develop a set 

of DFCs, which were outcomes of fundamental importance 

that were “achievable through the operation of Glen Canyon 

Dam, subject to the Law of the River and consistent with the 

Grand Canyon Protection Act” (Castle, 2010, p. 2) During 

the next 2 years, this ad hoc group worked with the AMWG 

and DOI to develop the set of DFCs, which were adopted by 

the AMWG and recommended to the Secretary of the Interior 

in February 2012. The set of DFCs formed the basis for the 

development of the resource goals for the LTEMP EIS. The 

resource goals express the fundamental objectives for the 

LTEMP EIS and are described in the nine subsections that follow (4.1 through 4.9).

Associated with the resource goals, the joint-lead agencies developed metrics to evaluate the relative performance 

of the LTEMP alternatives. These scientifically based performance metrics were developed in a series of workshops 

among SMEs working on the LTEMP EIS and were revised 

to incorporate feedback from Cooperating Agencies and other 

stakeholders. The performance metrics are intended to be 

objective measures of the performance of alternatives relative to goals for each affected resource being evaluated in the 

LTEMP EIS. Note that these performance metrics are not the 

full impacts analysis for the LTEMP EIS. Other sources of 

quantitative and qualitative information, in addition to the performance metrics, were used to assess the overall and relative 

performance of alternatives and their constituent elements for 

the LTEMP EIS. A summary of the performance metrics used 

in the decision analysis is given in table 1.

4.1 Aquatic Ecology

The resource goals associated with aquatic ecology 

focused on the persistence of HBC and other native fish, the 

quality of the rainbow trout (RBT) (Oncorhynchus mykiss) 

fishery, and absence or containment of nonnative aquatic 

species.

4.1.1 Humpback Chub

Resource Goal.—Meet HBC recovery goals including 

maintaining a self-sustaining population, spawning habitat, 

and aggregations in the natural range of the HBC in the Colorado River and its tributaries below the Glen Canyon Dam.

Performance Metric 1.—Expected minimum number of 

adult (greater than or equal to [≥] 200 millimeters [mm]) HBC 

in the Little Colorado River population during the LTEMP 

planning period (20 years).

Performance Metric 2.—Average temperature suitability 

index (scale 0–1) for HBC at RM 157 (Havasu Creek) and 

RM 213 (Pumpkin Spring) (fig. 1). The potential for a selfsustaining aggregation of HBC at each of these locations was 

based on a temperature suitability model that considered how 

well water temperatures met requirements for important HBC 

life-history stages (spawning, egg incubation, and growth) 

using triangular probability functions based on the reported 

ranges of suitable temperatures for each life-history stage 

(Valdez and Speas, 2007). The composite temperature suitability index was the geometric mean of the suitability indices for 

each of the life-history stages.

4.1.2 Other Native Fish

Resource Goal.—Maintain self-sustaining native fish 

species populations and their habitats in their natural ranges on 

the Colorado River and its tributaries.

In the analysis for the EIS, temperature suitability 

indexes for other native fish, similar to the one for HBC, were 

developed. They did not, however, show much differentiation 

among alternatives, and they did not differ much from each 

other and from the HBC temperature suitability metric. These 

metrics were not, subsequently, used in the decision analysis; 

the HBC temperature suitability metric was assumed to stand 

in for other native fish as well.

4.1.3 Rainbow Trout Fishery

Resource Goal.—Achieve a healthy high-quality recreational RBT fishery in the Glen Canyon National Recreation 

Area and reduce or eliminate downstream RBT migration 

consistent with NPS fish management and ESA compliance.

Performance Metric 3.—Rainbow trout catch rate in 

Glen Canyon National Recreation Area (age 2+ fish per 

angler-hour).

Performance Metric 4.—Rainbow trout emigration rate 

(number of age-0 RBT moving into Marble Canyon from Glen 

Canyon per year).

Performance Metric 5.—Abundance of high-quality RBT 

(greater than [>] 16 inches total length) in the Glen Canyon 

reach.

4.1.4 Nonnative Aquatic Species

Resource Goal.—Minimize or reduce the presence and 

expansion of aquatic nonnative invasive species.

In the analysis for the EIS, temperature suitability metrics 

for nonnative warm-water and cold-water fish and aquatic 

parasites, similar to the one for HBC, were developed. They 

did not, however, provide much differentiation among the 

alternatives. These metrics are reported in the EIS but were 

not used in the decision analysis.
10 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Table 1. Summary of the 18 performance metrics used in the decision analysis.

[The range for each performance metric captures the amount of variability in the metric because of the effects of the different alternatives, the hydrological and 

sediment traces, and structural and parametric uncertainty. The range shown was used in the swing-weighting elicitation. HBC, humpback chub; RBT, rainbow 

trout; #, number; >, greater than]

Number Resource goal Performance metric Units Desired 

direction Range

1

Humpback chub

Minimum number of adult HBC # adults Increase 3,000–8,500

2 HBC temperature suitability Index (0–1) Increase 0.0–0.2

3

RBT fishery

RBT catch rate Fish/angler-hour Increase 1.0–5.0

4 RBT emigration rate Trout/year Decrease 15,000–125,000

5 Abundance of high-quality RBT # fish > 16 inches Increase 400–1,200

6

Archaeological and 

cultural resources

Wind transport of sediment index Index (0–1) Increase 0.0–0.5

7 Glen Canyon flow index Days/year Decrease 0–75

8 Time-off-river index Index (0–1) Increase 0.60–0.95

9

Hydropower and energy Hydropower generation Million $/year Increase 120–200

10 Hydropower capacity Million $/year Increase 10–50

11

Recreation

Camping area index Index (0–1) Increase 0.0–0.5

12 Fluctuation index Index (0–1) Increase 0.0–1.0

13 Rafting use index Visitor-days/year Decrease 0–1,300

14 Riparian vegetation Riparian vegetation index Sum of ratios Increase 2.0–6.0

15 Sediment Sand load index Proportion (0–1) Increase 0.0–0.6

16

Tribal resources

Marsh vegetation ratio Ratio Increase 0.0–1.5

17 Mechanical removal Years (out of 20) Decrease 0–5

18 Trout management flows Years (out of 20) Decrease 0–20

4.2 Archaeological and Cultural Resources

Resource Goal.—Maintain the integrity of potentially 

affected National Register of Historic Places eligible or listed 

historic properties in place, where possible, with preservation 

methods used on a site-specific basis.

Performance Metric 6.—Wind transport of sediment 

index (WTSI). This WTSI metric captures the availability of 

fine sediment for movement by wind. The potential settlement 

of such sediment over historic properties may provide protection from erosion. The WTSI is the product of the sand load 

index (SLI; see section 4.7, “Sediment”), which is a proxy for 

the availability of sediment, and a flow factor, which is higher 

when low flows happen during the windy season. The WTSI 

can range between 0 and 1, with higher values indicating more 

opportunity for transport of sediment.

Performance Metric 7.—Glen Canyon flow index (number of days per year that the maximum daily flow is greater 

than 23,200 cubic feet per second [ft3

/s]). There are a number 

of archaeological sites within the Glen Canyon National Recreation Area that could be negatively affected by high flows. 

Ninemile Terrace is considered representative of other archaeological sites on terraces within the Glen Canyon National 

Recreation Area; the toe of its slope is at an elevation that corresponds to flows from Glen Canyon Dam of 23,200 ft3

/s.

Performance Metric 8.—Time-off-river index. This 

metric reflects the availability of discretionary time off river 

for rafting parties, which allows for greater visitation of 

archaeological sites. From the standpoint of protection of 

archaeological and cultural sites, such discretionary time is 

not desirable. The time-off-river index is calculated from 

the mean daily flow levels in the river: daily flows less than 

10,000 ft3

/s receive a score of 1 (because rafting progress is 

slow and there is little discretionary time); daily flows greater 

than 31,500 ft3/s receive a score of 0 (because rafting progress 

is fast, allowing the most discretionary time); and daily flows 

between 10,000 and 31,500 ft3/s receive a score based on 

linear interpolation. The daily scores are averaged within seasons: summer (May–August), winter (November–February), 

and spring and fall (March–April and September–October, 

respectively). An annual value is calculated using a weighted 

average across seasons, with higher weight (0.54) given to 

summer than to spring and fall (0.31) or winter (0.15). The 

final metric ranges between 0 and 1, with higher values indicating better potential to protect archaeological resources.
4 Resource Goals and Performance Metrics 11

4.3 Hydropower and Energy

Resource Goal.—Maintain or increase Glen Canyon 

Dam electric energy generation, load following capability, and 

ramp rate capability; and minimize emissions and costs to the 

greatest extent practicable, consistent with improvement and 

long-term sustainability of downstream resources.

Performance Metric 9. —Value of hydropower generation 

(million dollars per year). The value of hydropower energy 

production over a 20-year trace is discounted at 3.375 percent 

per year. The total net present value of energy production is 

divided by 20 to provide an annualized measure.

Performance Metric 10.—Value of hydropower capacity (million dollars per year). The WAPA enters into longterm firm contracts that obligate them to deliver firm electric 

service to their customers. For the purpose of the LTEMP EIS, 

the value of these contracts is estimated by first calculating the 

amount of power (megawatt [MW]) that can be contracted, 

then multiplying by the replacement cost for that capacity 

(dollars per MW-year). The capacity itself is calculated by 

finding the 90-percent exceedance value (10th percentile) for 

the daily maximum generation in August (August is the month 

with the highest demand) during a 20-year trace. The replacement cost used was $50,100/MW-year based on a natural gas 

combustion turbine.

Additional metrics associated with hydropower generation, including the retail rate effects on residential and nonresidential consumers, are examined in the LTEMP EIS but were 

not available in time to be included as part of the decision 

analysis.

4.4 Natural Processes

Resource Goal.—Restore, to the extent practicable, 

ecological patterns and processes within their range of natural 

variability, including the natural abundance, diversity, and 

genetic and ecological integrity of the plant and animal species 

native to those ecosystems.

A quantitative performance metric to capture this 

resource goal was not developed because of its multifaceted 

complexity. In the EIS, the alternatives are compared qualitatively against this resource goal, but it was not used in the 

decision analysis.

4.5 Recreational Experience

Resource Goal.—Maintain and improve the quality of 

recreational experiences for the users of the Colorado River 

ecosystem. Recreation includes, but is not limited to, flatwater 

and whitewater boating, river corridor camping, and angling in 

Glen Canyon.

Performance Metric 11.—Camping area index. This 

metric captures the availability of medium (16–25 people) and 

large (>25 people) campsites on beaches along the Colorado 

River. The camping area index is the product of the SLI (see 

section 4.7, “Sediment”), which measures the availability of 

sediment for beach forming, and a flow factor, which reflects 

exposure of the beaches. The flow factor is calculated from the 

maximum daily flows. A score of 1 is given to flows less than 

8,000 ft3/s (because the lower flow means a lower elevation 

of the water and hence more beach exposure); a score of 0 is 

given to flows greater than 31,500 ft3/s; and an intermediate 

score is determined by linear interpolation for flows between 

8,000 and 31,500 ft3/s. The flow factor is averaged over days 

within each season, and then over seasons, with higher weight 

given to the summer season (0.54) than the spring and fall 

(0.31) or winter (0.15) seasons. The camping area index can 

range between 0 and 1, with higher values indicating greater 

availability of medium and large campsites.

Performance Metric 12.—Fluctuation index. This index 

measures the fraction of time that the daily fluctuations in 

flow are within a tolerable range; tolerable is defined based 

on the study by Bishop and others (1987). As the mean 

daily flow increases, greater fluctuations are tolerable. For 

mean daily flow in the range from 5,000 to 8,999 ft3/s, the 

daily fluctuation index is 1 if the daily fluctuation is less 

than 3,400 ft3/s; for daily flow in the range from 9,000 to 

15,999 ft3/s, the fluctuation index is 1 if the daily fluctuation 

is less than 4,800 ft3/s; for daily flow in the range from 16,000 

to 31,999 ft3/s, the fluctuation index is 1 if the daily fluctuation is less than 7,200 ft3

/s; and for daily flow greater than 

32,000 ft3/s, the fluctuation index is 1 if the daily fluctuation 

is less than 9,800 ft3/s. In all cases, the daily fluctuation index 

is 0 if the daily fluctuation is greater than 10,000 ft3/s. For 

fluctuation index values between 1 and 0, the index is determined by linear interpolation between the tolerable threshold 

and 10,000 ft3/s. The daily values for the fluctuation index are 

averaged within seasons, then averaged across seasons with 

higher weight (0.54) given to summer months than to spring 

and fall (0.31) or winter months (0.15). The fluctuation index 

can range between 0 and 1, with higher values indicating a 

better visitor experience as influenced by fluctuation in water 

levels.

Performance Metric 13.—Glen Canyon rafting use metric 

(average boat seats lost per year because of high-flow experiments [HFEs]). During HFEs and for 2 days before and after, 

Glen Canyon day-rafting operators cannot take recreational 

passengers. This metric calculates the average number of boat 

seats lost per year.

Several other metrics reflecting the recreational experience were analyzed in the LTEMP EIS. In some cases, these 

did not help to discriminate among the alternatives, and in 

other cases, these metrics were highly correlated with other 

metrics. For these reasons, the metrics were not included in 

the decision analysis and are not described here.

4.6 Riparian Vegetation

Resource Goal.—Maintain native vegetation and wildlife habitat, in various stages of maturity, such that they are 
12 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

diverse, healthy, productive, self-sustaining, and ecologically 

appropriate.

Performance Metric 14.—Riparian vegetation index. 

This index summarizes predicted changes during the 20-year 

LTEMP period in the relative cover of native vegetation community types and the relative diversity of community types, as 

described by a seven-state state-and-transition model (Ralston 

and others, 2014). The index is calculated as the sum of four 

component ratios: the ratio of final to initial native vegetation 

cover, the ratio of final to initial vegetation state diversity, the 

ratio of final to initial native to nonnative dominant vegetation 

state, and the ratio of initial to final cover of arrowweed. An 

index value of 4.0 indicates an unchanged vegetation condition; values greater than 4.0 indicate improved vegetation 

conditions; values less than 4.0 indicate degraded vegetation 

conditions.

4.7 Sediment

Resource Goal.—Increase and retain fine sediment 

volume, area, and distribution in the Glen, Marble, and Grand 

Canyon reaches above the elevation of the average base flow 

for ecological, cultural, and recreational purposes.

Performance Metric 15.—The SLI (the cumulative sand 

load transported by high flows [greater than 31,500 ft3/s] 

divided by the cumulative sand load transported in total). The 

range of this metric is 0–1, with higher values reflecting the 

potential for larger sandbars because more of the sediment is 

transported at higher river volumes and hence higher elevations. A number of other metrics that summarize the transportation and deposition of sediment were analyzed in the LTEMP 

EIS. They were all fairly highly correlated; only the SLI was 

used in the decision analysis.

4.8 Tribal Resources

Resource Goal.—Maintain the diverse values and 

resources of traditionally associated tribes along the Colorado 

River corridor through Glen, Marble, and Grand Canyons.

In discussions with tribal representatives during the 

course of the development of the LTEMP EIS, a large number 

of important resource goals were identified. A concerted effort 

was made to understand these goals as clearly as possible, but 

it was challenging to express some of them in terms amenable to measurement. Although all these goals are important 

aspects of the importance to the tribes of the Grand Canyon, 

the Colorado River, and their management, not all of them will 

be affected by the alternatives being considered in the LTEMP 

EIS. All of these resource goals are discussed and evaluated 

in the LTEMP EIS. The full set of goals is described in this 

section, but only those for which performance metrics were 

identified are included in the decision analysis.

4.8.1 Health of the Ecosystem

The ecosystem in Glen, Marble, and Grand Canyons is 

more than the sum of its parts and, in the view of the tribes, 

should be healthy as a whole. Historically, in the GCDAMP, 

the overall health of the ecosystem has been examined 

by evaluating the status of each part, but this reductionist 

approach might possibly miss some important aspects. There 

are a variety of indicators of ecosystem health that could be 

considered including, but not limited to, the following: the 

health of the river and its ability to sustain life; the color of 

the water; the absence of contaminants, pollutants, and disease 

in the water; the potability of the water; the quality of the 

water that reaches Lake Mead; and the viability and health of 

wildlife and plants in Glen, Marble, and Grand Canyons. It is 

important to understand that, for many tribes, the Colorado 

River is a sentient being and the spiritual center of the ecosystem because it has the capability of giving and taking life and 

is prone to anger if mistreated. The health of the ecosystem 

depends on the health of the Colorado River.

Because of the holistic nature of this resource goal, it was 

difficult to find a single performance metric that summarized 

it. A narrative analysis of this resource goal is found in the 

LTEMP EIS, but it was not included in the decision analysis.

4.8.2 Sites of Cultural Importance

There are specific sites within Glen, Marble, and Grand 

Canyons that are important for cultural reasons; and for 

preservation of tribal, religious society, kiva group, and clan 

history (for example, shrines, sacred sites, ancient burial sites, 

springs, plant collection areas, mineral collection areas, offering places, and other elements). These sites can be threatened 

by erosion, loss of sediment inputs, and intrusive human use 

(especially nontribal, outside visitors). Flow and nonflow 

actions (for example, education, permitting, research and 

monitoring, and interpretation) may affect these sites.

Performance Metric 16.—Marsh vegetation ratio (ratio 

of frequency of wetland states during the course of 20 years to 

frequency of wetland states if the initial abundance remained 

unchanged). Wetlands are a rare and important habitat type 

along the Colorado River; within Glen, Marble, and Grand 

Canyons, there are currently approximately 4.6 acres of such 

habitat. Maintenance and increase of marsh vegetation is 

important from a tribal perspective. The metric is calculated 

by looking at the marsh and shrub wetland states in the riparian state-and-transition model (Ralston and others, 2014) 

during the course of a 20-year projection and comparing that 

to the current abundance. The metric is a ratio; thus, a value of 

1.0 indicates an unchanged abundance of marsh (on average), 

a value greater than 1.0 indicates an average increase, and a 

value less than 1.0 indicates an average decrease.

A number of aspects of this resource goal are also 

reflected in other performance metrics described in previous 

sections. The WTSI (performance metric 6) focuses on the 

availability of fine sediment for transfer to protect National 
4 Resource Goals and Performance Metrics 13

Register eligible or listed sites. Such sites, however, do not 

represent the full set of places of tribal concern, so other 

metrics must also be considered. The Glen Canyon flow 

index (performance metric 7) reflects the protection of some 

important sites in Glen Canyon. The time-off-river index (performance metric 8) measures the degree to which the discretionary time off river of rafters might affect cultural resources, 

including those of tribal importance. 

In additions to these performance metrics, a number of 

other considerations are evaluated narratively in the LTEMP 

EIS, including riparian vegetation diversity, native fish diversity, and access to springs.

4.8.3 Respect for Life

To many of the tribes associated with the Colorado River 

and Glen, Marble, and Grand Canyons, life itself is sacred. 

Human activities should protect and promote life, not destroy 

it. There are two aspects to this resource goal: first, minimize 

the taking of life; and second, encourage the expansion and 

proliferation of life. These are both complex concepts. The 

tribes recognize that it is appropriate for humans to take other 

life in some circumstances, especially when it promotes other 

life (particularly, our own consumption for survival), but this 

taking needs to be minimal and respectful because there are 

spiritual consequences associated with the taking of life. The 

promotion of life is also important but does not necessarily 

imply a return to historical or “natural” conditions. The Glen 

Canyon Dam has encouraged new life in Glen, Marble, and 

Grand Canyons, so a return to predam conditions is not necessarily implied by this objective. It is worth noting that many of 

the tribes do not make a strong distinction between native and 

nonnative species.

Performance Metric 17.—Frequency of mechanical 

removal, as measured by the average number of years in 

which trout mechanical removal trips happen. Several of the 

alternatives considered in the LTEMP EIS include the use of 

electrofishing and removal of nonnative trout at the confluence of the Colorado River with the Little Colorado River in 

an effort to reduce the piscivory of juvenile HBC by trout. 

A number of tribes, especially Hopi and Zuni, have raised 

concerns about this taking of life. As a coarse measure of the 

cultural, spiritual, and ethical effects of killing trout, this performance metric allows distinctions among alternatives, favoring those that make an effort to minimize mechanical removal. 

But the nature of the take, the purpose behind it, the methods 

of the take, the disposition of the trout taken, and the mind set 

of those killing the fish all also affect the sacred treatment of 

living beings.

Performance Metric 18.—Frequency of trout management flows, as measured by the average number of years in 

which TMFs happen. Several of the alternatives considered 

in the LTEMP EIS include the potential for use of TMFs. The 

TMFs, designed to reduce reproduction or survival of juvenile 

trout, are considered to be killing by some tribes and should 

be minimized. Alternatives that include TMFs are likely to 

differ in how often the flows are triggered, so this performance 

metric helps to distinguish among the alternatives.

4.8.4 Sacred Integrity of Grand, Marble, and 

Glen Canyons

Glen, Marble, and Grand Canyons are sacred to many 

tribes, and the preservation of their sacred integrity is important. The sanctity of Glen, Marble, and Grand Canyons may 

be threatened by human activities and behaviors, development, and the presence of artificial structures and activities. An 

important aspect of the sanctity is the intentionality of visitors: 

when outsiders enter Glen, Marble, and Grand Canyons (on 

boat or hiking trips), the respect they show to the canyons 

and Colorado River can affect the spiritual integrity. There 

are many consequences of the disturbance of this sanctity, 

including, but not limited to, the following: a reduction of the 

spiritual strength of plants gathered and used by the Navajo 

for medicinal and cultural purposes; an inability to retire 

Navajo sacred objects into the Colorado River when they have 

become too old for continued use; weakening of the sacred 

role the canyons play as a final resting place for Hopi; and 

an overall disruption of the state of mind and spirit of Zuni 

religious leaders and of their experience of being within a very 

sacred place that embodies the Zuni emergence, migrations, 

and communion with the spirits of Zuni ancestors.

This resource goal, although of profound importance 

to the tribes, is not thought to differ measurably across the 

alternatives under consideration in the LTEMP EIS because 

it is not driven by flow operations from Glen Canyon Dam or 

currently envisioned attendant activities. This goal is evaluated 

in the narrative EIS analysis but not in the formal decision 

analysis.

4.8.5 Stewardship

According to their traditions, several of the tribes 

understand that they have been given a sacred stewardship 

responsibility for the preservation and harmony of the world; 

for example, the Hopi have a covenant with the caretaker of 

this world, Masaw, to be stewards of the earth; other Tribes 

have similar stewardship ethics grounded in spiritual traditions. To maintain these stewardship responsibilities, the tribes 

need to be an active part of stewardship of the Glen, Marble, 

and Grand Canyons. This stewardship includes the following: 

ceremonial activities, whether performed in the canyons or 

in the villages; participation in management of the canyons, 

including water management, through traditional practices and 

western management activities; and education to maintain cultural knowledge and connection with the canyons. The tribes 

note that the Federal government also has stewardship responsibilities that arise out of Federal legislation; because Federal 

involvement has sometimes taken stewardship responsibility 

from the tribes, it is critical that the Federal government be 

accountable for its stewardship. At times, some tribes believe 
14 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

the presence of the Federal government has made it more difficult for them to carry out their stewardship responsibilities. 

Successful development of joint stewardship among the tribes 

and Federal government will require continued building of 

mutual respect and trust between those entities.

Tribal stewardship opportunities are not tied to individual 

alternatives being considered in the LTEMP EIS but could be 

crafted to apply to any of the alternatives; thus, this resource 

goal, although of critical importance to the tribes individually 

and to the ongoing relationship between the tribes and Federal 

government, may not help distinguish among the alternatives. 

This goal is evaluated in the narrative EIS analysis but not in 

the formal decision analysis.

4.8.6 Tribal Connections to the Canyons

The spiritual, historical, and cultural connections that 

tribes have to Glen, Marble, and Grand Canyons require the 

protection of sacred sites and the integrity of the canyons as a 

whole, but protection alone is not enough. The tribes also need 

opportunities for access, education, and stewardship to keep 

their connections vibrant. Access can be undermined by physical barriers and by the effects of human activity that decrease 

the power of those sites and the experience when at them (for 

example, lack of privacy, disturbance of the soundscape, and 

viewshed).

Like the sacred integrity and stewardship resource goals, 

this resource goal is not thought to differ across the alternatives. The flow operations of Glen Canyon Dam are not likely 

to affect tribal access, education, spiritual ceremonies, or 

other connections to Glen, Marble, and Grand Canyons. This 

resource goal may be more appropriately addressed through 

government-to-government consultation in other forums. This 

goal is evaluated in the narrative EIS analysis but not in the 

formal decision analysis.

4.8.7 Economic Opportunity

The Glen, Marble, and Grand Canyons, Colorado River, 

and Glen Canyon Dam are sources of economic benefit for 

the tribes in the area. The canyons provide tourism and other 

opportunities that enhance the economic well-being of tribes. 

(As an important note, tourism can also undermine the wellbeing of tribes in aspects other than economic; see the other 

tribal resource goals.) The Glen Canyon Dam provides affordable electricity for tribal needs and for development projects.

The hydroelectric performance metrics (especially hydroelectric generation, performance metric 9) may reflect one 

aspect of economic opportunity in that provision of affordable 

hydroelectricity is a component of economic development. 

Other economic effects are evaluated in the narrative EIS but 

were not included in the formal decision analysis.

4.8.8 Tribal Water Rights and Supply

Tribes in the area depend on the Colorado River for 

many of their water needs; so the preservation of established, 

traditional, and desired water rights, now and into the future, 

is important to them. There are a number of claims to water 

rights that have been asserted by the tribes but for which there 

are not yet quantified rights through decree or negotiated 

settlement; some Tribes have indicated that these claims to 

water rights are as important as established water rights.

Based on its purpose and need, the LTEMP EIS was not 

intended to include any alternative that violated agreed-upon 

water rights. The effect of the Lake Powell water elevation on 

Navajo access to water was examined and was not determined 

to differ across alternatives.

4.8.9 Process Objectives

The tribes also expressed several important process 

objectives—objectives that govern how the LTEMP decision 

is made, rather than what decision is made. The first of these is 

the genuine incorporation of tribal input to the LTEMP process 

as a reflection of Federal trust responsibilities. The second is 

the importance of incorporating learning to improve management with time; in this spirit, an experimental approach that 

can result in adaptive management is favored.

Because these were process objectives, they were not 

evaluated in the formal decision analysis, which focused on 

objectives that could help discern among the alternatives.

4.9 Water Delivery

Resource Goal.—Ensure that water delivery continues in a manner that is fully consistent with and subject to 

the Colorado River Compact, Upper Colorado River Basin 

Compact, Water Treaty of 1944 with Mexico, decree of the 

Supreme Court in Arizona v. California, and provisions of the 

Colorado River Storage Project Act of 1956 and the Colorado 

River Basin Project Act of 1968 that govern allocation, appropriation, development, and exportation of the waters of the 

Colorado River Basin; and consistent with applicable determinations of annual water release volumes from Glen Canyon 

Dam made pursuant to the Long-Range Operating Criteria 

for Colorado River Basin Reservoirs, which are currently 

implemented through the 2007 interim guidelines (Bureau of 

Reclamation, 2007).

The scope and need of the LTEMP EIS was designed to 

preserve existing agreements concerning water delivery; thus, 

none of the alternatives considered were meant to differ with 

regard to aspects of water delivery. A number of water delivery metrics were evaluated to make certain that this part of the 

purpose and need was upheld. None of the alternatives were 

determined to differ with regard to these metrics; thus, they 

were not included in the decision analysis.
5 Alternatives 15

5 Alternatives

Seven alternatives were considered in the LTEMP EIS. 

Each of these alternatives is a complex combination of flow 

and nonflow actions, often with condition-dependent and 

information-dependent triggers for various components. 

Four of the alternatives (B, C, D, and E) also contain variants 

(called long-term strategies) to represent different combinations of actions that would be taken once critical uncertainties 

were resolved. In total, 19 long-term strategies were evaluated. 

A brief summary of these alternatives is given here to provide 

context for the decision analysis with an emphasis on those 

elements that differ across alternatives. A full description of 

these alternatives, with the rationale for each, is given in chapter 2 of the LTEMP EIS.

5.1 Action Elements

The alternatives are composed of flow actions, which 

govern how water flow through Glen Canyon Dam is to be 

managed on hourly, daily, and monthly scales, as well as nonflow actions, which govern other activities meant to complement the flow actions to achieve the resource goals.

5.1.1 Monthly Release Pattern and Other Base

Operations

The seven alternatives differ in the operational characteristics that govern flow through Glen Canyon Dam (table 2). 

All alternatives follow the 2007 interim guidelines (Bureau 

of Reclamation, 2007), which specifies the annual volume 

of water to be passed through Glen Canyon Dam. The alternatives differ in how they apportion the annual volume to 

monthly volumes, ranging from patterns that match releases 

under the previous EIS to patterns that match monthly demand 

for electricity to patterns that approximate the unregulated 

flow in the Colorado River. The minimum flow at any 

time differs across alternatives, with a range from 5,000 to 

8,000 ft3/s. The maximum flow for base operations is the same 

for all alternatives (25,000 ft3

/s). The daily range specifies the 

limit to how much variation there can be between the minimum and maximum flow on a given day and ranges between 

0 and 12,000 ft3/s across alternatives. The ramp rates limit 

how quickly the flow can change; for all alternatives the ramp 

rate for increasing flow is 4,000 cubic feet per second per hour 

(ft3

/s-h); the ramp rate for decreasing flow is lower, ranging 

from 1,500 to 4,000 ft3

/s-h.

5.1.2 High-Flow Experiments

Since 1996, there have been a number of experimental high-flow releases of water through Glen Canyon Dam, 

designed to mimic natural high flows and deposit sediment at 

higher elevations, in an effort to rebuild sandbars and restore 

other ecosystem processes in Glen, Marble, and Grand Canyons. High-flow releases exceed the operational maximum 

daily flow and can be as high as 45,000 ft3

/s depending on the 

number of generation units available. The Bureau of Reclamation (2011a) established a protocol for determining when HFE 

releases were warranted and how they should be implemented. 

To date, the HFEs that have been implemented under the 2011 

HFE protocol have happened in the fall, although the HFE 

protocol allows them to happen in the spring after 2015. The 

LTEMP EIS alternatives differ in if, when, and how they allow 

fall HFEs and spring HFEs (table 3). In addition, some of 

the alternatives allow proactive spring HFEs; these are HFE 

releases designed to protect sand supply in years when the 

annual guidelines call for high annual volumes (>10 million 

acre-feet). Most of the HFEs are limited to a maximum duration of 96 hours, but some of the alternatives allow extendedduration HFEs if adequate sediment supply is available 

(table 3).

5.1.3 Trout Management Flows

High flows, whether through high annual volumes or 

HFEs, have been determined to increase RBT production in 

the Glen Canyon reach. Because of the concern about the 

effect of RBT on HBC, methods have been sought to manage RBT populations. One proposed method is TMFs. These 

are patterns of high-water release (for example, 20,000 ft3

/s) 

for 2–7 days to encourage young-of-the-year trout to move to 

higher elevation shallow-water habitats followed by a rapid 

drop to a low flow (for example, 5,000 to 8,000 ft3

/s) during 

the day to strand young trout and expose them to sunlight and 

heat. The intention is to reduce RBT production and, hence, 

reduce emigration of trout from Glen to Marble Canyon, lowering the risk to HBC farther downstream. The TMFs have not 

previously been implemented at Glen Canyon Dam. They are 

considered an experimental element in several of the alternatives (table 3).

5.1.4 Other Flow Manipulations

A variety of other flow manipulations are built into the 

alternatives. Low summer flows are designed to produce 

warmer conditions for juvenile HBC growth in the Colorado 

River at the confluence with the Little Colorado River by 

reducing flows from July through September. These flows 

would be implemented experimentally in those alternatives 

that allow them (table 3). 

Sustained low flows for benthic invertebrate production 

(“bug flows”) are an experimental action considered under 

Alternative D for restoring mayflies, stoneflies, and caddisflies to Glen and Marble Canyons. If implemented, steady 

minimum flows would be provided every weekend from May 

through August, to ensure that eggs laid during weekends 

would not be subject to drying because of lower water levels 

during egg development. Demand for electricity is lower on 
16 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan Table 2. Operational characteristics of Long-Term Experimental and Management Plan alternatives. [maf,, megaacre-foot; kaf, kiloacre-foot; ft3/s, cubic foot per second; >, greater than] Elements of base operations Alternative

A B C D E F G

Monthly 

pattern in release volume

Historic monthly 

release volumes. 

Same as 

Alternative A

Highest volume 

in high electric demand months; February–June volumes proportional to contract rate of delivery; lower volumes August–November

Comparable to 

Alternative E, but August and September volume increased, with additional volume taken from January through July 

Monthly volumes proportional to the contract 

rate of delivery, but with a targeted reduction in August–October volumes; volume released in October, November, and December=2.0 maf in > 8.23-maf years

Relative to Alternative A, higher 

release volumes in April–June; lower volumes in remaining months

Equal monthly 

volumes, adjusted with changes in runoff forecast.

Minimum 

flows (ft3/s)

8,000 during day; 5,000 during night

Same as 

Alternative

A

Same as 

Alternative

A

Same as 

Alternative

A

Same as 

Alternative A 5,000 5,000

Maximum 

flows (ft3/s)

25,000 Same as Alternative

A

Same as 

Alternative

A

Same as 

Alternative

A

Same as 

Alternative

A

Same as 

Alternative

A

Same as 

Alternative A.

Daily range 

(ft3/s)

5,000, 6,000, or 

8,000 depending on monthly volume

December and January: 12,000February, July, and 

August: 10,000

October, November, 

March, June, and September: 8,000

April and May: 

6,000

Equal to 7 × monthly 

volume in kaf in all months;

2,000 ft3/s following 

significant sediment input in summer or fall

Equal to 10 × monthly 

volume in kaf in June–August, and 9 × monthly volume in kaf in other months; not to exceed 8,000 ft3/s

Equal to 12 × monthly 

volume in kaf in June–August, and 10 × monthly volume in kaf in other months;

2,000 ft3/s following significant sediment input in 

summer or fall

0 0 

Ramp rates 

(ft3/s-hr)

4,000 up1,500 down

4,000 up4,000 down 

November–March 

3,000 down in other 

months

4,000 up2,500 down

4,000 up2,500 down

4,000 up2,500 down

4,000 up1,500 down

4,000 up1,500 down.
5 Alternatives 17

Table 3. Triggered and experimental elements of the Long-Term Experimental and Management Plan long-term strategies.

[A filled circle (●) indicates the element in question is implemented fully in the long-term strategy; an open circle (○) indicates the element in question is 

implemented in some partial degree; and -- indicates that element is not part of that long-term strategy. HFE, high-flow experiment]

Element

Alternative

A B C D E F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Fall HFEs ○ ○ ○ ● ● -- ● ● ● ● ● ● ● -- ● -- -- ● ●

Spring HFEs ○ ○ ○ ● ● -- -- ○ ○ ○ ○ ○ ○ -- -- -- -- ● ●

Proactive spring HFEs -- -- -- ● ● -- -- ● ● ● ● -- -- -- -- -- -- -- ●

HFEs longer than 96-hour duration -- -- -- ● ● -- ● ○ ○ ○ ○ -- -- -- -- -- -- -- ●

Trout management flows -- ● ● ● -- -- -- ● ● -- ● ● -- -- -- -- ● -- ●

Low summer flows -- -- -- -- ● -- -- ○ ○ ○ -- -- ● -- -- ● -- -- --

Bug flows -- -- -- -- -- -- -- -- ● -- -- -- -- -- -- -- -- -- --

Hydropower improvement flows -- -- ● -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --

Steady flows before HFEs -- -- -- ● ● -- ● ● ● ● ● ● ● -- ● -- -- ● ●

Steady flows after HFEs -- -- -- ● ● -- ● ● ● ● ● -- -- -- -- -- -- ● ●

Mechanical trout removal ● ● ● -- -- ● ● ● ● ● ● -- -- ● ● -- -- -- ●

weekends, so timing the benthic invertebrate production flows 

during the weekends minimizes the effect on the value of 

hydropower generation.

Hydropower improvement flows are an experimental 

action considered under Alternative B to increase hydropower 

generation during high-demand months (December–February 

and June–August) in years when the annual volume is less 

than or equal to (≤) 8.23 million acre-feet. These flows work 

largely by allowing a greater daily range and higher ramp 

rates.

Some of the alternatives use steady flows before an 

HFE to conserve sediment for the high-flow release, after 

an HFE to reduce erosion of newly built sandbars, or both 

(table 3). These steady flows stop the load-following patterns 

of hourly releases and are sometimes called “load-following 

curtailment.”

5.1.5 Nonflow Actions

Under a 2011 assessment (Bureau of Reclamation, 

2011b), experimental methods of controlling trout populations were evaluated and implemented. These methods include 

mechanical removal of brown trout (Salmo trutta) and RBT 

at the Little Colorado River through electrofishing, selective 

removal, and beneficial use of the fish removed. Many of the 

LTEMP EIS alternatives allow for continued use of mechanical removal with specific requirements for when such removal 

would be triggered (table 3).

Under all alternatives except Alternative A (no-action 

alternative), the NPS would implement experimental vegetation restoration to modify the cover and distribution of plant 

communities along the Colorado River. This restoration would 

include removal of nonnative species, prevention of new introductions, planting of native species, management of vegetation at campsites, and removal of windrows that block wind 

transport of sediment.

5.1.6 Long-Term Strategies

Four of the seven alternatives (Alternatives B, C, D, and 

E) were conceived as experimental and adaptive strategies, 

with elements that would be deployed experimentally to allow 

resolution of uncertainty and subsequent adaptation of the 

strategy in response to the information acquired; thus, within 

each of these alternatives are a number of implicit long-term 

strategies that might result after resolution of uncertainty. 

Not all of those possible long-term strategies were analyzed, 

but a representative set from each alternative was articulated 

and analyzed to capture the possible range of environmental 

effects and evaluate the importance of resolving the underlying uncertainty; thus, for example, under Alternative C, 

whether or not to implement fall HFEs, spring HFEs, TMFs, 

low summer flows, or mechanical removal will depend on the 

efficacy of those actions in achieving their intended purpose, 

the strengths of undesirable side effects of those actions, and 

the underlying effect of RBT on HBC. The long-term strategies C1, C2, C3, and C4 represent a range of possible solutions once such uncertainty is resolved; if, for instance, spring 

HFEs have a beneficial effect on sediment deposition and only 

a small effect on RBT production, TMFs are ineffective, but 

low summer flows benefit HBC, then long-term strategy C2 

might be the best version of Alternative C to use. All of the 19 

long-term strategies (table 3) were evaluated against the critical uncertainties described in sections 6.2.1, 6.3.1, and 6.5.1.
18 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

5.2 Alternative A (No Action)

Alternative A, the no-action alternative, represents 

continued operation of Glen Canyon Dam under the existing 

guidelines in Bureau of Reclamation (1996), interim guidelines in Bureau of Reclamation (2007), and the two 2011 EAs 

(Bureau of Reclamation, 2011a, 2011b), as well as other applicable guidance. This alternative has a base operation pattern 

that was called MLFF in Bureau of Reclamation (1996). Both 

of the 2011 EAs expire after 2020, so the elements specified 

in them (HFEs and mechanical removal) would cease at that 

point (Bureau of Reclamation 2011a, 2011b). Beyond the 

actions discussed in Bureau of Reclamation (2011a, 2011b), 

there are limited experimental options that can be tested under 

this alternative.

5.3 Alternative B

Alternative B was designed to increase the value of 

hydropower generation while limiting negative effects to other 

resources. This alternative was submitted by the Colorado 

River Energy Distributors Association (CREDA), a member of AMWG. The base operations are designed to generate more valuable power than Alternative A by increasing 

the daily range and ramp rates to allow power generation to 

more closely follow power demand. This alternative limits 

the implementation of HFEs to one every other year because 

HFEs release water without generating power and require 

water volumes to be shifted from months when the value of 

power is potentially higher to other months. To moderate some 

of the negative effects of this water-release pattern on other 

resources, Alternative B includes mechanical removal, evaluation of TMFs, and experimental implementation of vegetation 

restoration. 

5.3.1 Alternative Long-Term Strategies

Alternative B includes two long-term strategies, one with 

its base operations (B1) and one that includes hydropower 

improvement flows (B2), maximum power plant capacity 

flows implemented as many as four times during the LTEMP 

period, in years when the annual volume is ≤8.23 million 

acre-feet (table 3). Such years, because of their lower volume 

of release, typically require the most purchases by WAPA to 

meet contractual demand, so the higher flows could mitigate 

costly purchases in high-demand months. Whether B1 or B2 is 

a better long-term strategy, when the effects across all resource 

goals are taken into account, depends on the strength of negative effects of greater power generation on other resources. 

Specific hypotheses for these negative effects, which are 

uncertain, were not explicitly articulated when Alternative 

B was submitted. The long-term strategies B1 and B2 were 

evaluated, however, against the set of uncertainties developed 

in the context of other alternatives.

5.4 Alternative C

Alternative C was developed by the joint-lead agencies 

to maintain or improve multiple resources, with some priority 

placed on HBC, sediment, and hydropower. Called the “condition-dependent adaptive strategy” in interim public documents 

during the preparation of the EIS (Bureau of Reclamation and 

National Park Service, 2014), Alternative C has informationdependent and condition-dependent triggers, seeks to test 

critical hypotheses, and contains explicit instructions for how 

new insights will affect subsequent implementation of action 

elements.

Compared to many of the other alternatives, the base 

operations for Alternative C shift monthly volumes from 

the monsoon months (August through November) to highdemands months (December, January, and July) to reduce 

sediment transport during the monsoonal high-sediment-input 

period and limit effects on the value of power generation. The 

reduced volume in late summer is also intended to provide 

warmer river temperatures for the benefit of HBC and other 

native fish.

Alternative C includes a number of condition-dependent 

(triggered) elements; among them are fall HFEs, spring HFEs, 

proactive spring HFEs, extended duration HFEs, TMFs, low 

summer flows, load-following curtailment, and mechanical removal of trout. The specific details of the triggers for 

these events are found in chapter 2 of the EIS. Most of these 

elements are experimental, and so there are informationdependent triggers as well; that is, monitoring and evaluation 

of all these elements would be completed, and elements would 

be dropped or retained depending on if the evidence supported 

their retention. 

5.4.1 Alternative Long-Term Strategies

Four long-term strategies (C1, C2, C3, and C4) were 

developed to capture the range of possible outcomes from 

adaptive implementation of Alternative C (table 3). The 

motivation for the creation of these long-term strategies was 

the recognition that there are critical uncertainties that impede 

identification of the best long-term strategy. The key questions 

concerned the following: the magnitude of increase in RBT 

production in the Glen Canyon reach caused by fall HFEs, the 

relative effects of temperature and trout density on survival 

and growth of juvenile HBC at the Little Colorado River, and 

the effectiveness of TMFs in reducing RBT production. The 

relevance of these uncertainties arises because of the possible 

tradeoffs among the objectives of supporting the HBC population, maintaining a strong rainbow trout fishery, increasing 

sediment deposition and sandbar building, and affirming 

tribal values concerning the respect for life (especially aquatic 

life). Each of the action elements under consideration (HFEs, 

TMFs, low summer flows, and mechanical removal) is meant 

to increase achievement of one or more of those objectives, 

but the question is if the benefits to one objective come at 

the expense of others. Alternative C was designed under the 
5 Alternatives 19

assumption that the best mix of action elements (the best 

long-term strategy) depends on resolution of the critical 

uncertainties.

Long-term strategy C1 allows use of all the triggered 

elements from Alternative C except for low summer flows and 

mechanical removals; such a strategy has been hypothesized 

to be warranted, for instance, if the effect of spring HFEs on 

trout production was weak, TMFs were effective, and low 

summer flows were not effective. Long-term strategy C2 is 

similar to C1 except that it calls for low summer flows but not 

TMFs. Long-term strategy C3 does not allow HFEs and might 

be called for if HFEs are determined to cause a very strong 

increase in trout recruitment and other actions are ineffective 

at mitigating the effect of trout on HBC. Long-term strategy 

C4 allows fall HFEs but not spring HFEs and manages the 

effect of trout on HBC through mechanical removal. 

Whether or not the resolution of the critical uncertainties 

actually matters to the ranking of the long-term strategies is 

one of the central questions in section 8.3, “Expected Value of 

Information.”

5.5 Alternative D

Alternative D was developed by the joint-lead agencies 

after a full analysis of the other six LTEMP alternatives had 

been completed. The initial analysis suggested that there were 

strong characteristics of Alternatives C and E that contributed 

to achievement of resource goals and motivated creative thinking about how to achieve more of those benefits in a single 

alternative. Alternative D adopts characteristics of Alternatives 

C and E.

The base operations for Alternative D are in many 

respects a compromise between those of Alternatives C and E 

(table 2). The monthly pattern of release is most similar to that 

of Alternative E, with an increase in the August and September 

volumes (and a corresponding decrease in the January through 

June volumes). The daily range for flows is about midway 

between the ranges from Alternatives C and E.

The experimental and adaptive options for Alternative D 

include all the elements considered in the any of the alternatives, except for hydropower improvement flows (table 3). The 

experimental implementation calls for early testing of TMFs 

in the first 2–5 years without having to meet the conditiondependent triggers and delayed consideration of spring HFEs 

until the third year of implementation.

5.5.1 Alternative Long-Term Strategies

Four long-term strategies (D1, D2, D3, and D4) were 

developed to capture the range of possible outcomes from 

adaptive implementation of Alternative D (table 3). The longterm strategies in this set look more similar than in the sets for 

Alternatives C and E because they were created after an initial 

analysis of the other six alternatives and, thus, reflect insights 

from that analysis. For instance, long-term strategies D1 

through D4 are all similar with regard to implementation of 

HFEs because the initial analysis suggested that the benefits of 

HFEs outweighed the costs, even in the face of the uncertainties tested. There remain, however, other long-term strategies 

implicit in Alternative D; for example, if spring, proactive, or 

extended HFEs are determined to have very strong negative 

effects on resources of importance that cannot be mitigated by 

other actions, they could be removed from long-term implementation. The four long-term strategies for Alternative D, 

however, are thought to capture the most important differences 

that might arise in long-term implementation.

Long-term strategy D4 might be considered the base 

strategy under Alternative D because all action elements 

except low summer flows and bug flows are in operation 

(table 3). Long-term strategy D1 adds low summer flows. 

Long-term strategy D2 adds implementation of bug flows and 

would be supported if early experimental tests of bug flows 

determined they had the intended effects in restoring important native benthic invertebrate communities without having adverse effects on other resources. Under this long-term 

strategy, low summer flows would only be implemented in the 

second 10 years. Long-term strategy D3 is similar to D1 but 

removes TMFs. 

The motivations for the different long-term strategies in 

Alternative D are the same critical uncertainties that motivate 

the long-term strategies in Alternative C, with a specific focus 

on the effectiveness of TMFs, the influence of temperature on 

juvenile HBC growth and survival, and the effect of bug flows 

on other resources.

5.6 Alternative E

Alternative E was developed by representatives of 

the seven states in the Colorado River Basin to provide for 

recovery of HBC while protecting other important resources, 

including sediment, the RBT fishery at Lees Ferry, aquatic 

food base, and hydropower resources. Called the “resourcetargeted condition-dependent” strategy in interim public 

documents during the preparation of the EIS (Bureau of 

Reclamation and National Park Service, 2014), Alternative E 

has information-dependent and condition-dependent triggers, 

seeks to test critical hypotheses, and contains explicit instructions for how new insights will affect subsequent implementation of action elements.

The base operations for Alternative E seek greater value 

of hydropower production than Alternatives C and D (table 2). 

The monthly volumes largely follow hydropower demand 

and are proportional to the contract rate of delivery but with 

a reduction in August through October volumes to conserve 

sediment during the monsoon period. The daily range is large 

compared to the other alternatives. The ramp rates are the 

same as in Alternative C and D.

Alternative E includes a number of condition-dependent 

(triggered) elements; among them are fall HFEs, spring 

HFEs, TMFs, low summer flows, load-following curtailment 
20 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

(before but not after HFEs), and mechanical removal of trout 

(table 3). Implementation of spring HFEs is delayed until the 

second 10 years to allow testing of TMFs in the early years 

(with long-term implementation of spring HFEs conditional 

on TMFs being an effective tool). An important experimental 

element of Alternative E is early testing of the response of 

RBT populations to TMFs and fall HFEs in a 2x2 factorial 

design, replicated two to three times (in hopes of experiencing 

both cold and warm water temperatures). The specific details 

of the triggers for these events are found in chapter 2 of the 

EIS. Most of these elements are experimental, and so there 

are information-dependent triggers as well; that is, monitoring 

and evaluation of all these elements would be completed, and 

elements would be dropped or retained depending on if the 

evidence supported their retention.

5.6.1 Alternative Long-Term Strategies

Six long-term strategies (E1, E2, E3, E4, E5, and E6) 

were developed to capture the range of possible outcomes 

from adaptive implementation of Alternative E (table 3). The 

motivation for the creation of these long-term strategies was 

the recognition that there are critical uncertainties that impede 

identification of the best long-term strategy. The key questions 

concerned the following: the magnitude of increase in RBT 

production in the Glen Canyon reach caused by fall HFEs, the 

relative effects of temperature and trout density on survival 

and growth of juvenile HBC at the Little Colorado River, the 

effectiveness of TMFs in reducing RBT production, and the 

effect of high flows on sediment retention.

Long-term strategy E1 uses spring and fall HFEs when 

triggered, along with TMFs to manage the trout populations; 

this long-term strategy would be favored, for example, if 

HFEs are determined to be beneficial to sediment conservation 

and TMFs are effective in controlling trout populations. Longterm strategy E2 also uses spring and fall HFEs but uses low 

summer flows instead of TMFs; this strategy might be favored 

if temperature is a more important factor in HBC survival 

than trout predation and TMFs are ineffective. Long-term 

strategy E3 does not allow HFEs, TMFs, or low summer flows 

and uses mechanical removal to manage trout density at the 

Little Colorado River when needed; such a strategy might be 

favored if HFEs greatly increase production of trout, TMFs are 

ineffective in managing trout populations, HBC are strongly 

affected by trout predation, and low summer flows do not 

provide enough advantage to HBC to warrant use. Long-term 

strategy E4 allows fall HFEs but not spring HFEs and relies 

on mechanical removal to manage trout at the Little Colorado 

River. Long-term strategy E5 is like E3 in not allowing HFEs 

but uses low summer flows rather than mechanical removal to 

support HBC population growth. Long-term strategy E6 also 

does not use HFEs and uses TMFs when needed to manage 

trout. As with Alternatives C and D, these long-term strategies are not meant to be a comprehensive set of the possible 

permutations of Alternative E that might arise from adaptive 

implementation but, rather, are meant to represent a broad 

range of possible outcomes, effectively bracketing the strategies that might arise. 

As with Alternative C and D, whether or not the resolution of the critical uncertainties actually matters to the ranking 

of the long-term strategies is one of the central questions in 

section 8.3, “Expected Value of Information.”

5.7 Alternative F

Alternative F was designed to follow a more natural 

pattern of flows, limiting sediment transport and providing warmer temperatures in the summer months. The base 

operations provide higher release volumes in April–June (to 

mimic spring runoff), including a May 1 high-flow spike 

of 45,000 ft3/s (essentially an untriggered HFE) and lower 

volumes the remainder of the year. The daily flows are steady 

with no daily fluctuations (no hydropeaking). The only 

modification to base operations would be sediment-triggered 

HFEs (fall and spring). In keeping with the desire to have this 

alternative better match predam conditions than the other alternatives, it does not use TMFs or mechanical removal.

5.8 Alternative G

Alternative G was designed to maximize the conservation 

of sediment by providing year-round steady flows. The base 

operations provide equal monthly volumes and steady daily 

flows. In practice, the monthly volumes would have to be 

adjusted as the forecasts for annual volume change through the 

year, but the intent is to keep flows quite steady. This alternative does allow all forms of HFEs to be used to maximize 

sandbar building and sediment conservation. 

6 Quantitative Methods

The 19 long-term strategies were evaluated against the 

18 performance metrics and 5 sources of uncertainty: the 

hydrology during the 20-year LTEMP period of performance, 

the sediment inputs during the 20-year LTEMP period of 

performance, the relative effects of temperature and trout 

predation on juvenile HBC survival and growth, the effectiveness of trout management flows, and the effect of fall HFEs on 

trout recruitment. A total of 10 modeling teams built a linked 

set of analyses to simulate the performance of the long-term 

strategies; the performance metrics were calculated from the 

simulation outputs. This section describes the methods used to 

complete these analyses.

6.1 Modeling Overview

The basis of the analysis of the alternatives was a large, 

linked set of simulation models that were meant to represent the 

Colorado River ecosystem below Glen Canyon Dam (fig. 2). 
6 Quantitative Methods 21

Unit outages

and power

characteristics

for all traces

GTMax Lite for

Glen Canyon

only

Daily and

hourly flows

Tributary sand

Sand budget

model

Monthly

release

volumes and 

elevations

Natural flow

traces

CRSS

Water delivery

values

Hydropower

metrics

Months with

HFEs and

adjusted

volumes

GTMax Lite for

Glen Canyon only

Lake Powell

and Mead

elevations

Sand

budget model

Water temperature

model

Sediment

metrics

Monthly water

temperature

Refined daily

and hourly flows

and generation

Customer and

power

systems data

Daily and hourly

generation

AURORAxmp Model

capacity expansion

and dispatch

configurations

GTMax Lite for

large SLCA/IP

plants other

than GCD

Historical

operations

Spreadsheet

model for

small SLCA/IP

facilities

Power

systems and

ratepayer impacts

Humpback chub

aggregation metrics

Thermal

suitability model

Trout

metrics

Humpback

chub metrics

Other native

fish metrics

Invasive aquatic

species metrics

Tribal resources

and values metrics

Recreation

(camping, angling,

boating) metrics

Archaeological and

cultural resources

metrics

Humpback chub/

trout model

Riparian vegetation

model

Riparian vegetation

metrics

Inputs or intermediate calculationsModelsFinal metrics, impacts, or values

EXPLANATION

Colorado River Simulation SystemGlen Canyon DamGeneration and Transmission Maximization Model High-flow experimentSalt Lake City Area Integrated Projects

CRSS

GCD

GTMax Lite

HFE

SLCA/IP

Figure 2. Model flow for the analysis of the Long-Term Experimental and Management Plan long-term strategies against the performance metrics.
22 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Each long-term strategy encoded a set of rules that governed 

annual, monthly, daily, and hourly flow through the dam, as 

well as mechanical removal of nonnative fish from the Colorado River at its confluence with the Little Colorado River and 

vegetation treatments along the river corridor. The long-term 

strategy, coupled with 21 hydrological traces and 3 sediment 

traces (to represent environmental variation), determined the 

flows through the Glen Canyon Dam. The daily and hourly 

flows then were sent to models for vegetation, sediment, and 

temperature. The temperature and flow outputs then informed 

models for RBT, HBC, and other aquatic organisms. The outputs of all these models were collected and used to calculate 

performance metrics under all the resource goals. The details 

of all these modeling methods are provided in the technical 

appendixes to the LTEMP EIS; a synopsis is provided here.

6.2 Hydrology

The hydrological modeling requires three sets of input: 

a time series of hydrological flow into Lake Powell, a time 

series of sediment input into the Colorado River from the 

Paria River and the Little Colorado River, and the operational 

rules determined by a long-term strategy. The output of the 

hydrological modeling is the monthly, daily, and hourly flows 

through Glen Canyon Dam for the 20-year LTEMP period 

of performance. This requires three modeling systems. First, 

the Colorado River Simulation System (CRSS), built with 

the commercial software package RiverWareTM (Zagona and 

others, 2001), generates monthly volumes and lake elevations 

taking into account the operations of the 12 primary reservoirs 

on the Colorado River. Second, the Generation and Transmission Maximization model (GTMax-Lite) generates daily 

and hourly flows taking into account patterns of demand and 

supply of electric power in the Colorado River Storage Project 

(CRSP) region. Third, a sand budget model tracks sediment 

in Glen, Marble, and Grand Canyons; and is used to indicate 

when sediment conditions warrant high-flow experiments 

(if the long-term strategy in question allows them). After the 

HFEs are identified, a second pass through GTMax-Lite is 

required to refine the daily and hourly flows. Note that the 

effect of TMFs on hydrology were not modelled because this 

would have required a full coupling of the hydrology and fish 

models (with feedback from the fish models to the hydrology 

models), which was a computing task that was beyond the 

resources available. 

6.2.1 Critical Uncertainty

To model uncertainty in future inflow, the indexed 

sequential method (Ouarda and others, 1997) was used to 

resample the historical record of natural flow in the Colorado 

River system during the 105-year period from 1906 through 

2010. Every fifth trace was selected from this series to produce 

21 hydrological traces; every long-term strategy was evaluated 

against all 21 hydrological traces.

To investigate the possible effects of climate change on 

the performance of the long-term strategies, CRSS was run 

with 112 natural flow traces developed from downscaled general circulation model projected hydrological traces (Bureau 

of Reclamation, 2012). These climate scenarios were based 

on 16 general circulation models from the Coupled Model 

Intercomparison Project Phase 3 (CMIP3) using three forcing scenarios (A2, high; A1b, medium; and B1, low) (Maurer 

and others, 2007). The LTEMP models were not run with the 

112 hydrological traces; rather, the 112 hydrological traces 

from the climate-change scenarios were used to place weights 

on the 21 index sequential historical traces. The model results 

are compared using historical (equal) and climate-change 

weights on the 21 hydrological traces.

6.3 Sediment Dynamics

A sand budget model (Russell and Huang, 2010; Wright 

and others, 2010) was used to track sand storage and transport from Lees Ferry (RM 0) to Bright Angel Creek (RM 87) 

(fig. 1). The sand budget model takes as input hourly hydrographs at the reach boundaries (RM 30, RM 61, and RM 87) 

developed based on release schedules from Glen Canyon Dam 

and hourly sand delivery from the Paria and Little Colorado 

Rivers. The model tracks sand storage and movement through 

three reaches (Upper Marble Canyon, RM 0 to 30; Lower 

Marble Canyon, RM 30 to 61; and Eastern Grand Canyon, 

RM 61 to 87) using empirically-based rating curves formulated on a particle-size-specific basis. The outputs of the model 

include hourly time series of the sand transported at the downstream border of each reach, and the sand budget for each 

reach. From these outputs a variety of performance metrics, 

including the SLI (performance metric 15), can be calculated.

6.3.1 Critical Uncertainty

Sediment input to the Colorado River between Lees Ferry 

and Bright Angel Creek comes primarily from the Paria River 

and the Little Colorado River (fig. 1). Variation in sediment 

input is driven by spatial and temporal variation in precipitation. To capture uncertainty about the time series of future 

sediment input, three 20-year traces of sediment input were 

created from historical records using an index sequential 

method. For Paria River input, 49 reconstructed historical 

traces were available (index years 1964–2012). For Little 

Colorado River input, 18 reconstructed historical traces were 

available (index years 1995–2012). The cumulative 20-year 

sediment delivery for each trace was calculated; and the traces 

corresponding to the 10th, 50th, and 90th quantiles of cumulative sediment delivery were selected to represent the range of 

possible uncertainty. The traces from corresponding quantiles 

for the Paria River and Little Colorado River delivery were 

coupled in the simulations. Weights were placed on the three 

sediment traces so that the weighted mean and standard deviation of the 20-year Paria River sediment delivery for the three 
6 Quantitative Methods 23

traces matched the mean and standard deviation for the full set 

of index sequential Paria River traces.

6.4 Temperature

The average monthly water temperatures along the river 

corridor from Glen Canyon Dam to Lake Mead were forecast 

using the monthly flow and air temperatures forecasts. The 

release temperature from Glen Canyon Dam was simulated 

with the CE–QUAL–W2 model (Cole and Wells, 2006). This 

served as an input to a river temperature model that calculated 

the gains and losses of heat as the water moved downstream 

(Wright and others, 2009). For the purposes of the decision 

analysis, water temperature was an intermediate variable used 

as input to the fish models but not used directly as a performance metric.

6.5 Coupled Rainbow Trout-Humpback Chub 

Dynamics

A coupled RBT–HBC model was used to simulate the 

population dynamics of RBT in the Glen Canyon reach, 

movement of RBT from Lees Ferry to the Little Colorado 

River confluence, and the population dynamics of HBC in 

the Little Colorado River reach. The RBT population dynamics were simulated with an age-structured population model 

that accounted for the effects of water flow on recruitment, 

survival, growth, and downstream emigration (Korman and 

others, 2012). The RBT performance metrics are calculated 

from this model. The RBT catch rate (performance metric 3) 

was calculated from the age-specific abundances, age-specific 

vulnerabilities, and a catchability coefficient. The abundance 

of RBT greater than 16 inches (considered high-quality RBT; 

performance metric 5) was calculated from the age-specific 

abundances and size-at-age characteristics. The annual emigrants (performance metric 4) were computed as a fraction of 

the recruitment.

The RBT movement model predicts the monthly abundance of RBT in each 1-mile segment of the Colorado River 

from RM 0 to RM 150 taking into account movement rates, 

natural mortality, and implementation of mechanical removal. 

Movement and survival were density independent and were 

not affected by flow or temperature.

A size- and location-structured population model was 

used to predict the adult population size of HBC at monthly 

intervals (Yackulic and others, 2014). The HBC population 

size in two locations, the Colorado River and the Little Colorado River (a tributary of the Colorado River), was accounted 

for in the model. Survival of juvenile HBC in the Colorado 

River depended on the abundance of RBT in the Little Colorado River reach of the Colorado River; their growth depended 

on RBT abundance and temperature. Growth rates in other 

size classes in the Colorado River depended on water temperature. From this model, the minimum number of adult HBC 

during each 20-year trace was calculated, and the expected 

minimum value (performance metric 1) was calculated by taking the mean across traces.

6.5.1 Critical Uncertainties

A number of critical uncertainties concerned parameters 

in the RBT and HBC models. Each of these uncertainties was 

expressed as alternative sets of model parameters. All longterm strategies, with all sediment and hydrology traces, were 

run separately against the combinations of model parameters. 

The alternative sets of model parameters were assigned 

weights to represent their empirical support, and weighted 

averages of the performance metrics were calculated across 

them.

Uncertainty about the effect of fall HFEs on recruitment of RBT in the Glen Canyon reach was expressed as two 

hypotheses: one that there is no effect on recruitment, and 

another that proposed recruitment would increase at the same 

rate as seen with spring HFEs (Korman and others, 2011) but 

for only 1 year instead of 2 years. An expert panel consisting 

of four fish biologists who were familiar with RBT dynamics 

in the Colorado River was convened in March 2014. A modified Delphi process with four-point elicitation (Speirs-Bridge 

and others, 2010) was used to estimate the likelihood of the 

two hypotheses; the resulting individual estimates were aggregated by simple averaging.

Uncertainty about the effect of TMFs in reducing RBT 

recruitment in the Glen Canyon reach was expressed as two 

hypotheses by the expert panel: TMFs reduce recruitment by 

10 percent, and TMFs reduce recruitment by 50 percent. The 

two levels of effectiveness were chosen to represent equally 

plausible extremes.

The final area of uncertainty concerned the effects of 

RBT abundance and temperature on the growth and survival 

of juvenile HBC in the Little Colorado River reach of the 

main stem Colorado River. Four hypotheses were generated 

(fig. 3A and 3B): a strong effect of temperature on growth and 

a strong effect of RBT on growth and survival (f1), a weak 

effect of temperature on growth and a weak effect of RBT on 

growth and survival (f2), a weak effect of temperature and a 

strong effect of RBT (g1), and a strong effect of temperature 

and a weak effect of RBT (g2). The parameter values for the 

four hypotheses were chosen from the ends of diameters on 

the 90-percent confidence ellipsoid for the joint parameter 

likelihood (Yackulic and others, 2014) and, thus, were equally 

weighted.

6.6 Fish Habitat Suitability

Temperature suitability for HBC and other native and 

nonnative fish at a number of sites between Lees Ferry (RM 0) 

and Diamond Creek (RM 225) (fig. 1) was evaluated with 

species-specific models for the effect of temperature on survival, reproduction, and growth (Valdez and Speas, 2007). The 

details of all these methods and results are in the LTEMP EIS. 
24 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

f1, g1

f2, g2

f1

f2

g1

g2

0.25 0.30 0.35 0.40 0.45 0.50 0.55 0.60 0.65

-0.35

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

Temperature effect on growth

Trout effect on growth

-0.45 -0.4 -0.35 -0.3 -0.25 -0.2 -0.15 -0.1

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

 0

Survival intercept

Trout effect on survival

A. Survival parameters B. Growth parameters

Figure 3. Uncertainty in the humpback chub population dynamics as represented by four alternative sets of parameters. A, 

survival parameters; and, B, growth parameters

The only one of those metrics that was retained for the decision analysis was the average temperature suitability for HBC 

at Havasu Creek (RM 157) and Pumpkin Spring (RM 213) 

because most of the other metrics exhibited very little difference across alternatives. Monthly water temperatures in 

the main stem Colorado River at the various sites under each 

long-term strategy were used to calculate the suitability for 

spawning, incubation, and growth during critical periods of 

the annual cycle. The overall temperature suitability at a site 

was the product of the suitability for the three component 

processes during the course of the year and could theoretically 

vary between 0 and 1 (performance metric 2).

6.7 Archaeological and Cultural Resources

The performance metrics associated with archaeological 

and cultural resources (performance metrics 6–8; table 1) did 

not require separate models but, rather, were calculated from 

intermediate sediment and flow variables.

6.8 Hydropower

As noted above in the “Hydrology” section (section 6.2), 

CRSS, the sand budget model, and GTMax-Lite are used 

in combination to simulate the water release and, therefore, 

hydropower generation on an hourly basis for each long-term 

strategy (fig. 2). The GTMax-Lite optimized the economic 

value of hourly energy produced at Glen Canyon Dam. This 

model determined an hour-by-hour pattern of generation (in 

megawatt hours) and water releases (in cubic feet per second) 

that satisfied the operating constraints imposed by each alternative, such as upramp and downramp rates, maximum change 

in the release during a rolling 24-hour period, maximum 

hourly release, and others. Hourly electricity market prices 

during the 20-year LTEMP period were determined using the 

AURORAxmp (Aurora) model to simulate the operation of the 

Western Interconnection of which Glen Canyon Dam is one of 

several thousand generating plants. Hourly prices determined 

by Aurora for 2013 were benchmarked against 2013 day-ahead 

market prices published by the Intercontinental Exchange. 

Prices forecast by Aurora for 2013 were compared against historical Intercontinental Exchange prices, and future electricity 

prices were adjusted accordingly.

For a given long-term strategy, the value of hydropower 

generation (performance metric 9) was calculated by combining the forecasts of the hourly generation profile and electricity market price. The net present value (NPV) of this 20-year 

time series was calculated assuming a 3.375 percent discount 

rate, and then divided by 20 to present the results on an annualized scale.

Hydropower capacity from Glen Canyon Dam is also 

marketed in the form of long-term firm contracts to provide power up to a given level. Energy demand is highest in 

August, so to calculate the marketable capacity (in megawatts), the peak daily generation was tabulated for each day 

in August across the 20 years of a trace (for each long-term 

strategy). From these 620 daily values (20 years x 31 days 

in August), the 90-percent exceedance value (10th quantile) 

was calculated, representing the marketable capacity for that 

trace. In 90 percent of August days (and much more often in 

other months), WAPA can be confident of being able to deliver 

that amount of power without having to purchase it. The net 

present value of that capacity (performance metric 10) was 

calculated by multiplying by the levelized cost of capacity 

plus fixed annual operating and maintenance (O&M) expenses 

of a thermal power plant constructed to replace capacity 

lost at Glen Canyon Dam. The selection of the replacement 

technology was based on insights obtained from the power 

systems economic analysis. The Aurora model runs made for 

this analysis determined that a natural gas combustion turbine 

would be the type of generating unit most likely constructed 
7 Consequence Analysis Results 25

as a replacement for lost Glen Canyon Dam capacity. The cost 

to construct that type of unit, spread over its book life, was 

determined to be $50,100/MW-year, including capital investment costs, allowance for funds during construction, and fixed 

O&M costs. The source of cost data for replacement power 

plant capacity was U.S. Energy Information Administration 

(2014). In the analysis in the LTEMP EIS (chapter 4 and 

appendix K), the daily August generation values for all traces 

were combined before the exceedance value was calculated; 

this results in some slight differences in the median values 

presented but not systematic bias.

6.9 Recreation

The performance metrics associated with recreation 

(performance metrics 11–13; table 1) did not require separate 

models but, rather, were calculated from intermediate sediment and flow variables.

6.10 Riparian Vegetation

The effects of the long-term strategies on riparian vegetation were simulated with a state-and-transition model for 

Colorado River riparian vegetation downstream from Glen 

Canyon Dam (Ralston and others, 2014). The model tracks 

seven vegetation communities (bare sand, marsh, shrub wetland, tamarisk, cottonwood-willow, arrowweed, and mesquite) 

on channel margins and sandbars in the New High Water 

Zone and Fluctuation Zone as affected by the depth, timing, 

and duration of inundation. Six geomorphic submodels are 

included: lower separation bar, upper separation bar, lower 

reattachment bar, upper reattachment bar, lower channel margin, and upper channel margin; only four of these are unique 

(the parameters for the upper separation bar, upper reattachment bar, and upper channel margin are all equal). The model 

starts in 1 of 25 possible states (the 7 vegetation communities 

crossed with the 6 geomorphic submodels; not all of the 42 

combinations are possible) and then tracks the vegetation state 

in each subsequent year as affected by the alternative-specific 

flows. The output is the number of years spent in each vegetation state during the 20-year LTEMP period of performance. 

These results are summed across the 25 starting states and then 

compared to the results that would have happened if the starting state was maintained for the 20-year period. In comparing 

the native to nonnative states, and in calculating the native 

diversity, the results are weighted by the current estimated area 

of each vegetation state.

Four component ratios are calculated as intermediate results: the ratio of cumulative to initial native vegetation cover, the ratio of cumulative to initial vegetation state 

diversity, the ratio of cumulative to initial native to nonnative 

dominant vegetation state, and the ratio of initial to cumulative cover of arrowweed. The final value (riparian native states 

and diversity index; performance metric 14) is the sum of the 

four component ratios; thus, an index value of 4.0 indicates 

an unchanged vegetation condition, values greater than 4.0 

indicate improved vegetation conditions, and values less than 

4.0 indicate degraded vegetation conditions.

7 Consequence Analysis Results

In this section, the results for the individual performance 

metrics are presented. In most cases, these results are reported 

as means over all sources of uncertainty with boxplots used 

to show the variance induced by uncertainty. Unless otherwise noted, the historical weighting of hydrological traces 

was used; the sensitivity of the results to climate change (as 

captured by weighting of the hydrological traces) is discussed 

in section 8.4, “Effects of Climate Change.”

7.1 Humpback Chub Results

Across alternatives, the expected (mean) minimum number of adult HBC during the 20-year LTEMP period of performance differed by about 500 from Alternative A (fig. 4, bottom 

panel). The variation across hydrological traces within an 

alternative was greater than the magnitude of variation across 

alternatives. Many of the long-term strategies were demonstrably better than Alternative A; the biggest exceptions (C2 

and F) were long-term strategies that increased RBT recruitment (through HFEs) without tools to manage RBT populations (TMFs and mechanical removal). The best-performing 

long-term strategy (E6) did not allow HFEs of any type and 

included triggered use of TMFs if needed. The benefit of low 

summer flows to HBC (compare D1 to D4) is not discernible, 

likely because of their infrequent use.

The set of 16 scenarios that captured critical uncertainty did not affect the dominance of long-term strategy E6 

(table 4); that is, if the only desired outcome is to maximize 

the HBC metric, the best long-term strategy is E6 regardless 

of the effect of fall HFEs or TMFs on RBT recruitment or 

the relative effects of temperature and RBT on juvenile HBC 

growth and survival. For the purpose of the HBC objective, 

therefore, the expected value of information across the critical 

uncertainties that were articulated is 0.

Across the uncertainty represented by the hydrological 

traces, there is a small value of information (table 5). For most 

hydrological traces, the best long-term strategy for HBC was 

E6; but long-term strategies B2, C3, D1, and E5 were also 

favored in some traces. In the face of uncertainty, the best 

strategy is E6 with an expected (mean) minimum adult population size of 5,708. If the hydrological trace could be known 

in advance of choosing an action, the appropriate strategy (B2, 

C3, D1, E5, or E6) would be chosen; the expected minimum 

adult population size would be 5,725, which is an increase 

of 0.30 percent. Of course, the hydrological trace cannot be 

known in advance of choosing an action because it represents 

environmental variation during the next 20 years, but the 

inclusion of these results is helpful in understanding the value 
26 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

3,000

4,000

5,000

6,000

7,000

-1,000

0

1,000

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Minimum number of adult humpback chub

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 4. Minimum number of adult humpback chub (performance metric 1) forecast for 19 long-term strategies with equal weighting 

of traces.

of information for resolvable uncertainty (the 16 scenarios) 

relative to the value of information for background environmental variation.

The value of information depends on the set of long-term 

strategies evaluated. The calculations just described assume 

the decision maker is choosing from the entire set of 19 longterm strategies. In practice, it is more likely that one of the 

alternatives would be chosen, and the long-term strategy 

would be chosen from within the set associated with that 

alternative. The value of information in resolving the critical uncertainties is 0 for all alternatives except Alternative C, 

which has a value of information of 8.5 adult HBC (table 6).

Temperature suitability for HBC was forecast at a 

number of locations in the main stem Colorado River, and in 

most cases the suitability was low (less than 0.03 on a scale 

of 0 to 1). At the two locations farthest downstream (Havasu 

Creek, RM 157; Pumpkin Spring, RM 213; fig. 1), the average 

suitability was around 0.1 (on a scale of 0 to 1). The temperature suitability for HBC was affected more by hydrology 

than by the long-term strategy; only Alternative F stood out 

from the other alternatives (fig. 5). Alternative G, with its 

year-round steady flows, provided the best suitability (mean 

0.102); its ranking above the other long-term strategies was 

not affected by any of the critical uncertainties (table 6). There 

was a small effect of the hydrological trace on the ranking of 

alternatives based on temperature suitability (EVXI; 0.77 percent; table 6).

7.2 Rainbow Trout Fishery Results

The RBT performance metrics differed substantially 

across long-term strategies (figs. 6–8). The RBT catch rate 

(performance metric 3) and RBT emigration rate (performance 

metric 4) were highly correlated with one another because 

both were driven by the RBT population size. Catch rate and 

emigration rate were highest for those long-term strategies that 

implemented many HFEs but did not allow TMFs (C2, C4, 

D3, E2, and F) or that provided steady flows that supported 

RBT recruitment (G) (figs. 6–7). The abundance of highquality RBT (greater than 16 inches in length, performance 

metric 5) was negatively correlated with RBT population size; 

the best long-term strategies were those that restricted or did 

not allow HFEs (B1, B2, C3, E3, E5, E6) (fig. 8).

Apart from the HBC metric when only the strategies 

under Alternative C were considered, the RBT performance 

metrics were the only ones affected by the critical uncertainties; the fall HFE hypothesis had an EVXI of 0.26 percent, 

3.50 percent, and 0.98 percent for the three performance 

metrics, respectively (table 6). Across all performance metrics 

and all uncertainties, the strongest effect was the effect of the 

combined hydrological/sediment trace on the RBT catch rate 

with an EVXI of 7.53 percent (table 6). The uncertainty about 

the effectiveness of TMFs did not affect the top ranked longterm strategy for any of the trout or other performance metrics 

(table 6).
7 Consequence Analysis Results 27

Table 4. Effect of critical uncertainty on minimum adult humpback chub population size (performance metric 1).[The cell entries show the expected minimum humpback chub population size for each long-term strategy in each combination of hypotheses. The hypotheses are if high-flow experiments increase rainbow trout recruitment (N, no; Y, yes), how effective rainbow trout management flows are (H, high effectiveness; L, low effectiveness), and the relative effects of temperature and rainbow trout on juvenile humpback chub survival and growth (f1, f2, g1, and g2). A solid dot (●) indicates the result is averaged over this uncertainty. The highlighted long-term strategy is the best performing alternative in each row. HFE, high-flow experiments; TMF, trout management flow; RBT, rainbow trout]

Fall 

HFE TMF Model A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

N H f1 5,204 5,712 5,858 5,463 4,766 5,408 5,402 5,655 5,576 5,174 5,645 5,839 5,500 5,580 5,574 5,572 5,900 4,409 5,082N H f2 5,116 5,413 5,520 5,268 4,772 5,253 5,246 5,324 5,275 5,050 5,318 5,482 5,298 5,357 5,352 5,351 5,521 4,545 5,028N H g1 5,195 5,774 5,941 5,430 4,632 5,377 5,372 5,655 5,556 5,116 5,646 5,874 5,500 5,597 5,593 5,582 5,947 4,184 5,042N H g2 5,135 5,396 5,486 5,324 4,901 5,303 5,295 5,368 5,329 5,125 5,359 5,493 5,327 5,375 5,369 5,376 5,527 4,773 5,078N L f1 5,204 5,653 5,799 5,251 4,766 5,408 5,402 5,439 5,367 5,003 5,432 5,790 5,500 5,580 5,574 5,572 5,857 4,409 4,803N L f2 5,116 5,382 5,487 5,132 4,772 5,253 5,246 5,214 5,168 4,959 5,209 5,457 5,298 5,357 5,352 5,351 5,500 4,545 4,855N L g1 5,195 5,710 5,877 5,189 4,632 5,377 5,372 5,420 5,332 4,931 5,415 5,822 5,500 5,597 5,593 5,582 5,902 4,184 4,739N L g2 5,135 5,367 5,457 5,206 4,901 5,303 5,295 5,265 5,228 5,036 5,257 5,470 5,327 5,375 5,369 5,376 5,507 4,773 4,922

Y H f1 4,803 5,340 5,538 4,824 4,210 5,408 4,365 5,266 5,179 4,781 5,258 4,936 4,608 5,580 4,727 5,572 5,900 4,352 4,603Y H f2 4,870 5,193 5,327 4,878 4,412 5,253 4,559 5,104 5,051 4,820 5,100 4,948 4,715 5,357 4,808 5,351 5,521 4,494 4,709Y H g1 4,748 5,349 5,578 4,720 4,038 5,377 4,232 5,214 5,111 4,680 5,208 4,861 4,496 5,597 4,643 5,582 5,947 4,126 4,549Y H g2 4,916 5,201 5,317 4,974 4,571 5,303 4,679 5,168 5,124 4,914 5,161 5,019 4,814 5,375 4,886 5,376 5,527 4,726 4,762Y L f1 4,803 5,280 5,475 4,639 4,210 5,408 4,365 5,015 4,939 4,575 5,010 4,880 4,608 5,580 4,727 5,572 5,857 4,352 4,364Y L f2 4,870 5,159 5,291 4,755 4,412 5,253 4,559 4,972 4,924 4,702 4,969 4,913 4,715 5,357 4,808 5,351 5,500 4,494 4,557Y L g1 4,748 5,282 5,509 4,521 4,038 5,377 4,232 4,950 4,860 4,472 4,947 4,801 4,496 5,597 4,643 5,582 5,902 4,126 4,292Y L g2 4,916 5,170 5,285 4,864 4,571 5,303 4,679 5,042 5,001 4,793 5,036 4,988 4,814 5,375 4,886 5,376 5,507 4,726 4,621

Grand mean

● ● ● 4,991 5,392 5,541 5,016 4,527 5,335 4,874 5,247 5,181 4,876 5,241 5,269 5,015 5,477 5,103 5,470 5,708 4,450 4,741

Effect of fall HFE hypotheses

N ● ● 5,162 5,551 5,678 5,283 4,768 5,335 5,328 5,417 5,354 5,049 5,410 5,653 5,406 5,477 5,472 5,470 5,708 4,478 4,943

Y ● ● 4,835 5,247 5,415 4,772 4,308 5,335 4,459 5,091 5,024 4,717 5,086 4,918 4,658 5,477 4,766 5,470 5,708 4,424 4,557

Effect of TMF hypotheses

● H ● 4,991 5,415 5,565 5,098 4,527 5,335 4,874 5,337 5,268 4,950 5,330 5,290 5,015 5,477 5,103 5,470 5,724 4,450 4,847● L ● 4,991 5,368 5,516 4,933 4,527 5,335 4,874 5,157 5,094 4,801 5,152 5,248 5,015 5,477 5,103 5,470 5,691 4,450 4,636

Effect of temperature and RBT hypotheses

● ● f1 4,995 5,488 5,660 5,030 4,475 5,408 4,859 5,334 5,256 4,874 5,327 5,340 5,034 5,580 5,131 5,572 5,878 4,379 4,702● ● f2 4,987 5,281 5,402 5,000 4,584 5,253 4,887 5,148 5,099 4,877 5,144 5,188 4,993 5,357 5,067 5,351 5,510 4,518 4,780● ● g1 4,961 5,519 5,718 4,949 4,322 5,377 4,776 5,299 5,204 4,789 5,293 5,316 4,975 5,597 5,096 5,582 5,925 4,154 4,645● ● g2 5,021 5,279 5,382 5,084 4,728 5,303 4,972 5,206 5,166 4,962 5,199 5,232 5,059 5,375 5,117 5,376 5,517 4,748 4,839
28 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan Table 5. Effect of hydrological trace on minimum adult humpback chub population size (performance metric 1). [The cell entries show the expected minimum adult humpback chub population size for each long-term strategy, under each hydrological trace, averaged over the sediment traces and critical uncertainties. The bottom row shows the mean over the hydrological traces. The highlighted long-term strategy is the best performing alternative in each row. The best strategy in the face of uncertainty has an expected mini- mum population size of 5,708; the expected minimum population size if the hydrological trace could be known before the strategy was selected is 5,725 (average of the highlighted cells). Thus, the expected value of information, for resolving uncertainty about the future hydrology, is an increase of 17 adults in the minimum humpback chub population.] Trace

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

h00 3,680 4,570 4,575 4,258 3,286 4,113 3,772 4,616 4,574 3,732 4,617 4,615 3,875 4,252 3,986 4,182 4,887 3,419 4,326h01 3,789 4,804 4,812 4,279 3,284 4,050 3,670 4,692 4,613 3,740 4,692 4,708 3,779 4,182 3,892 4,136 5,038 3,357 4,519h02 3,742 4,403 4,406 3,991 3,304 3,964 3,656 4,372 4,325 3,609 4,370 4,344 3,820 4,131 3,936 4,085 4,592 3,391 4,155h03 4,035 4,325 4,434 4,007 3,476 4,307 3,959 4,306 4,206 3,909 4,287 4,402 4,197 4,576 4,266 4,610 4,837 3,493 4,208h04 5,047 5,443 5,591 4978 4,401 5,345 4,892 5,083 5,026 4,930 5,071 5,147 4,944 5,455 5,076 5,450 5,639 4,430 4,674h05 6,386 6,619 6,901 6,166 5,940 6,873 6,164 6,234 6,152 6,126 6,236 6,318 6,335 6,998 6,433 7,008 6,998 5,767 5,226h06 4,200 4,497 4,794 4,048 3,726 4,563 4,102 4,324 4,258 4,097 4,292 4,242 4,240 4,735 4,330 4,739 4,735 3,664 3,918h07 4,206 4,386 4,555 4,274 3,972 4,618 4,207 4,445 4,347 4,245 4,445 4,535 4,448 4,857 4,518 4,857 4,918 3,765 4,421h08 5,176 5,303 5,378 4,988 4,733 5,364 5,064 5,317 5,278 5,165 5,323 5,262 5,274 5,523 5,304 5,543 5,523 4,364 4,831h09 5,435 5,561 5,726 5,297 5,193 5,580 5,269 5,746 5,704 5,528 5,730 5,486 5,414 5,714 5,453 5,703 5,713 4,864 4,953h10 6,053 6,160 6,258 6,036 5,913 6,413 6,049 6,035 5,970 5,900 6,030 5,984 5,976 6,362 6,048 6,377 6,362 5,792 5,433

h11 6,788 7,118 7,349 6,630 6,365 7,433 6,577 6,710 6,646 6,551 6,735 6,792 6,802 7,544 6,867 7,553 7,544 6,003 5,526

h12 6,310 6,560 6,677 5,854 5,580 6,628 5,879 6,048 5,930 5,906 6,040 6,242 6,200 6,967 6,314 6,998 6,970 5,035 5,052h13 4,511 4,965 5,111 4,567 4,005 4,868 4,372 4,657 4,614 4,350 4,673 4,791 4,565 5,080 4,643 5,071 5,340 4,026 4,107h14 4,120 5,185 5,313 4,644 3,663 4,391 4,045 5,029 4,986 3,931 4,991 4,828 4,151 4,513 4,236 4,479 5,166 3,713 4,436h15 4,130 4,543 4,715 4,097 3,601 4,368 3,981 4,256 4,160 4,024 4,186 4,342 4,011 4,458 4,124 4,442 4,745 3,668 4,202h16 5,013 5,313 5,674 5,172 4,716 5,772 5,186 5,229 5,159 4,966 5,218 5,292 5,135 5,682 5,201 5,679 5838 4,761 4,568h17 4,994 5,213 5,375 4,948 4,615 5,210 4,942 5,347 5,281 5,079 5,347 5,221 5,036 5,317 5,099 5,309 5,471 4,638 4,938h18 6,419 6,586 6,692 6,192 5,814 6,653 6,201 6,476 6,432 6,328 6,489 6,503 6,457 6,803 6,505 6,818 6,857 5,905 6,009h19 6,678 7,055 7,339 6,506 5,895 7,076 6,354 6,540 6,468 6,242 6,555 6,880 6,511 7,210 6,674 7,202 7,570 5,832 5,801h20 4,098 4,617 4,678 4,395 3,590 4,450 4,004 4,721 4,675 4,027 4,725 4,710 4,150 4,659 4,254 4,629 5,117 3,559 4,267

Mean 4,991 5,392 5,541 5,016 4,527 5,335 4,874 5,247 5,181 4,876 5,241 5,269 5,015 5,477 5,103 5,470 5,708 4,450 4,741
7 Consequence Analysis Results 29

Table 6. Expected value of information, by performance metric, for resolving critical uncertainty.[See table 1 for a list of performance metrics. The best performance is the expected value of the performance metric across all uncertainties for the long-term strategy that maximized or minimized the performance metric. The value of perfectly resolving uncertainty across the 16 formal hypotheses about critical parameters is shown when all 19 long-term strategies are considered; only the two long-term strategies of alternative B are considered; and only the four, four, or six long-term strategies of alternatives C, D, and E are considered, respectively. Partial values of information are shown for resolving only the uncertainty about the effects of fall high-flow experiments on rainbow trout recruitment, efficacy of trout management flows, and effects of temperature and rainbow trout on humpback chub. Finally, the partial values of information are shown for resolving uncertainty about the combined hydrological-sediment trace, just the sediment trace, and just the hydrological trace. For the performance metrics not shown, all values of information were 0. min, minimum; HBC, humpback chub; TempSuit, temperature suitability index; Qual, quality; RBT, rainbow trout; WTSI, wind transport of sediment index; GC flow, Glen Canyon flow index; TOR, time-off-river index; CAI, camping area index; Veg, riparian vegetation index; SLI, sand load index; max, maximum; EVPI, expected value of perfect information; EVXI, expected value of partial information; HFE, high-flow experiment; TMF, trout management flow; ~, value less than 0.01 percent]

Performance metric

1 2 3 4 5 6 7 8 9 11 14 15 16

minHBC TempSuit RBT 

Catch

Emigrate Qual 

RBT

WTSI GC flow TOR Power CAI Veg SLI Marsh

Desired direction max max max min max max min max max max max max max

Best performance 5,708 0.102 3.367 22,415 956 0.465 18.4 0.840 150.41 0.451 3.954 0.576 1.101Best action E6 G F E6 E6 G E3 G B2 G D4 G E6EVPI (16, all) 0.0 0.000 0.009 785 9.3 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVPI (16, B) 0.0 0.000 0.000 0 0.0 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVPI (16, C) 8.5 0.000 0.000 237 0.8 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVPI (16, D) 0.0 0.000 0.000 0 0.0 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVPI (16, E) 0.0 0.000 0.000 0 0.0 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVXI (fall HFE) 0.0 0.000 0.009 785 9.3 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVXI (TMF) 0.0 0.000 0.000 0 0.0 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVXI (HBC) 0.0 0.000 0.000 0 0.0 0.000 0.0 0.000 0.00 0.000 0.000 0.000 0.000EVXI (traces) 19.0 0.001 0.254 1,033 8.4 0.003 0.9 0.001 0.35 0.007 0.224 0.021 0.035EVXI (sediment) 0.0 0.000 0.000 37 0.0 0.000 0.0 0.000 0.00 0.000 0.012 0.000 0.000EVXI (hydrology) 17.0 0.001 0.198 636 5.5 0.002 0.9 0.001 0.35 0.004 0.212 0.015 0.032

As a percent of the best performance

EVPI (16, all), in percent ~ ~ 0.26 3.50 0.98 ~ ~ ~ ~ ~ ~ ~ ~EVPI (16, B) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~EVPI (16, C), in percent 0.15 ~ ~ 1.06 0.09 ~ ~ ~ ~ ~ ~ ~ ~EVPI (16, D) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~EVPI (16, E) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~EVXI (fall HFE), in percent ~ ~ 0.26 3.50 0.98 ~ ~ ~ ~ ~ ~ ~ ~EVXI (TMF) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~EVXI (chub) ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~EVXI (traces), in percent 0.33 1.39 7.53 4.61 0.88 0.59 4.78 0.10 0.23 1.63 5.66 3.57 3.20EVXI (sediment), in percent ~ ~ ~ 0.17 ~ ~ ~ 0.03 ~ ~ 0.30 ~ ~EVXI (hydrology), in percent 0.30 0.77 5.89 2.84 0.58 0.44 4.77 0.08 0.23 0.96 5.36 2.58 2.91
30 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0

0.05

0.10

0.15

-0.10

-0.05

0

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Temperature suitability index

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 5. Temperature suitability index for humpback chub at river mile 157 and 213 (performance metric 2) forecast for 19 long-term 

strategies with equal weighting of traces.

1

2

3

4

5

-2

-1

0

1

2

Rainbow trout catch rate, in fish per hour

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 6. Rainbow trout catch rate (performance metric 3) forecast for 19 long-term strategies with equal weighting of traces.
7 Consequence Analysis Results 31

20,000

40,000

60,000

80,000

100,000

-50,000

0

50,000

Rainbow trout emigration rate, in fish per year

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 7. Rainbow trout emigration rate from Glen Canyon (performance metric 4) forecast for 19 long-term strategies with equal 

weighting of traces.

600

800

1,000

-200

0

200

400

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Rainbow trout greater than 16 inches

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 8. Abundance of high-quality rainbow trout (performance metric 5) forecast for 19 long-term strategies with equal weighting of 

traces.
32 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

7.3 Archeological and Cultural Resources 

Results

The protection of archeological and cultural resources 

was reflected in three performance metrics. The WTSI (performance metric 6) was largely driven by the SLI (performance 

metric 15) and was highest for those long-term strategies that 

implemented HFEs often (fig. 9). The best-performing alternative for WTSI was Alternative G (mean WTSI, 0.465); this 

ranking was not affected by any of the critical uncertainties or 

uncertainty in the sediment input (table 6). For some hydrological traces, long-term strategy C1 outperformed G, but the 

value of information related to hydrological uncertainty was 

small (0.44 percent).

The Glen Canyon flow index (performance metric 7) differed little among alternatives with a mean of around 20 days 

per year in which the flow exceeded 23,200 ft3/s (fig. 10). 

Only Alternative F differed from the other long-term strategies 

(mean 36.8 days/year). The long-term strategies that did not 

allow HFEs (C3, E3, E5, and E6) had the best performance 

(smallest Glen Canyon flow index). The top ranking long-term 

strategy was not affected by critical uncertainty or sediment 

trace (table 6) but did vary depending on the hydrological 

trace (EVXI, 4.77 percent).

The time-off-river index (performance metric 8) was 

affected most strongly by the hydrological trace but was also 

weakly affected by the long-term strategy (fig. 11). Alternatives B and C were indistinguishable from Alternative 

A; Alternatives D, E, and G performed slightly better than 

Alternative A; and Alternative F performed noticeably worse 

because high flows during the peak rafting season allow 

quicker trips and more discretionary time at camping stops. 

Long-term strategies E1 and G were the best-performing 

strategies across hydrological and sediment traces, with only 

a small value of information associated with the trace uncertainty and no value of information associated with the critical 

uncertainties (table 6).

7.4 Hydropower Generation and Capacity 

Results

The best-performing long-term strategy for hydropower 

generation (performance metric 9) was B2, a long-term 

strategy designed to match power generation more closely 

to demand (fig. 12). For most of the long-term strategies, the 

annual value of hydropower generation is within $3 million 

of the value from Alternative A, except for Alternatives F and 

G for which annual generation is $6 million to $8 million less 

than Alternative A (fig. 12). The effect of hydrology on hydropower generation is much greater than the effect of the alternatives, with the average annual power generation over 20 years 

varying more than $60 million across hydrological traces (fig. 

12). The dominance of B2 as the best long-term strategy for 

hydropower generation, however, is robust to uncertainty in 

hydrology with a value of information of only $350,000 per 

year (EVXI, 0.23 percent, table 6); for 4 out of 21 hydrological traces, long-term strategy E3 outranked B2.

The value of hydropower capacity (performance metric 

10) was more sensitive to the choice of long-term strategy than 

hydropower generation with long-term strategies differing 

by nearly $20 million per year (fig. 13). Long-term strategy 

B2 was the best-performing strategy across all uncertainties, including hydrological trace. The long-term strategies in 

Alternative B outperformed Alternative A; all the remaining 

long-term strategies had a lower value of hydropower capacity 

than Alternative A (fig. 13). The value of capacity was somewhat sensitive to hydrological trace (the range of performance 

within a long-term strategy varied by $8 million to $17 million 

per year across hydrological traces), but the identification of 

the best-performing alternative was not (EVXI, 0).

7.5 Recreational Experience Results

The camping area index (performance metric 11), a metric composed from the SLI to represent beach formation and a 

flow factor to represent beach exposure, was highly correlated 

with the SLI (performance metric 18). The best-performing 

long-term strategies (especially C1, F, and G) include frequent 

implementation of HFEs; the worst-performing long-term 

strategies (for example, C3, E3, E5, and E6) do not permit 

HFEs (fig. 14). The identification of the top-ranked long-term 

strategy was affected by the combination of hydrological and 

sediment trace (EVXI, 1.96 percent) but not by any other 

uncertainties (table 6).

The fluctuation index (performance metric 12), which 

reflects the fraction of time the daily flow fluctuations are in a 

tolerable range for recreation, was highest for Alternatives F 

and G and lowest for Alternative B (fig. 15). Alternative D was 

comparable to Alternative A. The effect of the long-term strategies within an alternative was small compared to the effect 

of the base operations within each alternative. The fluctuation 

index was not strongly affected by any of the uncertainties, 

including hydrological trace.

The Glen Canyon rafting use metric (performance metric 13), which reflects the visitor-days lost per year because 

of HFEs, was best for those long-term strategies that did not 

permit HFEs (for example, C3, E3, E5, and E6) and worst for 

Alternative F (fig. 16). Under Alternative F, nearly 1,000 boat 

seats per year in Glen Canyon are expected to be lost because 

of HFEs. The rafting use metric showed variation as a result 

of hydrological trace, but the identification of the best-performing long-term strategies was not affected by any of the 

uncertainties.

7.6 Riparian Vegetation Results

The riparian vegetation index (performance metric 14) 

was the sum of four ratios that represented measures of 

different aspects of the vegetation community with a 

score of 4 indicating maintenance of current conditions. 
7 Consequence Analysis Results 33

0

0.2

0.4

0.6

-0.2

0

0.2

0.4

Wind transport of sediment index

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 9. Wind transport of sediment index (performance metric 6) forecast for 19 long-term strategies with equal weighting of 

traces.

0

20

40

60

80

-10

0

10

20

Glen Canyon flow index, in days per year 

in which flow exceeds 23,200 cubic feet per second

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 10. Glen Canyon flow index (performance metric 7) forecast for 19 long-term strategies with equal weighting of traces.
34 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0.7

0.8

0.9

-0.08

-0.06

-0.04

-0.02

0

0.02

Time-off-river index

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 11. Cultural resources time-off-river index (performance metric 8) forecast for 19 long-term strategies with equal weighting of 

traces.

120

140

160

180

-10

-5

0

Value of hydropower generation, in millions of dollars per year

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 12. Annualized net present value of hydropower generation (performance metric 9) forecast for 19 long-term strategies with 

equal weighting of traces.
7 Consequence Analysis Results 35

10

20

30

40

-20

-10

0

Value of hydropower capacity, in millions of dollars per year

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

equal weighting of traces.

Figure 13. Annualized net present value of hydropower capacity (performance metric 10) forecast for 19 long-term strategies with 

0

0.2

0.4

0.6

-0.2

0

0.2

0.4

Camping area index

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 14. Camping area index (performance metric 11) forecast for 19 long-term strategies with equal weighting of traces.
36 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0.2

0.4

0.6

0.8

1.0

-0.6

-0.4

-0.2

0

0.2

Fluctuation index

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 15. Fluctuation index (performance metric 12) forecast for 19 long-term strategies with equal weighting of traces.

0

500

1,000

0

500

1,000

Glen Canyon rafting use metric, in visitor-days per year

lost because of high-flow experiments

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Long-term strategy

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Figure 16. Glen Canyon rafting use metric (performance metric 13) forecast for 19 long-term strategies with equal weighting of traces.
7 Consequence Analysis Results 37

The index ranged between 2 and 6 across alternatives and 

traces (fig. 17), indicating a 50-percent degradation or a 

50-percent improvement in vegetation conditions, respectively. The highest mean vegetation index was associated with 

long-term strategy D4 (riparian vegetation index, 3.954), but 

13 of the 19 long-term strategies were ranked first depending 

on the hydrological and sediment trace. The value of information for resolving uncertainty about the hydrological and 

sediment trace before committing to a long-term strategy was 

0.224 (an improvement of 5.66 percent; table 6). The choice 

of the long-term strategy was not affected, however, by any of 

the critical uncertainties (table 6).

7.7 Sediment Results

The SLI (performance metric 15) measured the potential 

for sand bar formation by reporting the proportion of sand 

transported during flows greater than 31,500 ft3/s. The SLI was 

most strongly affected by the frequency of HFEs; the longterm strategies with the highest SLI (especially C1, C2, D1, 

D2, F, and G) allowed frequent HFEs, and the long-term strategies with the lowest SLI (C3, E3, E5, and E6) did not permit 

HFEs (fig. 18). Uncertainty in the combination of hydrological 

and sediment trace did affect the ranking of the alternatives 

(EVXI, 3.57 percent; table 6) with 6 of the 19 long-term strategies favored in at least 1 trace.

7.8 Tribal Resources Results

Several performance metrics were developed to evaluate 

resources of specific importance from a tribal perspective. The 

marsh vegetation ratio (performance metric 16) forecast the 

preservation or expansion of the wetland vegetation communities along the Colorado River (fig. 19). For context, the current 

extent of marsh vegetation is 4.6 acres. The long-term strategy 

with the highest mean marsh vegetation ratio was E6 (mean 

marsh vegetation ratio, 1.101), representing about a 10-percent 

increase in marsh community area, but long-term strategy E3 

was most commonly ranked first among the 63 traces. Alternatives C, F, and G exhibited consistent losses of marsh vegetation compared to Alternative A, whereas Alternatives D and E 

exhibited modest increases (fig. 19). The variance in the index 

induced by variation in hydrology was larger than the variance across alternatives (fig. 19). The value of information for 

resolving hydrological and sediment uncertainty before committing to a long-term strategy was 3.20 percent (table 6).

The frequency of mechanical removal (performance metric 17) was 0 for those long-term strategies that do not permit 

this management tool (C1, C2, E1, E2, E5, E6, and F) and 

varied between 0 and 6 years out of 20 for the other long-term 

strategies (fig. 20). For those long-term strategies that permitted mechanical removal, the frequency was influenced by the 

emigration rate of RBT from Glen Canyon (performance metric 4), which in turn was influenced by the frequency of HFEs. 

Because the long-term strategies with the lowest frequency 

were determined simply by whether mechanical removal was 

allowed or not, the identification of the top-ranked strategy 

was not influenced by any of the uncertainties.

The frequency of trout management flows (performance 

metric 18) was 0 for those long-term strategies that did not 

allow their use (10 out of the 19 long-term strategies) and 

varied between 0 and 20 years out of 20 for the remaining 

strategies (fig. 21). Under Alternative G, TMFs were triggered 

on average more than one-half of the years, and sometimes in 

all 20 years, depending on the hydrological trace. For the strategies that allowed TMFs, the frequency of their use was correlated with RBT abundance and, hence, with the RBT catch 

rate (performance metric 3) and emigration rate (performance 

metric 4). Again, because the long-term strategies with the 

lowest frequency of TMFs were determined by the admissible 

tools in the strategy, the value of information associated with 

the various uncertainties was 0.

7.9 Full Consequence Table

A summary of the performance of the 19 long-term strategies against the 18 performance metrics is represented in a 

consequence table (table 7). There is neither a single long-term 

strategy that performs best for all performance metrics (such 

a strategy would be shaded in yellow across the corresponding row), nor a single long-term strategy that performs worst 

for all performance metrics (such a strategy would be shaded 

dark blue across the row); furthermore, no long-term strategy 

is either consistently better or consistently worse than Alternative A across all performance metrics. Interestingly, the 

strategies that are worst for a number of performance metrics 

(for example, B2, C3, E5, F, and G) are also best for other performance metrics. Other long-term strategies are neither best 

nor worst on any performance metrics (for example, B1, C4, 

D1, D2). This pattern of performance indicates that there are 

important tradeoffs among the long-term strategies, and the 

best alternative cannot be identified without considering the 

relative value of the resource goals and performance metrics.
38 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

2

4

6

-3

-2

-1

0

1

Riparian vegetation index

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 17. Riparian vegetation index (performance metric 14) forecast for 19 long-term strategies with equal weighting of traces.

0

0.2

0.4

0.6

-0.2

0

0.2

0.4

Sand load index, in proportion of sand transported 

during flows greater than 31,500 cubic feet per second

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 18. Sand load index (performance metric 15) forecast for 19 long-term strategies with equal weighting of traces.
7 Consequence Analysis Results 39

0

0.5

1.0

1.5

-1.0

-0.5

0

0.5

1.0

Marsh vegetation ratio

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 19. Marsh vegetation ratio (ratio of average marsh cover over 20-year period to current marsh cover, performance metric 16) 

forecast for 19 long-term strategies with equal weighting of traces.

0

2

4

6

0

2

4

6

Frequency of mechanical removal, in years out of 20

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 20. Frequency of mechanical removal (performance metric 17) forecast for 19 long-term strategies with equal weighting of 

traces.
40 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0

5

10

15

20

0

5

10

15

20

Frequency of trout management flows, in years out of 20

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 21. Frequency of trout management flows (performance metric 18) forecast for 19 long-term strategies with equal weighting of 

traces.
7 Consequence Analysis Results 41

Table 7. Full consequence table showing the mean value of each performance metric for each long-term strategy.[The columns are the performance metrics, numbered as in table 1. For most performance metrics, higher values are desired (yellow), but for five metrics, lower values are desired (pink). The rows are the long-term strategies. The values in each column are color-coded, with the best-performing long-term strategy shaded in yellow, the worst-performing long-term strategy shaded in dark blue, strategies that perform better than status quo (Alternative A) shaded in orange, and strategies that perform worse than status quo shaded in turquoise. HBC, humpback chub; TempSuit, temperature suitability index; RBT, rainbow trout; WTSI, wind transport of sediment index; GC flow, Glen Canyon flow index; TOR, time-off-river index; CAI, camping area index; FI, fluctuation index; GC raft, Glen Canyon rafting use metric; Veg, riparian vegetation index; SLI, sand load index; MR, mechanical removal; TMF, trout management flow]

Longterm 

strategy

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

HBC Temp Suit RBT Catch RBT Emigrate Quality RBT WTSI GC flow TOR Power Cap. CAI FI GC raft Veg SLI Marsh MR TMF

High High High Low High High Low High High High High High Low High High High Low Low

A 4,991 0.097 2.11 36,699 769 0.159 22.7 0.823 148.5 28.5 0.139 0.786 49 3.66 0.211 0.72 0.07 0.0

B1 5,392 0.097 1.67 29,586 867 0.171 23.1 0.823 149.4 30.4 0.146 0.420 71 3.87 0.234 0.80 0.44 3.0B2 5,541 0.097 1.46 24,172 920 0.144 23.1 0.823 150.4 32.4 0.122 0.256 72 3.12 0.222 0.17 0.30 3.1C1 5,016 0.082 2.23 43,683 748 0.377 21.8 0.824 147.3 20.8 0.376 0.935 315 3.18 0.536 0.25 0.00 6.5C2 4,527 0.079 3.18 66,890 640 0.365 21.8 0.823 147.2 19.5 0.371 0.929 307 3.18 0.534 0.25 0.00 0.0C3 5,335 0.079 1.90 33,559 830 0.043 18.5 0.821 148.9 20.8 0.043 0.924 0 2.83 0.065 0.25 0.74 0.0C4 4,874 0.079 2.72 55,076 707 0.334 21.0 0.823 147.6 20.8 0.335 0.928 83 2.98 0.483 0.25 2.80 0.0

D1 5,247 0.094 2.02 40,784 811 0.379 23.5 0.835 146.6 23.8 0.359 0.741 348 3.67 0.531 0.75 1.67 3.9D2 5,181 0.095 2.15 43,981 796 0.378 23.6 0.835 146.1 19.6 0.360 0.784 351 3.69 0.535 0.76 2.02 6.9D3 4,876 0.095 2.63 55,811 711 0.378 23.5 0.836 146.8 23.8 0.359 0.724 348 3.70 0.533 0.78 2.95 0.0D4 5,241 0.097 2.03 40,936 810 0.380 23.5 0.836 146.7 25.1 0.358 0.741 348 3.95 0.529 0.84 1.69 3.8

E1 5,269 0.090 1.93 37,614 826 0.311 21.3 0.839 148.0 22.8 0.303 0.568 177 3.54 0.456 0.62 0.00 2.6E2 5,015 0.086 2.33 47,450 761 0.297 21.3 0.837 147.9 21.8 0.292 0.534 174 3.84 0.443 0.85 0.00 0.0E3 5,477 0.087 1.68 28,499 891 0.030 18.4 0.836 149.3 22.8 0.028 0.517 0 3.93 0.046 1.10 0.47 0.0E4 5,103 0.087 2.19 42,806 781 0.281 20.9 0.838 148.1 22.8 0.272 0.529 79 3.93 0.415 0.91 1.73 0.0E5 5,470 0.083 1.68 28,561 890 0.029 18.5 0.835 149.1 21.8 0.028 0.517 0 3.87 0.046 1.05 0.00 0.0E6 5,708 0.087 1.42 22,415 956 0.032 18.8 0.837 149.3 22.8 0.030 0.518 0 3.93 0.049 1.10 0.00 2.4

F 4,450 0.030 3.37 71,869 592 0.299 36.8 0.749 141.0 11.2 0.406 0.997 919 3.14 0.558 0.14 0.00 0.0

G 4,741 0.102 2.81 58,533 702 0.465 24.7 0.840 142.9 18.0 0.451 0.981 512 3.40 0.576 0.42 3.05 11.0
42 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

8 Decision Analysis Results

The purposes of completing a formal decision analysis 

as a component of the evaluation in the LTEMP EIS was to 

explicitly examine the influence of the relative importance 

assigned to the resource goals on the ranking of the alternatives and long-term strategies, and to understand the effect of 

uncertainty on this same ranking. As described in section 3, 

“Decision Analysis,” the two primary tools used to support 

this investigation were MCDA and the expected value of 

information.

8.1 Swing-Weighting Method and Results

The swing-weighting method (Winterfeldt and Edwards, 

1986) was used to develop weights for the performance 

metrics, reflecting individual stakeholder expressions of the 

relative importance of those metrics. In April 2014, a 2-day 

workshop was held to present a preliminary analysis of 

the long-term strategies against the performance metrics to 

interested stakeholders. After discussion of the meaning and 

interpretation of each of the performance metrics, the stakeholders were given a swing-weighting response form with 

instructions (table 8) to complete within several weeks. The 

swing-weighting method asks a decision maker or stakeholder 

to consider each resource goal, the performance metric that 

reflects it, and the range over which the performance metric 

varies; and to compare the relative importance of the performance metrics by evaluating how valuable it would be to 

change the performance of an alternative from the worst score 

for a performance metric to the best score. The stakeholders were first asked to rank the performance metrics in this 

manner and then to assign a score between 0 and 100 to each 

metric to reflect a more nuanced interpretation of the ranking. Because it is difficult to evaluate 18 performance metrics 

simultaneously, and because there were natural groupings of 

the metrics, the swing-weighting exercise used a two-level 

structure. The 18 performance metrics were assigned to 

8 higher-level groups (table 8). The stakeholders first ranked 

and scored the metrics within each group. Then, to rank and 

score the higher-level groups, the stakeholders were asked to 

think about the importance of changing the scores of all the 

metrics within a group for their worst levels to their best levels 

at the same time. While completing this exercise, the stakeholders had access to a preliminary consequence table (similar 

to table 7) and a spreadsheet that automatically calculated the 

weights on the performance metrics as the swing-weighting 

sheet was filled in.

A total of 27 Federal, State, tribal, and private agencies 

and organizations were invited to participate in the decision 

analysis by expressing their view of the relative importance of 

the performance metrics using the swing-weighting method 

(Winterfeldt and Edwards, 1986). The invitations were sent to 

agencies that were either members of AMWG or Cooperating 

Agencies for the LTEMP EIS; 14 agencies, including the 2 

joint-lead agencies, elected to participate (table 9).

From the responses to the swing-weighting exercise, 

weights were calculated for the 18 performance metrics 

separately for each participating stakeholder. To compare these 

vectors of weights, principal components analysis was used to 

reduce the dimension of the comparison from 18 to 2. The first 

two principal components explained 52.6 percent of the variation in weights across stakeholders. The first principal component was positively correlated with the weight on the frequency of mechanical removal, SLI, WTSI, fluctuation index, 

and riparian vegetation index; and negatively correlated with 

hydropower generation and capacity (fig. 22A). The second 

principal component was positively correlated with HBC population size, Glen Canyon rafting use, and the camping area 

index; and negatively correlated with RBT emigration and the 

Glen Canyon flow index (fig. 22A). The differences among the 

participating stakeholders can then be plotted with these two 

components (fig. 22B). Representatives from the utility industry (CREDA, UAMPS, and SRP) tended to put more weight 

on the hydropower generation and capacity metrics than other 

stakeholders. The agencies concerned with management of the 

RBT fishery (International Federation of Fly Fishers [IFFF] 

and Arizona Game and Fish Department [AZGFD]) were in 

the middle of the spectrum on the first component but had 

strong positive values for the second component, reflecting 

an emphasis on HBC, high-quality RBT, and RBT catch rate. 

The FWS, Arizona Department of Water Resources (ADWR) 

and the Hopi, Hualapai, and Navajo Tribes placed relatively 

more emphasis on increasing vegetation and minimizing RBT 

emigration than other stakeholders. The principal components 

for two of the nongovernment organizations (Grand Canyon 

River Guides [GCRG] and National Parks Conservation Association [NPCA]) placed an emphasis on camping area index, 

SLI, and WTSI. Note that the joint-lead agencies (Reclamation 

and NPS) separately completed the swing-weighting exercise, 

and their separate weights were used as input to the principal 

components analysis, but they elected to average their weighting vectors for presentation and discussion. The joint-lead 

principal components fall near the center of the spectrum of 

stakeholders. Although some of these patterns compare well 

with the expressed views of the stakeholder agencies, others 

are harder to explain; it is important to note that the higherorder principal components still contained explanatory power, 

so not all the differences among agencies can be summarized 

with the first two components.

8.2 Multicriteria Decision Analysis

The weights on the performance metrics, unique to 

each agency, tribe, or organization, were combined with 

the consequence table to generate a weighted performance 

metric for each long-term strategy, allowing a comparison 

of the long-term strategies that integrates the differences in 

response across performance metrics and the relative value 
8 Decision Analysis Results 43

Table 8. Swing-weighting response form.[Stakeholder agencies were invited to share their perspectives on the relative importance of the resource goals and performance metrics using this swing-weighting exercise. The results were compiled to generate agency-specific weights on the performance metrics; GLCA, Glen Canyon National Recreation Area; GRCA, Grand Canyon National Park; min, minimum; M, million; B, billion; >, greater than; yr, year; hr, hour; #, number; TMF, trout management flow]

Agency:Participants: Higher level Sublevel

Goal Performance metrics Units Desired direction Worst case Best case Rank Score Rank Score

Humpback chub

Number of adult humpback chub (20-yr min) # of adults High 3,000 8,500Temperature suitability index (river mile 157 and 213) index (0–1) High 0 0.20

Rainbow trout fishery

Angler catch rate index fish/angler-hr High 1.0 5.0Lees Ferry trout emigration estimate trout/yr Low 125,000 15,000Number of trout > 16 inches total length # fish High 400 1,200

Archaeological and 

cultural resources

Wind transport of sediment index (WTSI) index (0–1) High 0 0.50GLCA flow effects of historic properties days flow > 23.2k Low 75 0Time-off-river index (TOR) index (0–1) High 0.60 0.95

Hydropower

Value of energy generation (mean annual) Million $/yr

Billion $/20-yr

High 120M/

2.4B

200M/

4B

Value of capacity (mean annual) Million $/yr 

Billion $/20-yr

High 10M/

0.2B

50M/

1B

Recreation

GRCA camping area index (average of 3) index (0–1) High 0 0.50Fluctuation index index (0–1) High 0 1.0GLCA rafting use index boat seats lost/yr Low 1,300 0

Riparian vegetation

Vegetation index (sum of four ratios) sum of ratios High 2.0 6.0 1 100

Sediment

GRCA sand load index (sand transported > 31,500) proportion High 0 0.6 1 100

Tribal

Change in marsh vegetation ratio High 0 1.5Mechanical removal (years out of 20) yrs Low 5 0Frequency of TMF (years out of 20) yrs Low 20 0
44 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Table 9. Agencies participating in the swing-weighting exercise.

[Representatives from 27 Federal, State, tribal, and private agencies were invited to participate in the swing-weighting exercise. The set of invited agencies 

included the Adaptive Management Working Group and cooperating agencies for the Long-term Experimental and Management Plan Environmental Impact 

Statement. Of the 27 agencies invited, 14 participated. AMWG, Adaptive Management Working Group; CA, cooperating agency; NGO, non-governmental 

organization]

Agency Affiliation AMWG CA Participant

Bureau of Indian Affairs Federal Yes Yes No

National Park Service Federal Yes Yes Yes

Bureau of Reclamation Federal Yes Yes Yes

U.S. Fish and Wildlife Service Federal Yes Yes Yes

Western Area Power Administration Federal Yes Yes No

Havasupai Tribe Tribe No Yes No

Hopi Tribe Tribe Yes Yes Yes

Hualapai Tribe Tribe Yes Yes Yes

Navajo Nation Tribe Yes Yes Yes

Pueblo of Zuni Tribe Yes Yes No

Southern Paiute Consortium Tribe Yes No No

Fort Mojave Indian Tribe Tribe No Yes No

National Parks Conservation Association NGO Yes No Yes

International Federation of Fly Fishers NGO Yes No Yes

Grand Canyon River Guides NGO Yes No Yes

Arizona Department of Water Resources State Yes No Yes

Arizona Game and Fish Department State Yes Yes Yes

Colorado River Board of California State Yes Yes No

Colorado Water Conservation Board State Yes No No

Colorado River Commission of Nevada State Yes Yes No

New Mexico Interstate Stream Commission State Yes No No

Wyoming State Engineer’s Office State Yes No No

Utah Division of Water Resources State Yes No No

Salt River Project Public power utility No Yes Yes

Upper Colorado River Commission State and Federal No Yes No

Colorado River Energy Distributors Association Power purchase contractor Yes No Yes

Utah Associated Municipal Power Systems Power purchase contractor Yes Yes Yes

the particular agency places on those performance metrics. 

For the joint-lead agencies, using their average set of weights, 

Alternative D (and specifically, long-term strategy D4) performed better than the other alternatives (fig. 23). Alternatives 

C, D, E, F, and G all substantially outperformed Alternative A 

(status quo), provided HFEs were implemented. The long-term 

strategies that did not implement HFEs (C3, E3, E5, and E6) 

were all demonstrably poorer than Alternative A, as measured 

by the weighted performance. The weighted performance of 

Alternative B was similar to that of Alternative A, with longterm strategy B1 slightly better than long-term strategy B2.

Alternative D was created after a preliminary analysis of 

the other alternatives. In the April 2014 preliminary analysis, 

the results of which were discussed by the joint-lead agencies 

and the Cooperating Agencies, Alternative C and E exhibited 

roughly equivalent performance with some differences in 

ranking across stakeholders. Alternative D was created as a 

hybrid between Alternative C and E, taking features of each 

that had contributed to positive performance in the preliminary analysis. One of the advantages of separating objective 

weighting from alternative evaluation in MCDA is that the 

weights only depend on the ranges of the performance metrics, 

not on the alternatives under consideration, so new alternatives 

can be evaluated without having to re-elicit the swing weights. 

Because Alternative D was created from the insights gained in 

the preliminary analysis of alternatives, it stands to reason that 

Alternative D outperformed Alternatives C and E. It is interesting, however, that the gain in performance is fairly small, 

suggesting that only marginal gains in performance remain 

with the management tools available, against the backdrop 
8 Decision Analysis Results 45

-0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4

0.4

0.3

0.2

0.1

0

-0.1

-0.2

-0.3

-0.4

-0.5

First component 

First component 

TMF MR

Marsh vegetation

SLI

Vegetation

index

Glen Canyon rafting use

Fluctuation index

Camping area index

Hydropower capacity

Hydropower

generation

Time-off-river

Glen Canyon flow index

WTSI

High-quality rainbow trout

Trout emigration

Rainbow trout catch rate

HBC temperature 

suitability

Adult HBC

(minimum)

Second component

A. Loading plot

-5 -4 -3 -2 -1 0 1 2 3 4

3

2

1

0

-1

-2

-3

GCRG

NPCA

IFFF

Navajo

Hualapai

Hopi CREDA

UAMPS

SRP

AZGFD

ADWR

FWS

Second component

B. Score plot

Joint-lead average

UAMPS

IFFF

AZGFD

GCRG

NPCA

Utah Associated Municipal Power Systems

International Federation of Fly Fishers

Arizona Game and Fish Department

Grand Canyon River Guides

National Parks Conservation Association

 

CREDA

SRP

ADWR

FWS

Colorado River Energy Distributors Association

Salt River Project

Arizona Department of Water Resources

U.S. Fish and Wildlife Service

 

HBC

TMF

MR

SLI

WTSI

 >

Humpback chub 

Trout management flow

Mechanical removal

Sand load index

Wind transport of sediment 

index

Greater than 

EXPLANATION

EXPLANATION

Figure 22. Principal components analysis of the weights on the performance metrics across stakeholder agencies. A, loading plot; 

and, B, score plot.
46 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0.3

0.4

0.5

0.6

0.7

-0.1

0

0.1

0.2

Weighted performance

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Long-term strategy

Figure 23. Joint-lead agency weighted performance across 18 metrics forecast for 19 long-term strategies using equal weighting of 

hydrological traces.

of uncertainty in hydrology and the complex set of tradeoffs 

among the resource goals.

One of the important motivations for MCDA was the 

opportunity to include and analyze differing viewpoints from 

stakeholder agencies regarding the importance of the various 

resource goals and their performance metrics. The agencyspecific weightings did affect the ranking of alternatives 

(table 10). For most of the participating agencies, long-term 

strategy D4 had the highest mean-weighted performance 

(with the mean taken over hydrology and sediment traces 

and critical uncertainties). For two of the agencies (UAMPS, 

CREDA), long-term strategy B2 had the highest weighted 

score, and for one organization (GCRG), Alternative G had the 

highest weighted score. For all participating agencies except 

CREDA, long-term strategy D4 outperformed Alternative 

A. For all agencies, long-term strategy C3 performed worse 

than Alternative A, and for most of those agencies, it was 

the worst-performing long-term strategy. Alternative F was 

a polarizing strategy: for three agencies it was the worst-performing strategy, whereas for four others, it performed better 

than the status quo. Most of the differences across stakeholders in the MCDA ranking of the long-term strategies can be 

explained by the first principal component in the weighting 

(fig. 22). For those agencies that placed more weight on the 

performance metrics on the left side of the diagram (fig. 22A), 

Alternative B performed very well because it performs best 

for power generation and capacity; for those agencies that 

placed more weight on the performance metrics on the right 

side of the diagram (fig. 22A), Alternative B was outperformed 

by Alternative D. Across all participating stakeholders, the 

mean values for the long-term strategies in Alternative D were 

greater than those for Alternatives C and E, suggesting that the 

improvements made in crafting Alternative D are robust to the 

weights on the performance metrics.

Across stakeholders, the top-ranked long-term strategy 

was effectively tied with a number of other long-term strategies, against the backdrop of uncertainty induced by the 

hydrology and sediment traces; for example, using the jointlead agency weights on the objectives, Alternative D4 had the 

highest mean performance, but the differences between D4 

and next nine long-term strategies (D1, C1, D2, G, C2, D3, 

E1, E2, and E4) had ranges that included zero (fig. 24, bottom 

panel); that is, based on the results of the MCDA, alternatives C, D, E, and G were nearly indistinguishable using the 

joint-lead agency weights. This pattern held for the weights 

from most stakeholders as well (fig. 25), with between 2 and 

11 long-term strategies effectively tied. For all stakeholders 

except CREDA, long-term strategies D4 and D1 were either 

the best-performing or effectively tied with the best-performing long-term strategy.

The differences among the long-term strategies within 

Alternative D are difficult to discern (fig. 23). The tradeoffs 

associated with using TMFs are marginal (compare D1 to D3). 

Similarly, the tradeoffs associated with “bug flows” do not 
8 Decision Analysis Results 47

Table 10. Weighted performance for 19 long-term strategies using the weights on the performance metrics expressed by 14 participating agencies.[The aggregate scores across performance metrics, weighted using the agency-specific swing-weighting results, are shown for each long-term strategy. For each participating agency, the best-performing longterm strategy is shaded in yellow, the worst-performing strategy is shaded in dark blue, strategies that perform better than Alternative A are shaded in orange, and strategies that perform worse than Alternative A are shaded in turquoise. The 21 hydrological traces were weighted equally. Thirteen additional stakeholder agencies were invited but declined to participate. JL, joint-lead agency average; FWS, U.S. Fish and Wildlife Service; ADWR, Arizona Department of Water Resources; AZGFD, Arizona Game and Fish Department; SRP, Salt River Project; UAMPS, Utah Associated Municipal Power Systems; CREDA, Colorado River Energy Distributors Association; IFFF, International Federation of Fly Fishers; NPCA, National Parks Conservation Association; GCRG, Grand Canyon River Guides]

JL FWS ADWR AZGFD SRP UAMPS CREDA Hopi Hualapai Navajo IFFF NPCA GCRG

A 0.479 0.508 0.483 0.448 0.472 0.448 0.459 0.515 0.530 0.477 0.508 0.450 0.429

B1 0.488 0.504 0.511 0.450 0.493 0.485 0.474 0.512 0.538 0.495 0.511 0.474 0.443B2 0.454 0.434 0.447 0.402 0.491 0.495 0.484 0.457 0.477 0.446 0.504 0.416 0.384C1 0.615 0.539 0.484 0.508 0.458 0.410 0.410 0.574 0.599 0.521 0.544 0.604 0.637C2 0.602 0.515 0.465 0.518 0.426 0.376 0.391 0.570 0.591 0.511 0.549 0.589 0.631C3 0.376 0.433 0.378 0.369 0.418 0.400 0.412 0.411 0.410 0.374 0.445 0.315 0.280C4 0.559 0.507 0.452 0.497 0.441 0.392 0.405 0.532 0.544 0.488 0.529 0.555 0.573

D1 0.619 0.573 0.542 0.540 0.489 0.450 0.436 0.596 0.630 0.559 0.553 0.634 0.648D2 0.607 0.574 0.526 0.535 0.470 0.424 0.414 0.581 0.615 0.547 0.534 0.630 0.642D3 0.599 0.557 0.526 0.540 0.472 0.425 0.428 0.584 0.614 0.550 0.544 0.621 0.637D4 0.628 0.590 0.560 0.553 0.500 0.460 0.445 0.610 0.646 0.574 0.559 0.647 0.662

E1 0.589 0.535 0.522 0.506 0.475 0.447 0.430 0.572 0.607 0.535 0.550 0.587 0.592E2 0.583 0.539 0.533 0.515 0.459 0.428 0.418 0.579 0.616 0.542 0.547 0.588 0.594E3 0.400 0.488 0.482 0.411 0.450 0.445 0.434 0.463 0.483 0.445 0.461 0.373 0.319E4 0.560 0.543 0.532 0.509 0.468 0.436 0.427 0.563 0.597 0.536 0.534 0.575 0.569E5 0.401 0.482 0.476 0.407 0.442 0.438 0.426 0.461 0.482 0.440 0.461 0.370 0.318E6 0.412 0.498 0.492 0.415 0.460 0.462 0.440 0.469 0.491 0.451 0.467 0.382 0.326

F 0.559 0.465 0.396 0.484 0.311 0.269 0.293 0.509 0.536 0.431 0.475 0.535 0.622

G 0.605 0.559 0.478 0.532 0.456 0.385 0.397 0.563 0.588 0.524 0.514 0.634 0.669
48 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0.3

0.4

0.5

0.6

0.7

-0.3

-0.2

-0.1

0

D4 D1 C1 D2 G C2 D3 E1 E2 E4 C4 F B1 A B2 E6 E5 E3 C3

D4 D1 C1 D2 G C2 D3 E1 E2 E4 C4 F B1 A B2 E6 E5 E3 C3

Weighted performance

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Long-term strategy

Performance of each long-term strategy

Pairwise comparison to the best-performing long-term strategy (D4), controlling for trace

Figure 24. Joint-lead agency weighted performance across 18 metrics, sorted by mean performance, forecast for 19 long-term 

strategies using equal weighting of hydrological traces.

produce a clear favorite (compare D1 to D2). Finally, although 

there is the suggestion that the costs of low summer flows outweigh the benefits (compare D1 to D4), the difference is small 

against the backdrop of the variation driven by hydrology. 

The patterns within Alternative D are shown for the joint-lead 

agency weighting (figs. 23–24) but hold for the other stakeholder weightings as well (fig. 25). 

8.3 Expected Value of Information

A second important purpose for the use of formal decision analysis tools was to examine the effect of uncertainty on 

the ranking of the alternatives. Although the effect of uncertainty on the ranking of alternatives based on single performance metrics was generally small (table 6), it is conceivable 

that the weighted performance could be more sensitive to 

uncertainty if the effects of the uncertainties affect the subtle 

balancing of tradeoffs. The analysis of the value of information on the weighted performance, however, revealed that 

the effect of the uncertainties was nearly 0 (table 11). For the 

joint-lead agency weighting and all but two of the participating stakeholders, the best-performing long-term strategy was 

the same across the 16 hypotheses that represented critical 

uncertainty (table 11); for example, long-term strategy D4 

was preferred in the joint-lead agency weighting regardless of 

whether fall HFEs have an effect on RBT recruitment or not, 

regardless of whether TMFs are 10 or 50 percent effective in 

reducing RBT recruitment, and regardless of the relative influence of temperature and RBT on juvenile HBC survival and 

growth. For two stakeholders, there was a very small effect of 

the critical uncertainty: for IFFF, C2 was preferred more than 

D4 in 2 of the 16 hypotheses; and for GCRG, D4 was preferred more than G in 1 of the 16 hypotheses (table 11). Even 

in these two cases, however, the expected value of information 

for resolving the uncertainty represented by the 16 hypotheses 

was less than 0.1 percent.

At first glance, these results may seem puzzling. Many 

intense discussions within AMWG through the years have 

focused on concern that uncertainty about the response of the 

system to management prevents the identification of a best 

management strategy, and differences concerning recommendations for management have been explained as differences 

in interpretation of the scientific evidence. The uncertainties 

examined in this analysis (effect of fall HFEs, effectiveness 

of TMFs, and relative influence of temperature and RBT on 

juvenile HBC) have been central to previous discussions. 

But the results of this analysis do not support the conclusions 

that resolution of this uncertainty is important in choosing 

among the long-term strategies examined. There are several 

explanations for these results. First, there is less uncertainty 

than informal conversations imply. After several decades of 

intense study of the resources affected by the operation of 

Glen Canyon Dam, the AMWG partners have learned a great 

deal. Uncertainty remains, of course, but it is bounded; for 
8 Decision Analysis Results 49

-0.3

-0.2

-0.1

0

D4 D2 D1 G D3 E4 E2 C1 E1 C2 A C4 B1 E6 E3 E5 F B2 C3

-0.2

-0.1

0

B2 B1 A D4 E6 D1 E3 E1 D3 E4 E5 E2 D2 C3 C1 C4 G C2 F

-0.1

0

D4 D1 E1 C2 E2 D3 C1 D2 E4 C4 G B1 A B2 F E6 E3 E5 C3

-0.6

-0.4

-0.2

0

G D4 D1 D2 C1 D3 C2 F E2 E1 C4 E4 B1 A B2 E6 E3 E5 C3

Weighted performance (pairwise difference from best, controlling for trace)

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Long-term strategy

Figure 25. Weighted performance across 18 metrics, for four stakeholder agencies, sorted in descending order of mean performance. 

The four panels show the results for the stakeholders at the outer boundary of the score plot (fig. 22B; U.S. Fish and Wildlife Service, 

Colorado River Energy Distributors Association, International Federation of Fly Fishers, and Grand Canyon River Guides).
50 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan Table 11. Expected value of perfect information for resolving various uncertainties, as a function of the agency-specific weights. [For each of the 16 combinations of hypotheses that describe the critical uncertainties, the best-performing long-term strategies are listed, using the agency-specific weights on the performance metrics. The shading of each cell is associated with the alternative from which the best-performing long-term strategy comes. See table 4 for a description of the critical uncertainties. The expected value of information for resolving critical uncertainty is zero for all the stakeholders except IFFF and GCRG. The expected value of partial information for resolving uncertainty about the combination of sediment and hydrology trace, sediment trace alone, and hydrology trace alone is shown. JL, joint-lead agency average; FWS, U.S. Fish and Wildlife Service; ADWR, Arizona Department of Water Resources; AZGFD, Arizona Game and Fish Department; SRP, Salt River Project; UAMPS, Utah Associated Municipal Power Systems; CREDA, Colorado River Energy Distributors Association; IFFF, International Federation of Fly Fishers; NPCA, National Parks Conservation Association; GCRG, Grand Canyon River Guides; HFE, high-flow experiment; TMF, trout management flow; EV, expected value; EVPI, expected value of per- fect information; EVXI, expected value of partial information] Critical uncertainties Stakeholder Fall HFE TMF Temp trout Federal State Tribe NGO JL FWS ADWR AZGFD SRP UAMPS CREDA Hopi Hualapai Navajo IFFF NPCA GCRG No High f1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No High f2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No High g1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No High g2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No Low f1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No Low f2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No Low g1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

No Low g2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes High f1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes High f2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes High g1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes High g2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes Low f1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

Yes Low f2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 C2 D4

G

Yes Low g1 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4 D4Yes Low g2 D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 C2 D4

G

Average best D4 D4 D4 D4 D4 B2 B2 D4 D4 D4 D4 D4

G

EV 0.628 0.590 0.560 0.553 0.500 0.495 0.484 0.610 0.646 0.574 0.559 0.647 0.669EVPI 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0000 0.0004 0.0000 0.0000EVPI, in percent 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.07 0.00 0.01EVXI (trace), in percent 1.50 1.39 1.54 1.84 0.89 0.10 0.00 1.06 1.32 1.00 1.23 1.87 2.50EVXI (sediment), in percent 0.02 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.13 0.14 0.00EVXI (hydrology), in percent 1.10 0.93 1.27 1.58 0.75 0.09 0.00 0.87 1.12 0.80 0.91 1.53 1.89
8 Decision Analysis Results 51

example, the uncertainty in the effect of temperature and RBT 

on juvenile HBC is large, in the sense that the parameters governing those relations are uncertain by a factor of two (fig. 3A

and 3B); the trout effect on survival could be as small as -0.15 

or as large as -0.4. But, the decades of study of the interactions of RBT and HBC have indicated conclusively that there 

is an effect of RBT on juvenile HBC survival, and there are 

effects of both RBT and temperature on juvenile HBC growth. 

Second, the value of information is strongly affected by the 

set of alternatives considered. It only asks if the uncertainty 

impedes the choice of a best alternative from among the set 

evaluated. It does not ask if there could be another alternative, 

not yet identified, that would be sensitive to the uncertainty. 

So, for example, it was thought that the value of low summer flows would be sensitive to the effect of temperature on 

HBC growth; therefore, C2, for example, might be favored 

compared to C1 if model f1 holds, but C1 might be favored 

compared to C2 if model f2 holds. This presumes, though, 

that low summer flows are triggered in enough years and 

involve flows low enough to lead to substantial changes in the 

temperature in the Colorado River at the confluence with the 

Little Colorado River to produce a demonstrable difference 

between models f1 and f2; this was not the case. But, if there 

were another alternative that could produce large differences 

in temperature (for example, a temperature control device on 

Glen Canyon Dam), these uncertainties may have played a 

greater role. Third, in a multiple-objective decision, the influence of the weights on the objectives might override the effect 

of uncertainties; for example, the costs associated with low 

summer flows (on recreation and hydropower, for example) 

may override the potential benefits (to HBC) enough that the 

effect of the uncertainty does not matter.

8.4 Effects of Climate Change

Although the value of information associated with resolving critical uncertainty was small, there was an indication that 

the uncertainty associated with hydrological input had some 

influence on the ranking of the long-term strategies (table 11). 

For the joint-lead agency weighting, foreknowledge of the 

hydrological and sediment trace could lead to a 1.5 percent 

improvement in the weighted performance across objectives 

(table 11). Such knowledge is not possible because it would 

require accurate prediction of the monthly precipitation in the 

Colorado Basin over the next 20 years. But in demonstrating 

that hydrological input might be important in the selection of 

a long-term strategy, it raises the question about the potential 

influences of climate change.

The 21 reconstructed historical hydrological traces that 

were used in the LTEMP analysis represent possible 20-year 

sequences if the future is like the recent past (1906-2010). 

The 112 hydrological traces generated as part of the Basin 

Study (Bureau of Reclamation, 2012) represent the best current understanding of what might happen because of climate 

change. Mean annual inflow to Lake Powell is quite different 

under these two sets of traces (fig. 26). The median flow 

over the climate-change traces is lower than for the historical traces, although it is within the interquartile range of the 

historical traces. About 30 percent of the potential future distribution of flows, however, is not captured by the distribution 

of the historical traces. The historical 20-year trace with the 

lowest mean inflow has an annual flow of about 8.5 million 

acre-feet; 30 percent of the climate-change traces fall below 

this point, suggesting that the set of historical traces may not 

be representative of future conditions.

To examine part of the potential influence of climate 

change, a reweighting of the historical traces was calculated 

to better match the mean and variance of Lake Powell inflow 

seen in the climate-change traces. The climate-change weighing put about 18 percent of the weight on the single hydrology 

trace with the lowest input to Lake Powell. This reweighting 

of the hydrological traces was then used in all the MCDA 

calculations to examine the effect of those weights on the 

performance of the long-term strategies. For the joint-lead 

agency weights on the performance metrics, the weights on the 

hydrological traces had a small effect on the aggregate performance but did not change the rankings of alternatives (fig. 27). 

A similar pattern was seen for other stakeholder weightings. 

Although this result suggests the ranking of alternatives is 

robust to uncertainty about climate change, it is important to 

7

8

9

10

11

12

13

Historical traces Climate change traces 

Lake Powell inflow, 

in millions of acre-feet per year 

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

EXPLANATION

Figure 26. Mean annual inflow to Lake Powell during the 20-year 

period of the Long-Term Experimental and Management Plan 

for 21 historical hydrological traces and 112 climate-changeinfluenced hydrological traces.
52 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

0.3

0.4

0.5

0.6

0.7

-0.1

0

0.1

0.2

Weighted performance

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

A B1 B2 C1 C2 C3 C4 D1 D2 D3 D4 E1 E2 E3 E4 E5 E6 F G

Maximum value

Minimum value

75th percentile

50th percentile 

(median)

25th percentile

Interquartile

range

Mean

value

EXPLANATION

Long-term strategy

Performance of each long-term strategy

Pairwise comparison to the no-action alternative (A), controlling for trace

Figure 27. Joint-lead agency weighted performance across 18 metrics forecast for 19 long-term strategies using the climate-change 

weighting of hydrological traces.

keep in mind that the traces that were run through the modeling did not include plausible scenarios from the low end of the 

climate-change distribution.

9 Discussion

The analysis in this report is meant to add to the analysis 

in the LTEMP EIS itself. For those resource goals that could 

be evaluated with quantitative performance metrics, the tools 

of decision analysis allowed us to integrate the consequence 

analysis with the value judgments of stakeholders to formally evaluate the tradeoffs among the resource goals and to 

examine the influence of uncertainty on the performance of the 

long-term strategies. It is important to note, however, that not 

all the resource goals of importance could be evaluated quantitatively. The LTEMP EIS contains a narrative analysis of a 

greater number of resource goals than were included in the 

decision analysis, and this greater set of goals may be influential for the stakeholders and joint-lead agencies.

9.1 Evaluation of Alternatives

The alternatives examined in the LTEMP EIS and their 

long-term strategies differed in performance across the metrics 

considered. The full consequence table (table 7) shows strong 

tradeoffs: long-term strategies that perform well on any one 

performance metric may not perform well on others; thus, the 

choice of a preferred alternative will require the Secretary of 

Interior to weigh the importance of the various resource goals. 

The MCDA was designed as one way to provide input to the 

Secretary in her decision, by allowing the joint-lead agencies 

and stakeholders to express their interpretation of the importance of those resource goals and trace that interpretation 

through to the rankings of the alternatives. The best-performing long-term strategy did depend on the weights given to the 

performance metrics. Among the participating agencies, three 

alternatives rose to the top: Alternatives B, D, and G. Alternative B (especially long-term strategy B2) was favored by those 

agencies that emphasized the importance of the hydropower 

resource. Alternative G was the top- or second-ranked alternative for those agencies that emphasized restoration of natural 

processes, like beach building and native vegetation. For the 

remainder of the participating agencies, the analysis ranked 

Alternative D (especially long-term strategy D4) the highest. 

Alternative D was created after preliminary analysis revealed 

the strengths and weaknesses of the other alternatives and 

performs marginally better than the alternatives from which it 

was designed (C and E).

A number of other suggestions arise from the decision 

analysis regarding the long-term strategies. For most of the 

stakeholders, the benefits of HFEs seem to outweigh the costs, 

even in the face of uncertainty about their effects, at least as 
9 Discussion 53

measured by the performance metrics included in this analysis. The other modifications (for example, TMFs, mechanical 

removal, low summer flows, and bug flows) produce equivocal 

results; their inclusion is neither convincingly demonstrated 

nor precluded by the decision analysis.

Several caveats are warranted in interpreting this report. 

First, the performance metrics evaluated represent only a subset of the resource goals that might be influenced by the alternatives. The LTEMP EIS provides a fuller discussion of other 

goals, and the stakeholders and decision makers will need to 

judge if the formal decision analysis represents a sufficient 

degree of completeness. Second, not all stakeholders with an 

interest in the LTEMP EIS chose to participate in the decision analysis; thus, the array of viewpoints represented (for 

example, fig. 22 and table 10) may not capture the full range 

or appropriate distribution of viewpoints. Third, even for those 

stakeholders that participated, it may be difficult to express a 

nuanced set of values through the swing-weighting process. 

The set of MCDA methods used assumes the resource goals 

can be traded against each other in a linear, additive, and independent fashion. In fact, the value a stakeholder places on one 

objective might depend on how well an alternative is also performing on another objective; such dependencies require other 

decision analysis methods that were not used here. Fourth, 

some of the performance metrics may not have directly 

captured the resource goals they were meant to represent. For 

instance, the SLI is a proxy for sediment deposition and retention but not necessarily directly correlated with it. Fifth, some 

of the performance metrics (like hydropower generation) were 

expressed on natural scales that can be readily understood, 

but some of the performance metrics were expressed on proxy 

scales (for example, SLI) or constructed scales (for example, 

fluctuation index) that are difficult to understand; this mixture 

of scales makes the swing-weighting judgments challenging. 

The structuring of objectives and the development of 

appropriate performance metrics are critical steps in MCDA 

and require diligence to meet the assumptions of a linear, 

additive value model (the form of MCDA used in this analysis). One of the concerns is the possible inclusion of means 

objectives, leading to possible double counting in the weighting process; for example, there was considerable discussion 

about the inclusion of the RBT emigration rate (performance 

metric 4)—was it fundamental or merely a means to conserving native fish populations? For some stakeholders, it was 

fundamental as an expression of the conservation of natural 

processes. But for other stakeholders, weight may have been 

placed on this objective as an expression of the effect of trout 

on native fish, thus, leading to possible double counting. 

Another concern is the possible inclusion of preferentially 

dependent objectives; for example, were RBT catch rate 

(performance metric 3) and abundance of high-quality RBT 

(performance metric 5) treated as independent in the swingweighting exercise, or did the participants assume that if you 

had one, you would have the other and so assign weight based 

on simultaneous achievement? The two-stage swing-weighting 

may have helped reduce this concern, but with the large 

number of performance metrics being evaluated, cognitive 

mistakes in the swing-weighting exercise could still have happened. When applying decision theory to real problems, with 

all their subtleties and complexities, it is difficult to achieve 

perfect adherence to the assumptions of the analysis, but diligence toward those assumptions is needed.

All these caveats are fair, and the interpretation of the 

results should account for them. But the analysis also represents an effort to use the best available science to explicitly examine the ranking of alternatives as influenced by an 

analysis of consequences and the relative values placed on the 

resource goals.

9.2 Motivation for Adaptive Management

The purpose of doing the EVPI calculation was to understand the importance of resolving uncertainty (and the relative 

importance of resolving the different sources of uncertainty) 

before beginning experimental design. This puts the focus of 

uncertainty on its value to the decision maker, rather than its 

value as a point of scientific discovery. Although much has 

been written about the many scientific uncertainties about 

how the Colorado River ecosystem responds to management, 

less has been documented about how resolution of those 

uncertainties would explicitly affect and improve management decisions. The value of information analysis identifies 

the following: what long-term management alternative is best 

assuming each of the competing hypotheses, what long-term 

management alternative is best in the face of uncertainty (if 

uncertainty cannot be resolved), how much long-term management can improve through resolution of uncertainty, and 

which components of uncertainty contribute most to the value 

of information. Of course, what constitutes “best” depends 

on how the decision maker values the multiple resource goals 

within their statutory framework, so the analysis includes the 

sensitivity of the value of information to the weights on the 

objectives.

The results presented in this report (including the MCDA 

and EVPI analyses) provide valuable information for the construction of an experimental or adaptive design that is targeted 

to the most importance sources of uncertainty and has the best 

chance of resolving that uncertainty. The design of an adaptive 

strategy needs to consider three important elements: the value 

of the information being sought, the power of any experimental or adaptive design to resolve that uncertainty, and the 

short-term costs to the resource goals of pursuing the reduction of uncertainty. Although there are optimization algorithms 

to derive adaptive strategies taking these factors into account, 

for example, active adaptive stochastic dynamic programming (Williams, 1996), such tools are difficult to use in such a 

complex system.

Interestingly, the EVPI analysis suggests that there is not 

much advantage in an experimental approach because the resolution of the uncertainties articulated is not expected to alter 

the choice of long-term strategy. This suggestion, however, 
54 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

hinges on the assumption that the uncertainties examined are 

a complete set. What about the “unknown unknowns,” the 

uncertainties that we cannot yet articulate, but which we might 

nevertheless discover? We cannot analyze their importance, 

without being able to estimate their effect, and we also cannot 

design a monitoring system to detect them reliably. But, ongoing monitoring of the important resource goals, coupled with 

experimental implementation of the novel elements of any 

chosen alternative, is likely to reveal any surprises that would 

affect the achievement of the resource goals. 

9.3 Decision Analysis and the National 

Environmental Policy Act

A typical analysis under the NEPA looks quite a bit 

like a formal decision analysis: resource goals (objectives) 

are articulated, alternatives are designed and described, and 

alternatives are evaluated against the individual resources of 

concern. These analyses are often qualitative, but a mix of 

qualitative and quantitative tools is also common. What is 

often missing from such analyses is a formal way to synthesize 

the results across the resources of concern. To the extent that 

NEPA is meant as a way to disclose to the public the environmental effects of the considered actions, such a synthesis is 

not needed. What decision analysis adds is a formal, quantitative way for the decision maker to consider and express the 

relative importance of the various resource goals in choosing a 

preferred alternative. In our view, then, decision analysis provides a useful companion to the analyses completed in an EIS.

The use of formal decision analysis in the context of a 

NEPA process was unfamiliar to a number of the Cooperating Agencies and other stakeholders, and several noted their 

discomfort with its use in comments to the joint-lead agencies. 

One-half of those stakeholders invited to participate in the 

swing-weighting exercise chose not to do so (table 9). They 

cited a number of concerns: skepticism that all the resources 

could be evaluated quantitatively, distrust of the methods of 

MCDA generally, a preference for providing input in a format 

they felt comfortable with, and a desire not to be forced to 

make an explicit value judgment on all the resource goals. 

Underlying the concerns was a stated fear that the decision 

analysis methods would be used to make the decision rather 

than just support an understanding of the alternatives. At all 

points, the joint-lead agencies and DOI leadership assured the 

stakeholders that the decision analysis was an effort to explore 

the performance of the proposed alternatives as deeply as possible with quantitative tools but was being used as only one of 

many sources of input in the transparent, deliberative process 

sought under NEPA. The effective use of decision analysis 

tools in the context of future NEPA processes will benefit from 

the insights of additional case studies, and from improved 

communication and training with regard to the theory, practice, justification, and benefits of decision analysis.

10 Summary

This report describes a formal decision analysis led by 

the U.S. Geological Survey, in cooperation with the Bureau 

of Reclamation, National Park Service, and Argonne National 

Laboratory, to support the development and evaluation of 

alternatives for the Glen Canyon Dam Long-Term Experimental and Management Plan. A set of 12 resource goals formed 

the basis of this evaluation, with 18 performance metrics used 

to provide quantitative measures of the resource goals. A total 

of 19 long-term strategies associated with 7 alternatives were 

evaluated against the performance metrics using a series of 

linked simulation models. Stakeholder input was elicited using 

the swing-weighting method, and this input was coupled with 

the quantitative evaluation of the alternatives in a multicriteria 

decision analysis. For 10 out of 13 stakeholder weightings 

presented, Alternative D (in particular, long-term strategy 

D4) outperformed the other alternatives. For the remaining 

stakeholder weightings, Alternatives B and G were the top 

performers. These rankings were robust to the uncertainties 

examined; the value of resolving uncertainty was never greater 

than 7.5 percent for any performance metric, and never greater 

than 2.5 percent for any stakeholder-weighted performance. 

This analysis is not a substitute for the full qualitative analysis 

found in the Long-Term Experimental and Management Plan 

Environmental Impact Statement, but does provide a transparent way to synthesize the analyses that could be quantified.

11 References Cited

Bishop, R.C., Boyle, K.J., Welsh, M.P., Baumgartner, R.M., 

and Rathbun, P.R., 1987, Glen Canyon Dam releases and 

downstream recreation—An analysis of user preferences 

and economic values: Flagstaff, Ariz., Glen Canyon Environmental Studies, Report No. GCES/27/87, 398 p. [Also 

available at http://www.gcmrc.gov/library/reports/cultural/

Recreation/Bishop1987.pdf.]

Bureau of Reclamation, 1995, Final Environmental Impact 

Statement—Operation of Glen Canyon Dam, Colorado 

River Storage Project, Arizona: Washington, Bureau of 

Reclamation, 337 p.

Bureau of Reclamation, 1996, Record of decision—Operation 

of Glen Canyon Dam—Final environmental impact statement: Washington, Bureau of Reclamation, 15 p.

Bureau of Reclamation, 2007, Final Environmental Impact 

Statement—Colorado River interim guidelines for lower 

basin shortages and the coordinated operations for Lake 

Powell and Lake Mead: Boulder City, Nev., Bureau of Reclamation, Upper and Lower Colorado Regions.
11 References Cited 55

Bureau of Reclamation, 2011a, Environmental assessment—Development and implementation of a protocol for 

high-flow experimental releases from Glen Canyon Dam, 

Arizona, 2011 through 2020: Salt Lake City, Utah, Bureau 

of Reclamation, Upper Colorado Region.

Bureau of Reclamation, 2011b, Environmental assessment—

Non-native fish control downstream from Glen Canyon 

Dam: Salt Lake City, Utah, Bureau of Reclamation, Upper 

Colorado Region.

Bureau of Reclamation, 2012, Colorado River Basin water 

supply and demand study—Study report: Washington, 

Bureau of Reclamation, 85 p.

Bureau of Reclamation and National Park Service, 2014, 

Alternatives for consideration in the Glen Canyon Dam 

Long-term Experimental and Management Plan Environmental Impact Statement: Salt Lake City, Utah, Bureau of 

Reclamation, Upper Colorado Region; Denver, Colorado, 

National Park Service, Intermountain Region.

Bureau of Reclamation and National Park Service, 2015, 

Draft Environmental Impact Statement—Glen Canyon 

Dam Long-term Experimental and Management Plan: Salt 

Lake City, Utah, Bureau of Reclamation, Upper Colorado 

Region; Denver, Colorado, National Park Service, Intermountain Region.

Castle, Anne, 2010, Direction to desired future conditions 

ad hoc group: Washington, D.C., U.S. Department of the 

Interior, 3 p.

Cole, T.M., and Wells, S.A., 2006, CE–QUAL–W2—A twodimensional, laterally averaged, hydrodynamic and water 

quality model (ver. 3.5): Vicksburg, Miss., U.S. Army 

Engineering and Research Development Center, Instruction 

Report EL–06–1. 

Edwards, W., 1971, Social utilities, in Lesser, Arthur, ed., 

Decision and risk analysis—Powerful new tools for management: Hoboken, N.J., Engineering Economist for American Society of Engineering Education, Proceedings of the 

Triennial Symposium sponsored by Engineering Economy 

Divisions, American Society for Engineering Education, 

and American Institute of Industrial Engineers, 6th, U.S. 

Naval Academy, Annapolis, Md., June 19–20, 1971, p. 

119–129.

Felli, J.C., and Hazen, G.B., 1998, Sensitivity analysis and the 

expected value of perfect information: Medical Decision 

Making, v. 18, no. 1, p. 95–109.

Figueira, José, Greco, Salvatore, and Ehrogott, M., eds., 2005, 

Multiple criteria decision analysis—State of the art surveys: 

New York, Springer, v. 78, no. 1, 1,045 p.

Flug, Marshall, Seitz, H.L.H., and Scott, J.F., 2000, Multicriteria decision analysis applied to Glen Canyon Dam: Journal 

of Water Resources Planning and Management, v. 126, no. 

5, p. 270–276. [Also available at http://dx.doi.org/10.1061/

(ASCE)0733-9496(2000)126:5(270).]

Goodwin, Paul, and Wright, George, 2004, Decision analysis 

for management judgment (3rd ed.): West Sussex, United 

Kingdom, John Wiley & Sons, 477 p.

Gregory, Robin, Failing, Lee, Harstone, Michael, Long, Graham, McDaniels, Tim, and Ohlson, Dan, 2012, Structured 

decision making—A practical guide for environmental management choices: West Sussex, United Kingdom, WileyBlackwell, 312 p.

Gregory, Robin, Keeney, Ralph, and Winterfeldt, Detlof von, 

1992, Adapting the environmental impact statement process 

to inform decision makers: Journal of Policy Analysis and 

Management, v. 11, no. 1, p. 58–75. [Also available at 

http://dx.doi.org/10.2307/3325132.]

Herath, Gamini, and Prato, Tony, eds., 2006, Using multicriteria decision analysis in natural resource management: 

Hampshire, United Kingdom, Ashgate Publishing Limited.

Howard, R.A., 1966, Information value theory: IEEE Transactions on Systems Science and Cybernetics, v. 2, no. 1, 

p. 22–26. [Also available at http://dx.doi.org/10.1109/

TSSC.1966.300074.]

Johnson, F.A., Moore, C.T., Kendall, W.L., Dubovsky, J.A., 

Caithamer, D.F., Kelley, J.R., Jr., and Williams, B.K., 1997, 

Uncertainty and the management of mallard harvests: Journal of Wildlife Management, v. 61, no. 1, p. 202–216. [Also 

available at http://dx.doi.org/10.2307/3802429.]

Keeney, R.L., 1996, Value-focused thinking—A path to creative decisionmaking: Cambridge, Mass., Harvard University Press, 432 p.

Kiker, G.A., Bridges, T.S., Varghese, A., Seager, T.P., and 

Linkov, Igor, 2005, Application of multicriteria decision 

analysis in environmental decision making: Integrated 

Environmental Assessment and Management, v. 1, no. 2, 

p. 95–108. [Also available at http://dx.doi.org/10.1897/

IEAM_2004a-015.1.]

Kimbrough, Sue, Vallero, Dan, Shores, Richard, Vette, Alan, 

Black, Kevin, and Martinez, Victoria, 2008, Multi-criteria 

decision analysis for the selection of a near road ambient 

air monitoring site for the measurement of mobile source 

air toxics: Transportation Research Part D—Transport and 

Environment, v. 13, no. 8, p. 505–515. [Also available at 

http://dx.doi.org/10.1016/j.trd.2008.09.009.]
56 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Korman, Josh, Kaplinski, Matthew, and Melis, T.S., 2011, 

Effects of fluctuating flows and a controlled flood on incubation success and early survival rates and growth of age-0 

rainbow trout in a large regulated river: Transactions of the 

American Fisheries Society, v. 140, no. 2, p. 487–505. [Also 

available at http://dx.doi.org/10.1080/00028487.2011.5720

15.]

Korman, Josh, Martell, S.J.D., Walters, C.J., Makinster, 

A.S., Coggins, L.G., Yard, M.D., and Persons, W.R., 2012, 

Estimating recruitment dynamics and movement of rainbow trout (Oncorhynchus mykiss) in the Colorado River 

in Grand Canyon using an integrated assessment model: 

Canadian Journal of Fisheries and Aquatic Sciences, v. 

69, no. 11, p. 1827–1849. [Also available at http://dx.doi.

org/10.1139/f2012-097.]

Kulkarni, R.B., Burns, R.L., Wright, John, Apper, Bryan, 

Baily, T.O., and Noack, S.T., 1993, Decision analysis 

of alternative highway alignments: Journal of Transportation Engineering, v. 119, no. 3, p. 317–332. [Also 

available at http://dx.doi.org/10.1061/(ASCE)0733-

947X(1993)119:3(317).]

Linkov, Igor, Satterstrom, F.K., Kiker, G.A., Batchelor, C.J., 

Bridges, T.S., and Ferguson, E.A., 2006a, From comparative risk assessment to multi-criteria decision analysis 

and adaptive management: Recent developments and 

applications: Environment International, v. 32, no. 8, p. 

1072–1093. [Also available at http://dx.doi.org/10.1016/j.

envint.2006.06.013.]

Linkov, Igor, Satterstrom, F.K., Kiker, G.A., Bridges, T.S., 

Benjamin, S.L., and Belluck, D.A., 2006b, From optimization to adaptation: shifting paradigms in environmental 

management and their application to remedial decisions: 

Integrated Environmental Assessment and Management, 

v. 2, no. 1, p. 92–98. [Also available at http://dx.doi.

org/10.1897/1551-3793(2006)2[92:FOTASP]2.0.CO;2.]

Linkov, Igor, Satterstrom, F.K., Kiker, G.A., Seager, T.P., 

Bridges, T.S., Gardner, K.H., Rogers, S.H., Belluck, D.A., 

and Meyer, A., 2006c, Multicriteria decision analysis: 

a comprehensive decision approach for management of 

contaminated sediments: Risk Analysis, v. 26, no. 1, p. 

61–78. [Also available at http://dx.doi.org/10.1111/j.1539-

6924.2006.00713.x.]

Marcot, B.G., Thompson, M.P., Runge, M.C., Thompson, F.R., 

McNulty, Steven, Cleaves, David, Tomosy, Monica, Fisher, 

L.A., and Bliss, Andrew, 2012, Recent advances in applying 

decision science to managing national forests: Forest Ecology and Management, v. 285, p. 123–132. [Also available at 

http://dx.doi.org/10.1016/j.foreco.2012.08.024.]

Martin, T.G., Burgman, M.A., Fidler, Fiona, Kuhnert, P.M., 

Low-Choy, Samantha, McBride, Marissa, and Mengersen, 

Kerrie, 2012, Eliciting expert knowledge in conservation science: Conservation Biology, v. 26, no. 1, p. 29–38. 

[Also available at http://dx.doi.org/10.1111/j.1523-

1739.2011.01806.x.]

Maurer, E.P., Brekke, Levi, Pruitt, Tom, and Duffy, P.B., 

2007, Fine-resolution climate projections enhance regional 

climate change impact studies: Eos, Transactions, American 

Geophysical Union, v. 88, no. 47, p. 504. [Also available at 

http://dx.doi.org/10.1029/2007EO470006.]

McFadden, J.E., Hiller, T.L., and Tyre, A.J., 2011, Evaluating 

the efficacy of adaptive management approaches: Is there 

a formula for success?: Journal of Environmental Management, v. 92, no. 5, p. 1354-1359.

National Oceanic and Atmospheric Administration, 1995a, 

Environmental assessment for the structural restoration of 

the M/V Alex Owen Maitland grounding site, Key Largo 

National Marine Sanctuary, Florida: Cambridge, Mass., 

Industrial Economics, Inc.

National Oceanic and Atmospheric Administration, 1995b, 

Environmental assessment for the structural restoration of 

the M/V Elpis grounding site, Key Largo National Marine 

Sanctuary, Florida: Cambridge, Mass., Industrial Economics, Inc.

Nobrega, R.A.A., O’Hara, C.G., Sadasivuni, R., and Dumas, 

J., 2009, Bridging decision-making process and environmental needs in corridor planning: Management of 

Environmental Quality—An International Journal, v. 

20, no. 6, p. 622–637. [Also available at http://dx.doi.

org/10.1108/14777830910990744.]

Ouarda, T.B.M.J., Labadie, J.W., and Fontane, D.G., 1997, 

Indexed sequential hydrologic modeling for hydropower 

capacity estimation: Journal of the American Water 

Resources Association, v. 33, no. 6, p. 1337–1349. [Also 

available at http://dx.doi.org/10.1111/j.1752-1688.1997.

tb03557.x.]

Prato, Tony, 1999, Multiple attribute decision analysis for 

ecosystem management: Ecological Economics, v. 30, no. 

2, p. 207–222. [Also available at http://dx.doi.org/10.1016/

S0921-8009(99)00002-6.]

Raiffa, Howard, and Schlaifer, R.O., 1961, Applied statistical decision theory: Cambridge, Mass., Graduate School of 

Business Administration, Harvard University.

Ralston, B.E., Starfield, A.M., Black, R.S., and Van 

Lonkhuyzen, R.A., 2014, State-and-transition prototype 

model of riparian vegetation downstream of Glen Canyon Dam, Arizona: U.S. Geological Survey Open-File 

Report 2014–1095, 30 p. [Also available at http://dx.doi.

org/10.3133/ofr20141095.]
11 References Cited 57

Runge, M.C., 2011, An introduction to adaptive management 

for threatened and endangered species: Journal of Fish and 

Wildlife Management, v. 2, no. 2, p. 220–233. [Also available at http://dx.doi.org/10.3996/082011-JFWM-045.]

Runge, M.C., Bean, Ellen, Smith, D.R., and Kokos, Sonja, 

2011a, Non-native fish control below Glen Canyon Dam—

Report from a structured decision-making project: U.S. 

Geological Survey Open-File Report 2011–1012, 74 p. 

[Also available at http://pubs.usgs.gov/of/2011/1012/.]

Runge, M.C., Converse, S.J., and Lyons, J.E., 2011b, Which 

uncertainty? Using expert elicitation and expected value 

of information to design an adaptive program: Biological 

Conservation, v. 144, no. 4, p. 1214–1223. [Also available 

at http://dx.doi.org/10.1016/j.biocon.2010.12.020.]

Russell, Kendra, and Huang, Jianchun, 2010, Sediment 

analysis for Glen Canyon Dam environmental assessment—

Upper Colorado Region, AZ: Denver, Colo., Bureau of 

Reclamation, Denver Technical Services Center, 36 p.

Sheehy, Daniel, and Vik, Susan, 2002, Applying decision 

analysis methods to NRDA restoration planning: Proceedings of the Annual Ecosystem Restoration and Creation 

Conference, 29th, Hillsborough College, Plant City, Florida, 

May 2002, p. 113–127.

Speirs-Bridge, Andrew, Fidler, Fiona, McBride, M.F., Flander, Louisa, Cumming, Geoff, and Burgman, M.A., 2010, 

Reducing overconfidence in the interval judgments of 

experts: Risk Analysis, v. 30, no. 3, p. 512–523. [Also available at http://dx.doi.org/10.1111/j.1539-6924.2009.01337.x.]

Stich, Bethany, and Holland, J.H., 2011, Using a multiple 

criteria decision-making model to streamline and enhance 

NEPA and public participation processes: Public Works 

Management & Policy, v. 16, no. 1, p. 59–89. [Also available at http://dx.doi.org/10.1177/1087724X10390227.]

Turner, N.J., Gregory, Robin, Brooks, Cheryl, Failing, Lee, 

and Satterfield, Terre, 2008, From invisibility to transparency—Identifying the implications: Ecology and Society, v. 

13, no. 2, art. 7, 14 p.

U.S. Energy Information Administration, 2014, Assumptions to the annual energy outlook 2014: Washington, U.S. 

Department of Energy, DOE/EIA–0554(2014).

Valdez, R.A., and Speas, D.W., 2007, A risk assessment model 

to evaluate risks and benefits to aquatic resources from a 

selective withdrawal structure on Glen Canyon Dam: Salt 

Lake City, Utah, Bureau of Reclamation, 36 p.

Walters, C.J., 1986, Adaptive management of renewable 

resources: New York, MacMillan Publishing Company, 

374 p.

Walters, C.J., and Holling, C.S., 1990, Large-scale management experiments and learning by doing: Ecology, v. 

71, no. 6, p. 2060–2068. [Also available at http://dx.doi.

org/10.2307/1938620.]

Winterfeldt, D. von, and Edwards, W., 1986, Decision analysis 

and behavioral research: Cambridge, United Kingdom, 

Cambridge University Press 624 p.

Williams, B.K., 1996, Adaptive optimization of renewable 

natural resources—Solution algorithms and a computer 

program: Ecological Modelling, v. 93, nos. 1–3, p. 101–

111. [Also available at http://dx.doi.org/10.1016/0304-

3800(95)00217-0.]

Wright, S.A., Anderson, C.R., and Voichick, Nicholas, 2009, A 

simplified water temperature model for the Colorado River 

below Glen Canyon Dam: River Research and Applications, 

v. 25, no. 6, p. 675–686. [Also available at http://dx.doi.

org/10.1002/rra.1179.]

Wright, S.A., Topping, D.J., Rubin, D.M., and Melis, T.S., 

2010, An approach for modeling sediment budgets in 

supply-limited rivers: Water Resources Research, v. 

46, no. 10, p. 1–18. [Also available at http://dx.doi.

org/10.1029/2009WR008600.]

Yackulic, C.B., Yard, M.D., Korman, Josh, and Haverbeke, 

D.R., 2014, A quantitative life history of endangered 

humpback chub that spawn in the Little Colorado River—

Variation in movement, growth, and survival: Ecology and 

Evolution, v. 4, no. 7, p. 1006–1018. [Also available at 

http://dx.doi.org/10.1002/ece3.990.]

Zagona, E.A., Fulp, T.J., Shane, R., Magee, T., and Goranflo, 

H.M., 2001, RiverWare—A generalized tool for complex 

reservoir system modeling: Journal of the American Water 

Resources Association, v. 37, no. 4, p. 913–929. [Also 

available at http://dx.doi.org/10.1111/j.1752-1688.2001.

tb05522.x.]

12 Appendix 1. Disclaimers 59

12 Appendix 1. Disclaimers

In preparatory discussions with the Adaptive Management Working Group (AMWG) stakeholders and Cooperating 

Agencies, the joint-lead agencies and U.S. Department of the 

Interior (DOI) wanted to be clear about the role of the decision analysis in the Long-Term Experimental and Management Plan (LTEMP) Environmental Impact Statement (EIS) 

process. The following disclaimer is to be included in the EIS 

when describing the use of formal decision analysis. (Note that 

during the discussions with the stakeholders, the term “structured decision analysis” was used as shorthand for the process 

being used. In this report, we follow the common practice 

in the literature and simply refer to the process as “decision 

analysis.” The disclaimers below are reproduced verbatim, and 

so retain the use of the term “structured decision analysis”). A 

number of the participating stakeholders added disclaimers of 

their own when they submitted their swing-weighing results; 

these are also included below.

12.1 Standard Disclaimer

In an effort to provide multiple opportunities for interested stakeholders to provide input in the LTEMP process, 

the National Park Service (NPS) and Bureau of Reclamation 

(hereinafter referred to as “Reclamation”) have decided to 

incorporate facilitated structured decision analysis (SDA) into 

the LTEMP EIS process. The SDA has been used previously 

for one aspect of the Glen Canyon Dam Adaptive Management Program (GCDAMP), the “Environmental Assessment 

for Non-Native Fish Control below Glen Canyon Dam.” 

The use of SDA in the LTEMP process is not required by 

the National Environmental Policy Act (NEPA), nor does it 

replace the NEPA impact analysis. 

Participation in the SDA process is a voluntary opportunity for stakeholder input. The NPS and Reclamation recognize that any input provided during the SDA effort does not 

replace the need and opportunities for formal public comment 

that are required steps in the NEPA process. Such formal 

comments on the Draft and Final EISs will be regarded as the 

formal and official positions of any commenting entity. 

The use of SDA is an effort to cast a complex decision 

setting into a transparent, comprehensive but compressed 

form to help the decision makers and stakeholders see the 

essential elements; it may not, however, capture all nuances 

perfectly. The NPS and Reclamation recognize that the metrics 

for identified resource goals in the SDA do not necessarily 

reflect consensus or agreement among participants; moreover, 

the swing-weighting values to be applied to the metrics for 

identified resource goals may not reflect the broader policies or the importance of issues for any participant or agency. 

Stakeholders have had and will have formal opportunities 

to express their values through standard steps in the NEPA 

process, especially the submission of alternatives and public 

comments; for some stakeholders, these steps may allow them 

more flexibility to express their values in a familiar form than 

the decision analysis. 

For these and other reasons, neither the co-leads nor the 

swing-weighting participants are bound by any outcomes or 

results of the SDA process. The NPS and Reclamation will use 

the results of the SDA process as one of multiple sources of 

information to inform the NEPA process, but the SDA process 

itself will not be used in isolation from other input to select 

the preferred alternative; rather, the NPS and Reclamation will 

choose a preferred alternative based on their statutory missions 

and responsibilities, giving consideration to legal, economic, 

environmental, technical, and other factors, as well as formal 

public input.

12.2 Arizona Department of Water Resources 

Disclaimer

 The Arizona Department of Water Resources (ADWR) 

understands that the use of SDA in the LTEMP process is 

not required by NEPA, nor does it replace the NEPA impact 

analysis; furthermore, the ADWR recognizes that any input 

provided during the SDA effort does not replace the need and 

opportunity for formal public comment that are required steps 

in the NEPA process. Such formal comments on the Draft and 

Final EISs, in addition to the attached swing-weight exercise 

input, will be regarded as the formal and official positions of 

the ADWR.

The ADWR further recognizes that the swing-weighting 

values that have been applied to the metrics for identified 

resource goals do not reflect the broader policies or the importance of issues for the State of Arizona. The importance and 

priority of the values of ADWR are more accurately reflected 

within the “Resource Targeted Condition-Dependent” alternative, being necessarily incorporated during alternative creation. For these and other reasons, the ADWR is not bound by 

any outcomes or results of the SDA process.

Moreover, ADWR understands that the NPS and Reclamation will not solely rely on the results of the SDA process 

to select a preferred alternative; rather, the NPS and Reclamation will choose a preferred alternative based on their statutory 

missions and responsibilities, giving consideration to legal, 

economic, environmental, technical formal public input and 

other factors.
60 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

12.3 Arizona Game and Fish Department 

Disclaimer

The Arizona Game and Fish Department (AGFD) 

understands that the use of SDA in the LTEMP process is not 

required by NEPA, nor does it replace the NEPA impact analysis; furthermore, the AGFD recognizes that any input provided 

during the SDA effort does not replace the need and opportunity for formal public comment that are required steps in the 

NEPA process. Such formal comments on the Draft and Final 

EISs will be regarded as the formal and official positions of 

the AGFD, in addition to the attached swing-weight exercise.

The AGFD further submits that the swing-weighting 

values that we have provided for the metrics for identified 

resource goals do not reflect the broader policies or the importance of issues for the State of Arizona as a whole; rather, they 

represent AGFD values as they relate to our specific statutory 

authority and mission. For these and other reasons, the AGFD 

is not bound by any outcomes or results of the SDA process.

The AGFD understands and expects that the NPS and 

Reclamation will not solely rely on the results of the SDA 

process to select a preferred alternative; rather, the NPS and 

Reclamation will choose a preferred alternative based on their 

statutory missions and responsibilities, giving consideration to 

legal, economic, environmental, technical, formal public input 

and other factors.

12.4 Salt River Project Explanatory Letter

The Salt River Project (SRP) submits the attached swingweighting exercise response under the following conditions: 

• that this letter entitled “SRP’s SDA Evaluation” remain 

attached to the SRP swing-weighting response table, 

and 

• that the information submitted herein is not authorized 

for use, or attribution, beyond the purposes of this specific exercise in the current LTEMP EIS process. 

1. Hydropower (100).—The SRP believes that the hydropower resource goal is the only one that represents both 

societal and environmental benefits. Greater quantities 

of hydropower provide cost effective service to people 

who value electricity to support many aspects of their 

lives. Greater quantities of this renewable, carbon-free 

resource also serve to avoid emissions and water use by 

other electric generating facilities. In the initial round of 

swing weighting, energy and capacity are valued equally. 

1. The energy metric does not appear to function in a 

consistent, intuitive manner; for example, a more 

flexible alternative with more energy production in 

higher-value months sometimes performs worse than 

a less flexible alternative with less energy production 

in higher-value months. 

2. While the capacity metric currently looks at July, 

SRP’s peak demand requirement most often occurs 

in August, and our understanding from the Western 

Area Power Administration (WAPA) is that August is 

a better choice for peak planning purposes from their 

perspective as well. The SRP believes that utilizing 

August for peak demand planning would produce 

more credible capacity metric results. 

2. Humpback Chub (75).—The SRP recognizes the environmental significance of the endangered humpback 

chub (HBC) (Gila cypha). The SRP supports the assembly of scientific evidence that establishes clear cause and 

effect relationships between flow regimes and a recovery of this endangered species. The “Number of Adult 

Humpback Chub” metric looks specifically at chub 

population, so the SRP places more emphasis on this 

metric than the “Temperature Suitability Index” metric. 

3. Archaeological and Cultural Resources (60).—The 

Archeological and Cultural Resources goal represents an 

important societal benefit that the SRP supports; furthermore, SRP’s perception is that, as presently crafted, the 

Archeological and Cultural Resources goal contains a 

more accurate representation of tribal interests than the 

“Tribal” resource goal. The SRP supports alternatives 

that science shows will preserve these resources. 

1. At the April Workshop, it was mentioned that the 

Time-off-river index was the issue that could lead to 

degradation of archeological and cultural resources; 

thus, this metric was scored highest. 

2. There has been some discussion about the wind 

transport of sediment index (WTSI). Specifically, 

some LTEMP representatives indicated that they 

would not want architectural and cultural sites buried 

by sand. This metric was, therefore, scored less 

highly.

3. Based on new data from high-flow experiments 

(HFEs), it is unclear how HFEs and sediment affect 

downstream architectural and cultural sites. Because 

there are some winners and some losers, it is uncertain how the single “GLCA flow effects of historic 

properties (Ninemile)” metric can be representative 

of all sites; for example, science shows that HFEs 

will not increase sand bars before river mile (RM) 

30 since there is no silt inflow. Even below RM 30, 

some sandbars cannot be built up to previous levels 

because even the maximum output of GCD cannot 

push sand high enough; thus, the SRP is not weighing this option as heavily as the others. 

4. Trout Fishery (40).—The SRP recognizes the recreational value of the trout fishery and weighs the benefits 

against the environmental dangers of trout migration on 

downstream resources in the Grand Canyon ecosystem. 
12 Appendix 1. Disclaimers 61

Since science appears to show that trout stay relatively 

close to Lees Ferry, SRP would not weigh “Lees Ferry 

trout emigration estimate” metric as heavily as if trout 

emigrated more through the canyon. As long as trout are 

not affecting downstream resources, the other metrics 

are also important to the trout fishery; thus, the “Angler 

Catch Rate Index” and “Number of trout > 16 inches 

total length” sublevel metrics are scored lower than the 

emigration metric but are still valued. 

5. Recreation (20).—The SRP recognizes the societal value 

of recreation and believes Glen and Grand Canyons 

should be enjoyed by anyone who visits. While most 

visitors who visit the Grand Canyon do not raft down 

the canyon, the subset of people who do so should be 

considered. 

1. The “GRCA camping area index” is an important 

metric because people need safe areas to camp for 

the night. This is the highest rated metric. 

2. The fluctuation index is less highly rated because it 

does not take the time of day of the fluctuation into 

account. 

3. The GLCA rafting use index does not appear to have 

enough swing from low to high values to be rated 

highly. The swing of 600 people per year that could 

not take a trip down the river is not the same scale 

of an impact as a 1,000 fish population increase for 

HBC or a $9 million per year increase in hydropower energy value. 

6. Riparian Vegetation (20).—The SRP recognizes the 

effect of riparian vegetation on the ecosystem of the 

Grand Canyon and believes it is has valuable environmental impacts. The aggregation of all metrics into 

this single metric hides some of the complexities; for 

example, CDAS and SASF perform worst for native/

nonnative ratio but best for arrowweed. These are not 

able to be considered separately, which makes specific 

value judgments difficult. 

7. Sediment (0).—The SRP observes that the effects of 

sediment are accounted for in the HBC, Archaeological 

& Cultural, Recreation, and Riparian Vegetation metrics. 

The SRP’s perception is that sediment is a means to an 

end. We believe that a separate sediment metric does not 

make sense in the same manner that a separate temperature or HFE metric does not make sense. 

8. Tribal (0).—The SRP believes that tribal interests should 

be recognized as a valued dimension of the LTEMP EIS 

process; however, as presented assembled and stated, 

the Tribal Resource goal does not appear to effectively 

represent tribal interests. The results for “Change in 

Marsh Vegetation” were questioned at the April Workshop and were not changed before this swing weighting. 

Trout should only be removed when they are endangering HBC, and the decision of how to remove them would 

preferably comply with tribal interests.

12.5 Utah Associated Municipal Power 

Systems Explanatory Letter

Utah Associated Municipal Power Systems (UAMPS) 

representing over 30 UAMPS members contracting for 

power generating output of Glen Canyon Dam is pleased to 

submit our response to the swing-weight exercise related to 

the LTEMP EIS process. The UAMPS is both an LTEMP 

EIS Cooperating Agency, a member of Glen Canyon Dam 

Adaptive Management Workgroup, and has been involved 

with most activities related to the operation of Glen Canyon 

Dam since environmental studies began in the mid-1980s. 

The UAMPS is grateful for the opportunity to participate in 

the EIS process as a member of the AMWG and as a Cooperating Agency contributor. We are appreciative of the great 

amount of work performed by both Federal government 

employees and contractors performing studies and assembling 

vast amounts of reporting documents, as well of those interested parties who have been following this LTEMP process.

We continue to be concerned with the valuation of the 

electric generation from Glen Canyon Dam and the reduction of power capacity and see this loss will be required to 

be replaced by resources using fossil fuels emitting carbon 

dioxide, which create other environmental problems and 

considerations. Thus far, we have not seen any expression of 

impacts of capacity switching except for some costs of capacity data for natural gas generation. In that regard, we wish to 

support the comment submitted by the Colorado River Energy 

Distributors Association (CREDA) that the cost estimates 

for natural gas combined-cycle generation in today’s costs 

included in the performance metrics are way undervalued and 

should be corrected. We also hope additional cost analysis will 

eventually be included to address the lost power generating 

flexibility at Glen Canyon Dam needed for the western power 

grid. The flexibility provided by this hydropower is not free 

and will need to be made-up by some other power generating 

resource paid for by all utilities connected to the WECC power 

grid. Thus far, this loss of generating flexibly and the environmental impacts resulting from the shift to fossil fuels has only 

been casually addressed. We see the environmental objectives 

in this EIS seem to have priority over the far reaching climate 

change impacts that the electric industry in the west faces 

more and more as it struggles meet electric demands.

12.6 Colorado River Energy Distributors 

Association Submittal Letter

This letter [dated April 18, 2014] is an integral part of 

CREDA’s SDA swing-weighting exercise submittal. CREDA 

members are all non-profit wholesale customers of the Salt 
62 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Lake City Area Integrated Projects (SLCA/IP) firm electric 

and transmission resources, of which Glen Canyon Dams 

is the largest generating facility. Although all members are 

non-profit entities, CREDA membership is diverse. Therefore, responses provided on the attached exercise cannot be 

ascribed to any individual CREDA member, as individual 

members did not have the ability to participate in the 5 days of 

workshop nor complete the exercise individually. Further, the 

responses are submitted with the understanding that the results 

will be reported with participant attribution and included in 

the draft/final Long-Term Experimental and Management Plan 

(LTEMP) Environmental Impact Statement (EIS). Further, 

it is not clear to CREDA in what other forums this submittal 

is intended for use or attribution. Given those assumptions, 

CREDA submits its SDA swing-weighting response under the 

conditions that a) this letter shall remain attached to the SDA 

swing-weighting response table, and 2) the information submitted herein is not authorized for use or attribution beyond 

purposes of this specific exercise in the LTEMP EIS processes 

and documents.

SDA AND LTEMP: Having participate fully in the 

August and March-April SDA workshops, it has become 

clear to CREDA this this process may not lend itself readily to a system and resources as complex and interdependent as we see in the resources of interest in the LTEMP. As 

the performance metrics and models have been developed 

and evolved, CREDA still has concerns about some of the 

underlying Resource Goals, and their inconsistency with the 

AMWG-approved Desired Future Conditions. Although many 

comments and concerns have been expressed on some of the 

Goals in various forums and through various medium, the 

Goals have remained unchanged since LTEMP scoping, As we 

were advised on April 1, if a participant still has concerns with 

a Resource Goal, or if the participant believes the elements 

of a metric are contradictory, inappropriately “linked” to a 

Resource Goal, or a create a potential legal or policy conflict, 

the participant should value the metric with a very low or 0 

value. We appreciate the time and effort involved in developing and facilitating the SDA process, but suggest that it may be 

more applicable and useful for either determining stakeholder 

values of individual attributes of a resource, or analyzing a 

reduced number of alternatives (particularly when many of 

the LTEMP alternatives were “split up” for this exercise). We 

commend the stakeholders who devoted a significant amount 

of time and effort to the August 2013 and this year’s process.

SDA, HYDROPOWER ANALYSIS AND LTEMP: 

As you are aware, CREDA is participating in this process as 

a member of the Glen Canyon Dam Adaptive Management 

Work Group (AMWG), representing “contractors for the 

purchase of Federal power produced at Glen Canyon Dam”. 

As such representative, CREDA’s submittal of the Balanced 

Resource Alternative represents the value we ascribe to the 

hydropower resource, notwithstanding we have some outstanding questions about how this alternative was ultimately 

modeled for the SDA process. Participants in the April workshop were advised that the hydropower performance metric 

was incomplete, and that a key component of the metric, 

capacity, had to be developed by Argonne in an inordinately 

short period of time (April 1–8), further, that the “ratepayer 

analysis” portion of the hydropower analysis was noted as 

“still under discussion” in the Performance Metrics document 

provided to the workshop participants. CREDA offers detailed 

comment herein on the energy, capacity and ratepayer analysis 

components of the hydropower resource, as we were invited to 

do as soon as possible after the April 1 workshop. We understand and expect that the electric resource flexibility component of the metric will be assessed as part of the draft/full EIS 

process, and that air quality impacts, such as carbon offsets 

will also be a part of the overall hydropower analysis. We offer 

the following comments and suggestions for use in assessing 

hydropower resources going forward in the LTEMP process.

ENERGY: The information provided to the SDA 

workshop participants on April 1 has results that are unintuitive. For example, one can compare RTCD4 with BR1. 

Both have Fall HFEs. BR1 has an average output of 14,500 

cfs during July and August, while RTCD4 has an average 

output of 12,000 cfs and 11,000 cfs respectively. Furthermore, BR1 has an average fluctuation range that is 140% 

of MLFF, while RTCD4 has an average fluctuation range 

that is 114% of MLFF. Based on common knowledge, BR1 

should perform better for energy than RTCD4. However, 

based on modeling results, RTCD4 performed better. This 

example, along with others discussed during the April 4 

meeting, call the energy results into question. We urge the 

LTEMP co-leads to utilize the expertise of the cooperating 

agency utility experts in energy analysis. We would also 

like to see more detailed energy results, as was agreed in 

the April 4 conference call.

CAPACITY: The information provided to the 

SDA workshop participants on April 10 indicates that 

the capacity metric uses July as its peak month. CREDA 

reaffirms its suggestion on March 31 that August be used 

as the peak month to reflect actual utility peak demand 

experience, both past and as projected for the period of the 

LTEMP EIS. Also included in the April 10 information is a 

capacity value of $65,000/MW-yr to be used in the swingweighting process, based upon the following factors:

• 620-MW natural gas combined cycle plant 

• Capital cost of $917/kW 

• Fixed operation and maintenance cost of $13.17/kW 

• 30-year lifetime 

• Discount rate of 3.8%

CREDA believes, based on discussion with its 

utility members who are also cooperating agencies in the 

LTEMP, that a more accurate estimate of the capacity value 

for a large, natural gas combined cycle facility is in the 

range of $82,000/MW-yr–$132,000/MW-yr.

As a consulting service to the electric utility industry, the Electric Power Research Institute (EPRI) regularly 
12 Appendix 1. Disclaimers 63

prepares estimates of the costs associated with developing 

various types of energy resources, including gas-fired combined cycle facilities. EPRI’s publicly available Generation 

Technology Options Report (published 2/19/2013) can 

be accessed at the linked site. EPRI information provided 

below can be found in Table 1-2 of their report.

Having worked with EPRI for a number of years 

with respect to resource characteristics and cost information for various electric resource technologies, a CREDA 

member’s experience suggests that EPRI’s valuations often 

do not fully account for the generally higher elevations and 

harsher ambient conditions that exist in CREDA member 

service regions relative to the assumptions used in EPRI’s 

technology assessment efforts. Consequently, the CREDA 

member’s internal cost estimates are higher. A comparison 

of key factors is provided below. 

In summary, CREDA believes that the currently 

proposed values for LTEMP capacity valuation result in an 

understatement of Glen Canyon hydropower capacity value 

on the basis of a credible, publicly available, industry standard source for such information (EPRI) as well as on the 

basis of utility specific information and experience. We urge 

the LTEMP co-leads to utilize the expertise of the cooperating agency utility experts in capacity analysis.

RATEPAYER ANALYSIS: As was recently 

communicated to Ass’t. Secretary Castle, CREDA and 

its electric utility members believe that this analysis must 

be based on and reflect impacts to the product/resource 

produced through the operation of Glen Canyon Dam, 

which is wholesale electric power and energy. On 4/1/14, 

we again objected to an attempt to develop and include a 

retail rate analysis, which is not required by NEPA, and 

which will likely be incorrect and misleading because it 

will not be possible to obtain sufficient and credible data 

given the time constraints and budget dollars associated 

with the LTEMP process. The chart below provides some 

perspective about the complexities involved in electric 

utility retail rate development. Consistent with the scope of 

federal agency responsibility, to the extent any ratepayer 

analysis is required (which we don’t believe is the case 

under NEPA), we urge the LTEMP co-leads to focus on the 

wholesale level, and consider utilizing cooperating agency 

utility expertise.

Finally, CREDA recognizes the time and effort spent by 

all the SDA participants and believes that a key outcome from 

the workshops and exercises is the opportunity afforded the 

AMWG stakeholders, LTEMP co-leads, Argonne National 

Labs and GCMRC personnel and contractors to learn more 

about and appreciate the complexities and interdependencies 

of the resources addressed through the Adaptive Management 

Program, as well as the challenges faced by the Secretary of 

the Interior in balancing the resources associated with the 

operation of Glen Canyon Dam. It has also become clear 

through the model development and results supporting the 

SDA process that the resources of concern are “performing 

well” under the current operational and management parameters. The LTEMP co-lead and cooperating agencies should 

strongly consider this information.

Factor LTEMP EPRI CREDA

1 Capital Cost $917/kw $1,025/kw–1,325/kw $1,130/kw–$1,426/kw

2 Fixed O&M Cost $13.17/kw-yr $15/kw-yr $23/kw-yr–$25/kw-yr

3 Discount Rate (1) 3.8% 5% Higher than 5%

4 Resulting Capacity Cost (2) $65,000/MW-yr $82,000/MW-yr–$101,000/MW-yr $108,000/MW-yr–$132,000/MW-yr

(1) The discount rates of CREDA members are proprietary and confidential information. 

(2) Lower cost represents wet cooling, higher cost represents dry cooling.
64 Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan

Publishing support provided by: 

West Trenton, Tacoma, and Rolla Publishing Service Centers

For more information concerning this publication, contact:

Director, USGS Patuxent Wildlife Research Center

12100 Beech Forest Road, STE 4039

Laurel, MD 20708-4039

301-497-5500

Or visit the Patuxent Wildlife Research Center Web site at: 

http://www.pwrc.usgs.gov/

Renewables

requirements

and carbon 

restriction

Markets in the

west—RTOs,

ISOs, capacity

markets, carbon 

markets

Extra RPS

energy that is 

needed

Extra lowcarbon energy

that is Needed

Type of plant

to build

Cost of fuel

and 

purchased 

power

Merchant

capacity,

scarcity

premiums

Retail

rates

Potential revenue

erosion and solutions

for fixed cost

recovery

Customer

expectations for

rates (more

dynamic, flexible,

etc.)

Demand side

load

reductions

Extra

flexibility that

is needed

Extra energy

that is 

needed

Extra capacity

that is needed

Load

growth

Demand side

load

changes

Type of

time of use

programs,

interruptible

load, etc.

Cost and

effectiveness

of programs

Electric

vehicles,

etc.

Rooftop solar

(lower energy

and capacity

needed, and 

makes less

opportunity for 

cost recovery)

Capacity and energy

changes from every

other contract change

and power plant

retirement/addition

Capacity and

energy changes

from GCD EIS

Capacity and

energy changes

from GCD rate

negotiations in

2024

Representative Factors in

Retail Rate Development

Runge and others—Decision Analysis to Support Development of the Glen Canyon Dam Long-Term Experimental and Management Plan—SIR 2015–5176 

ISSN 2328-0328 (online)

http://dx.doi.org/10.3133/sir20155176