U.S. Department of the Interior

U.S. Geological Survey

Example Metrics for the Long-Term 

Experimental and Management Plan

Technical Work Group Meeting

October 13, 2021

Helen C. Fairley, Lucas S. Bair, Charles B. Yackulic

U.S. Geological Survey

Southwest Biological Science Center

Grand Canyon Monitoring and Research Center

U.S. Department of the Interior

U.S. Geological Survey Photo Credit: Amy Martin
Introduction

• LTEMP defines 11 goals

– How do we know if goals are being achieved?

– Need to define performance metrics!

• Section 6.1(c) of the LTEMP ROD* states:

“The DOI, in consultation with the AMWG, will develop monitoring 

metrics for the goals and objectives using those in Appendix C as a 

starting point.” 

(Note: Appendix C = performance metrics developed by Runge et al. 

(2016) to help select the preferred LTEMP alternative.)

• FY 21-23 TWP, Reclamation Project C.12

* Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term Experimental and Management Plan Final 

Environmental Impact Statement, December 2016. Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and 

National Park Service, Intermountain Region, Lakewood, Colorado. 
Metrics Project Objectives

• Determine technically feasible, sustainable metrics to provide 

objective measures of progress towards attaining the 11 

LTEMP resource goals.

• Use metrics to periodically evaluate and report status of the 

11 LTEMP resource goals relative to pre-LTEMP (pre-2016) 

condition and LTEMP projections.

– Improve annual/cyclical reporting by distilling large amounts of scientific 

information into a set of standardized metrics that can be communicated to 

DOI managers and GCDAMP stakeholders via standardized methods and 

graphics on a regular (annual?) basis.

– Inform other planning efforts (e.g., specify knowledge gaps in the 

context of resource goals).
Draft Criteria for Selecting Metrics

• Reflects the expected performance outcome of each LTEMP 

goal or measures a performance outcome that triggers a 

compliance action or experiment

• Quantifiable (metric = measurable)

• Technically and financially feasible to measure

• Compatible with the metrics used to select LTEMP preferred 

alternative
Metrics Description

• Metric Type: (direct observation, estimate based on model, etc.)

• Relevant Criteria: relationship to established criteria

• Data required: data needed to produce the metric

• Metric calculation: How will metric be calculated?

• Frequency: How often will the metric be measured?

• Presentation: How will the metric be presented graphically?

• Interpretation: How will the metric be interpreted (historic 

range, trend, thresholds?)

• Limitations and uncertainties of the metric
Project Timeline: FY22 – October 2021 TWG: 

• GCMRC presents draft LTEMP performance metrics (sample) 

• TWG provides input during meeting; discussion of how best to 

graphically represent metrics to meet stakeholder needs. 

• Input from TWG (due end of October: Friday, 10/29/21)

– December 2021: 

• GCMRC pilots a sample of metrics in the FY2021 Annual Report, including 

sample graphics

– January 2022:

• GCMRC solicits feedback from the TWG regarding sample reporting of 

metrics in the FY2021 Annual Report. 

– February 2022:

• GCMRC solicits feedback from the AMWG and DOI regarding the sample 

of reported metrics in the FY2021 Annual Report.
U.S. Department of the Interior

U.S. Geological Survey

Photo Credit: National Park Service

Example Metrics for the Long-Term 

Experimental and Management 

Plan: Hydropower and Energy

7
Hydropower and Energy

• *Resource Goal: Maintain or increase Glen 

Canyon Dam electric energy generation, load 

following capability, and ramp rate capability, 

and minimize emissions and costs to the 

greatest extent practicable, consistent with 

improvement and long-term sustainability of 

downstream resources.

• Maximize the economic value of hydropower.

8

* Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term 

Experimental and Management Plan Final Environmental Impact Statement, December 2016. 

Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and National Park 

Service, Intermountain Region, Lakewood, Colorado. 
Hydropower and Energy Metrics*

• Economic value of energy generation

– Estimated using optimization model of GCD release 

and energy generation (e.g., GTMax) 

• Economic value of generation capacity

– Estimated using system model of the larger 

electricity sector and power system expansion costs

9

* Runge, M.C., LaGory, K.E., Russell, Kendra, Balsom, J.R., Butler, R.A., Coggins, L.G., Jr., Grantz, K.A., Hayse, John, 

Hlohowskyj, Ihor, Korman, Josh, May, J.E., O’Rourke, D.J., Poch, L.A., Prairie, J.R., VanKuiken, J.C., Van Lonkhuyzen, 

R.A., Varyu, D.R., Verhaaren, B.T., Vesekla, T.D., Williams, N.T., Wuthrich, K.K., Yackulic, C.B., Billerbeck, R.P., and 

Knowles, G.W., 2015, Decision analysis to support development of the Glen Canyon Dam Long-Term Experimental and 

Management Plan: U.S. Geological Survey Scientific Investigations Report 2015–5176, 64 p., http://dx.doi.org/10.3133/ 

sir20155176.
Economic Value of Energy

Do Not Cite

Price

Quantity

Supply

Hydropower

Coal

Natural gas
Economic Value of Energy

Price

Quantity

Supply

Demand

Hydropower

Do Not Cite
Hydropower Objective

12

Reservoir 

levels

Hourly 

energy 

prices

Ramp up

Min Onpeak

Max

Ramp 

down

Daily 

range

Monthly volume

Min Offpeak

Economic 

value of 

energy

Hourly flow to energy 

conversion

Flow constraint set

• Maximize the economic value of hydropower over time 

subject to operational constraints.

Do Not Cite
Example Metric Presentation

13

Trout 

Management 

Flows

Do Not Cite
U.S. Department of the Interior

U.S. Geological Survey

Photo Credit: Bureau of Reclamation

Example Metrics for the Long-Term 

Experimental and Management 

Plan: Recreational Experience

14
Recreational Experience

• Resource Goal: Maintain and improve the 

quality of recreational experiences for the 

users of the Colorado River ecosystem. 

15

Photo Credit: Bureau of Reclamation

* Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term 

Experimental and Management Plan Final Environmental Impact Statement, December 2016. 

Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and National Park 

Service, Intermountain Region, Lakewood, Colorado. 
16

Photo Credit: Bureau of Reclamation
Whitewater Conceptual Model

17

Number of 

Whitewater 

Rafters

Sandbar Area

Size of Rapids

Crowding

Time Off River

Navigation

Sandbar Area

Fluctuating Flows

Unique trip 

characteristics 

(i.e., geology, 

wilderness).

Whitewater 

Net Economic 

Value

Other

Function of 

Average Flow

Trip Attributes

Willingness to Pay

Do Not Cite
LTEMP Performance Metrics

• Camping Area Index: Accounts for optimal campsite area 

building and maintenance flows and sediment load.

• Navigational Risk Index: Measure of navigation difficulty 

based on the number of days during which the daily minimum 

flow was less than 8,000 cfs.

• Fluctuation Index: Measures the degree to which 

combinations of flows and fluctuations are within a range 

identified as preferable by experienced boat guides.

• Time Off River Index: Relates the level of flows to visitors 

being able to spend time ashore visiting attractions. 

18

Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term Photo Credit: Bureau of Reclamation

Experimental and Management Plan Final Environmental Impact Statement, December 2016. 

Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and National Park 

Service, Intermountain Region, Lakewood, Colorado. 
19

LTEMP Performance Metrics

0.0

0.2

0.4

0.6

0.8

1.0

3,000 7,000 11,000 15,000 19,000 23,000 27,000 31,000

Recreational Expericnce Indecies

Constant Colorado River flow in cubic feet per second

Camping Area Index

Time Off River Index

Do Not Cite
20

Chris J. Neher, John Duffield, Lucas S. Bair, David Patterson, Katherine Neher. 2017. Testing the limits 

of temporal stability: willingness to pay values among Grand Canyon whitewater boaters across 

decades. Water Resources Research, 53, 10,108– 10,120, 10.1002/2017WR020729. 
Participation

21

Upper Grand Canyon Whitewater 

Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term 

Experimental and Management Plan Final Environmental Impact Statement, December 2016. 

Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and National Park 

Service, Intermountain Region, Lakewood, Colorado. 
Example Metric Presentation

Do Not Cite 22
U.S. Department of the Interior

U.S. Geological Survey

Photo Credit: Bureau of Reclamation

Example Metrics for the Long-Term 

Experimental and Management 

Plan: Humpback chub

23
Humpback chub

• Resource Goal: Meet humpback chub recovery goals, 

including maintaining a self-sustaining population, 

spawning habitat, and aggregations in the Colorado 

River and its tributaries below the Glen Canyon Dam.

24

* Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term 

Experimental and Management Plan Final Environmental Impact Statement, December 2016. 

Bureau of Reclamation, Upper Colorado River Region, Salt Lake City, Utah and National Park 

Service, Intermountain Region, Lakewood, Colorado. 
Humpback chub

• Performance metric 1: Expected minimum number of 

adult HBC in the Little Colorado River population 

during the LTEMP planning period (20 years).

25
Humpback chub

• Performance metric 1: Adult HBC abundance in the 

Little Colorado River population.

• Metric Type: Estimated

• Data required: Sampling in the LCR and mainstem CR

• Metric calculation: Multistate mark-recapture model 

• Frequency: Annually

26

Preliminary, do not cite
Presentation
Interpretation

• Relative to Biological opinion triggers (along with other 

metrics derived from multistate model)

• Relative to LTEMP predictions under similar conditions

28

Preliminary, do not cite
Limitations/Uncertainties

• Estimation adult abundance:

– Capture probability heterogeneity, long-term behavioral 

responses, assumptions about partially sampled mainstem.

• Forecasting adult abundance

– Future foodbase conditions.

– Impacts of temperatures above 16-ish.

– Brown trout*, catfish, and any new nonnatives

– Climate change and juvenile production in the LCR

– (see appendix F for more details.)

29

Preliminary, do not cite
Conceptual models

30
31

Department of Interior, 2016, Glen Canyon Dam Long Term Experimental and Management Plan Final 

Environmental Impact Statement, December 2016. Bureau of Reclamation, Upper Colorado River Region, Salt 

Lake City, Utah and National Park Service, Intermountain Region, Lakewood, Colorado. 
Colorado River

Little Colorado 

River

resident

Preliminary, do not cite
Humpback chub

Recruitment

LCR juveniles

No flooding

LCR adults

CR juveniles

CR adults

RBT near LCR

Water 

temperature

Adult HBC 

Colorado River Abundance Little Colorado River

CF translocated 

juveniles

CF adults

Removals

(Yackulic et al., 2018; 2020; 2021; Dzul et al., 2021; In prep.)

Other stuff:

1) LCR Catfish?

2) LCR Sand?

3) Mainstem food?

Food

SRP

BNT?

SMB?

Water 

temperature?

CCF?

Turbidity
Humpback chub

• Performance metric 2: Average 

temperature suitability for HBC at RM 

157 and RM 213.

• Dibble et al. (2021) provides an 

improved version.

• But in near future, would like to move 

towards actual abundance estimates 

for western GC.

34

Preliminary, do not cite
Questions