Review of TWG Feedback on Draft 

Performance Metrics for the Long-Term 

Experimental and Management Plan

Technical Work Group Meeting

October 13, 2022

Helen C. Fairley

USGS Southwest Biological Science Center, 

Grand Canyon Monitoring and Research Center

U.S. Department of the Interior

U.S. Geological Survey
Presentation Outline

 Project Overview 

 Stakeholder Review and General Comments

 Discussion re: general comments/issues

 Specific Metrics by Goal

 Discussion re: specific metrics

 Next Steps
Acronyms Used in This Presentation

 AMWG: Adaptive Management Working Group

 CRE: Colorado River Ecosystem 

 CREDA: Colorado River Energy Distributors Association

 DOI: Department of Interior

 GCD: Glen Canyon Dam

 GCMRC: Grand Canyon Monitoring and Research Center

 GRCA: abbreviation for Grand Canyon National Park

 GLCA: abbreviation for Glen Canyon National Recreation Area

 HBC: Humpback Chub

 LCR: Little Colorado River

 LTEMP: Long Term Experimental and Management Plan

 ROD : Record of Decision

 TWG: Technical Work Group

 TWP: Triennial Work Plan

 WAPA: Western Area Power Administration
Project Overview (Purpose and Need)

 LTEMP defines 11 Goals

 How do we know if Goals are being achieved?

 Need to define performance metrics

 Section 6.1(c) of the LTEMP ROD*

“The DOI, in consultation with the AMWG, will develop 

monitoring metrics for the goals and objectives using those in 

Appendix C as a starting point.” 

(Note: Appendix C = performance metrics developed by Runge 

et al. (2016) to help select the preferred LTEMP alternative.)

 FY 21-23 TWP, Reclamation Project C.12

* Department of Interior, 2016, Record of Decision for the Glen Canyon Dam Long Term Experimental and 

Management Plan Final Environmental Impact Statement, December 2016. Bureau of Reclamation, Upper Colorado 

River Region, Salt Lake City, Utah and National Park Service, Intermountain Region, Lakewood, Colorado. 
Monitoring & Metrics

Monitoring & metrics serve a variety of purposes. 

Principal types and reasons for monitoring include:

 Effectiveness (or Performance) Monitoring

1. To assess effectiveness of policy, plan, or legislation

2. To evaluate progress towards achieving management 

objectives or regulatory standards

 Surveillance Monitoring

3. To detect incipient trends (“early warnings”)

4. To determine resource status in order to decide 

appropriate management actions 

 Validation Monitoring

5. To increase our understanding of resource dynamics

6. To develop and refine models or predictions
This Project

 Focus on defining metrics to monitor effectiveness 

of LTEMP (i.e., Are we achieving the LTEMP Goals?)

 “To assess effectiveness of policy . . .” and 

 “ track progress towards achieving management 

objectives . . .”

 Specific focus is on defining metrics for assessing 

and tracking achievement of LTEMP Goal outcomes

 metrics for other objectives & purposes to be defined later

 Performance metrics tell us “achieving / achieved” 

or “not achieved” -- but not necessarily why

 This will not be a comprehensive Monitoring Plan
LTEMP Performance Metrics: Criteria

 Reflects the expected performance outcome of each 

LTEMP goal, not the underlying “means objectives”

 Quantifiable (metric = measurable)

 Technically & financially feasible to measure (e.g., 

sustainable to monitor over a long period of time)

 Relevant to the metrics used to select LTEMP 

preferred alternative
Metrics design & selection

Some basic tenets of metrics design and 

selection include:

 Prioritize quality of metrics over quantity

 Design metrics that are easy to understand

 Design metrics that are easy to compare 
Example: Humpback Chub (Gila cypha) 

 Goal Statement: Meet humpback chub recovery goals, including 

maintaining a self-sustaining population, spawning habitat, and 

aggregations in the Colorado River and its tributaries below the GCD 

 Performance Metrics

 3.1 Current tier of HBC in LCR aggregation

 3.2 Grand Canyon-wide HBC abundance

 3.3 Proportion of Grand Canyon ecosystem with 

evidence of all 3 life stages

 Surveillance metrics (“drivers” of outcome): 

 Water qualities (temperature, turbidity, etc.)

 Nutrients (e.g., SRP) and food base quality

 Predator loads

 LCR flood magnitude & frequency
Conceptual diagrams capture 

underlying “drivers” of goal outcome
Draft Metrics Report & TWG Review

 Draft report sent to TWG June 9, 2022; discussed at 

June TWG meeting 

 Reviews received from AZ, NM, CO, CA, TWG chair, 

and WAPA/CREDA (Thanks to all who provided 

comments!!!)

 Comments consolidated into two documents; GMRC 

PIs reviewed all & responded to most

 Some TWG comments are in conflict; some will 

require substantial work to address

 Some changes have not been made to the draft, 

pending further discussion with TWG
General TWG Comments & Issues

 Uncertainty about the role/function of metrics and 

how they will be used in decision-making

 Some stakeholders would like more metrics (e.g., 

WAPA would like to include ALL variables from 

previous Knowledge Assessments)

 Some stakeholders suggest combining individual 

metrics, while others propose opposite approach

 Frequency that metrics will be reported (annually vs. 

less frequently in some cases)
Examples of Surveillance Metrics 

(Physical “drivers” of goal outcomes; important 

to monitor but NOT = performance metrics)

Metric Name Measurement Location(s) Frequency Method

Relevant 

Goals

Daily/monthly/annual releases m3/sec (cfs) GCD, LF, Phantom, DC 15 min increments auto sample All

Daily range (magnitude of fluctuations) m3/sec (cfs) GCD, LF, Phantom, DC 15 min increments auto sample All

Water Temperature Degrees C (F) GCD, -8 mi, LF, 5 sed gages 15 min increments auto sample 2,3,5,6,9,10

Turbidity fnu 6 mainstem sed gages 15 min increments auto sample 2,3,5,6,7,9,10

Dissolved Oxygen mg/L GCD, -8 mile, LF 15 min increments auto sample 2,3,5,9,10

Ph unitless GCD, LF monthly grab sample 2,3,5,9,10,11

Phosphorus (SRP, TDP, TP) mg/L GCD, LF, Paria monthly grab sample 2,3,5,9,10,11

Nitrogen (TN, NO23, NH4, TDN) mg/L GCD, LF monthly grab sample 2,3,5,9,10,11

Available Sunlight (Canyon shading)

PPFD μmol m-2 

s-1

Entire Cre- Yard et al. 

2005 instantaneous modeled

2,3,5,9,10,11

Sediment mass balance (inputs/export) metric tons 6 mainstem gages 15 min increments auto sample/modeled 1,2,6,7,11

Weather/climate parameter: Air 

Temperature Degrees C (F)

LF, 11mi, 24.5, 70, 125, 

223mi 4 min increments auto sample

2,6,11

Weather/climate parameter: Wind 

intensity km/hr 6 weather stations 4 min increments auto sample

1,6,7,11

Weather/ climate parameter: Wind 

direction degrees 6 weather stations 4 min increments auto sample

1,6,7,11

Weather/climate parameter: 

precip.intensity mm/hr 6 weather stations 4 min increments auto sample

1,6,7,11

Weather/climate parameter: 

precip.amount mm/hr 6 weather stations 4 min increments auto sample

1,6,7,11

weather/climate parameter: humidity 6 weather stations 4 min increments auto sample 6,11
Knowledge Assessment Metrics vs. 

Performance Metrics

NATURAL PROCESSES

Restore, to the extent practicable, 

ecological patterns and 

processes within their range of 

natural variability, including the 

natural abundance, diversity, and 

genetic and ecological integrity 

of the plant and animal species 

native to those ecosystems.

2017 KA 2019 KA 2022 GCMRC Metrics

• not assessed in 2017 not assessed in the 2019 Knowledge 

Assessment

• Deviation from the natural 

annual hydrograph 

• Deviation from the natural 

daily hydrograph

• Springtime Gross Primary 

Productivity in Marble and 

Grand Canyon

• Percent EPT 

AQUATIC FOOD BASE

(This is not an LTEMP goal)

• Food base diversity

• Secondary production

• Food base diversity

• Secondary production

• not assessed in the 2022 

GCMRC Metrics (but see 3rd

and 4th metrics above)

HUMPBACK CHUB

Meet humpback chub recovery 

goals, including maintaining a 

self-sustaining population, 

spawning habitat, and 

aggregations in the Colorado 

River and its tributaries below the 

Glen Canyon Dam.

• Adult population in western Grand 

Canyon

• Adult population that spawns in 

LCR

• Juvenile chub population in CR 

near LCR

• Juvenile chub population in 

western Grand Canyon

• not assessed in the 2019 

Knowledge Assessment

• Current tier of humpback chub 

in the LCR aggregation

• Grand Canyon-wide 

abundance of adult humpback 

chub 

• Proportion of the Grand 

Canyon ecosystem with 

evidence of all 3 life stages of 

humpback chub 

WATER QUALITY

(This is not an LTEMP goal)

• GCD outflow concentrations of 

dissolved oxygen 

• GCD outflow concentrations of 

metals (e.g., selenium, mercury, 

uranium, etc.) 

• GCD outflow concentrations of 

nutrients (e.g., phosphorous, 

nitrogen) 

• GCD outflow concentrations of 

phytoplankton, zooplankton, 

chlorophyll a 

• GCD outflow salinity, TDS, 

specific conductance 

• GCD outflow temperature 

• GCD outflow turbidity/sediment 

load 

• GCD outflow temperature and 

dissolved oxygen in support of 

condition for trout 

• GCD outflow temperature in 

support of condition for chub

• GCD outflow phosphorus 

concentrations in support of 

ecosystem productivity

not assessed in the 2022 GCMRC 

Metrics
KA Metrics vs. Performance Metrics

2017 KA 2019 KA Metrics Draft

Recreational 

Experience

Maintain and improve 

the quality of 

recreational 

experiences for the 

users of the Colorado 

River Ecosystem. 

Recreation includes, 

but is not limited to, 

flatwater and 

whitewater boating, 

river corridor camping, 

and angling in Glen 

Canyon. 

• Backpacking/day-use shore 

access 

• Backpacking/day-use shore 

availability 

• Backpacking/day-use usable 

campsite area 

• Flatwater floating in Glen 

Canyon NRA 

• GLCA walk-in angling access 

and safety 

• GLCA walk-in angling trout 

abundance 

• GLCA walk-in angling trout 

condition 

• GLCA watercraft angling 

access and safety 

• GLCA watercraft angling 

trout abundance 

• GLCA watercraft angling 

trout condition 

• Whitewater (WW) boat 

mooring 

• WW navigational risk 

• WW navigational risk 

Diamond down 

• WW river crowding (i.e., 

rapids, beaches) 

• WW river running experience 

(i.e., rapids) 

• WW time on river 

• Glen Canyon river 

corridor access 

and safety 

(includes angling, 

boating, hunting, 

etc.)

• GLCA rainbow 

trout condition

• GLCA rainbow 

trout abundance

• WW experience -

rapids and time 

on river

• WW experience -

reduced beaching 

risk

• WW experience -

crowding and 

navigational risk

• WW usable 

campsite area

Economic 

Value of 

Recreational 

Experience

(WTP)
Metric Description Format

Goal X Statement

Background (sets stage and rational for selecting specific metrics)

Conceptual diagram (shows “means variables” & “ecological drivers”)

Metric X.1 (Metric Name)

• Metric Type: direct observation/measurement, estimate based on model, etc.

• Relevant Criteria: Describe how it relates to the established criteria for metrics

• Data required: Data required to produce the metric

• Metric calculation: How metric will be calculated

• Frequency: How often will the metric be measured?

• Presentation: How will the metric be presented (graphically; example provided)

• Interpretation: How will metric be interpreted (historic range, trend, thresholds?)

• Limitations and uncertainties of the metric

• NEW Addition: Current status of metric 
Goals/Metrics still under discussion

 1. Archaeological and Cultural Resources. Maintain the integrity of 

potentially affected NRHP-eligible or listed historic properties in place, 

where possible, with preservation methods employed on a site-specific 

basis. Status: Agreement on 3 metrics; still discussing need for 4th

 7. Sediment. Increase and retain fine sediment volume, area, 

and distribution in the Glen, Marble, and Grand Canyon reaches 

above the elevation of the average base flow for ecological, 

cultural, and recreational purposes. Status: Agreement on 2 

metrics; discussion on hold pending resolution of TWG issues

 8. Tribal Resources. Maintain the diverse values and resources 

of traditionally associated Tribes along the Colorado River 

corridor through Glen, Marble, and Grand Canyons. Status: 

discussion in progress, no agreement on specific metrics yet


Draft Metrics, Goal 2

 Goal 2 – Natural Processes

Restore, to the extent practicable, ecological patterns and 

processes within their range of natural variability, including the 

natural abundance, diversity, and genetic and ecological integrity 

of the plant and animal species native to those ecosystems

 2.1 Seasonal Timing of River Flow

 2.2 Sub-daily fluctuation in River Flow

 2.3 Springtime Gross Primary Productivity

 2.4 Percent EPT
Draft Metrics, Goal 3

 Goal 3- Humpback Chub (Gila cypha)

Meet humpback chub recovery goals, including maintaining a selfsustaining population, spawning habitat, and aggregations in the 

Colorado River and its tributaries below the Glen Canyon Dam 

 3.1 Current tier of HBC in LCR aggregation

 3.2 Grand Canyon-wide HBC abundance

 3.3 Proportion of Grand Canyon ecosystem with 

evidence of all 3 life stages
Draft Metrics, continued

 Goal 5 – Other Native Fishes

Maintain self-sustaining native fish species populations and their 

habitats in their natural ranges on the Co. River and its tributaries

 5.1 - 5.3: Proportion of the Grand Canyon ecosystem (GCE) with 

evidence of all 3 life stages of bluehead sucker (Catostomus 

discobolus) (5.1), flannelmouth sucker (Catostomus latipinnis) 

(5.2), razorback sucker (Xyrauchen texanus) (5.3)

 5.4 Proportion of GCE with speckled dace (Rhinichthys osculus) 

(any life stage)

 5.5 Proportion of GCE with extirpated species (any life stage)

 Goal 9 – Rainbow Trout Fishery

Achieve a healthy high-quality recreational rainbow trout fishery in 

GCNRA and reduce or eliminate downstream trout migration 

consistent with NPS fish management and ESA compliance

 9.1 Trout Angler Catch Rate

 9.2 Rainbow Trout (Oncorhynchus mykiss) Abundance 
Draft Metrics, continued

 Goal 4 – Hydropower

Maintain or increase Glen Canyon Dam electric energy generation, 

load following capability, and ramp rate capability, and minimize 

emissions and costs to the greatest extent practicable, consistent 

with improvement and long-term sustainability of downstream 

resources

4.1 – Economic Value of Hydropower

 Goal 6 – Recreation 

Maintain and improve the quality of recreational experiences for 

the users of the Colorado River Ecosystem. Recreation includes, 

but is not limited to, flatwater and whitewater boating, river 

corridor camping, and angling in Glen Canyon

6.1 – Economic Value of Recreation Experience
Draft Metrics, continued

 Goal 10 – Non-native aquatic species

Minimize or reduce the presence and expansion of aquatic 

nonnative invasives. 

 10.1 - 10.4: Average number of “risky species” per 

habitat segment in Grand Canyon Ecosystem

 10.5 - 10.8: Average number of “risky species” 

with evidence of recent recruitment per habitat 

segment in Grand Canyon Ecosystem

 10.1, 10.5 = Low risk species 

 10.2, 10.6 = medium risk species 

 10.3, 10.7 = high risk

 10.4, 10.8 = very high
Draft Metrics, continued

 Goal 11 – Riparian vegetation

Maintain native vegetation and wildlife habitat, in various stages of 

maturity, such that they are diverse, healthy, productive, selfsustaining, and ecologically appropriate

 11.1 Total plant cover

 11.2 Native plant richness

 11.3 Native to non-native plant species cover ratio
Next Steps

 Today: Provide additional input on draft metrics 

(purpose/need/focus of this project; need for 

more or fewer individual metrics; frequency that 

metrics will be reported, etc.)

 October-December 2022: GCMRC will continue to 

revise draft report with latest input from TWG

 January 2023: GCMRC will pilot example metrics 

at Annual Reporting Meeting 

 January-March 2023: Finalize metrics report
Questions?

July 1973 Questions?

May 2022

River Mile 122.3, left bank. Top photo:

July 1973, photographer unknown.

Bottom photo: May 9, 2022 by A.H. Fairley