Presentation to the Budget Ad Hoc Group

David P. Braun and Robert S. Unnasch, Sound Science LLC

June 16, 2020
Presentation Outline

 Review purposes

 List of projects reviewed

 Review panel selection

 Review panel members

 Review panel instructions

 Overall review findings

 Individual review findings

 Webinar discussion by project
Purposes of Review

 External, expert, independent, neutral-party peer 

review of draft triennial work plan and budget

 Science Advisors frequently asked to review draft annual 

and triennial work plans and budgets

 Reclamation request to Science Advisor (contractor):

 Review to “provide critical comments regarding the 

quality of science, integration with other projects, and 

overall progress toward answering questions about the 

operation and management of Glen Canyon Dam”
Projects Reviewed

: Streamflow, Water Quality, and 

Sediment Transport and Budgeting

Sandbar and Sediment Storage 

Monitoring and Research

Riparian Vegetation Monitoring 

and Research

 Includes Reclamation C.7-C.8, 

Experimental Vegetation 

Treatment

Effects of Dam Operations and 

Vegetation Management on 

Archaeological Sites

Controls on Ecosystem 

Productivity: Nutrients, Flow, and 

Temperature

Aquatic Invertebrate Ecology

: Humpback Chub Population 

Dynamics

Salmonid Research and Monitoring 

Project

Warm-water Native and Non- Native Fish Monitoring and 

Research

Socioeconomic Research

Geospatial Science, Data 

Management and Technology 

Project

Overflight Remote Sensing in 

Support of Long-Term Monitoring and LTEMP

Economic Impacts of Electrical 

Production at Glen Canyon Dam
Review Panel Selection

 Selection criteria

 Well-established, demonstrated topical expertise

 Demonstrated ability as independent, neutral reviewer

 Ability to review multiple related projects preferred

 No existing or potential conflicts of interest

 Can be challenging: GCMRC scientists collaborate widely

 Selection process

 Master list based on professional knowledge; literature; 

suggestions from Reclamation, GCMRC, previous reviewers

 Draft list reviewed by GCMRC, Reclamation for potential 

conflicts

 Final list ranked and contacted by Sound Science LLC
Review Panel Members

 Prof. Ellen Wohl, Department of Geosciences, Colorado State 

University: Projects A, B

 Prof. Mark Dixon, Department of Biology, University of South Dakota: Projects C, D, and Reclamation Projects C.7-C.8

 Prof. Mark Sweeney, Department of Sustainability & Environment, University of South Dakota: Project D (geology)

 Dr. Emma Rosi, Cary Institute of Ecosystem Studies: Projects E and F

 Prof. Julian Olden, School of Aquatic and Fishery Sciences, University of Washington: Projects G, H, I

 Prof. Bonnie Colby, Department of Agricultural and Resource Economics, University of Arizona: Projects J, N

(Projects K, L reviewed by Dr. Robert Unnasch, Sound Science)
Review Panel Instructions

 Conflict of Interest disclaimer

 Review foci

 Clarity and scientific quality consistent with 2016 

LTEMP ROD goals?

 Meets needs to assess resource status and trends, effects 

of experimental and management actions, potential 

other drivers and constraints?

 Integration with other projects proposed under TWP?

 Feasibility of accomplishing stated three-year goals?

 Contributes to adaptive management of resources and 

experimental and management actions per 2016 ROD?
Review Findings Overall

 Projects with Few Reviewer Concerns or Suggestions

 A, B, E, F, G, H, K, L

 Projects with More Reviewer Concerns or Suggestions

 C, D

 Projects with Greatest Reviewer Concerns, Suggestions

 Reclamation C.7-C.8

 Project I, Element I.3

 Project J

 Project N
Project A

Streamflow, Water Quality, and Sediment Transport 

and Budgeting

 Science questions appropriate and relevant to 

assessing resource status and trends, effects of 

experimental and management actions

 Well integrated with other projects in TWP

 3-year goals and elements highly feasible

 Contributions central to adaptive management of 

resources and experimental and management actions 

prioritized in 2016 LTEMP ROD
Project B

Sandbar and Sediment Storage Monitoring and 

Research

 Science questions appropriate and relevant to 

assessing resource status and trends, effects of 

experimental and management actions

 Well integrated with other projects in TWP

 3-year goals and elements highly feasible

 Contributions central to adaptive management of 

resources and experimental and management actions 

prioritized in 2016 LTEMP ROD
Project C (+)

Riparian Vegetation Monitoring and Research

 Well thought out

 Strong conceptual foundation and excellent 

integration of elements

 Should result in better prediction of vegetation 

responses to flows and management under a wider 

range of conditions than sampled recently in CRe

 Multiple data sources and predictive models will help 

inform the non-flow vegetation plans and actions
Project C (–)

Riparian Vegetation Monitoring and Research

 Proposal should address historical, current, or future 

remote sensing methods or products that could help 

support modeling and guide monitoring

 Project should address plant species age structure and 

demography (recruitment niches) in analyses of 

monitoring data, meta-analysis of regional data, and 

construction of hierarchical models

 Proposal should address groundwater dynamics as possible 

influence on riparian vegetation

 Proposal should more fully address how project relates to 

the LTEMP experimental flows
Reclamation Projects C.7,C.8 (+)

Experimental Vegetation Treatment

 Proposal solid and well thought-out, scientifically well 

supported

 Basic vegetation management objectives clearly described

 Rests on “excellent start” on metrics to prioritize reaches 

and sites for management

 Appropriate adoption of adaptive management framework 

to inform restoration actions across life of program

 Close collaboration with Tribes and GCMRC will help to 

make the projects relevant to stakeholder concerns 

(including protection of cultural resources)
Reclamation Projects C.7,C.8 (–)

Experimental Vegetation Treatment

 Adaptive management process requires explicitly 

identifying hypotheses/questions and discussing how they 

will be addressed – missing from proposal

 Proposal should address how C.7-C.8 restoration, 

monitoring link to GCMRC Projects B, C, D, L

 Proposal should differentiate reach-based vs. site-based 

criteria to identify Priority Treatment Areas

 Different vegetation management objectives may warrant 

different prioritization criteria

 Budget information confusing; not sufficient for review
Project D (+)

Effects of Dam Operations and Vegetation Management 

on Archaeological Sites

 Well written

 Reasonable hypotheses informed by previous research

 Identifiable goals

 Scientific quality high, using proven methods (such as 

LiDAR) to address primary research questions

 E.g., whether HFEs and resultant sandbar building increase 

eolian sand transport to dune areas, resulting in higher 

preservation potential for archaeological sites and cultural 

artifacts

 New and older repeat photos will help assess vegetation 

change over time
Project D (–)

Effects of Dam Operations and Vegetation Management 

on Archaeological Sites

 Proposal needs more clarity on dune types and sites 

representing pre- and post-dam river flows and altered 

eolian sediment connectivity

 Element D.1 needs more clarity on whether the project 

addresses (1) dunes associated with modern fluvial sources 

(MFS), (2) dunes associated with pre-dam Colorado River 

flows (relict fluvial sources; RFS), or (3) both

 Alternative hypotheses possible for importance of dune, 

sandbar vegetation management to eolian dynamics

 Proposal needs to clarify lack of budgeting for Element D.4 

given importance of the element
Project E

Controls on Ecosystem Productivity: Nutrients, 

Flow, and Temperature

 Project well grounded in previously collected data and 

understanding of the system

 Project seeks to maintain existing datasets and 

advance understanding

 Project will fill clearly identified knowledge gaps, with 

well-conceived, doable approaches, including for 

aquatic macrophyte investigations (commended)

 Proposed outcomes and budget appropriate and well 

justified in the plan
Project F

Aquatic Invertebrate Ecology

 Project addresses crucial data needs for GCDAMP

 Projects E, F well linked; combination can provide new 

insights into how food web base responds to dam 

operations and long-term changes in the river

 Project makes maximal use of older data, continues 

important citizen science, continues important study 

of “bug flow” impacts, and aligns well with AMP needs

 Budget reasonable given proposed work; adequate to 

accomplish project goals
Project G

Humpback Chub Population Dynamics

 Project will continue important monitoring activities 

mandated by Biological Opinion associated with 2016 

LTEMP ROD

 Project clearly identifies its scientific questions, 

follows robust methods; findings will be relevant to 

address BiOp Conservation Measures

 Project might consider future investigations of parasite 

composition beyond Asian tapeworm

 Proposed products and budget appropriate
Project H

Salmonid Research and Monitoring Project

 Project uses a robust combination of field, modeling, 

and laboratory techniques to evaluate the response of 

rainbow trout and brown trout to experimental flows, 

other management actions

 Proposal for Element H.2 appropriately identifies how 

budget constraints unfortunately will reduce number 

of reaches with mark-recapture sampling from 3 to 

only 1 with FY2022

 Loss of replication will reduce quality/strength of 

inference important to AMP goals. Can this be avoided?
Project I (+)

Warm-water Native and Non-Native Fish 

Monitoring and Research

 Project priority, overall design, and proposed products 

all appropriate
Project I (–)

Warm-water Native and Non-Native Fish Monitoring and Research

 Project should also address negative impacts of competition with 

invasive fish species

 Proposal discussion of invasive species management is outdated. 

GCDAMP should comprehensively review literature and conduct 

formal modeling to identify gaps in knowledge on invasive species 

management

 Monitoring trip plan for Element I.1 will introduce gaps in knowledge 

due to skipped surveys in some reaches. Could other Element budgets 

be reduced to ensure all monitoring trips maintained for I.1?

 Proposal needs to address eDNA problems with inflated false absences 

and inaccurate estimates of abundance

 Proposal should strongly consider stable isotope or fatty acid analyses 

to determine diet composition, to complement and address limitations 

in conventional gut content analysis

 This would also help identify potential competition between native and 

invasive fishes, in addition to predation
Project J (+)

Socioeconomic Research

 Project is an essential component of overall Work Plan

 Project provides a valuable opportunity to build on 

existing GCDAMP socio-economic approaches and 

models

 Improved understanding of socio-economic 

implications of flow experiments is essential to good 

working relationships among participating federal 

agencies, stakeholders, and elected officials

 Element J.1 has clear objectives and appropriately 

chooses cost-effectiveness analysis over cost benefit 

analysis
Project J (–)

Socioeconomic Research

 Project should address regional economic impacts, e.g., changes in 

employment and business activity as linked to GCD operations and flow 

experiments. Changes in recreation activity linked to themes explored in 

project could have ‘ripple effects’ on the regional economy. 

 Earlier phases of the GCDAMP included regional economic impact studies. 

Why are these not included in this or other recent work plans?

 Proposal for Element J.2 should more explicitly address integration of 

economic findings with Element J.1 and vice versa:

 Both will assess cost-effectiveness in strategies to support native fish and should 

be “... analyzed in a comparable manner and integrated to identify optimal 

combinations of actions.”

 Element J.3 should seriously consider alternative metrics and alternative 

survey approaches “... along the lines of contingent valuation instruments”

 Earlier phases of the GCDAMP included state-of-the-art valuation research. 

Why are these not included in this or other recent work plans?
Project K

Geospatial Science, Data Management and Technology 

Project

 Project addresses foundational needs of GCDAMP to manage 

and share data, crucial for adaptive management

 Proposal background statement clearly highlights need to 

maintain existing information while providing guidance and 

support to implement new technologies for data acquisition on 

long-term projects

 Review commends GCMRC leadership within USGS in using 

cloud-based resources to effectively manage, archive, and share data

 Project team is very collaborative and works effectively with a 

diversity of researchers within GCMRC and its partners. This is 

not easy given great diversity of information managed, but team 

is managing these challenges well
Project L

Overflight Remote Sensing in Support of LongTerm Monitoring and LTEMP

 Plan and budget well-thought-out

 Will produce imagery critical to research and 

monitoring in canyon

 Crucial to repeat overflight in next 2-3 years

 Project team has great experience with this work

 Greatly improved satellite imaging over next few years 

will likely provide same 20 cm. resolution expected 

from 2021 overflight. Team should plan to address this 

in next generation of Project (for FY2024-2026)
Project N (+)

Economic Impacts of Electrical Production at Glen Canyon 

Dam

 Project is an essential component of the Work Plan

 Hydropower and Energy Resources are central elements in the 

regional economy

 Improved understanding of socio-economic implications of 

experiments in GCD operations is essential to good working 

relationships among participating federal agencies, stakeholders, 

and elected officials

 Proposed work provides opportunities to further refine 

approaches and models to estimate economic effects of 

operational changes at GCD

 Proposed operational metrics are well-suited to assessing effects 

of operational changes on the economic value of hydropower
Project N (–)

Economic Impacts of Electrical Production at Glen Canyon Dam

 Proposed financial metrics, while “...central, and commonly used in decision 

frameworks related to hydropower production and dam operations,” “... are not 

a complete measure of the economic value to society of changes in cost and 

availability of hydropower and energy resources”

 A complete suite of metrics would include changes in consumer and producer 

surplus, measured in $/MW and $/MWh, associated with changes in GCD 

operations

 Project should consider including “... a complete economic evaluation [of] 

changes in regional energy costs with changes in GCD operations, and the 

associated changes in consumer and producer surplus”

 Proposal “... does not include work related to regional economic impacts” such 

as “... changes in employment and business activity” with changes in GCD 

operations “... through ‘ripple effects’ in the regional economy”

 Review notes, as with Project J, that regional economic impact studies related to 

GCD operations were important features of earlier phases of the GCDAMP. Why 

are these not included in this or other recent work plans given the 

understandable interests of Reclamation and many regional stakeholders? 
Final Discussion