Independent Scientific Advisory Board

for the Northwest Power and Conservation Council, Columbia River Basin Indian Tribes, and NOAA Fisheries

and

Independent Scientific Review Panel

for the Northwest Power and Conservation Council; 851 SW 6th Avenue, Suite 1100; Portland, Oregon 97204

A Joint ISAB and ISRP Review of the 

Draft Research, Monitoring & Evaluation Plan 

for the NOAA-Fisheries 2000 Federal Columbia 

River Power System Biological Opinion

January 15, 2004

ISAB&ISRP 2004-1
Review Authors

Joint ISRP and ISAB Members

Charles C. Coutant, Ph.D., senior resource ecologist, Oak Ridge National Laboratory, Oak 

Ridge, Tennessee.

Daniel Goodman, Ph.D., an expert in ecological risk assessment at Montana State University in 

Bozeman.

Susan Hanna, Ph.D., professor of agriculture and resource economics at Oregon State 

University (also an IEAB member).

Lyman McDonald, Ph.D., consulting statistician at Western Ecosystems Tech., Inc., Cheyenne, 

Wyoming, formerly a professor at the University of Wyoming.

Brian Riddell, Ph.D., an expert in international fisheries management at the Department of 

Fisheries and Oceans Canada, Nanaimo, British Columbia.

ISRP Members

Nancy Huntly, Ph.D., professor of wildlife biology at Idaho State University.

William Liss, Ph.D., professor of fisheries at Oregon State University.

William Smoker, Ph.D., professor of fisheries, aquaculture technology and genetics, at the 

University of Alaska Fairbanks, Juneau Center for Fisheries and Ocean Sciences.

Richard R. Whitney, Ph.D., consulting fisheries scientist, Leavenworth, Washington, formerly 

a professor in the School of Fisheries, University of Washington.

Richard Williams, Ph.D., ISRP Chair, an expert in population and evolutionary genetics, 

ecology. Graduate Affiliate Faculty, Aquaculture Research Institute, University of Idaho.

ISAB Members

Robert Bilby, Ph.D., an expert in riparian ecology at Weyerhaeuser Corporation.

Peter A. Bisson, Ph.D., a specialist on habitat issues at the Olympia (Washington) Forestry 

Sciences Laboratory of the U.S. Forest Service.

Eric J. Loudenslager, Ph.D., ISAB Chair, an expert in genetics and fish culture, and a hatchery 

manager at Humboldt State University, California.

David P. Philipp, Ph.D., an expert in conservation genetics and reproductive ecology at the 

Illinois Natural History Survey, University of Illinois.
i

ISAB and ISRP Review of the Draft Research, Monitoring & 

Evaluation Plan for the NOAA-Fisheries 2000 Federal Columbia 

River Power System Biological Opinion

Contents: 

Introduction........................................................................................................................1

General Overview/Executive Summary ..........................................................................3

General Issues..............................................................................................................3

Answers to the four NOAA Questions ........................................................................7

Overall ISRP/ISAB Recommendations.....................................................................11

ISRP/ISAB Recommendation on Sections of the Plan..............................................11

Comments on RME Plan Front Matter: Introduction, Overview, Project 

Implementation, Compliance Monitoring and Regional Coordination......................14

General Comments ....................................................................................................14

Detailed Comments on the RME Plan Introduction and Overview ..........................16

A. Tributary .....................................................................................................................18

General Comments ....................................................................................................18

Answers to NOAA Questions....................................................................................18

Detailed Comments...................................................................................................23

B. Hydrosystem................................................................................................................25

General Comments ....................................................................................................25

Answers to NOAA Questions....................................................................................25

Detailed Comments...................................................................................................31

C. Hatchery/Harvest .......................................................................................................36

1. Hatchery.....................................................................................................................36

General Comments ....................................................................................................36

Answers to NOAA Questions....................................................................................36

2. Harvest: Action 167. Improving estimates of incidental mortalities in fisheries.....42

General Comments and Answers to NOAA Questions.............................................42

Detailed Comments...................................................................................................44

D. Columbia River Estuary and Plume .........................................................................46

General Comments ....................................................................................................46

Answers to NOAA Questions....................................................................................46

Detailed Comments...................................................................................................48

E. Data Management .......................................................................................................58

General Comments ....................................................................................................58

Answers to NOAA Questions....................................................................................58

Appendix 1. Comments on “Monitoring Strategy for the Upper Columbia Basin” .62
ISAB&ISRP 2004-1 RME Plan Review

1

ISAB and ISRP Review of the Draft Research, Monitoring & 

Evaluation Plan for the NOAA-Fisheries 2000 Federal Columbia 

River Power System Biological Opinion

Introduction 

In September 2003, NOAA Fisheries and the Northwest Power and Conservation Council 

requested that the ISAB and ISRP review the Action Agencies and NOAA Fisheries’ 

draft Research, Monitoring & Evaluation Plan for the NOAA-Fisheries 2000 Federal 

Columbia River Power System Biological Opinion.1 This draft plan was developed jointly 

by staff from NOAA Fisheries, the Bonneville Power Administration, the Army Corps of 

Engineers, and the Bureau of Reclamation, and is designed to help implement the NOAA 

Fisheries 2000 Federal Columbia River Power System (FCRPS) Biological Opinion 

(BiOp) and the Federal Caucus Basinwide Salmon Recovery Strategy. The purpose of 

the RME Plan is to initiate and coordinate a process for evaluating the status of affected 

stocks of ESA listed anadromous salmonids and the effectiveness of mitigation and 

conservation actions. The focus of the Plan is on the BiOp and Recovery Strategy 

requirements as they relate to ESA-listed salmonids. The ISRP and ISAB recognize the 

importance of the Plan in providing a roadmap to demonstrate that recovery actions are 

having the desired effect or to identify actions that are not succeeding. 

The ISRP and ISAB have looked forward to reviewing the document over the past year. 

Early in 2003, the ISRP completed reviews of proposals submitted in response to BPA’s 

Mainstem/Systemwide solicitation and a subsequent set of Request for Studies intended 

to meet research, monitoring, and evaluation gaps for Reasonable and Prudent 

Alternative actions under the 2000 FCRPS BiOp. These reviews were conducted without 

the context of an ISAB/ISRP review of the RME Plan -- the subject of this review. That 

stated, this review benefits from the ISRP’s experience with the proposal review process 

and the review of earlier RME group products.2

The draft plan is limited to the specific requirements of the NOAA Fisheries FCRPS 

BiOp. Additional RME requirements of the U.S. Fish and Wildlife Service (USFWS) 

FCRPS BiOp for ESA-listed resident fish will be integrated with this RME program as 

they are developed in coordination with resident fish recovery planning. This RME 

program will also be integrated with the broader RME needs of the Federal All-H 

Strategy and the Northwest Power and Conservation Council’s (NPCC) Fish and Wildlife 

(F&W) Program, in coordination with other regional Federal, state and tribal RME 

programs. The Action Agencies and NOAA Fisheries are working with these other 

regional entities to identify areas of program overlap, coordination efficiencies, and 

funding responsibilities.

 

1

 The draft Research, Monitoring and Evaluation (RME) Plan is available at 

http://www.nwr.noaa.gov/1hydrop/hydroweb/fedrec.htm)

2

 See the ISRP’s webpage on the Northwest Power and Conservation Council’s website for the set of six 

reviews related to the Action Agencies’ RME effort: www.nwcouncil.org/library/isrp/Default.htm. 
ISAB&ISRP 2004-1 RME Plan Review

2

NOAA Fisheries requested that the ISAB [and ISRP] review focus on the following 

questions: 

1. If implemented, will the plan provide useful information about the status of ESAlisted salmon species? In what ways could the plan be improved to provide better 

information?

2. If implemented, will the plan provide useful information about the effectiveness of 

hydro, habitat, hatchery and harvest actions at improving the status of ESA-listed 

salmon species? In what ways could the plan be improved to provide better 

information?

3. In the context of the BiOp, has the plan identified the most critical RME gaps? Are 

there additional important gaps that it has not identified?

4. The plan describes the need for better coordination of data collection and data 

management throughout the Basin. If implemented, will the plan lead to improved 

coordination of data collection and data management?

The ISAB/ISRP review of the draft RME plan begins with a general overview/executive 

summary that includes a summary of the ISAB/ISRP recommendations and identifies 

broad issues that apply to the overall framework and implementation of the Plan. This 

overview section is followed by detailed ISAB/ISRP reviews of each section of the RME 

plan: habitat, hydrosystem, hatchery/harvest (presented together in the RME Plan), 

Columbia River estuary and plume (received as a separate document), and data 

management. Under each section, the ISAB/ISRP provides general comments, answers 

to the NOAA Fisheries’ questions, and detailed comments intended to help improve the 

final RME Plan. 

In addition to the RME Plan, the ISAB/ISRP were also requested to review a draft report 

prepared by BioAnalysts, Inc. for the Upper Columbia Regional Technical Team and the 

Upper Columbia Salmon Recovery Board. This report, titled Monitoring Strategy for the 

Upper Columbia Basin, was meant to provide specific guidance to various stakeholders 

in the Upper Columbia River Basin for implementing a coordinated monitoring program. 

In this sense, it contained far more specifics than the RME Plan, and it pertained almost 

exclusively to tributary habitat restoration. A review of this document is limited to habitat 

recovery evaluation and is included in an appendix of this report.
ISAB&ISRP 2004-1 RME Plan Review

3

General Overview/Executive Summary 

The plan is well organized and quite comprehensive overall. It represents an excellent 

start, and its authors should be commended for their work. The RME document has broad 

applicability and should be instructive to all practitioners. However, the ISRP/ISAB 

review identifies some large contextua l questions that would likely limit the effectiveness 

of the Plan, independently of how well it addresses the smaller issues. We highlight these 

important general concerns first below, followed by a summary of answers to the four 

NOAA questions, a summary of overall recommendations for the Plan, and a summary of 

major recommendations on each section of the Plan.

General Issues

Regional Coordination

The RME document presents what is inherently a regional plan to implement the BiOp; 

as such, it should clearly address the issue of regional coordination and should attempt to 

build effective regional coordination. Regional coordination must be a key part of the 

Plan. Although references are made throughout to the need and plans for interagency 

coordination in data management, monitoring, and evaluation, the Plan would be 

improved if it gave more detail on the process to enable that coordination. Success of the 

Plan depends on State, Federal, and Tribal agencies being willing to make changes in 

current sampling and data collection methods. The current draft does not make clear how 

it will encourage or ensure that the needed coordination and avoidance of duplication will 

take place. The Plan should better specify where the ongoing and state partnerships fit in, 

describe the coordination work to date, and explicitly identify strategies and plans to get 

the region involved. This RME effort presents the opportunity for planning at a basin 

scale, and coordination and efficiencies must be emphasized. The ISAB/ISRP suggest 

that the Plan include specific actions intended to foster implementation of regional 

planning for large-scale and long-term answers. The RME Plan could take a lead role in 

suggesting a regional plan to meet overall salmon recovery goals. For instance, many 

programs are PIT tagging fish, but there is no project aggregating all the sets of PITtagged fish to answer broader questions. The Plan could present the number of PIT-tags 

needed overall to estimate SARs, as well as identify specific tagging sites needed to 

achieve the marking levels necessary for regional data needs. The RME Plan also could 

foster needed coordination through planning exercises to develop experimental design or 

to analyze data and scope quantitative targets. 

Conceptual Framewo rk and Coordination across the Hs

The current RME plan is developed to follow the BiOp RPA action by action without 

sufficient attention to the overall intent of the BiOp. The RME plan needs a framework 

that ties all the Hs and RPA actions together and ensures agreement on the overall 

evaluation basis for recovery. This framework could then be used to coordinate the 

experimental design and data collection across the RME plan sections to facilitate an 

efficient and clearly integrated RME program. The Plan would benefit from explicit 
ISAB&ISRP 2004-1 RME Plan Review

4

development of an integrative conceptual model. Such a model could integrate the key 

questions over the life cycle of listed stocks and explain how the pieces of the RME plan 

fit into answering the overall key questions and key uncertainties. Such a model could be 

used to integrate components of the Plan into an explicit experimental design, guide 

priorities within the Plan, and aid in achieving regional participation and coordination. 

A specific example of an essential component of the monitoring and evaluation plan that 

integrates all actions is provided in the next paragraph. 

Monitoring Designs for Natural Populations

The presumptive bottom line for recovery is the population growth rate (lambda) for the 

wild stocks. However, a monitoring program design of naturally produced salmon 

spawning in the natural environment is not adequately described in the RME plan. 

Monitoring data needs are described in the tributary habitat sections and the potential 

interactions of hatchery and wild fish are noted in the hatchery section. However, the 

Plan does not provide information on the populations to be assessed, and the sampling 

methods and biological parameters expected in those programs. The streams and 

populations to be monitored and ability to consistently collect data on naturally produced 

salmon will influence the region’s ability to assess progress towards recovery. The 

absence of an actual design of the annual adult monitoring programs for the natural 

populations leaves significant uncertainty concerning this central feature of the 

monitoring and assessment plan. The Plan needs to ensure a quantitative basis for 

monitoring abundance trends and productivity in both adult and juvenile natural 

populations. If variability in annua l adult monitoring programs were to exceed the 

changes in abundances between years, then the ability to assess recovery using lambda 

would be severely compromised. Specification of species, streams, and methods in the 

Plan would indicate acceptance of an assessment basis for recovery evaluation and 

facilitate coordination between agencies.

Issues of Experimental Design and Existing Data

More explicit discussion of experimental design is critically needed throughout the Plan. 

The Plan should explore thoroughly the strengths and benefits of treatment and control 

(including natural experiments or active manipulations) versus regression/correlation 

designs, and should give general procedures for deciding on preferable design. It should 

also state specific designs for specific questions in specific places (or concrete procedures 

and plans for deciding these), as these decisions are made. The Plan should explore the 

possibilities for using various designs or at least include an approach for having these 

background experimental design scoping exercises done as part of the implementation of 

the Plan. Design options can be assessed from subbasin or other regional inventory data, 

and one can calculate which approach is likely to provide the more effective study design 

for specific questions in specific places. The Plan should explore the ranges of conditions 

and combinations of conditions present in the Basin and should present designs for 

studies that can provide data to address cause and effect relationships between fish 

metrics and these conditions (whether naturally occurring or the result of planned or 

incidental human actions). Satisfactory designs may sometimes involve natural 

experiments and regression/correlation analyses, or may include establishment of true 

manipulative experiments, with applied treatments and complementary controls. It would 
ISAB&ISRP 2004-1 RME Plan Review

5

be helpful for the region if the possibilities for achieving these designs, as well as the 

likely power of each approach, were explored at the beginning of the RME process. This 

information dictates what experimental designs are reasonable to pursue. We expand on 

this discussion of design and make several suggestions as to what is the best approach in 

our comments on the Tributary Habitat Monitoring section of the Plan. 

Sampling designs and targets also should include more quantitative scoping, either in the 

Plan itself or through processes scheduled to occur early in the implementation of the 

Plan. It would help if the Plan’s authors explored what size effects are detectable under 

various scenarios and with what effort, and showed how this information would be used 

in deciding experimental designs and prioritizing efforts. 

Additionally, the Plan should consider more thoroughly the distinctions between and 

interrelationships of status and effectiveness. Effectiveness of actions does not equal the 

only source of change in status or trend, and the Plan should include design and analysis 

elements that can best identify problems and possibilities. 

Plan Development and Implementation 

The Plan may need a tiered development approach. For instance, it may make sense to 

get status and background monitoring in place and implemented for several years before 

determining the more specific experiments. If so, exploration of data collected to date 

and what it suggests about critical needs, effective designs, etc, should be included in the 

phase-in stages. The data that are currently available (despite their limitations, which 

need to be specified) can and should be used in quantitative exploration of the costs and 

benefits of specific experimental designs to ask specific questions. The Plan should be 

tiered and sequenced, including a plan not just for implementation, as in the pilot studies, 

but also for adding desired or needed elements (e.g., larger channels, new species, etc.) 

and for eliminating parts that are no longer needed (e.g., scenario for response to 

cutbacks in funding, response to extinctions, retirements, answered uncertainties, etc.). It 

would be useful to imagine 5-year, 10-year, and 20-year scenarios. Additionally, the Plan 

largely sidesteps the issue of building coordination, but the Plan’s implementation design 

could directly address this anticipated difficulty (perhaps a “critical uncertainty”). It may 

be that the RME Plan could structure group efforts to solve concrete problems in an 

integrated forum (e.g., the pilot projects, scoping workshops, etc.) and so begin to 

accomplish coordination. 

Prioritization of Elements of the Plan

The elements of the Plan should be clearly prioritized. Its current organization closely 

follows specific RPAs but does not make clear their priority. It should also specify 

critical uncertainties in a way that is linked to key elements of the Plan rather than as 

stand-alone items. The Plan also should specify a relative priority for each of the 

monitoring initiatives, because the cost estimates provided, which are probably 

conservative, suggest the daunting expense of such an ambitious program and will likely 

impose compromises in RME design. Even though the RME Plan calls for widespread, 

expensive monitoring, the resources needed to implement such a program surely will be 

inadequate to achieve every objective in the Plan, thereby necessitating the prioritization 
ISAB&ISRP 2004-1 RME Plan Review

6

of monitoring activities. It would be helpful, therefore, if the Plan identified what aspects 

of the monitoring efforts were considered most essential and should receive priority 

funding. It should give highest priority to key integrative bottom-line measures (e.g., 

SARs) as core elements of the program. It also should prioritize critical uncertainties and 

should link these intimately into the program rather than having them as stand-alone 

projects. It should include quantitative targets and backgrounds (e.g., number of fish to be 

tagged). The current emphasis on piecemeal actions and RPAs does not supply adequate 

integration or prioritization of the effort. Better integration of sections also should lead to 

economy, consistency of outcomes, and higher usefulness of the Plan, as well as to better 

linkage to science and management questions.

Costs and Benefits of Measurement Precision

The RME report emphasizes that accuracy and precision of metrics should be determined 

and reported. Although the ISAB/ISRP agrees that precision and accuracy should be 

reported, the Plan should also present the rationale for the standards of precision and 

accuracy that are selected. In the current Plan, many of the standards selected seem 

arbitrary and perhaps unrealistic. The cost of precision is high, and the Plan should more 

carefully weigh (and explicitly present) how much precision is needed to answer specific 

questions, as well as what precision is likely possible, given what is known or can 

reasonably be assumed about the parameters to be measured and the techniques used to 

measure them. This is another area in which the Plan must weigh various costs and 

benefits and select, with explanation, the apparently most efficient monitoring plan to get 

the needed level of precision. That said, we note that at some point, planning must stop, 

limits on funds and resources must be established, and the monitoring plan must be 

finalized and come in under budget. In the end, it is difficult or impossible to finalize and 

implement a monitoring program in the field without knowledge of the funds and other 

resources available. 

Pilot Projects

The plan to begin with pilot projects that will test methods could be used more 

ambitiously to advance many other acknowledged difficulties (e.g., regional 

coordination, inconsistent use of data collection protocols, lack of background data 

analysis, need to better define data needs and data management protocols). The Plan’s 

implementation design should be expanded conceptually to include the needed analytical 

inventory of existing data, as well as planning of experimental designs. Pilot projects 

could include a survey of the potential for establishment of (1) applied treatment/control 

designs, (2) sample surveys of natural experiments, or (3) sample surveys of 

probabilistically selected sites. The Plan should explore the strengths and weaknesses of 

each of these techniques for answering specific questions. Such planning exercises could 

be developed and implemented by NOAA or other Action Agencies as an explicit part of 

the RME process and could be used to build buy-in to the Plan, as well as to link with 

Subbasin and other planning efforts. 
ISAB&ISRP 2004-1 RME Plan Review

7

Incorporation of the Corp’s AFEP Program

The RME Plan states that projects supported through the Corps’ Anadromous Fish 

Evaluation Program (AFEP) process will constitute adequate hydropower RME for this 

plan and will receive adequate review by that process which is already in place. We are 

concerned about the deference paid in this RME plan to the Corps’ AFEP program and 

raise it as a general issue. Although other Action Agencies’ programs were scrutinized, 

all RME related to AFEP is simply accepted with no apparent scrutiny. In this draft plan 

all dam-specific RME is assumed acceptable because it is vetted through the AFEP 

review process. That may be a mistake. The ISRP is independently reviewing the AFEP 

program. It has found that adequate review is unlikely to happen, because there is 

generally not sufficient time for review of final study plans prior to their adoption. The 

ISRP’s yet to be completed review of the Corps AFEP process in the past few months 

indicates that the AFEP deserves as much scrutiny for overall RME goals as other 

programs, both for planning of research and monitoring and determining effectiveness of 

actions. Because the AFEP dominates the hydropower side of RME, the RME plan would 

be improved if it included the AFEP work. The ISAB/ISRP suggests that the ISRP AFEP 

review (due in early 2004) be considered a part of this RME review.

Answers to the four NOAA Questions

In part the purpose of the ISRP/ISAB’s current review is to inform NOAA Fisheries if 

the Plan adequately meets the needs in the BiOp. The answer to this varies from section 

to section. In this overview, the ISAB/ISRP provides brief summaries of our answers to 

the questions. The subsequent section of the report provides more detailed answers for 

each section.

Question 1: Will the Plan provide useful information on the status of ESA-listed 

salmon? How could it be improved? 

• Tributary Habitat: The Tributary RME Plan should provide useful information 

about the status of listed salmon. Potential improvements include reconsideration 

of the method to evaluate the accuracy of the sampling programs. 

• Hydrosystem: The Hydrosystem RME Plan will likely provide much useful 

information, but the most pertinent information is approached only vaguely, 

namely whether the status of stocks is improving at acceptable rates.

• Hatchery: RPA 182 – The overall approach should provide useful information, 

but details of the proposed activities need to be better developed to ensure they 

will be adequate. RPA 184 - This element of the Plan requires further 

development, especially developing an explicit conceptual framework for 

integrating data to reach conclusions about populations or ESUs. 

• Harvest: No. The Plan’s response to a single Harvest RPA action (167) is a very 

limited consideration of harvest. It ignores the other four harvest related RPA 

actions, and as such, it presents an extremely narrow consideration of harvest.
ISAB&ISRP 2004-1 RME Plan Review

8

• Estuary/Plume: This section is too vague to be useful in its current form; specific 

objectives, metrics, and methods are needed.

• Data Management: Considered as a “statement of the problems for data 

management” the Plan is clearly written and if the problems are solved, solutions 

would provide useful information. However, it does not have proposed 

“solutions.” The plan is a statement of intentions to develop a plan in the future. 

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

• Tributary Habitat: The combination of status and trend monitoring with more 

intensive investigations of cause-effect relationships in the Action Effectiveness 

Research (AER) watershed scale and project-based approaches should provide a 

clear indication of the response of salmon to tributary habitat actions. The more

intensive AER efforts are well conceived, but several elements of this approach 

could compromise the quality of the information produced. Watershed-scale 

monitoring often occurs with little or no control over where, when, or what 

management and restoration actions will be applied. Control over where 

restoration actions are applied is required to maintain the integrity of reference 

sites. The response of salmon to a restoration project depends on both the action 

type and where the action is applied. 

• Hydrosystem: The effectiveness of RME could be better judged if it was 

explicitly oriented toward whether ESA stocks are improving at acceptable rates. 

However, data variability may prevent clear evidence. Uncritical acceptance of 

the Corp’s AFEP Program may be inappropriate.

• Hatchery, RPA 182: The proposed actions should produce useful information. 

RPA 184: The Plan is not adequately developed to effectively address the issue of 

effectiveness of hatchery reforms and conservation hatcheries, especially to 

wild/natural fish. Neither what will be measured nor how the data will be 

evaluated are clear.

• Harvest: In terms of action effectiveness, the study of catch-and-release mortality 

will improve the knowledge base for incorporating harvest impacts into an overall 

status assessment for listed species. But the current plan is inadequate unless 

there is further justification presented as to why: 1) the other four harvest related 

RPAs were ignored, and 2) the issue of total fishing mortality through the life 

cycle of the species was not addressed. 

• Estuary/Plume: No. Objectives of habitat restoration are not stated clearly and 

specifically and goal statements are general and vague.

• Data Management: Considered as a “statement of the problems for data 

management” the Plan is clearly written and if the problems are solved, solutions 

would provide useful information. However, it does not have proposed 

“solutions.” The plan is a statement of intentions to develop a plan in the future. 
ISAB&ISRP 2004-1 RME Plan Review

9

Question 3: In context of the BiOp, has the Plan identified the most critical RME 

gaps? Are there additional gaps? 

• Tributary Habitat: The gaps identified are reasonable. However, the selected 

habitat metrics are extensive, dominated by physical-chemical parameters, and 

not prioritized; the ISRP/ISAB suggest more attention be given to aquatic food 

webs. Additionally, the context and goals for the proposed monitoring should be 

more clearly explored and presented. Methods to fill the gap in information on 

larger channels will need to be implemented as they are developed.

• Hydrosystem: The draft plan needs to be revised to take into account a new 

NOAA Fisheries white paper (Williams et al. 2003) 3 on salmon survival, which 

obviates much of the consideration of D and related factors in the current draft 

RME plan. Use of actual data is preferred over models such as SIMPAS. Damspecific survival studies are still needed to fix problems in spite of the RME 

plan’s emphasis on reaches and the whole hydrosystem. 

• Hatchery: Methodologies to synthesize existing data on the impacts of hatchery 

production on natural populations and detect effects at the population or ESU 

levels remain a critical gap. A conceptual model of expected or possible effects 

could facilitate this.

• Harvest: The assessment of harvest RME gaps is not adequate, nor is the selection 

of one RPA and metric well justified.

• Estuary/Plume: No, it is not clear that an assessment of critical gaps has been 

completed.

• Data Management: Not all critical gaps are addressed. The Plan neglects the 

obstacle to “Evaluation” posed by the facts that the list of indicators that will be 

derived from the primary “Research and Monitoring” data have not been 

finalized. Further, the policy decision thresholds according to which conclusions 

will be drawn from those estimates remain undefined.

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management? 

The ISRP/ISAB believe that coordination of data collection and data management 

could be improved by stronger initial and long-term strategic planning as the Plan 

is developed and implemented. Unless a specific coordination plan is developed it 

is unlikely that improved coordination will result. The plan should address how to 

ensure consistency of methods (data collection and management) and format 

across projects. It should also specifically address the process through which 

information needs of decision-makers will be understood, as well as the process 

for learning, adaptation, and revision across projects within a pre-determined, 

 

3

 Williams et al. 21 December 2003. Preliminary Draft: Effects of the Federal Columbia River Power 

System on Salmon Populations. NOAA Fisheries, Northwest Fisheries Science Center, Seattle, WA. 

www.salmonrecovery.gov/remand/analysis_reports/Tech_Memo_for_FCRPS_impacts_22_Decembe.pdf
ISAB&ISRP 2004-1 RME Plan Review

10

structured process and design. Coordination of data collection and data 

management should be an explicit part of the Plan, and these might best be 

developed and implemented at the pilot project stage. 

Sections of the RME Plan differ in attention given to data coordination. The 

Tributary habitat sections give little attention to the coordination problems 

inherent in the implementation of multiple, complex habitat recovery actions, or 

to the coordination problems that will attend data management issues. The 

Hydrosystem section would be improved if more attention were given to 

including metadata in PIT-tag data management and if data from radiotelemetry 

and acoustic telemetry were integrated with the PIT-tag data. The Hatchery RME 

requires that tagging and marking of hatchery fish be sufficient for their 

enumeration on the spawning grounds within specified statistical limits. Thus, the 

Plan should explain what methods will be used to identify which spawning 

populations, as well as which hatchery releases are adequately monitored and 

marked versus which need to be upgraded. The Hatchery and Tributary 

components of the Plan should be better coordinated to ensure the adequate 

enumeration of the hatchery-origin fraction on the spawning grounds. The 

scientists who will use the data to estimate abundance and productivity for natural 

populations should be active participants in choosing sampling locations and data 

collections. The section on Harvest does not address data coordination or 

management. The Estuary section addressing coordination is too brief and 

incomplete to assess whether it will improve coordination of data collection and 

management; the Plan continues to call for an oversight group, which does not 

indicate a fully coordinated process even with the estuary. 

Finally, in review of the Data Management section we endorse the NPCC and 

NOAA Fisheries agreement May 3, 2003, to move forward with development of a 

Columbia Basin Cooperative Information System (CBCIS). However, goals of 

this high level information management needs assessment document cannot be 

achieved by funding alone. Substantial policy issues must be solved to make 

significant progress. In the short term, it is critical that data can be freely 

accessed by all concerned in the region to enhance the best, complete scientific 

analyses. 
ISAB&ISRP 2004-1 RME Plan Review

11

Overall ISRP/ISAB Recommendations

The ISAB/ISRP recommend that the RME group take the following actions to improve 

the Plan:

• Strengthen the integration among the major sections of the Plan, to facilitate 

efficient and integrated data collection and to coordinate experimental design, 

sampling, and data sharing. 

• Base experimental designs and data precision targets on thorough analysis and 

synthesis of all pertinent existing data, and present these analyses and syntheses 

as part of the documentation for the rationale of the program.

• Explore what is needed to meet the basic objectives of the BiOp, rather than 

restrict design only to reflect what is dictated by the current specific RPA actions 

in the 2000 BiOp. 

• Develop an explicit plan for phased development and implementation. 

• Develop specific priorities, including core integrative or essential measures, that 

are based on an assessment of precision and power versus cost, need, and value, 

for elements of the Plan. (The proposed monitoring may be more than is needed 

and may include some infeasible elements or be missing critical elements.) 

• Include a conceptual model (or set of related models representing all of the H’s) 

to aid in coordination, prioritization, evaluation, and modification of regional and 

project-specific contributions. 

ISRP/ISAB Recommendation on Sections of the Plan 

Tributary:

• Expand the monitoring to include some attention to food resources for juvenile 

salmonids.

• Prioritize the extensive set of proposed measurements. 

• Consider alternatives to paired treatment-control experimental designs.

• Consider the watershed-level evaluation of action effectiveness.

• Re-evaluate the measurement precision targets for habitat parameters and 

biological response variables.
ISAB&ISRP 2004-1 RME Plan Review

12

Hydrosystem:

• Explicitly integrate the Army Corps of Engineers Anadromous Fish Evaluation 

Program (AFEP) studies at mainstem hydropower sites in RME planning rather 

than accept all studies without scrutiny. 

• Take a more direct approach and carefully define what needs to be estimated to 

establish if survival targets for adult and juvenile salmonids have probably been 

achieved, then explain in detail how those estimates can be obtained, including 

description of what will be measured where, what the design is, what the sample 

sizes of marked fish will be, and what statistical models will be employed. 

• Explore whether a 3% improvement in adult survival as specified in the BiOp is a 

measurable (and thus a reasonable and realistic) goal, considering variability in 

data. 

• Reconsider the current RME Plan’s focus on delayed differential mortality, extra

mortality, and related delayed mortality and latent mortality of smolts in light of 

the new NOAA Fisheries white paper (Williams et al. 2003).

• Incorporate metadata in PIT-tag information, and develop a plan for integrating 

radio- and acoustic-telemetry in data management.

• Incorporate project-specific as well as overall performance metrics.

• Avoid extrapolation of survival estimates and instead emphasize collection and 

use of direct data as shown in the white paper by Williams et al. 2003.

Hatchery/Harvest:

• Better coordinate Hatchery RME with other RME efforts, especially tributary 

RME.

• Develop an analytical framework to quantify the effects of hatchery reforms on 

the natural spawning populations.

• Develop the framework to assess ESU level viability based on the population 

level response metrics for evaluating hatchery reform.

• Include an explicit plan for marking sufficient fish to obtain needed overall 

performance metrics. 

• More thoroughly develop Harvest RME and better integrate it with the rest of the 

Plan. 
ISAB&ISRP 2004-1 RME Plan Review

13

Estuary/Plume: 

• State clear, specific objectives of the RME program and construct the program to 

meet those objectives. 

• Conduct and present an assessment of information needs, existing information, 

critical gaps, and priority of identified critical uncertainties. 

Data Management:

• Prioritize for the key essentials: that each data collection activity result in verified 

electronic data and metadata that are quickly available for use, and that each 

database system be housed in an organization that can guarantee data integrity, 

maintenance, access, and documentation. 

• Ensure that all electronic data and metadata are accessible by one of the current 

Columbia River Basin Fish and Wildlife Program funded databases, while 

moving to creation of a CBCIS.

• Use the pilot projects to refine methods and procedures of QA/QC, data transfer 

and entry, data access rules, metadata standards, and methods to link data for 

queries.

• Take advantage of existing, potential data centers.
ISAB&ISRP 2004-1 RME Plan Review

14

Comments on RME Plan Front Matter: Introduction, 

Overview, Project Implementation, Compliance Monitoring 

and Regional Coordination

General Comments

The ISRP and ISAB reviewed the document “Research, Monitoring and Evaluation Plan 

for the NOAA-Fisheries 2000 Federal Columbia River Power System Biological Opinion 

(Draft of 9/11/03, with section on Estuary/Plume of 9/30/03). The document states that it 

“defines an RME program that is limited to the specific requirements of the NOAAFisheries FCRPS BiOp” and that “Additional RME requirements of the U.S. Fish and 

Wildlife Service (USFWS) FCRPS BiOp for ESA-listed resident fish will be integrated 

with this RME program as soon as they are developed in coordination with resident fish 

recovery planning”. Integration with the Northwest Power and Conservation Council’s 

(Council) Fish and Wildlife Program (FWP) and coordination with other regional, 

Federal, state tribal RME programs are also noted as necessary and intended. 

Although the Plan repeatedly notes the need for regional coordination and integration 

with other programs of other action agencies and with the Council’s Fish and Wildlife 

Program, these are rarely dealt with explicitly in the Plan, which is cast as stand-alone 

and appears to have been written by relatively few authors from four federal 

organizations. Throughout the Plan, the need for coordination and integration is 

frequently noted as a hurdle. 

The Plan is cast as designed to support the BiOp evaluations of performance standards 

specified in the 2000 BiOp and to address uncertainties in the RPAs therein. As such, it is 

quite narrowly written and follows the RPAs apparently without much attempt to 

integrate and prioritize them. The ISAB/ISRP strongly recommend that the RME Plan be 

more generally and broadly cast, to meet the spirit of the BiOp and to mesh with the 

complementary regional RME needs and goals, rather than being tied directly and 

without obvious integration or prioritization to the 2000 BiOp RPAs. This could 

contribute to development of the needed regional coordination that the Plan notes, and 

should make the Plan a more robust long-term effort. While achieving improved 

coordination within the time deadlines of the BiOp Remand Decision is a tall order, we 

stress the need to vigorously address this important problem as quickly as possible.

The RME Plan states three core scientific principles to be emphasized: that RME data 

have known accuracy and precision, that RME actions be implemented within a basinwide experimental framework, and that data management be regional. The Plan identifies 

six central components as essential to meeting the requirements of the 2000 BiOp: 

Populations and Environmental Status Monitoring (abundance, trend, and condition of 

fish populations and key environmental attributes), Action Effe ctiveness Research (for 

hydrosystem and offsite mitigation actions on status measures), Critical Uncertainty 

Research (addressing key uncertainties in assessment of population survival), 
ISAB&ISRP 2004-1 RME Plan Review

15

Implementation and Compliance Monitoring (tracking execution of management 

actions), Data Management, and Regional Coordination. The emphases on defined data 

quality, experimental approaches, and regional organization are appropriate and laudable, 

but the ISRP/ISAB recommend that the Plan develop specific approaches and actions to 

meet these important goals. The data quality targets need more empirical justification, 

including formal analysis of data quality needs and of the costs, benefits, and feasibility 

of quality targets; the Plan needs to state explicit experimental designs and their rationale 

throughout; and specific processes to achieve regional involvement should be part of the 

Plan. 

The RME document includes an Overview, but this overview is presented separately for 

each section of the Plan and does not present a true overview of the entire Plan nor 

integrate the sections. The sections are organized according to the four Hs: Tributary 

population and environmental status and restoration action effectiveness monitoring 

(Habitat, 84 pages plus a 31 page attachment on methods), Hydrosystem RME Plan 

(Hydrosystem, 30 pages plus 3 attachments), Hatchery/Harvest RME Plan (Hatchery and 

Harvest, 20 pages), and Data Management Plan (21 pages). The Estuary/Plume section 

was delivered in a separate document (52 pages). These sections were written by 

separate working groups and no formal integration of the sections is given in the Plan; it 

appears that there was insufficient conversation or integrated planning among the groups 

that produced the separate sections. The ISRP/ISAB strongly recommend that the overall 

Plan be more closely integrated and that the conceptual and methodological framework of 

the integration be explicitly developed and stated in the Introduction to or Overview of 

the Plan. Specific suggestions for improved integration have been provided in an earlier 

ISAB report (ISAB 2001-7 “ A Review of Salmon Recovery Strategies for the Columbia 

River Basin”).

The Plan states that a project tracking system has been developed for programmatic 

reporting and will be developed further to accommodate requirements of the Federal 

Caucus Salmon Strategy; this is intended to assure that actions are executed as 

prescribed. 

Regional coordination is presented most often in the Plan as a challenge or hurdle and it 

undoubtedly is such. The Plan states that significant progress toward much-needed 

regional participation and coordination has been accomplished, and it lists the key groups 

that are involved in overlapping programs for overall RME and for specific components 

(e.g., Hatchery, Harvest, Hydrosystem). The Plan describes the sorts of coordination that 

are desired (integration of objectives, collaborations to improve power of studies and 

reduce their costs, coordination and standardization of data collection and reporting, etc). 

However, the Plan does not identify concrete actions to achieve these goals, and it should 

focus more effort on development of specific plans to promote and achieve needed 

coordination. For instance, pilot projects could include actions designed to build regional 

coordination; the Plan could pose workshops or other fora to inventory and analyze 

existing data, coordinate with Subbasin planning, and build efficient experimental and 

sampling designs for RME using these pieces of background information.
ISAB&ISRP 2004-1 RME Plan Review

16

Detailed Comments on the RME Plan Introduction and Overview

The Table of Contents should be expanded to include all the subsections so the reader can 

more easily find specific subjects.

The plan would be enhanced by including a glossary and list of acronyms, as the 

estuary/plume plan does.

Introduction 

How it is anticipated that having biological and physical performance standards will 

address uncertainties should be clarified (p.1).

The core scientific principles that are outlined in section B are good ones. Some 

discussion should be included as to how to move ongoing projects (the many that are 

listed in tables throughout the document) toward meeting those principles; that is, collect 

data according to statistically derived sampling designs, conduct controlled experiments, 

and develop data consistency across projects. 

P.5, the estuary plume plan avoided the use of the term “critical uncertainties,” reserving 

it for hydrosystem related uncertainties specified in the BiOp. Should the word (p.5, #4, 

3

rd bullet) be modified here? 

Overview of the BiOp RME Plan

P.7, under “ecosystem status questions”: it’s not clear why “distribution of adult 

salmonids” shouldn’t be under “population and habitat status monitoring questions”

Regarding the discussion of the questions to be answered by the Action Effectiveness 

Research plan: at some point, with multiple actions under experimentation, the question 

of effectiveness should also address cost-effectiveness. Actions that are comparable in 

terms of their biological effectiveness should be compared on the basis of cost. This has 

been a big piece missing from research under the Fish and Wildlife Program and other 

funding sources in the Columbia River Basin, and some mention of it in this document 

would be useful in raising the issue. 

P.9, 1st sent: “compliment” should be “complement”

P.11, para 2: some discussion should be included as to how the research conducted under 

the AFEP can be integrated with this plan. Is integration intended?

P. 12: It wouldn’t seem that shifting from survival indexing (outcomes) to a simple 

confirmation that passage management actions are being implemented (inputs) would 

satisfactorily address the question of effectiveness.
ISAB&ISRP 2004-1 RME Plan Review

17

P.18, top paragraph. Who is going to coordinate all these RME programs, and how will 

coordination be done? This deserves some thought and careful planning, and it’s not clear 

that there is a specific plan for coordination.

P.20: more detail as to the CBFWA project will be helpful, as the project has changed 

since the ISRP reviewed the proposal. What does it mean to say that the primary focus of 

the CBFWA plan will be development of technical products?

P.22: did SAIC make recommendations in more detail than those listed in Table 2.1? If 

so, they should be included. The 8 listed are too general to mean much, and they have no 

operational content.

P.22, in the list of coordination process groups, should the STT of the PFMC be 

included?
ISAB&ISRP 2004-1 RME Plan Review

18

A. Tributary

General Comments

In general, the RME Plan does an excellent job in weaving between the requirements of 

the BiOp, recommendations of the ISAB/RP, States and Tribes, and others. The status 

and trends monitoring component, in particular, incorporates many of the comments and 

recommendations that have been provided by the ISAB and ISRP on monitoring. The 

ISAB strongly supports the establishment of the three pilot studies in the Wenatchee, 

John Day, and Salmon subbasins and implementation of the RME effort. Furthermore, a 

lot of monitoring has already been done in these pilot areas, and we encourage the RME 

participants to build upon the previous work.

Answers to NOAA Questions

Question 1. If implemented, will the plan provide useful information about the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

Overall, the ISAB believes that the tributary RME plan is very well thought out and will 

provide useful information about the status of ESA listed salmon. The “ecosystem 

status” and “population and habitat status” questions address core issues that are needed 

for an effective monitoring program. The authors of the RME Plan have clearly taken the 

recommendations of ISAB 2003-2 (A Review of Strategies for Recovering Tributary 

Habitat)

4

 to heart.

We applaud the RME Plan’s emphasis on assessing juvenile as well as adult salmon. 

Certainly, both are needed but many monitoring programs emphasize adult counts at the 

expense of information on juvenile fish. Certainly, adult counts are fundamental to the 

monitoring program if lambda is to be the basic measure of success or failure, and we 

urge continued improvement in adult monitoring techniques. However, we believe the 

number of smolts per female may be the most useful overall indicator of habitat trends at 

the watershed scale over time, and we are pleased to see this metric highlighted in the 

RME Plan. Estimating juvenile abundance through the period of freshwater rearing and 

life-history stage specific survival rates should greatly improve the identification of those 

factors most responsible for controlling smolt production. Although it is important to 

include juvenile fish surveys in the pilot studies, we recognize that estimation of juvenile 

abundance in a large-scale status and monitoring program is difficult. This was Idaho’s 

experience in the Supplementation Study; estimation of juvenile abundance was dropped 

on many streams. In the Oregon Coastal Coho monitoring, pools are surveyed for 

presence-absence of juveniles, but stream-wide juvenile population assessments are not 

 

4

 ISAB 2003-2. A Review of Strategies for Recovering Tributary Habitat. 

www.nwcouncil.org/library/isab/isab2003-2.htm
ISAB&ISRP 2004-1 RME Plan Review

19

performed. We acknowledge that there is no easy answer, and juvenile monitoring of the 

kind needed to yield smolts produced per returning adult will be very difficult and 

expensive, even in the pilot areas. Despite these problems, juvenile abundance data 

should be included in the monitoring effort as this information may be the most valuable 

in terms of evaluating the effectiveness of tributary habitat actions.

The plan suggests that the accuracy of the probabilistic sampling scheme to estimate 

abundance of adult salmon will be verified by comparing it with values from the current 

adult protocol. It is naïve to assume that the standard for testing the sampling based 

approaches will be the ongoing census based surveys that will act as the ‘truth’ against 

which the sampling data can be compared. Remember Wayne Fuller’s rule number one: 

“The good old _____ are not that good”? Fill in the blank with “data.” The verification 

process must include a thorough assessment of the accuracy of the current protocol. 

Unless this assessment indicates that the current data represent an accurate, total count of 

adult fish, the verification process should not rely on the current census information. 

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

The combination of status and trend monitoring with more intensive investigations of 

cause-effect relationships in the Action Effectiveness Research (AER) watershed scale 

and project-based approaches should provide a clear indication of the response of salmon 

to tributary habitat actions. The status and trend monitoring was well developed in the 

Plan. However, we believe the long list of variables proposed for evaluation may 

ultimately be detrimental to the program. Monitoring of all the proposed parameters 

would be extremely expensive and would make the program a highly vulnerable target 

during times of tight funding. Restricting the variables to be measured to those that are 

most significant to fish, measurable with reasonable certainty, and most apt to be 

influenced by the management actions being applied would produce a more economical 

list -- one that would be better able to withstand the vagaries of budget cycles. The list of 

proposed monitoring variables should be assessed to identify economies in implementing 

the status and trends portion of the monitoring plan. 

Some additional thought as to how status and trends monitoring sites will be managed 

should be included in the Plan. For example, there may be a significant advantage to 

keeping the locations of the sites a secret. If the local managers in charge of 

implementing management actions and the public do not know the locations, they cannot 

introduce bias (unintentional or intentional) in the way the monitored sites are treated. 

Although it may be difficult to retain the anonymity of the sites, especially from local 

managers, maintaining secrecy may help eliminate a potentially serious element of error. 

Also, it should be acknowledged by the authors that repeated sampling may alter some 

sites sufficiently that they are no longer representative, an issue recognized by Scott 
ISAB&ISRP 2004-1 RME Plan Review

20

Overton, OSU. A process should be included for identifying and eliminating these 

“worn-out” locations and selecting replacement sites.

The more intensive, Action Effectiveness Research efforts are well conceived but several 

elements of this approach could compromise the quality of the information produced. 

Watershed-scale monitoring often occurs with little or no control over where, when, or 

what management and restoration actions will be applied. Control over where restoration 

actions are applied is required to maintain the integrity of reference sites. The success of 

the watershed scale effort, in particular, depends on some ability to restrict the 

application of actions within watersheds designated as controls. We recognize the 

political difficulties related to controlling the application of management actions in the 

Columbia Basin. Nonetheless, the importance of the information that can be generated 

by the proposed RME effort should be sufficiently compelling to evoke some basinwide 

recognition of the critical need for the establishment and maintenance of reference sites. 

At a minimum, a commitment to establish and maintain selected reference watersheds 

should be a component of the subbasin plans covering the three pilot study areas. Those 

developing subbasin plans should work with the RME project leads to select reference 

sites.

Where suitable treatment-control paired watersheds with similar potential productivity 

cannot be located, we recommend use of statistical association analysis 

(correlation/regression) and modeling with time and time x treatment interaction 

predictor variables. This approach may be a more feasible analysis plan for data that are 

likely to be generated. Maintaining reference watersheds in a condition that provides 

adequate controls for treated watersheds may be difficult over extended time periods. 

Even with the development of methods to remove hidden biases from observational 

studies (Rosenbaum 2002), it is unlikely that these paired watershed comparisons will be 

done with sufficient care over the necessary evaluation period, and the scientists 

analyzing the data may have vested interests in the outcome.

The analyses of action effectiveness also could be done by correlation-regression type 

testing and modeling, which would reduce some of the concerns raised in the preceding 

paragraph (but not eliminate them entirely). If it is unlikely that treatment or control sites 

can be maintained as such for more than a year or two, we suggest that the design of the 

action effectiveness monitoring be changed to focus more on analysis by a statistical 

association approach (see Roni et al. 2002). This approach would involve selecting 

watersheds or sites reflecting a range of management actions, and with a range of time 

since implementation of the actions. These sites would supplement the sites 

probabilistically selected by the EMAP methods in the status and trends monitoring. 

Regression modeling with time and the interaction of time with management actions as 

independent predictor variables may be the best method for analysis of the data, given 

that new management actions will be implemented during the course of the study. Our 

recommendation would entail a significant change in experimental design from a series 

of treatment-control experiments. It is critical that a clear plan for the analysis of RME 

data be developed before data are collected as the analytical method to be employed will 

dictate the manner in which the data is collected. Correlation/regression modeling cannot 
ISAB&ISRP 2004-1 RME Plan Review

21

provide definitive evidence of a cause and effect relationships in a single data set. 

However, correlation/regression modeling can provide solid conclusions given adequate 

replication over space and time with corroborating results. 

The tributary habitat section of the RME proposal places considerable emphasis on 

understanding whether or not specific types of actions are effective. We believe the 

focus on action types may hinder the evaluation of the overall effectiveness of tributary 

habitat restoration efforts in the basin. The response of salmon to a restoration project 

depends on both the action type and where the action is applied. The project-based 

approach to evaluating the effectiveness of types of actions will certainly highlight the 

significance of project location in determining biological response. However, the 

question of whether or not the process being used to select project location and type 

(project prioritization) is working cannot be answered with this approach. The question 

of the effectiveness of the processes being used to develop restoration plans is much more 

significant in terms of understanding overall effectiveness of tributary habitat actions 

than would be an improved understanding of the variability in biological response to a 

specific class of restoration actions. The watershed scale approach will enable the 

evaluation of whole restoration programs, including the process used to select projects. 

For this reason, we believed this approach is more promising than the project-based 

approach.

The authors may be overly pessimistic about their ability to separate the effects of types 

of actions on sub-population level responses in the watershed-scale approach. The very 

complete information the authors propose to collect on the fish populations should enable 

the estimation of life-history stage specific survival values. This information may help to 

identify the types of actions that are most effective. The objective of nearly all 

restoration projects is the improvement of some specific habitat characteristics, such as 

increased pool area or reduced fine sediment in spawning gravel. Generally, these habitat 

objectives can be associated with a single salmon life-history stage and the information 

on watershed-scale survival during this life-history stage will indicate if the actions have 

achieved the desired biological response. 

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional gaps it has not identified?

The list of habitat metrics to which fish populations will be compared is dominated by 

physical-chemical parameters, including a variety of geomorphic and water quality 

measures. For years this has been the traditional approach to relating fish populations to 

their habitat in streams. Unfortunately, we worry that this approach undervalues the 

importance of trophic considerations, i.e., the sources and abundance of food resources 

for rearing salmonids. This is particularly true at the landscape scale (e.g., HUC-7 

subwatershed or larger) where there has been very little research on landscape indicators 

of trophic condition. Available food resources can be altered by human activity in the 

same way that physical habitat and water quality can be altered, but changes in the 

trophic structure of streams have not received nearly the attention that physical-chemical 
ISAB&ISRP 2004-1 RME Plan Review

22

changes have been given. Analyses of macroinvertebrate communities are usually 

framed in terms of detecting water pollution; for example, the RIVPACS method 

described in the Monitoring Strategy for the Upper Columbia Basin is a sensitive 

technique (developed in Europe) for relating changes in invertebrate presence or absence 

to what would be expected in an “unmanaged” or relatively unaltered watershed. But 

while measuring macroinvertebrate community changes may involve state-of-the-art 

methods of water quality bioassessment, techniques such as RIVPACS or BIBI do not 

provide strong inferences about the relative productivity of watersheds for salmonids.

Productivity involves a very complex interaction between having a safe place to live and 

getting enough food to satisfy life cycle requirements. In a comparison of the production 

of stream-dwelling salmonids worldwide, Bisson and Bilby (1998)5 observed that the 

most productive streams often did not possess habitat attributes associated with “ideal” 

salmon and trout rearing conditions – cold water, heavily forested watersheds, and very 

coarse channel substrate. Instead, the most productive streams tended to be characterized 

by moderate temperatures, nutrient-rich conditions, relatively open channels, abundant 

aquatic plants, and low to moderate gradients. They speculated that these conditions 

favored the types of macroinvertebrates that constituted most of the food resources for 

rearing salmonids, and that trophic considerations merited an importance equal to the 

quality of the physical and chemical environment.

The ISAB does not believe that there is currently a simple, direct way of monitoring the 

trophic condition of streams in the context of understanding salmon population status. 

More research is needed. However, we do encourage additional experimentation with 

assessment techniques that include estimates of both aquatic and terrestrial 

macroinvertebrates, and the relationship between macroinvertebrates and salmonid 

survival and growth (the latter requiring multiple censuses annually). We encourage 

additional research on the relationship between nutrient concentrations, physical habitat 

conditions, and macroinvertebrate abundance. While it may be true that metrics of 

trophic condition are poorly developed, the importance of such tools in understanding the 

capacity of watersheds to produce salmon suggests that development of such metrics be 

given a high priority in the targeted gaps research program.

There is little discussion about how monitoring habitat and population status will be 

interpreted with regard to progress in achieving “desired future conditions”. We realize 

that specifying such conditions involves certain policy choices, but making such a large 

investment in monitoring suggests that the context and goals for monitoring should be 

fairly clearly spelled out. Presumably, population goals developed through the TRT 

process will constitute one target. The BiOp mentions habitat performance standards that 

usually center around environmental hazard thresholds for salmonid spawning or rearing 

(e.g., maximum percent fine sediment in spawning areas), but the RME Plan is relatively 

silent on these standards. It would be helpful for the RME Plan to address how desired 

future conditions will be identified and how monitoring data will be used to track 

 

5

 Bisson, P.A. and R.E. Bilby. 1998. Organic matter and trophic dynamics. Pages 373- 398. In: R.J. 

Naiman and R.E. Bilby (editors). River Ecology and Management: Lessons from the Pacific Coastal 

Ecoregion. Springer-Verlag, New York, New York, USA.
ISAB&ISRP 2004-1 RME Plan Review

23

progress toward achieving desired conditions. The ISAB 2003-2 Tributary Habitat 

Report recommends, for example, that habitat recovery be tracked as the rate of change in 

the direction of achieving the natural range of conditions that would likely occur in 

relatively unmanaged watersheds. If the RME Plan agrees with this approach, or 

something like it, some added explanation would be useful.

The final gap is one that the authors of the RME plan recognize and clearly state; the 

protocols and approaches being tested are specifically designed for wadeable streams. 

The ISAB/RP agrees that the limitation to these smaller systems is appropriate as 

adequate sampling protocols for many of the most important variables have not been 

developed for large channels. However, as data from the RME effort are compiled and 

analyzed, the caveat that some important parts of the channel network were not included 

in the probabilistic sampling framework should be prominent. Also, the RME effort 

should expand as proper protocols for these larger channels are developed. 

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management?

The RME Plan breaks tributary monitoring into four categories: (1) status and trends 

monitoring, (2) habitat restoration action effectiveness monitoring, (3) programmatic and 

implementation issues, and (4) data management. Of these four categories, most 

attention is given to status and trends monitoring and restoration action effectiveness 

monitoring. The coordination problems inherent in the implementation of multiple, 

complex habitat recovery actions are not given the same level of detail, nor are the 

coordination problems that will attend data management issues. In fairness to the 

authors, many of the problems involving interagency coordination and cooperation with 

state, tribal, and interest groups cannot be adequately addressed in this report. However, 

the challenges of coordination may add to the expense of the RME effort for tributary 

habitat. Thus the cost estimates provided may be less than will actually be needed to 

coordinate and implement data collection and management. Even the cost estimates 

provided suggest the daunting expense of such an ambitious program and will likely 

impose compromises in RME design. Both the RME Plan and Upper Columbia 

Monitoring Strategy call for widespread, expensive monitoring, but surely the resources 

needed to implement such a program will be inadequate to achieve every objective in the 

Plan. This will necessitate prioritizing monitoring activities. It would be helpful if the 

Plan identified what aspects of the monitoring efforts were considered most essential and 

would receive priority funding.

Detailed Comments

Page 38. The RME Plan correctly states that it is very helpful to know both the precision 

and accuracy of different methods for measuring fish populations and their habitat. 

Furthermore, the Plan proposes precision targets for different types of measurements. For 

example, it recommends that “controllable” variance components due to within year, 

among year, across crew, etc, sources be kept within a coefficient of variation (CV) of < 
ISAB&ISRP 2004-1 RME Plan Review

24

15% for population, biological condition (aquatic community indicators), and chemical 

water quality measurements, and < 25% for physical habitat measurements. We certainly 

agree that specification of precision and accuracy is worthwhile. However, the level of 

precision and accuracy required of a measure should be related to the degree of change 

expected of a parameter as a result of the application of management actions. A CV of 

15% or 25% may not be required if responses in a parameter are quite large. In addition, 

we are not sure that the specified CV targets can be achieved for the various population 

and habitat parameters. There are very few studies, unfortunately, that report precision 

error for many of the measures specified in the RME Plan, and we are not provided with 

citations that suggest these targets are really achievable. Furthermore, “controllable” 

variance might include within year variability (due to time of sampling) or inter-crew 

variability (due to differences in sampling techniques), but among year variability will be 

strongly influenced by annual climate differences and these are not controllable. We 

believe that (1) additional research is needed to establish realistic precision targets for 

each of the parameters in the Habitat Status Monitoring section, and (2) additional 

refinement of the targets is needed to distinguish controllable from uncontrollable 

sampling variation. Our concern is that establishing CV targets for population and 

habitat monitoring that range from < 15% to < 25% may be setting the bar excessively 

high. The authors may wish to check with Russ Thurow at the Forest Service Rocky 

Mountain Research Station Lab in Boise for up-to-date information on population 

variability.
ISAB&ISRP 2004-1 RME Plan Review

25

B. Hydrosystem 

General Comments

On the whole, this section of the draft plan has many good features and is a credit to its 

writers. It should serve as a good basis for a final document. We provide comments that 

are intended to assist in finalization. NOAA Fisheries requested that the ISAB and 

ISRP’s review focus on four questions. The following general comments are organized 

by the questions. 

Answers to NOAA Questions

Question 1: If implemented, will the plan provide useful information about the status 

of ESA-listed salmon species? In what ways could the plan be improved to provide 

better information?

The most pertinent usefulness issue seems to be whether the Plan will provide 

information that will assist managers in determining whether goals for recovery are being 

met. While the Plan will provide much other useful information, it does not appear that it 

can be used to determine whether the status of ESA listed stocks is improving or not. A 

general impression of the Hydrosystem section of the RME Plan is that it generates a 

checklist that proffers rather vague promises that every issue will be addressed. A more 

direct approach would have been for the authors to carefully define just what it is they 

think they need to estimate in order to establish that survival targets for adult and juvenile 

salmonids have probably been achieved, and then explain in detail how they intend to 

obtain those estimates. What exactly will be measured where; what is the design; what 

will be the sample sizes (of marked fish released); and what statistical models will be 

employed? There now exists enough knowledge about realistic ranges of expectable 

survival rates and expectable detection rates that full specification of the design and 

sample sizes will allow a very credible prediction of what precision will be achieved in 

the future estimates. Only then would we know whether the proposed monitoring is likely 

to answer the needs.

Question 2: If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

The first two sentences of the answer to question 1 also apply to this question. That is, a 

plan aiming more to the point would focus on ensuring RME work that would answer 

whether the status of the ESUs has improved or not. The present suite of efforts may 

accomplish this, but the Plan is not clearly focused on demonstrating it.
ISAB&ISRP 2004-1 RME Plan Review

26

There are several places in the Hydrosystem portion of the Plan where it is assumed or 

stated that projects supported through the Corps’ Anadromous Fish Evaluation Program 

(AFEP) process will constitute adequate hydropower RME for this plan, and will receive 

adequate review by that process, which is already in place. We are concerned about the 

deference paid in this RME plan to the Corps’ AFEP. Although other Action Agencies’ 

programs were scrutinized, all RME related to AFEP appears to be simply accepted with 

no scrutiny. In this draft plan all dam-specific RME is assumed acceptable because it is 

vetted through the AFEP review process. That may be a mistake. The ISRP is 

independently reviewing the AFEP. It has found that adequate review is unlikely to 

happen through the AFEP process, because there is generally not sufficient time for 

review of final study plans prior to their adoption. The ISRP’s yet to be completed review 

of the Corps AFEP process in the past few months indicates that the AFEP deserves as 

much scrutiny for overall RME goals as other programs, both for planning of research 

and monitoring and determining the effectiveness of actions. Because the AFEP 

dominates the hydropower side of RME, it should be included in this RME Plan. Perhaps 

it could be included as a “Phase 2” RME Plan rather than being included in this plan, 

with appropriate notes to that effect in this plan. However it is accomplished, the AFEP 

should be an integral part of this FCRPS RME plan. The ISAB/ISRP suggests that the 

ISRP’s AFEP review (due in early 2004) be considered a part of this RME review. 

With respect to adult passage and improvements in survival, va riability may thwart 

efforts to measure stock improvements, especially over the short term. The plan mentions 

annual variability in quality (precision and accuracy) of the estimates due to “correction 

factors”. In addition, we note that annual estimates may vary because of variation in the 

“true” value, due to changes in operations at the dams, river conditions and other factors. 

From the discussion in the Plan’s text, it appears that the requirement in the BiOp for a 

3% improvement in adult survival may be unreasonable and unrealistic, simply because 

of the impracticality of measuring that small an improvement. The same difficulty might 

occur for the target of 9 or 10% improvement in juvenile survival, as mentioned on page 

165, depending on experimental design and sample sizes. This subject deserves more 

discussion because it might be interpreted as a key weakness in determining the status of 

ESUs in the first place. On page 157 under the heading “Progress and Compliance Tests, 

ESU Coverage” the Plan states “This means that only qualitative evaluations may be 

possible at the check-ins for most ESUs.” It may be possible, using the methods 

described in the preceding part of the section, and the appended Smith et al. paper, to 

make a statement about how large an estimated improvement would have to be to 

reasonably conclude that it was real with a given sample size.

The December 21, 2003 (draft) NOAA Fisheries white paper (Williams et al. 2003) on 

effects of the hydrosystem on survival presents the best available synthesis to date and 

raises some serious questions that should now be addressed in the RME plan. The white 

paper acknowledges (p 53, Williams et al.) that “...the relative role of the hydrosystem in 

overall stock performance is still uncertain.” That is an important statement that should 

motivate the RME effort to analyze, more pointedly, why that uncertainty exists, and 

specifically what (and how much) new data can be acquired to resolve that uncertainty, as 
ISAB&ISRP 2004-1 RME Plan Review

27

well as how much of the uncertainty is likely to be unresolvable in the foreseeable future. 

The NMFS analysis also raises an important new issue—the evidence, not evaluated 

systematically heretofore, that there is a significant handling effect in PIT tagging that 

reduces the survival rates of the PIT tagged fish by a detectable amount. The conclusion 

of that draft document (p 53) is that “...although PIT tagged wild chinook salmon 

provide useful data when assessing the difference in return rates of different treatment 

groups, they do not provide good information on the absolute adult return rates of fish.” 

This requires further investigation, and its implications for the intended BiOp evaluations 

need to be explored in the RME Plan. This uncertainty likely merits some specific 

additional experimentation to quantify the effects.

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional important gaps that it has not identified?

We believe there are more crucial RME gaps associated with the Hydrosystem than 

estimates of “D”, which occupy center stage in this section of the Plan. This emphasis in 

the Plan springs from the BiOp itself. The rather confusing questions of D and related 

issues of extra mortality, delayed mortality, and latent mortality potentially caused by the 

hydrosystem have been clarified by the NOAA Fisheries draft white paper noted earlier 

(Williams et al 2003). The RME draft plan needs to be revised based on this paper.

Prior to reading the white paper, the panel was concerned about the whole RME effort 

related to D, EM, and related delayed mortality and latent mortality of smolts. Five of the 

ten actions listed in Table 4.1 deal with this aspect of the hydro-related issues that are 

discussed in the Plan. The ISRP/ISAB are painfully aware that the definition of PATH 

"extra mortality" (EM) was elusive, that the appearance of "differential delayed 

mortality" owing to transportation was vulnerable to uncertainties caused by substituting 

assumed values or extrapolations into places in the calculation where specific data were 

needed, and that reach and dam survival rates are quite variable from year to year. We 

also are gratefully aware that the mathematics and statistics of survival rate estimation by 

mark-recapture methods is very well developed and sophisticated (and there are people 

available who know these techniques inside and out). The white paper specifically 

proposes abandonment of methods previously used involving the SIMPAS model with its 

flaws and use direct mark-recapture methods in their stead to obtain survival estimates. 

Because adequate numbers of recoveries of marked juveniles are available since 1997, it 

is now possible to do so, and the white paper provides those estimates for outmigration 

years from 1997 to 2000. That paper includes analyses of the survival estimates thus 

obtained and concludes, among other things that, considering the high average return rate 

of in-river migrant spring/summer chinook (4%) in recent years, there is little or no room 

for expression of delayed mortality. To increase SARs for transported fish so that T/I 

would be estimated as equal to 1, would require a 6% return rate for transported fish, 

which is an unrealistic expectation. 

There are other information gaps that should be discussed in the Plan, and we outline 

them here. As noted in answer to Question 1, it would be helpful for the RME team to 

better address the basic question, “What kinds of research, monitoring and evaluation 
ISAB&ISRP 2004-1 RME Plan Review

28

studies need to be undertaken to help managers decide whether or not a stock’s status is 

improving?”

The RME studies would be helpful in making management decisions such as, “What 

percentage of juvenile migrants should be transported in a particular year?” The white 

paper notes that a very high proportion of migrants is transported (85%), but that the 

return rates vary from year to year and within a year, with fish transported early in the 

year showing the lowest return rates, lower than in-river migrants. The RME plan might 

be revised to address such issues raised in the white paper.

Satisfactory implementation of the RPAs spelled out in the BiOp, as is the focus of this 

plan, may not necessarily lead to recovery. The RME plan develops an “Action 

Effectiveness Research” category (see page 165), and defines it as follows: “This 

approach shifts focus from annual survival indexing, focusing instead on confirming that 

the suite of adult passage management actions prescribed in the BiOp are satisfactorily 

implemented.....”.. This raises the question whether a convincing argument has been 

developed to suggest that the list of actions in the BiOp will lead to recovery. This RME 

plan can contribute to that assessment.

If problem sites for survival are to be fixed, project-specific survival still needs attention. 

On page 149, at the third paragraph from the bottom, project specific survival is given 

short shrift. The text on page 150-151 continues this emphasis. Although the emphasis 

appears to be upon survival estimates through specified reaches, as spelled out in the 

BiOp, the result might be to fail to identify individual projects that are sources of 

significant mortality. Because actions have to be taken at the project level (even if 

implemented systemwide), where overall system performance is affected is very 

important. The need is for both overall system metrics and project-specific metrics to get 

the full picture. Combining transported and in-river survival estimates prevents 

identification of in-river factors that affect performance and the ability to fix them, which 

might allow a phasing out of the wholly artificial transportation process. 

The use of extrapolation of survival estimates has been questioned, as the Plan notes. The 

plan would be better, in our opinion, if it advised that the practice be abandoned, carrying 

with it the use of models, such as SIMPAS that attempt to substitute a modeling exercise 

for obtaining real data. The white paper (Williams et al. 2003) makes it clear that PIT-tag 

estimates of survival through the reach from Lower Granite to below Bonneville dam are 

now possible and provide better information. The ISAB has previously recommended 

that the PIT tag estimation procedure be adopted for the purposes described in the Plan. 

We learn from Williams et al. that this has only been possible since 1997 when adequate 

numbers of recoveries of marked fish have been recovered below Bonneville Dam. 

If models like SIMPAS are used, they should be well explained, including their 

limitations. On page 155, the Plan refers to the use of SIMPAS, which is proposed for 

estimating survival of juvenile salmonids during in-river passage in cases where 

empirical estimates are not available. It refers to an updated version of SIMPAS. 

Considering the importance of the missing data thus provided, it seems to be equally 
ISAB&ISRP 2004-1 RME Plan Review

29

important to provide a full description of the version of SIMPAS that was used. Perhaps 

this could be provided as an attachment similar to the two attachments already provided 

or as a reference to a source. In our discussion above, we have recommended 

abandonment of SIMPAS where it is used for the purpose of estimating relative return 

rates of transported fish and in-river migrants when real data are not available over large 

reaches or many steps in the model. It may be useful for evaluating relative survival 

expectations of fish exposed to proposed operational or engineering options.

Estimation of direct survival during transportation is called for on page 151 and is needed 

to fill a significant gap. The plan states that direct survival is presumed to be 98 percent, 

but no direct estimates are available. A study to provide a direst estimate will serve the 

purpose of replacing a questionable assumption that has been made. However, the design 

of such a study presents a significant challenge. On page 153 the Plan states that it is not 

clear which agency has responsibility for conducting a study of direct survival. The 

responsibility needs to be decided soon because this could be an important contribution. 

Further explanation of proposed mid-Columbia actions seems needed. On page 155 under 

the heading “Middle Columbia ESUs (Chinook and Steelhead)”, the Plan states “...stocks 

from this drainage were assigned a secondary position governed by opportunity in using 

fish dedicated for other purposes.” This statement demands a fuller explanation. 

Recovery of listed fish that originate in mid-Columbia tributaries above dams that are not 

part of the FCRPS is dependent to some degree upon actions of the non-federal 

hydrosystem, which may be beyond the scope of the FCRPS 2000 BiOp. However, some 

of these actions are taken in concert with NOAA Fisheries and the FCRPS. In fact, the 

Plan might make reference to the Habitat Conservation Plans (HCPs), that are agreements 

entered into by NOAA Fisheries with Douglas and Chelan County PUDs that specify 

survival targets for adult and juvenile salmonids at three of the five mid-Columbia 

projects and lay out a process for producing estimates for compliance. Approval of the 

mid-Columbia HCPs requires a federal action because these HCPs are intended to satisfy 

the requirements of the FERC, the Endangered Species Act, the Stevens-Magnuson Act, 

and several other federal laws. These agreements might be described in the Plan. 

In addition, FERC (a federal entity subject to coordination with NMFS with respect to 

endangered species) has in place requirements for spill levels that are designed to provide 

certain survival levels for juvenile salmonids at the two Grant County PUD projects. On 

the same page we read, “Empirical estimates of in-river survival from the first FCRPS 

dam encountered to BON tailrace are required.” We wonder what is the reason for 

limiting the estimates to the first FCRPS dam encountered? In fact, are not Grand Coulee 

and Chief Joseph dams, as well as the upstream storage systems, part of the FCRPS? In 

addition, do they not have an effect on survival of adult and juvenile salmonids? The 

application of flow augmentation originates from the federal storage system above Grand 

Coulee and Chief Joseph dams. Grand Coulee and Chief Joseph dams also contribute to 

problems with total dissolved gas in the mid-Columbia Reach. The serious shortcomings 

of this “first dam encountered” approach should be recognized. 
ISAB&ISRP 2004-1 RME Plan Review

30

It seems as though these matters should at least be discussed in the Plan, preferably with 

incorporation of survival standards where these are available, such as in the HCPs. 

Similarly, reference should be made to the BiOp developed for the offshore fisheries, 

which take place on the listed stocks under the provisions of the Stevens-Magnuson Act. 

It would be appropriate to include a brief summary of the incidental take provisions 

allowed there. The whole life cycle of listed salmon and steelhead should be included in 

the discussion.

Question 4: The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to improved 

coordination of data collection and data management?

There are significant data management gaps regarding the hydrosystem that are not 

addressed by the Plan. The current PTAGIS system for PIT tag detection data is a fine 

model for a shared database with high standards for data organization, excellent public 

access, and essentially universal and rapid acquisition of the detection data from 

experiments conducted by many different organizations and individuals. More programs 

should adopt this approach. Nevertheless, the PTAGIS system suffers from one very 

serious shortcoming (which we have pointed out before). It does not contain the required 

meta-data (i.e., supporting information) for each release group. Because of this lack, 

PTAGIS data users do not have an indication (from the database) as to which tagged fish 

were representative of that year’s wild or hatchery outmigration versus which fish were 

part of an experimental treatment that actually should be evaluated as a distinct statistical 

stratum. It is very telling that even a high level group of “insiders” namely the authors of 

the current NOAA Fisheries draft white paper (for the remand analysis) of the effects of 

the hydrosystem, had to acknowledge (p 14 of the 12/21/03 draft on the web) that their 

own analysis did not have access to the meta-data necessary for culling unrepresentative 

experimental release groups from the analysis.

A second very serious gap in data collection coordination and data management is the 

independence of the various radio-tagging and acoustic-tagging efforts from the PIT 

tagging effort. These data are not contributed to PTAGIS (or any other common 

database). For this reason, these data are not generally accessible for conducting the kinds 

of syntheses that are possible with aggregate of PIT tag data. The planning for the radiotagging and acoustic-tagging efforts tends to focus narrowly on very specific questions 

about survival or passage route at a particular dam, without much regard for the 

contribution that the data from these tagged fish will (or might) make to estimates 

addressing larger questions. As for PIT tags, these larger questions include overall

survival rates (especially as components of lambda for purposes of BiOp evaluation), 

transportation effectiveness, and possible effects of outmigration history on subsequent 

survival as reflected in the SAR). Since these larger questions may be the real bottom line 

in subsequent jeopardy determination or recovery evaluations, it is a loss if the 

investment in radio-tagging and acoustic-tagging does not add as much as it might to the 

effective sample size for resolving these questions. Given the relatively shorter effective 

life of these tags (compared to PIT tags), the seemingly more invasive nature of their 

application, and the different opportunities they present for subsequent detection, there 
ISAB&ISRP 2004-1 RME Plan Review

31

should be significant benefits if the planning for the radio-tag and acoustic-tag 

experiment were folded into the planning for PIT tagging, in a comprehensive way, and if 

the relative merits (and costs) of the various technologies were weighed appropriately in 

that planning effort.

Detailed Comments 

Page 143, Table 4.4: It would be helpful if there were an appendix or table that gave the 

full project title without acronyms. What is highlighted (see legend)—is it the capital 

X’s? Need to be clearer. There appears to be a missing link between Table 4.4 and the 

text. The table shows no “X” mark for RME 191, while the text says “This general survey 

indicates that all hydro RME actions are being actively pursued at some level and that 

every action except one (191) is being addressed by more than one research effort.” 

Perhaps the words “except one” were meant to appear earlier in the sentence. On the 

other hand, on page 146 it is said, “Action 191 involves expanding an existing COE adult 

counting program and does not require a specific project. These activities fall under the 

auspices of the established COE Fish Passage Program.” Elsewhere, it is suggested that 

counts might be extended into the winter and documenting fall back through the juvenile 

passage facilities, page 146. But the Plan also notes, under the heading “Physical 

Performance Standards” that “These standards are guidelines for operating the system. 

They include flow targets and spill schedules. The BiOp does not call for specific tests to 

determine compliance with the guidelines, nor does it call for additional mechanisms to 

monitor these beyond procedures in place. So this plan does not treat this further.” To be 

as transparent as possible this plan should describe or refer to sources for information on 

what those mechanisms for monitoring and compliance consist of. 

Page 3, heading for section B: A point of fact is spelled principle, not principal. All uses 

of these words need to be checked in the document, for the spelling is often wrong. 

Page 3, bottom paragraph, first line: tenses are way off in this sentence. It would be better 

to read: Monitoring data that lack accompanying...assessments are of far less...

Page 3, bottom paragraph, fourth line: there should be hyphens after sampling and error.

Page 3, bottom paragraph, last line: principle, not principal. 

Page 5, second paragraph, first line, principal, not principle. 

Page 5, item 3, why only critical uncertainties related to EM and D? There are certainly 

other uncertainties. Is it because they are spelled out in the BiOp?

Page 11, first line: We suggest that the term “hydro-corridor” perpetuates the notion that 

the mainstems are just corridors for migrations and not real habitats for migrating and 

rearing juvenile salmonids. Certainly, some salmonids do just zoom through, but even 
ISAB&ISRP 2004-1 RME Plan Review

32

then they need special habitat features to assist that migration. We suggest using 

migration habitat of mainstem rivers instead. 

Page 11, third paragraph: Again, why not other major uncertainties besides D and EM? 

Page 12, top bullet: We cringe at more transportation, but if the experiment will be 

useful, we support it. 

Page 12, first paragraph under Evaluation and Assessment: This paragraph is vague. Do 

you mean that a suite of tests can be assumed to give multiple measures that will instill 

confidence when they have similar results? In second line, procedure should be plural. 

Page 139, second paragraph: should health be added to survival? This would account for 

gas bubble disease monitoring and other condition metrics. 

Page 139, third and fourth paragraphs: In the context of the rest of the report, deference to 

the Corps’ AFEP program seems like a mistake. Focus on EM and D seems to be shortchanging other critical uncertainties. 

Page 140, next to last paragraph: It might be useful to briefly describe the “procedures in 

place” or show where they can be found. 

Page 142, table 4.3: Too many acronyms. Please define or spell out. 

Page 144, next to last paragraph: When you say there are four projects to generate inriver survival estimates, are you suggesting that they are duplicative, supportive, 

necessary? What is a reader to grasp from the apparent redundancy? Also, the 

project/proposal numbers don’t mean much without a key (see above comment). 

Page 145, top paragraph: What about adult counts at dams and the calculated betweendam losses? How accurate are they? Could they be calibrated with PIT tags so that the 

dam counts would be more reliable as well as having a long history? See also 4th

paragraph on this page. 

The proposed shift from radio tracking adult salmonids to estimate between dam losses to 

use of PIT recovery data, is a useful and economic shift. The PIT recovery data are 

needed for estimation of SARs in any case, and the in-river survival estimates can be 

obtained as a secondary benefit of the PIT recoveries.

Page 145, second paragraph: Why is it not possible to determine this? 

Answering the “why” is important for improving things. 

Page 145, third paragraph: Where is the information noted in line 3 given in this report 

(would be more useable with specific cross referencing). 
ISAB&ISRP 2004-1 RME Plan Review

33

Page 146, Action 191 Gap assessment: Dam counts and survival estimates need 

calibration with PIT tags (related to Action 192). But the data need to be evaluated, not 

just have detectors installed. 

Page 146, next to last paragraph: Will every reader know what “high-Q-detection” is? 

Page 147, top sentences. We would add that new ideas are needed. 

Page 147, first full paragraph: It would be useful to reproduce Fisher’s (2002) appendix 

table 8 here or in an appendix. Readers of this report shouldn’t have to track down Fisher 

2002. Also, they shouldn’t have to assume that the AFEP forum is adequate to vet those 

projects. 

Page 147, D section heading: It is confusing to start a heading with a one-letter acronym. 

This suggests a listing with this being D (of A, B, C, D). Better to start with words, like: 

Delayed Mortality (D; Actions 185, 186, 187). 

Page 147, last full paragraph: Why has the review of the Corps proposal not been done? 

Page 148, top: With all the current high interest in performance differences between 

hatchery and wild fish, is use of hatchery fish wise? 

Page 148, EM heading: same comment as for D heading. 

Page 149, Action Plan. A short paragraph introducing this section on “Action Plan” 

would be helpful for those who get lost in the details and are not sure what’s coming up 

next and why. 

Page 149, ESU-specific monitoring: need space after ESU in sentence. 

Page 150, first line of full paragraph: Table is 4.5. 

Page 150, Table 4.5: the heading might say currently estimated. H and W might be 

defined in the heading, as well as other acronyms. 

Page 151, top paragraph, fourth bullet: Wouldn’t you want to identify which collector 

dam? Lumping all collection does not allow discrimination of any effects between 

collection projects. 

Page 151, 2nd paragraph: Shouldn’t it say In-river estimation procedures...? (You don’t 

mean the results should be jiggered to agree, do you?) Or what do you mean? 

Page 151, next to last paragraph: ...estimates to be...
ISAB&ISRP 2004-1 RME Plan Review

34

Page 153, top paragraph, 4th line: It seems that the responsibility ought to be sorted out 

quickly, for this could be an important and timely contribution. 

Page 153, second paragraph: Reiterate earlier comment about combining in-river and 

transport data, to the loss of analytical capability and ability to actually fix anything. 

Page 153, Next to last paragraph: The method of Zabel (actually Smith and Zabel) ought 

to be briefly described here, with reference to the attachment. 

Page 153, bottom bullets: The ISAB and ISRP have been recommending for years that 

the detectors be started earlier and continued later. What’s the problem with doing so? 

Page 154, item 1: It would be clearer if you said ...from a combination of empirical and 

... (the combination seems important). The paragraphs that follow all seem good. 

Page 155, top paragraph, last line (the reference order for “respectively” gets lost—better 

to parenthetically say chinook and steelhead. 

Page 155, last line of paragraph headed by Populations monitored: “tghus”? 

Page 156, next to bottom paragraph: It is not clear that the paragraph that follow are a 

summary of Attachment 3. Might identify it as such (assuming this is so). 

Page 158, top bullet: Why is fallback considered only at BON and the terminal dam? 

Aren’t fallback rates at intermediate dams important, too? Terminal dam might be 

defined. 

Page 158, second paragraph: The estimates do not make use of many years of dam 

counts, either, apparently. Note that the last line should have detection not detected. 

Page 166, top. This project specific approach is what was referred to above as being 

needed rather than the systemwide approach. Fixes are at the projects, but add up to the 

system as a whole. 

Page 167, bottom full paragraph: The compiled secondary (calculated) data are important 

and need a repository. PTAGIS still needs to keep the primary data. 

Page s 169-182: The Attachment 1 by Steve Smith is an excellent update of the BiOp 

survival estimates (baseline) using current information. The adjusted values should give 

the best estimate of the baselines for improvements. Incidentally, “Attachement” is not 

spelled correctly in the footer (it’s Attachment). 

Page 183, 3d paragraph, line 2, insert hatchery before the second fish. Attachment 2 is a 

valuable contribution. 
ISAB&ISRP 2004-1 RME Plan Review

35

Page 188-206: Attachment 3 is a fine example of detailed thinking that is needed to 

determine progress (2005) and compliance (2010) with BiOp goals. It should be reviewed 

by a statistician to determine its technical accuracy. Conceptually, it seems fine. 

Page 190. top paragraph, second line: Should be ...are to be achieved on...

Page 190, bottom paragraph of text, line 7: “needed” should be indeed.
ISAB&ISRP 2004-1 RME Plan Review

36

C. Hatchery/Harvest

1. Hatchery

General Comments

The Hatchery portion of this RME plan is intended to fulfill BiOp RPA 182 and 184, the 

Harvest portion of this RME plan is intended to fulfill BiOp RPA 167. The RME plan 

addresses the complication that hatchery fish create when action agencies attempt to 

establish the status (abundance and productivity) of component populations of listed 

ESUs (RPA 182). It also addresses whether hatchery reforms can reduce suspected 

deleterious effects of artificial production activities on listed populations and whether 

conservation hatchery activities can contribute to recovery (RPA 184). The ISAB and 

ISRP agree that these topics cover the most pressing problems that hatchery programs 

generate for evaluating the effect artificial production has on natural populations and 

whether artificial production can contribute to recovery. 

Answers to NOAA Questions

Question 1. If implemented, will the plan provide useful information about the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

Hatchery related RME to provide information on the status of ESA-listed salmon species 

is contained in actions to fulfill RPA 182. These actions consist of “counting” hatchery 

fish on the spawning grounds as a component of habitat tier 1 or tier 2 population status 

monitoring and determining the relative reproductive contribution of natural spawning by 

hatchery-origin adults using existing studies and “new” projects as a component of 

critical uncertainty research (CUR). The technology (tagging and marking, genotyping, 

fish collecting) and analytical methodologies (statistical models and other quantitative 

methods) all are sufficiently developed to effectively address the RPA. If implemented 

this information will improve determining the status of ESA-listed salmon species. 

However, details of the actions proposed to fulfill RPA 182 need to be developed more 

fully to ensure that these studies provide the Columbia River Basin with useful 

information for determining the status of listed salmon species.

The “masking problem” -- the uncertainty in the abundance and productivity (lambda) 

estimates for natural-origin fish populations due to mixing with hatchery-origin fish -- is 

clearly stated. Describing the consequences of masking warrants additional 

consideration. The last sentence of page 208 states, “As a result, estimates of recruits 

per spawner for the naturally reproducing component of the population can be inflated.” 

This statement may be true but estimates could also be underestimated. If the number of 

hatchery fish spawning is low but there is an increase in the number of hatchery fish in 
ISAB&ISRP 2004-1 RME Plan Review

37

the next return, then the R/S estimate will be high. However, if there were large numbers 

of unmarked hatchery fish in the spawning population that have poor fitness and 

“normal” numbers in the return, then the R/S of natural fish will be underestimated.

If either the counting portion or reproductive performance of hatchery fish portion of 

RPA 182 actions are inadequate, then progress to reduce the uncertainty in status 

assessments will be compromised. ISAB and ISRP concerns about "counting" are 

detailed below in our answer as to how adequately the Plan addresses coordination of 

data collections. ISAB and ISRP concerns about evaluating the reproductive 

performance of hatchery fish spawning naturally follows.

For the reproductive performance of the hatchery-origin adults portion of RME actions 

for RPA 182, the suggested approach is to use DNA pedigree analysis generated in 

existing studies, as well as in a few new studies. Studies comparing hatchery-origin and 

natural-origin reproductive success in the Pacific Northwest are beginning to produce 

results. They reveal instances where there is little difference between hatchery-origin and 

natural-origin reproductive success and instances where there is considerable difference. 

At this time there is only conjecture as to the mechanisms accounting for these 

differences. We expect that spawning effectiveness would vary over time in response to 

a number of factors, and we do not know the full range of factors or the range of 

spawning effectiveness. One issue is how to control for inter-mating effects before 

implementation of genetic monitoring. Past interactions will definitely influence the 

relative reproductive success in each comparison. Consequently, the number and type of 

studies required needs clarification and more thought so the variation in this parameter 

can be estimated. 

For the purposes of fulfilling RPA 182 and improving listed ESU status assessment, the 

ramification of these observations is that differences between natural-origin and hatcheryorigin adults need to be evaluated using populations of the listed ESU contrasted with 

hatchery populations spawning with it. The uncertainty of the spawning effectiveness of 

hatchery-origin adults is unlikely to be resolved in a general way. 

For some populations of listed ESUs (i.e., Snake River fall chinook ESU and Lyons Ferry 

fall chinook hatchery population), the population sizes and physical setting of the 

spawning terrain make using pedigree analysis inappropriate. In those cases, using 

alternative analytical approaches for estimating the proportion of juveniles or adults 

produced by hatchery-origin adults spawning naturally might be required. If the 

programs were only initiated in 2003, then there will only be 1 to 2 data points for F2

adult productivity when the 2010 “check point” occurs. (There could also be a few more 

estimates if measured at the smolt stage.) If the data are this limited, then less rigorous 

measures over more studies (than are currently being conducted) could be considered. In 

those streams and studies where pedigree analysis is feasible, measuring reproductive 

difference in terms of F2 productivity does not consider any explanation of reduced 

reproductive fitness. A few detailed studies into the F3 generation with back-crosses 

would be wise.
ISAB&ISRP 2004-1 RME Plan Review

38

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

RME to provide information on the effectiveness of hatchery actions at improving the 

status of ESA-listed salmon species is contained in actions to fulfill RPA 184 - The 

Effectiveness of Hatchery Reforms and Conservation Hatcheries. In contrast to the 

actions outlined to fulfill RPA 182, which the ISAB and ISRP believe are nearly 

sufficient, the actions outlined for RPA 184 are not adequately developed.

The RME actions for RPA 184 is a review (listing) of existing studies of hatchery 

research. The purpose of the list is to facilitate a gap analysis of whether sufficient 

hatchery action effectiveness research is underway. The research is to determine: 1) if 

hatchery reforms will reduce the deleterious impacts of artificial propagation and reduce 

ESU extinction risk, and 2) if conservation hatchery programs can contribute to recovery. 

The RME plan states (p. 223): it appears that sufficient studies directed at the 

effectiveness of conservation activities are underway (item 2 above). No such assertive 

statement is provided that sufficient studies of hatchery reform are underway (item 1 

above). Rather, several issues were identified as gaps relating to the effectiveness of 

hatchery reforms in reducing extinction risk. 

The gaps identified are: 

• methodologies and analytical models to synthesize the studies of hatchery reform and 

transform the results of the studies into required improvements to lambda and 

extinction risk probabilities; 

• benefit/risks of steelhead kelt reconditioning; 

• studies of predation by hatchery steelhead smolts and by spring chinook hatchery 

smolts, and studies of short-term competition between hatchery and natural-origin 

fish for food and space in tributary spawning and rearing habitat.

A framework and methodology for evaluating hatchery reform is needed. At the present 

time deleterious impacts to listed ESUs from hatchery releases is only reasonable 

conjecture. No quantitative estimates of hatchery impacts to any of the metrics –

abundance, productivity, diversity, geographic distribution – that form the basis for 

determining the viability of a salmonid population is available. If there are no estimates 

of the impacts from artificial propagation, how can we measure the reduction of impacts? 

The ISAB and ISRP have serious concern about even the ability to measure a reduction 

in extinction risk. Is this even a reasonable starting point for an assessment?
ISAB&ISRP 2004-1 RME Plan Review

39

The ISAB and ISRP do not concur with the conclusion that sufficient studies of 

conservation hatchery activities are underway. The ISAB supplementation report6

concludes that the Columbia River Basin does not have an adequate experimental design 

in place to determine if “supplementation” will provide a demographic benefit and 

quantify the cost to natural spawning fitness from interbreeding between hatchery-origin 

and natural-origin salmon. There may be sufficient studies underway to develop the fish 

culture techniques for captive broodstock and captive rearing programs, but these are not 

studies of whether the products of these programs make a meaningful contribution to 

recovery. The RME plan could provide a vehicle for the Columbia River Basin to 

develop a large-scale experimental approach to evaluating supplementation.

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional gaps it has not identified?

Action 182 Performance Standards, Current Projects, Gap Assessment, and Action 

Plan. The performance standard for 2003 BiOp check-in is that adequate studies have 

been initiated. Table 182-a lists a number of projects investigating relative reproductive 

success of hatchery fish. A formal assessment of the sample design, methodology, and 

data collection is required before concluding that the projects underway will address the 

questions in the BiOp. The current implication is that all the projects in this table are 

equally good or acceptable. That implication needs to be verified. Spring/Summer 

chinook supplementation and steelhead supplementation in Idaho are listed as projects. 

Based on a recent ISAB review of these activities, these projects do not provide data 

appropriate to evaluate differential reproductive success of natural and hatchery-origin 

adults. It would be helpful if the document discussed the extent to which studies in a 

given geographic area are likely to be generalizable to another area. The same comment 

applies to the studies listed in Tables 184-1 and 184-2. Table 182-a only comments on 

half of the issue; there is no treatment of the “counting” problem itself.

Table 182-a should be complemented with another table of ESUs, the number of 

acceptable projects, project reference numbers, the number of populations included, 

methods, and number of years of data collected. 

“Plans for addressing gaps in Action 182” on page 214 needs to be updated. The ISRP 

and Workgroup have completed evaluations of new projects and some have been 

approved for funding. Although the ISAB appreciates that the RME plan is a fluid 

document, to be most useful throughout the Columbia River Basin, the RME plan needs 

to be very clear and transparent as to how individual existing and new projects serve to 

fill RPA 182 obligations.

 

6

 ISAB 2003-3. A Review of Salmon and Steelhead Supplementation. 

www.nwcouncil.org/library/isab/isab2003-3.htm
ISAB&ISRP 2004-1 RME Plan Review

40

Action 184 Performance Standards, Current Projects, Gap Assessment, and Action 

Plan. The performance standard for action 184 is that an unspecified number of studies 

be underway that are sufficiently precise that they can detect changes in survival of 

naturally produced fish (emphasized words added) as a result of reforms or conservation 

hatchery activities.

Twenty studies of the effectiveness of hatchery reforms are listed in Table 184-1 and 42 

studies of the effectiveness of conservation hatchery actions are listed in Table 184-2. 

The studies in these lists were evaluated to “identify apparent gaps in priority research” 

(page 215). Two categories of gaps were identified for evaluating the effectiveness of 

hatchery reforms, and no gaps were identified for conservation hatchery actions.

The first gap identified in category 1 is a keystone to evaluating hatchery reform. 

“Methodologies or analytical models for synthesizing the results and detecting the effects 

at the population and ESU levels of a myriad of hatchery reforms and conservation 

hatchery activities in terms of their effects on extinction risk or recovery”.

In fact, until these methodologies are established, reviewing the list of projects is 

unjustified. It is against these criteria for precision and methodology that studies will be 

evaluated to determine if they will produce useful information. On page 215 “evaluating 

reforms” lies in isolating the effect of the reform in a controlled study and quantifying it 

in terms of its effect on population viability. This criterion is not listed in Table 184-1, 

and the ISAB and ISRP are doubtful that many of the studies in the previous table even 

have this objective. Can they be directed toward addressing this objective? Is anything 

being done to promote data consistency across projects?

The text in the hatchery reform section of RPA 184 is unclear as to what they are going to 

measure and how that will be evaluated. P. 223: "As noted previously, most studies of 

hatchery reforms necessarily will focus on effects on individual lots of fish at a particular 

life stage." P. 224: "Action 184 studies should outline the method employed to isolate 

and estimate the effects of a particular hatchery reform or conservation hatchery activity 

on survival, and how it is proposed that these effects will be extrapolated to extinction 

risk and/or recovery of the affected listed populations or ESUs."

The appropriate study design will evaluate the effect of the reform on the abundance and 

productivity of naturally produced fish. From the statement that measurements will be 

made on individual lots of fish, the ISAB and ISRP are concerned that the focus of the 

measures is on how effective hatchery reform is in altering the survival and behavior of 

hatchery fish. For example does the use of acclimation ponds reduce residualism and 

straying, and do NATURES rearing procedures improve survival or behavior of hatchery 

reared smolts? These effects are important to measure, but they are not the measurement 

of interest in evaluating the effects of hatchery reform on a reduction to extinction risk. 

That measurement needs to assess the change in the abundance and productivity of 

naturally produced salmon.
ISAB&ISRP 2004-1 RME Plan Review

41

Projects like “NATURES rearing” and acclimation ponds measure the usefulness of these 

actions for altering hatchery fish survival or behavior (increased survival in the case of 

NATURES rearing and decrease straying in the case of acclimation ponds). They 

contrast the behavior/result of two hatchery treatments. There is no evaluation of 

whether NATURES rearing program or programmatic changes as a result of success 

(reduced hatchery production and release) have any impact on abundance, productivity, 

diversity, or geographic distribution of any potentially affected natural populations. 

Quantifying the impact of hatchery programs on natural populations requires evaluating 

the status of the natural population, not the hatchery population.

There is a general challenge in Columbia River Basin recovery planning to figure out 

how to aggregate individual population assessments into ESU level assessments. These 

are problems general to all four Hs, not just hatchery reforms. It is not clear from the 

RME plan that this subtlety is understood.

To recapitulate, before the projects in Table 184-1 can be assessed, the methodologies of 

analysis must be established. The ISAB and ISRP recommend beginning with a 

conceptual outline of the types of deleterious impacts anticipated – direct predation on 

emergent natural fry by hatchery smolts, competition, disease transmittal in tributary 

habitats, competition in the ocean, estuary, and mainstem habitats, genetic consequences 

of interbreeding – and then developing analytical approaches to measure these 

interactions. A first effort could involve the continued “mining” of the extant data. 

Recent work by NOAA Fisheries on the effects of brook trout on spring/summer chinook, 

competition in the ocean, and steelhead smolt releases in the Salmon River system are 

examples. Mark Chilcote (steelhead) and Tom Nickelson (coho) have had their 

regression analysis of hatchery spawning levels versus productivity published by CJAFS. 

These analyses could provide a provisional basis for setting performance thresholds for 

hatchery/natural interaction levels, i.e., proportion of hatchery fish that can spawn with a 

natural population, numbers of juveniles that can be released to compete, the number that 

can residualize, etc. Hatchery reform studies could then contribute by identifying any 

actions that could lower the incidence of interaction below threshold performance 

metrics.

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management?

Action 182 requires counting hatchery fish that are mixed with natural-origin salmon and 

steelhead on the spawning grounds. To accomplish this hatchery fish must be tagged and 

marked at a sufficient scale to be enumerated on the spawning grounds within specified 

statistical limits. The status of hatchery marking programs is not reviewed in the RME, 

and this is a significant omission. In the recent draft APRE report recruits per spawner, 

smolt to adult survival, escapement, and total catch are available for only 4.6, 35.6, 20.7, 

and 33.3% of Columbia Basin hatchery programs. In the September 2003 BiOp check-in, 

marking of “federally funded” hatchery releases for harvest is noted, along with a 
ISAB&ISRP 2004-1 RME Plan Review

42

statement “Work on a comprehensive marking plan continues”. An adequate 

comprehensive marking plan is required for RPA 182 actions to provide useful 

information on the status of ESA listed populations. There is a need to decide on the 

precision required for the abundance and productivity estimates in the status assessment. 

This precision will in turn dictate what monitoring is needed to count hatchery fish and 

the level of hatchery marking necessary. Once this is established, the methods that will 

be used to identify not only which spawning populations and which hatchery releases are 

adequately monitored and marked but also which should be upgraded need to be 

developed and made transparent. The details of these prerequisites would be best 

described in the RME document. If they are not sufficiently described in the RME 

(which they are not now), they should be added as a "gap" with an RFS or RPA identified 

to address the gap. 

The enumeration of the hatchery-origin fraction on the spawning grounds will be 

accomplished as part of tier 1 or tier 2 status monitoring described in the Tributary RME 

plan. The cohesion required between the Hatchery RME workgroup and the Tributary 

RME workgroup to ensure that the data collections are adequate is not evident in the 

RME plan. The scientists that will be using the data on spawning effectiveness of 

hatchery fish and the counting data to perform estimates of abundance and productivity 

for natural populations should not surrender the choice of locations and data collections 

on the “counting” portion of the task to the tributary habitat status monitoring effort. 

There needs to be annual assurance that the sites and precision of the estimates will be 

usable.

2. Harvest: Action 167. Improving estimates of incidental mortalities in 

fisheries 

General Comments and Answers to NOAA Questions

Question 1. If implemented, will the plan provide useful information about the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

No. The Plan’s response to the single Harvest RPA action 167 is a very limited 

consideration of harvest. It ignores the other four RPA actions, and as such, it presents 

an extremely narrow consideration of harvest. The BiOp has five RPA actions associated 

with harvest (see page 9-145 of 2000 BiOp): 

1. RPA 164 - selective fishing gear testing and development; 

2. RPA 165 - Fishery Management and Stock Assessment Models to address effects 

of selective fishing; 

3. RPA 166 - implement and enable changes in catch sampling programs for mass 

marking programs and/or selective fishery regimes; 

4. RPA 167 - incidental mortality, effects of selective fishing; 

5. RPA 168 - develop methods for crediting harvest reforms. 
ISAB&ISRP 2004-1 RME Plan Review

43

It was not apparent to the ISAB/ISRP why the other RPA actions were not addressed. 

Furthermore, in general, this response to Action 167 was minimal. There are incidental 

mortalities in ocean and coastal fisheries, plus there is an obvious issue of expanding 

mass-mark selective fisheries for hatchery-produced fish. Incidental mortality in the 

hydrosystem also occurs between enumeration sites and may well be attributed to fishing 

impacts (i.e. inter-dam losses), such as the possible “ghost nets” in Bonneville pool or 

other sites. The issue of relating incidental mortality to a listed species was not addressed 

at all. Are current programs collecting tissue samples to identify the stocks present 

during fishing, and how accurately can specific listed stocks be identified?

Because we are uncertain why the response only focused on RPA 167, we are uncertain 

of the expectations for harvest in general. In our opinion, this response would not be 

expected to provide the necessary information on harvest effects on listed species. 

Further, the standard for assessing fishing effects is the “effect on ultimate spawning 

(reproductive) success” (page 225). This assessment will be extremely difficult and will 

likely be a minor component of total fishing mortality. There is a need to ensure that the 

immediate and short-term mortality is estimated well (and first), and then consider how to 

assess the long-term impacts on reproductive potential.

An additional consideration is that this response refers to one research proposal in the 

basin. The ISRP has reviewed proposals for that research in the past and has provided 

extensive comment (most recently for the Mainstem & Systemwide project #35018). The 

committee, therefore, finds it very difficult to accept that “no specific gaps have been 

identified at this time”, in the Gaps Assessment section. Many other sections of the RME 

plan used tables of projects to outline activities, a more comprehensive presentation of 

projects and data requirements to meet the Harvest related RPAs (identified above) would 

be a useful addition to this section of the Plan.

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

In terms of action effectiveness, the study of catch-and-release mortality will improve the 

knowledge base for incorporating harvest impacts into an overall status assessment for 

listed species. As commented above, however, the current plan is inadequate unless there 

is justification presented as to why the other four RPAs were ignored. In addition, the 

issue of total fishing mortality through the life cycle of the species was not addressed. 

In our opinion, the Plan would be improved by consideration of how to incorporate all 

sources of mortality, i.e., from each of the four H’s, into a comprehensive assessment 

covering the entire life-cycle of the anadromous salmonids.
ISAB&ISRP 2004-1 RME Plan Review

44

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional gaps it has not identified?

The assessment of RME gaps related to Harvest is inadequate. As commented in the 

answer to the first question, the ISAB/ISRP was concerned about the focus of this 

response on the “effect on ultimate spawning (reproductive) success”. The response did 

not provide support for focusing on this issue relative to estimates of immediate and 

short-term mortality. Other issues limited to catch-and-release fisheries include 

cumulative mortality due to multiple encounters, and identification of stocks actually 

present in fisheries. (Other information needs were identified under the first question.) 

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management?

The response on harvest does not address any aspects of data coordination or 

management. This is of course a serious omission because there are obvious data 

coordination and collection issues associated with integrating information on harvest with 

other aspects for the Basin-wide RME plan. For example, what level of stock resolution 

must be used in monitoring harvest impacts for consistency with other assessment 

programs? Are there specified levels of precision and accuracy that sampling programs 

should be designed to achieve? Are the data being collected comparable with data being 

collected in ocean and coastal fisheries so that data collected within the Basin can be 

integrated with those fishery programs to provide a life-cycle assessment of all mortality 

sources? 

Detailed Comments

(from page 225 RME draft) “Action 167 is presented in Section 9.6.3.2.2 of the BiOp and 

states: The Action Agencies shall work with NMFS, USFWS, and Tribal and state fishery 

management agencies to develop improved methods for estimating incidental mortalities 

in fisheries, with particular emphasis on selective fisheries in the Columbia River basin, 

doing so within the time frame necessary to make new marking and selective fishery 

regimes feasible.”

The ISAB has underlined portions of this statement to emphasize the “action” 

components of the RPA, and that were commented on in the RME draft material. The 

RME authors note the need to estimate the incidental mortality, and that:

ß the “critical question relates to effect on ultimate spawning (reproductive) success,”

ß the purpose of action 167 is to improve estimates of incidental mortality rates for 

existing fisheries and to determine or verify rates in new or experimental fisheries, 

ß the Action Agencies are required to have initiated studies and/or developed methods 

by the 3-year check-in.
ISAB&ISRP 2004-1 RME Plan Review

45

Although the ISAB sees a basis for the second and third bullets (assuming there is some 

basis for the use of the three-year check-in), we do not understand the emphasis on 

“ultimate spawning success” in this response. This concern is heightened in the very brief 

Performance Indicator and Standards section that again emphasizes this point and 

reiterates that it must be measured with “sufficiently accurate and precise estimates as 

needed to make fishery management decisions in the context of listed fish.” While there 

was agreement on the need to study the potential long-term effects of catch-and-release 

fishing, the basis for emphasis on reproductive impacts is not provided. What evidence is 

there that this is the critical question, and what would be used to establish the 

performance standard? It is not apparent to the ISAB why there is not an equal emphasis 

placed on improved estimates of immediate and short-term mortality by stock. It is also 

not clear if current assessment models account for these mortalities rates and the potential 

for cumulative mortality effects through multiple fisheries.

Based on the current projects, the RME needs assessment and Gap assessment sections 

conclude that “no specific gap has been identified at this time.” This statement is not 

well supported since there is no evidence presented that the current programs are 

adequate. A table of projects and objectives should be provided as with actions 182 and 

184. What is the likelihood that these studies will meet the experimental standard defined 

above (i.e., assess impact on reproductive potential)? More detail could be given about 

the ongoing experiments with tooth-tangle nets in the Zone 6 commercial fishery. Is 

immediate incidental mortality being measured? How does this study relate to survival 

of incidental catch and subsequent reproductive success? How will the existing studies 

address this Action item?

The action plan for 167 also needs much more detail. Rather than wait for selective 

fishery proposals as they emerge, a more proactive approach would be to identify 

research needs and issue an RFP to address those needs. For example, within the context 

of a life history model and using current knowledge of mortality rates on listed species, 

how much incidental mortality may be allowable to achieve rebuilding goals and how 

precisely can incidental mortality rates be predicted?

In the Overview on Tributary RME there is the statement that “In order to track the status 

of a population, spawner escapement and removals en route to the spawning ground must 

be estimated.” This statement seems to ignore ocean harvest, especially for fall chinook. 

The RME Plan authors may be planning to use current estimates of ocean harvest, but 

they should at least discuss the accuracy and precision of the coded wire program on 

hatchery fish for estimation of the harvest of wild populations and ESUs. Also, they 

should be concerned about the incidental take and mortality of released spring-summer 

chinook and steelhead in the ocean. The ISAB continues to think that the coded-wire 

program is in trouble and questions whether estimates based on hatchery fish are 

representative of harvest rates on wild fish. 
ISAB&ISRP 2004-1 RME Plan Review

46

D. Columbia River Estuary and Plume 

General Comments

The overall structure of the draft Columbia River Estuary and Plume Plan is reasonable 

and provides a good framework within which to develop a plan, though fundamental 

pieces are missing. To assist in the development of this draft section into a completed 

plan for the Columbia River estuary and plume, we provide extensive comments by 

section. However, there is a significant organization issue that needs to be considered 

before addressing specific comments. 

The plan is structured around three objectives: status monitoring, action effectiveness 

research, and uncertainties research. The plan then addresses an Action Plan and a Risk 

Assessment dependent upon existing projects in the estuary and lower Columbia River. 

However, in Appendix A the authors list the RPAs that address the estuary and should be 

the focus of this report. At no point do the authors’ address the fundamental question of 

whether these projects actually meet the requirements under these RPAs. We agree that 

this plan should provide “data on the performance of the estuary program” and “increase 

the knowledge-base”, but in the absence of a specific assessment of the information 

needed to satisfy the multiple RPAs, the reader is left to assume that the existing projects 

are sufficient. The authors do return to this issue in consideration of the Action Plan, but 

we recommend that the report should begin with a consideration of what is required and 

why. In the absence of this background, the question of what is required to meet the 

RPAs pervades this entire plan, and any strategic consideration of the RPAs is not 

evident.

The reviewers were also concerned about the depth of review and comment relative to 

each RPA. The ISRP recently reviewed one extensive document addressing only RPA 

159 (Johnson et al. 2003, reviewed in ISRP2003-13). Three of the authors of this report 

were also involved in the previous report, so we can assume coverage of the RPA 159 

should be authoritative. However, this does draw into question how extensive the other 

projects were and how thorough the gap assessment was. This issue is commented on 

further in this text but specifically concerns the process used in assessing gaps.

Answers to NOAA Questions

Summary 

In general, the document provides a good framework within which to develop a plan, but 

it is lacking fundamental pieces. The reported timeline shows that many of these pieces 

are scheduled to be completed in Fall 2003; perhaps final review of the Plan should await 

these pieces. Even without the missing pieces, the Plan needs to tighten up the objectives 

to make them more specific by rephrasing them and expressing them in measurable terms 

where possible. There are many measurement, sampling, and analysis issues that are left 

unaddressed by the Plan. Because this plan does not explicitly consider the RPAs and 
ISAB&ISRP 2004-1 RME Plan Review

47

what RME plan would be necessary to address the RPAs, we must conclude that this 

draft is inadequate and must be thoroughly reassessed. Brief comments to the questions 

are provided but the detailed comments on the draft are expected to be of more value. 

Question 1. If implemented, will the plan provide useful information about the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

The status monitoring statement is too vague to be useful. Objectives should be 

rephrased, include more specifics, and include statements of specifically what will be 

measured and how. 

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

The assessment of effectiveness of actions is limited by the lack of a clear and specific 

statement of objectives for habitat restoration. Goal statements are very general and 

sometimes poorly worded. For example, the FCRPS goal is expressed as to “contribute” 

to the increase. As with status monitoring, action effectiveness objectives should be 

rephrased, include more specifics, and include statements of specifically what will be 

measured and how. While we like the structure of implementation, effectiveness, and 

validation as steps in the Action Effectiveness Research, the objective statements are 

confusing. 

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional gaps it has not identified?

It is not clear that a thorough assessment of critical gaps has been conducted. The section 

lacks a specific assessment of the information needed to satisfy the BiOp RPAs and does 

not address whether ongoing projects are sufficient to meet RPA requirements. Strategic 

consideration of the RPAs is not evident.

The section on uncertainty research is usefully organized around management questions 

that are being addressed by current programs, but some attempt should be made to 

prioritize these uncertainties according to the degree of importance for decisionmaking. 

Areas of critical uncertainty not being addressed by ongoing projects should also be 

identified.
ISAB&ISRP 2004-1 RME Plan Review

48

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management?

The section addressing coordination is too brief and incomplete to assess whether it will 

improve coordination of data collection and management. Unless a specific coordination 

plan is developed it is unlikely that improved coordination in the estuary will result. The 

plan should address how to ensure consistency of methods (data collection and 

management) and format across projects. It should also specifically address the process 

through which information needs of decision-makers will be understood as well as the 

process for learning, adaptation and revision across projects within a pre-determined, 

structured process and design. Within the estuary province, this plan continues to call for 

an oversight group, which does not indicate a fully coordinated process even with the 

estuary.

Detailed Comments

Summary 

This estuary/plume section needs to be reviewed after revisions. At present, this text does 

not seem consistent with main body of the text. For example, under Status Monitoring, 

the summary refers to listed salmon frequently, but there is no reference to any work on 

stock identification for identification of component stocks and members of each ESU?

Preface

The preface identifies the RME plan as a work in progress, and as such is incomplete. It 

is missing identified performance targets, a sampling design, methods for data collection 

and analysis, methods for monitoring, a conceptual model, a plan for data management, 

and a coordination plan. It does include a timeline for completing these elements, but as 

they are the central elements of the Plan their inclusion is central to having a workable 

plan. It is hard to review the document as a plan without these elements, and it is also 

seems that it is putting the cart before the horse to present performance indicators, 

monitoring variables, etc without having specified the end point (performance targets.)

As the document notes, actions recommended in this plan will be funded elsewhere, so it 

is particularly important that the basic components of the Plan be complete so that the 

overarching framework, rationale and identification of critical gaps in knowledge are 

clear to funding agencies. 

Contents

The overall structure of the Plan is logical and systematically developed. The glossary 

and list of acronyms are helpful.
ISAB&ISRP 2004-1 RME Plan Review

49

x. Glossary, Adaptive management ... reasonable definition but would prefer to include 

reference to a “structured learning process”, the text does not follow this definition very 

closely.

x. Glossary, Estuary ... includes river miles zero to 146, but many other documents will 

refer to estuary as zero to 46 miles, and then the tidally influenced river from 46 Rm to 

146 Rm. Why the estuary is so broadly defined in this section? There is a concern with 

most of the report about what environments are actually considered. Are the lower river 

tributaries included and considered?

Background and Purpose

The introductory paragraph lists RME components of RPA actions. This plan is designed 

to address the monitoring and research component. It would be helpful in the introduction 

to include some reference as to how the companion Columbia River Estuary and Plume 

RME parts (protecting and enhancing; compliance monitoring) will be carried out.

P1. Figure 1 and text, how is the Willamette River included in this definition of estuary? 

Is any of the lower Willamette included, and if not, why not?

P2, para 3: it is stated that the purpose of the Columbia River Estuary and Plume RME 

plan is to provide a scientific basis for the Columbia River Estuary and Plume RME 

program. Is this correct? This is a confusing statement, since the programs cited in this 

text are the Columbia River Estuary and Plume RME program. What is meant by this 

statement? Is it actually that this plan is intended to be the evaluation basis of the 

program and review against the BiOp tasks? Isn’t the purpose of the Plan to provide a 

strategic framework for conducting research, monitoring and evaluation? A plan is a 

decision tool and guiding framework as to how scientific information will be developed 

and used, but the Plan doesn’t provide a scientific basis as such. It is important not to lose 

the sense of the strategic guidance characteristic of a plan.

P2: Status monitoring, “monitor trends in status of the ecosystem” ... two questions come 

to mind: status relative to what? And how would you assess the status of an ecosystem? 

Ecosystems are dynamic and will change, so what are you to compare against?

P2: Status monitoring refers to “habitat tracking ...” This wording is not very clear, what 

is tracking? Should re-word to improve clarity in future text.

P2, footnote 2: even if the term “critical “ is too specific a meaning in terms of the BiOp, 

it is still necessary to have some way to denote when an uncertainty is “key.” Not all 

uncertainties can or even need to be resolved, so there needs to be a way to direct funding 

to the most important.

P3: it would be helpful to report annual average discharge for both pre- and post-dam 

time periods.
ISAB&ISRP 2004-1 RME Plan Review

50

P4, section 1.3: The description is confusing. What coordinates all these separate RPA 

actions? It sounds like there is a lot of duplication. Is there any process to assess the 

effect of all actions to determine areas of potential duplication?

P5, the description of the CRE&P in the first paragraph makes it sound as if performance 

standards are already developed. Is there intent to use these?

How will the COE’s General Investigations Study fit with the work proposed in this 

plan? The two would appear to be duplicative. Does the COE study have quantified 

objectives?

P5, bottom: more detail about the two categories of performance standards should be 

included here. 

P6, table 1: is “recovery” defined in the context of the performance standard?

Developing the performance standard – defined here as a specified numerical objective or 

target – would seem to be needed before the rest of the Plan is developed, not inserted 

afterwards. The process seems backwards. 

What is specifically the difference between performance standards and objectives, given 

the definition in Table 1? The plan treats the two as different. How were objectives 

developed without performance standards?

Are BiOp goals not quantified? This discussion of objective and goals generates the 

obvious question and concern about the specification of goals in the BiOp. This is again 

an example where these is clear need to relate this text back to the original objectives and 

requirements of the RPAs.

P7. 2nd para., the sequence of development applied in this text provides a logical 

framework for development of the Plan.

Section 2.0 Goals

P.9 Columbia River Estuary and Plume RME goals: Action effectiveness: to determine 

effectiveness the effects have to be assessed in relation to stated objectives for habitat 

restoration. Are these objectives clearly stated and where?

P.9 These statements are very general and sometimes poorly worded. For example, in the 

FCRPS goal, is the only goal really to just “contribute” to the increases? And what 

would a reasonable contribution be? In the Columbia River Estuary and Plume RME 

goals, the status monitoring statement is vague:

ß status ... relative to what, is it related to a starting point in time, or to a future 

goal?

ß trends ... is this simply a trend in a parameter over time, or does it involve 

evaluation of variability and the ability to measure a trend against this variability?
ISAB&ISRP 2004-1 RME Plan Review

51

ß listed salmon ... are the program goals only with respect to list salmon and if so, 

how will those listed stocks be identified?

ß usage and survival ... presumably if listed salmon used the estuary more but the 

survival rate did not increase, you would be more interested in the survival rate. 

So is the basic goal to increase use or just ultimately the survival?

ß survival ... do you mean to imply survival just in the estuary or overall?

Under Uncertainties, the goal is now to “resolve” ... this is a strong goal but differs from 

the program statement to identify uncertainties (page 2). This is just an example of the 

need for consistent clear goals in this text. “Uncertainties” should be “Uncertainties 

Research” to be consistent with other sections of the document. Should be “Resolve key 

uncertainties” rather than “resolve uncertainties”

Section 3.0 Conceptual Model

P11. to P13 Conceptual Model ... while we agree with the value of developing a 

Conceptual Model, this section does little to provide that or assist this plan. For example, 

Figure 5 does not fit the description of Figure 5 1n the 2nd para., page 12. Figure 5 is 

actually only labeled as a “Feeding sub model ...”

- last sentence 2nd para, page 12, “restoring” should be restoration

- para. Page 13, point (2), delete “of” from phrase

- last sentence page 13, spelling of capabilities

- p.13, typo: “trough” should be “through” 

- Figure 5 ... at examine survival should involve predation, not in the model.

Substantial work is needed on this Conceptua l Model section before it benefits the 

development of this Plan. This section doesn’t provide enough detail to be a useful 

conceptual model. Does the model as presented advance understanding or help to focus 

research or monitoring resources? 

Section 4.0 Status Monitoring

P15: The list of objectives, now phrased as questions, should be rephrased in standard 

objective format. E.g. Objective 1: To determine the ecosystem status of the Columbia 

River Estuary and Plume. The benefit of this wording is that is focuses attention on the 

need to quantify the elements of “ecosystem status.”

All objectives contain lists of “selected” or “included” variables. Which will be 

monitored? All those listed or a subset? 

ß Objective 2: the discussion should be more specific about which variables are to be 

measured to determine timing, location and presence.

ß Objective 3: The text indicates that methods to determine survival below Rm 146 are 

not yet finalized. For consistency, shouldn’t the same methods be used as above Rm 

146?

ß Objectives 4 and 5 should be specific about what variables will actually be 

measured.

ß What will be measured under objective 6? 
ISAB&ISRP 2004-1 RME Plan Review

52

P17-20: In Table 2, it is not clear which attributes will be measured. Will all of them be 

measured?

P15. Section 4.2 ... Objective 1 apparently describes “ecosystem” traits for status 

monitoring, but they are all just physical features. Aren’t there any biological features to 

be considered in an ecosystem assessment? Question 2 refers to biological features but 

only of juvenile salmonids.

P15. Section 4.2 ... Objective 2 now refers to “life history diversity, which is 

hypothesized to be important to salmon resiliency.” Presumably the authors are referring 

to observed phenotypes that use different environments as juveniles (life histories) but 

there will not be any direct proof that these are genetic traits or life history strategies. Is 

this a critical uncertainty that needs to be added to your text? Also, how is resiliency 

used in this context? It is never defined.

P16. Section 4.2 ... Objective 3 Survival. This will be the most difficult parameter to 

assess but it is not apparently how this objective will be monitored or exactly what will 

be measured. We need a clear statement of the required attribute so that how it is to be 

monitored can be assessed.

P16. Section 4.2 ... Objective 4. Water quality directly affects salmonid fish use ... but it 

may not directly affect survival. In extreme cases it could affect survival but presumably 

you do not intend to only monitor for extreme water quality issues.

P16. Section 4.2 ... Objective 5. Is this objective included because all tributary streams 

entering the lower 146 miles are included in the “estuary” definition and to be included in 

this Columbia River Estuary and Plume RME plan? If so, this needs to be made much 

clearer in the Study Area section ... and if not, then why does this objective refer to 

spawning areas?

P16: How can performance targets be developed in FY04 after sampling, data collection 

etc has already begun?

P18. Table 2. ID. SM2 ... a monitoring program does not monitor “Life history 

diversity”, the expression of these life histories, i.e., Juvenile distributions, would be 

monitored. We disagree that the Indicator is “Life history diversity” and the text for 

Rationale seems to equate spatial diversity with life history strategies. This debate is not 

needed in this table or for the monitoring program. An important observation from the 

monitoring program may be to prove this linkage.

P18. Table 2. ID. SM2 ... Indicator “Usage”. Is this indicator necessary? The three 

attributes can each be incorporated in “Juvenile Distributions” or in “Growth”. 

Residence time is an attribute related to Growth, and Spatial distribution and Migration 

pathways are both attributes related to Distribution.
ISAB&ISRP 2004-1 RME Plan Review

53

Section 5.0 Action effectiveness Research

The review committee questioned why flow management wasn’t considered a “primary 

management action?” This is a potentially important consideration that is omitted.

P21. Section 5. The same comments apply to the Action Effectiveness Research 

objectives as stated above for the Status Monitoring objectives. Objectives should be 

rephrased, include more specifics, and include statements of specifically what will be 

measured and how. While we like the structure of implementation, effectiveness, and 

validation as steps in the Action Effectiveness Research, there is some confusion about 

the objective statements. 

 

ß Objective 1 involves both individual projects and their collective role in meeting 

program goals. These are important but different aspects of implementation. The 

implementation of a project can be monitored, but the collective value of these 

projects to meet the overall goals requires a more comprehensive assessment than a 

monitoring program. This is actually undertaken in Section 7.0. Would it be clearer 

to have two sub-objectives associated with implementation monitoring? Also, it is 

not clear how adaptive management, as in the glossary, pertains to this objective. 

ß Objective 2 ... agreed

ß Objective 3 ... need to be clear on what estimate of survival is being validated. If 

the cumulative effect of projects is being evaluated, then the survival to be assessed 

should be survival through the estuary (as defined in the glossary). However, it is 

not clear that this is the evaluation standard. Further, is it the intention to associate 

survival trends to specific habitat restoration projects? This may never be possible.

Since the answer to the question posed as objective 3 is critical to determining 

effectiveness of habitat restoration actions, variables to be measured need to be 

clearly specified.

P22: what is meant by “directed analysis?” More specific description of the type of 

analysis should be provided. Is this somehow different from analysis? 

P22, under section 5.3, last sent of para 1: this is an interesting question, and if it is 

important it should be listed as an objective. 

P22, last sentence on page is too vague. There should be a better developed plan to ensure 

coordination of research.

P22. Section 5.3 1st para. The section on conceptual foundations never referred to habitat 

capacity, opportunity, and realized functions. Should move to that section and more 

adequately describe the basis for these terms (as described on page 24). 

P23. 1st para. Last sentence. “monitoring, cumulative effects, is not currently covered”

... this is a major shortcoming and severely limits our review of this Plan. 
ISAB&ISRP 2004-1 RME Plan Review

54

P23. Table 4. It is not clear why certain Attributes may address Opportunity and 

Capacity, but others that are very similar do not? For example, essentially all of AER 1 

could address both.

P23. Table 4. AER 3 Resilience ... what is this Indicator and how would it be monitored?

Foraging Success under the Growth Indicator ... how would this be monitored?

P24. The first paragraph and the box should all be before Table 4 so that the table is 

better explained. 

P24. Last sentence of first para. “If habitat opportunity and capacity are acceptable 

relative to historical levels, and realized function is acceptable according to monitoring, 

this may serve as a surrogate for measurement of the effectiveness of habitat restoration 

actions in the estuary.” While this may seem like a fairly important comment, it is so 

subjective and poorly defined that it cannot be assessed. What would it be a surrogate 

too, do you mean a surrogate for the cumulative effect of the projects? How determines 

what is acceptable?

P24 and P25. last two paragraphs. There is a serious concern for the use of historical 

conditions in this context. While the temporal component is important to consider, you 

may have very little control over temporal effects due to annual climate variation. 

Further, how would you ever know the “Historical Days per Year Available” in the 

equation (page 25)? Given the changes in the hydrosystem and estuary landscape, 

couldn’t you accomplish the same by standardizing to a flow level in the current hydrosystem, or to some “potential level” that is set as an interim goal? This may keep people 

focused on the current situation and what is achievable as opposed to the distant past.

P24: What is the specific meaning of “directed research?” As above, the use of these 

adjectives creates questions whether something different is implied.

P25, it’s not clear in the Plan why methods of data collection and analysis will not be 

developed until FY04. 

Section 6.0 Uncertainty Research

The organization of this section around management questions could be a useful 

approach. However, these questions seem to be the initial questions that the current 

programs are addressing but without any effort to identify areas not being covered? Is it 

possible to prioritize among these key uncertainties according to their relative importance 

to management decision making?

ß Uncertainty 2a: how would the sediment core research address the question of how 

hydrographic change relates to habitat function or quality? What indicators of 

function or quality would be measurable in the sediment?

ß Uncertainty 2b: As with the Columbia River Estuary Plan, more attention should be 

given to the meaning and relevance of “historic.” What would determine a condition 
ISAB&ISRP 2004-1 RME Plan Review

55

as historic? How about changes in habitat condition over time within the “historic” 

(by which is meant pre-European influence?) time period. What is the relevance of 

“historic” given current uses and settlement patterns?

ß The concept of “historical conditions” deserves more careful development. Given the 

dynamic system of the Columbia River Estuary, more consideration should be given 

of an historic range of conditions, of the key salmon-promoting properties of this 

range of conditions, and of the extent to which these properties are achievable today. 

ß Question 4 does not seem appropriate; surely the answer depends on the confidence 

required in assessing specific objectives, many of which will vary between projects.

Section 7.0 Action Plan

Section 7.1 Project-Level Assessment

P31, 7.1: How will the “action-level assessment” actually be done?

P31. Section 7.1.1 Project Inventory ... good idea but it is difficult to identify programs. 

Could an appendix be provided with the project title, lead scientists, abstract of proposal, 

and any cited publications? In Table 7, also include years of actual data, several are new 

projects that are not yet implemented or funded. Should also identify that two of the 

projects are in the plume and do not consider the estuary.

P31, last sentence under 7.1.1. “The Action Agencies are working to coordinate these 

projects to form a cohesive EP-RME program”: isn’t that what this RME document is 

supposed to direct? Is there a plan to ensure that this plan is used in that way? 

P32. This paragraph identifies an important omission that should be clearly identified in 

this report, but why is this not identified in the critical uncertainties research?

P36. Table 8 is a good presentation of coverage, but a suggestion is to separate existing 

programs from those that are New with No data yet. It is misleading to show that these 

new programs are on par with ongoing and productive programs. Caption Table 8, “but 

data still in pipeline” needs to be clarified. 

Cov seems to refer to Indicator and to Attribute, these assessments are not explained in 

the text, which conducts the assessment and assigns the values? With multiple attributes 

in an Indicator, how is the Cov value for the Indicator determined? It does not seem to be 

simply the lowest common value in the attributes.

P37. Table 8 Survival ... this assessment in particular is optimistic. Three of the six 

projects noted in this row are new with no data! P2 involves the plume only, and P3 can 

not yet provide an estimate of survival through the estuary. What is the basis of this 

“complete coverage” assessment? 

The general comment on Table 8 is that the re is no documentation of how the various 

assessments of coverage were determined. Who did the assessment, was it reviewed (the 

authors of the projects should not have been involved)?
ISAB&ISRP 2004-1 RME Plan Review

56

P38. Table 9. Similar concerns to above for Table 8. Uncertainty U2b relies on P23 but 

this is a bathymetry project in the outer estuary only. How does this relate to food webs?

Section 7.1.3 Risk Assessment

P39. Risk Assessment. Risk is defined as the likelihood that programs goals and 

objectives will not be met if pertinent data are not available. Is the latter phrase really 

necessary, does it add something to this definition?

P39. Status Monitoring, 1st para. While the risk assessment may be useful, this paragraph 

is not very clear. Is the first criterion asking “Could program goals be met with current 

knowledge?” And then, is the second criterion “How vulnerable is the success of a 

project regarding implementation and successfully acquiring the related data? 

P39. Table 10. In several rows, the assessment of vulnerability seems dependent on the 

assessment of Data Disparity, why would this be the case? These two criteria could be 

assessed separately and then scores added. The risk assessment does not refer to the 

extent of habitat disruption in the current environment or to species. It would seem that 

these should be considered in assessing risk to the RME objectives.

P39, 7.1.3 More detail is required as to how the “risk assessments” were conducted. Who 

did the subjective assessments? What methods were used? How sensitive are the results 

to those who did the assessments? I.e., would another group performing the same risk 

assessment be likely to replicate the findings?

P40. Table 10 has a lot of 5’s: how will indicators be prioritized among those with the 

same scores? The document should discuss how Action Effectiveness Research 

indicators can be prioritized given that most were determined to be at risk.

P40. and P41. Tables 11 and 12 are apparently to be completed. Tables 11 and 12 do not 

have numerical scores as does Table 10. What is the difference?

Section 7.2 Program-Level Assessment

P42. This section on monitoring oversight is quite brief and tentative. It should be 

developed to be more of a plan. Surely a missing point here is that the projects must be

considered within agreed goals and objectives, otherwise, why monitor? Generally this 

section is incomplete, makes broad encompassing statements that need not pertain to all 

projects, and really draws into question who is running this RME ship?

P42. Sec 7.2.2: How will consistency of methods (data collection and management) and 

format be ensured across projects? In order to produce data useful to decision makers, 

the document notes that the Estuary and Plume RME data team will have to understand 

the information needs of decision makers responsible for adaptive management and 

program evaluation. The document should discuss the process by which this knowledge 

will be acquired. It won’t happen without a systematic plan.
ISAB&ISRP 2004-1 RME Plan Review

57

P43. Sec 7.2.3 Adaptive Management: does adaptive management include adjusting 

program objectives as new information is acquired, or adjusting methods and approaches 

for meeting specified objectives? More detail on adaptive management should be 

included in this plan. What is the process for learning, adaptation, and revision? The 

protocols from the basinwide RME should be included here. This form of adjustments 

can be done, but this text is not consistent with the definition of adaptive management in 

the glossary. To be consistent, sampling program adjustments should be conducted 

within a pre-determined, structured learning process and design.

P43. Sec 7.2.4 Funding. Why is this included in this plan?

Section 7.3 Recommendations

The recommendations in this list should be more thoroughly developed with discussion 

of timing, constraints, potential problems and approaches. 

P43. Section 7.3.1. 2nd para. The suggestion of another monitoring site in the estuary is a 

decision within the RME project, why would they choose one in the estuary over any 

other? This again draws attention to how the tributaries in the estuary are being 

considered in the overall RME.

P44. Table 13. The value of this table is not clear. The implication seems to be that most 

recommendations are being addressed by existing or new projects. What is then gained 

from this table? Surely there are uncertainties that still need study or improved 

monitoring. The need for this table should be clarified.

P45. Recommendations. This section seems incomplete, as it is the messages seem trivial. 

For the first two, one needs to ask “for what?” The oversight group should exist now, 

and the last three recommendations are simply work tasks of the oversight group.

P45. Timeline ... again refers to need for an oversight group. The agencies and groups 

funded by all these sources are provided huge amounts of money. If they cannot organize 

an oversight group, then the region should reconsider where to recommend funding.
ISAB&ISRP 2004-1 RME Plan Review

58

E. Data Management 

General Comments

All sections of the RME Plan included a statement to the effect that “There is no formal 

database in place that houses all of the information necessary to generate annual 

production, productivity and recovery progress performance metrics for populations, 

ESUs and fish habitat. One must be established.” The ISAB and ISRP do not believe this 

is feasible or in the best interests of scientific analysis of the data in the short term, given 

the history of independent development of databases in the region, the independent nature 

of agencies and other players in the region, and (we believe) the need to move toward a 

system consisting of a mixture of distributed databases and information portals. However, 

the ISAB/ISRP endorse the overall recommendations in the Data Management Section 6, 

namely:

• Develop an overall RME information system architecture—a detailed blueprint of 

the design of the RME system.

• Take advantage of existing, potential data centers. Include information 

portals/distributed database-management system tools as necessary to consolidate 

data and communicate using the Internet.

• Develop a data management cost-sharing approach to achieve BiOp requirements.

• Promote the free exchange of information and development of a systems view of 

the Columbia River Basin.

The document correctly recognizes the obstacles posed by the facts that: 1) the lists of 

variables that will be measured have not yet been finalized, either in principle or in detail, 

and 2) there are significant institutional and policy issues that must be solved for effective

storage and retrieval of primary data. 

Answers to NOAA Questions

Question 1. If implemented, will the plan provide useful information about the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

Question 2. If implemented, will the plan provide useful information about the 

effectiveness of hydro, habitat, hatchery, and harvest actions at improving the status of 

ESA-listed salmon species? In what ways could the plan be improved to provide better 

information?

Given the nature of data management (i.e., data storage, retrieval and evaluation), we 

provide one answer to the first two questions.
ISAB&ISRP 2004-1 RME Plan Review

59

The data management plan attempts to address the data requirements for BiOp Actions 

179-199. Considered as a “statement of the problems for data management” the plan is 

clearly written, and if the problems are solved, solutions would provide useful 

information. However, it does not have proposed “solutions.” The plan is a statement of 

intentions to develop a plan in the future. 

There will eventually be some decisions about what variables will be measured in a 

monitoring mode in each of the other modules of the Plan: tributary habitat, hatchery, 

hydrosystem, etc. Hopefully, some consensus will be achieved about measurement 

methods, and the approved methods will be documented. Hopefully, some guidelines will 

be adopted for choosing the times and places where measurements are made, and this too 

will be documented. Realistically, there may be several different bases for choosing when 

and where to measure, but each should have a documented rationale and a detailed 

procedure for implementation, so it should be possible to associate each measurement 

with one of a short list of designs that it conforms to. At a minimum, then, an effective 

data management plan must guarantee that each data collection activity takes 

responsibility for getting its data recorded in some electronic database system in a timely 

manner, where every data entry will record, in defined electronically interpretable fields 

(i.e., NOT in comment fields): 

• What variable was measured (this description should include the medium in 

which it was measured);

• What value was obtained;

• What method was used for the measurement (referring to a common manual of 

approved methods, and the associated quality control and quality assurance 

procedures);

• The location where the sample was taken or the measurement was made, as 

appropriate;

• The date and time when the sample was taken or the measurement was made, as 

appropriate; and

• The design type (referring to a common manual of approved designs, and the 

documentation for how that design was implemented).

Question 3. In the context of the BiOp, has the plan identified the most critical RME 

gaps? Are there additional gaps it has not identified?

The document has identified some of the critical “Research” and “Monitoring” gaps 

elsewhere in other segments of this review. However, it neglects the obstacle to 

“Evaluation” posed by the fact the list of indicators that will be estimated (derived 

statistics) from the primary “Research and Monitoring” data have not been finalized, and 

that the policy decision thresholds according to which conclusions will be drawn from 

those estimates remain undefined. 

When it becomes apparent how the data are actually to be used, those responsible for the 

analysis will develop interfaces (and links between various distributed databases as 
ISAB&ISRP 2004-1 RME Plan Review

60

needed) to package the data in ways that are convenient for the analysis. If decision rules 

are adopted, that define how conclusions are drawn, this may suggest particular designs 

(including minimal sample sizes and minimal precision for some of the measurement 

methods) that provide a high probability of conclusive results, within whatever cost 

constraints are operative at that time. 

When the first attempts are made to draw conclusions, it may emerge that the data that 

have been collected up to that time will prove inadequate or excessive for the purpose. 

This will naturally lead to suggestions for revisions of the design (or even revision of the 

list of variables that are measured). The possibility of such iterative false starts should 

motivate all the policy-level players in this enterprise to attempt to crystallize their 

statistical analysis techniques and decision rules as soon as is feasible. When statistical 

procedures and decision rules are defined in advance, it is technically possible to analyze 

proposed designs and determine in advance the probability that the design will deliver 

conclusive results after some stated period of time. This gives added depth to the concept 

of planning, and is much encouraged.

Question 4. The plan describes the need for better coordination of data collection and 

data management throughout the Basin. If implemented, will the plan lead to 

improved coordination of data collection and data management?

The RME Data Management Group has recommended that the Columbian Basin 

Cooperative Information System (CBCIS, see remarks below) effort be the basis for the 

development of the needed RPA 198 action item and as the foundation for the extensive 

regional RME coordination necessary to achieve standardization of data collection and 

reporting protocols. 

The ISAB/ISRP endorse the NPCC and NOAA Fisheries agreement May 3, 2003, to 

move forward with the following actions relative to development of a Columbia Basin 

Cooperative Information System (CBCIS): to receive public comment on the CBCIS 

report, to propose a draft administrative structure for CBCIS, to identify a budget and 

cost-sharing agreement, and to approach the stakeholders about commitments for a 

CBCIS-style regional information system. We share the RME Data Management 

Workgroup’s position that goals of this high level information management needs 

assessment document cannot be achieved by funding alone. Substantial policy issues 

must be solved to make significant progress. 

The following overall conclusions from the SAIC report read as if they could have been 

taken directly from some of our ISAB/RP reports:

• There is no comprehensive, systematic, planned information system in the 

Columbia River Basin 

• There are no common protocols for field collection of data 

• The collected data are of variable quality and sometimes unknown quality 

• There is no single inventory of available data or the location, and the data are kept 

in many different locations in different database systems 

• Data analysis needed to create information is an inefficient and frustrating process 
ISAB&ISRP 2004-1 RME Plan Review

61

• When data can be located and extracted they often must be validated before they 

can be used 

• Data consolidation is needed but the data quality issues and disparate systems 

compound this problem 

• There are data gaps that are currently not being satisfied 

• No single organization or authority has authority or responsibility for fixing this 

regional data management crisis.

The SAIC report looks at the big picture; from (1) obtaining stakeholder buy in, to (2) 

establishment of an administrative framework for a CBCIS, to (3) setting of regional 

goals and objectives, and to (4) development of data collection protocols and metadata 

requirements. The recommendations are in one-sentence bullets with no detail on how to 

accomplish the desired end results. For example, number 19 is “Develop basinwide 

monitoring protocols and data standards addressing data collection, storage and analysis.” 

This one recommendation seems to cover everything that SAIC was charged with and 

everything that the Action Agencies are dealing with in the RME Plan. The 

recommendations in the SAIC seem to be complete, but the devil is in the details! 

However, if a CBCIS is implemented, then improved coordination of data collection and 

data management will occur.

In the short term it is critical that data can be freely accessed by all concerned in the 

region to enhance the best, complete scientific analyses. Each database system must be 

housed in some organization with enough stability to guarantee integrity, maintenance, 

common access and documentation. In addition, we recommend that before projects are 

funded, the proposals should include evidence that the resulting primary data and 

metadata can be electronically accessed by at least one of the currently funded Columbia 

River Basin Fish and Wildlife Program databases: StreamNet, Fish Passage Center, 

PTAGIS, Coded Wire Tag Recovery (CWT), Regional Mark Information System (RMIS) 

or Data Analysis in Real Time (DART). The Principal Investigators of projects, e.g., the 

pilot RME work on status and trend monitoring of tributary habitat in the John Day, 

Wenatchee, and Salmon Subbasins, might be responsible for and request funding to store 

their own data and metadata in a database under their control in the short term, but if 

complete scientific analysis is to occur, then the data should be electronically accessible 

by another primary (e.g., StreamNet) or secondary tier database (e.g., DART) responsible 

to the Council’s Fish and Wildlife Program. Evidence in proposals that standards have 

been developed for electronic transfer of information will help work out the necessary

methods and procedures addressed in the RME Plan.

Solutions for the big picture policy and funding problems will require some time and 

debate. In the meantime science can help make some progress at the grass roots level. 

We can endorse, e.g., the pilot programs for status and trend monitoring in the John Day, 

Wenatchee, and Salmon Subbasins and insist that they have common data collection 

protocols, procedures for recording metadata, etc. Having real data in hand (not 

hypothetical, from three states, collected by common protocols, and with assurance that 

these data are accessible by at least one of the Fish and Wildlife Program’s current 

databases) will of necessity help solve the problem of how to store and retrieve them.
ISAB&ISRP 2004-1 RME Plan Review

62

Appendix 1. Comments on “Monitoring Strategy for the Upper 

Columbia Basin”

Macroinvertebrates

Macroinvertebrate studies are divided into two types – “transport” and “composition”. 

The transport studies refer to the export of invertebrates and coarse organic detritus from 

small, fishless headwater streams to larger fish-bearing streams. The way in which the 

results of such a study would be used to assess the productive capacity of those streams 

that did contain fish could have been made more explicit. In addition, the presumed 

leader of this study, Dr. Mark Wipfli, has recently accepted a position at the University of 

Alaska at Fairbanks, so his ability to carry out this work is uncertain.

The “composition” type of macroinvertebrate studies appeared (as in the RME Plan) to 

emphasize changes in invertebrate community structure in response to altered water 

quality (“stressors”), but did not clearly address how such data would be used to monitor 

habitat restoration. It was also not clear how the RIVPACS method, the technique of 

choice, would be calibrated.

The monitoring strategy did not explain how macroinvertebrate studies would be used to 

address the issue of trophic conditions in streams, i.e., the relative abundance of food 

resources for rearing salmonids, or the changes in food availability that would 

accompany habitat restoration.

Adult Escapement

For species such as bull trout (not adult salmon and steelhead), snorkel surveys are 

recommended, with electrofishing being used as a secondary method for a subsample of 

snorkeled sites. Two questions need clarification: (1) how will electrofishing be used (if 

at all) in streams too deep to wade, but not to snorkel, and (2) what fraction of the 

snorkeled sites will be electrofished for calibration purposes? The latter question is 

important because electrofishing is generally more expensive to conduct than snorkeling, 

and carries with it additional logistical difficulties, e.g., transporting the fish shocker to 

the sites.

Water Temperature and Other Water Quality Indicators

Temperature monitoring focuses solely on daily and weekly maximum temperatures, 

presumably during summer. Diel thermal variation or winter minimum temperatures are 

not addressed, although these can be important to fish. It is relatively easy to maintain 

year-round temperature loggers at monitoring sites.

In addition to temperature, other indicators are recommended. These include turbidity, 

conductivity, pH, and dissolved oxygen. The monitoring strategy proposes that 

effectiveness monitoring include automated, continuous water quality loggers at the 
ISAB&ISRP 2004-1 RME Plan Review

63

upstream and downstream boundaries of treatment and control sites. Setting aside the 

difficulties with maintaining paired treatment-control sites (see above), the cost of 

implementing such an approach may be prohibitive. The Hydrolab DataSonde® 4a 

logger, recommended in the Plan, costs approximately $10,000-15,000 per unit, 

depending on how many types of sensors are included. If there were 15 treatment-control 

pairs (30 sites total, each with two loggers), costs for water quality monitoring would 

range from $600,000-900,000. Such a cost may exceed the budget for water quality 

indicators, and this would be strictly for equipment costs; data analysis would be an 

additional expense.

________________________________________

________________________________________

w:\em\ww\isrp\1 final isrp reports\isab&isrp 2004-1 rme plan review.doc