INDEPENDENT SCIENTIFIC REVIEW PANEL

RETROSPECTIVE 

REPORT 2007 

ADAPTIVE MANAGEMENT IN THE 

COLUMBIA RIVER BASIN 

ISRP 2008-4 

APRIL 11, 2008
Independent Scientific Review Panel

for the Northwest Power & Conservation Council 

851 SW 6th Avenue, Suite 1100 

Portland, Oregon 97204 

www.nwcouncil.org/fw/ISRP

ISRP Members 

J. Richard Alldredge, Ph.D., Professor of Statistics at Washington State University. 

Robert Bilby, Ph.D., Ecologist at Weyerhaeuser Company, an expert in riparian ecology (former 

ISAB member, began ISRP in FY 2008). 

Peter A. Bisson, Ph.D., Senior Scientist at the Olympia (Washington) Forestry Sciences Laboratory 

of the USFS. 

John Epifanio, Ph.D., Associate Professional Scientist at the Illinois Natural History Survey. 

Linda Hardesty, Ph.D., Associate Professor of Range Management at Washington State University. 

Charles Henny, Ph.D., Senior Research Scientist at the U.S. Geological Survey in Corvallis, 

Oregon. 

Colin Levings, Ph.D., Scientist Emeritus and Sessional Researcher, Department of Fisheries and 

Oceans. Canada. 

William Liss, Ph.D., Professor Emeritus of Fisheries at Oregon State University (former ISAB 

member, ISRP member in FY 2007, now Peer Review Group member).

Eric J. Loudenslager, Ph.D., Hatchery Manager and Adjunct Professor of Fisheries Biology, 

Humboldt State University, California. 

Katherine Myers, Ph.D., Principal Investigator of the High Seas Salmon Research Program at the 

School of Aquatic and Fishery Sciences, University of Washington.

Thomas P. Poe, M.S., Consulting Fisheries Scientist, an expert in behavioral ecology of fishes, 

formerly with the U.S. Geological Survey. 

Bruce Ward, Fisheries Scientist, Ministry Of Environment, Aquatic Ecosystem Science Section, 

University of British Columbia, Vancouver, B.C., Canada. 

Staff 

Erik Merrill, J.D., ISRP and ISAB Coordinator, Northwest Power and Conservation Council. 

2 
Retrospective Report 2007 

ADAPTIVE MANAGEMENT IN THE COLUMBIA RIVER BASIN 

EXECUTIVE SUMMARY.......................................................................................................................... 1

BACKGROUND........................................................................................................................................... 2

THE RETROSPECTIVE CHARGE.................................................................................................................... 3

RECURRENT THEMES FROM PAST ISRP RETROSPECTIVE REPORTS............................................................. 3

Monitoring and Evaluation................................................................................................................................. 4

Tributary Habitat Restoration, Monitoring, and Reporting of Accomplishments.................................. 6

Habitat Restoration Effectiveness Monitoring.................................................................................................... 6

Implementation Monitoring................................................................................................................................ 7

Effectiveness Monitoring ................................................................................................................................... 7

Intensively Monitored Watersheds..................................................................................................................... 8

Wildlife .................................................................................................................................................. 9

Artificial Production.............................................................................................................................. 9

Mainstem ............................................................................................................................................. 10

Ocean and Estuary .............................................................................................................................. 11

Project and Program Review Process................................................................................................. 11

Inadequate Reporting of Results.......................................................................................................... 12

2007 ISRP REVIEWS................................................................................................................................. 12

TABLE OF FISCAL AND CALENDAR YEAR 2007 ISRP REPORTS WITH WEB LINKS .................................... 12

ADAPTIVE MANAGEMENT.................................................................................................................. 14

MAJOR 2007 PROGRAMMATIC ISSUES ............................................................................................ 17

PRODUCTION............................................................................................................................................. 17

Progress on ResultsReporting ............................................................................................................. 17

Select Area Fisheries Enhancement Project (SAFE)........................................................................................ 17

Umatilla Initiative Review ............................................................................................................................... 18

Lake Roosevelt Kokanee Review..................................................................................................................... 18

ISRP Review of the Hungry Horse/Flathead Lake Project............................................................................... 19

Resident Fish Substitution: Challenges in Focal Species Management .............................................. 19

Challenges to Assessing Competition and Predation with Invasive Non-native Species..................... 20

Interspecific Hybridization as a Conservation Challenge for Resident Fishes ................................... 20

Supplementation .................................................................................................................................. 21

HABITAT................................................................................................................................................... 23

Subbasin Habitat Programs (John Day, Grand Ronde, Umatilla)...................................................... 23

Monitoring and Evaluation -- Next Steps ............................................................................................ 26

Regional Partnerships....................................................................................................................................... 26

Emerging M&E Technology ............................................................................................................................ 29

Benefitting Wildlife: Rethinking Crediting and Monitoring in the Wildlife Program.......................... 30

Prioritization..................................................................................................................................................... 30

Monitoring Approaches.................................................................................................................................... 30

FUTURE DIRECTIONS FOR THE PROJECT REVIEW PROCESS ................................................ 32

The Solicitation for Innovative Projects and a Role for Research Funding...................................................... 33

The Umatilla Initiative Review Process as a Model Approach ........................................................................ 33

Retrospective Report 2007 

ADAPTIVE MANAGEMENT IN THE COLUMBIA RIVER BASIN 

EXECUTIVE SUMMARY 

“However beautiful the strategy, you should occasionally look at the results.” 

 Winston Churchill 

“Failure is the tuition you pay for success.” 

 Walter Brunell

In 2007 the Independent Scientific Review Panel (ISRP) completed over twenty reports 

containing reviews of Fish and Wildlife Program projects. This Retrospective Report focuses on how 

projects are changing their objectives, strategies, and methods based on learning from the results of 

their actions. We accomplish this by looking at themes that emerged in previous ISRP retrospectives, 

examining a subset of projects that were reviewed in Fiscal Year 2007, and investigating how 

sponsors applied the results of their past projects to proposed future actions and monitoring. Some 

of our findings include: 

Effectiveness Monitoring 

Habitat restoration projects should include some form of effectiveness monitoring in their plans. 

This is essential to demonstrating that the project is achieving desired habitat results. Effectiveness 

monitoring does not have to be costly; in fact, showing some evidence of a beneficial habitat trend is 

usually sufficient for the ISRP’s needs. We do not in most cases expect individual habitat project 

sponsors to demonstrate target population benefits. However, demonstrations of population benefits 

at the watershed scale using approaches such as intensively monitored watersheds are always helpful. 

We do not wish to place an unfair monitoring burden on individual project sponsors, and we 

encourage collaboration among habitat restoration participants in documenting physical habitat 

improvements and population benefits. The ISRP continues to emphasize that each project without 

an effectiveness monitoring plan represents a lost learning opportunity. 

Artificial Production 

The ISRP understands that early in 2008 the Ad Hoc Supplementation Work Group will submit a 

final report outlining recommendations for evaluating the demographic and long-term fitness effects 

of supplementation. We recognize the significant progress made by the regional scientists and look 

forward to reviewing the report. We specifically anticipate that a plan for a comprehensive evaluation 

of supplementation is close to implementation. Such monitoring and evaluation should provide 

information to guide decisions on the efficacy of supplementation and its contribution to the 

restoration of salmon and steelhead. 

1 
Mainstem 

In 2007 the ISRP and ISAB completed a review of the Comparative Survival Study’s (CSS) Tenyear Retrospective Summary Report1, one of the largest M&E projects in the Fish and Wildlife 

Program. The CSS is a field study of the survival of PIT-tagged spring/summer Chinook and PITtagged summer steelhead through the Snake and Columbia River hydrosystem from smolts through 

returning adults, with a focus on relative survival of smolts with different migration routes (e.g., inriver, transported, different routes of dam passage, and different numbers of dams passed). For the 

most part, the CSS Ten-Year Retrospective was effective in answering the concerns posed by the 

ISAB’s review of the CSS 2005 Annual Report (ISAB 2006-3). It also provided improved clarity in 

the presentation and explanation of the sophisticated methodologies used in analyses of CSS data. 

However, the ISRP, with concurrence of the ISAB, found that one of the major project 

objectives (Upriver/Downriver Comparisons) did not meet scientific review criteria, because of 

inevitable confounding from other factors in establishing cause(s) of upriver/downriver differences 

that may be detected, regardless of sample size and detection power that could be achieved. 

Project and Program Reviews 

The ISRP’s major effort for the year was conducting follow-up reviews to address unresolved 

issues from the review of 540 FY 2007-09 proposals. In addition, we reviewed proposals seeking to 

test innovative technologies and methods; completed a retrospective review of results reporting in 

FY 2007-09 proposals; provided a potential framework for future project reviews; and with the ISAB 

jointly reviewed a handbook of salmonid field protocols. In this Retrospective Report, we examine a 

subset of projects that were reviewed in FY07, and assess how the project sponsors learned from 

their past actions and, based on lessons learned, proposed future actions and monitoring. 

Adaptive Management in the Fish and Wildlife Program 

Almost without exception, project sponsors state that they use adaptive management to modify 

the tasks and work elements in proposals. At the same time, however, project proposals almost never 

provide (1) an experimental design to identify whether biological objectives have been met by 

employing specific strategies or (2) a decision tree that would be used to modify management based 

on updated scientific information. 

Often project sponsors propose to continue to employ tasks and work elements even when 

monitoring data indicates that biological objectives are unattainable because of environmental and 

biotic conditions. Sponsors of these projects are understandably reluctant to abandon these efforts in 

which they have invested much time and energy. A structured or integrated decision management 

process, linking the biological requirements needed to sustain fish populations together with the legal 

and cultural setting, is one approach to create the human dimension required to confront unrealistic 

biological objectives. Exploration of decisions and alternatives requires clear problem definition, 

establishing agreed objectives, development of alternatives over several iterations, and evaluation of 

consequences. The latter requires stakeholder and expert input, aided by simulation modeling. 

 

1 www.fpc.org/documents/CSS.html 

2 
BACKGROUND 

THE RETROSPECTIVE CHARGE 

In addition to scientific review of individual fish and wildlife projects funded by the Bonneville 

Power Administration, the 1996 Amendment to the Northwest Power Act charges the Independent 

Scientific Review Panel (ISRP) to annually review the results achieved from prior year expenditures. 

The Council’s 2000 Fish and Wildlife Program further defines the retrospective review charge, stating 

that the ISRP’s report should focus on the measurable benefits to fish and wildlife made through 

projects funded by Bonneville and previously reviewed by the ISRP. The ISRP’s findings should 

provide biological information for the Council’s ongoing accounting and evaluation of Bonneville’s 

expenditures and effectiveness in meeting the objectives of the program. The Program also states 

that the ISRP should summarize its province review findings and identify the major basinwide 

programmatic issues articulated within the province reviews. 

The ISRP most recently summarized the majority of its major basinwide programmatic issues 

and province review findings in the programmatic section of its FY 2007-09 project review report 

(ISRP 2006-4a2). Those findings were in addition to those identified in the 1997-2005 Retrospective 

Report (ISRP 2005-143), which summarized almost a decade of project reviews and attendant 

programmatic issues. Thus, the ISRP’s FY 2007-09 programmatic comments combined with the 

1997-2005 Retrospective Report constitute a thorough treatment of scientific issues that have arisen 

from the project reviews. Because basinwide scientific review issues had been so recently 

summarized, the ISRP’s 2006 Retrospective Report (ISRP 2007-14) focused on the extent that 262 

ongoing projects in FY 2007-09 reported physical habitat and biological results, summarized data 

analyses, and identified adaptive management actions. In the 2006 retrospective and 2007-09 project 

review, the ISRP identified reporting of project results in proposals as one of the significant 

deficiencies in the program. This 2007 Retrospective Report logically follows these earlier 

retrospective reports and focuses on how projects are changing their objectives, strategies, and 

methods based on the results of their actions -- adaptive management. The ISRP accomplishes this 

by looking at themes that emerged in previous retrospectives, examining a subset of projects that 

were reviewed in Fiscal Year 2007, and investigating how they applied the results of their past actions 

and monitoring to proposed future actions and monitoring. 

RECURRENT THEMES FROM PAST ISRP RETROSPECTIVE REPORTS 

The ISRP provides this abbreviated summary of recurrent themes from past retrospective 

reports and programmatic summaries of project and subbasin reviews to identify for readers the 

major findings of earlier reviews without requiring readers to obtain and read the original documents. 

For those themes where the ISRP is aware of updates on the status of efforts by the basin comanagers to address these challenges, the ISRP provides here a short statement on progress. For 

those themes that arose during the 2007 ISRP reviews, an expanded discussion of the themes, 

including adaptive management considerations, constitutes the body of this 2007 ISRP Retrospective. 

Themes without a progress statement may be actively being addressed by the basin co-managers, but 

the ISRP is not knowledgeable about the status. 

 

2 www.nwcouncil.org/library/isrp/isrp2006-4a.pdf 3 www.nwcouncil.org/library/isrp/isrp2005-14.htm 4 www.nwcouncil.org/library/isrp/isrp2007-1.htm 

3 
Monitoring and Evaluation 

Perhaps, the most dominant recurrent theme through all the ISRP’s past retrospective reports 

was the lack of adequate monitoring and evaluation in fish and wildlife projects throughout the Fish 

and Wildlife Program. 

In its first retrospective report (ISRP 2005-14), the ISRP noted that development of a 

systemwide monitoring and evaluation program was in a formative stage with three relatively new 

initiatives. First was the Collaborative Systemwide Monitoring and Evaluation Project (CSMEP), 

which was a Fish and Wildlife Program project administered by the Columbia Basin Fish and Wildlife 

Authority (CBFWA). Second, the Federal Action Agencies proposed a draft Research, Monitoring, 

and Evaluation Plan. Third, a cooperative monitoring and evaluation program in the Pacific 

Northwest was established by an ad hoc partnership of biologists from concerned federal, state, and 

tribal agencies under the name Pacific Northwest Aquatic Monitoring Partnership (PNAMP). The 

need to develop a coordinated, systemwide monitoring and evaluation program has been recognized 

by the ISRP from our initial reviews to the present, and the ISRP continues to recommend that the 

Council support the effort. 

Progress: CSMEP has reviewed the strengths and weaknesses of data to provide estimates of the Viable 

Salmonid Population (VSP) parameters (abundance, productivity, diversity, and geographic distribution) in 

representative subbasins. Using the information from this review they developed several alternate monitoring and 

evaluation designs for the Snake River for status and trends of Chinook salmon populations. CSMEP developed 

status and trends, harvest, hatchery, habitat effectiveness, and monitoring integration workgroups. Each workgroup is 

developing options for monitoring metrics important to the management of Columbia River salmon. The ISRP recently 

reviewed the CSMEP 2007 Annual Report and Snake River Pilot Study Reports and found that some elements of 

the large project are making good progress (e.g., tributary habitat) while others (e.g., hatchery and harvest effectiveness 

monitoring) will require additional planning.5 

Beginning with programmatic comments in the first ISRP review of project proposals, the ISRP 

recommended that smolt-monitoring, PIT-tag, radio telemetry, coded wire tag, and sonic tag projects 

should be subjected to a comprehensive programmatic review that gives special consideration to the 

complex interactions between projects. This review was considered critical because regulations 

requiring mass marking of hatchery fish and selective fisheries have significant implications for the 

integrity of the data collected by the coded wire tag program and for the analysis and interpretation 

of management uncertainties addressed by the other tagging programs. 

Progress: Some concerns about tagging projects have been addressed by the ISRP data management review 

(ISRP 2000-3), the ISAB Harvest Report (ISAB 2005-4), and ISAB and ISRP reviews of the Comparative 

Survival Study (ISAB/ISRP 2007-6, ISAB 2006-3) and reviews of other mainstem uncertainties (i.e. 

transportation, spill, latent mortality) (ISAB 2007-1, 2006-1, 2). In July of 2007 the Council assigned the ISAB 

and ISRP to jointly conduct a review of fish tagging projects and objectives in the Columbia River Basin. That review is 

currently underway. 

In the second retrospective report (ISRP 2006-4a), the ISRP recommended that multi-year 

projects should be required to report both project implementation results and measurable ecological 

benefits at agreed-upon milestones, or annually, as a condition for continued funding. Future 

 

5 www.nwcouncil.org/library/isrp/isrp2008-1.htm 

4 
proposals and the BPA database should be linked to enable reporting of biological results in addition 

to task completion. 

The second ISRP report also suggested establishing a statistical support facility to provide 

assistance to projects that have limited statistical expertise. The facility would answer questions about 

design and analysis and provide workshops on statistical topics of common interest within the 

Program. By “facility” the ISRP meant a group, pool, or list of individuals or agency staff with 

statistical expertise who are available to project sponsors to assist in the development of monitoring 

designs or analysis of data. 

Progress: On an ad hoc, informal, basis, CSMEP is providing statistical assistance to project sponsors who 

have joined the CSMEP network but do not have “in house” statistical and monitoring design expertise. It is too early 

in this effort to identify whether it can serve a portion, or the entirety, of the statistical support required by project 

sponsors. In addition, Bonneville funds the University of Washington through the Statistical Support for Salmonid 

Survival Studies project (1989107006) to provide statistical support to Columbia Basin investigators requesting 

assistance with the design and analysis of tagging studies. The purpose of the project is to ensure that the maximum 

information can be reliably extracted from tagging studies funded under the Council’s Fish and Wildlife Program. This 

service has been primarily used by a small group of investigators monitoring salmon passage survival but has recently 

been used to develop a creel survey and related monitoring for Lake Rufus Woods rainbow trout stocking efforts. 

Highlighting and further developing and coordinating the support offered by CSMEP, the Statistical Support project, 

and other potential efforts to cover a wider range of project types and sponsors could help meet this need for a statistical 

support facility. 

In its FY 2007-09 programmatic recommendations, the ISRP also recommended that the 

Council and BPA abandon the fixed 5% monitoring and evaluation cap for habitat projects and the 

implication that M&E funds be limited to compliance and implementation monitoring. The ISRP 

agreed there should be cost-effective M&E at a sustainable funding level, but the 5% cap in the 

solicitation is an arbitrary policy decision – that is, it may be too much for some projects and too 

little for others. Scientifically sound M&E is very project and issue specific, and monitoring budgets 

should reflect this reality. 

Alternative metrics for evaluation may be required for some administrative and educational 

projects. Fish tagging projects could be measured by the utility of the data to be used to estimate the 

vital statistics used in harvest and escapement. 

Progress: The ISRP (ISRP 2007-14) provided additional suggestions for developing performance metrics for 

administrative, regional coordination projects. 

 

6 Throughout this report, the names of Fish and Wildlife Program projects are followed by the projects’ nine digit BPA 

project number in parentheses. Generally, the first four digits indicate the year the project began. These numbers are useful 

because they can help the reader easily access the proposals and reviews of proposals in the Fish and Wildlife Program’s 

proposal database maintained on the Columbia River Basin Fish and Wildlife Authority’s web site at: 

www.cbfwa.org/projects/default.asp. For the UW Statistical Support proposal go to: 

www.cbfwa.org/solicitation/components/forms/Proposal.cfm?PropID=121 

5 
TRIBUTARY HABITAT RESTORATION, MONITORING, AND REPORTING OF 

ACCOMPLISHMENTS 

In the first retrospective report (ISRP 2005-14) it was the ISRP’s expectation that selection of 

habitat proposals in the future will be determined in part by their conformity with Subbasin Plans. 

Such an approach is a logical consequence of the Subbasin Planning effort, and indeed, validates the 

work and analysis conducted by many hundreds of individuals throughout the Columbia River 

subbasins. This requisite should make reviews by the ISRP more manageable and transparent, and 

reward efforts that tie projects to the Plans. 

In the programmatic comments for the 2007-09 proposal review (ISRP 2006-4a) the ISRP 

recommended the Council pay close attention to the implementation of tributary dam removals in 

the Columbia Basin (e.g., Condit, Marmot, and Hemlock) and ensure, perhaps through targeted 

RFPs, that dam decommissioning and post-removal effects are properly monitored. The ISRP also 

suggested that the Council should consider using the Columbia Basin Water Transaction Program’s 

criteria to evaluate proposals for improving irrigation system efficiency to preserve instream flow. 

Progress: The Council recommended and Bonneville budgeted FY 2007-09 funding for the US Forest Service’s 

Hemlock Dam Removal project (200707700). A companion WDFW project, Adult Steelhead Monitoring in Trout 

Creek (200721500), to evaluate the effectiveness of the dam removal and to count adults was not recommended for FY 

2007-09 Bonneville funding. The ISRP is not aware if this monitoring is funded through other sources (cost share was 

not shown in the proposal). As recommended earlier by the ISRP, this emphasizes the need of the Underwood 

Conservation District’s Wind River Watershed Restoration project (199801900) to orient its ongoing monitoring and 

evaluation efforts on detecting potential changes resulting from the Hemlock Dam removal. Although not a Fish and 

Wildlife Program Project, the Marmot dam was removed in the Sandy River Subbasin, and the USGS and others are 

monitoring the effects.7 

Regarding the ISRP’s recommendation to evaluate water conservation projects’ contributions to preserving instream 

flow, the Columbia Basin Water Transaction Program and Council staff evaluated the set of FY 2007-09 proposals 

that included water conservation actions. However, the ISRP does not know how that information was used by the 

Council and Bonneville in making project funding decisions. The Water Transaction Program continues to work with 

Bonneville on developing criteria and an evaluation approach for water conservation projects. 

Habitat Restoration Effectiveness Monitoring 

Over the last several years the ISRP has used a simplified terminology consisting of implementation

monitoring (was the project implemented as planned?) and effectiveness monitoring (did the project 

achieve the desired results?). We realize, however, that some practitioners have used several terms to 

describe the different types of habitat monitoring, in which effectiveness monitoring is divided into 

two categories – effectiveness (did the project achieve the desired habitat benefits?) and validation

monitoring (did the project result in benefits to the target population?). We want to make it clear that 

the ISRP’s assertion that all habitat projects deserve some level of effectiveness monitoring does not

mean that population monitoring should accompany every project. Because of the limited geographic 

scale of most habitat improvement efforts, determining the population benefits of individual projects 

is usually impossible. The effects of habitat restoration projects on populations are much better 

addressed using the intensively monitored watershed approach where experiments are applied at a 

scale appropriate to assess biological responses. 

 

7 See http://or.water.usgs.gov/projs_dir/marmot/index.html

6 
When we ask that habitat projects include some form of effectiveness monitoring we mean that 

post-implementation conditions should be monitored to establish whether the physical characteristics 

of the restoration site are trending in the desired direction. For example, if the project involves 

fencing a riparian area to exclude livestock from sensitive streamside vegetation, it is very helpful to 

know how the vegetation is responding to livestock exclusion. We acknowledge that some forms of 

passive restoration (such as fencing) take years to produce desired results, and we therefore have 

recommended some of the monitoring tools that can simply be used to track improvement over time 

(e.g., photo points). Active habitat restoration actions such as large wood placement in streams or 

culvert replacement produce much more rapid improvement, and these improvements (e.g., 

documentation of expanded fish distribution) can be measured over a much shorter time period. 

The ISRP’s 2006 Retrospective Report (ISRP 2007-1) provides a detailed list of both 

implementation and effectiveness monitoring metrics for habitat projects (see pages 12-19). The 

tables for effectiveness monitoring actions include suggestions for both habitat and population level 

monitoring. To avoid confusion, we emphasize that project sponsors should include some form of 

habitat improvement monitoring in their plans. This is essential to demonstrating that the project is 

achieving desired habitat results, and it does not have to be costly – in fact, showing some evidence 

of a beneficial habitat trend is usually sufficient for the ISRP’s needs. We do not, in most cases, 

expect individual habitat project sponsors to demonstrate population benefits. However, 

demonstrations of population benefits at the watershed scale (e.g., improved egg to smolt survival) 

are always helpful, and where individual project sponsors are in a position to contribute to 

watershed-scale population monitoring those contributions will be most welcome. Additionally, 

documentation of physical habitat improvement can always be provided by other monitoring efforts 

as long as there is a clear explanation of the monitoring linkage in the project description. The ISRP 

does not wish to place an unfair monitoring burden on individual project sponsors, and we 

encourage collaboration among habitat restoration participants, both with respect to documenting 

physical habitat improvements and population responses, to increase cost-effective knowledge 

sharing. However, we continue to note that each project for which there is no effectiveness 

monitoring represents a lost learning opportunity. 

In the 2006 retrospective report (ISRP 2007-1) the ISRP discussed three types of tributary 

habitat monitoring and provided recommendations for reporting at each level as follows: 

Implementation Monitoring 

“There are many types of habitat improvement projects currently being funded under the Fish 

and Wildlife Program. All should be monitored for implementation, and the ISRP supports the need 

for accurate implementation metrics as a necessary first step in any M&E effort. The ISRP 

appreciates that each project will not be held to a one-size-fits-all M&E standard.” 

Effectiveness Monitoring 

“Some form of effectiveness monitoring should accompany any habitat project. The overall level 

of effectiveness monitoring that have accompanied many projects in the past has not been adequate 

to address the basic question – are they working? Project sponsors should collect and analyze data 

that document whether the project is achieving, or is failing to achieve, its stated objectives and is 

realizing desired habitat and/or target population and/or multi-species benefits.” 

7 
Intensively Monitored Watersheds 

“The ISRP recommends continuing support of Intensively Monitored Watersheds. To properly 

address population-level response of fish species to habitat restoration, we support a network of 

intensively monitored watersheds (IMWs) in which restoration efforts can be coordinated in a way 

that will facilitate experimental learning by applying enough similar treatments to produce statistically 

robust results, coupled with thorough inventories of adult, juvenile, and smolt abundance. Many 

IMWs currently exist in the Columbia River Basin and elsewhere in the Pacific Northwest (some 

funded outside the Fish and Wildlife Program), but there are still significant geographical gaps in 

IMW coverage. The ISRP recommends that additional IMWs be added to this regional network over 

time. Although we realize that the cost of implementing IMWs is high, we note that this approach 

provides the most scientifically powerful way of addressing the question – Are tributary habitat 

restoration projects working?” 

Progress: In 2007, and over the past several years, various Pacific Northwest and Columbia River Basin 

projects and processes have made significant progress on habitat M&E. For implementation monitoring, the Bonneville 

Power Administration is making some progress on reporting of implementation monitoring (in this case task completion 

monitoring) through the PISCES database. The Columbia Basin Fish and Wildlife Authority has created the Status 

of The Resource (SOTR), a web-based report that shows project location and includes reporting of fish and wildlife 

population status where available. 

The Pacific Northwest Aquatic Partnership (PNAMP) in concert with the American Fisheries Society compiled 

a set of recommended field protocols for monitoring and collecting fish population data. These protocols are intended to 

be a reference guide that offers the potential for some standardization of methods and aid to the training of aquatic 

scientists and technicians. The ISRP and ISAB reviewed the handbook and thought it would be a useful start to 

standardizing protocols.8 

 As noted above, the Collaborative Systemwide Monitoring and Evaluation Project (CSMEP) has made some 

progress developing a comprehensive regional M&E program, including development of a habitat monitoring pilot study 

in the Salmon River, Idaho.9 In addition, the Integrated Status and Effectiveness Monitoring Program (ISEMP) has 

made progress on designing, implementing, testing, and evaluating status and trends, and effectiveness monitoring 

programs for salmon and steelhead populations and habitat in the Columbia River Basin. Over the past several years, 

the ISRP has reviewed the study designs for ISEMP pilot studies in the John Day, Snake, and Wenatchee subbasins 

and found them scientifically sound.10 

Finally, although we have recommended that some form of effectiveness monitoring accompany every habitat project, 

we emphasize that habitat data collection, analysis, and interpretation need not necessarily be completed within that 

project. For example, it can be done by a different entity at a larger scale than the individual project. 

The 2008 Program Amendments offer an opportunity to link the various efforts together into a regional RM&E 

program. The ISRP discusses these issues in greater detail in this report’s “Monitoring and Evaluation -- Next Steps” 

section below. 

 

8 www.nwcouncil.org/library/isab/isabisrp2007-5.htm 

9 See the ISRP’s March 7, 2008 review of CSMEP: www.nwcouncil.org/library/isrp/isrp2008-1.htm. 

10 For the ISRP’s review of ISEMP’s John Day study plan, see www.nwcouncil.org/library/isrp/isrp2007-8.htm. 

8 
WILDLIFE 

In the first ISRP retrospective report (ISRP 2005-14) the panel noted that the Management Plans 

portion of subbasin plans tended to pay far less attention to wildlife than to fish and often did not 

include much consideration of landscapes, ecosystems, and overall biodiversity. There is a critical 

need to evaluate (and demonstrate, if possible) where and when habitat restoration efforts increase or 

sustain fish and wildlife populations and at the same time maintain or increase biodiversity. 

Overall, much progress appears to have been made in developing productive scientific review 

and dialogue about wildlife. Several challenges remain for the wildlife portions of the Fish and 

Wildlife Program. First, integration of all elements of the Fish and Wildlife Program remains to be 

realized in the continuing development and implementation of subbasin plans. Second, additional 

time and thought must be given to criteria and procedures for selecting focal species that will be 

useful and effective in monitoring and evaluating project effectiveness. Third, the focus on 

ecosystems and biodiversity that is a central emphasis of the Council’s 2000 Fish and Wildlife 

Program is only beginning to be incorporated into actions. 

The ISRP also recommended “that the wildlife and fish habitat protection programs be better 

integrated and that projects be evaluated on criteria that favor those projects with documented 

benefits to both terrestrial and aquatic species.” 

In the second retrospective report (ISRP 2006-4a) the ISRP recommended that the Habitat 

Evaluation Procedure (HEP) should be used only as an initial scoring system for the mitigation 

agreements that underlie the Wildlife Program. It should not serve as the sole criterion for judging 

whether an agreement was worthwhile. 

Progress: Although some fine examples of wildlife projects exist among the projects listed, wildlife portions of the 

Fish and Wildlife Program may be considered in a formative stage with little or no connectivity among many of the 

diversified projects. Some projects have single focal species with quite specific monitoring efforts to understand a single 

species response to management activities. Other projects are broader in scope with many species targeted for monitoring 

efforts, i.e., emphasis on biodiversity. Many wildlife species exist and an approach for setting priorities needs to be 

established for both choosing focal species and targeting critical habitat needs for acquisition. Furthermore, approaches 

used for monitoring wildlife populations and collecting habitat data are varied. As with PNAMP and the American 

Fisheries Society’s development of salmonid monitoring protocols, perhaps a similar set of protocols, in conjunction with 

The Wildlife Society, can be developed that offer the potential for some standardization of methods for monitoring 

various wildlife species and their habitats. 

ARTIFICIAL PRODUCTION 

In the first retrospective report (ISRP 2005-14) the ISRP recommended that a defensible overall 

production plan be developed for each subbasin that integrates natural and artificial production 

elements and explicitly links them to prioritized habitat limiting factors and proposed habitat actions 

identified in the Subbasin Plan. 

Progress: The Fish and Wildlife Program’s artificial production effort has improved incrementally through 

comprehensive basinwide and individual hatchery program reviews. From 1997 through 2004, at Congress’ request, 

the Council undertook an Artificial Production Review Evaluation (APRE), producing principles for artificial 

production in an Artificial Production Review (APR, NWPCC 99-15), and a review of the purposes of individual 

programs and Hatchery and Genetic Management Plans (HGMPs) (APRE, NWPCC 2004-17), concluding with 

a report to Congress (NWPCC 2005-11). Subbasin planning in 2005 revealed a lack of integration of natural and 

hatchery production in subbasin plans. In response the Council began using the All-H Analyzer (AHA) developed by 

9 
the Puget Sound Hatchery Science Review Group (HSRG) to integrate the biological objectives for natural and 

artificial production and harvest. More recently Congress has directed that a Hatchery Scientific Review similar to the 

Puget Sound Hatchery Review be conducted in the Columbia Basin. This review is employing AHA as a tool to 

investigate the expected benefits from artificial production and the likely effects from integrating natural and artificial 

production. The ISRP (and/or ISAB) anticipates reporting to the Council how the final products from this review can 

be incorporated in the Fish and Wildlife Program to best provide harvest benefits and support restoration of natural 

spawning populations. 

In the second retrospective report (ISRP 2006-4a) the ISRP recommended that the Council issue 

an RFP to develop methods to evaluate the effects of large-scale artificial production programs 

designed primarily for harvest on the abundance, productivity, and diversity of naturally spawning 

salmon populations. Additionally, the ISRP recommended the Council issue an RFP to conduct 

studies of the effects of supplementation on long-term changes in fitness. 

Progress: See the more detailed summary of supplementation and the ad hoc supplementation workgroup on 

page 21. 

MAINSTEM 

The first ISRP retrospective report (ISRP 2005-14) commented that the ISRP was gratified that 

most of its recommendations regarding projects in the mainstem over the past eight review years had 

been adopted by the Council or another agency. The interchanges among the ISRP, the ISAB, the 

Council and the Council staff have been very positive. They yielded considerable progress toward 

developing a mainstem program that is scientifically sound, benefits fish, and has defined objectives 

and intended outcomes. In addition, intensive effort was initiated to provide for continual 

monitoring and evaluation of results without overwhelming the needs of other parts of the Fish and 

Wildlife Program. 

Nonetheless, research, monitoring, and evaluation on the mainstem have not been completed, 

and significant technical issues remain, especially if the often competing socioeconomic and fisheries 

interests are to be wedded as equal objectives. Recent events showed that the persistent issues of 

flow and spill, for example, have not been resolved. The mainstem programs of the Corps (AFEP) 

and the Council require improved interchange and coordination. 

Progress: In 2007 the ISRP and ISAB completed a review of the Comparative Survival Study’s (CSS) Tenyear Retrospective Summary Report11, one of the largest M&E projects in the Fish and Wildlife Program. The CSS 

is a field study of the survival of PIT-tagged spring/summer Chinook and PIT-tagged summer steelhead through the 

Snake and Columbia River(s) hydrosystem from smolts through returning adults, with a focus on relative survival of 

fish that traveled as smolts by alternative routes (e.g., in river, transported, different routes of dam passage, and different

numbers of dams passed). The CSS is important because it is one of the few organized attempts to systematically release 

PIT-tagged hatchery-reared and wild smolts into the Columbia River for the purpose of comparative monitoring and 

evaluation. Most aspects of the study, from its design and methods to the analytical results, continue to be strongly 

debated in the Region because the relative survival rates of salmonids under different hydrosystem operations and 

environmental constraints is a central concern of water and fish management policies. For the most part, the ISRP and 

the ISAB found that the CSS Ten-Year Retrospective was effective in answering the concerns posed by the ISAB’s 

review of the CSS 2005 Annual Report (ISAB 2006-3). However, the ISRP, with concurrence of the ISAB, found 

that one of the major project objectives (Upriver/Downriver Comparisons) did not meet scientific review criteria, because 

 

11 www.fpc.org/documents/CSS.html 

10 
of inevitable confounding from other factors in establishing cause(s) of upriver/downriver differences that may be 

detected, regardless of sample size and detection power that could be achieved. Nevertheless, he CSS Ten-Year 

Retrospective provided improved clarity in the presentation and explanation of the sophisticated methodologies used in 

analyses of CSS data. The scope of CSS investigations resulted in an extensive report, containing many detailed 

summaries of past and present work, and the report presents key data and data summaries in support of their major 

conclusions. 

The ISAB reviewed models and data to address post Bonneville Dam mortality (latent mortality) (ISAB 2007-

1) concluding that the hydrosystem likely causes some latent mortality. However, the ISAB recommended to the region 

that the components of latent mortality be treated in a single model, and strongly advised against continuing to try to 

measure latent mortality, and instead focus on estimating total mortality of in-river and transported fish. 

OCEAN AND ESTUARY 

The first retrospective report (ISRP 2005-14) recommended that the ISRP and Council should 

encourage innovative ecosystem-based research and monitoring in the estuary, with emphasis on the 

effects of the hydrosystem (altered flows, primarily) on all components of the ecosystem. 

The mainstem Columbia River between Puget Island (upper estuary) and Bonneville Dam 

remained largely un-assessed even after the subbasin planning process. This limitation has been 

identified by the ISRP and ISAB numerous times, but it still persists. Approximately 100 miles of 

river could be viewed as either a gauntlet common to all up-river and Willamette River salmonids or 

as a hundred miles of restoration opportunities. At this time there is apparently insufficient 

information to assess the importance of this large and highly modified subbasin. 

More thorough assessment and increased attention in regional research, monitoring, and 

evaluation plans are needed for the mainstem Columbia River between Puget Island (upper estuary) 

and Bonneville Dam. 

Progress: The ISRP reviewed several innovative proposals for improvement of the information base on ocean 

and estuary habitats. A project on eelgrass restoration (200751300) was recommended for funding, and a proposal 

dealing with acoustic tags to examine survival of Chinook in Columbia River plume (200750200) was ranked. The 

proposals that dealt with ocean and estuary studies were only a small proportion of the total number submitted. The 

ISRP continues to recognize this shortfall in the project mix, as identified in earlier retrospectives (ISRP 2005-14 and 

above) and encourages innovative ecosystem-based research and monitoring in the estuary, with emphasis on the effects of 

the hydrosystem (altered flows, primarily) on all components of the ecosystem. Conditions in the estuary, plume, and 

ocean are possibly very important in our understanding of key processes such as measuring total mortality that affect 

Columbia River salmonids. 

PROJECT AND PROGRAM REVIEW PROCESS 

The second ISRP retrospective report (ISRP 2006-4a) strongly recommends against further use 

of an annual solicitation that entails concurrent review of all new proposals and ongoing projects. 

Annual reviews tax the limits of the ISRP’s human and budgetary resources and increase the 

opportunity for inconsistencies among reviews. Features of the provincial reviews such as site visits 

and presentations are invaluable in increasing the thoroughness of ISRP reviews through a better 

understanding of projects (not just proposals) within their geographical and biological contexts. In 

addition, presentations with question and answer opportunities are a much more efficient way to 

clarify issues that the ISRP or project sponsors may not make explicit in written documents. 

11 
Progress: The Council has developed alternative paths for proposal and program review. Multiple 

proposal/project reviews for Lake Roosevelt, Umatilla River, Flathead Lake, and portions of the Clearwater subbasin 

during 2007 worked well for the ISRP. The ISRP has been briefed on, has reviewed, and supports the Council, 

Bonneville, and CBFWA staffs’ proposed approach to future project reviews. The ISRP looks forward to providing 

further input on the details of the process. See Future Directions for the Project Review Process on page 32 for more 

discussion. 

INADEQUATE REPORTING OF RESULTS 

In the 2006 retrospective report (ISRP 2007-1) the ISRP’s primary observation from our 

evaluation of the FY 2007-09 proposals for existing projects was that over 40% of the projects 

needed to improve their reporting of results. A variety of factors helped explain this deficiency in 

reporting. In the response loop 8% of the projects reported results at a sufficient level. This 

improved reporting indicated that in some cases the problem was inadequate summarizing, analyzing, 

and interpreting results in a project proposal rather than lack of data. 

Although the current proposal form includes instruction and emphasis on results reporting, the 

Council and ISRP should develop proposal forms that more explicitly require the reporting of data 

on physical habitats, biological objectives, summaries of data analysis, and the application of results 

to fish and wildlife management. 

ISRP proposal reviews should explicitly address the level and quality of reporting so the Council 

can use this information more effectively in developing their recommendations to Bonneville. This 

ISRP recommendation essentially applies to our internal ISRP review process rather than any 

potential Council action. 

Progress: As described below and a major theme of this report, the ISRP reviewed results reports from a 

number of projects that the ISRP found deficient in results reporting in the FY 2007-09 review. In addition, the 

Council and Bonneville have specifically considered how to treat the entire set of projects that the ISRP identified as 

deficient in the 2006 Retrospective analysis. Related to this, at its March 2008 meeting, the Council requested that the 

ISRP help identify and prioritize a list of reporting metrics for artificial production, wildlife, and habitat projects. At 

the time of the release of this report, the ISRP has just begun scoping the metrics review, and this 2007 Retrospective 

Report should inform our approach to the review. 

2007 ISRP REVIEWS 

For fiscal and calendar year 2007, the ISRP continued its role in providing scientific review to 

assist project sponsors in improving their projects and assist the Council and Bonneville in making 

project selection and implementation decisions. The ISRP completed a total of 21 reports/memos 

(see table below). The ISRP’s major effort for the year was conducting follow-up reviews addressing 

unresolved issues from the ISRP’s review of 540 FY 2007-09 proposals. In addition, the ISRP 

reviewed proposals seeking to test innovative technologies and methods; completed a retrospective 

review of results reporting in FY 2007-09 proposals; provided a potential framework for future 

project reviews; and with the ISAB jointly reviewed a handbook of salmonid field protocols. The full 

reports can be accessed via the web links provided in the table below. 

12 
Table of Fiscal and Calendar Year 2007 ISRP Reports with Web Links 

Number Title 

2007-18 Review of Ecological Restoration Strategies for Lapwai and Big Canyon Creeks

2007-17 Review of Project 2007-405-00 Rufus Woods Supplementation and Creel Project

ISAB/ISRP 

2007-6

ISAB and ISRP Review of the CSS Ten-Year Retrospective Summary Report

2007-16 Step Review of Sekokini Springs Isolation Facility

2007-15 Umatilla Initiative Review

2007-14 Input on Evaluation of Regional Coordination Projects

ISAB/ISRP 

2007-5

ISAB/RP review of the Salmonid Field Protocols Handbook

2007-13 Preliminary ISRP Review of Project 2007-405-00 Rufus Woods Supplementation and Creel 

Project

2007-12 Results Report of the Grand Ronde Model Watershed Program Habitat Restoration - 

Planning, Coordination, and Implementation Project, 1992-026-01

2007-11 Review of Combined Habitat Assessment Protocols (CHAP) work element in Project 

200600600, Habitat Evaluation Procedures (HEP)

2007-10 Lake Roosevelt Kokanee Program Review

2007-9 Review of FY 2007-09 Innovative Proposals

2007-8 Review of the ISEMP John Day Study Plan

2007-7 Review of the latest revision of the FY 2007-09 proposal 199101901 (dated 05/25/07), 

Hungry Horse Mitigation/Flathead Lake

2007-6 FY 2007-09 Follow-up Review of the Shoshone-Bannock Tribes’ Southern Idaho Wildlife 

Mitigation Project 199505702

2007-5 Final Review of Proposal 2003-063-00, Natural Reproductive Success & Demographic 

Effects of Hatchery-Origin Steelhead in Abernathy Ck, Washington 

2007-4 Review of Desert Wildlife Area O&M proposal

ISRP/IEAB

2007-3

SAFE Review 2007

2007-2 Response Loop Review of FY 2007 – 09 Proposal 2007-034-00: Columbia Cascade Pump 

Screen Correction

2007-1 2006 Retrospective Report

2006-7 Framework for ISRP Review of New and Ongoing Projects for the Columbia River Basin 

Fish and Wildlife Program

13 
ADAPTIVE MANAGEMENT 

USING PROJECT RESULTS IN RESTORATION DECISION-MAKING: COMPLETING 

THE ADAPTIVE MANAGEMENT LOOP 

Past ISRP retrospectives and programmatic comments have emphasized the need for sufficient 

monitoring and evaluation of Fish and Wildlife Program projects and reporting project results in 

proposals. One ISRP review question is: Does the project describe the adaptive management 

implication from past results whether successes or failures? 

The reason the ISRP has been focused on monitoring and evaluation and reporting results is that 

the Council’s Fish and Wildlife Program explicitly acknowledges that project work elements are 

uncertain to achieve their stated biological objectives. The 2000 Fish and Wildlife Program’s planning 

assumptions assert that “Management actions must be taken in an adaptive, experimental manner 

because ecosystems are inherently variable and highly complex. This includes using experimental 

designs and techniques as part of management actions, and integrating monitoring and research with 

those management actions to evaluate their effects on the ecosystem.” The program’s Scientific 

Foundation Principle 7. Ecological management is adaptive and experimental, states: 

“The dynamic nature, diversity, and complexity of ecological systems routinely disable attempts to command and 

control the environment. Adaptive management – the use of management experiments to investigate biological problems 

and to test the efficacy of management programs – provides a model for experimental management of ecosystems. 

Monitoring and evaluation dovetails with management experiments and so is essential for adaptive management to be 

successful. Experimental management does not mean passive “learning by doing” but rather a directed program aimed 

at understanding key ecosystem dynamics and the impacts of human actions using scientific experimentation and 

inquiry.”

Almost without exception, project sponsors state that they use adaptive management to modify 

the tasks and work elements in proposals. At the same time, however, project proposals almost never 

provide 1) an experimental design to identify whether biological objectives have been met by 

employing specific strategies or 2) a decision tree that would be used to modify management based 

on updated scientific information. Adaptive management components and principles in the Fish and 

Wildlife Program are largely conceptual and do not provide specific guidance to fish and wildlife 

managers. 

The 2000 Fish and Wildlife Plan’s Principle 7 implies the adoption of formal passive or active 

adaptive management which requires identifying hypotheses to test, establishing common metrics to 

be monitored, and designing an experimental procedure that will yield a test of the primary 

hypotheses. Passive adaptive management selects an acceptable initial management strategy based on 

available information and then proceeds to future decision points where the program is reassessed 

using the outcome from implementing restoration tasks. In contrast, active adaptive management 

implements different strategies (e.g., spill or transport of smolts), spatially or temporally, in a 

hypothesis testing framework. Very few individual projects implement these sophisticated designs to 

learn how the ecosystem responds from employing alternative management strategies. The ISRP 

believes it is unrealistic to expect all projects to include such management experiments. Barge 

transport of smolts, spill and bypass of smolts, habitat restoration projects, and hatchery production 

would be amendable to active and passive adaptive management. 

14 
Formal passive and active adaptive management are not the only processes that can be used for 

problem solving and decision making concerning Fish and Wildlife Program projects. Much of the 

adjustment to projects and Fish and Wildlife Program components actually resembles what has been 

referred to as “evolutionary” problem solving: improvement in practice by using a small number of 

independent projects that address a specific problem, with regular communication among the 

projects so the most promising methods are more widely applied. 

A transparent methodology to establish unambiguous thresholds for assessing project and 

program success is absent from many Program projects. Regardless of the decision making structure 

used, decision trees and mechanisms are needed to guide co-managers to alternative strategies, 

reassess goals, and reconfigure priorities when repeated efforts do not appear to yield tangible results. 

There are notable examples where programs have been substantially modified because of apparent 

failure with the primary strategy and new empirical evidence to explain the lack of success. One 

example is the decision to halt efforts to reintroduce kokanee in Flathead Lake because the failure of 

the preliminary efforts was determined to be due to lake trout predation, not a lack of primary 

production. 

More often however, projects propose to continue to employ tasks and work elements that have 

not yielded the anticipated biological benefits. In the FY 2007-09 Solicitation Review in 2006 and the 

follow-up reviews during 2007, there were a number of efforts that adequately reported results 

including the captive rearing of Snake River sockeye salmon, the artificial production program for 

kokanee in Lake Roosevelt, harvest programs to manage the abundance of lake trout in Flathead 

Lake, and the combined natural and artificial production of salmon and steelhead in the Umatilla 

River. But those results in the ISRP’s view fell short of the projects’ biological objectives. Sponsors 

of these projects are understandably reluctant to abandon these efforts in which they have invested 

much time and energy. Often alternative restoration strategies or focal species are not clearly 

apparent to the project proponents. 

Project appraisal based on results requires scientific and technical information. When programs 

are determined to be unattainable because of environmental and biotic conditions, transformation to 

alternative strategies or focal species undoubtedly will be more successful when the external social 

context (conflict among stakeholders, requirements to fulfill legal obligations) is explicitly considered 

during the process. Although this integration was a primary tenet of subbasin planning, this 

component of the Fish and Wildlife Program needs development as applied to program 

implementation and specific projects. 

A structured (or integrated) decision making process linking the economic, social, political, and 

biological requirements needed to sustain fish populations is one approach to evaluate complex 

multidimensional problems involving imperfect information and often conflicting public values. At 

the core of structured decision making is the exploration and development of a robust and 

representative set of objectives, which are used to help create and assess multiple alternatives (and 

test competing hypotheses) in an iterative, adaptable way. The goal of this exploration is to learn and 

gain insight into the consequences of the alternatives and make wiser decisions that better balance 

the competing interests. Exploration of decisions and alternatives can involve expert judgment, 

statistical models, structuring tools, scenario and multi-attribute analyses, multi-stakeholder input, 

planning workshops, the explicit treatment of uncertainty, and adaptive management. The process of 

structured decision making is flexible, but generally follows a six step sequence that is iterative (see 

the figure below). There is an emerging body of case studies that highlights how structured decision 

making would be amenable to application in many of the Columbia subbasins. A decision, once 

reached, requires an adaptive environmental assessment modeling and monitoring procedure, where 

15 
management actions can be treated as experiments, with modeled results tested in control and 

treatment studies. Failing et al. (2004)12 provide an example of a practical integration of probabilistic 

policy analysis and multi-stakeholder decision methods at a hydroelectric facility in British Columbia, 

Canada. 

Recent research highlights that carrying out a successful structured decision making process can 

be greatly facilitated with the assistance of an independent facilitator / decision analyst to assist all 

parties involved in clearly defining the decision and its context. Several facilitated meetings and 

discussions may be necessary to reach agreement even at this initial step, but this initial effort is 

important in setting the stage for subsequent iterative steps providing needed focus and potentially 

saving considerable time in the longer run, including avoiding expensive litigation. Integrated 

decision-making considers environmental, social, and economic issues while striving to find ways for 

all uses to co-exist with less conflict. For example, on the Umatilla River, water use is important for 

fish, agricultural irrigators, drinking, etc. Decision to alter flows involves several stakeholders in 

complex fashion, thus consideration of context must include all concerned through the iterative steps 

in the above figure, whether in this case of water use, or in other examples, such as development of 

additional hatchery capacity within the Columbia Basin (e.g., Chief Joseph Hatchery). The process 

has met with success in examples in British Columbia. One example is BC Hydro's water use 

planning program which resulted in 21 agreements between regulators and community groups for the 

operation of their hydro-electric facilities (Failing et al. 2004). Another example concerns the 

Seymour River, where decisions were required on continuation of hatchery operations during periods 

of low marine survival of wild steelhead, when hatchery fish were suspected to have a negative 

impact on wild fish. The decision process and related modeling exercises suggested alternatives that 

led to a compromise, which is now being monitored. 

Steps in the Decision Process 

Define Decision Context

Define Objectives/Interests

Iterate as required 

Implement and Monitor 

Evaluate and Select

Estimate Consequences

Develop Alternatives

 

12 Failing, L., G. Horn, and P. Higgins. 2004. Using expert judgment and stakeholder values to evaluate adaptive 

management options. Ecology and Society 9(1): 13. URL: http://www.ecologyandsociety.org/vol9/iss1/art13/. 

16 
MAJOR 2007 PROGRAMMATIC ISSUES 

In this report section, the ISRP examines a subset of projects that were reviewed in Fiscal Year 

2007 and investigates how the projects applied the results of their past actions and monitoring to 

proposed future actions and monitoring. The ISRP first address fish production projects and then 

habitat related projects including salmon habitat restoration and wildlife habitat strategies. 

FISH PRODUCTION 

PROGRESS ON RESULTS REPORTING 

Select Area Fisheries Enhancement Project (SAFE) 

The ISRP and Independent Economic Advisory Board (IEAB) reviewed two SAFE project 

reports. The ISRP concluded that the SAFE project is consistent with the Northwest Power and 

Conservation Council’s Fish and Wildlife Program and the Bi-State Lower Columbia River and 

Columbia River Estuary Subbasin Plan (ISRP&IEAB 2007-313). The SAFE fishery has been 

intensively monitored to assess catch of target and non-target stocks as well as effects on these 

stocks. The sponsors indicate that agencies specifically adapt management by modifying harvest 

regulations when deleterious impacts have been observed or anticipated. SAFE appears to meet its 

overall goals for yielding a relatively stable source and sufficiently high rate of harvest at present 

levels of production and release. Estimated survival rates for SAFE fish are on par with or exceed 

those achieved at lower Columbia River production hatcheries, and harvest of SAFE fish makes up a 

significant component of the lower Columbia River catch of salmon. Importantly, direct impacts to 

non-target or listed stocks, especially those above Bonneville Dam, appear limited. 

While the project sponsors conclude that the program’s production could be expanded to 

increase harvest, the sponsors did not clearly articulate a monitoring design or data analyses to 

support this conclusion. The ISRP identified several areas for improving the project sponsors’ 

presentation and approach. First, the presentation of monitoring methods and design could have 

been more comprehensive and complete. Second, a robust statistical analysis of the coded wire tag 

and other experimental study data was almost entirely absent. As such, the report did not present 

convincing evidence that an expansion of production to 11,300,000 smolts was needed, would 

improve harvest, or would have minimal impact on non-target stocks (especially during periods of 

anticipated or prolonged poor ocean conditions). The sponsors must not assume that because 

impacts appear minimal with the present scale of production and oceanic conditions, these will 

continue as the fishery expands or the environment changes. Third, harvest estimates are an 

important variable in establishing the program’s costs and benefits (per unit of catch – see the IEAB 

section of the SAFE review). 

Consequently, within an adaptive management context, the ISRP recommended continuing and 

enhancing rigorous monitoring, analysis, and results reporting to determine harvest and survival 

rates, impacts on non-target fish stocks, and stray rates of SAFE fish. If not already involved, a 

competent statistician should be consulted in project design and analysis of data. Alternative mass 

marking, such as thermal otolith marking techniques, should be explored to improve survival and 

catch estimates. The IEAB found the economic analysis of the SAFE project to be generally 

responsive to past ISRP/IEAB reviews, although some problems remained with documentation, 

 

13 See the ISRP report at www.nwcouncil.org/library/isrp/isrpieab2007-3.pdf 

17 
detail, and clarity of analysis (see ISRP&IEAB 2007-3 for details). The ISRP and IEAB found the 

combination of scientific and economic reporting and subsequent review to be a useful model. 

Umatilla Initiative Review 

The ISRP review of the “Umatilla Initiative” was intended as a comprehensive site and 

programmatic evaluation of the numerous but interrelated individual projects within the Umatilla 

River.14 To facilitate the review, the Confederated Tribes of the Umatilla Indian Reservation 

(CTUIR) and Oregon Department of Fish and Wildlife (ODFW) prepared, “Umatilla Projects 

Review: A presentation of the multiple Component Umatilla Basin Fisheries Restoration Program” 

to address specific questioned raised in the FY2007-09 proposal review. 

The ISRP organized projects into three categories reflecting each project’s primary objectives 

and tasks, providing a logical approach to evaluate project effectiveness: flow enhancement and fish 

passage; habitat enhancement; and artificial and natural production. Specifically, for flow 

enhancement and fish passage, Phase I and II of the water exchange program are in place and 

irrigation diversions have been upgraded to provide passage of both adults and juveniles. For habitat 

enhancement, numerous stream reaches have received large-wood additions and riparian zone 

fencing. For production enhancement, a hatchery and satellite facilities for holding and acclimating 

hatchery fish have been constructed, and spring and fall run Chinook and coho salmon have been reintroduced back into the watershed (after years of extirpation). Some fish are returning to the river, a 

fishery has developed, and cooperation with landowners is improving. The program, however, has 

not achieved its proposed salmon and steelhead production, escapement, and harvest goals. 

Production activities in the Subbasin are evaluated primarily through the Umatilla Hatchery 

M&E project (199000500). This Subbasin-wide effort has been essential for providing data and 

analysis for evaluating the hatchery-related program(s). Since water-exchange and habitat 

enhancement are intended to increase the abundance of salmon and steelhead, information reported 

for natural and artificial production is essential for a complete evaluation of the collective projects. 

The results reported in Grant et al. (2007) – Comprehensive Assessment of Summer Steelhead and Chinook 

Salmon Restoration Enhancement Efforts in the Umatilla Subbasin and summarized in the sponsors’ program 

review documentation Umatilla Projects Review: A presentation of the multiple Component Umatilla Basin 

Fisheries Restoration Program submitted by CTUIR and ODFW – are sufficient for a preliminary 

evaluation on the collective success of the subbasin effort. The monitoring effort and data analysis 

did not evaluate the separate contributions of the individual hatchery, water exchange, fish passage 

and habitat restoration components. Ultimately, it is not clear whether the updated monitoring plan 

can separate the effects of these individual actions. The ISRP recommended that additional 

information on life-stage specific mortality needs to be analyzed and reported for deciding how the 

project might be modified to close the gap between the goals of the Subbasin Plan and what is 

actually being achieved. 

Lake Roosevelt Kokanee Review 

The ISRP concluded that there is no scientific justification to continue artificial production of 

kokanee for stocking Lake Roosevelt or programs to develop and support naturally produced 

kokanee. The ISRP came to this conclusion based on the Lake Roosevelt program’s significant lack 

of success in producing both a tribal and recreational fishery for kokanee salmon. The results 

reported by the kokanee program’s experimental monitoring and evaluation efforts have documented 

 

14 See the ISRP report at www.nwcouncil.org/library/isrp/isrp2007-15.htm 

18 
that lack of success. Two major factors appear to be limiting this success: entrainment of large 

numbers kokanee through Grand Coulee Dam (particularly following the initiation of operations of 

the third powerhouse) and predation by walleye (whose population has increased significantly over 

the past eight years and now supports the major fishery in Lake Roosevelt). In addition, spring 

drawdowns have been significant in the past several years and have severely impacted kokanee 

spawning potential and natural production in the lower reaches of the tributaries. 

The ISRP believes, however, if it could be conclusively demonstrated that (1) the entrainment 

problem was controlled or significantly reduced, (2) the walleye population was significantly reduced 

and managed, and (3) drawdown levels were reduced or managed, then artificial production and/or 

support of natural production of kokanee could be re-initiated. In this example, three focused 

research projects are needed to implement adaptive management principles. 

ISRP Review of the Hungry Horse/Flathead Lake Project 

The ISRP’s final recommendation for this proposal was: Meets Scientific Review Criteria In Part 

(qualified). In their revised proposal, the project’s sponsors partially responded to the ISRP 

recommendations, but overall did not significantly improve key elements of the proposal. The 

project sponsors proposed a combination of habitat improvements in Flathead Lake and tributary 

streams, and lake trout removal in Flathead Lake using angling to decrease predation on bull trout 

and west-slope cutthroat trout focal species. The sponsors partially responded to the ISRP’s request 

to develop the rationale that the ongoing effort to reduce lake trout numbers via the fishing derbies 

might overcome the compensatory ability of the surviving lake trout. The sponsors provided a 

modeling exercise that describes how increased harvest could reduce the lake trout population. 

Unfortunately, reporting of results to date indicates that the lake trout population has not been 

reduced by angling and the angling efforts have not yet achieved a sufficient harvest. Further, the 

sponsor did not provide a rationale that this reduction would in turn provide a quantifiable increase 

in abundance of westslope cutthroat or bull trout. There is a need for a clear statement of a problem 

and an outline to resolve that problem that is both quantified and employs technically justifiable 

strategies and work elements. The ISRP suggested that the lake trout monitoring might be justified if 

shown to be part of a long-term fisheries plan for Flathead Lake, and funding for the fishing derbies 

might be justified if linked to a larger lake trout removal effort. 

Tributary stream habitat work could be justified if shown to be part of a well developed and 

prioritized restoration program or that the sites could be specifically linked to habitat restoration 

objectives in the subbasin plan. Currently, habitat restoration is being monitored almost exclusively 

with photo-point documentation. Additional metrics that represent trends in ecological conditions 

are also needed. These can be very simple, based on the Flathead watershed assessment and subbasin 

plan. What is needed is a more specific set of habitat objectives, a clear rationale that the sites 

selected for restoration are justifiable in terms of correcting factors that limit fish populations, and a 

strengthened effectiveness monitoring plan. The effectiveness-monitoring component should be 

sufficient to detect quantifiable habitat improvements and increases in fish populations or expanded 

distributions. 

RESIDENT FISH SUBSTITUTION: CHALLENGES IN FOCAL SPECIES MANAGEMENT 

During the FY 2007-09 project review, the ISRP was concerned that a number of resident fish 

substitution proposals selected substitution species that did not appear compatible with other species 

(predators and competitors) in the systems where they were going to be planted/introduced. In some 

cases the introduced fish posed risks to listed native species and were also possibly inconsistent with 

the general guidelines of the Council’s Fish and Wildlife Program. 

19 
The Council’s Program provides that anadromous fish losses due to the blocked areas need to be 

mitigated in part by resident fish species substitutions, provided that (1) populations of resident fish 

species remain healthy, (2) introduced species may be used if they were compatible with continued 

persistence of native species, and (3) an artificial production strategy may be used to “replace lost 

salmon and steelhead in blocked areas.” These production programs are to be executed consistent with 

the Council’s Artificial Production Review policies that include (1) maintaining appropriate risk 

management when using the tool of artificial propagation, and (2) making decisions on the use of 

artificial production in the context of fish and wildlife goals, objectives, and strategies at the subbasin 

and province levels. 

The ISRP found that these broad Fish and Wildlife Program guidelines/elements do not 

establish specific limits (i.e., how much risk?) or the methods (i.e., risk management protocols) to 

evaluate whether a proposed project is reasonably benign and likely to provide benefits without 

undesired consequences. Clear risk management criteria are needed for sponsors to follow when 

developing proposals for resident fish substitution projects and for the ISRP to use when reviewing 

such proposals. These concerns were raised to the ISAB and the Council. Each agreed that this was 

an important issue, and the Council assigned the ISAB to complete a review report on this issue and 

on the broader issue of non-native species impacts on the Fish and Wildlife Program. The ISAB 

report will include suggestions for resident fish substitution criteria and guideline development. This 

review is currently underway. 

CHALLENGES TO ASSESSING COMPETITION AND PREDATION WITH INVASIVE NONNATIVE SPECIES 

Competition and predation are frequently mentioned as the causative agents for decline of native 

species in Columbia River Basin watersheds where non-native species are present. However, the 

relative role of these biological interactions is poorly understood in many situations, and further work 

is required to determine their role relative to habitat factors such as spawning habitat modified by the 

hydrosystem. The potential for competition for food resources requires data on food supplies as well 

as spatial and temporal overlap of the potentially interacting species. 

Information on effects of competition and predation is required to guide restoration activities, 

but the essential data are frequently absent from specific lakes and rivers in the Basin where nonnatives are dominant. Should effort be placed on harvesting of non-native species to reduce 

competition and predation, or should emphasis be placed on habitat restoration? In general there is 

insufficient basic information on feeding habits of fishes in the reservoirs and long-term data sets are 

especially lacking. The carrying capacity of most reservoirs is not known, so food limitation is 

difficult to determine. While most jurisdictions require risk assessments for future introductions of 

non-native fishes, it is not clear how they can be conducted without such information. Clearly, 

complex ecosystem changes can be caused by introduction of non-native species. While basic 

information on feeding is required and contemporary methods such as stable isotope analyses can 

help, a trophic analysis model such as Ecopath (http://ecopath.org) might help unravel some of the 

relationships. The problem is not restricted to reservoirs in the Columbia River Basin but also may 

be present in tributaries where, for example, brook trout have been established. Thus a basin-wide 

workshop on the topic might be useful to share information, avoid duplication of efforts, and apply 

an adaptive management approach to the issue. 

20 
INTERSPECIFIC HYBRIDIZATION AS A CONSERVATION CHALLENGE FOR RESIDENT FISHES 

Hybridization between native and introduced species of fishes is increasingly proving to be a 

challenge for conserving or restoring aboriginal populations of salmonids. For example, efforts to 

recover degraded or “introgressed” populations of westslope cutthroat trout have been impeded by 

interbreeding with introduced and established populations of rainbow trout. Several issues confront 

these efforts such as: 

1. An understanding of the extent of hybridization having taken place (e.g., low level or recent 

interbreeding v. long-term or extensive interbreeding); 

2. What level (if any) of interbreeding can be reasonably accommodated for conserving or restoring 

aboriginal populations; 

3. The comparative effectiveness of chemical, mechanical, or biological eradication methods; 

4. The effectiveness of barriers and “quarantine” for native populations at risk from local upstream 

or downstream sources of introduced or hybridized trout populations; 

5. What constitutes an adequate source for re-establishing a “vacated”, extirpated, or eradicated 

habitat within the aboriginal range; 

6. The effectiveness of re-introducing propagated young v. transplanted adults v. natural 

recolonization; and, 

7. The efficacy of demographically swamping a population with a monophyletic source. 

The first two issues deal with detecting and assessing risks from changes to the structural 

elements of biological diversity, while the remaining five issues deal with approaches and specific 

methods for addressing threats created by interbreeding of divergent taxa. The ISRP acknowledges 

that reducing the threat to native trout posed by hybrid or introduced subpopulations is daunting and 

there are no easy solutions. The ISRP believes there are considerable logistical challenges associated 

with any undertaking to address the effect and legacy of former trout introductions. Any projects 

need to establish a logic pathway that links quantifiable outcomes with proposed treatment. It will 

also be critical to address these issues within a broader context, such as that currently under 

consideration by the ISAB in its non-native species review. Finally, robust M&E should accompany 

any program to answer questions of whether renovated lakes or streams support self-sustained 

populations of native trout (or other species) following eradication and repopulation. The ISRP 

recommends that M&E plans should include methods, protocols, as well as the kind of data 

collected, and specific hypotheses to be tested. We do not recommend measuring “everything” but 

rather those variables that can permit judgment as to whether the treatment(s) have worked or not. 

SUPPLEMENTATION 

Hatchery production of salmon and steelhead is a component of most subbasin plans 

implemented as part of the Columbia River Basin Fish and Wildlife Program. Most hatcheries are 

operated with the intent that the juvenile fish released will be harvested in the ocean or when they 

return as adults to the Columbia River. These hatcheries are referred to as harvest augmentation 

programs. The adults returning from juvenile releases from harvest augmentation programs are not 

intended to spawn naturally. 

21 
In contrast, a number of hatchery programs, referred to as conservation or supplementation, are 

operated with the intent that a significant fraction of the released fish will escape harvest to 

successfully reproduce in the wild. Juveniles from these supplementation programs are released into 

habitats to which they are expected to return and spawn, thereby providing a “demographic boost” 

that eventually leads to increased abundance of natural-origin adults in the salmon or steelhead 

population. 

The Council, when deciding to proceed with supplementation, recognized it as experimental, 

requiring risk assessment and monitoring (NWPPC 99-15, NWPCC 2000). The uncertainties are 

whether supplementation actually provides a demographic increase in natural production (the 

potential benefit) and whether supplementation leads to decreased natural-spawning fitness (the 

potential harm) in the wild stocks. The ISRP and ISAB have expressed the view that the uncertainty 

of both the benefits and the risks of supplementation are sufficient enough to put the merit of 

supplementation into question as a recovery strategy. The ISAB review of supplementation (ISAB 

2003-3) found that, in general, the correct parameters were not being evaluated in most 

supplementation projects. The ISRP and ISAB (ISRP&ISAB 2005-1515) provided a concise summary 

of possible approaches to monitoring supplementation and recommended that a uniform approach 

be established to evaluate the abundance, productivity, and capacity of target populations; to 

establish more reference locations; and to establish an evaluation of long-term fitness. The panel and 

board suggested that the Columbia River Inter-Tribal Fish Commission, NOAA-Fisheries, and the 

Council organize a workshop of the Basin’s fish managers to develop this comprehensive evaluation 

of supplementation within the Columbia River. 

Two workshops have been held: the first April, 6 and 7, 2006, and the second February 14 and 

15, 2007. The first workshop brought together the managers and biologists responsible for 

implementing many of the basin’s supplementation projects with the objectives:

1. To review the status of ongoing supplementation monitoring programs in the Columbia River 

Basin. 

2. To discuss ongoing or proposed monitoring programs with respect to a recent set of 

recommendations made by the ISRP and ISAB in the report: Monitoring and Evaluation of 

Supplementation Projects (ISRP&ISAB 2005-15). 

3. To identify areas where greater coordination among supplementation (and population) 

monitoring programs would be beneficial. 

The first workshop identified strong support among managers for an effort to coordinate 

monitoring and evaluation of supplementation across the Columbia River Basin. The effort will 

concentrate on two approaches: establishing monitoring protocols and analytical methods for longterm trend analysis for abundance and productivity in treated versus reference streams to evaluate 

demographic “boost” from supplementation; and intensive experimental studies to assess the longterm fitness effects of supplementation. 

The second workshop objectives were to outline the design of an evaluation of the demographic 

effects of supplementation on salmon and steelhead population abundance and productivity; and to 

further consider the design of a meta-analysis of projects using pedigree analysis that could be used 

 

15 www.nwcouncil.org/library/isrp/isrpisab2005-15.pdf 

22 
to assess the long-term effects of supplementation on natural spawning fitness of salmon and 

steelhead. 

Attendees of the second workshop concluded that with additional monitoring of reference 

streams the demographic effects of supplementation could be assessed, and an Ad Hoc 

Supplementation Work Group was created to coordinate with the CSMEP Hatchery Workgroup to 

establish standardized metrics to evaluate supplementation projects. Attendees also concluded that 

the objective to develop a design for a meta-analysis of projects using pedigree analysis was 

premature. There was a need to review and clarify the kinds of information that pedigree analysis can 

and cannot provide. During 2007 progress was made on both of these objectives, and a report is 

anticipated early in 2008. 

Also in 2007, the Fish and Wildlife Program project Reproduction of Steelhead in Hood River

(200305400) published a peer-reviewed article in Science: Genetic effects of captive breeding cause a 

rapid, cumulative fitness decline in the wild. Science 318:100-103. Michael Blouin and associates at 

Oregon State University (OSU) used an innovative pedigree analysis of the numbers of adult progeny 

produced by hatchery-origin adults spawning naturally that had either wild or hatchery parents when 

produced in the hatchery. This permitted a comparison of genetic effects un-confounded with 

environmental effects. They reported a 40% reduction in the natural-spawning fitness of hatcheryorigin steelhead that had hatchery-origin parents compared to hatchery-origin steelhead that had wild 

parents. 

The preferred contrast to evaluate the long-term fitness effects of supplementation would be to 

compare the adult progeny produced by natural-origin fish with different histories of 

supplementation in their pedigrees. The approach used by Dr. Blouin and the Hood River/OSU 

team is a reasonable surrogate when the preferred method is untenable owing to logistical 

constraints. If the effect measured in Hood River steelhead is observed in other steelhead and 

Chinook salmon, it would demonstrate a significant long-term genetic risk from supplementation. 

At the Council’s December 2007 meeting, Dr. Blouin presented the Hood River study’s results. 

He was followed by members of the Supplementation Work Group who identified other studies in 

the Columbia River and elsewhere that could provide similar information in the near future. 

The ISRP understands that the Supplementation Work Group will submit a final report outlining 

recommendations for evaluating both the demographic and long-term fitness effects of 

supplementation early in 2008. The ISRP recognizes the significant progress made by regional 

scientists, looks forward to reviewing the report, and is hopeful a plan for a comprehensive 

evaluation of supplementation is close to implementation. The monitoring and evaluation should 

provide information to guide decisions on the sufficiency of supplementation to contribute to the 

restoration of salmon and steelhead. 

HABITAT 

SUBBASIN HABITAT PROGRAMS (JOHN DAY, GRAND RONDE, UMATILLA) 

In the 2007-2009 Projects Review and 2006 Retrospective Report, the ISRP identified serious 

deficiencies in a majority of the projects in reporting results related to the project’s effectiveness in 

achieving the stated objectives. This deficiency was related in part to inadequate programs for 

monitoring and evaluating project success. The problem was particularly acute for habitat restoration 

projects. Without adequate monitoring and evaluation, it is difficult to determine whether a project is 

achieving or failing to achieve its objectives and is providing benefits to fish and wildlife. 

23 
Furthermore, without appropriate M&E, implementation of adaptive management is seriously 

impeded. In the 2006 Retrospective Report, the ISRP recommended to the Council that more 

emphasis should be placed on reporting results in project proposals and some form of effectiveness 

monitoring should be required of all habitat projects. As we have noted above, effectiveness 

monitoring associated with individual projects should most often focus on response of habitat rather 

than population responses. 

Three projects reviewed by the ISRP in 2007 differ in several ways, but all had significant habitat 

improvement components. In their initial review of these projects during the 2007-2009 Project 

Review process, the ISRP expressed concern about results reporting and the adequacy of monitoring 

and evaluation. These concerns were acute because the projects had been implemented for well over 

a decade and there was a clear need for some assessment of project success. Each of the sponsors of 

these projects prepared reports that responded in some way to the ISRP’s comments on results 

reporting and M&E. These reports were reviewed by the ISRP in 2007 to determine how well the 

sponsors had addressed the concerns. 

For the Grand Ronde Model Watershed Program Habitat Restoration - Planning, Coordination and 

Implementation Project (199202601) the ISRP called for "a report presenting quantitative and qualitative 

results to date pertaining to the effectiveness of the projects under their domain, a general summary 

and conclusions about overall project effectiveness, and the application of the results to 

management." The watershed program sponsors submitted the report for ISRP review on July 12, 

2007. 

In 2007 the ISRP also reviewed the sponsor’s response to the ISRP project review of the 

Mainstem, Middle Fork, John Day Rivers Fish Habitat Enhancement Project (198402100). The ISRP’s major 

concern was that this project, ongoing since 1984, lacked a comprehensive reporting of results 

demonstrating the effectiveness of the project. 

The ISRP also reviewed in 2007 a set of projects directed at restoration and protection of salmon 

and other aquatic biota in the Umatilla Basin. There are currently two large BPA-funded habitat 

enhancement projects in the Umatilla: Umatilla Subbasin Fish Habitat Improvement Project (198710002)

with ODFW being the lead entity, and Umatilla Anadromous Fish Habitat - CTUIR (198710001). In 

addition, Umatilla River Basin Stream Temperature Monitoring (200729300) involves a continuation of 

stream temperature monitoring at 31 long-term monitoring sites and Iskuulpa Watershed Project

(199506001) (CTUIR lead entity), which is specific to a single watershed, is designed to enhance 

habitat for focal wildlife and fish species. A fifth project – Morrow County Riparian Buffers Umatilla 

County Riparian Buffers (200202600) (Morrow County Soil & Water Conservation District lead entity) – 

involves enrolling farmers in riparian conservation programs to improve riparian zones. 

The Umatilla projects were reviewed individually during the 2007-2009 Projects Review. Central 

issues in the ISRP’s original review of the individual projects had to do with results reporting and 

M&E. The ISRP recommended these proposals be reviewed as a comprehensive package to clarify 

relationships among the projects, identify measures for effectiveness evaluation, and assess the 

overall benefits for salmon and other aquatic biota and their habitats. 

24 
Iskuulpa Creek and Meacham Creek, Umatilla River Subbasin

After reviewing the responses of the sponsors from each of these three subbasins to ISRP 

comments made in the 2007-2009 Projects Review, we abstracted general themes or elements 

common to all the projects in terms of how well they addressed the ISRP concerns about results 

reporting and M&E. 

All three projects maintained some level of monitoring and evaluation, principally 

implementation monitoring, but also some, usually qualitative, monitoring of project effectiveness. 

The ISRP remains convinced that M&E should be improved for all projects to enable more 

comprehensive, quantitative evaluation of project effectiveness. The lack of evaluation of biological 

effectiveness for all projects was especially striking. Sponsors generally acknowledged that in many 

cases M&E was inadequate or entirely lacking, but they clearly saw the need for monitoring and 

evaluation, and they expressed the desire to increase M&E efforts, consistent with available funding. 

Some are pursuing opportunities for collaboration with other projects and programs to facilitate 

monitoring activities. 

In many instances monitoring was sufficient to demonstrate some benefits derived from the 

project and assess, to some extent, whether the project was achieving its objectives. Project sponsors 

were able to identify not only parts of their projects that appeared to be successful but also those that 

failed to meet expectations. Often, adaptive changes were made to improve the project. Had some 

level of monitoring not taken place, these failures would not have been identified and appropriate 

project modifications could not have been implemented. Even so, all three projects could benefit 

from a more rigorous, systematic program for monitoring project effectiveness. 

In the sponsors’ responses, reporting of results, including data presentation, analysis, and 

interpretation was improved over the original proposals but still largely lacked the 

comprehensiveness needed for rigorous, quantitative assessment of project effectiveness. All projects 

had potentially important ongoing habitat restoration and protection work, but it was often unclear 

how well the individual habitat projects were achieving their objectives and were contributing to 

overall restoration goals. The projects were especially deficient in data related to biological 

population response. One reason for the insufficient reporting of results was inadequate 

summarization, analysis, and interpretation of physical habitat and biological data. Another, perhaps 

more serious problem, appeared to be insufficient physical habitat and biological data collection due 

to the lack of a rigorous monitoring and evaluation program. The lack of critical data underscores 

again the need for more comprehensive monitoring and evaluation. 

25 
Habitat projects collectively could benefit from enhanced biological monitoring. The projects 

focused monitoring efforts primarily on documenting implementation of habitat projects (e.g., 

number of stream miles fenced, number of culverts removed, number of habitat structures installed), 

and to a lesser extent on measuring physical changes in the stream following habitat restoration 

actions (e.g., stream channel changes resulting from placement of structures). To a far lesser degree, 

the biological effectiveness of habitat restoration on target populations was evaluated, that is, 

whether the implemented habitat action was actually improving fish abundance and provided a 

significant benefit to fish and wildlife (e.g., were fish using newly rehabilitated habitat, whether it is a 

section of stream previously inaccessible or newly created habitat following placement of instream 

structures). On larger spatial scales, where feasible, it would be highly desirable to determine adult 

escapement, smolt production, fry and parr abundance, and other appropriate biological metrics to 

determine whether habitat restoration actions within a larger area (e.g., watershed) have actually 

increased fish productivity (the “validation” monitoring described on page 6). Monitoring designs 

such as comparing a treated stream section with untreated sections upstream or downstream, or 

comparing treated watersheds with untreated ones, e.g., before-after/control impact (BACI) designs, 

can be effective ways of determining the efficacy of stream restoration actions. Several of the projects 

have proposed or are in the process of implementing such designs, and additional opportunities for 

this sort of work should be explored. The ISRP emphasizes that population-level response 

monitoring is not a requirement of every project, but the array of tributaries in the Umatilla subbasin 

with restored and unrestored conditions provides the various stakeholders with an excellent 

opportunity to design and collaborate on a biological response assessment program that would shed 

insight on the effectiveness of restoration actions. 

MONITORING AND EVALUATION -- NEXT STEPS 

Regional Partnerships 

The ISRP believes that progress is being made toward improved regional M&E coordination; 

improved integration of results from focused research efforts and individual monitoring programs; 

and development of improved indicators of habitat restoration effectiveness. Much of this progress 

has resulted from work supported by the Council and other organizations through a number of 

collaborative projects and efforts, the acronyms of which (e.g., PNAMP, CSMEP, ISEMP, IMWs) 

constitute an alphabet soup that would whet the appetite of even the most jaded bureaucrat. Because 

monitoring and evaluation is critical to learning lessons from past and ongoing projects, improved 

coordination among all interested parties will facilitate information transfer throughout the Columbia 

River Basin. 

For years the ISAB and ISRP have emphasized the need for adequate monitoring and evaluation 

of habitat restoration projects. In response to the need for better monitoring and for improved 

coordination of monitoring programs among federal, state, and tribal organizations, especially in light 

of tributary habitat improvement goals in the recent BiOps, natural resource stakeholders undertook 

several initiatives early in the decade to bring together the region’s diverse RM&E programs. Briefly, 

these include: 

The Pacific Northwest Aquatic Monitoring Partnership (PNAMP). This large partnership includes 

many federal, state, and tribal organizations. PNAMP’s goal is to provide a forum for coordinating 

aquatic habitat and salmonid monitoring programs.16 It has the broadest membership of any of the 

 

16 www.pnamp.org/

26 
regional aquatic M&E partnerships. Most of PNAMP’s recent projects have dealt with standardizing 

and improving habitat inventory methods. 

The Collaborative Systemwide Monitoring and Evaluation Project (CSMEP). CSMEP was initiated in 

2003 to coordinate efforts to improve the quality, consistency, and focus of fish population and 

habitat data to answer key monitoring and evaluation questions relevant to major decisions in the 

Columbia River Basin.17 CSMEP has a very strong adaptive management emphasis. It is 

geographically confined to the Columbia River Basin and is administered by the Columbia Basin Fish 

and Wildlife Authority, and partners include a number of federal, state, tribal, and private natural 

resource organizations. CSMEP has more of a fish population focus whereas PNAMP has more of a 

habitat focus. CSMEP’s relationship to PNAMP is illustrated in the diagram below. 

The Integrated Status and Effectiveness Monitoring Program (ISEMP) is noteworthy because the goal is 

to design, implement, test, and evaluate status and trends, and effectiveness monitoring programs, for 

salmon and steelhead populations and habitat in the Columbia River Basin.18 ISEMP is focused on 

three large “pilot projects” that include the John Day, Snake, and Wenatchee River subbasins, where 

aquatic habitat and salmon and steelhead populations will be carefully monitored. The program is 

administered by the Northwest Fisheries Science Center (NOAA Fisheries) and is designed to test 

the robustness of monitoring protocols, indicator metrics, and sampling designs currently used in 

monitoring programs. It is perhaps best characterized as an M&E methods development and 

assessment project, with the products being available for use in both PNAMP and CSMEP. 

Also noteworthy is the network of Intensively Monitored Watersheds (IMWs) that have been 

implemented throughout the Pacific Northwest. These long-term studies have the greatest likelihood 

of providing crucial information about the causal mechanisms that underpin our assumptions about 

restoration effectiveness. These watersheds typically range in size from relatively small (HUC-7) to 

mid-size (HUC-5) catchments in which habitat improvement treatments are applied in an 

experimental manner, i.e., with treatment and control sites, and where fish population responses will 

be monitored for several generations to determine the changes attributable to restoration. IMW 

studies are primarily long-term research, and restoration treatments are supposed to follow pre-

 

17 www.cbfwa.org/csmep/

18 www.nwfsc.noaa.gov/research/divisions/cbd/mathbio/isemp/index.cfm

27 
planned implementation schedules that are carefully monitored over many years. Funding for IMWs 

varies widely, and many sites are located out of the Columbia River Basin. Thus, there is no central 

clearinghouse or web site for IMW data. Nevertheless, results from IMWs will be used directly in 

PNAMP, CSMEP and ISEMP. Some long-term intensively monitored watershed studies also exist in 

British Columbia, although they are not formally tied to their U.S. counterparts. 

Most of these efforts began about five years ago and involved multi-year planning processes in 

which RM&E work plans were drawn up and reviewed – many by the ISRP. Only in the last two 

years or so have these programs begun to implement their proposed activities. It is premature at this 

point to judge whether those efforts constitute successes or failures. However, the ISRP has 

consistently supported the goals of these programs in the past, and, in general, we are optimistic that 

they will lead to improved feedback on the effectiveness of tributary habitat restoration actions to 

improve the success of future restoration projects. 

Virtually all of the collaborative efforts described above involve partnerships between multiple 

organizations and there are many instances where a scientist or agency is an active participant in 

several of these programs. As a result, the boundaries between these large-scale M&E programs are 

intentionally blurred, with information frequently being exchanged among the projects. However, at 

present there is no central repository for all the habitat M&E data being collected in the Columbia 

River Basin. This remains an issue worthy of attention, as many of the organizations collecting 

monitoring information are rapidly becoming swamped with data. This is especially true for the 

newer generation of remotely sensed environmental data, which often constitute extremely large data 

sets. The risk is that individual organizations may lose important habitat monitoring information or 

be unable to effectively share data without a regional clearinghouse for electronic storage and 

retrieval. The ISRP recognizes that such a repository will never be fully complete or up-to-date, but 

the development of new monitoring technologies deserves a parallel effort to properly archive the 

information that will be generated by the collaborative M&E programs mentioned above, as well as 

from the monitoring of individual restoration projects carried out as action items in subbasin plans 

(also see ISRP 2000-319). 

The ISRP therefore supports the goals and objectives of the regional M&E partnerships. 

Because we are in the early years of implementation for projects such as ISEMP and the IMWs, it is 

unrealistic to expect that answers for many of the very large, difficult habitat questions will soon be 

forthcoming, e.g., can tributary habitat restoration offset losses to operation of the hydrosystem? 

However, the Council embarked on a productive course of action in supporting the development of 

these M&E partnerships, and staying the course will give them an opportunity to generate 

information of broad use to policy makers. 

 

19 http://www.nwcouncil.org/library/isrp/isrp2000-3.htm 

28 
Emerging M&E Technology 

The ISRP recognizes that cost has been a barrier to 

monitoring habitat restoration projects, and for this reason 

we support the development and refinement of costeffective monitoring technologies in order to improve 

adaptive management. In particular, advances in remote 

sensing have made it possible to assess the condition of 

habitat features (e.g., vegetative cover and surface water 

temperature) over large areas in a relatively short time. 

Remote sensing will never replace more traditional methods 

of data collection for some environmental parameters, but 

for many aspects of implementation monitoring and some 

aspects of effectiveness monitoring the benefits of large scale 

remote sensing are obvious. Technologies such as high 

resolution Light Detection And Ranging (LIDAR) can 

provide enormous amounts of information on topography 

and forest condition during the process of generating digital 

elevation maps.20 However, the cost of collecting LIDAR 

data is high, and there is a steep curve in learning how to 

translate the discrete-return point clouds produced by LIDAR flights into usable habitat information. 

Nevertheless, we anticipate that the cost of LIDAR and other remotely-sensed coverage will 

gradually decline in the coming years and the analytical tools associated with them will mature, 

becoming easier to apply. The ISRP believes investments in new remote sensing technology can 

eventually pay off in the form of cost-effective monitoring techniques that provide rapid feedback on 

habitat restoration actions 

LIDAR image, USGS 

The ISRP also feels the time is right for focusing more attention on the “E” in M&E. Properly 

evaluating the benefits of habitat restoration requires, among other things, separating the effects of 

the project from natural environmental noise and determining whether the status and trend of the 

target habitats and species are responding to the restoration action as anticipated. This will likely 

include novel statistical approaches and new performance metrics. Interesting work on developing 

integrated response metrics is being done as part of the ISEMP project, and we strongly support this 

effort. Development of novel approaches to analyzing habitat and population monitoring data is 

needed, particularly as conditions in tributaries shift in response to climate change. Novel approaches 

are also needed to include changes in ocean survival as a factor influencing population changes. 

Having better evaluation tools will become an important part of the criteria for project selection. 

On a related issue, the ISRP suggests that habitat models deserve another review. It has been 

seven years since the ISAB reviewed the models being used for habitat improvement decisions in the 

Columbia River Basin (ISAB 2001-1). Since then, models such as Ecosystem Diagnosis and 

Treatment (EDT) have been widely used in developing subbasin plans and have continued to play a 

role in identifying restoration priorities. Little has been done to peer review or validate these models 

and to determine whether their predictions were accurate. Are there new models and decision 

support tools that can provide additional information for project selection and prioritization? A 

review of habitat models could also help inform the efforts of the regional M&E partnerships. The 

review could be a joint ISAB-ISRP effort. 

 

20 See http://lidar.cr.usgs.gov/

29 
BENEFITTING WILDLIFE: RETHINKING CREDITING AND MONITORING IN THE WILDLIFE 

PROGRAM 

Throughout the Fish and Wildlife Program’s history, land acquisition (by fee simple or easement) 

has been the major component of wildlife mitigation. As land is acquired with Program funds, it has 

become customary for the Program to continue to support operation and maintenance (O&M) of 

the acquisition in perpetuity. Thus, over time, an ever-increasing proportion of funds will be 

allocated to maintaining the status quo. This might appear to be an economic, rather than scientific 

issue, except that maintaining status quo does not improve the situation for wildlife. Most project 

objectives are quite general (improve habitat for mule deer), and many lack focal species or a full 

complement of focal species that represent the diversity of the species that were initially impacted by 

the hydrosystem development. Reporting has relied on activities (built X miles of fence) rather than 

the activities’ effects on habitat quality (control of trespass grazing increased winter forage for mule deer 

by 40%) or on populations of target species (additional winter forage, increased doe survival 15% and fawn 

production now exceeds the replacement level for this herd). 

This acquisition and reporting pattern has evolved based upon the assumption that habitat is an 

acceptable surrogate for populations. This assumption underlies procedures like Habitat Evaluation 

Procedures (HEP; http://www.fort.usgs.gov/Products/Software/HEP/) and a more complex, but 

conceptually similar procedure, Combined Habitat Assessment Protocols (CHAP). While HEP may 

have represented the state of the science underlying wildlife management when it was formulated, 

HEP is now seldom used by wildlife habitat researchers. Implementation of the Endangered Species 

Act in recent decades has spurred evaluation of the “habitat as surrogate” concept and found it 

lacking, both biologically and legally. This assumption has not been tested within the scope of 

Program activity, but such tests would be impossible in most cases due to a lack of target species or 

measurable habitat objectives based upon documented needs of the population in question. Given Bonneville’s 

commitment to HEP and Habitat Units (HUs) for crediting against losses, there can still be better 

strategies for using Program resources to evaluate responses to wildlife habitat management activities 

and to mitigate wildlife losses. Habitat is necessary, but not sufficient for restoring populations. Thus, 

we find ourselves in a quandary over choosing lands for acquisition as well as determining if the 

lands obtained and the wildlife program are meeting biological objectives. 

Prioritization 

The first of these strategies is prioritization. Subbasin plans were intended to provide a basis for 

prioritizing Program investments within each watershed based upon scientific assessments of 

resource and population conditions and opportunities related to focal species. Not all of the plans 

were able to develop priorities. Only a small number of the FY2007-09 proposals substantively used 

the plans, most just referenced pages in the plan that supported their proposal. There was little 

evidence that the plans actually influenced thinking about what must be done and how best to do it. 

To date most acquisitions have been opportunistic – the land was available and it supported some 

kind of wildlife, usually generalist rather than specialist species. There could be a rubric for 

evaluating, even guiding acquisitions based on priorities among focal species. For example, a great 

deal more deer habitat has been acquired than neo-tropical migrant bird habitat. By considering all 

species equally, as now appears to be the case, the Program is unlikely to mitigate for lost species 

diversity. Land acquisitions could also be ranked on the relative importance of a parcel to a particular 

population of focal species. Factors like relative scarcity of particular habitats, contiguity to other 

protected habitat, role of the parcel in the lifecycle of a species and degree of restoration (if any) 

needed could be used to compare acquisition opportunities, or even to predetermine an acquisition 

strategy. Such a structure of species and parcel priorities would also allow determination of priorities 

among subbasins, which is currently difficult. This model of decision making is being used 

30 
successfully by organizations such as The Nature Conservancy that rigorously seek the greatest 

impact of expending scarce resources. The Willamette proposal (20072600: Acquisition of a Conservation 

Easement over 1084 acres of Upland Prairie and Oak Habitat, Willamette Subbasin) is a fine example of this 

strategy at work and addresses both rare habitats and rare or unique species. This approach might 

even lead to RFPs directed at species or habitats currently neglected within the Program. 

Monitoring Approaches 

Several wildlife projects have shown tremendous wildlife responses to management activities, 

especially single focal species projects where specific research procedures were used for monitoring 

the response to habitat improvement, e.g., sharp-tailed grouse in eastern Washington. However, 

when habitat management is aimed at multiple species or wildlife in general, can a standard approach 

be used, or must the “monitoring” approach decision be made on a case by case basis? And, who 

makes the decision, or who decides after the fact that the monitoring approach used was 

appropriate? Perhaps wildlife habitat experts should be invited to make presentations to the 

ISRP/ISAB. Presentations to the panel on this subject matter have not been made in recent years. 

The approach for general “crediting” seems set (and not subject to change), but the approach or 

series of approaches for “monitoring” different focal species of interest needs to be addressed. 

Greater emphasis on focal species would allow development of measurable objectives, and 

monitoring approaches that actually document the effectiveness of management practices relative to 

the focal species. Under the present system, there has been minimal monitoring of the benefits of 

acquisitions or O&M to actual wildlife populations. Even with no change in the criteria for evaluating 

proposals for acquisitions and O&M, there is room for significant improvement in writing specific, 

measurable objectives that would lead to credible monitoring, evaluation, and adaptive management. 

A common example of this opportunity is invasive species (weed) management. Many O&M 

proposals that dealt with this issue included budget items for personnel, 4-wheelers, and other 

resources to spray weeds. Then they reported how many acres were sprayed, rarely identifying what 

species were targeted, or to what effect. Many private landowners and agencies are finding it more 

economical and more effective to contract out what spraying must be done to professionals, often 

with County Weed Boards. Further, herbicide use is only one aspect of an integrated pest 

management (IPM) plan that involves adjacent landowners, local government, and others in 

managing invasive species in a holistic manner that includes limiting opportunities for new species to 

arrive in the area, surveillance to quickly control new populations before they become established, 

and using a suite of complementary methods to control or eliminate established populations. This 

might include targeted grazing, use of bio-control insects, establishing more desirable species that can 

out-compete invasives, and maybe the use of herbicides. This state-of-the-art IPM approach is not 

being used within O&M programs that emphasize the status quo. 

More specific management objectives would lead to this more comprehensive approach. For 

example one objective might be to develop a GIS map of all invasive species on a parcel and then 

keep it up to date. This would be followed by goals such as eliminating certain species on specific 

acreages and halting spread of other species. Recording each treatment on the GIS system leads to 

very specific monitoring regimes to follow the results of each treatment. Such an approach may be 

more cost-effective and would undoubtedly be more ecologically effective than current practice. It 

would be very useful for the ISRP to review a subset of early acquisitions, supported by regular 

O&M funds, to see what has actually been accomplished towards program goals. This could lead to 

criteria for identifying the most promising acquisitions and perhaps to operations standards that must 

be maintained to receive O&M funds. Another option might be linking further O&M funding to 

31 
successful M&E. Proper management should lead to the need for fewer inputs over time as 

ecological function is restored and the system becomes more self regulating. Funding should shift 

over time from O&M to more M&E if management is successful. The current system rewards a lack 

of success with continuing funds to do the same thing over and over again, as in the weed 

management example. Most grant programs have a cost-sharing component that the Fish and 

Wildlife Program does not, because it is a mitigation program for which logically and legally costshare is not required unless in lieu provisions of the NW Power Act apply. However, consideration of 

benefits from cost-share and other funding mechanisms used by other land acquisition programs may 

be instructive to increase the efficiency and reduce the long-term costs of the wildlife program. A few 

of the 2007-09 proposals included provisions for future management funding (e.g., the Willamette 

endowment idea). A provision for a limited period of O&M funding might lead to more cost-share 

type commitment for long-term operation, or creativity such as demonstrated by the proposed 

endowment. The above are just a few ideas of how more benefits to wildlife might be generated by 

the Program. 

FUTURE DIRECTIONS FOR THE PROJECT REVIEW PROCESS 

With the experience gained from all its reviews over the past decade and particularly the review 

of 540 FY 2007-09 proposals, the ISRP’s first FY 2007 report provided suggestions on future review 

processes (ISRP 2006-721). The ISRP recommended in its 2005 Retrospective Report (ISRP 2005-14) 

and again in its Programmatic Comments to the 2007-09 Solicitation Report (ISRP 2006-4a), that 

future processes be modeled after the sequential multi-year provincial reviews, with potential 

alterations to more efficiently address program needs through targeted and topical (wildlife O&M, 

systemwide RM&E, lamprey, and such) solicitations. A staggered review process that provides for 

site visits, presentations, and response loops between sponsors and the ISRP improve the Fish and 

Wildlife Program because the ISRP gains a more thorough understanding of projects and projects are 

improved by incorporating the ISRP’s constructive suggestions. 

In the 2005 Retrospective Report, the ISRP also recommended that alternative review paths be 

investigated for continuing and new projects. For example, long-term operations and maintenance 

projects could receive administrative review or programmatic review of common methods, other 

continuing projects could receive periodic scientific review for progress attained (with funding of 

non-performers discontinued), and new projects could be reviewed both technically and 

administratively for responsiveness to targeted solicitations. The annual review process might thus 

concentrate on new proposals and a subset of the continuing projects. 

The ISRP’s experience in the FY 2007-09 review process further validated these earlier 

observations and recommendations. The ISRP specified the following elements to consider for 

future Fish and Wildlife Program project reviews. 

1. Establish separate solicitations (RFPs) and review tracks for new projects targeted to specific 

problems including systemwide information gaps or key limiting factors in a particular subbasin. 

 

21 www.nwcouncil.org/library/isrp/isrp2006-7.htm 

32 
2. For ongoing projects with large out-year obligations, return to multi-year, sequential reviews 

making use of the assessments, objectives, strategies, and prioritizations (if done) in subbasin 

plans. 

3. The sequenced review of ongoing projects is intended to provide for a thorough review by the 

ISRP of programs that are multi-faceted in their activities and potentially involve sponsors from 

different institutions. 

4. The ISRP review process would include (1) preliminary examination of a proposal, past reports, 

monitoring and evaluation data, and evidence of adaptive management; (2) a tour of past and 

proposed project sites; (3) presentation of the proposal (preceding or following the site visit 

depending on the review type and logistics) with an opportunity for questions from the ISRP; 

and (4) a preliminary ISRP review with a response loop to provide sponsors with the opportunity 

to incorporate ISRP suggestions. A three-day site visit schedule for an ongoing program might 

include: Day 1 – program overview and proposal presentation, Day 2 - site tour, and Day 3 – 

discussion and question and answer session. A written report by the ISRP would complete the 

review. 

5. In addition to the elements requested in the proposal form for the FY 2007-09 solicitation, the 

proposal form for ongoing projects should be revised to add sections requesting explicit 

presentation of the objectives proposed in the preceding solicitation cycles, what has and has not 

been accomplished, and how benefits to the focal species are being evaluated. 

6. A first step in initiating such a review is examining projects funded through the FY 2007-09 

solicitation and aggregating projects and topics with Council staff to sort projects to be reviewed 

by topic and by subbasin/province (geography). 

This review approach was successfully initiated in FY 2007 with the in-depth review of the set of 

projects that constitute the Umatilla Initiative and the RFP approach used to solicit innovative 

proposals. These two reviews are described in more detail below. 

The Solicitation for Innovative Projects and a Role for Research Funding 

As the ISRP had recommended, funds were allocated in FY 2007 for a solicitation for short-term 

innovative proposals aimed at testing new protocols and ideas on the ground. From the response, it 

is apparent that potential sponsors saw value in this opportunity: Fifty nine proposals were 

submitted, and of these, the ISRP found that nine fit the terms of the solicitation and met scientific 

criteria. The Council recommended six for funding. It is too early to know the ultimate contribution 

of these projects. We recommend that such a solicitation be repeated at regular intervals, every one 

to three years, to maintain consistent participation and interest in the solicitation. The value of this 

program element might be greater if a slightly longer time period could be funded, particularly where 

projects are tied to seasonal phenomena. 

It was apparent that there was a perceived need by sponsors for some mechanism to fund 

research prior to the on-the-ground, proof-of-concept stage favored in this year’s solicitation. In 

general, pure research proposals have not fared well in the general solicitations for some very good 

reasons related to likely benefits to fish and wildlife. If we knew it were a benefit, it would not be 

research, but there is the risk that unsuccessful ideas would not generate direct benefits to fish and 

wildlife. The converse risk is that for lack of specific knowledge, we squander opportunities to 

benefit fish and wildlife. Thus research should be a calculated risk. 

33 
There are some recognized research questions that, if answered, could add enormously to the 

Program’s success. One approach to this is targeted RFPs designed to generate the needed 

information through credible research for direct application to the Program. It may also be valuable 

to have a limited open research RFP with criteria that link proposals tightly to Program goals. There 

are undoubtedly talented scientists that see research opportunities we do not, but which would 

greatly enhance the Program’s effectiveness. Because research is a high risk enterprise, cost-sharing 

and other stringent requirements could be justified. Limiting the duration of funding would be 

appropriate. Successful research, after some initial Program-funded work, should merit funding from 

more research oriented programs, in which case Program funding would be leveraged into future 

work with benefits to the Program, but no further expense to the Program. 

Together, innovative, targeted, and research solicitations would be a limited component of the 

overall Program investment that should generate disproportionately positive benefits to fish and 

wildlife. 

The Umatilla Initiative Review Process as a Model Approach 

In the 2007-09 proposal review the ISRP ranked many proposals from the Umatilla subbasin as 

Not Fundable (qualified). The individual proposals often did not have clear biological objectives, did not 

provide results, and the relationship between the various projects in the subbasin were not obvious. 

Consequently the Council instructed the subbasin co-managers/project sponsors to respond to the 

ISRP concerns. The sponsors, the CTUIR and ODFW responded by producing a summary review 

document and hosting an ISRP site visit in May 2007. The process of “packaging” the set of closely 

related project proposals under a unified programmatic-type presentation in which the 

interrelationships can be clearly specified proved to be effective and efficient. It eliminated the need 

for each separate project proposal to meet standards set for ISRP review by Congress in the 1996 

Power Act Amendment. The ISRP has often found in individual proposals a failure to include 

sufficient provisions for monitoring and effectiveness evaluation. Often this could be corrected by 

improved coordination among the projects to ensure that within the set there will be metrics 

designed to develop some measure of effectiveness in benefiting fish and evidence that monitoring 

and evaluation will actually be carried out. This serves as an example where program/subbasin level 

M&E meets the needs of “provisions for M&E” required for an individual project. 

Site Visit with Umatilla program, Council, and ISRP participants 

34 
35 

The ISRP recommends that the procedure be adopted for similar watershed-scale efforts, for 

example the Yakima-Klickitat Fisheries Program. 

Further, the ISRP recommends that, once it has reviewed and the Council approved a set of 

proposals at the watershed scale, future reviews of those subbasins might be conducted on a multiyear cycle, or abbreviated to include only review of project proposals that represent some significant 

change in direction, as might occur as a result of adaptive management. 

________________________________________ 

w:\em\ww\isrp projects and reports\1 final isrp reports\isrp 2008-4 2007 retro.doc