Independent Scientific Advisory Board

for the Northwest Power Planning Council

and the National Marine Fisheries Service

851 SW 6th Avenue, Suite 1100

Portland, Oregon 97204

Model Synthesis Report

An Analysis of Decision Support Tools Used in

Columbia River Basin Salmon Management

March 2, 2001

ISAB 2001-1

Peter A. Bisson

Charles C. Coutant

Daniel Goodman

Robert Gramling

Dennis Lettenmaier

James Lichatowich

Eric Loudenslager

William Liss

Lyman McDonald

David Philipp

Brian Riddell
ISAB 2001-1: Model Synthesis Report

i

ISAB Model Synthesis Report

Contents:

INTRODUCTION.........................................................................................................................................................1

MODELS IN COLUMBIA RIVER BASIN SALMONID MANAGEMENT...............................................4

SALMON SCIENCE AND THE SALMON RECOVERY EFFORT ................................................................................... 4

PRIMER OF ECOLOGICAL MODELING....................................................................................................................... 4

THE THEORY OF ECOLOGICAL MODELING.............................................................................................................. 6

APPROACHES TO MODELING IN THE COLUMBIA RIVER BASIN............................................................................ 7

REVIEW OF MODELS: RESULTS VERSUS APPROACHES.......................................................................................... 9

COMPARISON OF THE ANALYTICAL APPROACHES (SUMMARY TABLES).......................................................... 11

SYNTHESIS OF THE ANSWERS TO THE ISAB’S QUESTIONS................................................................................. 24

EVALUATION OF THE MODELING EFFORTS........................................................................................................... 27

MOVING FORWARD WITH SUPPORT FOR MULTIPLE MODELING EFFORTS....................................................... 30

USE OF MODELS IN THE DECISION PROCESS IN THE COLUMBIA RIVER BASIN.................31

THE SCIENCE-POLICY INTERFACE.......................................................................................................................... 31

INDIVIDUAL MODELS AND THE DECISION PROCESS............................................................................................. 36

COMMUNICATION AND SOURCES OF CREDIBILITY.............................................................................................. 39

APPENDIX I: TECHNICALITIES OF QUANTIFICATION OF UNCERTAINTY..............................41

APPENDIX II. DESCRIPTION OF DECISION SUPPORT SYSTEMS USED TO GUIDE

SALMON RECOVERY PLANNING IN THE COLUMBIA RIVER BASIN ...........................................46

ECOSYSTEM DIAGNOSIS AND TREATMENT (EDT)............................................................................................... 46

CUMULATIVE RISK ASSESSMENT (CRI)................................................................................................................ 49

BAYESIAN BELIEF NETWORKS (BBN) OF THE INTERIOR COLUMBIA BASIN ECOSYSTEM MANAGEMENT

PROJECT ..................................................................................................................................................................... 53

PLAN FOR ANALYZING AND TESTING HYPOTHESES (PATH)............................................................................. 58

APPENDIX III. WRITTEN RESPONSES OF THE PRINCIPAL SCIENTISTS TO THE ISAB’S

QUESTIONS ABOUT THEIR ANALYTICAL APPROACHES ..................................................................64

ICBEMP ANSWERS.................................................................................................................................................. 66

CUMULATIVE RISK INITIATIVE ANSWERS............................................................................................................. 75

PATH ANSWERS PART I - GENERAL AND FLUSH.............................................................................................. 87

PATH ANSWERS PART II - CRISP AND ALPHA LIFE CYCLE MODEL............................................................100

PATH ANSWERS PART III - EXPERIMENTAL MANAGEMENT ..........................................................................104

ECOSYSTEM DIAGNOSIS AND TREATMENT ANSWERS.......................................................................................111

CRITFC COHORT MODEL ANSWERS...................................................................................................................120
ISAB 2001-1: Model Synthesis Report

1

ISAB Model Synthesis Report

An Analysis of Decision Support Tools Used in

Columbia River Basin Salmon Management

Introduction

Several major analytical efforts are underway to support decision-making for

salmon restoration in the Columbia Basin. These efforts include NMFS’ Cumulative Risk

Initiative (CRI), the Plan for Analysis and Testing Hypotheses (PATH), the Northwest

Power Planning Council’s use of the Ecosystem Diagnosis and Treatment protocol

(EDT), the US Forest Service’s and Bureau of Land Management’s effort undertaken for

the Interior Columbia River Basin Ecosystem Management Plan (ICBEMP-BBN), and

the Columbia River Intertribal Fish Commission’s COHORT model (CRITFC). Within

the region, conflict has developed over the role these models should play in the decisionmaking process. Part of this conflict is manifested in the current debate over the need for

multiple modeling approaches, such as CRI and EDT, when considerable time and money

has already been invested on the PATH models. This debate is exacerbated because the

models were developed for different purposes and they take different analytical

approaches, use somewhat different data sets, and make different assumptions.

 Policy-makers (and the public) are faced with important decisions concerning

salmon restoration. The conclusions of the various models contribute to the decisionmaking process. However, because the objectives, data, and assumptions vary among the

models, it is not surprising that their conclusions differ. Decision-making could be

hampered if the region becomes engulfed in a “collision of models,” with various interest

groups and agencies advocating the conclusions that best support their interests and

mandates. Such a debate will undoubtedly occur as an inevitable consequence of the

gravity and implications of impending decisions, and the lack of creditable scientific

conclusions concerning the probability (feasibility) of outcomes following any particular

management intervention. The ISAB undertook this project with the intent of bringing

some clarity to the regional debate. The purpose of the project was two-fold:

1. Comparative Synthesis - - to clarify the questions or problems that each model

was designed to address and to provide an overview of each of the models and a

synthesis that describes both consensus conclusions and areas of disagreement

among the models. We focused on the main results or conclusions of the models

and did not perform an in-depth evaluation of the structures of the models and the

quality of data used to calibrate and validate them. Given the complexities of the

models and the histories of their development, the latter would be a daunting task.

2. Assessment of the Role of Models in the Decision-making Process - - to assess

how science interfaces with decision-making, specifically relating to the roles

models play in the process.
ISAB 2001-1: Model Synthesis Report

2

In undertaking this project, the ISAB requested the assistance of the principal

scientists involved in each analytical approach. We asked that they 1) provide written

responses to a series of questions about their modeling approach (the verbatim responses

are attached in the Appendix III), 2) give an oral presentation of their responses and

participate in a discussion of the responses with the ISAB, and 3) participate in a panel

discussion with the ISAB and the representatives of each of the modeling approaches.

The ISAB asked the scientists to address the following questions in their written

responses:

1. What is the purpose of your modeling effort? What questions or problems were

your models designed to address?

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s

conclusions relative to the following:

a. The efficacy of dam breaching or draw down to natural river levels for

delisting ESA species and restoring diverse and productive populations of native

fishes throughout the Columbia River Basin?

b. The efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

Consider hatcheries that mitigate for lost habitat and those used to supplement

depleted stocks.

c. Allocation of harvest and harvest levels needed to delist ESA species and

restore diverse and productive populations of native fishes throughout the

Columbia River Basin?

d. The efficacy of restoration of tributary and mainstem habitat for delisting ESA

species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

3. What kinds of information or data are needed to run your model?

4. What are the strengths and weaknesses of your model?

5. What are the assumptions of your model?

6. How does your model address uncertainty?

7. All models make predictions. Why do you think your model’s predictions are

accurate?

8. How does your modeling effort relate to or contrast with the other modeling

efforts?

9. What advice would you give decision-makers on how they should use your model

to support decisions regarding salmon recovery in the Columbia River Basin?
ISAB 2001-1: Model Synthesis Report

3

The ISAB held a meeting with the principal scientists on May 3, 2000. We asked

them to focus their presentations on the following areas:

1. Provide a brief overview of the analytical approach. Include the kinds of primary

data that are needed, quantities derived from the primary data, and conceptually

take us through the analytical steps in your approach.

2. Discuss the major assumptions of the approach.

3. Address how the approach deals with uncertainty.

4. Discuss how you believe the analytical approach should be used to support

decision-making.

Subsequent to the ISAB’s meeting with the scientists, we met individually with

the following regional decision-makers on June 7 and 8, 2000: Lori Bodi (Senior Advisor

for Fish and Wildlife at the Bonneville Power Administration), William Stelle (then

Regional Administrator, National Marine Fisheries Service), Robert Lohn (Director of

Fish and Wildlife Division, Northwest Power Planning Council), Donald Sampson

(Executive Director, Columbia River Inter-Tribal Fish Commission), and Roy

Hemmingway (Salmon Advisor, Oregon Governor’s Office). The ISAB asked the

decision-makers to address the following questions at the meeting:

1. How do you believe the results produced by the analytical frameworks and other

scientific information should be used in the decision-making process?

2. What kind of information do you think the analytical frameworks should provide

to best inform decision-making?

3. Do you believe communication between scientists, particularly modelers, and

decision-makers need improvement, and if so, how could it be improved?

4. All the analytical frameworks contain a high degree of uncertainty, in part due to

the lack of high quality data. Do you believe the level of uncertainty is being

adequately communicated to you? How do you incorporate scientific uncertainty

in the decision-making process?

5. The five analytical frameworks represent different approaches to somewhat

different problems. What is your view of the benefits and detriments for decisionmaking of having alternative analytical frameworks? From your perspective, do

you view the existence of alternative analytical frameworks and results that

sometimes conflict as an impediment to decision-making? In making decisions,

will you give more weight to some or one of the frameworks over the others? If

so, why?
ISAB 2001-1: Model Synthesis Report

4

6. What could the ISAB include in its review of the analytical frameworks that

would make the review most useful to you?

The ISAB did not request the decision-makers to provide written responses to the

questions.

This report is a synthesis of the answers to the questions posed by the ISAB and

subsequent discussions with the principal scientists and decision-makers.

Models in Columbia River Basin Salmonid Management

Salmon Science and the Salmon Recovery Effort

The popular press has suggested that Columbia Basin salmon management

constitutes a hugely expensive "failure" of "salmon science." We will not comment here

on the difficult and inflammatory question of whether the recovery effort, to date, should

be judged a failure. We will take this opportunity, however, to correct the misperception

about the role of science in the recovery effort. Put most simply, the conventionally cited

huge amounts of money spent on the recovery effort were not spent on "salmon science."

The bulk of the money was spent, and continues to be spent, on management actions.

Historically, the role of science in the process has not been large. Some of the

management actions lack a strong scientific basis, and the results of some actions have

not been monitored with enough thoroughness to determine their effect.

Overall, two general conclusions are evident. First, the science that was done has

contributed substantial knowledge. Second, more science would have contributed more

knowledge. There are now prospects for increasing the investment in science; the

challenge will be to direct that investment at the right priorities.

The ISAB has reviewed the major modeling approaches that bear on Columbia

Basin salmon management. While these approaches constitute "large" modeling projects

by the usual standards of ecological modeling, they require a very small fraction of the

cost of the recovery effort. The ISAB believes that the investment in modeling and data

collection and analysis in the Basin should increase at least in proportion to the

desperately needed increase in investment required to establish an acceptable monitoring

program.

Primer of Ecological Modeling

Ideally, ecological models, like models of any complicated system, constitute a

way of organizing and communicating information. When modeling is done thoughtfully,

it provides a systematic and objective way of predicting what we can predict reliably, and

identifying what we cannot predict reliably. When the predictive power is slight--as is the

case when models are applied to many of the current salmon management questions--a

good model serves the very important roles of generating hypotheses and pin pointing the
ISAB 2001-1: Model Synthesis Report

5

crucial gaps in information. Knowledge about the information gaps provides a valuable

guide for setting priorities for new data collection, and it suggests new experiments to

resolve the critical uncertainties.

Ecological models provide a means of examining the combined implications of

sets of assumptions about mechanisms and sets of measurements on the system of

interest. The motivation for this exercise may vary. The motivation is often quite

different in academic and management contexts.

In an academic setting, the greatest interest usually centers on use of the model to

"test" the assumptions, in order to gauge the current state of theoretical understanding.

This is generally carried out by quantifying indices of internal consistency between the

data and the relationships implicit in the assumed mechanisms. Often, the goal will be to

compare alternative sets of assumptions, to see which are more likely to be true. These

academic uses of models are tolerant of very large discrepancies between the model and

reality. The model, by design, is often highly idealized, in order to shed light on

particular mechanisms of interest, whereas the reality that is the source of measured data

is expected to be far more complicated. The mismatch is treated as "noise," that can be

largely ignored provided it is not so large as to obscure the contrast between alternative

sets of assumptions.

In a management setting, the greatest interest usually centers on the use of models

to make predictions -- generally these are predictions about the consequences of

alternative interventions that are under consideration. Here, it is taken for granted that the

model is an approximation, and probably an oversimplification, and that the actual

outcomes will not exactly match the predictions. But the consequences of the decision

will result from the actual outcomes. For this reason, discrepancies between the model

and reality are of crucial concern. Owing to limitations of our theoretical knowledge and

limitations of our available data, these discrepancies may be large, and unavoidable.

Because the predictions of the models become a basis for management decisions, the

discrepancies between the model and reality cannot just be dismissed as noise. The

discrepancies need to be analyzed very carefully to provide a characterization of the

uncertainty of the predictions.

The point of the uncertainty analysis for a management model is two-fold. One

goal, which should be obvious, is to reduce the uncertainty to the extent that is feasible

(even if this is achieved at the expense of loss of elegance or loss of a clear relation to

academic theories). The second goal, which is a little less obvious, is to quantify the

uncertainty as accurately as possible, in order to provide a rational basis for bet-hedging

and setting margins of error in the use of the predictions in a decision process.

Characterization of uncertainty in predictive use of a model is inherently

probabilistic, and is carried out by statistical procedures. The quantification of

uncertainty is reported in terms of measures of the spread in a probability distribution of

the discrepancy between prediction and actuality. The reason for the focus on

probabilistic characterization is that deterministic discrepancies can simply be absorbed
ISAB 2001-1: Model Synthesis Report

6

by calibration. If we know that a prediction is always 10% high, we simply adjust our

prediction accordingly. Uncertainty is concerned with the component of the discrepancy

that is not so consistent. In other words, uncertainty characterization is concerned with

determining the probability that the prediction will be high, or low, by any stated amount.

As a technical shorthand notation, the uncertainty characterization is often communicated

in terms of the mean or variance of a mathematically defined probability distribution.

The Theory of Ecological Modeling

The "classic" paper on the theory of ecological modeling is R. Levins (1966)

 " The strategy of model building in population biology." Amer. Sci. 54:421-431. This

paper, however, was written from a perspective of academic science, and is not especially

concerned with management, so it is only part of the story. It neglects statistics and the

quantification of uncertainty. For this reason, this classic paper will provide insight on

what is good about CRI, but not about its shortcomings; conversely, it will explain the

worst weaknesses of PATH but will neglect PATH's strengths; and it will be totally at a

loss for dealing with either the strengths or weaknesses of EDT and ICBEMP-BBN, since

expert systems and GIS had not yet been developed at the time the paper was written.

Briefly, a practical update of this theory of ecological modeling would recognize

that "scientific model" means many things to many people. So much so, that the intention

to classify and analyze all the possibilities is rather daunting. It may be helpful to try to

maintain a pragmatic viewpoint and focus on models in a very particular context.

In the present context, we are interested in comparing and evaluating a set of

models whose primary purpose is environmental decision making-- more particularly,

decision making about big budget interventions and big budget data collection programs,

under conditions of high uncertainty and in a very large and complicated ecosystem.

Thus, these models are being used to make predictions under circumstances in which we

cannot reasonably expect the predictions to be very accurate. To use these models and

their predictions intelligently, we have to approach this use in the spirit of intelligent

gambling. The first rule of gambling is that to place intelligent bets, you have to know the

odds. In the present context, this means that we would need to know, in a quantitative

way, how good the predictions of the respective models are. This knowledge is invariably

probabilistic.

In other words, "knowing how good the predictions are" means being able to state

that if the model predicts X, we have assurance that with probability Y the true outcome

will be within plus or minus Z of the prediction X. "Knowing how good the predictions

are" means actually having solid numbers for Y and Z, given X. When we know how

good the predictions are, in this sense, we can make sensible decisions in light both of the

prediction (X), and the probability (Y) that the actual outcome will not be what was

predicted, but will deviate by a stated amount (Z). This probabilistic knowledge lets us

hedge our bets and spread the risk of being wrong in proportion to the probability of

being wrong and the costs of being wrong.
ISAB 2001-1: Model Synthesis Report

7

So, the worth of a model in a decision-making context depends on knowing how

accurate its predictions are. All other things being equal, we of course prefer a more

accurate model to a less accurate model. But if the model is substantially inaccurate, as is

likely inevitable in Columbia River salmon management, knowing more about

quantifying the inaccuracy is actually more important than making marginal

improvements in the accuracy. This is a difficult concept for many people to come to

grips with. The lack of understanding on this point often leads to distorted priorities in an

overemphasis on "research to reduce uncertainties" and irrelevant controversy about

which estimate is "best."

Quantification of the accuracy of the model predictions is generally accomplished

by summarizing the fit of model predictions to actual measurements of the predicted

quantities (See Appendix I). This is very technical; there are several ways of doing it, and

there is not a clear professional consensus on which way is best. Regardless of the

mathematical details, there is a known list of requirements for success in the endeavor.

Among the most important requirements are the necessity for knowing the precision of

the data themselves, a careful identification of actual data versus derived or surrogate

quantities, and a rigorous separation between the data used to tune or initialize the model

and the withheld data that are then compared to predictions. This last requirement is

surprisingly tricky, and depends on a tractable and transparent model structure and a

fairly deep understanding of spatial and temporal correlation patterns in the data. A final

important and subtle consideration in quantifying the accuracy of model predictions is

correcting for the fact that the statistical procedure is inherently retrospective, while our

interest usually is in the accuracy of predictions about the future.

Approaches to Modeling in the Columbia River Basin

There are now at least three distinct philosophies of modeling represented in the

models available for decision support in Columbia Basin salmon management. Typically,

scientists are well trained in at most one of the philosophies. These philosophies are:

1. Decision-analysis, embodied most clearly in PATH-FLUSH and PATH-CRiSP,

2. Statistical, embodied most clearly in CRI, and

3. Expert-system, embodied most clearly in ICBEMP-BBN and EDT.

A more detailed description of the models is given in Appendix II.

Of course, the actual models are not 100% pure manifestations of the respective

philosophies, but still each model clearly relies much more heavily on one approach than

others. In addition, the strengths and weaknesses of the various approaches do influence

the usefulness of the various modeling efforts.

Of the three approaches, the decision-analysis approach is most closely directed at

providing management advice, and it is the most formal about factoring uncertainty into
ISAB 2001-1: Model Synthesis Report

8

the analysis. If this modeling approach is successful, it has the potential to be the most

useful to decision-makers.

The decision-analysis approach, however, is very difficult to implement

successfully. Its success depends crucially on the engagement of the actual decisionmakers in framing the questions that need to be answered, identifying the management

options that are under consideration, and in defining the "values" put on the various

possible outcomes. The decision analysis approach also requires clear communication

between the technical analysts and the decision-makers, including communication about

complicated matters of risk, probability, and uncertainty. Such engagement and

communication is difficult to achieve in the institutional setting of Columbia Basin

salmon management, where there is so much fragmentation of decision-making authority.

The statistical approach is scientifically the most classical of the three, and can

operate with a large degree of detachment from policy. It proceeds by testing hypotheses

and estimating life history parameters with available data. This has the advantages of

clarity, rigor, and empirical objectivity. The limitation of this approach is that the scope

of the questions that can be answered with satisfactory conclusiveness is restricted by the

availability of data. In a domain that is data-poor, too many pressing questions may go

unanswered. This approach may be scientifically correct, but it does not address the

needs of the managers who recognize that "no decision" is still a decision.

Expert-system approaches fill gaps in the data with expert opinion. In the context

of the salmon problem, expert opinion allows consideration of the most concrete menu of

specific options for actual management. Expert opinion is, admittedly, a weaker basis for

scientific prediction than is a mathematical relationship validated with an archive of

quantitative empirical data. It is important to recognize, however, that at the level of

spatial resolution and environmental detail required to make salmon management

decisions, there are no available validated mathematical formulae for predicting reliably

the effects of management actions on salmon, and there is no adequate data archive for

deriving such formulae.

In this light, the expert-system approach may well be a reasonable and practical

method for providing tentative answers to some management questions that do need to be

addressed quickly. There is a need, now, to make quite specific assessments about a

broad spectrum of possible management interventions, to decide which interventions are

worth trying, and to decide where, and on what scale, they should be implemented. It is

clear, however, that the tentativeness of the answers from the expert-system approach

needs to be kept in mind when these answers are factored into management decisions.

When there is a need to make decisions in the face of significant uncertainty,

there are valid uses for both secure information and more speculative information such as

that derived from expert opinion. In the short term, speculative information is the only

option for filling the gaps in the secure information. The two kinds of information should

serve different roles in the decision process, and things will go badly if those roles get

confused.
ISAB 2001-1: Model Synthesis Report

9

Basically, the speculative information generates hypotheses, whereas secure

scientific information accumulates through the careful testing of hypotheses and

estimation of critical life history parameters. The two can work together if the hypotheses

and estimates are used as a basis for designing and implementing management

experiments. Monitoring of the results then provides data to test the hypotheses or

estimate new parameters. In this way, the expert system and statistical approaches can be

complementary, and decision-analysis can optimize the mix.

Prudence demands that we perform management experiments on such a scale that

we can afford the consequences of any probable outcome. The contribution of decisionanalysis is the balancing of the prospects for each experiment: the value of the desired

outcomes, the costs of the undesired outcomes, and the value of the secure information

that will be obtained in either case.

For decision-makers to accept scientific advice on the merits of experimental

management, the nature and extent of the uncertainties have to be explained in language

that is both realistic and vivid. It is not a matter of attempting to teach the decisionmakers to like uncertainty -- why should anyone do that? The important thing is learning

to cope with unavoidable uncertainty. This is especially important when the uncertainty is

large, as is common in environmental problems.

Review of Models: Results versus Approaches

Most of the models we reviewed are, to some extent, works in progress. This has

important implications for interpretation of our comments about these efforts. Our

comments for each model, accordingly, apply to the state of the model at a particular

stage in its development, that is, at the time when we investigated it. To the extent that

the developers of the models view their own efforts as ongoing, rather than finished

products, we might best frame our review as comments on the respective modeling

"approaches" rather than a judgment on a specific set of results.

Roughly ranking the models according to the degree of finality of the effort, it

seemed that ICBEMP-BBN was presented as the effort that had come to the most

definitive closure, in the sense of defining a question, attempting to answer it, and

delivering the answer for the consideration of the decision-maker who had requested it in

the first place. ICBEMP-BBN still has the potential for being updated with new data and

for further refinement, but at least for the present phase of its use in a decision making

process, its developers and users are treating it as being completed.

The PATH effort had the second greatest degree of closure, in that a set of PATH

results was delivered for consideration by NMFS in developing their comments for the

Anadromous Fish Appendix to the Army Corps of Engineers draft "Lower Snake River

Juvenile Salmonid Migration Feasibility Study." In addition, PATH has ceased to exist

as an organization within the past year, but some of the component models used in PATH
ISAB 2001-1: Model Synthesis Report

10

continue to evolve, most notably CRiSP at the University of Washington Columbia Basin

Research program, and continuing analytical efforts by the state and tribal fish and

wildlife agencies (STUFA) directed primarily at smolt passage and smolt to adult

survival, and PATH-EM.

We have witnessed two presentations of CRI, spaced about a year apart, with

substantial changes in CRI over that time period being apparent. CRI explicitly represents

itself as an "approach" rather than a specific product or model, and both briefings

described ongoing analyses and plans for further development. We reviewed in some

detail a very early draft of the CRI work intended for inclusion in the Anadromous Fish

Appendix.

The EDT effort and the CRITFC model seem to have achieved the least closure.

In both cases, the results shown to us were presented as demonstrations of what to expect.

In the case of EDT, the planned systematic comparison of the model predictions against

empirical data had not yet been carried out. The Council has chartered the Regional

Assessment Advisory Committee (RAAC) to manage an effort to compare model

predictions against empirical data and to advise the Council on regional analytical needs

as part of the Council’s subbasin planning effort. Notwithstanding the tentative nature of

the EDT results that were presented to us, the basic underlying model and the nature of

the modeling approach were distinctive and clearly presented. The CRITFC model was

presented in a form that demonstrated the model structure, but some parameter values

used in this model were based on estimates that were known to be outdated, and plans for

further work were described.
ISAB 2001-1: Model Synthesis Report

11

Comparison of the Analytical Approaches

Summary of the Written Answers to the ISAB’s Questions

Purpose of Model.

ICBEMP-BBN Compare relative trends in aquatic habitat and population status of

six salmonid species among three Federal land management

alternatives.

EDT Compare a set of comprehensive visions (events and actions

affecting animal populations and their habitat) in terms of the

performance of salmon populations. EDT estimates cumulative

effect of one or more individual actions.

CRI Synthesize data and predict trends, dynamics and risks of

threatened and endangered salmonids on the West Coast of North

America.

PATH - Flush Evaluate management options intended to lead to recovery of

Snake River listed salmon and steelhead stocks. Only Snake River

ESA species are considered.

PATH - CRiSP Evaluate the effectiveness of a limited number of Snake River

salmon recovery actions including breaching the lower four Snake

River dams, increased transportation, and reduced harvest.

PATH-EM PATH Experimental Management (EM) analyzes trade-offs

between biological effects of actions, i.e., how actions influence

persistence and recovery, for Snake spring/summer chinook.

Emphasize how long one would need to monitor population

dynamics to detect changes.

CRITFC COHORT Provide an easy-to-use interface designed to allow users to test

salmon recovery options. Provide the essence of existing models

for the following factors in a single package: ocean harvest

analyses, in-river harvest, PATH downstream passage, and EDT

subbasin habitat.
ISAB 2001-1: Model Synthesis Report

12

Data Needs of Model.

ICBEMP-BBN Predictions or GIS coverages of 6th code HUCs on Federal Lands

for: road density, grazing intensity, ground disturbing activities,

exotic species, etc. Many of these are extrapolated values, not

primary data.

EDT Habitat information at the HUC-6 level on some 40 habitat

parameters. Biological information is required at the population

level (rules relating survival to habitat conditions). Many of these

“data” are extrapolated values, not original primary data.

CRI Analyses start with time series of spawner counts to estimate

recruits per spawners or annual rates of population growth. Data

on the age structure of returning fish are also needed. Many of

these 'data' are extrapolated values, not original primary data.

PATH - Flush Run reconstruction requires: redd counts, dam counts of adults,

harvest rates, spawner age composition, hatchery brood removal,

and hatchery natural spawner abundance. Other data include: flow

rates, travel times, reach survival estimates, passage indices, etc.

Many of these 'data' are extrapolated values, not original primary

data.

PATH - CRiSP CRiSP passage model predictions on smolt survival require data on

hydrosystem operations, water quality conditions and stock release

information.

PATH-EM Annual estimates of spawning escapement and age estimates, past

harvest rates, upstream survival rates, future harvest rates, and

estimates of the effects of actions on life-cycle survival (e.g.,

correlations between spawning escapement and parr to smolt

survival estimates). Many of these 'data' are extrapolated values,

not original primary data.

CRITFC COHORT Subbasin habitat data such as temperature and sediment data,

mortality rates at various life history stages (such as overwintering,

estuary/early ocean, and annual ocean mortality rates), as well as

harvest rates. The data needed are those necessary to calculate the

instantaneous mortality rate at each life stage. Many of these

“data” are extrapolated values, not original primary data.
ISAB 2001-1: Model Synthesis Report

13

Strengths of the Model.

ICBEMP-BBN Logically explicit and transparent. Results are quantifiable and

spatially explicit. The framework is flexible allowing

incorporation of both quantitative and qualitative information;

analyses can be replicated and assumptions can be tested to

determine relevance to the results.

EDT EDT accounts for cumulative effects of factors such as spatial

temporal interactions, all attributes, competition, and predation

effects. Density dependent factors are included. It translates

combinations of actions at any scale into biological performance

responses (population productivity, abundance, and life history

diversity).

CRI CRI is simple, relatively easy to duplicate, and responsive to new

and/or updated data. CRI is attempting to deal with observation

error.

PATH - Flush PATH is a cooperative, multi-agency process with peer-review by

external scientists. The decision analysis approach made it

unnecessary to have complete agreement to determine which

actions had the best chance of achieving recovery over the range of

uncertainty.

PATH - CRiSP The CRiSP passage model is calibrated with all the available

current data.

PATH-EM PATH-EM is a Ricker population projection model of Recruits vs.

Spawners. Management regimes can be as simple or complex as

desired by the model user, effecting all stocks permanently (e.g.,

drawdown) or different stocks in different ways over time (e.g.,

different nutrient enrichment schedules for different stocks and

years).

CRITFC COHORT The model has a simple user interface allowing anyone to run the

model. It is one of the few models to include the potential effects

of habitat change on population growth
ISAB 2001-1: Model Synthesis Report

14

Weaknesses of the Model.

ICBEMP-BBN Limited data and knowledge at relevant scales forcing substantial

reliance on expert opinion, and surrogates for key elements or

processes. Results can be viewed only as relative trends among

alternatives.

EDT Lack of ground truthing of input data and peer review to ensure

that rules are consistent with current information and knowledge.

Need modeling of genetic effects to be expanded. When applied to

the entire Columbia Basin EDT takes hours to run and requires lots

of computer space.

CRI Pessimistic, if one assumes that ocean conditions will soon

improve. CRI is not a formal decision theoretic approach.

PATH - Flush The complexity of analyses in PATH was not transparent to many

individuals. There is limited data for building models, testing

hypotheses about the past, and testing hypotheses about the future.

PATH - CRiSP There is no way to characterize differences in survival among fish

of a release group for different passage routes or because of presmolt conditions. The PIT tag studies in the tributaries above the

hydrosystem do not support a simple travel time survival

relationship as is contained in the CRiSP and FLUSH models.

PATH-EM There is strong reliance on spawner-recruit estimates, which may

contain systematic errors. Estimates of an action’s effects are

needed as model inputs.

CRITFC COHORT There are no density dependent effects, no stochasticity, no time

lags built in for hydrosystem or habitat changes, does not allow

changes in flow, and population sizes are not projected into the

future.
ISAB 2001-1: Model Synthesis Report

15

Assumptions required by the Model.

ICBEMP-BBN The assumed conditional probabilities adequately represent key

relationships, processes, and interpretations of management

direction. Proposed restoration and management will proceed as

described. Local habitat, biological potential, and ocean and

migratory conditions are the primary determinants of species status.

EDT EDT is a flexible, expert (cause and effect) system that begins with

a detailed habitat description and links it via assumptions to

ecological function and biological performance of specific species.

Two parameter density-dependent survival functions are assumed

for each life cycle segment (typically the Beverton-Holt equation is

used).

CRI No density-dependence in current dynamics. No directional trends

in the variation in recruits per spawner.

PATH - Flush The prospective analysis correctly quantifies the range of possible

future conditions. Under each management option, the hydroregulation model correctly computes mean monthly flows, the

passage models correctly compute passage survivals and the

Bayesian life cycle simulation model (BSM) correctly generates a

range of spawner abundances.

PATH - CRiSP CRiSP assumes survival is a function of travel time and

environmental water quality including temperature and total

dissolved gas levels.

PATH-EM Future conditions will be like 1978-1994 absent management

intervention and the estimates of an action’s effects (which are

model inputs) are accurate.

CRITFC COHORT Spawning and rearing are evenly distributed throughout currently

used habitat regardless of habitat quality. Habitat improvements are

applied first to the lowest quality habitat. A per-dam mortality rate

of 17% is assumed. John Day drawdown reduces mortality by 1⁄2 of

a dam. Hatchery fish upon release are treated as being identical to

natural origin fish with the exception of a 50% post release

mortality applied to the hatchery fish.
ISAB 2001-1: Model Synthesis Report

16

Conclusions with respect to Dam Breaching.

ICBEMP-BBN Not useful for evaluating the efficacy of dam breaching or

drawdown. Models reflect the expert's beliefs that dams have a

dominant influence on status of anadromous salmonids.

EDT Breaching dams was effective in restoring mainstem habitats to the

benefit of chinook performance. Dam removal provided significant

benefits to listed Snake River stocks, but less or minor benefit to

Upper Columbia River stocks.

CRI Dam breaching would need to increase ocean/estuarine survival by

100% or more in order to ensure positive growth rates large enough

to mitigate extinction risk.

PATH - Flush In most cases, the “natural-river” options met the survival

standards under the most pessimistic assumptions. None of the

“transportation” options met the recovery standard, except under

very optimistic assumptions. The absolute values of PATH

population projections are likely optimistic.

PATH - CRiSP Drawdown was better than transportation as a recovery measure for

spring chinook while harvest was a possible recovery measure for

fall chinook. New data now challenge the PATH conclusions.

PATH-EM Effects of dam breaching, e.g., effects of drawdown of four Lower

Snake projects, have been modeled by PATH-EM. Some

preliminary results are in the PATH-EM Report, but were not given

during the presentation.

CRITFC COHORT CRITFC has not reached final conclusions on the questions posed.

Based only on an aggregate of the Middle Fork Salmon R. spring

chinook they conclude: 1) spring chinook populations cannot be

rebuilt without dam breaching, and 2) survival gains from other

factors are needed to allow the population to rebuild, even with

dam breaching.
ISAB 2001-1: Model Synthesis Report

17

Conclusions with respect to Hatcheries.

ICBEMP-BBN Efficacy of hatcheries was not addressed.

EDT Conclusions were not given. EDT brackets alternatives by using a

range of assumptions about effectiveness of hatcheries. The initial

assumption was that hatcheries fulfilled their promise and were

comparably effective as natural populations.

CRI Because basic data needed to assess the impacts of hatchery fish

are lacking, including reproductive output of wild spawning

hatchery fish, it is impossible to obtain answers to many questions

regarding the impact of hatchery fish on wild fish.

PATH - Flush Preliminary results, based on limited data, suggest that artificial

propagation of spring/summer chinook has not significantly

contributed to declines in wild populations of Snake and upper

Columbia spring/summer chinook.

PATH - CRiSP PATH did not address hatcheries in a scientific manner.

PATH-EM Effects of hatcheries on wild populations, e.g., effects of reducing

Snake hatchery releases, have been modeled by PATH-EM. Some

preliminary results are in the PATH-EM Report, but were not given

during the presentation.

CRITFC COHORT CRITFC has not reached final conclusions on the questions posed.

Considering only demographic effects, hatcheries are an effective

tool for maintaining, reintroducing, and rebuilding existing

populations.
ISAB 2001-1: Model Synthesis Report

18

Conclusions with respect to Changes in Harvest.

ICBEMP-BBN Allocation of harvest levels was not addressed.

EDT Harvest reductions had little impact on spring chinook stocks but

were important to the abundance of fall chinook. EDT has not

attempted to assess impacts of harvest rates on ESA status.

CRI Reducing harvest would allow three ESUs (Fall Chinook in the

Snake River, Upper Willamette Chinook, Lower Columbia

Chinook) to grow. Harvest reductions by themselves could not

esure growing populations in the remaining ESUs.

PATH - Flush Declines in survival of Snake River Basin populations occurred at

the same time as substantial declines in harvest rates, contrary to

the assumption that harvests are significant contributors to declines

in survival of Snake River Basin populations after 1974.

PATH - CRiSP The mixture of hypotheses were optimistic in their estimates of

future productivity of stocks, so harvest reductions predict

optimistic recovery probabilities.

PATH-EM Effects of different harvest strategies have not been modeled by

PATH-EM. Future harvest strategies are required as input for

running the model.

CRITFC COHORT CRITFC has not reached final conclusions on the questions posed.

Limited analysis indicates that total elimination of all harvest, by

itself, is not sufficient to maintain or rebuild spring chinook

populations in the Snake River or above Priest Rapids dam.
ISAB 2001-1: Model Synthesis Report

19

Conclusions with respect to Tributary and Mainstem Habitat

Improvement.

ICBEMP-BBN All alternatives were projected to improve habitat on FS/BLM

lands and to maintain current strong populations with some

rebuilding of depressed populations. The preferred alternative

showed the largest relative increases.

EDT Breaching dams was effective in restoring mainstem habitats to the

benefit of chinook performance. Land-use changes and practices in

tributaries were, over the long term, effective in restoring

abundance and distribution of tributary populations.

CRI In regression modeling, salmon productivity was found to be more

related to: 1) good quality riparian zones in non-forested areas, 2)

low water temperatures, and 3) low gradients, as contrasted to the

number of dams passed.

PATH - Flush The significant decline in Snake River Basin populations after 1974

does not coincide with habitat degradation. Degradation in habitat

occurred mostly prior to 1974 and spawning and rearing habitat for

some of the Snake River Basin populations has remained in good

or pristine condition.

PATH - CRiSP PATH had no essential contribution to the effect of restoring

diversity of habitat in tributaries or the mainstem. Assessments for

restoring or creating mainstem fall chinook habitat were

approached in a cursory manner.

PATH-EM Effects of specific tributary and mainstem habitat improvements

can be modeled by PATH-EM. For example, effects of

improvement in rearing habitat (increasing egg-smolt or parr-smolt

survival via carcass or nutrient enrichment) have been studied.

Some preliminary results are in the PATH-EM Report, but were

not given during the presentation.

CRITFC COHORT CRITFC has not reached final conclusions on the questions posed.
ISAB 2001-1: Model Synthesis Report

20

How is uncertainty addressed?

ICBEMP-BBN Key relationships in the networks are represented by conditional

probabilities. The model outputs were expressed as the probability

of future habitat capacity and salmonid population status.

EDT The implications of uncertainty are addressed through sensitivity

analysis (e.g. by presenting "best case-worst case" outcomes of

actions). EDT is a steady-state model that does not incorporate

stochastic elements.

CRI Compute confidence intervals for estimates of annual population

growth. In case of little data, CRI uses an upper and lower bound

for parameters or perform numerical calculations over a wide range

of potential values.

PATH - Flush The Bayesian life cycle simulation model (BSM) was used to carry

uncertainties in climate and stock-production relationships through

the analysis. The decision analysis approach considered all possible

combinations of all relevant uncertainties.

PATH - CRiSP Uncertainty in PATH models was expressed through a weighting of

the hypotheses by outside experts. The degree of certainty in

conclusions may have been overstated.

PATH-EM Summarizing the sampling distribution via confidence intervals

assesses uncertainty in parameter estimates. The population

projection model respects parameter uncertainty by drawing values

from the joint sampling distribution of the estimates.

CRITFC COHORT The user can conduct a sensitivity analysis on variables that are of

concern.
ISAB 2001-1: Model Synthesis Report

21

Accuracy of predictions?

ICBEMP-BBN The models are accurate in the sense that they faithfully reflect the

beliefs of experts as to the influence of land management on

aquatic habitats and salmonids.

EDT EDT's depiction, like that of any model, is imperfect. EDT

compares relative effects of different habitat changes on salmon

performance.

CRI Some predictions are accurate, e.g., if downstream survival was

increased to 100%, then some populations are still at severe risk.

Some predictions are not. CRI depends on relative rank of

predictions, not accuracy.

PATH - Flush Emphasize that relative ranking of actions may be more important

than absolute standards. Rank order of actions was insensitive to

choice of life-cycle model configurations and passage models.

PATH - CRiSP CRiSP model predictions for spring chinook hydrosystem survival

have been proven accurate by comparisons with PIT tag data.

PATH-EM The fit of the model has been tested by addition of new index

stocks and by excluding/including recent years of spawner/recruit

data. Of course, the future is unknown and for example, the

model’s assumption about low-survival under recent ocean

conditions may not hold.

CRITFC COHORT Population growth rates from the model for spring chinook are of

similar magnitude to observed rates estimated independently by

others.
ISAB 2001-1: Model Synthesis Report

22

Relation to other models?

ICBEMP-BBN Each modeling effort had very different objectives. ICBEMP-BBN

provided a broad scale but more intensive analysis of current and

proposed management of FS/BLM lands than the other models.

ICBEMP-BBN analyses concerning migration survival did not

change relative differences of projected effects among the three

alternatives.

EDT The relationship between CRI, PATH, and EDT is complementary.

EDT can incorporate the analytical results of the statistics-based

PATH and CRI. Both EDT and ICBEMP-BBN rely on expert

opinion, but EDT gives finer scale analyses. EDT might

complement the CRITC model by expanding its analytical

capabilities.

CRI CRI is data based, focusing on annual rate of population change

and the risk of extinction for eleven different ESUs. PATH studied

two ESUs and focuses on hydropower. EDT is an expert system

summarizing effects of habitat descriptors, but does not incorporate

much fish population dynamics. ICBEMP-BBN seems to be

primarily a habitat-focused analysis.

PATH - Flush Concentration on Snake River stocks allowed PATH to perform a

comprehensive analysis of two data rich stocks (spring/summer

chinook and fall chinook). CRI, with its emphasis on rate of

growth, is charged with analyzing all ESA listed anadromous

stocks, regardless of the amount or quality of data available. EDT

models both anadromous and resident fishes at basin-wide or

province-wide scales.

PATH - CRiSP PATH synthesized much of the past data. The other modeling

projects have used the PATH data analyses as a foundation for their

work.

PATH-EM Compared to CRI, PATH-EM allows a more systematic

exploration of trade-offs among population risk measures,

experimental design, and knowledge gained. For example, PATHEM allows the growth rate of populations to decrease with

increased population size.

CRITFC COHORT We have attempted to represent the general behavior of PATH,

PSC, and TAC analyses without much of the complexity and detail.
ISAB 2001-1: Model Synthesis Report

23

How should the model be used to support decisions?

ICBEMP-BBN The ICBEMP-BBN model was specifically created to help

decision-makers select a management plan for FS/BLM lands. It

should be used in concert with other models and information to

weigh the merits of possible decisions affecting management of

other public and private habitat, hydropower, hatchery operations,

and harvest.

EDT Models highlight risks and tradeoffs of alternative actions. No

model does it all for most decisions. Use EDT in conjunction with

other models such as the CRI, PATH, and CRITFC tools. Take

advantage of the strengths of each model system and appreciate the

different perspectives afforded by each. Do not use models as

analytical bludgeons in political warfare.

CRI Decision-makers might be well-served by drawing on all of the

available analytical tools. EDT and CRI are not ‘dueling’ models;

in fact, they attempt to predict totally different response variables.

PATH and CRI could be profitably combined, or at a minimum,

learn from one another. ICBEMP-BBN seems to be primarily a

habitat-focused analysis.

PATH - Flush The decision analysis of PATH showed which actions were most

likely to benefit the stocks under the widest range of assumptions.

Analyses using the same data and the Leslie matrix model

advocated by CRI give strikingly similar results about the relative

effectiveness of the proposed alternatives in preventing extinction

and achieving recovery of listed Snake River stocks.

PATH - CRiSP Neither CRiSP, nor any other passage or life cycle model now

available, is adequate to address the issue of delayed mortality or

survival of fish above the hydrosystem. The models need to be

further developed and new experiments need to be planned to

characterize the impacts of passage experience on fish survival.

PATH-EM Take the individual population dynamics predictions with several

grains of salt: changes in climate and other uncontrollable factors

make predictions of extinction times, probabilities, etc. very

uncertain. However, have confidence in the overall conclusions

from the EM modeling.

CRITFC COHORT This model can be an effective tool for teaching local watershed

councils about the impact of conditions outside individual

subbasins upon local restoration efforts. The model can be used for

broad scale trade-off analyses across the 4-H’s.
ISAB 2001-1: Model Synthesis Report

24

Synthesis of the Answers to the ISAB’s Questions

What is the purpose of your modeling effort? What questions or problems were your

models designed to address?

Purposes of the decision support systems (models) are different. Two (PATHFLUSH and PATH-CRiSP) are primarily designed only to evaluate Snake River salmon

recovery actions including breaching the lower four Snake River dams, increased

transportation, and reduced harvest. Two (ICBEMP-BBN and EDT) are primarily

designed to evaluate large-scale habitat management alternatives across the landscape of

the Columbia River Basin and three (CRITFC, CRI and PATH-EM) are primarily for

prediction of trends in populations of salmonids based on counts of fish and rates of

change in population parameters.

What kinds of information or data are needed to run your model?

Data needs are different, because the purposes of the models are different. PATHFLUSH and PATH-CRiSP models concentrate on estimating survival based on flow and

other data as fish enter the lower Snake River. ICBEMP-BBN and EDT concentrate on

predicted effects of habitat changes over large areas based on spatial habitat data.

CRITFC, CRI, and PATH-EM concentrate on predicted trends in sizes of populations

based on fish counts and rates of change of population parameters (e.g., survival rates) in

the presence of various management actions. One characteristic common to all of the

models is that information required to run the models includes not only primary data

measured by documented procedures, but also derived values each depending on yet

other (perhaps undocumented) assumptions and ‘models.’

What are the assumptions of your model?

Not only are the purposes and data needs different among the models; the models

depend on different assumptions made by the respective developers concerning the state

of nature. Some of the key assumptions made are highlighted in Table 1, and the

presenters described others (Appendix III). Two of the models, ICBEMP-BBN and EDT,

are “expert systems” depending heavily on opinions (i.e., assumptions) expressed by

panels of experts. All of the models implicitly assume that future climate and ocean

conditions will be like those in the recent past.

What are the strengths and weaknesses of your model?

Given that purposes, data needs, and assumptions are different, it is not surprising

that the models have different strengths and weaknesses (Table 1). Some involve

complex interactions of physical and biological factors (ICBEMP-BBN, EDT, PATH-
ISAB 2001-1: Model Synthesis Report

25

FLUSH, and PATH-CRiSP), others (CRI and CRITFC) depend on relatively simple

mathematical procedures that can be easily programmed and duplicated, while PATHEM fits somewhere in-between. Simplicity and complexity are each simultaneously a

strength and a weakness. The simpler models allow relatively more direct point estimates

of life history parameters under a given set of assumptions, but lack the ability of the

complex models to help understand mechanisms of cause and effect relationships. The

more complex models deal better with the comparison of management alternatives, but

usually can only rank alternatives on their expected benefits. It would be nice of course

to have the advantages of all the decision support systems in one “model”, but this seems

to be impossible.

How does your model address uncertainty?

Only one model (CRI) reported an attempt to deal with variation in the data due to

sampling (observational error) so that confidence intervals can be placed on predicted

values. However, CRI was not totally successful in this attempt and along with EDT,

PATH-EM, and CRITFC, performed sensitivity analysis (e.g. presented "best case-worst

case" outcomes of actions). The other models, ICBEMP-BBN, PATH-FLUSH, and

PATH-CRiSP, utilized probabilistic arguments either by standard procedures or Bayesian

methods to assign probabilities to expected outcomes. None of the models address the

issue of the feasibility of implementing management actions in the face of social and

economic constraints.

All models make predictions. Why do you think your model’s predictions are accurate?

In comparison of major management alternatives, all of the models depend

primarily on the relative rank of predictions, not accuracy of the actual predicted

numerical values. The rank orders of effects of alternative actions were judged to be

accurate by the model presenters.

How does your modeling effort relate to or contrast with the other modeling efforts?

The modeling efforts have very different objectives. Both EDT and ICBEMPBBN rely on expert opinion to rank effects of habitat changes over large areas, but EDT

gives finer scale analyses. CRI, PATH-EM, and CRITFC attempt to predict the trends of

stocks of fish, but with different approaches and information. PATH-FLUSH and PATHCRiSP concentrate on analysis of only two Snake River ESUs. Nevertheless, the general

conclusions when faced with the same objectives and when based on more-or-less the

same information are often in close agreement with respect to the predicted rank order of

effects of management alternatives. As PATH-FLUSH concluded “The decision analysis

of PATH showed which actions were most likely to benefit the stocks under the widest

range of assumptions. Analyses using the same data and the Leslie matrix model

advocated by CRI give strikingly similar results about the relative effectiveness of the

proposed alternatives in preventing extinction and achieving recovery of listed Snake

River stocks.”
ISAB 2001-1: Model Synthesis Report

26

What is the efficacy of dam breaching or draw down to natural river levels for delisting

ESA species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

There is general agreement among the models that breaching dams is effective in

restoring mainstem habitats to the benefit of listed fall chinook and breaching would

provide significant benefits to both Snake River fall and spring listed stocks. All models

also agree with the obvious and important conclusion that breaching dams on the Lower

Snake River will do little or nothing to help the other ten listed ESA species. With respect

to the Snake River spring chinook population, there is general agreement among the

PATH models that dam breaching or draw down are better than transportation (with the

dams in place) as a recovery measure. CRI and CRITFC conclude that survival gains

from other factors such as tributary and estuarine habitat improvement are needed to

allow the Snake River spring chinook population to rebuild, even with dam breaching.

ICBEMP-BBN is not useful in judging the efficacy of dam breaching.

What is the efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

No satisfactory conclusions were given concerning the efficacy of hatcheries for

delisting ESA species. Only EDT and PATH-EM seems to have taken an initial look at

modeling the effect of reduced hatchery production (for wild steelhead in the Snake

River).

What is the allocation of harvest and harvest levels needed to delist ESA species and

restore diverse and productive populations of native fishes throughout the Columbia

River Basin?

The effect of changing harvest received varied and limited attention by the

modeling efforts. CRI concluded that reducing harvest would allow only three ESUs (Fall

Chinook in the Snake River, Upper Willamette Chinook, Lower Columbia Chinook) to

grow, a conclusion not much different from the PATH determination that harvests were

not significant contributors to declines in survival of Snake River Basin populations after

1974, or the CRITFC note that elimination of all harvest, by itself, is not sufficient to

maintain or rebuild spring chinook populations in the Snake River or above Priest Rapids

dam. ICBEMP-BBN, EDT, and PATH-EM have not attempted to assess impacts of

harvest alternatives on ESA stocks.

What is the efficacy of restoration of tributary and mainstem habitat for delisting ESA

species and restoring diverse and productive populations of native fishes throughout

the Columbia River Basin?

The two broad scale habitat models, ICBEMP-BBN and EDT, both agree that

over the long term improving tributary habitat is effective in maintaining or restoring

abundance and distribution of tributary populations. The rather obvious conclusion is
ISAB 2001-1: Model Synthesis Report

27

supported that breaching dams was effective in restoring mainstem habitats to the benefit

of listed Snake River fall chinook. CRI related salmon productivity to characteristics of

tributary habitat. PATH, in contrast, concluded that tributary habitat degradation was not

responsible for the decline in Snake River stocks following the construction of the lower

Snake River dams.

What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River Basin?

The decision support systems have different strengths and weaknesses and

policy-makers would be well served by drawing on all of the available analytical tools.

Six of the seven presenters implied that their model should be used in conjunction with

other models, because in the words of one “No model does it all for most decisions. Use

EDT in conjunction with other models such as the CRI, PATH, and CRITFC tools. Take

advantage of the strengths of each model system and appreciate the different perspectives

afforded by each.” The ISAB wholeheartedly endorses this recommendation. Results

from a model, given knowledge of its assumptions and quality of data used, cannot

decrease the information available to a decision-maker.

Evaluation of the Modeling Efforts

None of the models presently in use in the Columbia River Basin is complete

enough to serve as the sole decision support tool for the region. The models available to

support decisions in the Columbia Basin serve different functions and all have strengths

and weaknesses. The models differ in the problems they were attempting to address, the

analytical approaches to the problems, the assumptions underlying each of the

approaches, the quantity and quality of the available data, and the rigor with which they

deal with the complex life cycles and habitats of the species.

PATH scores high for its attempt to operate in a decision context that was explicit

about uncertainties, and for attempting to formally quantify uncertainty in a reasonably

sophisticated way. PATH’s weakness was in the very narrow range of factors that it took

into consideration, and in the tortuous model structure that posed an obstacle to diagnosis

of model behavior. CRI, by contrast, scores very high for clarity and tractability of model

structure, but it lacks statistical treatment to quantify uncertainty. CRI encompasses a

broader range of factors than the original PATH process, but it still appears deficient in

necessary detail for habitat variables and spatial and temporal structure. Both CRI and

PATH rely on derived quantities in place of actual data, in ways that confound a proper

uncertainty analysis. CRI clearly is investing heavily in attempts to remedy these

deficiencies.

EDT scores very high marks for comprehensiveness of factors taken into account,

and for spatial and temporal detail in the data it uses. It is not yet clear whether EDT is

sufficiently rigorous about maintaining the separation between actual measurements and

derived quantities, or in quantifying the uncertainty of the derived quantities. The

predictive modeling component of EDT is essentially an expert system -- this is a
ISAB 2001-1: Model Synthesis Report

28

systematic amalgamation of the judgment of a panel of experts that then gets coded into a

set of quantitative relationships that can operate on data input to generate predictions. In

principle, the construction of an operating rule set from the expert judgment can be done

in a way that is transparent and open to diagnosis. We do not yet know whether EDT has

achieved this. The process of eliciting expert judgment may include obtaining opinion

about certainty, but that too is expert judgment, rather than a statistical comparison to

actual measurements. So it looks as if EDT does not yet include a satisfactory treatment

of uncertainty.

ICBEMP-BBN like EDT gets high marks for comprehensiveness and for spatial

and temporal detail. ICBEMP-BBN went one important step beyond EDT by formally

eliciting expert judgment about uncertainty and propagating this uncertainty through the

model predictions. ICBEMP-BBN was constructed as a Bayesian Belief Network, which

confers the advantage of explicit representation of uncertainty as probabilities. This in

turn offers the potential of a smooth transition from expert system based analysis to

statistically based analysis, as more data become available to develop more empirical

models of the key relationships between management, habitat, and salmon population

responses. Essentially, the Bayesian Belief Network can serve as a prior distribution for

classical Bayesian statistical inference, merging the expert opinion with empirical

measurements. The Bayesian statistical framework lends itself to value-of-information

calculations that can help guide decisions about future monitoring, ground truthing, or

experimental interventions to help resolve critical uncertainties. ICBEMP-BBN did not

avail itself of this potential, but it is an attractive option that should be considered as EDT

evolves toward calibration and validation.

Overall, the region needs modeling efforts that collectively combine the best

features of all the models we have reviewed. Features of ICBEMP-BBN and EDT could

be used to formulate working hypotheses concerning, for example, causes of salmonid

declines and potential effectiveness of management interventions. Features of ICBEMPBBN could be used to communicate the uncertainty of the working hypotheses. Features

of CRI could be used to test hypotheses wherever data were available. Features of PATH

could be used to quantify the uncertainty of the tests of the hypotheses and to place

recommendations for management in a risk assessment context. Furthermore, CRI might

serve as an inspiration for clarity, rigor, and openness.

The models the ISAB reviewed are best at ranking the expected effects of

management alternatives. The general conclusions of the models are often in close

agreement with respect to the predicted rank of management alternatives when

addressing similar problems and using the same data sets. They are not good, however, at

giving absolute numerical predictions and they do a poor job of accurately estimating

what the policy-makers may need most, namely, a credible scientific analysis of the

probability (feasibility) that some measurable degree of salmon recovery will be achieved

with any particular management action.

There are two important ways in which scientists can help environmental decision

processes to cope better with uncertainty. The first is the purely technical contribution of
ISAB 2001-1: Model Synthesis Report

29

explicitly quantifying the relevant uncertainty. Statements of the respective probabilities

of alternative scenarios are a natural way to communicate uncertainty when the decision

is essentially placing a bet about which scenario actually will materialize. Recent

progress in practical methods for quantifying uncertainty is an exciting development that

deserves to be used in Columbia Basin salmon modeling.

The second important area where scientists can contribute is in helping the

decision-makers craft decision rules that get formalized before the analysis is undertaken.

Decision rules define the measurements that will be made, the statistical operations that

will be performed on the data, and the threshold magnitudes of estimated quantities at

specified levels of certainty that will serve as criteria for the decision. Such specifications

help remove ambiguity from the way science is used in a decision -- even when there is

uncertainty in the data or models. Committing to these specifications in advance helps

dispel suspicions that the analysis may be manipulated to achieve a particular outcome.

All the models are severely constrained by lack of data. Modeling controversies in

the salmon arena have largely been an unproductive distraction from the real scientific

problem of inadequacy of the available data for addressing many of the important

management questions. Some of the debate that now centers on competing models could

be resolved with the right data. The present paucity of data creates more scope for

alternative assumptions in the models. Sophisticated, responsible modeling takes all the

plausible alternative assumptions into account with weighting according to their

respective concordance with the data that are available. This need not lead to "modeling

wars."

The data problem, unfortunately, will not be solved as easily as the modeling

problem or the data base management problem. Where scientific leadership and

institutional innovation needs to be exercised is in the prioritization, design, and

implementation of large-scale monitoring linked to management experiments. There is at

present no institutional center of authority for addressing the prioritization, design and

coordination issues for large-scale monitoring. Prioritization and design issues inherently

presume centralization, so the entrepreneurial "distributed" paradigm that the ISRP has

recommended for data management, and that the ISAB is here recommending for

modeling, will not solve the data problem.

Decision-makers would be well served by drawing on all the available analytical

tools. Decision-makers would benefit by focusing on areas of consensus among the

models or the weight of evidence provided collectively by the models. Areas of

disagreement among the models may pinpoint uncertainties that require further

investigation. In considering how results of models make their way into the decisionmaking process, it is helpful to recall the roles of models. They provide ways of

organizing and communicating information, generating hypotheses, and pin pointing the

crucial gaps in information. The modeling efforts are not ends in themselves; they are not

final, definitive answers, but rather they are ongoing processes for continuously

increasing knowledge.
ISAB 2001-1: Model Synthesis Report

30

Moving Forward with Support for Multiple Modeling Efforts

At this point in the evolution of northwest salmon science, there is no compelling

reason to fund large collaborative modeling efforts. For the amount of real data that are

available, the kinds of models that are actually justifiable really are not that complicated.

Small groups of researchers should be adequate to pursue development of the models.

And the coherence of vision of a small group may encourage clarity in the resulting

model. That coherence is very difficult to achieve as the group gets large.

The past hope in encouraging the development of a region-wide collaborative

modeling effort was to achieve "scientific consensus." In fact, the lesson of reflecting on

what we have learned in the last period of time is that scientific consensus does not really

emerge from regional modeling committees. Scientific consensus is a product of a much

larger scientific audience, whose primary route of exposure to the information in question

through reading articles, mostly in peer-reviewed books and journals; and where major

differences are settled by the results of attempts to duplicate analyses of the same data or

attempts to duplicate experiments in the same system.

So, the correct way to proceed with modeling needs is to fund specific groups to

undertake specific modeling projects. For any modeling topic that is expected to be in

any way controversial, one of the conditions of the contract should be delivery of a

version of the report in a form suitable for peer-reviewed publication. The length

limitations of this publication format may preclude some important detail, so it is to be

expected that there will be "long-form" reports as well (which may include data archives)

that should be in electronic form to be made accessible on the Web.

The expectation will be that the reports delivered in a form suitable for peerreviewed publication actually will be submitted for publication, but it is recognized that

the review (and possible revision) and publication process usually involves a lag time of

six months to two years. The long lag time should not be used as an excuse not to pursue

publication. It is true that managers and administrators often believe that they need the

results "yesterday," but it is also true that the same questions come up again and again,

year after year, often without satisfactory resolution.

There would be merit to funding several groups to pursue modeling questions.

There would even be merit to a degree of overlap, so that the same data sets might be

analyzed from somewhat different technical perspectives, which would encourage the

evolution of scientific insights. The actual dollar cost of modeling, as a fraction of the

overall salmon recovery budget, has been very small, so this is not the place to look for

cost cutting.
ISAB 2001-1: Model Synthesis Report

31

Use of Models in the Decision Process in the Columbia River Basin

The Science-Policy Interface

For scientific results to enable management interventions that are effective in

accomplishing societal goals, there must be not only a belief by decision-makers in the

credibility of the science and its relevance to the problems being addressed, but also

effective communication between scientists and decision-makers, so that decision-makers

are well informed of the consequences of management alternatives. Each of the present

collection of prominent salmon-relevant modeling approaches that we reviewed has

something useful to offer policy-makers provided the policy-makers understand the logic,

supporting evidence, and limitations and assumptions associated with each model.

Without that understanding, of course, there is a temptation to use the modeling results in

superficial and spuriously definitive ways that can add confusion to the deliberations. In

addition, misuse and misunderstanding have the potential to create an atmosphere that

undervalues science.

Unfortunately, there is no standing formal institutional mechanism for

synthesizing the results or for clarifying the interpretation of the various salmon models

for the various policy makers on an ongoing basis. In this connection, that decisionmaking authority for various aspects of salmon management is as fragmented as the

modeling efforts.

The following sketch and accompanying text is our attempt to provide an analysis

of potential impediments at the policy-science interface that were revealed in our

discussions with decision-makers and scientists. The sketch portrays a system of selfregulating behaviors exhibited by both policy-makers and scientists (Bella 1995, 1996a,b,

1997). The idea is that policy-makers and scientists become “locked up,” trapped as it

were, in a system of behaviors that form self-reinforcing loops, thus tending to perpetuate

the status quo. We do not intend to imply that the patterns of behavior depicted in the

sketch are operative under all circumstances.

The behaviors are represented in boxes that are connected by arrows. The

incoming arrows describe reasons for the behavior and the outgoing arrows describe

consequences of the behavior. Proceed forward or backward along any arrow. If you

move forward, say “therefore” and if you move backward say “because.” Start at the top

left of the sketch.

Decision-makers and scientists often not only “speak different languages,” but

also operate with different interests, goals, and belief systems. Seldom are decisionmakers educated as scientists and thus, they sometimes ask scientists to answer questions

that are not amenable to a scientific approach or phrase questions in a way that is unclear

to scientists. Scientists, in turn, are often not trained and have little experience in the

policy-making process in which the legal and societal dimensions of any proposed

solution must be considered. Some scientists are simply not interested in the policy

implications of their work, often viewing themselves as independent of the policy
ISAB 2001-1: Model Synthesis Report

32

process, and focusing mainly on conducting scientific research within their area of

expertise with the belief that their research results will somehow feed into the decisionmaking process. Scientists sometimes tend to view the politics of the policy-making

process as value-laden, subjective, and sometimes corrupt.

As a result of differences in their background experience, education, and interests,

the needs of decision-makers often are not clear to scientists, and decision-makers do not

comprehend what science is able to accomplish relative to a particular problem. Thus, the

expectations of scientists and decision-makers differ, with decision-makers often

expecting science to accomplish, over a relatively short time period, more than it is

capable of accomplishing given limitations of data quality and quantity, environmental

variability and uncertainty, limited background and experience of scientists, and so forth.
ISAB 2001-1: Model Synthesis Report

33

Decision-makers don’t

frame scientific questions

Needs of the decision-makers are

not clear to scientists

Scientific results are not in a form

that is meaningful to decision-makers

Decision-makers are not

adequately informed

Recovery alternatives

are highly controversial

Scientists adhere to rigorous,

precise interpretations of analyses

Decisions are made

that are socially and

politically expedient

Scientists decide that their

preferred alternatives are

not politically acceptable

Scientists propose alternatives they

believe are politically acceptable

Scientists don’t believe

their research impacts

decisions

Scientists disagree on

appropriate recovery strategies

More research is

needed to resolve

uncertainty

Important decisions

are deferred

Decision-makers

are seldom trained

as scientists

Scientists are seldom

trained as decision-makers

Decision-makers and

scientists differ in their

expectations of science
ISAB 2001-1: Model Synthesis Report

34

As a consequence of this shortage of meaningful communication, decision-makers

are often not as well informed as they could be of the full range of management

alternatives and uncertainties. When decision-makers do not frame appropriate scientific

questions or fail to make their needs clear to scientists, two things can occur. First,

research is performed that does not address critical management questions. Second,

technical results are communicated to decision-makers in a form that is not meaningful to

them. As an example, technical measures such as the risk of extinction, expressed as the

probability of extinction over 24, 48 or 100 years, are perfectly acceptable scientific

measures of the chances of extinction, but these concepts and the time frames to which

they apply may be difficult for decision-makers to interpret and implement into a

defensible decision.

Decision-makers most often rely on trusted experts, who are usually members of

their own staffs or independent scientific groups, to assess and interpret, “translate” as it

were, results of technical analyses into a form that can feed into the decision process

more readily, without seizing the decision from the decision-maker.

Another pathway that leads to poorly informed decision-making could originate

when management interventions are highly controversial and have major political, social,

and economic ramifications. Scientists sometimes respond to this situation by retreating

“into the science”, becoming overly precise and rigorous in communicating the results of

their analyses to decision-makers. Scientists act this way because they believe that in

doing so the decision-maker will have accurate facts and the scientists themselves will

not be held responsible for any errors in interpretation. In other cases, scientists may

have so much time and energy invested in a particular modeling approach that they

become advocates for the results of the model, losing sight of the uncertainties. Less than

well-informed decision-makers should be concerned about their ability to communicate

clearly and justify their decisions to political leaders, the stakeholders, their constituents,

and the general public. To bridge communication gaps, many decision-makers without

formal scientific training have become very knowledgeable about science and its

limitations. Conversely, many scientists have become more conversant in the decisionmaking process

Technical analyses

and results

Trusted

experts

Decisional significance;

Credibility of the analysis

Decisionmaker

Translation and interpretation

process to bring technical

information into the realm of

human experience
ISAB 2001-1: Model Synthesis Report

35

Problems arise when decision-makers are not adequately informed, for whatever

reason, or their expectations are not met. Under these circumstances they may relegate

scientific findings to a less integral position in the decision-making process and choose

an action or alternative that is the most politically expedient. Upon viewing decisions of

this kind, scientists often believe that their research has had little effect on the decision.

They sometimes believe that the only research that gets used in decision-making is

research that supports a decision that has already been made, implicitly or explicitly.

To complicate things further, scientists often believe that scientific findings

should be preeminent in the decision-making process and so do not fully appreciate the

decision-maker’s need to consider the societal implications of the decision at hand.

Sometimes in an effort to make their work more relevant to the decision-making process,

or through pressure from the decision-maker or interest groups, scientists propose

alternatives that are not their preferred actions, but ones they believe would be politically

acceptable, thereby perpetuating the system of behaviors that leads to politically

expedient decisions.

Scientific disagreement and conflict over appropriate recovery strategies also can

lead to politically expedient decisions. Disagreement can stem from differences in the

problems being addressed by different analyses, differences in analytical approaches to

the problems, differences in the assumptions underlying each of the approaches, the

quantity and quality of the available data, and the rigor with which each approach deals

with the complex life cycles and habitats of the species. Scientific debates are to be

expected; they are a part of the scientific process and are necessary for winnowing

through the various scientific approaches to expose their weaknesses and strengths. These

debates can become preeminent for the involved scientists, but some of the decisionmakers we spoke with found them confusing and are more interested in identifying

broad-scale patterns or areas where the various approaches agree.

Unfortunately, decision-makers can use scientific uncertainty to justify politically

expedient decisions or to defer decisions pending further research, a strategy that is often

believed by both scientists and decision-makers to reduce uncertainty and narrow the

scope of the decision. An example of a deferred decision was NMFS’ decision to delay

breaching the four lower Snake River dams. When decisions are deferred for reasons not

obvious or not defensible to scientists, who believe they have sufficient scientific support

to justify a decision, they again feel their research is being ignored in favor of politics.

This is the way many scientists feel about the decision to delay breaching.
ISAB 2001-1: Model Synthesis Report

36

Individual Models and the Decision Process

Most of the analytical frameworks that the ISAB reviewed were developed to

address the specific needs of an agency. In these cases the model probably will be of

greatest utility to the sponsoring agency. Two of the modeling efforts we reviewed

seemed to enjoy very successful communication with their respective decision-makers.

These were ICBEMP-BBN and CRI. In both cases, the decision-making authority was

part of the same organization that had ownership in the model and scientists developing

the models belong to the organization. In both cases, the decision questions that the

modeling effort was intended to inform were well-defined in advance of the modeling

project, the models were directed at those questions, and the decision-makers seemed not

to experience difficulty in understanding and using the model results. It is noteworthy

that in those two cases, the treatment of uncertainty in the models did not seem to pose a

special obstacle to the decision-maker who was going to use the results.

The EDT model was developed to serve two rather distinct purposes: (1) to

articulate a particular desired vision for the future of the basin with a prescription for

associated management strategies, and (2) to appraise the expected effectiveness of

specific proposed projects in the context of the watersheds where they would be located.

The strategic vision application is not tightly linked to an identified decision process. The

decision process of project funding by the Fish and Wildlife Program intends to

incorporate the project proposal application, but that role has not yet been formalized or

implemented.

The CRITFC model has yet to be used for decision-making, and its results have

not yet been promoted as arguments for influencing a particular decision. The history of

PATH in relation to decision making processes, however, is complicated enough to

warrant the following separate section.

The Institutional History of PATH

It is difficult to characterize PATH accurately and fairly, because it was never just

one thing; many things happened under the PATH umbrella. It is important, however, to

understand PATH accurately and fairly for two reasons: 1) because PATH did attempt

some worthwhile things that have not yet been duplicated by the other modeling efforts,

and 2) because feelings have run strong on several aspects of the modeling debate, and

some of the PATH participants still function as a distinct constituency (e.g., PATH-EM,

this report).

PATH was a "process" that was initiated as part of the settlement of the litigation

over the 1995 Hydro BiOp. At that time (1995) there were two dueling models, FLUSH

and CRiSP. FLUSH was the work of a coalition of states and tribes and put a heavy

emphasis on the effects of flow on smolt survival during downstream passage. CRiSP

was the work of Dr. Jim Anderson's group at the University of Washington, funded by

BPA, and it emphasized other factors affecting smolt survival during downstream
ISAB 2001-1: Model Synthesis Report

37

passage. NMFS, the agency responsible for the BiOp under ESA, did not have ownership

of either model, and at the time of the 1995 Biological Opinion, NMFS basically declined

to decide between them. The legal resolution, at that time, was to set up PATH, with

representation of all the main parties to the debate, as a scientific forum that would be

given the assignment to achieve scientific consensus in time for the next BiOp on hydro

operations, which was then scheduled for 1999.

Though NMFS had representation in PATH, NMFS did not have administrative

responsibility or control over PATH. The bulk of PATH funding came from BPA through

the Council’s Fish and Wildlife Program, and some of the PATH funding came via other

BPA mechanisms. PATH developed its own organizational structure with the

engagement of a consulting company (ESSA) that provided a guiding facilitator and with

the establishment of a Scientific Review Panel of distinguished fisheries scientists. Later,

as the technical strategy for the PATH analysis took shape, PATH also engaged some

additional fisheries modeling expertise, some of it from outside the Columbia Basin, to

actually carry out the central mathematical analysis.

PATH then set out on four separate, but related enterprises. These were:

1. continuing work on the FLUSH and CRiSP models, more or less independently of

one another and not calibrated to the same data sets,

2. continuing work to consolidate a common database of "run reconstructions" for a

list of "index stocks,"

3. construction of a new hypermodel that would incorporate both FLUSH and

CRiSP as modules for calculating downstream passage smolt mortality and that

would fit all its other parameters via Bayesian inference, calibrating to the "run

reconstructions" of "index stocks,"

4. construction of a decision theoretic framework for incorporating the expert

judgement of the scientific review panel into a set of recommendations that would

respond to the accumulated evidence (including the results from the hypermodel), the

uncertainty, and some appraisal of the risks.

These four pieces each made sense in their own way, but they did not fit together

satisfactorily. Superficially, one would think that the priority charge to PATH was to

diagnose and reconcile the differences between FLUSH and CRiSP, but, taken together,

components (1) and (3) of the PATH enterprise largely precluded that resolution. The

technical Bayesian analysis in component (3), while genuinely leading edge in its

sophistication, was very complicated and difficult to explain to potential customers for

PATH's results. Component (4) edged into the business of making recommendations

about hydro operations and the merits of dam breaching, which may have been perceived

in some quarters as going beyond the original PATH charter.
ISAB 2001-1: Model Synthesis Report

38

Notwithstanding a large amount of good work, PATH was vulnerable to the

criticism that it had not answered the question asked of it, and that it was trying to answer

policy-sensitive questions that had not been asked of it. It was not quite that simple, of

course, because it was not entirely clear who was in charge of PATH, or to whom PATH

was supposed to answer. Furthermore, because of its "consensus" structure, PATH was

notoriously slow in operation, and it generated unwieldy volumes of reports.

The PATH funding was piecemeal. Many of the subprojects of PATH were

presented annually to the Fish and Wildlife Program as separate proposals for funding. In

1999, the ISRP recommended that the PATH proposals not be funded. Subsequently, the

Northwest Power Planning Council, which makes the final decisions on program funding,

decided to terminate funding for PATH as an organization.

For the past year, NMFS has been doing its own modeling and data analysis to

support its own decision processes under its ESA responsibilities. The NMFS approach,

so far, has been simpler and more statistically consistently than was PATH.

CRiSP development continues in Dr. Jim Anderson's group. Elements of FLUSH

are maintained by some of the players in the coalition of states and tribes. This coalition

has formalized itself as an organization called STUFA, but with CRITFC spinning off yet

another modeling effort that resembles EDT and ICBEMP-BBN in its use of a database

of habitat characteristics and with the overlay of a conventional treatment of population

dynamics.

The review functions of PATH's scientific review panel, the facilitation function

of ESSA, the hypermodel effort, and the consensus function that PATH as a whole was

supposed to accomplish, no longer occur in that forum. Nevertheless, many of the

individuals who were involved in PATH continue to work on Columbia Basin salmon

modeling and data analysis issues--and some results of the PATH activities are

continuing to move into the peer reviewed scientific literature. The habitat-centered

modeling efforts of ICBEMP-BBN and EDT have taken up important aspects of salmon

modeling with which PATH, for the most part, was not concerned.

The final product from PATH as a functioning organization was the report on

Experimental Management, PATH-EM. This report constituted a continuation of PATH’s

commitment to evaluate the time necessary to detect effects of anticipated management

actions and to provide some preliminary mathematical estimates of the effects, though it

may have been a somewhat rushed and incomplete effort owing to the termination of

PATH’s funding.

The logic of decision analysis often leads to a conclusion that when relevant

uncertainties are large, it is imprudent to commit to long-term courses of action unless

these courses of action have a built-in flexibility to respond to new information. In fact,

the decision options to be weighed include the relative investment in continued data

collection versus the merits of experimental manipulations. The relative merits of the two

options should be judged in terms of the value of the information they provide, quite
ISAB 2001-1: Model Synthesis Report

39

aside from the direct outcomes any experimental manipulation expressed in terms of the

desired objective (such as more fish available for harvest, or an increased probability of

population persistence). The value of new information is quantified in terms of its

potential use to influence the selection of subsequent courses of action. The formalization

of such flexible decision rules together with the cost-benefit of continued data and

experimental manipulation are encapsulated in the technical theory of Adaptive

Management.

Among the Columbia River basin salmon modeling efforts, the PATH

experimental management report represents the most substantive attempt, to date, to

provide an analysis of the prospects for actual Adaptive Management. The analysis

described specific experiments and provided a preliminary assessment of the types of

information that the experiments would provide, and how much time they would require

for completion. Because there seems to be a temptation, in many quarters, to co-opt the

term “adaptive management” merely to put a positive spin on vague management plans, it

is important that this kind of technical analysis of adaptive management resume as an

ongoing enterprise connected to actual decision making in the Basin.

Communication and Sources of Credibility

The best resolution of scientific disagreement is through publication in the open

scientific literature, and provision of access to model code and to data files allowing

independent verification. Data access should include access to the original primary data

and metadata. If there is not access to the primary data, derived quantities, when treated

as if they were data, may carry error that escapes scrutiny. Because publication imposes

time delays that may be inconsistent with decision timetables, it is advantageous for a

modeling project to have good lines of communication with decision-makers. If a

modeling effort is motivated by a desire to contribute to a particular decision, it is

especially helpful to invest up front in enough communication to ensure that the model

really is addressing the right question.

The present culture of Columbia Basin salmon science has not put a great

premium on publication. This greatly magnifies the importance of the decision-makers' a

priori trust in any particular modeling effort that they choose to use. This is not entirely a

desirable situation. A greater role of publication in the open literature in establishing

credibility for the use of science in Columbia Basin decision-making would be good. A

greater reliance on the mechanisms of normal scientific discourse might also reduce some

of the contentiousness that has characterized the history of scientific debate over key

issues in Columbia Basin salmon management.

We might note that the recent experience of an attempt at a scientific consensus

"process" operating within the conventional Columbia Basin culture was not especially

encouraging. The process proved to be very slow and cumbersome--so much so that it

probably was no quicker than publication might have been. Furthermore, the consensus

did not hold.
ISAB 2001-1: Model Synthesis Report

40

Realistically, it is inevitable that institutional trust will give selected models and

modelers an inside track for access to decision-makers, but a culture of publication and

respect for published results could still influence the standards of quality control and the

habits of discourse for resolving scientific disagreement. We view as a very positive

development the recent example of CRI in espousing a culture of publication and a stance

of healthy skepticism about traditional data sources.
ISAB 2001-1: Model Synthesis Report

Appendix I - 41

Appendix I: Technicalities of Quantification of Uncertainty

Theory can usefully distinguish three main kinds of "error" as contributing to the

uncertainty of a model's predictions. These are measurement error, parameter error, and

process error. The label "error" in this theoretical language is meant to refer to a source of

variance, not to a "mistake." The role of variance in our characterization of this type of

error is crucial. We quantify the error in terms of a variance, because we acknowledge

that we cannot know the actual value of the error in a given instance, but only can know

its probability distribution. If we knew the actual value of the error, we would just

subtract that from our estimate, and then there would not be any error.

Measurement error refers to the random influences that cause an observation to

deviate from the true value that is being measured. For example, we may obtain

observations in the form of redd counts in an attempt to census the size of a spawning

population. However, because this is not an exhaustive count of the population, but

rather a count in an index area, during a particular time window, which then is expanded

to estimate the entire spawning population for that year, the estimate of population size

has an error component.

Parameter error refers to random influences that cause a parameter estimate to

deviate from the true value that we wish it to represent. Use of measurements that are

subject to measurement error can be one contributor to parameter error. Another

important source of parameter error that can arise even with no measurement error is

sampling variation. Consider, for example, that our parameter of interest is the average

number of recruits per spawner at a particular population size. We know that the actual

number of recruits per spawner in a given generation is subject to random environmental

influences. If our data set consists of only a few years’ observations of numbers of

recruits for brood years when the spawner population was in a particular size range, that

small sample may misrepresent the true average, even if the measurements themselves

had no measurement error. In reality, for a quantity like average number of recruits per

spawner at a particular population size, we would expect the parameter error to be

substantially influenced both by measurement error and sampling variation.

A further contributor to parameter error is simple mis-specification of a statistical

extrapolation procedure. Consider for example, that the parameter of interest is the alpha

of a Ricker production equation. That parameter represents the limiting value of recruits

per spawner in the limiting condition of a vanishingly sparse population. Since our data

come from populations that are not (yet) vanishingly sparse, we need somehow to

extrapolate from the estimates of recruits per spawner at observed densities to an estimate

of what the value should be at the limiting density where the parameter is thought to

express itself in pure form. In the case of the Ricker equation, we do this by linear

regression, since the equation represents the relation between spawner density and

recruits per spawner as a linear relation, in which the parameter alpha is the y-intercept.

This linear regression seems a clever work around for the absence of actual data at zero

density, but what if the true relationship between spawner density and average recruits

per spawner is not exactly linear? Then the linear regression will have introduced an error
ISAB 2001-1: Model Synthesis Report

Appendix I - 42

component over and above the sampling variation in the brood years for which we have

observations, and the measurement error in those observations. We see, therefore, that

correctly characterizing the parameter error is a rather complicated undertaking.

Finally, when we apply a model to make predictions of a process that itself is

subject to random variation, called process error, the process error will contribute to the

variance in the distribution of the discrepancy between the prediction and what actually

will occur. The influence of process error -- the inherent randomness in the process -- will

cause prediction error even if we knew the parameters exactly and our measurements

were perfect. Consider the case of predicting the number of recruits, from knowledge of

the number of spawners that give rise to that brood and from knowledge of the

parameters of the Ricker equation that applied to that stock. The Ricker equation gives

the "average" number of recruits per spawner that we should expect from a given brood.

Even if we knew the Ricker parameters exactly, the prediction is only for the "average."

On the other hand, the actual number that will return in the year being predicted will be

influenced by random environmental variation that in principle is not predictable, because

it is future noise in the process.

While we may not be able to fully predict the deviation owing to the random

process error of this sort for a given future set of returns, we should be able characterize

its distribution. That is, we should, in this example, be able to say that the number of

recruits will be a distribution, more or less centered over the predicted average from use

of the Ricker equation applied to our estimate of the number of spawners that gave rise to

this brood, and using the parameter estimates that we obtained from fitting the Ricker

equation to some historical data. This distribution of the predicted returns will also have a

spread, measured usually by its standard deviation, that reflects the uncertainty in the

prediction. This prediction uncertainty will have contributions from measurement error in

the estimate of the starting number of spawners, parameter error owing both to sampling

variation in the brood years from which data were available to estimate the parameters of

the Ricker equation, and owing also to measurement error in those data, and process error

in the actual mechanisms that influence the recruits per spawner for a given brood.

Because this error exists, any manager who intends to make decisions based on

the prediction needs to know it and take it into account. In this stock recruitment example

we expect the total prediction error to be considerable, based on our experience from the

substantial scatter that we see in the fit of observations to empirical stock recruitment

curves. This scatter is a combined result of measurement error, parameter error, and

process error. The first practical question that arises in this connection is: can we

adequately estimate the total prediction error just from the scatter in these empirical

plots?

The answer, unfortunately, is no. The scatter in the plot has contributions from all

three sources of error, and it represents a distribution of the discrepancy between

measurements and prediction -- not between prediction and truth. And the measurements

are subject to measurement error, and the predictions are subject to measurement error,

parameter error and process error. If we use the discrepancy between measurements and
ISAB 2001-1: Model Synthesis Report

Appendix I - 43

prediction in the scatter plot as a quantification of the prediction error, we will have

overestimated the short-term prediction error. The misrepresentation of the long-term

prediction error will likely be even larger and, for rather complicated reasons, we will not

even know whether the long-term prediction error will have been over estimated or under

estimated.

Bear in mind that our estimation procedure for this stock recruitment relationship

example was based entirely on a short-term relationship, namely that between spawners

and recruits over the span of one generation. If we then use this to make a longer-term

prediction, over the span of several generations, the three error sources (measurement,

parameter, and process) will propagate differently. Note that a modeling application such

as a calculation of the probability of extinction has this character of projecting for several

generations into the future.

Measurement error for initial conditions (basically the estimate of the number of

spawners for this brood, in this example) enters the projection once, as a simple

multiplicative factor. Parameter error will be a fixed value that holds for the duration of

the projection, but its effect will compound, more or less exponentially, with each

generation. Process error gets re-sampled each generation, because a new expression of

the environmental "noise" manifests itself, so there will be a degree to which the process

error "averages out" over the generations, even though its effect compounds in each

generation. Thus we will get quite different results for the prediction and for the

prediction error of the projection, depending on how we apportion the error that we see in

the scatter plot among measurement error, parameter error, and process error.

As a result, it is not so simple a matter as automatically assigning all the

regression scatter to the process error (which is what the parameter estimation step of the

Dennis model does), or automatically equating the jack-knife cross-validation residual

sum of squares with prediction error (which is what would happen in a naive comparison

of observed versus predicted conditions in a watershed with the EDT model). The correct

calculation of measurement error, parameter error, and process error is a technically

involved undertaking that is not simply superimposed on a modeling effort as an

afterthought. It has to be built into the modeling, and it is a demanding part of the project.

The technical demands escalate as the model becomes more complex. For

example, fairly simple traditional methods for putting a conventional "confidence

interval" on the estimate of a single parameter may not go too far astray for error analysis

purposes. As soon as there is more than one parameter involved, however, a whole world

of ambiguities open up if we try to estimate the prediction error by simply propagating all

the "confidence distributions" independently for all the parameters.

First of all, the parameter errors are generally not independent; and secondly,

conventional confidence distributions do not have the right probability properties for use

in error propagation. This is reasonably well known in the technical risk assessment

literature, but the solution involves considerable sophistication, and there is not a clear

consensus in this literature on all the details of implementing the solution. Broadly
ISAB 2001-1: Model Synthesis Report

Appendix I - 44

speaking, the key elements of the solution involve Bayesian methods to compute a "joint

posterior distribution" on the entire set of relevant parameters. The joint distribution takes

account of the correlation in error between various parameters, and usually results in a

much smaller computed value for the total prediction error than would result from

independently propagating all the confidence distributions on the parameters. Formally, a

posterior distribution has the right mathematical properties for combination with other

probabilities.

Difficulties with propagating parameter error are revealed clearly enough in

simple applications of the Dennis model for estimating distributions of time to extinction.

The forward projection stage of the Dennis model is a Brownian motion model, which

represents process error in its diffusion term. In the parameter estimation stage of the

Dennis model, confidence limits are put on the "average growth rate" parameter, because

this falls out directly from the regression machinery that estimates this parameter as a

slope. This average growth rate parameter becomes the drift term in the Brownian motion

forward projection. Unfortunately no confidence interval is put on the estimate of the

"variance in growth rate" which becomes the diffusion term in the Brownian motion

projection, even though the uncertainty in this parameter is bound to be large (the

sampling variance for a variance is notoriously large, and declines slowly as the sample

size increases), and this parameter has a large influence on the result.

Furthermore, it is not really clear in the Dennis model framework how to

propagate the confidence calculation for the one parameter, the drift term that is given a

confidence interval, through the prediction of extinction times in the projection stage.

Simple expedients, such as separately calculating distributions of extinction times using

the upper 95% confidence limit, the central estimate (MLE), and the lower 95%

confidence limit of the average growth rate parameter create new problems. What is the

meaning, for example, of the lower 15% tail of the time to extinction using the upper

95% confidence limit of the average growth rate parameter? It is not mathematically

legitimate to fold the "confidence" from a parameter estimate into the probability of

outcome in a projection conditioned on a particular data set. The two quantities are of

fundamentally different kinds -- too different to be combined.

The concession to simplicity of omitting density dependence in the Dennis model

saves it the difficulty of having to deal with even more parameters that would be even

more uncertain. The omission of density dependence, however, leads to other serious

problems. Part of the reason that conventional stock recruitment analysis has not revealed

strong density dependence in the Snake River chinook is, almost certainly, because the

carrying capacity of this system has not remained constant during the interval spanned by

the data. The dominant dynamics for the past several decades has been of populations

tracking the downward spiral of their carrying capacity in a way that is "density

dependent," but does not give a pretty Ricker-curve picture if we try to analyze it,

incorrectly, with an analysis that assumes constancy of the density dependence

parameters for the duration of the data set.
ISAB 2001-1: Model Synthesis Report

Appendix I - 45

These comments on the treatment of uncertainty analysis in simple applications

of the Dennis model do not detract from the many very positive aspects of CRI. In

addition, for the most immediate uses of this chapter of CRI in decision-making, these

criticisms are probably irrelevant. The immediate use of the Dennis model in CRI has

been to determine whether the various stocks are in imminent risk of extinction, and the

answer has already come in that they are. Building in more uncertainty analysis, just to

show that the risk of extinction perhaps is even higher, is probably unnecessary. Where

the uncertainty analysis will be important to the actual decision-making will be in the

detailed analyses to make selections among various interventions.
ISAB 2001-1: Model Synthesis Report

Appendix II - 46

Appendix II. Description of Decision Support Systems Used to Guide

Salmon Recovery Planning in the Columbia River Basin

This appendix contains descriptions of different decision support tools (models) currently

informing salmon policy in the Columbia River Basin. We include descriptions of four

such quantitative analytical systems: Ecosystem Diagnosis and Treatment (EDT),

Cumulative Risk Initiative (CRI), Bayesian Belief Networks (BBN) of the Interior

Columbia Basin Ecosystem Management Project, and Plan for Analyzing and Testing

Hypotheses (PATH). The descriptions were taken with minor editing directly from

reports by primary authors of each modeling effort and are therefore in their own words.

Lars Mobrand of Mobrand Biometrics, Inc. described EDT, Peter Kareiva of the NMFS

Northwest Fisheries Science Center described CRI, Bruce Rieman of the USDA Forest

Service Intermountain Research Station described BBN, and David Marmorek of ESSA

Technology, Ltd. described PATH.

The following summaries do not contain all of the information needed to

understand the analytical systems in detail. Readers are referred to appropriate

publications by project authors for more complete descriptions.

Ecosystem Diagnosis and Treatment (EDT)

Steps in the EDT analysis

The basic steps of EDT are captured in its name, Ecosystem Diagnosis and Treatment:

Ecosystem: The description of the biophysical environment of species and

populations of concern.

Diagnosis: Evaluation of the “health” or quality of the environment with

respect to specific species or populations.

Treatment: The analysis of the impact of different strategies in changing

the existing environment toward one that is more compatible with the

needs of the species or population of interest.

More explicitly, in EDT we:

1. Describe the habitat template in terms of a set of physical attributes of the current

terrestrial and aquatic environment at the level of the HUC-6 (hydrologic unit code) .

2. Assess the habitat template in regard to how it affects biological performance

measured as productivity, capacity and life history diversity of salmon and other

species – i.e. derive a “survival landscape” based on habitat conditions.

3. Evaluate how regional alternatives might change biological performance by relating

strategies to changes in environmental attributes.
ISAB 2001-1: Model Synthesis Report

Appendix II - 47

4. Capture our accumulated knowledge about the relationship between salmonids and

other aquatic and terrestrial species and their environment in the form of documented

“rules” that explain or hypothesize survival responses to their habitat – i.e. organize a

repository of knowledge and information.

5. Perform sensitivity analysis to guide refinement and modification of management

strategies and or objectives and development of monitoring and evaluation plans.

More details on EDT and the Framework Project can be found at the Framework web

site: http:/www.nwframework.com.

The data used in EDT

There are four major types of data used in EDT. The first data category consists of

the 108 strategies that are selected and combined to make up an alternative. In the

Framework Project, these are arrayed across 10 ecological provinces to make an

alternative focused on a particular vision.

In EDT, we distinguish information that is actually observed from information

that is derived from other information. Most of the information that is routinely used in

natural resource management is actually derived from a smaller set of real observations.

For example, counts of adult salmon at mainstem dams, abundance estimates of spawning

fish and the number of fish harvested are all basic fisheries information that is expanded

or derived from a much smaller set of actual observations. As you might expect, to

describe the habitat of the entire Columbia River, we have to derive a lot of the

information. We note where these derivations occur and base them on a set of explicit

rules. The description begins by filling in the available information for each of the 7,200

HUC-6 units. Where information is missing, the scientific literature and appropriate

experts are consulted to derive rules for filling in the gaps.

The next higher level of derived information is the assessment of the habitat with

respect to specific fish or wildlife species. Habitat quality is assessed for each life stage in

terms of productivity and capacity. For chinook salmon we distinguish habitat quality for

each of 16 life stages.

Finally, the productivity and capacity is integrated over the entire life history to

derive the overall estimated productivity and capacity. A number of life history pathways

are tested to assess the impact on life history diversity as well.

The different data types are related through a set of rules that are the heart of the

EDT expert system. They are the basis for describing habitat and relating habitat

observations to the higher level environmental attributes and the resulting biological

response. The rules capture the region’s expertise and are derived from empirical

research, the scientific literature and expert opinion.
ISAB 2001-1: Model Synthesis Report

Appendix II - 48

There are four types of rules that call on different types of expertise. The first

links each of the 108 strategies to one or more of 51 types of habitat observations. Those

devising the strategies and alternatives intend them to result in some change in the

environment. This is formalized by capturing the knowledge of how different types of

actions (strategies) affect our 51 descriptors of habitat conditions.

The second type of rule links the observations to 44 Environmental Attributes.

These rules describe how the habitat observations are expanded to account for incomplete

or missing information. These rules are devised by hydrologists, geomorphologists and

other physical scientists to expand the direct habitat observational data to all 7,200 HUC6 units for all time periods.

The third rule category is based on the knowledge of biologists regarding life

stage survival response or productivity of specific species to one or more of the

Environmental Attributes for each HUC-6. The resulting Biological Metrics can be

thought of as 19 graphs showing the relationship between life stage productivity and

Environmental Attributes.

The final type of rule is an algorithm that integrates over the entire life history

pathway to compute total productivity and capacity. This is done for each successful life

history pathway to estimate life history diversity.

The scale and resolution of the EDT analysis

EDT paints a picture of the biological and physical landscape of the Columbia

River. The “pixel” size of this picture is the HUC-6 (hydrologic unit code) of which

there are approximately 7,200 in the Columbia River basin. These are organized in a

spatial hierarchy to describe subbasin, ecological provinces and the Columbia River

basin. EDT describes the equilibrium condition of the basin as a result of a set of

strategies. Time is not a factor in EDT except in regard to the explicit description of the

various life history pathways of target species.

Biological resolution in EDT is limited only by the available data. The present

analysis assesses habitat conditions in terms of four species: two aquatic species, chinook

salmon and bull trout, and two terrestrial species, black bear and beaver. Data has been

assembled to distinguish 107 natural and 50 hatchery populations of chinook salmon as

well.

Measures of “performance” in the EDT analysis

EDT measures biological performance in terms of three population parameters:

productivity, abundance potential, and life history diversity. Productivity is the density

independent component of survival times the rate of reproduction. Abundance potential is

the carrying capacity of the habitat. Productivity and abundance potential are assessed

for habitat in each life history stage. They are integrated over the life history to describe

performance of a particular life history pathway. The grain and variation of the habitat
ISAB 2001-1: Model Synthesis Report

Appendix II - 49

description means that there can be more than one potentially successful pathway across

the “survival landscape.” This provides an assessment of the strategies in terms of their

positive or negative effect on life history diversity as well as population productivity and

abundance. All three parameters reflect the ability of the environment to support salmon

production in the long term.

General philosophy and aim of the EDT

The general philosophy of EDT is well grounded in established scientific

principles describing the influence of habitat structure on the characteristics of biological

communities. Specifically, EDT assumes that habitat characteristics determine biological

performance. Hence, changing habitat through natural events or by human action will

have a corresponding effect on biological performance.

A second philosophical point is that EDT recognizes that the scientific basis for

decisions regarding the future of the Columbia River cannot and will not be limited to

statistically “proven” knowledge. While statistically based information is important,

scientists are increasingly aware that the complex and dynamic nature of ecological

systems is currently, and perhaps always will be, imperfectly captured in statistical

relationships. Prudent management must take advantage of all available information.

For this reason, EDT uses both statistically based and heuristic knowledge.

The current application of EDT is to the Multi-Species Framework Project

sponsored by the Northwest Power Planning Council and federal and tribal management

agencies. The focus of the Framework Project is on the long-term vision for fish and

wildlife management in the Columbia River. Development of a vision involves

consideration of the types of strategies required and their impacts in terms of human

communities and other social and economic factors in addition to their ecological

impacts. EDT is being used in this context as a tool to facilitate long-term basin-wide

planning. However, it also will provide a basis for development of more detailed and

shorter term plans for individual subbasins.

Cumulative Risk Assessment (CRI)

The four key steps to a CRI analysis

1.) Estimate the risk of quasi-extinction for known populations

2.) Construct demographic projection matrices that depict current demographic

performance rates and in turn can be used to calculate annual population growth rates

(assuming a “current conditions”).

3.) Perform sensitivity analyses to assess where in the life cycles of salmonids there are

the greatest opportunities for promoting recovery, as measured by changes in the

annual population growth rate. This can be done several different ways. The simplest

is to manipulate the values in baseline matrices to represent particular demographic

improvements, and calculate the % increase in annual population growth rate that
ISAB 2001-1: Model Synthesis Report

Appendix II - 50

results. This increase in annual population growth can then be converted into an

estimated reduction in quasi-extinction risk.

4.) For those demographic improvements that give a noteworthy response in terms of

population growth, identify management actions that might accomplish those

improvements, and use statistical analyses or experimental studies to determine

whether there is evidence that those improvements are actually feasible with the

management action being considered.

More details can be found in several other documents available from the CRI website

(http:/www.nwfsc.noaa.gov/cri/).

The data used in CRI

The primary data used by CRI are time series of population counts, and recruits

per spawner ratios. An example is given below, from fall chinook salmon in the Snake

River.

Table 1. Counts for Fall chinook salmon.

year spawners

Recruits to spawning

grounds (total)

Recruits to

spawning grounds

(minus jacks)

1980 515 2294 1285

1981 878 1555 983

1982 1209 1810 1224

1983 909 1986 1115

1984 717 1764 934

1985 1080 654 541

1986 1403 706 539

1987 1064 373 292

1988 702 747 710

1989 815 656 529

1990 273 284 227

1991 767 300 206

1992 674

1993 883

1994 448

1995 226

1996 964
ISAB 2001-1: Model Synthesis Report

Appendix II - 51

From this, one can calculate an extinction risk, and estimate how much we need to

increase annual population growth to mitigate this risk, as shown:

Table 2. Quasi-extinction risks for Snake River fall chinook salmon (based on data from

1980-1996).

Avg. λ

avg. N over

last 5 years

p(one spawner

within 10 yrs)

p(one spawner

within 100 yrs)

Fall Chinook 1.13

(0.89-1.44)

639 <0.0001

(<0.0001-0.16)

0.06

(0.0002-1.0)

Table 3. Quasi-extinction probability for Snake River Fall Chinook associated with

particular increases in λ.

p(one spawner within 100 years)

% change in λ Fall Chinook

5 0.005

10 0.0003

15 1.3 x 10-5

20 7.1 x 10-7

It may also be possible to construct a detailed demographic matrix that can then

be used to simulate management experiments such as harvest reductions. Below, as an

example is the Snake River fall chinook salmon demographic matrix.

2 3 4 5 6

Age

frequency of

females (fx)

0 0.129 0.652 0.198 0.020

93-96 Ocean

harvest rate

(hx)

0.0123 0.0465 0.1368 0.1838 0.1953

Female eggs

per female

spawner

(mx)

1442.5 1566.5 1625.5 1625.5

Propensity

to breed (bx)

0 0.081 0.648 0.859 1.0

93-96 Mainstem adult harvest rate 0.174

93-96 adult Bon to Basin conversion rate 0.471

s1 0.0102

These parameters are then substituted into the following matrix where,

μ represents the age-specific fraction of ocean dwelling salmon that return to spawn (it

combine the probability of returning during that year, with the survival rate swimming

upstream, which includes harvest reductions as well as other mortality), s1 represents
ISAB 2001-1: Model Synthesis Report

Appendix II - 52

survival during the first year of life, sA is survival as adults living in the ocean, and hi

indicates ocean harvest rates on fish in ageclass i.

1 2 3 4 5 6

1 0 0 (1-μ)s1b3m3 (1-μ)s1b4m4 (1-μ)s1m5 (1-μ)s1m6

2 (1-h2)sA 0 0 0 0 0

3 0 (1-h3) sA 0 0 0 0

4 0 0 (1-b3)(1-h4)sA 0 0 0

5 0 0 0 (1-b4)(1-h5)sA 0 0

6 0 0 0 0 (1-b5)(1-h6)sA 0

This matrix can be used to simulate the consequences of reduced harvest, and

other management actions. Importantly, for many management actions (almost

everything other than harvest reductions) it is not certain whether a given action will

accomplish the desired demographic improvement. This is where the “feasibility studies”

come into play. For a feasibility study, the dependent variable will typically be recruits

per spawner, number of spawners, smolts per spawner, smolt-to-adult returns, or survival

during some life stage. Correlations are then sought between these measures of salmonid

productivity and variables such as number of hatchery releases, fraction of stream miles

failing to meet EPA water quality standards, and so forth. What CRI envisions as

feasibility studies also represent evidence that EDT uses in constructing its “rules”

relating stream attributes to salmon production.

The scale and resolution of CRI analyses

CRI is most effective when applied to distinct populations, or collections of

populations. This is because it focuses on population growth rate and a population’s risk

of extinction. The spatial scale at which CRI best operates ranges from subwatershed on

up to subbasin or basin. As it is currently developed, CRI is not equipped to deal with an

entire province or region comprised of many populations and multiple ESUs. However,

there are plans for extending the CRI to this large scale (beginning with a technical

workshop in December 2000 at NWFSC, which was aimed at multiple populations and

ESU-wide priority-setting). CRI would never be used at the fine scale of a particular

reach or stream. CRI could never inform us about reach-specific or small-scale

management actions. The output of CRI often takes the form of: “ if this, then the

expected response is ___”. CRI does not deal with individual fish at all, and also does

not deal with life history diversity. In the absence of data and statistical relationships, the

CRI does not venture very far with its analysis.

Measures of “performance” for the CRI analysis

The primary measure of performance for CRI is average annual rate of population

growth. This core measure is then the basis for two additional measures of performance:

risk of extinction over 10 years and 100 years, and the percentage by which annual

population growth is expected to increase with some management action. Although it is

impossible to validate “risk of extinction” as a performance measure, annual population

growth rate and % change in annual population growth can be validated – these are both
ISAB 2001-1: Model Synthesis Report

Appendix II - 53

measurable, and in fact are routinely available from the type of spawner or redd counts

typically made for salmonids.

CRI’s general philosophy and aims

CRI’s three most distinctive features are:

1.) an emphasis on simplicity and simple models, so that others outside NMFS can repeat

their own analyses with slight modifications of the assumptions, new data, different time

periods, different levels of risk averseness, and so forth,

2.) a staunch empiricist’s skepticism, such that a priority is placed on relationships

supported by data, and that otherwise must be couched as “if this, then that” statements,

3.) focusing on population dynamics or demography as the window through which to

evaluate management actions.

Bayesian Belief Networks (BBN) of the Interior Columbia

Basin Ecosystem Management Project

We were asked to evaluate the potential effects of three land management

alternatives on fishes and their habitats across the entire Interior Columbia River Basin

(ICRB). The ICRB represents an area of about 58 million hectares, and roughly 7,000 6th-

code watersheds (also known as subwatersheds) (Figure 1) supporting 88 taxa of native

fishes. Those alternatives included spatially explicit aquatic conservation and restoration

strategies that dictated such things as riparian buffers, watershed analysis, and a system

of designated watersheds where aquatic conservation was emphasized over all other

management objectives. Information available for this analysis included an existing,

probabilistic assessment of the current status and distribution of seven “key salmonid”

fishes. It also included independent, spatially explicit projections of the intensity and

distribution of ground disturbing/mitigating activities (e.g. logging, road building, road

closure) and anticipated changes in large scale patterns of wild fire associated with each

alternative.

Any evaluation of land management effects for the entire ICRB is a daunting task.

It is obviously complicated by the sheer size and complexity of the basin, and by the

relative lack of information and understanding of the physical and ecological processes

relevant at such a large scale. This analysis was further complicated by an urgency in the

decision process that dictated an analysis completed in the course of only a few months.

There have been two basic approaches to the evaluation of broad scale land

management proposals. One, characterized in earlier iterations of the ICRB planning

process and other large Federal Assessments has used a summary of expert opinion. In

essence, available information is arrayed before a panel of scientists or resource

management specialists who provide an interpretation of likely outcomes based on their
ISAB 2001-1: Model Synthesis Report

Appendix II - 54

background and experience. Although simulation or empirical models for some specific

linkages or environments might represent some of the available information, the system

is characterized largely in the participants minds. This approach proved unsatisfying for

several reasons: (1) the huge body of information made it difficult to conceptualize and

account for multiple interacting effects; (2) The influence of the experts' assumptions on

results of analyses could not be evaluated directly; (3) the analyses could not be revisited

or updated easily when management drivers or key assumptions were modified; and (4)

the results were not easily quantified or made spatially explicit and thus could not provide

a clear contrast among alternatives.

A second approach has been model based where predictions of species/system

responses are made through mathematical abstractions that attempt to characterize the

relevant physical and ecological processes. Formal, model based approaches directly

address the last three issues linked to expert judgement above. Important issues with

model based approaches include the invariable lack of information needed to

parameterize, or the computational resources and time to apply, very detailed models for

more than a handful of landscapes or populations. Although attempts to model whole

ecosystems have often incorporated remarkable complexity (e.g. EDT), it is not clear that

mechanistic detail necessarily leads to better predictions.

In our evaluation we attempted to resolve these issues by combining expert

judgement and a relatively simple, but formal model framework of Bayesian belief

networks (BBNs). Also known as Influence Diagrams, Causal Trees, or Probability

Networks, these networks are typically represented by box and arrow diagrams that

depict causal linkages among the physical and ecological factors that influence the

probability of an outcome or state of interest (e.g. the status of a bull trout population).

The linkages between elements or “nodes” of the diagram are represented by conditional

probability tables that reflect the probability or frequency of a particular state given the

states in all “parent” nodes that directly influence the one of interest.

Bayesian Belief Networks provided an appealing framework for the ICBRB

analysis for several reasons. First, the networks are explicit, formal representations of

our knowledge, beliefs, and uncertainty for complex ecological systems. Essentially these

networks provide a means of articulating what we think about the causal web influencing

species and their habitats. The analysis and the logic underpinning it become far more

transparent than one based solely on expert judgment.

Second, representation of the linkages and outcomes as probabilities

acknowledges the lack of precision in predictive models that may arise through limited

understanding of the system, inherent variation in biological and physical processes, and

lack of data. A distinct advantage of this approach is the ability to combine information

derived empirically and from expert opinion. The subjective element of “expert”

judgment is clearly a contentious point, but in reality for this kind of analysis there is no

other option. By incorporating information from multiple advisors, areas of consensus

and disparity of opinion become obvious and can be reflected directly in the conditional

probabilities. Where a particular relationship is understood well experts and data will
ISAB 2001-1: Model Synthesis Report

Appendix II - 55

tend to converge on a strong distinction in probabilities among alternative states. Where

it is not, differences of opinion or noise in the data will produce a more uniform

distribution among states reflecting either the lack of a strong ecological linkage or a lack

of understanding.

Third, the networks can be as simple or complex as information and

understanding allow. The absence of mechanistic detail does not necessarily limit the

utility of a model. For example there is good evidence of a relationship between the level

of land disturbance and the occurrence and persistence of bull trout. Most experts would

agree that substantial watershed disruption associated with high densities of roads and

logging will increase the likelihood that a bull trout population will not be strong. A

simple and potentially useful model might include only that single linkage. Alternatively,

a model could include additional detail such as the expected links between canopy

removal, stream temperature, juvenile growth and survival, population growth rate, and

extinction risk. The additional detail may be more appealing, but far more difficult to

support for an analysis across a huge region. The construction of networks is an exercise

in compromise that attempts to balance mechanistic detail with the strength of the

relationship implied by any linkage and the information and understanding available to

drive the model. Collapsing the model to fewer, simpler linkages does not necessarily

compromise the ability to make a prediction if the fundamental tie between the ultimate

driver (amount of land disturbance) and outcome of interest (bull trout status) is believed

to be relatively strong.

Finally, the computational demands are relatively simple. Networks can be

constructed easily with commercially available software or traditional statistical packages

suitable for matrix manipulation. Even the most complex networks can be updated and

resolved in a matter of minutes or even seconds on current PCs. For an analysis of the

ICRB this was extremely useful because it meant that we could replicate our analyses

across thousands of watersheds in the Basin limited only by our ability to develop the

tables of input information characterizing watersheds and the management changes or

trends expected with each alternative. That flexibility lends itself well to sensitivity

analysis that allows an exploration of model behavior, changing alternatives, and

competing worldviews.

Application to the ICRB

Our analyses focused on six salmonid fishes and their habitats. We considered

effects of the management alternatives on habitat on all lands, but for salmonids we

focused only on the potential spawning and rearing areas. We excluded consideration of

areas classified only as corridors or seasonal habitats for three reasons: 1) spawning and

rearing habitats are the critical areas found predominantly on Federal land and are the

habitats most sensitive to Federal land-use management; 2) spawning and rearing areas

are more likely to be in headwater systems; and 3) we have poor understanding and

ability to predict the influence of multiple effects over the very large and complex

catchments contributing to downstream habitats. We do not imply that federal land
ISAB 2001-1: Model Synthesis Report

Appendix II - 56

management does not influence more downstream areas, but that its effects will be more

evident, predictable, and important in spawning and rearing areas.

For the purposes of our evaluation, we assumed that landscape conditions and

management effects were immediate. Lotic habitats and fish populations are dynamic.

They are continually responding to changes in landscapes as a result of management and

natural processes. Some responses may occur quickly, whereas others may lag years or

decades behind the changes on the landscape. Our results therefore cannot be viewed as

absolute estimates of the number of strong populations or high capacity habitats that will

exist. Rather, they are measures of the relative differences that may be expected among

locations and species, given estimates of current and future landscape conditions

associated with each alternative.

Network Structures

Within the limits of available information, we attempted to represent physical and

biological processes most likely to influence distribution and dynamics of the salmonids

and their habitats. Our networks were characterized by a collection of components

(nodes) that represented environmental states or processes and two variables of primary

interest, aquatic habitat capacity and the future status of each salmonid.

Within the geomorphic constraints of any watershed, upslope disturbances (e.g.,

logging, roads, fire) that cause accelerated production of sediment, alter hydrologic

regimes, or alter the characteristics of riparian areas are generally accepted as primary

drivers that influence the condition of habitats for fishes in wildland systems. In our

network aquatic habitat capacity depended on: (1) generation and delivery of sediment;

(2) the occurrence of large channel reorganizing floods; and (3) the condition or integrity

of the riparian corridor. Sediment was defined as the relative amount of sediment

entering streams above natural rates. It was influenced by road density and ground

disturbance (i.e., logging, thinning and prescribed fire), topographic conditions (slope

steepness), and management activities designed to mitigate erosion or sediment delivery

(standards and guides). Hydrologic effects included probability of flood/debris-flow

events that could reorganize large portions of the steam network. These events were

influenced by the probability of having a flood-generating storm, by slope angle (slope 2)

and the occurrence of large fires that dramatically alter hydrologic function (fire-rain).

Riparian condition was perceived as those characteristics influencing shading and climate

moderation, bank stabilization and water storage, and delivery of coarse and fine organic

material. It was influenced by prior riparian condition, future grazing, and management

activities intended to conserve or restore riparian function (standards and guides).

Salmonids can be strongly influenced by the physical capacity and quality of

habitat. Status of many populations, however, is not strictly a function of local habitat

conditions. Introductions of exotic species or the loss of some keystone forms such as the

anadromous salmonids may have profound effects. Larger scale habitat fragmentation

and isolation from surrounding populations can also be important.
ISAB 2001-1: Model Synthesis Report

Appendix II - 57

For these reasons the future status of the salmonids in each subwatershed was

conditional on one or two factors that were independent of local habitat capacity. The

future status of resident salmonids (bull trout, Yellowstone and westslope cutthroat trout,

redband trout) depended upon aquatic habitat capacity and on biological potential of the

existing population. In addition to these two, status of anadromous salmonids (stream

type chinook salmon, steelhead) also depended on conditions influencing survival of

migrants to and from the ocean (migrant survival). Biological potential represented

constraints on population resilience, productivity, and size, dependent on current

condition of the population (current population status), the biotic effects associated with

introduction or loss of members of the associated fish community (exotic threat,

anadromous loss) and potential for demographic support from surrounding populations

(refounding and support, connectivity). Migrant survival depended both on conditions in

the migratory corridor (i.e., number of dams that must be passed by migrating fish) and in

the ocean.

Conditional dependencies in the aquatic habitat and population status networks

were estimated using a combination of expert judgement and empirical relationships. The

conditional probabilities linking directly with salmonid status were estimated

independently for each species. For many of the key nodes we relied on multiple experts

and averaged their estimated probabilities to reflect the relative uncertainty in collective

beliefs. Any disagreement among experts produced a more uniform distribution of

probabilities across states reflecting greater uncertainty in the conditional dependencies.

Available Information

Existing syntheses of landscape characteristics, fish assemblages, and an

interpretation of planned management activities based on the alternatives outlined in the

Supplemental Draft Environmental Impact Statement (SDEIS) represented the primary

information available for our analyses. The biophysical coverages summarized to

subwatersheds were obtained from the Interior Columbia Basin Ecosystem Management

Project. Predictions of the land management activities included estimates of road density,

mechanical ground disturbance, livestock grazing, and probability of large wildfire. In

addition, we developed our own series of rules to assign a level of conservation and

restoration (high, moderate, low) in each subwatershed based on the management

direction outlined in the SDEIS alternatives.

All inputs for the networks were summarized from equivalent (species status and

distribution) or finer resolution (landscape data derived at 1 km pixel) information.

Variables represented by the nodes in our network then are viewed as conditions

representative of entire subwatersheds.
ISAB 2001-1: Model Synthesis Report

Appendix II - 58

Plan for Analyzing and Testing Hypotheses (PATH)

Our primary focus in the work accomplished to date has been to make a start at

developing some tools and procedures for conducting quantitative analyses of

experimental actions. We have developed a set of experimental management (EM)

modeling tools that allow us to quickly assess the biological/conservation consequences

and learning opportunities of actions that affect overall survival of Snake River spring

and summer chinook salmon. These models are intended to provide a starting point for

additional work after PATH is discontinued.

We have used these models to conduct some preliminary screening and analyses

of the short-list of actions. These analyses are preliminary because:

1. We have not done a thorough assessment of the feasibility of implementing these

actions. Because of this, we have evaluated a set of generic and hypothetical

experimental actions without speculating about how these actions might be actually

implemented.

2. We have only looked at the effects of individual actions; combinations of actions may

be more effective.

3. We assume that an action will have some effect, then assess the resulting biological

and learning consequences. We have not assessed the weight of evidence in support

or against the assumed magnitude of effects.

4. In most cases, we have only looked at how long it would take to detect effects in

overall survival, from spawner-recruit data.

Our preliminary assessments should therefore be viewed as illustrations of “what

if” scenarios of management experiments. We address the question “Suppose that a

particular action could be feasibly implemented and had a particular effect on Snake R.

spring/summer chinook populations: What would the biological consequences of such an

action be, how difficult would it be to estimate that effect from spawner-recruit data with

reasonable confidence, and what are the resulting trade-offs between learning and

biological objectives?” These assessments are useful for developing and testing our EM

models, and for providing some broad guidance on the learning and conservation

implications of various actions.

Outputs

A. Biological

The primary output of the model is projected numbers of spawners and recruits

for seven Snake River index stocks of spring / summer chinook. From these, we calculate

probabilities of exceeding 1995 BiOp recovery and survival thresholds1 over 24 and 100

 

1 These are the probabilities that the number of spawners of 6 out of the 7 index stocks will exceed

survival and recovery threshold numbers of spawners. Survival thresholds range from 150 to 300 spawners;

recovery thresholds range from 350 to 1150 spawners, depending on the stock.
ISAB 2001-1: Model Synthesis Report

Appendix II - 59

years (survival standards) and 24 and 48 years (recovery standards). We also calculate the

probability of going to one spawner or less in a given year (over 10 and 100 years) as a

quasi-extinction metric similar to that used by CRI in their August 1999 document.

In order to calculate these metrics, we assume that actions will be maintained for

the duration of each metric’s time horizon (i.e. 24 and 100 years for survival

probabilities, 24 and 48 years for recovery probabilities, and 10 and 100 years for quasiextinction metrics). With the possible exception of the drawdown actions, this

assumption is probably not realistic because if one discovers a suite of actions that meets

survival and recovery requirements, one likely would not continue with the original

on/off experiment. The population metrics included here may thus be viewed as a relative

index of the biological consequences to the stocks, if the experimental actions were

continued indefinitely.

Probabilities of exceeding survival and recovery thresholds are lower in this

analysis than in previous PATH reports because of differences in some of the

assumptions and data used in the model:

• Because we use 1978-1994 as representative of current conditions, we are assuming

that the poor ocean conditions that existed in this time period continue into the future.

• We have assumed in most cases that extra mortality2is “here to stay”. That is, we

assume that the same high level of extra mortality that was experienced in 1978-1994

continues on into the future.

• This analysis uses updated spawner-recruit data, which includes spawner data up to

1999. Spawner numbers in these years were generally low, with zero spawners in

some years for Marsh Creek and Sulphur index stocks.

B. Learning

The main metrics of how much can be learned from an action are expressed in

terms of the probability of estimating effects of an action over various time frames, or,

conversely, how long it would take to estimate an effect with a certain level of

confidence. Various criteria can be applied to determine how long an experiment needs to

be run to estimate effect sizes that reflect the risk preferences of decision-makers. We

present three examples for illustration:

1) one approach might be to require the experiment to not have a negative estimated

effect on survival. In this case, decision-makers would want to know the probability

of estimating any non-zero effect on survival rates, and how this probability changes

as the experiment goes on. This is the least stringent of the three examples; the effect

can be estimated with high probability in a relatively short period of time.

 

2 Extra mortality is defined as any mortality occurring outside the juvenile migration corridor that is

not accounted for by: (1) productivity parameters in the spawner-recruit relationship; (2) estimates of direct

mortality within the migration corridor; (3) common year effects influencing both Snake River and Lower

Columbia River stocks; and (4) random effects specific to each stock in each year.
ISAB 2001-1: Model Synthesis Report

Appendix II - 60

2) decision-makers may want to know that the estimated effect of the action is close to

(say, 80% of) its hypothesized effect. When hypothesized effects are large, this is

generally the most difficult criterion to meet (i.e., probabilities of meeting it are

lowest).

3) if one applies standard criteria for designing experiments, we would want to be fairly

certain that we do not claim that an effect exists when in fact the action has no effect.

To do this, we define a critical effect size (Δm*), which is set at a level that

minimizes the probability (0.05 or less) of incorrectly concluding that there is an

effect when in fact there is none. The probability of detecting this critical effect size,

if it exists, is called the “power” of the experiment; the higher this probability, the

more “powerful” the experiment. This is the most difficult criterion to meet when

hypothesized effects are small.

Model Structure

The model is based on the Ricker model of Recruits vs. Spawners that is used for

most analyses of Pacific salmon populations. Natural log units are used to linearize this

model because this makes it easier to deal with the wide variability that characterizes

most spawner-recruit data sets and to estimate the model’s parameters3. The model can be

expressed as:

ln(Ri,t/Si,t) = ai + biSi,t + mt + εi,t

or alternatively as: R/S ∝ productivity carrying cap. year error term

 factor factor effect

for each stock, each stock, all stocks, each stock,

 all years all years each year each year

These parameters are estimated from historical spawner-recruit data, then used in

forward projections to simulate the effects of actions. Assumptions about the effects of

experimental actions are implemented in the model through the “mt ” or “year effect”

term, which can be thought of a general survival factor for each year that affects all

Snake River spring chinook stocks simultaneously. In the model, mt values are calculated

relative to the average survival rate from spawner to recruit over the entire historical time

period (1958 to 1994). For years when mt = 0, overall survival was equal to the long-term

average. When mt is positive, overall survival was better than average; when mt is

negative survival was worse than average. Because mt is in natural log units, every unit

increase (decrease) in mt increases (decreases) survival by a factor of 2.7 (1 / 2.7) relative

to the historical average. For example, when mt = 1, survival in that year was 2.7 times

the historical average. When mt = 2, survival in that year was 7.4X the historical average

(=2.7 X 2.7). When mt = -1, survival in that year was 0.37X the historical average (=1 /

2.7).

 

3 When natural log units are used the error term, which for spawner-recruit data is assumed to follow a lognormal distribution, is transformed into a normally-distributed parameter. This allows us to fit a linear

model to the log-transformed data.
ISAB 2001-1: Model Synthesis Report

Appendix II - 61

Modeling Process

1. Estimate model parameters (ai, bi, mt, εi,t) from historical spawner-recruit data (Figure

ES-1).

-2.5

-1.5

-0.5

0.5

1.5

2.5

0 500 1000 1500

Spawners (1957-1994)

ln(R/S)

} m_t = 1

m_t = -1 {

-3

-2

-1

0

1

2

1955 1965 1975 1985 1995

Brood Year

m_t (year effect)

Rel. change in survival

1

2.7

7.4

0.4

0.14

0.05

Figure ES-1: Estimation of mt from historical data (left) and historical time series of mt (right). Spawnerrecruit data shown in left panel is a hypothetical dataset generated for illustration purposes.

2. Specify a future time series of mt for simulating experimental actions. The future time

series of mt =

an historical mt value selected at random from the 1978-1994 mt values (this was

used as the base period because conditions between 1978 to 1994 were assumed

to be most like present conditions)

plus

a hypothesized effect on survival of the future action (this term is called Δm). For

example, consider a hypothetical experiment in which some action is turned on

and off in successive years. If this experimental action is hypothesized to cause a

2.7-fold improvement in survival in each year the action is implemented

(“treatment year”) relative to years where the treatment is not applied (“control

year”), the time series of Δm values for the forward simulation would be Δm = 1,

0, 1, 0, etc. for the duration of the experiment.
ISAB 2001-1: Model Synthesis Report

Appendix II - 62

-3

-2

-1

0

1

2

3

1995 2000 2005 2010 2015 2020

Year

m_t

mt selected from base period

mt + delta-mt

Rel. change in survival

1

2.7

7.4

0.4

0.14

0.05

The result is a future time series of Δm values that shows how an action is

hypothesized to change overall spawner-recruit survival rates from the survival rates

experienced between 1978-1994 (Figure ES-2 shows an example using the Δm=1/0 in

on/off years example).

Figure ES-2: Example future time series of mt values for forward projections of Δm=1,0 on/off

experiment.

3. Use the future time series of mt, along with historical estimates of the other model

parameters (a, b, ε) to project populations through the experimental period. Simulate

future data collection and analyses. Estimate Δm as the difference in average

simulated ln(R/S) in treatment and control years (Figure ES-3). Calculate

probabilities of recovery, survival, and extinction.

-3

-2

-1

0

1

2

1995 2000 2005 2010 2015 2020

Simulated year of experiment

ln(R/S)

1995

1996

treatment years

control years

-3

-2

-1

0

1

2

1995 2005 2015

Simulated year

ln(R/S)

Treatment years

Control years

Ave. 

treat

Ave. 

control

}

estimated

Δm

Figure ES-3: Method of estimating Δm from future time series of ln(R/S) data. Data are hypothetical

examples for illustration purposes only.
ISAB 2001-1: Model Synthesis Report

Appendix II - 63

4. Do this over multiple trials (i.e., many alternative futures) to get a frequency

distribution of estimated Δm and biological metrics for different lengths of experiments

(longer experiments = more data = better information) (Figure ES-4). Calculate

probabilities of detecting various levels of Δm. Earlier in Section ES.3 we presented three

examples of effects decision-makers may be interested in. These effect sizes can be

translated into terms of Δm (Table ES-2). The frequency distributions are used to

calculate the probabilities of estimating these Δm values.

Table ES-2: Δm equivalents of three example effect sizes decision-makers may be interested in

estimating.

Effect Corresponding Estimated Δm value

Experiment has no negative effect on survival Δm ≥ 0

Effect of the action is close to its hypothesized effect Δm ≥ 0.8 X the “true” hypothesized Δm value

Statistical “critical” effect size (Δm*) Δm* ≥ 1.64 X std. deviation of the estimated Δm

Figure ES-4: Distribution of estimated Δm values for a 1/0 on/off type of experiment of various

durations.

Inputs

The primary model inputs are time series of Δm values for each experimental action,

where these Δm’s represent hypotheses about how the action will affect overall survival

rates relative to those experienced from 1978 to 1994. These are specified for a series of

“generic” actions, in addition to the six experimental actions.
ISAB 2001-1: Model Synthesis Report

Appendix III: Questions - 64

Appendix III. Written Responses of the Principal Scientists to

the ISAB’s Questions about Their Analytical Approaches

Contents:

ICBEMP ANSWERS 66

CUMULATIVE RISK INITIATIVE ANSWERS 75

PATH ANSWERS PART I - GENERAL AND FLUSH 87

PATH ANSWERS PART II - CRISP AND ALPHA LIFE CYCLE MODEL 100

PATH ANSWERS PART III - EXPERIMENTAL MANAGEMENT 104

ECOSYSTEM DIAGNOSIS AND TREATMENT ANSWERS 111

CRITFC COHORT MODEL ANSWERS 120

In undertaking this project, the ISAB requested the assistance of the principal

scientists involved in each analytical approach. The ISAB asked that they: 1) provide

written responses to a series of questions about their modeling effort, 2) give an oral

presentation of their responses and participate in a discussion of the responses with the

ISAB, and 3) participate in a panel discussion with the ISAB and the representatives of

each of the modeling efforts. The principal scientists were informed that their responses

would be incorporated into and appended verbatim to the ISAB report. Each of the

scientists provided written responses, which were distributed to the other participants in

the May 2000 workshop at the Northwest Power Planning Council offices in Portland.

This appendix contains the responses.

The ISAB asked the principal scientists to address the following questions in their

written responses and oral presentation:

1. What is the purpose of your modeling effort? What questions or problems were your

models designed to address?

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s conclusions

relative to the following:

a. The efficacy of dam breaching or drawdown to natural river levels for delisting

ESA species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

b. The efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

Consider hatcheries that mitigate for lost habitat and those used to supplement

depleted stocks.

c. Allocation of harvest and harvest levels needed to delist ESA species and

restore diverse and productive populations of native fishes throughout the

Columbia River Basin?
ISAB 2001-1: Model Synthesis Report

Appendix III: Questions - 65

d. The efficacy of restoration of tributary and mainstem habitat for delisting ESA

species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

3. What kinds of information or data are needed to run your model?

4. What are the strengths and weaknesses of your model?

5. What are the assumptions of your model?

6. How does your model address uncertainty?

7. All models make predictions. Why do you think your model’s predictions are

accurate?

8. How does your modeling effort relate to or contrast with the other three modeling

efforts?

9. What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River Basin?
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 66

ICBEMP Answers

Submitted by:

Bruce Rieman, USDA Forest Service, Rocky Mountain Research Station

Philip Howell, USDA Forest Service, Pacific Northwest Research Station

James Peterson, Cooperative Fish and Wildlife Research Unit, University of Georgia

1. What is the purpose of your modeling effort? What questions or problems were your

models designed to address?

Our models were developed to evaluate the relative effects of proposed land

management plans on aquatic habitats and species on Forest Service and Bureau

of Land Management lands in the Interior Columbia Basin. Specifically, our

models were used to compare the expected changes (from current conditions) in

aquatic habitat capacity and future population status of six salmonid species

among three land management alternatives. The species examined are relatively

widely distributed on FS/BLM and included bull trout, westslope cutthroat trout,

Yellowstone cutthroat trout, redband (resident rainbow trout), stream-type

chinook, and steelhead. Our results were projected at 10 and 100 years from

present and summarized for 6th code hydrologic units containing FS/BLM lands,

all lands (public and private), and other subdivisions (e.g., ESUs) within the

ICBEMP management area. Relative trends for projected habitat capacity of the

6

th code watersheds associated with 17 other native fishes classified as sensitive

(Lee et al. 1997) were also summarized, but we did not project population status

for these species. The Supplemental Draft Environmental Impact Statement

describing the model in detail will be released for public comment in the near

future.

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s

conclusions relative to the following:

a. The efficacy of dam breaching or drawdown to natural river levels for

delisting ESA species and restoring diverse and productive populations of

native fishes throughout the Columbia River Basin?

Our models are not useful for evaluating the efficacy of dam breaching or

drawdown. Dam effects were included in our models because our direction was

to provide as much spatial detail as possible and because dams could potentially

obscure the effects of changes in spawning and juvenile rearing habitat associated

with each of the alternatives. Our models reflected a strong belief that mainstem

dams have an important influence on migrant survival that may constrain the

response of populations to habitat conditions.
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 67

We evaluated two scenarios: one in which migrant survival in Snake Basin

chinook and steelhead areas was classified as low and another where migrant

survival was classified as moderate. Even though the manipulation of migrant

survival produced positive changes in some outcomes in the status of chinook and

steelhead, the relative performance of the SDEIS alternatives did not change.

Although our models reflected the belief that dams have an important influence

on survival, that influence had essentially no effect on the projected differences

among the alternatives we were asked to evaluate. 

b. The efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

Consider hatcheries that mitigate for lost habitat and those used to supplement

depleted stocks.

The efficacy of hatcheries was not addressed in our models.

c. Allocation of harvest and harvest levels needed to delist ESA species and

restore diverse and productive populations of native fishes throughout the

Columbia River Basin?

The allocation of harvest levels was not addressed in our models.

 d. The efficacy of restoration of tributary and mainstem habitat for delisting

ESA species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

Our models are useful for considering spatially explicit trends that may result

through habitat and land use management on Federal lands. We do not believe it

is possible to predict the absolute change in habitat capacity or quality, or the

absolute change in abundance or productivity of the fish stocks associated with

these lands at this scale of analysis.

Our models focused on the effects of Forest Service/BLM land management and

restoration of spawning and initial rearing habitat for six salmonids fishes in the

Columbia Basin east of the Cascade Range and outside of federal lands managed

under the Northwest Forest Plan. Although we considered all public and private

lands within the ICBEMP management area to provide a larger context, the

alternatives and their outcomes varied only on FS/BLM lands. The relative effects

of proposed federal habitat conservation and restoration on listed and unlisted

species were also influenced by the species’ biological potential (e.g., current

status, threat of exotics, refounding and support of adjacent populations) and

mainstem migrant survival for anadromous species. 

Although the outcomes varied by alternative, species, and strata, all three SDEIS

alternatives projected positive long-term trends relative to current conditions in

aquatic habitat and to a lesser degree, in species status. All of the alternatives

were projected to maintain core areas, where populations are currently strong and

more widely distributed. We projected some rebuilding of depressed populations,

especially in watersheds identified for restoration emphasis. Although habitat
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 68

conditions were expected to improve under each alternative, there was more

uncertainty associated with the responses of many populations due to constraints

of non-habitat factors (e.g. isolation, exotic species, migrant survival). Some

declines in habitat conditions were projected in the short term under alternative 3.

3. What kinds of information or data are needed to run your model?

Predictions or coverages of the following for each 6th code HUC on Federal Lands

within the Interior Columbia River Basin analysis area:

• Road density

• Slope class

• Proportion of the area influenced directly by ground disturbing activities

(e.g. logging, skidding, prescribed fire) per decade

• An interpretation of the level of habitat mitigation and restoration

described or implied in the management alternatives

• Grazing intensity

• Past management history (e.g. wilderness, intensively or lightly managed

forest)

• Probability of large wildfire

• Occurrence of introduced fishes

• Status of existing populations of the particular species modeled

• Where current species status is ‘absent’, a prediction of potential for

spawning and rearing

• Status of populations in surrounding 6th code HUCs

• Connectivity among 6th codes HUCS within the encompassing 4th code

HUC

• Number of dams that must be passed by anadromous fish

4. What are the strengths and weaknesses of your model?

Past attempts to evaluate the alternatives of the previous ICBEMP DEIS and

similar large scale management proposals, such as FEMAT, focused on arraying

available information (e.g., management direction, projected levels of activity,

knowledge of fish or habitat distribution and status) and then asking one or more

experts to formulate opinions about the likely future status or trend of habitat and

populations across the affected range. This was an unsatisfying and contentious

approach for several reasons. The interpretations represented a synthesis that was

subjective and not logically explicit. It was difficult to conceptualize and account
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 69

for the complexity of the management direction and the multiple interacting

physical and biological effects. Although assumptions could be stated, it was hard

to determine how those assumptions may have influenced the outcome or

uncertainty in the outcome. Because the analysis was based on individual

interpretations of all of the data, it was difficult to replicate or update the analysis

when the management alternative or key assumptions were modified. The results

were not easily quantified and were not spatially explicit. Management and its

effects were generalized across a large heterogeneous landscape. It was difficult

to evaluate the relative differences among alternatives and the uncertainty in any

predicted outcome.

We believe our approach provided several distinct advantages. Complex physical

and biological interactions and management alternatives could be

compartmentalized into simpler, more comprehensible components. By

formalizing our understanding and assumptions, we provided a framework for

exploration of differences in the management alternatives that is quantifiable,

spatially explicit, and flexible. The networks can incorporate both quantitative

and qualitative information, which is essential to any analysis attempted at this

scale. Our assumptions and model relationships can be tested, and the analysis

repeated in a consistent, expedient manner. Critical uncertainties for management

can be determined, and the structure of the networks can be changed to explore

the sensitivity of results to those uncertainties. 

Despite these benefits these networks have important limitations. Our analysis is

limited by the subjective nature of the information available to us, by the quality

or accuracy of the data that do exist, and by our understanding of and ability to

describe key linkages in aquatic ecosystems at this scale. In many cases, the

landscape data represent only surrogates or correlates of key elements or

processes. Often the interpretation of any process was based on models requiring

extrapolation across spatial and temporal scales. We view the networks as very

simple abstractions that represent our attempt to capture the way we think about

systems and the relative importance of the different processes influencing fish.

They are a relatively simple means of replicating a professional interpretation of

conditions and effects in a consistent way across a very complex landscape. They

may exaggerate some patterns and under represent others. Uncertainty is explicit

in the use of conditional probabilities. This uncertainty reflects the limitations of

understanding and information but also means that trends and differences can be

obscured. Because much of the information represented in the networks is

subjective and represents the strength of the belief in those relationships, the

results can be viewed only as relative trends among alternatives and not as

predictions of the absolute number or true probabilities of high quality watersheds

or extant populations. 
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 70

5. What are the assumptions of your model?

Our key general assumptions are listed below. Other assumptions related to

interpretation of specific elements of the alternatives or the model are described

in other analysis documents. There are many assumptions inherent in the

development of the model structure and the parameter estimates (i.e., conditional

probability tables [cpts] ) that are not captured in this list. In some cases,

particularly where we used expert judgment to develop the cpts, those

assumptions are captured in the model but are not described in the model

documentation.

We assumed that the interpretation of the SDEIS and resulting predictions of

landscape characteristics and disturbance provided by the landscape team is

accurate in value and spatial representation. We similarly assume that information

derived from the ICBEMP Scientific Assessment (e.g., Quigley and Arbelbide

1997) and available in existing basin coverages is accurate. We know that errors

exist in the data and in some cases the relative magnitude of the error is not

quantified. However, we assumed that those errors do not meaningfully

compromise the results of the analysis.

We assumed that decreases in road density reflect the actual removal of roads and

most of their related adverse effects on the landscape. It is recognized that

removal may not include re-contouring if inappropriate, but does include revegetating and no vehicular use and the restoration of hydrologic function.

Our definition of aquatic habitat capacity implies that a range in habitat conditions

is possible even without human disturbance and that habitat will vary through

time in response to natural disturbance and vegetation succession. We do not

assume that optimum conditions always will exist in the absence of human

activity. We did assume that a subwatershed (HUC 6) in which sediment input,

riparian habitat, and hydrologic regime have not been substantially altered by

human activity will be more likely to contain aquatic habitat conditions that are

closer to optimum for indigenous salmonid species than a subwatershed where

one or more of these components have been considerably altered by human

activity.

We assumed that aquatic restoration needs are apparent and that the habitat

restoration methods to be used will be effective. 

We assumed that the each of the alternatives would be fully implemented to

achieve the objectives and meet the standards as described.

We assumed that field units will be staffed with adequate aquatic expertise to

effectively implement analysis, conservation, and restoration direction.

We assumed that there is greater uncertainty regarding the effectiveness of

restoration compared to conserving existing high quality aquatic habitat.

Some point source impacts, such as mining, could adversely affect aquatic habitat

condition under all alternatives, but there were inadequate data to determine their

potential effects in a spatial context.
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 71

We assumed that interim FS direction for roadless areas will be replaced by

comparable long-term direction for FS lands. That direction primarily restricts

road building, but does not change the management allocation. For example,

timber harvest and other uses could still occur but would be limited to access not

dependent on new roads. It was assumed the existing minimal level of road

construction on BLM lands will continue.

Even though the belief networks employ probabilities, we do not assume that they

are accurate estimates of true probabilities or "risks." Rather, they represent the

strength of our belief in the status or trends for particular elements of the system. 

We assumed that the differences among experts that estimated conditional

probabilities in our networks represented uncertainty in outcomes resulting from

inherent variability in the system and uncertainty in our understanding of nature.

Differences or confusion in the interpretation of or the definition of states of

nature reflected the limitations of our understanding.

It is known that the predicted results of management activities and alternatives at

10 and 100 years will continue to be influenced by succession and evolution of

the system into the future. For the sake of the evaluation, however, we assumed

that the biological response was an immediate result of the landscape conditions

at the point of evaluation (i.e., 0, 10, 100 years).

Where 6th code HUCS (subwatersheds) were not true watersheds but composites

of a high order mainstem reach and low order tributaries we assumed that habitat

conditions for fish in the subwatershed being evaluated were represented by the

mean of conditions in contributing subwatersheds. 

We assumed that the effects of federal land management activities on salmonid

fishes will be most influential and measurable in spawning and rearing habitats.

Our analysis was limited to subwatersheds classified as existing or potential

spawning and rearing habitat. We assumed then that Federal land management

does not significantly influence the condition of populations outside these areas.

We assumed that the status and trends of salmonids and their habitats are suitable

indicators of the conditions in aquatic ecosystems in general related to federal

land management.

We assumed that climate change would not influence the relative differences

among the alternatives.

We assumed that factors influencing the condition of habitats for fishes that are

not explicitly represented in the models are incorporated as additional uncertainty

about the likely future status.

For subwatersheds with multiple ownership or management direction, we

assumed that the net effect of the mix of prescriptions across the different

management areas was the area weighted average of the probabilities associated

with each.
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 72

We assumed that the uncertainty associated with each estimate of subjective

probability in the cpts is trivial compared to the overall uncertainty in the model,

and hence treat each of these probabilities as point estimates. Further, we did not

weight the information provided by individual experts and, therefore, assumed

that each had the same level of relevant knowledge.

We assumed that planning and analysis at levels of organization below the

Interior Columbia Basin (e.g. subbasin review and watershed analysis or EAWS)

would be necessary to effectively implement the direction in the SDEIS. We

assumed that these analyses where required would be effective and resulted in the

proper citing of conservation and restoration measures. Where these analyses

were not required we assumed greater uncertainty, and less success in producing

the intended outcome.

6. How does your model address uncertainty?

The uncertainty surrounding future outcomes was explicitly incorporated in our

model through the use of probability. We modeled the relationships among states

using conditional probabilities in the form of subjective (belief) and empirical

probability estimates. All model inputs were probabilistic with the exception of

average subwatershed slope, which was deterministic (i.e., known). The model

outputs were expressed as the probability that future habitat capacity and

salmonid population status were in each of 3 states.

7. All models make predictions. Why do you think your model’s predictions are

accurate?

We don’t think our predictions are accurate in the absolute sense. That is we

don’t believe the probabilities we estimate for a particular state in habitat

condition or species status reflect true probabilities. As stated in #4, our models

relied, to a great extent, on the subjective belief of experts due to the complex

nature and lack of empirical data and models for estimating the effects of land

management at large scales and over long time frames. Thus, the models should

be considered as explicit formalized thought-processes rather than empirically

based predictive models. Because we sampled a wide array of experts in the

physical and biological sciences, we believe that the models are “accurate” in that

they faithfully reflect the collective beliefs of experts as to the influence of land

management on aquatic habitats and salmonids. Human judgments of subjective

probability, however, are often inconsistent with probability laws and definitions.

In contrast, the empirically derived components of our model were relatively

accurate, with an average cross-validation error rate of 21%, across species and

components.
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 73

8. How does your modeling effort relate to or contrast with the other three modeling

efforts?

It appears to us that each effort had very different objectives.

Our models were designed to analyze the relative effects of the SDEIS

alternatives on freshwater salmonid habitat capacity, particularly on federal lands,

and future population status of both resident and anadromous salmonids

influenced by FS/BLM management. They were not designed to assess the risks

of all of the factors affecting salmonids in the Columbia basin throughout their

life cycles and various life histories or the risks associated with the Snake River

dams. PATH and EDT are large, complex models, whereas the effects of

mainstem dams in our models are reduced to a single variable and associated set

of probabilities generated by an expert panel. In essence that component of the

system that other groups have tried to analyze through very complicated

approaches was distilled to some very broad assumptions in our analysis. Our

panel assumed that dams have an important influence on the survival of salmon

and steelhead that can in some cases outweigh the effects of habitat--our models

did not tell us that. Based on some of the CRI results, the assumption used in our

model may have been too conservative. Based on other analyses they may have

been appropriate. Either may be right. It was not our intent to enter that debate

or to attempt to resolve the uncertainty. Because of the uncertainty, we

conducted two sets of analyses with a range of assumptions regarding the

influence of the Snake River dams. Regardless of the assumptions used, the

trends among the alternatives we were asked to evaluate remain the same.

Although differences in those assumptions make a large difference in the absolute

number of salmonid populations predicted to be present through our models, they

have no effect on the relative differences among the alternatives. We emphasize

that the absolute numbers are not particularly meaningful in interpreting our

results. Furthermore, any differences or similarities among our models and other

salmonid models for the Columbia in terms of how migrant survival is modeled

have no effect on the projected outcomes of the alternatives on freshwater habitat

capacity of anadromous species or other fishes on FS/BLM lands, where the

alternatives are most directly influential. The original aquatic science assessment

(Lee et al. 1997), subsequent published papers, and the SDEIS evaluation have all

stressed the importance of maintaining and restoring freshwater habitat to the

persistence of anadromous species, regardless of the relative impact of migrant

survival or other Hs. 

9. What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River Basin? 

As stated in #1, our models were designed to evaluate the relative merit of 3

proposed management alternatives for FS/BLM lands in the Basin by examining

their (relative) potential effects on the habitat of salmon and other fishes on those

lands. That is, our models were specifically created to help decision-makers select

a management plan for FS/BLM lands. The consequences of those activities on

salmon recovery, although an important consideration, will depend on many

factors outside of the scope of our models and the decisions made concerning
ISAB 2001-1: Model Synthesis Report

Appendix III: ICBEMP - 74

management of FS/BLM lands. Our models should be used in concert with other

models and information to weigh the merits of possible decisions affecting

management of other public and private habitat, hydropower and hatchery

operations, and harvest.

References Cited

Lee, Danny C.; Sedell, James R.; Rieman, Bruce E. [and others]. 1997. Chapter 4.

Broadscale assessment of aquatic species and habitats. In: Quigley, Thomas M.;

Arbelbide, Sylvia J., tech eds. 1997. An assessment of ecosystem components in

the interior Columbia basin and portions of the Klamath and Great Basins:

volume 3. Gen. Tech. Rep. PNW-GTR-405. Portland, OR: U.S. Department of

Agriculture, Forest Service, Pacific Northwest Research Station. 4 vol: 1057-

1713. (Quigley, Thomas M.; tech. ed.; The Interior Columbia Basin Ecosystem

Management Project: Scientific Assessment).

Quigley, Thomas M.; Arbelbide, Sylvia J., tech. eds. 1997. An assessment of ecosystem

components in the interior Columbia basin and portions of the Klamath and Great

Basins. Gen. Tech. Rep. PNW-GTR-405. Portland, OR: U.S. Department of

Agriculture, Forest Service, Pacific Northwest Research Station. 4 vol (Quigley,

Thomas M., tech. ed.; The Interior Columbia Basin Ecosystem Management

Project: Scientific Assessment).
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 75

Cumulative Risk Initiative Answers

Submitted by Peter Kareiva and Chris Jordan of CRI

NWFSC, NMFS

PROLOGUE

At the risk of repeating ourselves again, we would like to emphasize that the CRI is not

just a modeling effort. While we realize that all science and all analyses are models, the

CRI is best described as an organized chain of logic and analyses that for some steps

relies on models. One distinguishing feature of the CRI is that it stops short of using

numbers or calculations to make certain “leaps”; the CRI is unwilling to make

quantitative leaps across data gaps that are too wide and too deep.

In a previous review of the CRI, the ISAB argued for moving towards a formal decision

theoretic approach, and at the time of that review CRI researchers would probably have

agreed with the ISAB. However, the CRI research team would now disagree for five

intertwined reasons:

1. As we have become increasingly familiar with the data available for Columbia River

salmon populations, we have concluded that the data are generally so poor that many of

the pertinent management questions simply cannot be dealt with quantitatively (see the

Executive Summary of McClure et al 2000).4

2. The core of the existing decision theoretic approach regarding Snake River dam

breaching is an upstream/downstream comparison. However, there is clear statistical

evidence that the different ESUs (upriver versus downriver chinook) respond nonuniformly to the same ocean variables (Levin, submitted), implying differences in time

series cannot be attributed solely to dams even after a common environmental factor is

accounted for.

3. Upcoming management decisions must concern the entire Columbia River Basin and

its twelve listed ESUs. Yet a decision framework has only been attempted for the Snake

River chinook salmon. It is impossible within even a one-year time period to imagine

constructing a decision theoretic approach for the entire Columbia River Basin. Far more

pressing is the need to synthesize and standardize data across ESUs and index stocks, and

develop comparative approaches. Fundamental questions such as how many hatchery

fish are spawning in the wild and what is their reproductive efficacy have far greater

urgency than a decision analysis framework. In fact, without first answering such

questions a decision analysis framework is likely to be unrewarding.

 

4 Included in our responses to the ISAB’s Nive Questions are references to particular sections of a recent CRI analysis of

eleven salmonid ESUs in the Columbia River Basin. The document, A Standardized Quantitative Analysis of the Risks

Faced by Salmonids in the Columbia River Basin (referred to as McClure et al. 2000) is too lengthy to be read in its

entirety as part of the ISAB project, but is included as part of our response to illustrate the scale and scope of the CRI

effort. In addition, one can ground-truth our answers provided here by examining the relevant pages of this most recent

CRI product. This document is available from the CRI website.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 76

4. The emphasis on decision-theoretic approaches diminishes the emphasis on data. This

is not a logical necessity, but it is appears to be a “tendency”. Because modeling often

gives the illusion of an “answer”, there is reduced pressure to acquire or update data. For

example, until recently run-reconstruction data for Snake River Chinook beyond brood

year 1990 had not been updated, even though the data were available to update through

brood year 1994. Ironically these most basic of all population data were four years

behind the times while the region debated population-modeling strategy. The actual

population numbers from recent years had become less important in the scientific

discussion than modeling predictions about future population behaviors.

5. In its best manifestations, decision theory can provide guidance to policy-makers.

However, policy-makers may not be able to evaluate the “firmness” or “credibility” of

these answers. The CRI feels that the numbers from decision theory would represent an

illusion of science when there is really very little science to support particular choices.

Given the data, the CRI concludes that only bold management experiments can give

scientific answers to the key questions. Of course this conclusion itself is the outcome of

a decision process, but it is a matter of priorities and pragmatism as opposed to academic

acumen. Given the many analyses that need to be done, a decision theoretic calculation

to provide a “number” that says “we do not know enough to know what to do, so do bold

management experiments”, seems like a poor investment of scientific effort. Instead, the

CRI will be emphasizing the design of monitoring programs, and exploring ways to

estimate observation error because sampling error has been so neglected in the whole

debate.

In summary, asking for a formal decision theoretic analysis at this stage is like asking a

baby to walk before it can crawl. Such an analysis is a good idea, but there are many

more pressing needs to attend to before any such analysis is attempted. PATH made a

valiant effort at a decision theoretic component, and did a great service in synthesizing

data, and exploring quantitative hypotheses about fish passage. But PATH floundered

when it came to evaluating the merits of dam breaching relative to other management

options. One main reason PATH has faced such a challenging task is that without being

able to attribute upstream/downstream differences to dams, there simply may not be

adequate data to contrast dam breaching to other management options within the same

analytical framework.

In addition to the absence of the most basic data, a second problem with the Columbia

River Basin salmon science arena is that while we can find numerous review chapters and

perspectives papers, original analyses of actual data are remarkably scarce. Where are

the published analyses linking habitat attributes to recruits per spawners? Where are the

published analyses describing population trajectories in any quantitative manner? Where

are the published analyses quantitatively examining hatchery impacts on wild stocks, as

best as can be done? We suspect that the ISAB’s recommendation of establishing a

“decision theoretic framework” represents the same frustration we feel about an absence

of analyses. Where we perhaps differ from the ISAB is in believing that we need to do a

lot more basic data analyses before moving forward to a decision theory approach. To
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 77

take advantage of ISAB’s expertise, we have appended an EXCEL data file of time series

of spawner counts. We encourage any or all ISAB members to try out their ideas by

analyzing these time series using likelihood ratio or information theoretic methods, etc.

Note that for many of these time series, we have no information on hatchery fish, or only

crude guesses. In none of the time series has there been a direct estimate of the

reproductive fitness of hatchery fish that spawn naturally.

Perhaps the shortage of clear data analyses exists because many of the analyses got

bundled into large modeling efforts. Unfortunately, most of these analyses remain

unpublished, poorly documented, and in our opinion impenetrable. The CRI is trying to

initiate a culture of publishing in refereed journals, and publishing data papers, not just

perspective or broad overview papers. Four noteworthy papers have been submitted in

the last few months from the NWFSC efforts (not all are CRI papers, but they are

related). We encourage the ISAB to contact the authors of those papers (their e-mail

addresses are given below), and ask permission to review the work. We believe these

papers are beginning to add some clarity that was previously lacking:

John.G.Williams@noaa.gov: updated downstream survival estimates and SARs for

Snake River Chinook and Steelhead (using PIT tag data). Submitted to CJFAS.

Rich.Zabel@noaa.gov: a discussion of the temporal pattern of SARs in the Snake

River and comparison of stocks in which the hydropower hypothesis of “extra

mortality” is challenged. In press in CJFAS

Phil.Levin@noaa.gov: a critique of the upstream/downstream comparison, plus some

statistical analyses indicating that different Columbia River Basin ESUs (including

upriver versus downriver ESUs) respond differently to common ocean environmental

variables

Jim Regetz, jregetz@princeton.edu: analysis of spatial variation in recruits per

spawner for 22 chinook index stocks and the percentage of that variation explained by

simple habitat variables. Submitted to Conservation Biology.

Michelle Marvier, mmarvier@scu.edu: numerical experiments with stage-structured

matrices applied to Snake River Chinook salmon as a means of examining

management options. Soon to be submitted to Science.

Phil.Levin@noaa.gov: an examination of competition between hatchery and wild fish

as measured by ocean survival. Manuscript proceeding through the three levels of

internal NWFSC review.

If more analyses had been submitted to journals three or four years ago, the

documentation of the relevant science would be an order of magnitude better in the

region – as it is, the lack of clear and well-written documentation is a major limiting

factor in all discussions.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 78

ANSWERS TO THE NINE ISAB QUESTIONS

1. What is the purpose of your modeling effort? What questions or problems were your

models designed to address?

The CRI analysis is intended to synthesize data regarding the trends, dynamics

and risks of threatened and endangered salmonids on the West Coast of North

America. This scope (25 ESUs, ranging from California northward to the Puget

Sound and the Canadian border) is considerably broader than even the Columbia

River Basin. At this scale, the CRI will provide a standardized, yet flexible,

framework within which recovery teams on the west coast can:

a. consider the status of their ESUs of concern;

b. examine the likely effects of recovery actions;

c. help design monitoring and evaluation strategies for determining how well actions are

working and whether the ESUs as a whole are moving towards recovery.

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries.

The CRI analyses suggest that data and data analyses to address the merits of actions in

the 4-H’s are generally lacking, with the exception of harvest (in which case the analysis

is straightforward since a harvested fish is unambiguously a dead fish). Nonetheless, the

CRI does offer some analyses that are germane to a discussion of these four-H’s.

Harvest: At the ESU level, it is possible to calculate the percentage increase in lambda5

expected to result from particular reductions in harvest, using harvest rates from the time

period during which lambda itself was estimated (up until brood year 1994). Performing

this calculation makes it clear that by reducing harvest, the lambdas of three ESUs (Fall

Chinook in the Snake River, Upper Willamette Chinook, Lower Columbia Chinook)

could be sufficiently increased to well above 1.0, and the populations would be growing.

Conversely, it is also evident that harvest reductions by themselves could not remedy the

low lambdas in the remaining ESUs.

Hatcheries: Because basic data needed to assess the impacts of hatchery fish, including

reproductive output of wild spawning hatchery fish, are lacking, it is impossible to obtain

a satisfying answer to many questions regarding the impact of hatchery fish. Outside the

Snake River Basin, we often lack even estimates for the relative frequency of hatchery

fish on spawning grounds. As discussed in McClure et al. (2000), there are some data

supporting a hypothesis that hatchery fish compete with wild fish in the ocean and under

poor conditions can reduce the survival of wild fish (Levin, in review). If these analyses

 

5 The CRI measures population performance with “lambda”, the annual rate of population change estimated in a manner

that includes environmental variability, and as much as possible, accounts for sampling error. A lambda greater than one

results in an increasing population, while lambda < 1.0 implies a shrinking population.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 79

are on target, then it is possible that substantial improvements may be gained by

alterations of hatchery policy. However, for many ESU’s, due to the presence of

hatchery fish in counts of natural spawners, it is difficult to assess the most fundamental

demographic processes of populations.

Habitat: The CRI firmly believes that habitat restoration offers substantial opportunities

for population recovery. A preliminary analysis of spatial variation in recruits per

spawner data at the sub-watershed level revealed that a few simple habitat variables

explained over 60% of the variation among chinook stocks in their productivity (Regetz,

in review). The same 22 stocks did not reveal any such relationship with respect to

simple hydropower variables such as the number of dams between the spawning ground

and the ocean (which varied from 1 to 9). This is not evidence that dams and hydropower

are unimportant, but rather puts the success of the habitat regression model in the context

of other explanatory variables. A second analysis, better connected to specific parcels of

land within ESUs, has been provided by the Watershed Processes Program of the

NWFSC (section V.F.2 of McClure et al 2000). In this analysis salmon productivity was

found to be related to good quality riparian zones in non-forested areas, low water

temperatures, and low gradients.

Hydropower: Hydropower operations are usually discussed with respect to Snake River

Spring/Summer Chinook salmon, although, given the widespread presence of dams

throughout the Columbia Basin, they clearly impact all of the ESUs (see section V. of

McClure et al. 2000). Clearly the hydropower system can alter mortality. However,

rather than searching for extra mortality that must be attributed to some external cause or

disease, it would be more parsimonious to ask the following questions:

By how much is dam breaching likely to increase downstream survival, upstream survival

and estuarine/ocean survival? What is the evidence for such increases and what are

their consequences in terms of improved lambdas?

There are good data to show that downstream survival may actually decrease following

dam breaching because the in-river survival already comes close to matching pre-dam

conditions (Williams et al 2000), and because transport survival is very high. Upstream

survival is likely to increase by 15-30% with breaching, depending on estimates and

interpretation of current conversion rates. The big unknown, of course, is the expected

increase in ocean/estuarine survival due to removal of the four dams. Since the CRI

concludes this is such an important uncertainty, it ran analyses of potential increases in

survival that ranged from 20% to 180% in increments of 20% (see figure VI-5 of

McClure et al.). It appears that dam breaching would need to increase early

ocean/estuarine survival by 100% or more in order to increase lambda enough that

extinction risk were mitigated. It is worth noting that an increase of 100% corresponds

to fish that are one half as fit because of their travel through dams – a magnitude of

depressed fitness that ought to be detectable on an individual fish basis. As a result of

these analyses the CRI suggests that a future emphasis on fitness measures of individual

fish (especially in conjunction with PIT tag data) may provide a more tractable indicator
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 80

of whether dam breaching could be expected to recover spring/summer chinook salmon

in the Snake River.

3. What kinds of information or data are needed to run your model?

All CRI analyses start with time series of spawner counts to estimate recruits per

spawners or annual rates of population growth. In the few data rich cases the CRI

approach is to develop demographic projection matrices. Some estimate of the age

structure of returning fish is needed, but the extinction modeling and estimation of

lambda are robust to large and erratic errors in this age estimation. Hence the most

critical data are spawner counts. It is worth noting that even this most basic information

is often lacking, or is compromised by the presence of hatchery fish (see section V. of

McClure et al. 2000).

A second piece of critical information the CRI would like to have, or be able to estimate,

is observation error. The CRI will make a concerted effort over the next several months

to provide bounds to the magnitude of observation errors in spawner counts and will then

explore the implications of this uncertainty for monitoring programs (see discussion of

“detectability in section V.C of McClure et al, 2000).

4. What are the strengths and weaknesses of your model?

Weaknesses:

a.) The CRI does not provide answers in the form that policy makers seem to desire.

Policy makers want simple numbers – how many fish will we get if we do X? It is the

CRI’s contention that given the quality of existing data any such numbers would be

largely be “guesses” pulled out of thin air.

b.) The CRI is not a formal decision theoretic approach.

c.) Since the CRI focuses on data from 1980 onward, it does not cover an entire Pacific

Decadal Oscillation and hence does not incorporate ocean cycles or trends in ocean

conditions. Because recent years have reflected the poorest ocean conditions of the

modern record, this weakness means that the CRI is likely to overestimate risks if one

assumes that ocean conditions will soon improve. However, we feel that this overestimation is a cautious approach.

Strengths:

a.) The CRI is simple, and hence other scientists can repeat its analyses, as evidenced by

the fact that others have adopted the matrix approach and conducted their own estimates

of lambda.

b.) The CRI emphasizes documentation of data and data sources, and places a greater

premium on presenting basic data and hypothesis testing using data prior to modeling.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 81

c.) The CRI is attempting to deal with observation error by designing estimation

procedures that are robust despite its presence, and by focusing on obtaining direct

estimates of its magnitude. It is unclear that any analytical framework can be reliable if it

does not address sampling error in some fashion.

d.) The CRI is extremely responsive to new and/or updated data.

e.) The CRI places emphasis on transparency and presentation so that others may

understand its methods. One manifestation of this is the CRI’s emphasis on publishing in

peer-reviewed journals.

5. What are the assumptions of your model?

Because the CRI is such a simple quantitative analysis there are not many assumptions.

However two critical simplifications that the CRI has made to date need to be mentioned.

a. The first simplification is the assumption of a lack of density-dependence in current

dynamics. Previous models applied to salmon in the region have assumed a Ricker form

of density-dependence. Our analyses indicate that the statistical evidence to support such

density-dependence is absent (the density-dependence we occasionally do detect takes the

form of density-depensation, or declining productivity as abundance declines).

b. The second critical assumption has been to treat variability in recruits per spawner as

lognormal variation from a temporally homogeneous stochastic process (i.e., it assumes

there are no directional trends underlying the variation).

Both assumptions, and especially how one tests them with time series of data, are interrelated. In McClure et al (2000), the CRI reports results from likelihood ratio tests for

density dependence in the time series of 69 stocks, and finds strict density-dependence in

only eight cases. In all of these cases there is the suggestion of declining recruits per

spawner with declining numbers, or density-depensation (which is not the form of

density-dependence used in the Ricker formulation). Unfortunately, it is very hard to

distinguish between a hypothesis of density-depensation and a temporal trend. The CRI

is developing methods based on information theory to tease apart these two mechanisms.

6. How is uncertainty addressed?

Uncertainty is dealt with in several ways. First confidence intervals for estimates of

annual population growth are calculated. Second, when we have almost no data, as is the

case for the reproductive fitness of hatchery fish, we estimate lambdas using an upper and

lower bound for hatchery fish fitness (i.e., we assume the fitness of hatchery fish is zero

at its lowest or equal to that of wild fish at its highest). In other cases, we perform

numerical calculations over a wide range of potential values. For example, we calculate

the expected increase in lambda for Snake River spring/summer chinook salmon over a

broad range of potential improvements in estuarine/early ocean survival due to dam
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 82

breaching (ranging from 0% to 180%, in 20% increments; see Figure VI-5). A third way

of addressing uncertainty is to use multiple methods to calculate key parameters. For

example, we calculate sensitivity measures (change in lambda associated with a 10%

reduction in mortality) off baseline matrices that are derived in several different ways

e.g., using different time horizons (beginning in either 1980 or 1990) or using different

methods to derive matrix parameters (using SAR’s to back-calculate estuarine survival,

or starting with a literature value of 7% for estuarine survival). Lastly, we encourage

actual plots of data, such as mean recruits per spawner as a function of subwatershed

habitat (see figure extracted from Regetz manuscript in section V of McClure et al 2000).

7. Why do we think our model’s predictions are accurate?

We do not always think the inferences from our analyses are accurate – it all depends on

which inferences? (or which predictions?).

Examples of “accurate” predictions are: the conclusion that even if downstream survival

were increased to 100%, spring/summer chinook lambdas remain substantially lower than

1.0 and the populations are still at severe risk. After the fact, this is an obvious point.

But as a numerical result it puts much of the focus on the details of downstream passage

in perspective (suggesting that the emphasis of passage research now should turn to

consequences for survival below Bonneville dam). Another “accurate” inference is the

relative opportunity for improving lambda by acting on first year survival and

estuarine/early ocean survival relative to survival in other lifestages. At the ESU-level

across the entire Columbia Basin, our prediction that major harvest reductions could

achieve the needed improvements in lambda for Upper Willamette Chinook, Lower

Columbia Chinook, and the Snake River fall chinook is likely accurate.

Other predictions or estimates are only moderately accurate. For example our estimates

of lambda (annual population growth rates) are probably the best available summary of

the current trends in populations, subject to the uncertainty about how to adjust the

estimates for the presence of hatchery fish on spawning grounds.

Finally, there are aspects of our analyses whose accuracy needs further study. The best

example concerns “probabilities of extinction”. We do not require that probabilities of

extinction are accurate, but we do require that their relative ranking of risks among stocks

or ESUs is accurate. We have launched several analyses to determine how well relative

rankings of extinction risk stand up in the face of sampling error and fundamental

violations of the underlying model.

Lastly, speaking about models strictly in terms of “accuracy” does not do modeling

justice. We can learn from models quite independent of accuracy. A good example from

recent CRI analyses concerns what we have learned about the need to better resolve

hatchery inputs on wild spawner grounds. We have learned that it is foolish to take

seriously absolute predictions of extinction risk (or population growth) in the absence of

rigorous samplings of marked hatchery fish on spawning grounds and empirical

evaluations of the reproductive rate of those hatchery fish relative to wild fish. Without
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 83

modeling, this result is evident to some – but the exercise of estimating lambda under

different assumptions about the hatchery contribution makes the importance of this

uncertainty dramatically evident. This is NOT an uncertainty that other analytical

frameworks have drawn attention to.

8. How does your modeling effort relate to or contrast with the other three modeling

efforts?

The distinguishing features of the CRI are:

a. The CRI is the analytical approach that sticks closest to the actual data.

b. Unlike the other analytical approaches, it is possible to readily grasp the analyses

undertaken by the CRI, and redo the calculations under slightly modified assumptions.

c. The CRI examines eleven different ESUs in the Columbia Basin – none of the other

models tackle this biological and geographic scope.

d. The CRI focuses first and foremost on annual rate of population change (adjusted for

environmental variability), and the risk of extinction.

PATH is the only other analytical approach that starts with spawner counts as the key

input data and uses transient population dynamics as a key response variable. But unlike

CRI, PATH has produced analyses for only two of the Columbia Basin ESUs and tends

to focus on hydropower. EDT is a broader analysis than is CRI in terms of its willingness

to evaluate alternative management scenarios. However, EDT is primarily an expert

system. Although useful for generating hypotheses and summarizing spatially explicit

habitat descriptors, EDT does not incorporate much about fish population dynamics. To

see this, simply recognize the fact that the most recent four years of Snake River

Spring/Summer Chinook recruits per spawner data have no influence on EDT’s answers.

We do not know enough about ICBEMP to comment on it, except to mention that it

seems to be primarily a habitat-focused analysis. It is not clear that ICBEMP’s

framework could ever allow an evaluation of harvest or hatchery impacts.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 84

9. What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River basin?

Below is an excerpt from the EXECUTIVE SUMMARY of McClure et al:

Generally, ESUs or stocks with the most rapid rates of decline – the lowest lambdas --

require the most improvement to mitigate extinction risks. However, this generality is

complicated by the fact that low populations and high environmental variability can

exacerbate extinction risks beyond what might be expected from lambda alone. The

magnitude of improvements required in lambda ranged from less than 1% to as much as

65%, with most values falling between 5% and 20%.

The more difficult task is exploring opportunities for improving lambda, i.e., increasing

the number of recruits per spawner. The well-known “four H’s” (hydropower, habitat,

hatcheries and harvest) represent the human-influenced arenas in which management

can be altered in hopes of recovering ESUs. But because these four H’s vary enormously

in the areas occupied by different ESUs, it is unlikely that a simple prescription can be

drawn up that fits all ESUs. For example, the number of dams per kilometer varies from

0.4 to 2.8 depending on the region associated with each ESU. Land use characterization

also varies widely across regions occupied by ESUs, with some regions characterized by

a high percentage of rangeland (Upper Columbia and Snake Rivers), urbanization (lower

Columbia, and upper Willamette Rivers), or cropland (upper Willamette River). At the

finer scale of index stocks, preliminary analyses indicate that three habitat variables at

the subwatershed scale explain 60% of the variation in recruits per spawner: (1) percent

of land classified as urban, (2) proportion of stream length failing to meet EPA waterquality standards, and (3) the ability of streams to recover from sediment flow events.

Lastly, although nearly 100 hatchery facilities in the Columbia Basin release

approximately 150 million smolts annually, the magnitude of this hatchery production

varies by an order of magnitude among ESUs. The impact of this hatchery production is

difficult to analyze because of the lack of large-scale controlled experiments. Some

preliminary analyses suggests that in “poor ocean years” hatchery fish compete with

wild fish and lower the survival rates of the wild fish.

Rates of population decline and extinction risks vary widely across the Columbia River

Basin, suggesting that management needs vary in accord with these different levels of

risk. Most imperiled are Upper Columbia Spring Chinook, Middle Columbia steelhead,

Upper Columbia steelhead, and Upper Willamette steelhead ESUs.

The amount of improvement in recruits per spawner that is required to mitigate risks can

be modest (less than 1%) or quite large (as high as 65%). When needed improvements

are modest there are probably management options, but when needed improvements are

large there is little room to be selective about what actions are taken. We must do

everything possible to increase recruits per spawners before it is too late. A lambda of

0.9 means that in less than 7 years a population is likely to be reduced to half its current

level.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 85

Reductions of harvest represent an easily identified mechanism for improving recruits

per spawner in a few ESUs (Lower Columbia Chinook, Upper Willamette Chinook, and

Snake River Fall Chinook). In other ESUs we lack data for making confident

quantitative predictions about the likely effects of any particular management action.

This is even the case for the much-studied Snake River Spring/Summer Chinook salmon

where risks are substantial and the need for action is striking (particularly if one factors

in the recent declining trend in recruits per spawner). Although there is some evidence

that dam breaching is necessary for mitigating the extinction risk faced by Snake River

Spring/Summer Chinook salmon (especially given the lack of evidence that needed

improvements can be made by non-breaching management actions), it is highly unlikely

that dam breaching alone will recover these populations. Hence, even in this moststudied of all cases, actions will be predicated on uncertainty. But what is not uncertain

is the substantial rates of decline for Snake River Spring/Summer Chinook salmon and

even worse rates of decline for several other ESUs.

In summary, the scientific uncertainty surrounding the likely outcome of everything but

harvest reductions is not an argument for inaction, especially given the high risks faced

by several ESUs. Quite the contrary. This level of uncertainty is, however, an observation

that the public and policy makers should be aware of. From a scientific viewpoint the

ideal action is rapid, targeted management action with effective monitoring programs.

Secondly, establishment of quantitative links between management actions and salmon

productivity are obviously a priority area for research. The region has suffered from an

inattention to standardized reporting of data and analyses at a large scale and as a result

currently lacks the scientific information required to make quantitative assessments of

management scenarios. It is imperative that this last point be emphasized to the public

and policy makers: collectively we have failed to manage Columbia River Basin salmonid

populations and are now forced to undertake management actions as experiments,

accepting that some will fail, but if they are properly designed, we can learn from our

mistakes.

In general, we feel that decision-makers might be well-served by drawing on all of the

available analytical tools, either in their existing or modified form, and not just on CRI.

First, it is worth noting that EDT and CRI are not “dueling” models; in fact, they attempt

to predict totally different response variables. CRI examines annual rates of population

growth and extinction risk, neither of which are EDT inputs or outputs. However EDT

does identify “good habitat conditions” versus “bad conditions”, and assesses responses

to changes. Both of these represent hypotheses that could be tested using the machinery

of CRI. In addition, the spatial data synthesized by EDT might be used in empirical

models aimed at testing relationships between habitat and salmonid productivity as

measured by demographic or spawner data. Second, although PATH and CRI are in one

sense at odds with one another because of contrasting modeling and scientific

philosophies, even these two approaches could be profitably combined, or minimally

learn from one another. In fact, already CRI and PATH are learning from one another (in

spite of what often seems like an acrimonious scientific debate). For example, CRI has

altered the way it parameterizes its Snake River matrices, and now uses PATH SAR
ISAB 2001-1: Model Synthesis Report

Appendix III: CRI - 86

estimates as the starting point with which to solve for first year survival. Conversely,

PATH has begun to use a simplified matrix formulation to explore its ideas about extra

mortality, and has added calculation of extinction probability to its repertoire of output

statistics. Clearly CRI and PATH will never “agree” in the sense of approaching

analyses in the same manner – however, if they can maintain a scientific dialogue, there

is some opportunity for each approach to be improved. The main obstacles to a

reconciliation of PATH and CRI are that (i) the two approaches operate on different

timescales (the CRI seeks results in months, whereas PATH seeks results in years), and

(ii) once one steps beyond Snake River chinook salmon, the detailed and complicated

models typical of PATH clearly cannot be supported. We do not know enough about

ICBEMP to discuss how its approach may or may not complement CRI.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 87

PATH Answers Part I - General and FLUSH

Paul Wilson, CBFWA

Question 1.

PATH grew out of previous efforts by operating agencies and state, tribal, and federal

fisheries agencies to compare and improve models used to evaluate management options

intended to lead to recovery of listed salmon and steelhead stocks. The Plan for

Analyzing and Testing Hypotheses (PATH) is a formal and rigorous program of

formulating and testing hypotheses. It is intended to identify, address and to reduce

uncertainties in the fundamental biological issues surrounding recovery of endangered

spring/summer chinook, fall chinook, steelhead and sockeye stocks in the Columbia

River Basin.

The objectives of PATH are to:

a) determine the overall level of support for key alternative hypotheses from existing

information, and propose other hypotheses and/or model improvements that are more

consistent with these data (retrospective analyses);

b) assess the ability to distinguish among competing hypotheses from future

information;

c) advise institutions on research, monitoring and adaptive management experiments

that would maximize learning; and

d) advise regulatory agencies on management actions to restore endangered salmon

stocks to self-sustaining levels of abundance (prospective and decision analyses).

Question 2.

PATH analyses have focussed on Snake River ESA species. Species outside of the Snake

R. basin were not addressed in detail in PATH until very recently, so conclusions

presented here apply to Snake River ESA species only.

Retrospective Analysis (Spring/Summer Chinook Salmon Only)

1. Declines in numbers of spawning salmon exhibited population-specific patterns.

2. Declines were greater for Snake River Basin and upper Columbia River Basin

populations than for lower Columbia River Basin populations.

3. Declines in Snake River Basin populations after 1974 were greater than declines

before 1974, and greater than declines in lower Columbia River Basin populations

after 1974.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 88

4. Differences in survival and productivity between Snake River Basin populations and

lower Columbia River Basin populations coincide in space and time with

development of the Hydropower System.

5. The significant decline in Snake River Basin populations after 1974 does not coincide

with habitat degradation. Degradation in habitat occurred mostly prior to 1974 and

spawning and rearing habitat for some of the Snake River Basin populations has

remained in good or pristine condition.

6. The degree to which artificial propagation contributed significantly to declines in

survival of listed Snake River Basin populations is uncertain.

7. Declines in survival of Snake River Basin populations occurred at the same time as

substantial declines in harvest rates. These trends are contrary to the assumption that

harvests are significant contributors to declines in survival of Snake River Basin

populations after 1974.

8. Different salmon populations appeared to respond differently to changes in climate.

However, no climatic index could explain differences between declines in survival of

Snake River Basin and lower Columbia River Basin populations.

9. All models suggest that mortality of Snake River Basin populations in the Columbia

River downstream from Bonneville Dam and in the ocean (extra mortality) is greater

for “transported” fish than for “in-river” fish that migrated past dams and through

reservoirs. The efficacy of transportation is expressed as the “D-value”, which is a

reflection of the ratio of extra mortality of “transported” fish to extra mortality of “inriver” fish. The retrospective D values have been estimated to be less than one (0.63

to 0.34).

Prospective Analysis and Risk Assessment

(Spring/Summer Chinook Salmon and Steelhead, and Sockeye Salmon)

1. The hydropower system options examined capture three general approaches for

operating and/or configuring the hydropower system in the future.

a. Three options use barges to move juvenile fish around dams and reservoirs. One

option (A1) describes the 1995 BIOP. The other two options (A2/A2’) describe

improved transportation under conservative and liberal assumptions about the extent

and effectiveness of improvements.

b. Three options are called “natural-river”. Two options (A3-3yr and A3-8yr) assume

Snake River dams are removed after 3 years and 8 years, respectively. The other

option (B1) also draws the level of John Day Reservoir down. Both A3 options were

explored for spring, summer and fall chinook. Two options (A6/A6’) improve

existing passage facilities at dams and use flow augmentation to speed migration.

These options are called “optimum in-river migration”.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 89

2. The “optimum in-river migration” options (A6/A6’) were not analyzed in the same

way as the “transportation” and “natural-river” options. Instead of using a formal

quantitative risk analysis, the “optimum in-river migration” options were analyzed

relative to the “transportation” and “natural-river” options. This was necessary

because time constraints prevented hydrologists from developing the “optimum inriver migration” options in sufficient detail for the quantitative analysis. Results

showed that listed salmon were no more likely to survive and recover under A6 than

A2.

3. Of 14 uncertainties examined, seven proved to most influence estimates of survival

and recovery under each hydropower system option. Of these seven, the most

influential assumptions were those included in the passage/transportation models and

those describing the mortality of salmon after they have passed through the

hydropower system. These assumptions most influenced the results because they

accounted for the largest proportion of the mortality salmon experience over their life

cycle.

4. For spring and summer chinook salmon, two of the “natural-river” options (A3-3yr

and B1) exceeded all of the standards. A3-8yr met the 100-year survival and 48-year

recovery standards, but missed the 24-year survival standard by less than 1

percentage point. This was also true for the weighted results. None of the

“transportation” options met either the 24-year survival standard or the 48-year

recovery standard under equal weights. This was also true for the weighted results.

5. For spring and summer chinook salmon, the “natural-river” options had higher

probabilities of achieving the survival and recovery standards than the

“transportation” options. “Natural-river” options met the standards over a wide range

of assumptions. In fact, the “natural-river” options met the 100-year survival and the

48-year recovery standards under the most pessimistic set of assumptions. “Naturalriver” options were also less risky than the “transportation” options. The projections

had relatively little variability over the full range of assumptions.

6. For steelhead, a hydropower system option is likely to meet survival and recovery

standards if it meets the standards for Snake River spring and summer chinook

salmon. However, our analyses did not address whether the survival and recovery

standards for steelhead would be met if a hydropower system option fails to result in

an acceptable likelihood of survival and recovery for spring and summer chinook

salmon.

For sockeye salmon, recovery is less likely than for Snake River spring and summer

chinook salmon under “transportation” options, if high rates of descaling, which appear

to be associated with bypass screens, are a primary source of injury and mortality. This is

because “transportation” options rely on bypass systems to collect juvenile sockeye

salmon. The likelihood of recovery of sockeye salmon under “natural-river” options,

relative to other species, was not analyzed.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 90

Fall Chinook

1. Models that include passage model inputs fit the historical time series of Snake River

spawner-recruit information much better than simple models that only consider

inherent stock productivity. However, models of intermediate complexity that do not

include passage model inputs provide better fits than most models with passage

model input.

2. The best fits to the spawner-recruit data are obtained when D is estimated from the

spawner-recruit information, or when D is fixed below a value of 0.2. Models that fix

D at values ≥ 0.2 have a poorer fit to the spawner-recruit data, and models that

assume D=1 have a much poorer fit. Estimates of D and spawning effectiveness are

related because hatchery production and transportation of fall chinook began at

around the same time in the historical period.

3. Models that fix hatchery spawner effectiveness at a high value (0.7 or 1.0) provide

better fits to the historical spawner-recruit data than models that estimate E. However,

models that fix hatchery spawner effectiveness at low values (0.0) provide worse fits

to these data than models that estimate E. Estimated E values are generally around 0.7

to 1.0, unless the D value is fixed at a high value.

4. Including a common ocean effect that affects several Columbia River fall chinook

stocks results in a strongly better fit over models that do not include this effect. The

best estimates of these common effects are based on data from the Snake River and

Deschutes stocks.

5. All hydrosystem actions project an improvement in survival rates and in spawner

abundance because of assumptions about system operations built into the

management scenarios, such as flow augmentation at levels prescribed by the 1995

Biological Opinion.

6. Projected outcomes of actions depend strongly on what is assumed about D (the

estuary/ocean survival rate of transported fish, relative to the estuary/ocean survival

of non-transported fish).

7. All hydrosystem actions meet survival standards (probabilities of exceeding survival

escapement thresholds are greater than 0.7), regardless of what is assumed about the

estuary/ocean survival rate of transported fish.

8. All drawdown actions meet recovery standards (probabilities of exceeding recovery

escapement thresholds are greater than 0.5), regardless of what is assumed about the

estuary/ocean survival rate of transported fish. The drawdown actions (A3, B1)

exhibited the most robust response across those uncertainties considered to date, and

produced higher recovery probabilities (as well as higher average spawning

escapements) than other actions.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 91

9. There are non-breaching actions (actions that do not involve drawdown of dams) that

meet recovery standards, although there is no single non-breaching option that meets

recovery standards under all assumptions about the relative survival of transported

fish. If transported fish are assumed to have high relative survival (i.e. high D),

maximizing transportation will achieve recovery standards. If transported fish are

assumed to have low relative survival (i.e. low D), then retaining current system

configuration and allowing all smolts to migrate in-river achieves the recovery

standards.

10. In addition to transport survival assumptions, model results are sensitive to alternative

ocean and in-river harvest rate targets, alternative survival and recovery thresholds,

and (under the highest D assumption) alternative assumptions about upstream

survival rates of adults.

Sensitivity Analysis

(Spring/Summer and Fall Chinook)

1. Changing assumptions about the quality and quantity of freshwater spawning and

rearing habitat had minor influences on estimates of the probability of meeting

survival and recovery standards for spring and summer chinook salmon and did not

affect overall ranking of hydropower system options. Alternative assumptions about

FGE, conversion rates following drawdown, and effectiveness of the predator

removal program generally had small impacts on the probability of meeting survival

and recovery goals for both spring/summer and fall chinook.

2. Small reductions in already small spring/summer harvest rates had minimal effects on

the probability of meeting survival and recovery standards. These improvements in

the probability of meeting survival and recovery standards under “transportation”

options were not sufficient to change the ranking of hydropower system options.

“Natural-river” options still produced higher probabilities of meeting survival and

recovery standards than “transportation” options.

3. Small changes in ocean and in-river harvest of fall chinook salmon had minor effects

on the probability of meeting 48-year recovery standards for “transportation” options.

These changes in the probability of meeting recovery standards under

“transportation” options were not sufficient to change the ranking of hydropower

system options. Large changes (e.g. reducing in-river harvest by 50% and ocean

harvest by 50 or 75%) had a significant effect on probability of meeting recovery

standard. However, the standard was met with transport options only for one ‘D’

hypothesis examined, and “Natural-river” options still produced higher probabilities

of meeting survival and recovery standards than “transportation” options.

4. Explicitly incorporating additional mortality, i.e. from sources that may not be

reflected in the historical spawner and recruit data up to brood year 1990, into spring

and summer chinook salmon analyses affected all hydropower system options

equally, and thus did not change their ranking. These additional sources of mortality
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 92

approximated recent levels of predation on salmon smolts by Caspian terns and other

bird predators in the estuary. Incorporating the additional mortality caused all the

hydropower system options to miss the 24-year survival standard. It also caused the

“transportation” options to miss the 100-year survival standard and 48-year recovery

standard.

5. Increases in the survival rates of adult spring, summer, and fall chinook salmon

during their upstream migration after drawdown of John Day Dam had minimal

effects on overall results.

Overall Conclusions:

Prospective and Risk Analyses

1. The “natural-river” options exceeded all three standards NMFS uses to determine

jeopardy for spring, summer, and fall chinook salmon, with one exception. The

likelihood of survival of spring and summer chinook salmon missed the 24-year

survival standard by less than one percentage point when drawdown of Snake River

dams was delayed for eight years. In most cases, the “natural-river” options also met

the standards under the most pessimistic assumptions. None of the “transportation”

options met the recovery standard, except under very optimistic assumptions.

2. Weighting certain key assumptions to reflect recommendations by the Scientific

Review Panel did not significantly change the results. “Natural-river” options still

met all the standards, while “transportation” options failed to meet the recovery

standard.

3. “Natural-river” options were less risky than “transportation” options for spring/

summer and fall chinook salmon. The projected likelihood of meeting the jeopardy

standards under the “natural-river” options was least variable over the full range of

assumptions. See Marmorek et al. (1998)

Sensitivity Analyses

1. Changing assumptions about the quality and quantity of freshwater spawning and

rearing habitat, harvest rates, and survival rates of adult salmon to their birthplace had

minor influences on estimates of the probability of meeting survival and recovery

standards for spring, summer and fall chinook salmon and did not affect overall

ranking of hydropower system options.

2. Explicitly incorporating additional mortality into spring and summer chinook salmon

analyses equally reduced the likelihood of all hydropower system options meeting the

standards, and thus did not change their ranking. Incorporating the additional

mortality caused all of the hydropower system options to miss the 24-year survival

standard. It also caused the “transportation” options to miss the 100-year survival

standard and 48-year recovery standard.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 93

Questions 3, 5, and 6.

Kinds of data used in PATH modeling:

Run reconstruction data:

Redd counts from index areas; dam counts of adults; harvest rate estimates from CWT

recoveries; carcass scale data (for spawner age estimation); hatchery brood removal

estimates; hatchery-origin natural spawner estimates.

Passage modeling:

Physical Data. Both the CRiSP and FLUSH passage models use specific flow rate,

reservoir elevation, spill rate, and temperature (for FLUSH, fall chinook only) data in

their passage models. These variables influence several mechanisms within in the models

such as fish travel times, relative usage of dam passage routes, and predation rates.

Biological Data. Data on Initial Emigration Timing (CRiSP and Fall FLUSH only); fish

travel time estimates (from PIT-tags and coded-wire tags); reach survival estimates (from

NMFS, PIT-tags and CWTs); predator abundance and consumption data, survival

estimates for different dam passage routes (spill, turbine, bypass); spill effectiveness

estimates; fish guidance efficiency estimates; passage indices at dams; smolt descaling

data.

Other life stage data used:

Subbasin specific hatchery release data; juvenile dam counts; SAR estimates from CWT

and PIT-tag data (of both transported and non-transported fish); radio tag data for adult

upstream migration and pre-spawning mortality; climate data (e.g., sea surface

temperature, upwelling/downwelling transition dates, drought indices); land use data (e.g.

percent watershed logged).

Assumptions/Uncertainties:

Uncertainties in past conditions due to incomplete data and potentially confounding

influences generate a range of alternative assumptions about historical conditions, such as

the mortality of fish at specific dams in past years, or the success of past transportation

experiments. These alternative assumptions about the past, together with historical flow

information, are used in retrospective modeling analyses that generate quantitative

estimates of parameters needed to run models into the future. This requires running both

passage models, which estimate survival from Lower Granite Reservoir to Bonneville

Dam and life cycle models. Spawner-recruit data and environmental data (e.g., climate

indicators) are used for calibration of the life cycle models’ stock production functions

and other parameters. The retrospective modeling analysis quantifies our understanding

of the variability in survival rates, and the factors that affect them. Results from the
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 94

retrospective analysis are passed to the prospective analysis. The prospective modeling

analysis quantifies the range of possible futures, expressed as specific performance

measures. This set of possible futures depends on:

• the understanding and estimated parameter values gleaned from the retrospective

analysis;

• the specific future action under consideration (scenarios A1, A2, A2’, A3, B1). This set

of actions has been developed by the Implementation Team (I.T.), and draws from

previous experience of analyzing a much larger set of options (refs: Biological Opinion;

System Operating Review; System Configuration Study).

• the expected flows associated with each action; and

• assumptions about future conditions, including passage survival assumptions such

as fish guidance efficiency through bypasses around dams, and non-passage assumptions

such as harvest schedules, habitat improvements and future climate.

For the prospective analysis, the alternative hydrosystem management actions are

evaluated by simulating their consequences using a linked set of models in a four-step

process to generate performance measures:

1. A hydro-regulation model translates each management option into the mean monthly

flows which would be observed in the Snake and Columbia Rivers at various

locations, (the U.S. Army Corps’ HYSER model has been used for the scenarios

included in this report). The hydro-regulation model is run for the water years 1929-

1988 to generate a representative set of flows, and this information is used as input to

the passage models.

2. A passage model translates the projected set of flows, spills, and dam configurations

and operations for a given year into the estimated passage survival of both

transported and non-transported smolts through the migration corridor from the head

of Lower Granite Reservoir to the tail-race of Bonneville Dam. The passage models

simulate passage survival rates under each management action for a given set of

water years, to compute the improvement in survival relative to the retrospective

period. The longer term water record (i.e., 1929-1988) is considered in step 4. We

have used two different passage models, CRiSP and FLUSH, which use different

approaches to predicting passage survival rates.

3. One of the key pieces of information passed from the retrospective modeling analysis

to the prospective analysis are estimates of the ratio of post-Bonneville survival rates

of transported to that of non-transported fish. These ratios are generated by

combining estimates of historical passage survival rates with the results of

transportation experiments.

4. A life-cycle model (BSM) generates a range of possible spawner abundances for each

stock and year, under each management option. It does this by combining information

produced by the passage models (i.e., the projected passage survivals, fraction of fish

transported, and post-Bonneville survival assumptions) together with estimates of the
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 95

other (non-passage) influences on survival (i.e., stock productivity, adult survival

during upstream migration and harvest, post-Bonneville mortality, climate conditions,

and habitat changes). The life-cycle model performs a thousand simulations for a

given set of passage model inputs to ensure that the full range of possible ways the

system works, and thus the full range of possible futures, is adequately simulated, and

that the uncertainty in performance measures is properly estimated. These simulations

randomly select passage model outputs from each of the given set of water years

according to how frequently the flow in each kind of year occurred in the long term

historical record (1929-1992). For example, an extremely low flow year like 1977

(the lowest flow in the entire 1929-1992 period) is selected much less frequently than

a more typical flow year like 1979 or 1985. For spring/summer chinook, the life

cycle model also considers alternative assumptions with respect to whether upstream

and downstream stocks have some common responses to climate fluctuations

(DELTA approach) or respond independently (ALPHA approach).

The decision analysis approach (of which the Bayesian life cycle model is only one

component) allowed the analysis to consider all possible combinations of all relevant

uncertainties. The Bayesian life cycle simulation model (BSM) was essential to carry

uncertainties in climate and stock-production relationships through the analysis. BSM

was modified to allow for the possibility that upstream and downstream stocks have

different estuarine and ocean survival – this took extra time and made the analysis more

complex, but was an important alternative to the hypothesis of common year effects.

Question 7

The MLE (Maximum Likelihood Estimation) life cycle model, which uses retrospective

data to generate information used in the prospective life-cycle model (BSM), was

developed by an independent scientist (R. Deriso) in consultation with the other PATH

participants. The procedures used estimating parameters ultimately used in the analysis

of actions in the hydrosystem and other H’s were analyzed for sensitivity to such things

as spawner measurement error and different parameterizations. Biases arising from using

the wrong set of assumptions were tested for by generating simulated recruitment data

and assumed passage mortality, and then using the MLE model to estimate the

parameters (whose ‘actual’ values came from a distribution with specified mean and

variance). Lowest bias models found from this procedure were the ones used in the

decision analysis. The median predicted recruitment estimated by the MLE for each

brood year can be compared to the observed recruitment for each index stock [slide].

The life-cycle models were used not to make point estimates of future spawner

abundances, but to generate statistical distributions of these abundances. Any one run

(i.e., a setting with particular inputs and parameter distributions reflecting, e.g. a

particular passage model and extra mortality hypothesis) results in a distribution of

possible outcomes. The accuracy of the distribution produced by any one run of the BSM

depends on the accuracy of the hypotheses that run relies on (as well as the error inherent

in the life-cycle model configuration). Use of historical information on recruitment and

juvenile passage survival helped constrain the analyses of prospective actions to plausible
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 96

outcomes, although intersection of some hypotheses resulted in highly optimistic

outcomes. The influence of these highly optimistic runs on the overall conclusions was

minimized somewhat when SRP weightings were applied to the run results to generate

overall distributions.

Actions were categorized by assessing the performance of the stocks over a range of

combinations of likely hypotheses, in order to determine the ranking of actions with

respect to the performance standards (using either equal weighting of hypotheses or SRP

weighting). The documentation tried to emphasize that relative ranking of actions may

be more important than information on performance of an action relative to absolute

standards, given the knife-edge nature of the performance standards. Rank order of

actions was insensitive to choice of life-cycle model configuration (alpha or delta),

though in general delta was more optimistic with respect to performance standards.

The passage models used (especially for spring/summer chinook) represented very

different approaches and predicted substantially different pictures of the relative benefits

of transportation-based and drawdown scenarios. Drawdown scenarios were better than

transportation-based scenarios under almost all assumptions (spring/summer chinook) or

most assumptions (for fall chinook) for both models; the degree to which they were better

depended on passage model. The competing passage models and the different

hypotheses under which each was run encompass the likely outcomes of each

hydrosystem alternative. The ability of the passage models to predict reach survival was

judged by producing both ‘retrospective’ and ‘prospective’ model estimates to compare

to reach survival estimates [transparency]. The prospective-type predictions are

particularly useful because information that would not be available in advance of a

migration season is excluded from the prediction method. Extensive diagnostic output

was produced so the mechanisms and hypotheses of the models were explicitly presented

to SRP and others for comparison with empirical estimates and with rationale for model

hypotheses. More important, passage model output used in conjunction with

transportation effectiveness estimates in life-cycle models to compare predicted SAR and

R/S to observed values using multiple methods for estimating goodness of fit.

Question 8

PATH is an open, inclusive, fully collaborative, independently facilitated project. PATH

gets its priorities from an inter-agency policy team. It makes use of expertise in several

areas: fisheries biology; analytical methods (including modeling and statistics);

hydrosystem operations; familiarity with the quality of empirical information (e.g.

estimates of escapement, passage survival, smolt to adult survival, dam survival, barge

survival); conservation biology and decision analysis. PATH has a healthy balance of

expertise in these areas. The proof of this assertion lies in the PATH Weight of Evidence

report and associated submissions, which integrated together all of these different types

of expertise, in a balanced synthesis (see independent scientists’ and SRP scientists’

comments on the Weight of Evidence report, attached to this letter). Incidentally, PATH

does not “vote” on conclusions or hypotheses; the hypotheses, analyses and evidence that
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 97

carry the most weight are those which most strongly survive internal and external peer

review.

Unlike the other efforts, PATH has concentrated on Snake River stocks. Analyses of

other Columbia stocks have served primarily to aid in the Snake River decision analysis.

The concentration on Snake River stocks has allowed PATH to perform a most

comprehensive analysis of two of the stocks (spring/summer chinook and fall chinook).

PATH analyses focussed on data-rich stocks. “The PATH decision analysis is by far the

most comprehensive example of applied decision analysis for any resource management

problem that I know of. It is an extremely complex undertaking, but if anyone can

suggest a better approach, I’d like to hear about it” (Randall Peterman). PATH’s explicit

inclusion of uncertainty and variability is unprecedented in the region—it did not rely on

point estimates of vital rates when comparing management actions, for instance. PATH

considered all the H’s, but its charge was to help make a decision on long-term

hydrosystem configuration. PATH performance measures were primarily those of

NMFS's jeopardy standard, and did not attempt to estimate absolute extinction

probabilities. PATH used historical data on productivity of target stocks and downstream

stocks to put management efforts in context and to help estimate their likely impacts.

CRI has a much broader mandate. CRI is a non-collaborative, NMFS process, with input

from other entities in the form of comments on CRI analyses. CRI scientists have a broad

background in conservation biology, but limited experience working with anadromous

salmonids, and many are new to the region. CRI is charged with analyzing all ESA listed

anadromous stocks in the region, regardless of the amount or quality of data available. A

major objective has been consistency of approach to stocks in different river basins and

with vastly different quality and quantity of data available. Simplicity and transparency

are stated to be important goals of the process.

CRI has used some data or results developed in PATH. However, in the CRI analyses of

Snake River chinook stocks, inclusion of alternative hypotheses, such as experience at

one life stage affecting survival probability at a later stage, has been cursory. Point

estimates of vital rates have been used to compare the potential benefits of different

management actions. Two types of models have been used: a “Dennis-type” extinction

model to estimate likelihood of extinction in a given time, assuming that conditions do

not change from the 1980 to early or mid-90s period; and a deterministic Leslie matrix

model to estimate expected changes in the mean growth rate under the assumption of a

fixed amount of mortality being reduced in individual life stages. Data prior to 1980 are

not used, and no context has been provided to quantify the likelihood of actions in

different life stages actually being effective at improving the population growth rate.

CRI’s main analysis was a “baseline” scenario, which assumed no improvement over

recent conditions. PATH did not analyze a corresponding scenario, since current

operations (A1) were assumed to represent improvement over pre-1995 conditions.

EDT is a subbasin-focused modeling system. Environmental attributes of different

watersheds are described and quantified. EDT is primarily a single-entity process. EDT

also has a broader agenda than PATH, attempting to analyze the entire Columbia River
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 98

basin. It is broader than CRI in that it includes analysis of resident fish and wildlife

species, and not just ESA-listed anadromous species. The inclusion of expert opinion is

more extensive, but less formal than the processes used in PATH. The performance

measures are less sharply defined than in PATH or CRI.

Question 4

Strengths

• PATH is a cooperative, multi-agency process, representing a broad array of

viewpoints.

• Internal review and debate lends focus to arguments and hypotheses.

• All PATH work products are peer-reviewed by external scientists.

• The process includes a neutral facilitator and a number of independent scientists,

including experts on decision analysis and elicitation of expert opinion.

• Two different life cycle models. First model used year (climate) effect estimated

using downstream stocks; second modeled stocks in each region separately. First

prospective model used relative improvement (ratio of prospective to retrospective

survival estimates) in downstream passage and extra mortality hypotheses; second

model plugged in actual prospective estimates to model prospectively.

• Explicitly included natural environmental variation and human-induced variation in

vital rates.

• Did not attempt to model absolute extinction.

• Differing opinions on key hypotheses, even after weight of evidence. Decision

analysis approach of PATH made it unnecessary to have complete agreement to

determine which actions had best chance of achieving recovery over range of

uncertainty.

Weaknesses

• PATH is a multi-agency process: it takes time to reach agreement and/or consensus.

Sometimes difficult to meet internal and external deadlines.

• Resources (key personnel) are limited.

• There will always be limited data for building models, testing hypotheses about the

past, and testing hypotheses about the future.

• Factors not modeled include: behavioral, productivity and genetic interactions

between populations; the effects of the quantity and quality of mainstem river habitat;

hydrosystem impacts on the estuary.

• Did not attempt to model absolute extinction.

• Analyzes intrinsically complex natural resource problems in great detail. Not

transparent to casual observer. Difficult for most to read all of the documents.

• Minority opinions can obscure robustness of primary findings.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART I, General and FLUSH - 99

Question 9.

I would advise decision-makers to keep in mind that PATH was conceived of as a

preponderance-of-evidence approach from the start; there was never any possibility of

arriving at consensus on one action that, if implemented, could be guaranteed, beyond all

reasonable doubt, to recover the listed stocks. The decision analysis could show only

under which assumptions different outcomes were achieved for each analyzed action, and

also which actions were most likely to benefit the stocks under the widest range of

assumptions reflecting uncertainty in biological processes. PATH was a fully

collaborative process; NMFS and other federal agencies were intimately involved. It was

the collaborative process initiated by the region to develop quantitative biological

information to aid in making the 1999 decision on long-term operation and configuration

of the hydrosystem.

No other process has been so fully and thoroughly peer-reviewed, and no other process

has considered as much of the relevant information. Further, the results attained in

PATH are not dependent on the particular configuration of models used in PATH:

analyses using the same data but using the Leslie matrix model advocated by CRI give

strikingly similar results about the relative effectiveness of the proposed FCRPS

alternatives in preventing extinction and achieving recovery of listed Snake River stocks

(Oosterhout et al. 2000). The interagency Drawdown Regional Economic Workgroup

(DREW), which was charged with estimating the costs and benefits of the alternative

FCRPS actions, used PATH to provide information on biological outcomes of the

different actions.

References

Marmorek, D.R., C.N. Peters and I. Parnell (eds.) 1998. PATH final report for

fiscal year 1998. Prepared by ESSA Technologies Ltd., Vancouver, BC, 263 pp.

Oosterhout, G. and 8 others. 2000. A technical review of the National Marine

Fisheries Service Leslie matrix model of Snake River spring and summer chinook

populations. April 28, 2000.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART II, CRiSP - 100

PATH Answers Part II - CRiSP and Alpha Life Cycle Model

Jim Anderson, University of Washington

Phone: 206-543-4772

e-mail: jim@cbr.washington.edu

Date: March 20, 2000

1. What is the purpose of your modeling effort? What questions or problems were

your models designed to address?

The purpose of PATH was to evaluate the effectiveness of a limited number of Snake

River salmon recovery actions including breaching the lower four Snake River dams,

increased transportation, and reduced harvest. Hatcheries and habitat were evaluated in a

cursory manner but the effort in these areas was inadequate to say anything substantial.

Two of my contributions in PATH were developing the CRiSP passage model and the

alpha life cycle model in which the extra mortality was expressed independent of the

downstream stocks.

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s

conclusions relative to the following:

a. The efficacy of dam breaching or drawdown to natural river levels for delisting

ESA species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

The weighting of the analyses in PATH indicated that drawdown was better than

transportation as a recovery measure for spring chinook while harvest was a possible

recovery measure for fall chinook. After the conclusions were developed, publicized and

made official, new data from the passage and transportation studies were made available

that challenge the PATH conclusions. PATH did not update its analysis to reflect these

new findings. The specific PATH conclusions on the probability of recovery with dam

breaching vs. transportation are questionable.

b. The efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

Consider hatcheries that mitigate for lost habitat and those used to supplement

depleted stocks.

PATH did not address hatcheries in a scientific manner. Any results were little more than

the personal opinions from specific members of PATH and of the review panel.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART II, CRiSP - 101

c. Allocation of harvest and harvest levels needed to delist ESA species and

restore diverse and productive populations of native fishes throughout the

Columbia River Basin?

The conclusions on harvest levels needed to restore stocks were inextricably bound to the

assumptions of the intrinsic productivity of the stocks. The greater the productivity, the

more effective harvest as a recovery measure. For both spring and fall chinook, PATH

very likely overestimated the intrinsic productivity. This overestimation evolved from

the decision support framework that divided PATH into pro and con groups on many

issues. Since agendas drove the groups, the hypotheses were formulated with bias

towards particular beliefs. Thus, dam removal advocates developed models with high

hydrosystem mortality, which required a high intrinsic productivity to balance the large

mortality. The group favoring climate as a major contributor to fish decline proposed the

hypothesis that the climate would return to pre-1977 conditions, which again inferred a

high productivity in the future. The “here to stay” extra mortality hypotheses became so

convoluted with the mixture of assumptions intrinsic to PATH’s decision process that it

did not accurately represent the possibility that productivity may now be low in the

system. The upshot of all this was that the mixture of hypotheses gave very high

productivity so harvest reductions predict optimistic recovery probabilities.

d. The efficacy of restoration of tributary and mainstem habitat for delisting ESA

species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin?

PATH addressed single species issues and had no essential contribution to restoring

diversity in tributaries or the mainstem. Assessments for restoring or creating mainstem

fall chinook habitat were approaches in a cursory manner. The discussion of the changes

that may occur in the drawn down reservoirs was not linked to the estimated changes in

smolt survival or growth.

2. What kinds of information or data are needed to run your model?

The CRiSP passage model has been calibrated with several levels of information. The

water quality predictions have been calibrated with physical measurements on total

dissolved gas, temperature and flow. Smolt travel time and mortality have been

calibrated with the past and recent survival studies using brand release and PIT tag

experiments conducted on specific stocks from the Snake and mid-Columbia Rivers.

Predictions to evaluate future conditions use a suite of water years representing the

expected frequency of low, medium and high water years. Survival for each release is

identified in terms of locations and timing of release over the basin. Model predictions

on smolt survival require hydrosystem operations, water quality conditions, and stock

release information.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART II, CRiSP - 102

3. What are the strengths and weaknesses of your model?

The strength of the CRiSP passage model is that it is calibrated with all the available

current data and the model fits the data better than other models developed to describe

survival of smolts through the river system. A major weakness of the model is that all

fish from a release group are treated as equal. Presently there is no way to characterize

differences in accumulated stress from different passage routes or because of pre-smolt

conditions. This is particularly significant for subyearling chinook in which size is

correlated with survival and migration rate. With the next version of the model, CRiSP

1.7, we are addressing some of these problems. Size and growth are being included

through a bioenergetic model. In addition, we will include fish condition through a

vitality modeling approach that can deal with the accumulation of stress over time

(Anderson in press, Ecological Monographs – “A vitality based model relating stressors

and environmental properties to organism survival”).

Another issue is the assumption that survival is a function of travel time. The PIT tag

studies in the tributaries above the hydrosystem do not support a simple travel time

survival relationship as is contained in the CRiSP1.6 model and the FLUSH model. We

have developed a new theory to address this problem. It will be included in CRiSP1.7.

4. What are the assumptions of your model?

In the CRiSP 1.6 passage model survival is a function of travel time and environmental

water quality including temperature and total dissolved gas levels. In general, survival

decreases as an exponential function of travel time. The rate of mortality decrease

increases with temperature and when total dissolved gas level is above a threshold level.

The alpha life cycle used in PATH is a variant of the Ricker spawner-recruit model. It

assumes climatic variations are characterized by a decadal scale cyclic process and a step

process that changes with a cycle of 60 years.

5. How does your model address uncertainty?

Uncertainty in PATH models was expressed through a weighting of the hypotheses by

outside experts. In retrospect, this was inadequate and misleading. Since PATH

explored a large range of hypotheses, many results could be obtained. Ultimately the

weighting by the four review members determined the outcome of the PATH process

irrespective of the scientific logic or rigor of the results. Essentially the final PATH

results were the reflection of four scientists selected by the moderators. Their written

justification for their weighting revealed misconception and biases. The recent survival

studies do not comport with their conclusions and the fact that PATH did not reconcile

discrepancies between its conclusions and the new data should be of concern to decisionmakers. I believe the PATH decisions process did not resolve the uncertainty in the

PATH hypotheses. If anything, the degree of certainty in conclusions was overstated.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART II, CRiSP - 103

6. All models make predictions. Why do you think your model’s predictions are

accurate?

The CRiSP model predictions for spring chinook hydrosystem survival have been proven

accurate by comparisons with PIT tag data.

The success in characterizing fall chinook survival is less because the model does not

include important behavioral and bioenergetic factors and systemwide estimates of fall

chinook survival are currently unavailable.

The life cycle model based predictions on recovery probabilities are likely optimistic.

7. How does your modeling effort relate to or contrast with the other three modeling

efforts?

PATH, in spite of problems of balance, openness, and scientific rigor, did explore the

bounds of possible outcomes of dam breaching and transportation, and it synthesized

much of the data. The other modeling projects have used the PATH data analyses as a

foundation for their work. In addition, PATH formalized the mathematical foundations

for extra and delayed mortalities.

In our new juvenile passage modeling approach, we are addressing individual smolt

characteristics including size, growth, and vitality. This approach is unique in that all

other modeling efforts to date have considered fish at a population level that does not

differentiate between successful and unsuccessful individuals within the population.

8. What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River Basin?

CRiSP is the best available modeling system to describe the effect of hydrosystem

operations on smolt passage survival. It realistically characterizes the effects of flow,

spill, gas supersaturation and temperature on fish travel time and survival through the

hydrosystem. Neither CRiSP, nor any other passage or life cycle model now available, is

adequate to address the issue of delayed mortality or survival of fish above the

hydrosystem. To address these issues the models need to be further developed and new

experiments need to be planned to characterize the impacts of passage experience on fish

survival.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 104

PATH Answers Part III - Experimental Management

From: Charlie Paulsen

Rich Hinrichsen

Date: May 2, 2000

Subj: Reply to ISAB questions for modelers – PATH Experimental Management

Purpose

The complete report (20 pg. Executive summary, 65 pg. Main body, plus appendices) is

now on the BPA PATH web page, http://www.efw.bpa.gov/PATH/. We answer the eight

questions briefly in this memorandum; more complete information on methods, data, etc.

are in the report. The views in this memo are our own (Paulsen and Hinrichsen),

although the report proper is a PATH group effort.

In contrast to earlier PATH reports, which focussed on the effects of hydrosystem

management actions on Snake chinook population viability, the experimental

management (EM) report analyzes trade-offs between biological effects of actions (how

they influence persistence and recovery) and how much one could learn by treating

actions as experiments whose outcomes are uncertain. It is designed to answer questions

like the following:

• If an action increased the ratio of recruits to spawners (at low spawner numbers)

by, say, 50% above its recent average, how long would one need to be confident

that the action works as planned?

• What are the trade-offs between the length of the experiment and confidence in its

efficacy?

• How will population viability be affected by experimenting (e.g., action “on” in

odd-numbered years and “off” in even-numbered years) as opposed to turning the

action “on” as soon as possible?

Although PATH incorporated many uncertainties via a decision analysis approach, to

date no other work within the Columbia has addressed these design questions in detail for

ESA-listed stocks.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 105

Major conclusions

In investigating a variety of EM actions, the effects on population viability, and

associated learning opportunities, we have looked at a number of life stages, all for Snake

spring/summer chinook. These include

• Rearing habitat (increasing egg-smolt or parr-smolt survival via carcass or

nutrient enrichment);

• Downstream survival (drawdown of four Lower Snake projects);

• Downstream survival (turn transport on/off in alternate years); and

• Downstream/early ocean survival (reducing Snake hatchery releases).

The model structure (see replies to questions 4, 5, and 6) is quite general, however. The

focus is on the recruit/spawner ratio, and the model can accommodate survival changes at

any stage in the life cycle. We emphasize how long one would need to monitor

population dynamics to detect changes, not on the most probable effects of any given

action. Although the effects of the above actions are based on available data, they are

uncertain; if one could precisely predict their effects, no management experiments would

be required to reduce uncertainty. On one specific point, harvest, spring/summer chinook

harvest rules are used by the model to account for changes in harvest rates as a function

of changes in the abundance of returning adults. These rates can be changed easily, but

past work in PATH suggests that for spring/summer chinook, population viability is not

terribly sensitive to changes in harvest rules. We have not performed this type of

sensitivity with the EM model.

Model Inputs

The model inputs for “base case” population projections consist of annual estimates of

the following for Snake spring/summer chinook. Data series start in 1957, and run

through 1994. Note that the model does not rely on upstream/downstream comparisons,

or on passage models, for the base case.

• Spawning escapement, estimated from expanded redd counts.

• Spawner age estimates, from scale analyses and length-at-age information. These

are often averages from years with available data.

• Estimates of past harvest rates.

• Estimates of upstream survival, from dam counts, “turnoff” into lower river

tributaries, and harvest.

• A harvest “schedule” based on abundance in future years and rules for harvest

rates as a function of run-year abundance.

For any given action, the model also requires estimates of the effects on life-cycle

survival. The data sources for these are documented in the body of the EM report.

Examples include correlations between spawning escapement and parr -> smolt survival
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 106

estimates (latter from Paulsen and Fisher, 2000, in review), or correlations between

recruit/spawner ratios and hatchery releases. For drawdown actions only, we use passage

model output to estimate effects of drawdown on life-cycle survival.

Information Needs

Beyond the data noted above, the model needs a schedule for when EM actions will be

implemented (e.g., on/off in odd/even years for 10 years running). Similar information

on a stock-specific basis is required for actions that may effect only a subset of the index

stocks (e.g., carcass/nutrient enrichment).

Strengths and weaknesses

There are two important strengths of the model. First, as a population projection model,

it is very straight-forward regarding data requirements and the basic mechanics of the

population dynamics:

i t i t i i i t mt mi t i t R S a b S , , , , ,

ln( / ) = + + + Δ + e

Ri,t,age = maturei ,t ,age × Ri,t

= × × − × − ×∑ − age i t i t i t i t i t Ri t age age S prespwn conv mharv tharv , , , , , , ,(1 ) (1 )

i = subscript representing index stock

t = subscript representing brood year

Ri,t = recruits (adult offspring) from parent spawners Si,t

Si,t = spawners returning in year t

ai = average Ricker-a

bi = Ricker-b

mt = common deviation from average productivity

Dmi,t = hypothetical changes in productivity schedule (E.M. Design)

ei,t= Normally distributed process + counting error (white noise)

Ri,t,age = age-specific recruits (adult progeny)

maturei,t-age,age = age-specific maturity schedule

prespwni,t = pre-spawning survival

convi,t= upstream survival (conversion rate)

mharvi,t=mainstem harvest rate

tharvi,t = tributary harvest rate

Estimation of the model is accomplished by doing a least squares fit of

ln(Ri,t/Si,t) = ai + biSi,t + mt + ei,t

to the spawner-recruit data. The model contains 264 observations and 52 parameters

(including sigma^2).
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 107

The only complications (in population projection) arise because of the necessity of

tracking multiple age-structured populations through time.

Second, it seem to us that it is well-suited for estimating both the biological (population

dynamics) and information (experimental power) effects of EM management regimes.

These can be as simple or complex as desired by the model user, effecting all stocks

permanently (e.g., drawdown) or different stocks in different ways over time (e.g.,

different nutrient enrichment schedules for different stocks and years). In the report, we

show examples of the model’s use with a variety of ancillary data to estimate the effects

of EM actions, ranging from PIT-tag based SAR’s to hatchery releases and parr -> smolt

survivals derived for studies designed to detect arrival times at Lower Granite.

The model’s main weaknesses are the strong reliance on spawner-recruit estimates, which

may contain systematic errors (random variation should be absorbed into the process

error terms in the model), and the basic assumption that the estimates of an action’s

effects (which are model inputs) are accurate. On the other hand, if the effects of any

given action were known with certainty, there would be no need for an experiment, and

little need for population dynamics models.

Assumptions

Many assumptions are required for this or any population dynamics model. Among the

most important are:

• Future conditions will be like 1978-1994, absent management intervention;

• Harvest rules, conversion rates, and population age structure will be similar to

recent respective means and variances;

• Joint distributions of estimated model parameters (Ricker ai’s and bi’s, mt’s) apply

to the future.

For detecting the effects of EM actions, the assumption regarding future conditions like

1978-1994 can be relaxed, so long as changes in other conditions (e.g., climate) do not

coincide with the schedule for the EM action (e.g., on in odd years, off in even years).

Uncertainty

The uncertainty in parameter estimates is assessed by summarizing the sampling

distribution via confidence intervals. The population projection model respects

parameter uncertainty by drawing from the joint sampling distribution of the estimates

using technique of Gelman et al. (1995, p. 237) which treats the joint sampling

distribution as a “posterior” distribution or using “bootstrapping the residuals” (Efron and

Tibshirani 1993, p.113-115). Uncertainty in effects of management actions is addressed

using (optional) distributions of projected effects. Uncertainty in risk measures is

assessed using confidence intervals.
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 108

We assume that statistical distributions estimated from historical data will continue, and

that EM actions are not perfectly confounded with other actions or natural phenomena.

The latter is of most concern with actions that can be turned off or on only once (e.g.,

drawdown).

Prediction accuracy

We have tested the fit of the retrospective model extensively, including searches for

outlying data points, addition of more index stocks (another 5-10 have been developed),

excluding/including recent years of spawner/recruit data (1991-1994), and a variety of

other statistical techniques, and found it to be robust.

Of course, the future is unknown. If, for example, the 2000 returns of spring chinook,

currently running 3-5 times the 10-year average, signal a long-term change in marine

survival, the model’s assumption about low-survival recent conditions will not hold. On

the other hand, the EM design should be robust to most such changes.

Other modeling efforts

This is the only modeling effort we are aware of that has done a systematic exploration of

trade-offs among population risk measures, experimental design, and knowledge gained

via an experimental approach to reducing uncertainty. Initial steps by CRI in this regard

(McClure et al. 2000, section V-E) basically consist of before-and-after comparisons of

abundance for isolated populations, which can of course be influenced by a myriad of

natural and anthropogenic factors. The parr -> smolt survival estimates used as one

approach for estimating the effects of nutrient additions are drawn from one of very few

studies that directly estimate freshwater survival rates as a function of land use patterns.

Table. Current differences in CRI and EM modeling at a glance.

CRI (Dennis-type models)

McClure et al. (2000)

EM (Screening) Model

Covariability in growth

rates among stocks

No (in general), but index

stocks combined into single

population for some

analyses.

Yes

Can project responses to

experimental treatments

interspersed over time and

space.

No Yes

Recovery escapement

thresholds

No Yes

“Survival” escapement

thresholds

No Yes
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 109

Table. Current differences in CRI and EM modeling at a glance.

CRI (Dennis-type models)

McClure et al. (2000)

EM (Screening) Model

Extinction probabilities Yes Yes

Confidence intervals

reported for:

lambda (for ESU-level

only)

All model parameters,

probability of extinction

and exceeding recovery

escapement, fraction of

years survival escapement

exceeded.

Density dependence No Yes. Growth rate decreases

with increased population

size.

SR “data” time span BY 1980-1994 BY 1957-1994

Growth rate distribution Instantaneous growth rate is

Gaussian white noise (no

trend)

Density-independent

growth is decomposed into

common growth effect

(allows for trend) + white

noise

Population response ln(N(t+1)/N(t)), where N(t)

is a “weighted running

sum”

ln(Recruits/Spawner),

where recruits are adult

progeny arriving at

Bonneville Dam.

Time step Continuous. Reproduction

modeled as continuous

process

Discrete. Reproduction is

seasonal.

Observation error in SR

data

Handled using innovative

estimation technique and

“weighted running sum”

Limited weir counts used to

estimate ratio of process to

observation error variance.

Achieving target viability Uses changes in lambda Uses changes in

log(Recruits/Spawner),

upstream survival, harvest
ISAB 2001-1: Model Synthesis Report

Appendix III: PATH PART III, EM - 110

Advice

Our advice would be to take the population dynamics predictions with several grains of

salt: changes in climate and other uncontrollable factors make predictions of extinction

times, probabilities, etc. very uncertain. However, we have more confidence in the

overall results from the EM modeling: actions that affect all stocks, even if tested

rigorously over time periods of one or more decades, will likely have substantial residual

uncertainty in their effects. These “temporal” controls (on/off) must run for many years

because of high year-to-year variation in survival. Experiments that utilize more

traditional “spatial” controls (e.g., treat half the index stocks, using the remainder as

controls) have far more power to reduce uncertainty.

References:

Efron, G. and R.J. Tibshirani. 1993. An introduction to the bootstrap. Monographs on

Statistics and Applied Probability 57. Chapman & Hall/CRS, New York.

Gelman, A., J.B. Carlin, H.S. Stern, and D.B. Rubin. 1995. Bayesian data analysis.

Chapman & Hall. New York.

McClure et al. 2000. Draft document. Standardized quantitative analysis of the risks

faced by salmonids in the Columbia River Basin. Available at the CRI web site.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 111

Ecosystem Diagnosis and Treatment Answers

ISAB Model Review

Response to Questions Relative to the Ecosystem Diagnosis and Treatment Method.

1. What is the purpose of your modeling effort? What questions or problems were

your models designed to address?

The EDT method is a landscape-based approach for relating events and actions affecting

animal populations and their habitat to the long-term performance of species of interest

(Lichatowich et al. 1995; Mobrand Biometrics, Inc. 1999). The EDT method incorporates

a conceptual framework, a procedure, and a set of tools (Lestelle et al. 1996). The

framework provides the theoretical foundation, the procedure prescribes the steps in

planning and analyzing ecosystem information in support of decision-making, and the

tools are the databases and model components used to organize, analyze, and summarize

information.

Development of EDT was funded largely through the NPPC Fish and Wildlife Program

and in response to “concerns raised by the ISAB and its predecessors (ISG and SRG, e.g.

ISG 2000) regarding the need for a clear and explicitly stated framework to serve as the

rational, scientific basis for the FWP” (NPPC 1997; Mobrand et al. 1998).

EDT was designed to address cumulative effects of many environmental factors, in many

places. It is a tool for tracking the operating assumptions upon which adaptive

management action plans can be built. EDT can be used to:

• develop working hypotheses regarding action priorities; for example, diagnose

salmon performance problems and identify treatment alternatives;

• develop and document working hypotheses—the conditions required to achieve

specified objectives—for detailed and comprehensive action alternatives;

• assess potential benefits and risks associated with incorrect operating

assumptions—in other words, analyze consequences under alternate hypotheses;

• identify key uncertainties to guide research and monitoring and evaluation.

EDT was designed to be an expert system and, as such, differs from statistical models in

both purpose and design (Holling 1998, Hilborn and Mangel, 1997)—a key distinction.

Expert systems seek to be comprehensive and explanatory, they generate hypotheses

rather than test them, and they formulate operating assumptions upon which actions can

be based and risks understood. C.S. Holling (1999) states the distinction in another way

“...reductionist science, i.e. the science of parts, was essential to provide bricks for an

edifice, but not the strategic design of the edifice. Such strategic design is needed for

appropriate diagnosis and policy, and it has to emerge from a science of integration. A
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 112

science of integration combines research and application, is interdisciplinary and faces

the realization that knowledge of the system we deal with is always incomplete.”

EDT is a tool for building hypotheses. It depends upon the results of statistical analyses

to formulate new hypotheses, which in turn can be tested through statistical models. It is

a framework for integrating results from statistical analyses with knowledge and opinions

from all relevant scientific disciplines.

Expert systems like EDT consist of data linked to conclusions through a set of rules. EDT

was conceived to be flexible so that new knowledge, information, and data can be

incorporated, thus allowing the expert system to remain current. At the heart of EDT is

the structure that defines how data and rules are linked. This temporal-spatial-conceptual

architecture of data and rules is the Framework as defined and explained in the Multispecies Framework process (NPPC 1997; EWG 1999; Mobrand et al. 1997; Lestelle et

al.1996). The data and rules themselves are more ephemeral and replaceable.

EDT computes future capacity, productivity and diversity of salmon in the Columbia

Basin as a function of: a) the bank of genetic resources embodied by the animals alive

today and b) the environment available to them. The current version of the EDT expert

system, while incorporating genetic fitness factors, has a primarily environmental focus.

EDT constructs “survival landscapes” that describe the patterns of survival conditions for

salmon over time and space. EDT assesses the quality of the habitat in regard to the

biological template of specific species, such as chinook salmon (Mobrand et al. 1997).

The specific purpose of the application of EDT in the Multi-species Framework analysis

was to compare a set of comprehensive visions for the Columbia Basin in terms of the

prospective performance of salmon populations in each of ten provinces (McConnaha

1999). The first stage in this analysis focused on chinook salmon. Based on available

environmental descriptions of some 7,500 spatial habitat units (6-level HUCs) for the

Columbia River basin as well as information on hatchery production, harvest, and hydro

operations, we projected productivity, capacity and diversity for 73 natural and 50

hatchery populations of chinook salmon under 10 different scenarios. These scenarios

included the current, historic, no action conditions as well as seven alternative futures for

the Columbia River Basin. The analysis represents the long-term average performance

potential for chinook salmon.

2. Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s

conclusions relative to the following:

The EDT analysis performed so far for the Columbia Basin estimates cumulative effects

of all H’s. We have not attempted to analyze isolated strategies; instead, the Multispecies Framework analysis compared alternatives composed of prescribed mixes of all

H’s. Each of these alternatives was based upon a worldview and was modeled using

[operating] assumptions consistent with this worldview (Costanza 2000). As we finalize
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 113

the Multi-species Framework analysis, we will address the biological consequences when

alternate worldviews are true, in an attempt to shed some light on the benefits and risks

associated with strategies that emphasize some of the H’s more than the others.

Because we have not completed our strategies analysis, the answers below represent

general impressions of results so far.

a. The efficacy of dam breaching or drawdown to natural river levels for

delisting ESA species and restoring diverse and productive

populations of native fishes throughout the Columbia River Basin?

EDT and the Framework Project did not attempt to address the needs for delisting ESA

species. Drawdown and dam breaching were effective strategies for restoring mainstem

chinook habitat with a relatively low level of biological risk but with a great risk of social

change and political will.

b. The efficacy of hatcheries for delisting ESA species and restoring

diverse and productive populations of native fishes throughout the

Columbia River Basin? Consider hatcheries that mitigate for lost

habitat and those used to supplement depleted stocks.

The efficacy of hatcheries to replace natural habitats and ecological functions is not an

analytical problem but will only be solved by careful study. Absent firm information, our

approach has been to bracket alternatives that make heavy use of hatcheries by using a

range of effectiveness assumptions. Our initial assumption was that hatcheries fulfilled

their promise and were comparably effective as natural populations. We are contrasting

this with a view that says they are substantially less effective than natural populations.

c. Allocation of harvest and harvest levels needed to delist ESA species

and restore diverse and productive populations of native fishes

throughout the Columbia Rover Basin?

The alternatives that we examined assumed different harvest allocations and rates,

including strategies that greatly reduce harvest rates. In general, harvest reductions had

little impact on spring chinook stocks but were important to the abundance of fall

chinook. We have not attempted to assess impacts of harvest rates on ESA status.

d. The efficacy of restoration of tributary and mainstem habitat for

delisting ESA species and restoring diverse and productive

populations of native fishes throughout the Columbia River Basin?

We looked at a wide array of strategies to change habitat attributes in tributaries and

mainstem areas. Breaching dams, for example, was quite effective in restoring mainstem

habitats to the benefit of chinook performance. Land-use changes and practices in

tributaries were, over the long term, effective in restoring abundance and distribution of

tributary populations.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 114

3. What kinds of information or data are needed to run your model?

EDT requires qualitative and quantitative habitat information (baseline input data) and

generic information about the salmon species and their habitat requirements (rules

relating survival to habitat conditions). Habitat information is collected at the HUC-6

level (about 7,500 “pixels” in the U.S. portion of the Columbia River basin) in regard to

some 40 habitat parameters. Biological information is collected at the population level.

For chinook salmon, we have described 73 natural populations in the basin.

4. What are the strengths and weaknesses of your model?

Perhaps the greatest strength of the EDT approach is that it accounts for cumulative

effects—such as spatial temporal interactions, all attributes, competition, predation

effects. In addition, EDT:

• complements and is supported by statistical approaches,

• enhances the value of research and monitoring results,

• computes population performance in terms of productivity, capacity, and

diversity,

• translates combinations of actions at any scale into biological performance

responses,

• examines comprehensive alternative futures,

• diagnoses problems and prioritizes treatments,

• provides a repository of knowledge and the means for documentation and

accountability,

• provides a conceptual framework for understanding,

• identifies critical uncertainties (for directing monitoring activities and

research),

• separates policy from science,

• describes benefits and risks of approaches,

• supports trade-off analysis.

Among the weaknesses of the current version of the EDT model are:

• lack of tools for easy review and edit of all rules, (we are working to remedy

this problem)

• need for ground truthing of input data and routine peer review process to

ensure that the rules are well understood and consistent with current

information and knowledge, (a structured, independent certification process—

with standards for documentation and access—for data and rules would

greatly enhance the utility of this and other models)

• need for genetics components to be expanded to accommodate a wider range

of assumptions

• because of the dimensionality (many reaches, time periods, life stages and

attributes) the model is demanding in terms of computer resources (it takes

hours to run and requires lots of memory).
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 115

5. What are the assumptions of your model?

The fundamental assumption of EDT is that the Multi-species conceptual foundation and

framework are appropriate and useful to the decision-making process they are intended to

support. Specifically, EDT assumes that habitat is the template for biological

performance (Southwood 1977). EDT begins with a detailed habitat description and

links it to ecological function and biological performance of specific species.

For the most part, EDT is a flexible, expert system capable of incorporating a wide range

of hypotheses and assumptions. Each scenario modeled represents a unique set of

assumptions (worldview) and a specified suite of actions.

Two parameter density-dependent survival functions are assumed (Mousalli and Hilborn

1986) for each life cycle segment. Typically the Beverton-Holt equation is used; others

can be accommodated.

6. How does your model address uncertainty?

EDT is a steady-state model that predicts long-term outcomes. It does not incorporate

stochastic elements. It includes current population status only as genetic fitness factors

and, therefore, does not predict the length of time required to reach the long-term state. It

does not explicitly compute the extinction risk.

The implications of uncertainty are addressed through sensitivity analysis, which in turn

can guide recovery actions as well as monitoring and research efforts towards risk averse

solutions.

7. All models make predictions. Why do you think your model’s predictions are

accurate?

“All models are wrong. Some are useful.” (a paraphrase of a statement for which I can’t

recall the source). EDT provides an accurate depiction of the environment and biological

performance given our current scientific understanding and available data. Its depiction,

like that of any model, is imperfect because our knowledge is imperfect and because

nature is inherently complex. It does however provide a framework within which we can

accumulate knowledge and improve predictability over time.

EDT inputs consist of detailed descriptions of the landscape. It captures the obvious

features of the landscape on a fine scale in terms of many observable attributes. For

example, it includes length and width information of stream reaches; it routes these

reaches thus accounting for the connectivity (or lack thereof) along the salmon migration

pathways. The unique conditions of a watershed described this way, coupled with the

relatively well understood habitat requirements and patterns of movement of salmon,

allow EDT to predict steady-state population parameters and to compare relative effects

of different habitat changes on salmon performance in a way that is useful and

informative to decision making.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 116

In other words, the main reason EDT provides useful predictions is that it captures the

habitat conditions on a fine scale and analyzes their significance from the point of view

of salmon and other species.

8. How does your modeling effort relate to or contrast with the other three

modeling efforts?

The relationship between CRI, PATH, and EDT is complementary. The EDT analysis

can and should incorporate the analytical results of the statistics-based PATH and CRI.

The EDT model can be used to develop hypotheses regarding, for example, the

significance of specific habitat survival relationships, which in turn might be tested with

statistical models. CRI fits parameters to observed population data. The EDT model is

driven by habitat data and does not input any population numbers. See Kareiva et al.

1999.

The ICBEMP method is similar to EDT in concept and data requirements. EDT estimates

quantitative and cumulative effects of actions on salmon populations.

The CRITC model offers some very intuitive ways for the user to explore alternative

strategies and assumptions. EDT might complement this model by expanding its

analytical capabilities. Our impression is that the strength of the CRITFC model relative

to EDT is in the user interface; whereas EDT is analytically more powerful.

9. What advice would you give to decision-makers on how they should use your

model to support decisions regarding salmon recovery in the Columbia River

Basin?

We believe that models can best aid decision-makers by highlighting risks and tradeoffs

of alternative actions. No model does it all for most decisions. For this reason, we

strongly favor the concept of an analytical toolbox. EDT can and should be used in

conjunction with other models such as the CRI, PATH, and CRITFC tools. This would

allow the region to take advantage of the strengths of each model system and to

appreciate the different perspectives afforded by each. Unfortunately, models too often

become analytical bludgeons in political warfare. They are used to bolster political

positions based more on value sets than on scientifically derived information. We find

that this is the cause of much of the mistrust of models by decision-makers.

One approach that we are using to inform decision-makers is to analyze alternatives in

light of different worldviews and assumptions (Costanza 2000). This is an attempt to

highlight the underlying tradeoffs of different alternatives and their vulnerability to both

social risk (will we really make the social changes necessary?) and biological/

technological risk (can technology effectively substitute for natural ecological function?).

We believe that this type of analysis would be complemented by analysis of extinction

risk for specific populations, for example, that could be provided by other models such as

CRI.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 117

Hypothetical Decision Matrix Strategies/Actions

Worldview

Key

Assumptions

(4H’s)

Regulation

Restoration

...

Mix

Hatcheries

Transport

...

Holistic-Multispecies Vision A Vision B Vision C

Intermediate Vision D Vision E Vision F

Agricultural Vision G Vision H Vision I

EDT would flesh out the Visions in this decision chart. Operating assumptions apply on

the diagonal, alternate assumptions off the diagonal.

Reference Cited

Costanza, R. 2000. Visions of alternative (upredictable) futures and their use

in policy analysis. Conservation Ecology [online] 4(1): URL:

http://www.consecol.org/vol4/iss1/art5

Hilborn, R. and M. Mangel. 1997. The Ecological Detective: Confronting

models with data. Princeton University Press. Princeton, NJ.

Holling, C. S. 1999. Introduction to the special feature: just complex enough

for understanding; just simple enough for communication. Conservation Ecology

3(2): 1. [online] URL: http://www.consecol.org/vol3/iss2/art1

Holling, C.S. 1998. Two cultures of ecology. Conservation Ecology [online]

2(2): 4. Available from the Internet. URL: http://www.consecol.org/vol2/iss2/art4

Independent Scientific Group 2000. Return to the River 2000: restoration of

salmonid fishes in the Columbia River Ecosystem. Portland, OR, Northwest Power

Planning Council.

Lestelle, L.C. 1996, L.E. Mobrand, J.A. Lichatowich, T.S. Vogel. Applied

Ecosystem Analysis – A Primer: The Ecosystem Diagnosis and Treatment Method.

Bonneville Power Administration. Portland, OR.

Lichatowich, J. A., L. E. Mobrand, et al. 1995. An approach to the diagnosis

and treatment of depleted Pacific salmon populations in freshwater ecosystems.

Fisheries 20(1): 10-18.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 118

McConnaha, W. E. 1999. The Columbia River Multi-Species Framework

Project: Progress and Concepts. Seattle, WA, American Water Resources

Association. Proceedings, December 1999.

Mobrand, L.E., L.C. Lestelle, G.R. Blair, 1998. Recovery of a Columbia River

Watershed from an Ecosystem Perspective. A case study using the EDT method.

Final Report. Bonneville Power Administration. Portland, OR.

Mobrand, L. E., J. A. Lichatowich, et al.1997. An approach to describing

ecosystem performance "through the eyes of salmon”. Canadian Journal of Fisheries

and Aquatic Sciences 54: 2964-2973.

Mobrand Biometrics, Inc. 1999. The EDT Method. Portland, OR, Northwest

Power Planning Council.

Moussalli, E. and R. Hilborn 1986. Optimal stock size and harvest rate in

multistage life history models. Canadian Journal of Fisheries and Aquatic Sciences

43(1): 135-141.

Northwest Power Planning Council 1997. An integrated framework for fish

and wildlife managment in the Columbia River Basin. Portland, OR, Northwest

Power Planning Council.
ISAB 2001-1: Model Synthesis Report

Appendix III: EDT - 119

Categorical

conclusions about

Environmental

Attributes by baseline

scenario (current and

historic), reach, and

season

"Biological Rules"

(survival response to

deviations from the

optimum by specieslife stage)--"Training"

the Data Translator

"Data

Translator"

Biological Life history by species

Data and Info.

Trajectory

generator

Environmental

Data and Info

STORED

INFORMATION PROCESSING

INPUT

PROCESSING

Sensitivity Analysis

identifies key assumptions

"Benchmarks"

(optimal survival

conditions by

species-life stage)

Spatial network

"routed reaches"

Population

Structure

Identification of

population components

by location and time of

spawning

Alternative

Definitions

Strategy/Action Library

Scenario

generator

Trajectory

Computations

Trajectory

"integration"

TEMPORARY DATA

Survival Data Table

Survival Landscape

(19 survival rated

attributes/record)

7,500 reaches

12 months

20 life stages

10 species

2 baseline scenarios

36 million records

Trajectory Table

(100 populations)

1000 traj/pop

200 records/traj

20 million records

 Performance Table

Trajectory Profiles

(100 pop)

1000 traj

2 baseline scenarios

200,000 records

Population Table

100 pop

10 Alternatives

1000 records

Output

Reports

Geographic

Data and Info.

The Ecosystem Diagnosis and Treatment (EDT) method

Biological

Data and Info.

Biological

Data and Info.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 120

CRITFC Cohort Model Answers

Submitted by: Jeff Fryer, Phil Roger

1.) What is the purpose of your modeling effort? What questions or problems were

your models designed to address?

a. A communication and education tool

The cohort model has an easy-to-use interface designed to allow users to

test salmon recovery options. The user can easily manipulate harvest

rates, predator control efforts, smolt transportation rates and assumptions,

habitat restoration efforts, as well as environmental variables. The model

allows users with all levels of knowledge of salmon issues to design

salmon recovery scenarios and determine their result. Graphical output is

provided giving the resulting population growth rates under baseline

conditions, user-modified conditions, and user-modified conditions with

optimum habitat, hydro, and no harvest scenarios.

b. Stimulate users to provide additional information if they disagree with part of the

model

The data and equations used by the model are available to the user. If the

user disagrees with the either the data or equations, the user is welcome to

supply their own, which could then be incorporated into the model.

c. Demystify models and provide wider user access to quantitative tools

By providing an easy-to-use interface, making the data and equations

readily available to the user, and also providing references for data values,

we have provided a model that we hope will be widely used and inspire

some level of confidence in its results among users.

d. Provide the essence of several complex tools in a single package

1. PSC/PFMC ocean harvest analyses

2. TAC in-river harvest

3. PATH downstream passage

4. EDT subbasin habitat

e. Allow users to do their own trade-off analysis between the four “H’s”

The user can manipulate parameters affecting each of the “4 H’s”.

Population growth rates are calculated successively for the user-selected

scenario with optimum subbasin habitat, no harvest, and under an “optimal

hydro” scenario consisting of the breaching of the lower Snake River

dams plus a drawdown of John Day reservoir. This provides some

measure of the impact of three of the four “H’s” on the scenario selected

by the user.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 121

2.) Summarize the major conclusions of your modeling effort relative to the four H’s:

habitat, harvest, hydropower, and hatcheries. Specifically, state your model’s

conclusions relative to the following:

OUR MODEL IS STILL UNDERGOING DEVELOPMENT AND TESTING. WE

HAVE NOT REACHED FINAL CONCLUSIONS ON THE QUESTIONS POSED. THE

FOLLOWING STATEMENTS ARE PRELIMINARY AND SUBJECT TO CHANGE.

a. The efficacy of dam breaching or drawdown to natural river levels for delisting

ESA species and restoring diverse and productive populations of native fishes

throughout the Columbia River Basin

We have modelled only an aggregate of the Middle Fork Salmon R. spring

chinook populations in the Snake R. To the extent this aggregate is

representative of all Snake River spring chinook populations, we draw the

following conclusions:

1. Spring chinook naturally reproducing population growth rate is below

replacement and cannot be rebuilt without dam breaching.

2. Even with a high rate of supplementation, it is difficult to achieve a

population growth rate above replacement without survival gains from

the other H’s.

3. Survival gains from other H’s are needed to allow the population to

rebuild, even with dam breaching

4. Breaching 4 Snake River dams does nothing to help populations

outside the Snake River (except possibly by improving flows, which

we have not modelled).

5. Drawing down John Day to spillway crest produces some benefits for

all stocks originating above that point.

b. The efficacy of hatcheries for delisting ESA species and restoring diverse and

productive populations of native fishes throughout the Columbia River Basin?

Consider hatcheries that mitigate for lost habitat and those used to supplement

delisted stocks.

We have considered only the demographic effects of hatcheries at this

time. From this perspective, hatcheries are a very effective tool for

maintaining and rebuilding existing populations and as a source for

reintroducing salmon into areas from which they have been extirpated.

c. Allocation of harvest and harvest levels needed to delist ESA species and restore

diverse and productive populations of native fishes throughout the Columbia

River Basin?
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 122

Even total elimination of all harvest, by itself, is not sufficient to maintain

or rebuild spring chinook populations in the Snake River or above Priest

Rapids dam.

d. The efficacy of habitat improvements for delisting ESA species and restoring

diverse and productive populations of native fishes throughout the Columbia

River Basin?

Other than EDT, this is the only model that attempts to represent the

effects of improving habitat on population growth.

e. Other conclusions?

The relative proportions of the H’s needed to recover spring chinook

populations varies in important ways in different areas of the Columbia

Basin. For example, breaching Snake River dams benefits Snake River

populations, but is ineffective for recovering other populations. Habitat

improvements become more effective as one moves downriver and the

effects of the hydrosystem decreases. Maintaining some populations

appears impossible without using artificial production technology.

3.) What kinds of information or data are needed to run your model?

The data required to run the model and the associated documentation, and the

equations used, are all accessible through the model. The Cohort Model is an age based

predictive model based on the work of Thompson and Bell (1934) as described in

Sanders (1995) that proceeds successively from youngest to oldest age classes. Salmon

life history is broken into a series of stages with the number of fish leaving one stage

becoming the input for the next stage. Instantaneous mortality rates are calculated for the

various mortality factors for each life stage and then applied to determine the number of

fish entering the next stage. The data needed are those necessary to calculate the

instantaneous mortality rate at each life stage. This data includes subbasin habitat data

such as temperature and sediment data (or stream condition data which is transformed

into these values), mortality rates at various life history stages (such as overwintering,

estuary/early ocean, and annual ocean mortality rates), as well as harvest rates.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 123

4.) What are the strengths and weaknesses of your model?

Strengths:

a. Runs very quickly on a mid-range PC

b. Easy for new users to use

The model has a simple user interface allowing anyone to run the model.

We are working on making the model available via the Internet.

c. Structure, data sources, and assumptions are very transparent

All are easily accessible to the user running the model. Data can be easily

modified by making changes to a Microsoft Access database.

d. Mimics the general behavior of several more detailed models in a single

package

e. One of the few models to include the potential effects of habitat change on

population growth

Weaknesses:

a. There are no density dependent effects.

b. There is no stochasisity

c. All modifications are instantaneous-there are no time lags built in for

hydrosystem or habitat changes.

d. Population sizes are not projected into the future.

e. The effects of changes in flow are not modeled.

5.) What are the assumptions of your model?

Major assumptions:

• Spawning and rearing are evenly distributed throughout currently used habitat

regardless of habitat quality.

• Habitat improvements are applied first to the lowest quality habitat.

• Habitat can only be restored to “good” condition.

• A per-dam mortality rate of 17%.

• John Day drawdown reduces mortality by 1⁄2 of a dam.

• Hatchery fish upon release are treated as being identical to natural origin fish with

the exception of a 50% post release mortality applied to the hatchery fish

6.) How does your model address uncertainty?

Uncertainty is not addressed except to allow the user to specify what the user

thinks are the correct values. The user can therefore conduct a sensitivity analysis

on variables that are of concern.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 124

7.) All models make predictions. Why do you think your model’s predictions are

accurate?

Population growth rates from the model for spring chinook are of similar magnitude

to observed rates estimated independently by others (Petrosky, C.E. et al. 1995).

8.) How does your modeling effort relate to or contrast with the other three modeling

efforts?

• We have attempted to represent the general behavior of PATH, PSC, and TAC

analyses without much of the complexity and detail.

• We intend to compare the subbasin behavior of our model with EDT. The

original intention was to see if we could use EDT smolt output as the input to

our model. If successful, one might then call this model “EDT lite”.

• We have not compared our efforts to the CRI model. We are beginning this

effort with local NMFS staff. Analytically, there seems to be much similarity

between our methods.

9.) What advice would you give decision-makers on how they should use your model to

support decisions regarding salmon recovery in the Columbia River Basin?

• This model can be an effective tool for teaching local watershed councils

about the impact of conditions outside individual subbasins upon local

restoration efforts.

• The model can be used for broad scale trade-off analyses across the 4-H’s.

Additional detail can be obtained by using more detailed models, if needed.

For example, our model could be used to approximate the amount of benefit

possible from habitat improvements, followed by a local EDT analysis to

determine which sets of actions would be most effective within a subbasin.

Literature Cited:

Petrosky, C.E., H.A. Schaller, and R.C. Beamesderfer. 1995. Spawner - recruit

relationships for spring and summer chinook salmon populations in several Columbia

and Snake River subbasins. Unpublished.

Sanders, M.J. 1995. Introduction to Thompson and Bell yield analysis using

Excel spreadsheets. FAO Fisheries Circular 895. Rome.

Thompson, W.F. and F.H. Bell. 1934. Biological statistics of the Pacific halibut

fishery. 2. Effect of changes in intensity upon total yield and yield per unit of gear. Rep.

International Fishery (Pacific Halibut) Commission.
ISAB 2001-1: Model Synthesis Report

Appendix III: CRITFC Cohort - 125

________________________________________

\\nwppcu1\fish\em\ww\isab\1 isab final reports\isab 2001-1 model synth.doc