The Science

Consistency Review

A Tool To Evaluate the Use 

of Scientific Information in

Land Management

Decisionmaking

United States

Department of

Agriculture

Forest Service

FS-772

September 2003

The Science

Consistency Review

A Tool To Evaluate the Use 

of Scientific Information in

Land Management

Decisionmaking

United States

Department of

Agriculture

Forest Service

FS-772

September 2003

James M. Guldin, David Cawrse, Russell Graham, Miles

Hemstrom, Linda Joyce, Steve Kessler, Ranotta McNair,

George Peterson, Charles G. Shaw, Peter Stine, Mark

Twery, and Jeffrey Walter

Guldin is project leader, Southern Research Station, U.S. Department of Agriculture Forest Service,

Hot Springs, AR; Cawrse is resources staff officer, Shoshone National Forest, Cody, WY; Graham is

research forester, Rocky Mountain Research Station, Moscow, ID; Hemstrom is research ecologist,

Pacific Northwest Research Station, Portland, OR; Joyce is project leader, Rocky Mountain Research

Station, Fort Collins, CO; Kessler is staff director for Wildlife, Fish, Rare Plants and Rangeland,

Region 9 Regional Office, Milwaukee, WI; McNair is forest supervisor, Idaho Panhandle National

Forest, Coeur d’Alene, ID; Peterson is project leader, Rocky Mountain Research Station, Fort

Collins, CO; Shaw is national program manager for pathology research, Office of the Deputy Chief

for Research, Washington, DC; Stine is research biogeographer and program manager, Pacific

Southwest Research Station, Sacramento, CA; Twery is project leader, Northeastern Research

Station, Burlington, VT; Walter is district ranger, Clackamas River Ranger District, Mt. Hood

National Forest, Estacada, OR.
Abstract

The paper outlines a process called the science consistency review, which can be used to evaluate the use of

scientific information in land management decisions.

Developed with specific reference to land management

decisions in the U.S. Department of Agriculture Forest

Service, the process involves assembling a team of

reviewers under a review administrator to constructively

criticize draft analysis and decision documents.

Reviews are then forwarded to the responsible official,

whose team of technical experts may revise the draft

documents in response to reviewer concerns. The

process is designed to proceed iteratively until reviewers are satisfied that key elements are consistent with

available scientific information. Variations of the science consistency review have been applied elsewhere,

but this paper represents the first effort to standardize the process for application to decisions within the

agency.
iii

Preface

The Forest Service has a long history of using 

scientific information to support forest management.

Given the increased complexity of scientific information

and of management decisions especially on national

forests, additional tools can help land managers and

planners ensure that decisions about management of

national forests are consistent with the best available

science. There are several ways to accomplish this 

consistency, such as by using science consistency

reviews (SCRs), peer reviews, or science advisory boards.

This document provides specific guidelines for conducting

SCRs of national forest decisions. 

Formal SCRs have been conducted on several national

forest management decisions, such as those involving

the Tongass National Forest Plan and the Sierra Nevada

Framework. Those reviews, as well as descriptions of

the SCR process, have been published in several Forest

Service publications and technical journals. Until now,

however, decisionmakers have had no standardized

process to follow in conducting SCRs. To address this

limitation, the agency’s National Leadership Team

convened a work group to develop guidelines for the

conduct of SCRs that built on lessons learned from

previous reviews, were scalable to the amount of effort

appropriate to the decision being considered, and were

practicable. The work group developed draft guidelines,

which were then extensively reviewed and pilot tested. 

Decisionmakers should use these guidelines sparingly,

when appropriate, and only after careful consideration

of the cost of the review relative to the expected benefit.

The expectation is that formal, documented SCRs will

be used infrequently and only when the additional

level of thoroughness is judged necessary to ensure

that decisions are consistent with the best available

science. Until the Forest Service is more experienced

in the use of SCRs, managers should consider deciding

for themselves whether to implement a formal SCR.

In those limited cases where the SCR is appropriate,

decisionmakers should expect that it will be helpful 

in ensuring that the responsible official considers the

appropriate science and that the decision made is

consistent with that science. The use of these reviews

should also enhance the public’s recognition during

appeals and litigation that science factors have been

carefully considered and clearly documented. In those

limited cases where a documented SCR is prepared, 

it may actually reduce overall decisionmaking costs

and improve timeliness.
iv
Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Definition and Purpose of the Science Consistency 

Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

SCR Participants and Their Roles . . . . . . . . . . . . . .5

Administration of the SCR . . . . . . . . . . . . . . . . . . .7

The Conduct of the Review . . . . . . . . . . . . . . . . . .15

Science Consistency Review Products . . . . . . . . . .19

Implementing the SCR Process . . . . . . . . . . . . . . .21

Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . .23

Appendix 1—Frequently Asked Questions . . . . . . .25

Appendix 2—Lessons Learned . . . . . . . . . . . . . . . .31

1

Introduction

Concern over Federal land management decisions has

grown in recent years. Public debates over activities

on Federal land have been contentious, especially

regarding the management of national forests. Decisions

on the management and use of national forest lands

are based on many different considerations and values.

Although Federal land managers can make choices

concerning how to balance the various risks and

tradeoffs involved, their decisions face questions from

the public about whether the management direction

and its associated effects, outcomes, and outputs are

appropriate. 

Increasingly, this scrutiny centers on the question of

whether appropriate and relevant scientific information

has been considered in reaching a particular decision.

Decisions are made from an array of choices available

to policymakers and land managers, and scientific

information is critical to understanding and informing

those choices. Those with an interest in the outcome

of land use decisions generally support the notion that

analysis and decision documents should be consistent

with available scientific information.

The responsible official can choose from a number of

methods to evaluate the scientific information before

making such decisions. Examples include scientific

peer review, science advisory boards, science consistency

reviews (SCRs), or expert opinion. The purpose of this

white paper is to outline a recommended process for

the SCR, a formal review in which scientists or other

knowledgeable individuals are invited to assess the

pertinent scientific information and to determine 

how that information has been synthesized in the 

documentation of a given decision. Discussion centers

on several topics: 

1. the definition of the SCR and its application in 

decisions,

2. the roles and relationships of the participants, 

3. actions to be taken before the SCR begins,

4. the conduct and iterative nature of the review, and

5. a description of the products of the SCR.

The development of these guidelines does not represent

an implicit preference or agency endorsement of the

SCR approach over peer review, science advisory boards,

or other methods. Agency leaders believe that the SCR

process has value in certain situations, and they have

been concerned that no guidelines were available for

its application.

These guidelines, then, were developed to standardize

the SCR process and to guide land managers and others

who choose to employ it. Special attention was given

to ensuring the guidelines:

• Incorporated existing agency experience with science

consistency evaluations. They reflect the lessons 

learned in antecedent efforts such as the Tongass 

Land Management Plan Revision, the Interior 

Columbia Basin Final Environmental Impact 

Statement, the Sierra Nevada Framework for 

Conservation and Collaboration, and the Northern 

Great Plains Land Management Plan Revision. 

• Can be used for small or large decisions, for situations

of little or great complexity, and with emphasis on 

practical application and real-world constraints.

• Were peer reviewed by scientists, land managers, 

and administrators, both inside and outside the 

agency; their constructively critical comments led 

to modifications and improvements in the process.

• Were field tested on a small project on the Shoshone

National Forest, with the help of the Rocky Mountain

Research Station, which also resulted in refinements

of the process.

Thus, the guidelines reflect current thinking by both

National Forest System (NFS) and Research and

Development (R&D) personnel on the best way to

implement the SCR.

3

Definition and Purpose of the Science

Consistency Review

A science consistency review is defined as the

process used to determine whether an analysis or 

decision document is consistent with the best available

science. That review is accomplished by judging

whether scientific information of appropriate content,

rigor, and applicability has been considered, evaluated, 

and synthesized in the documents that underlie and

record land management decisions. Those documents

include both the analysis documents (environmental

assessments or environmental impact statements) and

the decision documents (decision memo, decision

notice, or record of decision). 

The SCR process is proposed as an optional tool for

agency officials to use in considering the adequacy

and thoroughness of scientific information supporting

land management decisions within the Forest Service.

The basic element of the SCR process is the review of

a draft document that asks and answers the following

four questions:

1. Has applicable and available scientific information 

been considered?

2. Is the scientific information interpreted reasonably 

and accurately?

3. Are the uncertainties associated with the scientific 

information acknowledged and documented?

4. Have the relevant management consequences, 

including risks and uncertainties, been identified 

and documented?

The overall SCR process encompasses actions before

the reviews begin, the conduct of the reviews, the 

revision of the draft document in response to reviews,

and the finalization of the decision. Figure 1 highlights

key elements and the iterative nature of the process. 

When carried to completion, the process will result 

in a final SCR report that land managers can use to

document that a decision is consistent with the best

available scientific information. The steps taken to

arrive at that final report, however, will be the most

important. Interim SCR reviews will highlight elements

that are not consistent with available scientific information,

and draft analysis and decision documents will be revised

accordingly in response to the interim review. Thus,

the end of the SCR process is almost anticlimactic—if

the final SCR report declares all elements to be fully

consistent, the responsible official then can assert

that a decision is consistent with the best available

science.

The SCR process has a number of other advantages as

well. Participants in the process are expected to define

the extent and scope of the scientific information related

to the particular document under review. They can

determine what scientific information might be applicable,

and judge its quality and usefulness. They can identify

and quantify the expected risks and consequences from

implementing the proposed action or the alternative

RA issues SCR report

RA issues final SCR report

RO revises draft document

Conduct the SCR

Interim

or final?

Initiate the SCR

Before the SCR begins

Review draft document

Final

RO issues the decision

Figure 1. Flowchart outlining the SCR process.

Interim
4

actions. Lastly, they must evaluate whether the effects

of implementing the proposed and alternative actions

are consistent with what would be inferred from the

scientific literature and expert judgment, and whether

those effects are reasonably interpreted and disclosed

in the reviewed document.

The final SCR report gives the decisionmaker a document

to justify that the information on which his or her

decision is based is consistent with available scientific

information. The final report discloses what elements

were evaluated and the criteria by which they were

evaluated for science consistency. It provides a product

in the form of a rating document disclosing the results

of the evaluation. Together, these elements enable land

managers and scientists to deal proactively with any

concerns, either inside or outside the agency, that some

body of relevant science might not have considered in

the decision.

The SCR process is used to support or enlighten a

decision, not to make one. SCR participants are not

authorized to validate, ratify, or make a decision; that

responsibility rests solely with the responsible official.

Neither the SCR reports nor the participants who 

prepare them should make judgments about the balance

managers achieve among competing objectives or what

levels of risk they should take. The reports must not

drive the decision process to a particular end, nor

should they make specific recommendations regarding

actions. When prepared properly, the SCR does not

advise a decisionmaker for or against a particular

course of action.

Moreover, just as the SCR is not intended to allow

reviewers to usurp the decisionmaking authority of

the responsible official, neither is it intended to make

reviewers responsible for the decisions or outcomes

that result from those decisions. Ultimately, the

responsible official must carry the weight of whatever

decisions are made.

Finally, the SCR process should not substitute 

for other established processes that may be more

appropriate. For example, the SCR is the wrong 

tool to use to evaluate assessments, bibliographies,

"state-of-the-art" papers, or scientific papers1

. These

should be reviewed using the scientific peer review

process rather than the SCR process. Having peer

review processes in place throughout the planning

process would aid in the science evaluation of the

draft analysis or decision document, but the peer

review process is separate from the SCR review as

described in this white paper.

1 However, this is not to say that SCR reports are independent

of the scientific literature. On the contrary, if a draft document

being reviewed by the SCR reviewers fails to include an

assessment or scientific paper that the reviewer thinks is

germane to the draft document, he or she must direct the

technical experts to that source.
5

SCR Participants and Their Roles

Responsible Official

The responsible official is the person in the Forest

Service hierarchy who is ultimately responsible for

making and executing the land management decision.

Typically this person is a regional forester, forest

supervisor, or district ranger. The responsible official

makes choices related to selecting a preferred alternative,

balancing risk, and considering tradeoffs associated

with ecological, social, and economic outcomes and

effects. Although the responsible official may not be

directly involved in the SCR review itself, he or she is

responsible for the content of the draft document

under review, how the feedback from the review is

considered, whether the feedback is incorporated into

revisions of the draft document, and how the decision

is made. 

Technical Experts

Technical experts are those individuals participating

directly or indirectly in the development of draft analysis

or decision documents subject to review. Often technical

experts are interdisciplinary (ID) team members; typically

they are professional employees, and they may or may

not have advanced academic degrees. Generally, the

technical experts on the ID team will directly review,

reference, and incorporate information from science

literature, expert opinion, or the science assessment,

if applicable, into the draft documents. They will also

be responsible for modifying draft documents based

on feedback in the SCR report.

Reviewers

Reviewers are the experts who actually perform the

SCR. A reviewer must have scientific credibility in the

field but not necessarily an advanced academic degree.

In some cases, a person with local or traditional

knowledge may be appropriate. Reviewers can be R&D

scientists; university faculty members; scientists with

State agencies, tribes, or other Federal agencies; or NFS

staff or technical experts not attached to the particular

project or unit under review. Reviewers can also be

scientists or experts from private companies, consultants,

or nongovernmental organizations.

Above all, reviewers must be seen as being fair and

impartial. They must not inject their own values or

preferences into their judgments. They must confine

themselves to technical judgments of scientific 

consistency as it is defined. It is probably impossible

for a reviewer to be completely objective in reviewing 

a draft document, but he or she should reduce the

amount of subjective elements used as much as 

possible. Though it may be difficult for some reviewers

to separate technical evaluations from their own values

or policy beliefs, doing so is critical to the impartiality

of the SCR report. 

For issues that are small in scope (as defined by 

geographic area, subject matter, or complexity), the

reviewers may act independently. For issues of larger

scope, the reviewers may need to have more interaction

as a team, potentially with a designated team leader.

Having reviewers work together, especially on complex

issues, may help sort out elements that have 

interdisciplinary focus.

Review Administrator

The review administrator is responsible for the conduct

of the review and any interim or final SCR reports. The

review administrator may be a R&D station director 

or a subordinate administrator or scientist designated

by the station director. Alternatively, the review

administrator may be a regional office staff director or

a staff director from another forest. He or she might

also be an appropriate research administrator outside

the agency, such as a dean or department head of a

land grant university identified in consultation with

the agency. 

For a given SCR, the review administrator is identified

by R&D leadership with input from the responsible

official. Whether the prospective review administrator

is inside or outside the agency, he or she should be

willing and able to serve. The review administrator’s

supervisor will need to concur with this selection as

well. Selection of the review administrator must be

made with sensitivity to the contention expected in 

the documents to be reviewed, the desire to maintain

appropriate scientific independence from the draft

documents, and the need to address concerns that

might arise over any perceived bias. 

Although the review administrator is responsible for

the SCR, in order for the process to be successful, he

or she should establish a collaborative relationship

with the responsible official. The review administrator

will perform the following:

• identify candidate reviewers, 

• select reviewers and negotiate their availability and 

schedules, 

• disseminate the materials to be reviewed to the 

review team, 

• coordinate the review itself, and 

• address disagreements among different reviews or 

reviewers. 

For a small-scale review, the review administrator may

be one of the reviewers, working either alone or as a

member of a small team. The review administrator

may have broader authority to direct the review team

and work with a separate review team leader. For more

extensive reviews, the review administrator summarizes

the reviews for the responsible official. Regardless 

of the scale of the project, the review administrator 

is responsible for preparing the SCR report and 

transmitting that report to the responsible official.
6

Others Involved

The broader public already has a number of opportunities

to comment on the technical aspects of draft documents

and their supporting materials. Typically, members of

Congress, advocacy groups, and the public, including

individuals and nongovernmental organizations, can

be involved in a decision through formal comment 

on draft documents (such as their review of a draft

environmental impact statement. The responsible official,

with the review administrator’s concurrence, may

choose to have any applicable SCR reports available

for public examination before a decision is issued.
7

Overall, the responsible official decides whether to 

initiate the SCR, and is responsible for providing the

review team with the documents for review. The review

administrator is then responsible for the review process

and reports, and the responsible official is responsible

for all modifications of the documents under review.

The SCR process is most effective if agency officials

work collaboratively rather than confrontationally. 

Occasionally, a R&D administrator may be particularly

interested in the use of science in a draft analysis or

decision document that could be clarified using the

SCR process. This situation should be addressed 

collaboratively since it is normally the responsible 

official who initiates and funds the SCR. An expert

may unilaterally initiate a technical or scientific review

of an analysis or decision document without the

responsible official’s collaboration or funding; but in

that event the responsible official reserves the right to

apply the SCR report or not, at his or her discretion. 

If strong disagreements cloud the decision to engage

or terminate the formal SCR process as described, 

the next level of line authority should resolve the 

disagreements. 

Both the responsible official and review administrator

should maintain appropriate administrative records

that document the initiation and conduct of the SCR

process. For example, the review administrator would

file letters that transmit the interim or final SCR reports;

the responsible official would file letters transmitting

the revised draft analysis or decision documents for

the subsequent iterative review. At the conclusion of

the SCR process, these letters would become part of

the administrative record of the decision.

The composition and structure of the team working on

an SCR is critical. These people represent the credibility

of the process to the scientific community and the public

at large. How this team is selected and managed has

significant bearing on the results of the process. The

responsibilities of the key participants at various stages

of the process are shown in table 1, and the overall

process for conducting the review is listed in table 2.

Discussion on each step follows.

Before the SCR Begins

The initial contact regarding the possibility of 

conducting an SCR should be made between the

responsible official and the appropriate R&D person2

,

because station and region leadership should be

aware of any SCR taking place within their respective

jurisdictions. That initial contact enables the responsible

official to review the various alternatives available for

evaluating the scientific information for the proposed

decision, and whether the SCR process is the best 

tool available to support his or her decision. Until 

a standard process is in place, informal connections

among managers and R&D scientists are the most

effective means to identify the need for an SCR. 

Many factors enter into the decision to choose the SCR

process rather than peer review, science advisory boards,

or other methods. Table 3 provides a conceptual

framework that summarizes the appropriate 

considerations and provides examples of the specific

risks and benefits of using an SCR. If a decision falls

on the higher end of the risk scale, conducting a

stand-alone SCR might not be warranted.

Two economic questions can also guide the responsible

official in deciding to choose the SCR: 

• Are the returns associated with the potential outcomes

from the decision greater than the costs of conducting

the SCR? 

• Is the cost of the SCR not going to be greater than 

the cost of making a wrong decision? 

There will be times, however, when the consequences

of not conducting an SCR are outweighed by its benefits,

despite the financial costs. If the controversy associated

with a decision is expected to be intense, a responsible

official may decide that the SCR process must be

engaged regardless of its cost, for reasons of legal

defensibility, public credibility, or sustainability. This

might include situations where the processes affected

by and involved in land policy, planning, management,

or decision alternatives are so complex that a high

degree of controversy results regardless.

Triggering the SCR Process

If the responsible official decides to engage the SCR

process, the review administrator becomes responsible

for its conduct and completion. Accordingly, the first

step is to identify the review administrator. If the review

administrator works for R&D, a letter from the supervisor,

station director, or deputy chief of R&D would be

appropriate; similarly, if the review administrator

comes from within the NFS, line authority to assign

the task would derive from the supervisor, regional

forester, or deputy chief of the NFS. A review 

administrator from outside the agency should 

have the support of his or her supervisor as well.

Table 4 (a and b) illustrates how using the SCR might

benefit the forest planning and project planning

processes. The SCR can assure the responsible official

that the best available science was used appropriately

at that step and that it will contribute to focusing subAdministration of the SCR 

2 One way to proceed with this is to establish a point of

contact within each R&D research station, and within the

Washington Office R&D staff, to handle requests or referrals

for SCRs of a regional or national scope, respectively. For

example, within the Southern Research Station, a logical

point of contact would be the Assistant Station Director for

Planning and Applications.
8

Table 1. Roles of participants in an SCR, in order of performance

Participant

Role

Identifies need for SCR*

. XX X** X

Initiate the SCR. XX X

Establish scope of the SCR. Develop budget; identify sources XX XX

of funding. Develop timeline. Establish standards for 

interaction between participants and public. Establish 

criteria for selection of reviewers.

Conducts assessments and environmental analysis. XX

Develops alternatives and effects analysis. Documents 

processes. Conducts interdisciplinary team meetings. 

Prepares documents for SCR review.

Establish administrative structure for review team. X XX

Select reviewers.***

Develop elements to be evaluated. X XX X X

Approve elements to be evaluated. X XX

Convenes reviewers; administers the review. XX

Performs the review. XX

Provides reviewers with requested documentation; XX

maydevelop additional documentation as needed.

Approves the interim/final SCR report. XX

Ensure communication between technical experts XX XX

and reviewers.

Ensure that interim or final SCR report does not X XX

advocate any particular decision and that scientific 

information andconsistency evaluations are value neutral.

Reconcile reviews. X XX

Review and comment on SCR report. XX X

Directs modification of document based on interim SCR report. XX

Modify document using interim SCR report. X XX

Responsible for and approves final SCR report. XX

Responsible for management decision. XX

Responsible Technical Reviewer Review

Official Expert Administrator

XX = primary responsibility

X = secondary responsibility

* SCR may be conducted on both analysis documents and

decision documents, such as drafts of an Environmental

Assessment or Environmental Impact Statement or a Decision

Notice or Record of Decision.

** A technical expert may request the responsible official to

initiate an SCR if the technical expert believes that areas of

their expertise would benefit from an unbiased review.

*** There is a relationship between the selection of reviewers

and the selection of elements. Identification of additional 

elements during the process may necessitate additional or

different reviewers.

  Modifications in the draft document may be subject to an

additional review by the reviewers (the iterative process).

Iterative steps for interim and final SCR reports** **
9

Table 2. Suggested process for conducting a formal SCR.

Initiating the review:

1. Initial contact between responsible official and R&D contact.

a. Explore alternatives for evaluating scientific information in the proposed decision.

b. Decide whether an SCR is the best tool to apply in the given case.

c. Select the review administrator, if responsible official has elected to use the SCR.

2. Letter from responsible official to review administrator asks for review.

a. Prepare a written charter, memorandum of understanding, or other authorizing document.

b. Establish the scope, content, review elements, and sideboards for the review.

c. Forecast the expected timeframe for the review.

d. Clarify the openness of the review among participants and with the public.

e. Determine how to pay for the SCR.

3. Review administrator establishes review team.

a. Consult with responsible official and other domain experts to find appropriate reviewers.

b. Establish administrative structure for the review team.

4. Review administrator and responsible official issue the charge to the review team.

a. Introduce team charter, documents to be reviewed, and appropriate supporting materials.

b. Reinforce the need for impartiality on part of reviewers.

Conducting the review:

1. Selecting the review elements.

a. Technical experts provide initial list of elements from document to reviewers.

b. Technical experts, reviewers, responsible official, and review administrator review the proposed elements.

c. Revised list of elements and justifications are returned to review administrator for final approval.

d. Elements should be written so they can be answered with "yes" or "no" under the evaluation criteria.

2. Scoring elements against criteria.

a. The elements and criteria form a matrix of rows and columns.

b. Each reviewer records a yes or no rating in each cell of the matrix.

c. Narrative justification statements must accompany ratings by which reviewers explain their evaluations and

offer suggestions for changes that would achieve consistency.

3. Reconciling reviews.

a. Reviewers forward the finished reviews to the review team leader or review administrator.

b. The review administrator reconciles variations in review content.

c. The review administrator collates the reviews and prepares a summary SCR report for the responsible official.

4. Preparing the review report.

a. Draft or interim reports are anticipated as part of an iterative process that may result in a revised document

that needs an additional iteration.

b. The review administrator may conduct or request a review of the final team report to ensure that the report 

does not advocate any particular decision or alternative, and that the evaluations are value neutral.

Completing the review:

1. The responsible official and review administrator jointly make the ultimate decision that the process is complete.

2. The interim or final report is transmitted from the review administrator to the responsible official. 

3. Responsible official directs technical experts to refine the draft document based on the SCR report.

4. If necessary, the revised document is returned to the review team for further review.

5. The process concludes when the final SCR report and the decision are issued by the review administrator and 

responsible official, respectively.
10

Table 3. A conceptual framework that the responsible official should consider in making the decision regarding

whether to conduct an SCR, the amount of effort to expend, and the time needed for an SCR 

Potential Effects* (Risks & Benefits)

Factor To Consider

Spatial and temporal scales • Localized site conditions • Broad geographic ranges and

• Small watershed multiple agencies

• Transcend organizational

boundaries

Duration of effects • Short-term effect on communities, • Long-term effect on communities,

economy, and/or environment economy, and/or environment

Scope of decision • Routine management actions • Large-scale regional and forest

(site-specific) plans or plan amendments

State of knowledge • Well-developed routine analysis • Emerging science and technology

• Professionally recognized science • Disputed findings and 

finding interpretations

Data availability • Well developed • Data gaps

• Generally accepted • Arguably insufficient data

• Associated risk small • Highly disputed

Scope of effects • Limited effect on or change to • Long-ranging associated risks to

communities the environment

Controversy • Limited in scope and action • Highly disputed and/or arguably 

insufficient data

Low (SCR probably not needed or High (SCR may be helpful and

met by smaller effort) may require larger effort)

Table 4a. Benefits that might be obtained from initiating the SCR at various steps in forest planning.

* Planning directives requires use of best available science at these steps.

Step in the Forest Incremental Benefits of an SCR

Planning Process Decision Made

Current condition* Identify resource condition Assess the adequacy and accuracy of the information on

and trends. condition and trends for the resources of interest.

Adequacy of existing Establish decisions to be Assess the reality of the desired future condition.

plan direction to achieve made in the plan revision.

the desired future 

condition

Proposed action Establish initial proposal Assess the potential of proposed action to move resource

for actions to move resource conditions toward desired future condition.

conditions toward desired 

future condition. 

Issue identification Identify the significant Assess the scientific basis for the issues and the potential

issues to be addressed in effects of the proposed action.

the Environmental Impact 

Statement.

* The term "effects" refers to risks, benefits, outcomes, and outputs.
11

Table 4b. Benefits of the SCR in project or other planning.

Step in the Forest Incremental Benefits of an SCR

Planning Process Decision Made

Alternative development Identify alternative actions 1. Assess whether there is a scientific basis that states

that meet the purpose and whether the alternatives address the range of significant

need, and address the issues.

significant issues. 2. Assess if the alternatives will move resource conditions

toward the desired future condition.

3. Identify applicable, but missing, information (e.g., 

assessments, literature, databases, traditional 

environmental knowledge, etc.)

4. Assess if assumptions used in alternative development

are adequately documented.

*** SCR review of analysis reports can ensure that the data, methodology, and conclusions are scientifically credible. SCR review

of monitoring reports can ensure that the data are being properly interpreted, monitoring plans use current methodology, and the

plans are designed to collect pertinent data.

Step in the Forest Incremental Benefits of an SCR

Planning Process Decision Made

Identify purpose and need Establish that there is a Assess the adequacy and accuracy of the understanding

need to take action. of the condition and trends for the resources of interest. 

Proposed action Establish the proposed Assess the potential of the proposed action to address

actions to address the the identified need.

identified purpose and need.

Issue identification Identify the issues to be Verify the scientific basis for why the issues are potential

addressed in the NEPA side effects to the proposal.

process.

Alternative development Identify alternative actions 1. Assess whether there is a scientific basis that the

hat meet the purpose and alternatives address the range of issues.

need and address the issues. 2. Assess if the alternatives will move resource conditions

toward the desired future condition.

3. Identify applicable, but missing, information (e.g., 

assessments, literature, databases, traditional 

environmental knowledge, etc.)

4. Assess if assumptions used in alternative development

are adequately documented. 

Analysis** Decide whether the analysis Assess adequacy of risk portrayal.

is adequate, given the level Assess rigor of analysis.

of risk and uncertainty.

Decision Decide what actions to Assess whether or not decision rationale that is based on

implement. science is scientifically supported.

Monitoring** Choose methodology and Assess whether monitoring plan adequately addresses

protocols for assessing uncertainty involving risks or the uncertainty of effects.

implementation of the plan.
12

sequent steps of the planning process as well as the

incremental decisions made at those steps. It is not

implied that the SCR should be used at each step of

the planning process. In determining when to use an

SCR, a responsible official should weigh many factors,

such as the complexity at that step of the planning

process, the risk to resources or to the success of the

planning process if inconsistency occurs, the intent 

to use an SCR at a future step or the use of one in a

previous decision, and administrative concerns such

as cost or timing.

The best time to initiate an SCR, however, is early in

the planning process. Both the overall planning and

the SCR process will proceed more smoothly if early

versions of the draft analysis and decision documents

are reviewed and revised. Generally, the interaction

among participants should begin with a review of the

first draft of the analysis document. If the process is

initiated well after these initial draft documents have

been prepared, one should expect a more complicated

review for reviewers, and a more extensive revision by

the technical experts. 

Initiating the SCR Review

The SCR review is initiated through the responsible

official’s formal written request (on official correspondence)

to the review administrator. For the administrative

record, the responsible official must retain the documents

that resulted in the initiation, conduct, and conclusion

of the SCR.

Several factors are important to consider as the

responsible official and review administrator outline

the overall administrative context of the proposed SCR.

One suggestion to facilitate mutual understanding is

to prepare a formal written charter, memorandum of

understanding, or similar document that states how

issues are to be handled for the SCR under consideration,

especially for those SCRs that have wide scope. The

decisions requiring consideration at the outset include,

but may not be limited to, the following issues:

• The first priority is to establish the scope of the 

review, emphasizing the relevant issues that will be 

the focus of the review. The initial request from NFS

determines the general outline of the SCR, but the 

review administrator and responsible official must 

work together to decide exactly what form the SCR 

will take. This includes decisions about whether to 

review all or part of a document, whether one or 

more iterations are anticipated, the number of team

members likely to be needed, and similar sideboards.

These decisions must reflect the funding available 

for the review.

• The timeframe for the review must be established. 

The length of the time commitment required of the 

team members should also be decided early. The 

time commitment to complete such a process can 

be significant, and the responsible official and 

review administrator need to be cognizant of that 

from the start. Time constraints will play a role in 

determining who can be involved and the expected 

cost of the review. 

• The degree of openness of the SCR process among 

the participants and with the public should be 

established. The SCR process is one where technical

feedback occurs among the participants, and 

procedures must be in place so that critiques 

remain constructive. Moreover, since reviewers 

traditionally have had less contact with the public 

than technical experts, the participants’ roles in 

interacting with the public must be established. 

The time to clarify these administrative relationships

is at the beginning of the process so as to minimize 

conflict.

• Finally, a key priority is to determine how to pay 

for the SCR. Details regarding the estimated cost of 

supporting the review team and the source of funds

(specific job codes) to conduct the review must be 

agreed to in advance. Depending on the scope of 

the matter, the cost of conducting the SCR could be

high. Reviewers’ compensation must be established;

this could include salary and travel reimbursement 

for agency reviewers, and travel reimbursement and

honoraria for reviewers outside the Federal 

Government. Generally, the NFS would be responsible

for the costs involved.

Selecting the Review Team

The review administrator is responsible for selecting

the members of the review team and for establishing

an administrative structure for the conduct of the

review. Although the review administrator selects the

reviewers for the SCR, the responsible official should

be consulted regarding the appropriateness of individual

reviewers. 

Reviewers should have three attributes: expertise,

credibility, and independence from authorship of the

draft document. Reviewers must represent the breadth

of expertise needed to address the elements under

review, and should be recognized in their fields as

having sufficient experience and knowledge to speak

on a given subject. They must have credibility in their

areas of expertise, or neither their fellow reviewers nor

the technical experts responsible for the modifications

to the draft documents will regard their comments as

authoritative. Finally, independence from authorship

of the draft document is needed, so that no one is put

in the awkward position of reviewing his or her own

work. Often, a technical expert responsible for writing

part of the draft document is a local or regional expert

on the subject involved; nevertheless, any responsibility

for authorship of the draft document should exclude

one from consideration as a reviewer of that document.

This does not mean, though, that an expert who is 

frequently cited in a draft document but who was not

involved in preparing it should not review that document;

in fact, such a person may be the ideal reviewer. 

The number of reviewers on a team should relate to

the scale and magnitude of the issues involved and the

available resources. There are potential advantages for

the perceived objectivity of the interim or final SCR

report if the composition of the team represents a

cross-section of the scientific community. The availability
13

and willingness of each team member to participate

should not be disregarded. Serving as a member of an

SCR team may require a major commitment of time

and energy, and reviewers should be made aware of

the expected commitment at the outset.

If the SCR is expected to be a major task, a team leader

may be needed; for small efforts, the review administrator

may assume this role. A team leader coordinates the

activities of the review team members. The team leader

should be willing and able to lead a group of scientists.

If the team has meetings, it might also be advisable 

to have an additional person functioning as a team

facilitator; this may apply only for efforts that include

large teams and more complex tasks.

The Charge to the Team

The initial charge to the SCR team is crucial to 

establishing working practices and managing the

resulting expectations of the group. This can be done

informally or formally using a charter or other appropriate

document, but both the responsible official and review

administrator need to sign any formal document used.

The charge or charter should be delivered to the team

at the first opportunity. Reviewers should be clearly

advised that the SCR takes no position and expresses

no opinion regarding which particular alternative

should be selected. 

The draft document under review along with appropriate

supporting materials must be introduced to the review

team in an effective manner. Reviewers should be

charged with reviewing those elements in the draft

document that pertain to their expertise, using the

standard criteria for evaluation discussed below.

Reviewers must have a clear understanding of the parts

of the draft document to be reviewed, as identified by

the review administrator and the responsible official. 

The team should be directed to seek consensus as

they conduct their review, but not given a mandate to

achieve it. The team members should be able to work

independently in their areas of expertise, but the

group as a whole should be able to work together and

provide input to each other’s reviews. The concept is

parallel to the interdisciplinary team of technical experts

who assembled the documents under review. Meetings

of reviewers might be valuable but are not required for

the work to be accomplished. Team members should

allocate tasks within the group among themselves or

at the direction of the review administrator or team

leader, as appropriate, such that all elements of the

review are covered. Discussions and assignments need

to be clear and focused. Should any disagreements or

disputes arise among reviewers when the initial charge

is presented, the review administrator should resolve

them.

15

The Conduct of the Review

The Review Process

The SCR process is designed to be flexible and scalable,

especially for small analysis or decision documents at

the project or plan amendment level. It is intended to

make efficient use of reviewer time and effort. For a

project-level or plan amendment, the time and effort

for each iteration of the SCR is analogous to that invested

in the review of scientific manuscripts of equivalent

length, and the iterative process will have a limited

number of cycles. For decisions with greater scale,

complexity, or contentiousness, the SCR may require

a larger review team, a larger investment in reviewer

time and effort, and a greater number of iterations.

The core of the SCR is a team review of a draft analysis

or decision document and supporting materials. The

elements to be reviewed will depend on the particular

document under review. The evaluation criteria are

more general, however, and have been standardized as

suggested elsewhere in this white paper. Although the

elements to review will differ, the evaluation criteria

are common to reviews in any situation.

The review should be structured as a matrix by which

each element is evaluated according to each of the

evaluation criteria (table 5). In this context, the elements

can be visualized as rows and the criteria as columns.

Each cell of the matrix should have explanatory notes

written by members of the review team that explain

the evaluation. In instances where the rating is less

than ideal, explanatory notes should state what

changes would make the document acceptable. 

Completed reviews are returned to the review 

administrator, who reconciles them and prepares an

interim or final review report. That report, including

pertinent material from the individual reviews, is

returned to the responsible official for potential revisions

of the draft document or supporting material. Draft or

interim reports are anticipated as part of the process,

and may result in a revised document that needs an

additional review. Closure may require several iterations,

but throughout the process the two interests, science

and decisionmaking, must remain separate and 

independent. Revisions of the draft document are

returned to the review team for additional review. 

Any changes to the document(s), however, are at the

discretion of the responsible official.

The cycle repeats until the review administrator and

the responsible official agree that the intent of the review

has been achieved, after the iterative cycle of interim

SCR reports and modifications of analysis and decision

documents has achieved its intent. Ideally, that point

in time will have two attributes: the review administrator

will approve a final SCR report that evaluates all elements

of a draft document to be science-consistent, and the

responsible official will issue a decision based on that

draft document.

Experience has shown that the ideally cooperative

relationship between reviewers and technical experts,

and perhaps also between the responsible official and

review administrator, might fall short of a final SCR

report that evaluates all elements as consistent. Such

an outcome could occur if the number of iterations

becomes excessive, if the response to an interim 

SCR report is insufficient, or if disagreement exists 

on how to make an element consistent. Under such

circumstances, the responsible official or review

administrator could prematurely terminate the SCR

process.

Selecting the Elements

The review administrator, in consultation with the

responsible official, selects the elements to review.

Since not all elements subject to an SCR are equally

important, the responsible official and his or her technical

experts (the management side) should collaborate with

the review administrator and his or her reviewers (the

review team) to identify those elements. The management

side brings knowledge of policy considerations, and the

members of the review team (as well as the managers)

bring knowledge of technical and scientific information.

The review administrator must use both areas of

knowledge to prepare a list of elements from the 

document that would benefit from an SCR.

The decision about which elements in the draft document

should be included in the review process can be based

on results from project scoping, past experience by the

technical experts or reviewers, or based on their own

merits. The following list of selection factors can help

identify what issues may be important to a particular

situation and whether the potential elements can be

addressed by an SCR:

1. The scientific information about an element is 

important to the potential decision.

2. An element is sufficiently important to affect the 

substance of the decision. 

3. Public interest or official policy has identified an 

issue as important. 

4. Excessive conflict or uncertainty exists within 

available scientific information.

5. A potential element is relevant to the agency and 

within its authority for decisions. 

6. The information prepared for the draft document 

has sufficient scientific credibility.

An element should meet both factors 1 and 2, plus at

least one of factors 3 through 6, to merit consideration

in the review.

Elements should be drafted so they can be answered

with a categorical yes or no rating. If an element is

identified for which there is no expert on the review

team, an additional reviewer with appropriate expertise

may be required. Issues of scale are important and

are best addressed in the elements.
16

Table 5. A sample excerpt from a matrix of elements and criteria to be used in evaluating science consistency.

For each element, technical experts should provide an explanation of the purpose and thinking behind the element to improve the

reviewer’s comprehension of the element to be rated. For each rating in each cell, reviewers should provide a note explaining why

and how the rating was derived, including suggestions that could be used to improve a "no" rating.

Elements Criteria for Decision

1. Old Forests and Associated Species

1.a. Spotted owl habitat requirements 

and availability

1.b. Spotted owl population trends

1.c. Silvicultural prescriptions to create

old forest habitat 

1.d. Conservation approaches for 

Canada lynx

2. Fire and fuels

2.a. Current and future fuel loading

2.b. Current and future wildfire trends

2.c. Smoke emissions and effects on 

human health

2.d. Effectiveness of treatments for fuel 

reduction

3. Nonnative invasive species

3.a. Effects of spotted knapweed on 

native communities

3.b. Treatments to control spotted 

knapweed

3.c. Effects of zebra mussels on native 

aquatic communities

4. Economic and Social Well-being

4.a. Rural community economic and 

population trends

4.b. Role of nontimber forest products 

in local livelihoods 

4.c. Recreational and economic 

importance of off-highway vehicles

4.d. Effects of timber harvest trends on 

local communities

Is the relevant Is the scientific Are the uncertainties Are the relevant

scientific information associated with the management

information reasonably relevant scientific consequences

considered? interpreted and information identified and

accurately acknowledged documented,

presented? and documented? including 

associated risks 

and uncertainties?
17

After the elements are drafted, the review team leader

or review administrator must confirm that elements

are relevant and correct. The responsible official should

agree that each element is germane to the decision.

The review administrator, however, under his or her

responsibility for the overall SCR, has final approval 

of the list of elements for review,. For those SCRs in

which a formal charter is developed, the final list of

elements can be included in the charter.

It is conceivable that a revised document in response

to an interim SCR report would result in modification

of the elements to review. Suggestions for new elements

might come from technical experts, reviewers, the

review administrator, or the responsible official. 

Again, final inclusion of a new element in a subsequent

iterative review would be at the discretion of the

review administrator.

The Evaluation Criteria 

Although the elements are specific to issues, the criteria

by which elements are evaluated are standard for all

cases and independent of the scale of the decision. The

following criteria are the recommended set of questions

that decisionmakers should use in the context of the

issues being addressed in the planning document and

in the decision itself. 

Criterion 1: Is the relevant scientific 

information considered?

The first criterion requires that the document under

review contain a sufficiently thorough array of information

from associated scientific literature, documented case

studies, scientific assessments, or expert opinion (as

applicable) addressed to form the scientific foundation

for a decision. Typical questions that might arise during

review of this criterion include the following:

1a. Is the breadth and depth of the scientific information

in the planning documents thorough enough to 

include the scientific consensus as well as any 

contradictory or conflicting areas?

1b. Are diverse sources of information (literature, 

assessments, case studies, expert opinion) 

referenced and synthesized adequately? 

1c. Is all the scientific information used in the 

document under review credible and applicable?

In the context of the planning process, many ways can

be used to incorporate scientific information. An SCR

must address whether the document recognizes this

core information and handles areas of controversy. 

Criterion 2: Is the scientific information 

reasonably interpreted and accurately presented?

This second criterion requires that the technical experts

understand the scientific information, that the information

is used in an appropriate manner (such as in application

to appropriate regions and physiographic conditions),

and that it is presented in a manner consistent with

the assumptions and interpretations provided by the

authors of the particular scientific literature being

cited. Typical questions that reviewers might ask

under this criterion include the following:

2a. Are the assumptions in the scientific literature,

assessments, or expert opinion concerning specific

fundamental points clearly identified? 

2b. For areas of controversy, are scientific disagreements

on the issues discussed? Are differing or opposing

views selectively used or fairly presented?

2c. Are the citations accurate, credible, and 

appropriately used? 

2d. Is the scientific information correctly interpreted 

and applied for the specific conditions and 

geographic areas in the draft document? Is the 

scientific information originally formulated at 

similar scales and geographic areas? If not, what

was offered as a bridge to the planning area 

spatial scale?

2e. Is the use of theory balanced and supported by 

data? Are fundamental points in the scientific 

information based on appropriate ecological, 

economic, or social theory? 

2f. Is the use of scientific information value neutral 

(unbiased)?

2g. Are the weights given to varied sources of 

information clear and appropriate?

Criterion 3: Are the uncertainties associated

with the relevant scientific information

acknowledged and documented?

Uncertainty in scientific information represents primarily

the statistical variability associated with data and

temporal and spatial scales. Variability in disturbance

regimes and pertinent extreme events constitutes

another form of uncertainty. Transferability of sources

of information and the theoretical bases of interpretation

also could cause uncertainty. Typical questions that

reviewers might ask under this criterion include the

following:

3a. Is uncertainty in the scientific information 

acknowledged, adequately disclosed, and 

appropriately described?
18

3b. Is the uncertainty from different sources of 

information reported clearly?

3c. Is the reliability of the data acknowledged and 

documented?

3d. Are there gaps in scientific knowledge that need 

to be acknowledged and documented? Has 

incomplete or unavailable information been 

recognized?

Criterion 4: Are the relevant management

consequences identified and documented,

including associated risks and uncertainties?

The previous three criteria dealt with the scientific

information. This criterion deals with the application

of that information to potential or selected management

actions. Consequences resulting from projected 

management alternatives carry various levels of risk

that must be addressed. Typical questions that reviewers

might ask under this criterion include the following:

4a. Are relevant consequences from each proposed 

management alternative identified, described, 

and analyzed appropriately?

4b. Has scientific information been used appropriately

to estimate the projected consequences of 

management actions? 

4c. Are projected management actions stated and 

documented in such a way that implementation 

effects can be predicted? 

4d. Is the risk associated with proposed actions and 

projected consequences reasonably addressed?

4e. Are the amount and quality of scientific information

commensurate with the proposed land management

activities and the resource risks and uncertainties

associated with those activities?

Scoring the Consistency of Elements Against

Criteria

The elements and criteria define the scope of the review.

Results must be clear and communicated effectively to

both the reviewers and the technical experts. If the

reviewers do not understand the questions or disagree

about what the questions are, the outcome will not 

be valid. The recommended approach is to use a matrix

of elements and criteria both for presenting the 

questions and for summarizing the result of a review,

with additional explanatory text supporting the entry

in each cell (table 5).

The recommended rating system for the review is to

produce one "yes" or "no" for each element under each

criterion, and to provide narrative documentation for

the rating that was assigned. Moreover, if the rating is

"no," the narrative should provide detail on what needs

to change to improve the rating.

The narrative is critical. The reviewer should offer several ways that an inconsistent element could be made

consistent. This gives the technical experts more latitude in modifying the draft document, and it steers

the reviewer away from the position of advocating one

course of action in a particular element.

Reconciling the Reviews

Multiple reviewers may disagree on some or many 

of the scores in the evaluation matrix. The review

administrator has the task of reconciling conflicting

reviews. The review administrator must establish the

nature of the reviewers’ disagreements and resolve

them objectively. In some cases a way to measure and

describe the degree of disagreement among the 

reviewers for each response may be advisable; the

level of disagreement provides information about the

level of confidence in the evaluation.

The review administrator must provide feedback that

separates personal bias from scientific differences among

the reviewers’ positions and the rigor or degree of effort

made by the reviewers. The following are suggestions

to resolve differing reviews, listed in order of those

requiring least effort to those requiring the most

effort. The review administrator—

1. Sends the responsible official the reviews as they 

stand, with no recommendation or summarization 

of conflicts.

2. Unilaterally chooses one answer in the review that 

seems to reflect the opinion of the majority of the 

team members or otherwise seems most appropriate.

3. Acts as an associate editor and adds his or her 

recommendations, giving more or less weight to 

each review.

4. Contacts reviewers, interprets their findings, and 

produces a reconciled report.

5. Prepares a unified team response, but includes 

dissenting opinions from reviewers.

6. Asks a third-party technical expert to review the 

reviews and evaluate the merit of each of the 

reviewers’ positions.

7. Convenes the reviewers in an evaluation panel, 

enabling them to discuss the evaluations and 

persuade each other to change their positions, 

but without forcing consensus. 

This process seeks a consensus of the group, but does

not mandate consensus. Some interaction among the

reviewers enables them to clarify their points and provide

more useful information to the responsible official. For

complex or large-scope issues, it may be useful to have

a team facilitator to help sort through differences. The

objectivity and independence of such a person can

prove to be a valuable asset to the overall process.
19

An SCR must be an interactive and iterative collaboration

between science and management. The responsible

official and his or her technical experts will be able to

use the SCR report in a number of ways, by revising

the reviewed document, by challenging the evaluation

result, or by issuing a decision that discloses an

incomplete consistency result. The latter two choices

set up a potentially irreconcilable process, and should

be avoided. The revision process is preferred in that it

offers the only means of obtaining a final SCR report

that declares a document is fully consistent with

available scientific information.

Draft or Interim Review Team Report

An anticipated outcome of the iterative SCR process 

is that draft review team reports (unofficial) or interim

review team reports (official) may result in a revised

document that needs an additional review. Draft reports

may be helpful for reviewers and technical experts who

are working closely together in the iterative process,

and these would be shared directly between reviewers

and technical experts. Interim reports provide a more

formal process, which would include formal transmittal

letters from the review administrator to the responsible

official, and these may be needed as part of the

administrative record for the decision. The responsible

official would forward the interim report to the appropriate

technical experts for use in revising the draft document.

Ultimately, the draft or interim report is provided to

the responsible official for one of two actions: to accept

it as is, or to use it as guidance to technical experts

for revisions that may be needed to address questions

raised by the review. Subsequent reviews may ensue.

Ultimately, the responsible official decides whether to

repeat the review process. This opens the possibility

that a responsible official might issue a decision

based on an analysis or decision document for which

the latest interim SCR report finds some elements to

be inconsistent with available scientific information.

This would fall short of the preferred way to terminate

the SCR process, but would be within the responsible

official’s authority.

Final Review Team Report

A final written report is the ultimate product of the

SCR process. The review administrator should submit

a transmittal letter, which can be filed as part of the

administrative record of the decision, along with the

interim and final reports to the responsible official.

The review administrator is responsible for preparing

an executive summary and for transmitting the body

of the review report. The written SCR report should

document the process of the review, including selecting

the reviewers, selecting the elements, assembling the

information under review, and describing how disagreements

among reviewers were reconciled. If a matrix of elements

and criteria was used with summary ratings, this should

be included as well. Narratives supporting the summary

ratings should also be included. Upon request from

one or more of the reviewers, and at the discretion 

of the review administrator, an additional dissenting

(minority) report may be transmitted along with the

final summary report. Finally, the review administrator

may conduct or request a review of the final team report

to ensure that the report does not advocate any particular

decision or alternative and that the evaluations are

value neutral. 

Completion

The responsible official decides when the process is

complete. Between completion and success lies the

report that is written for, delivered to, and accepted by

the responsible official. The review administrator should

transmit the completed report to the responsible official.

The SCR documents should become part of the planning

analysis file, and they should be made public at the

same time as the decision and the National Environmental

Policy Act (NEPA) document are made public.

Science Consistency Review Products

21

Implementing the SCR Process

Widespread implementation of the SCR process 

within the agency will require attention to several

organizational and policy issues beyond the scope of

this white paper. These issues include primary purpose,

analysis gridlock, potential for appeals, reviewer 

workload, and adaptive response to early SCR efforts.

Such issues point to the need for guidance from

agency leadership when the SCR process is made

available to the Forest Service at large.

According to the "primary purpose" financial management

strategy within the agency, costs for conducting the SCR

would generally be the responsibility of an appropriate

budget line item or combination of budget line items

within the NFS budget. In addition, some experience

with the conduct and cost of an SCR under the process

outlined in this white paper is needed for future budget

planning purposes.

The potential for the SCR process to add to the length

of time needed to issue a decision must be balanced

by the increased defensibility of a decision that can 

be documented as consistent with available scientific

information. If the benefits of using the SCR process

do not outweigh the added time required to issue the

decision, managers may want to consider tools other

than the SCR process outlined here. 

There is some question about whether conducting an

SCR for a given project will establish a formal or 

informal precedent for similar reviews of projects of

comparable size and complexity. There is also a concern

about whether using the SCR process in a given 

decision opens the door to appeals of similar decisions

where the SCR process was not used. The SCR process

must be implemented in a way that addresses these

concerns.

It is acknowledged that the SCR process will increase

the demand on reviewers’ time and effort. If the SCR

process becomes a required or widely used, rather

than elective, part of planning, then scientists and

administrators could spend a significant amount of

their time conducting these reviews. 

There may be instances where a scientist or research

administrator has concerns about the manner in which

agency officials are using science outputs and products.

Several ways exist to resolve such concerns, such as

direct communication, involvement with the NEPA

process externally, or progression of concerns internally

within the overall Forest Service organizational hierarchy.

There may be instances where a review administrator

unilaterally reviews a draft document using the SCR

process outlined herein; however, if both the responsible

official and review administrator are willing cooperators

in the SCR, it dramatically increases the likelihood

that the SCR report will be favorably received.

Finally, the SCR process proposed herein will no doubt

evolve as it is applied in different situations. It is

important to adaptively respond to early experience

with the SCR process as outlined in this white paper.

One would expect that NEPA coursework might be

modified to provide training in the SCR process to

technical experts and responsible officials; similarly,

review administrators and reviewers would benefit from

such training. In addition, technology transfer tools,

such as a Web site to facilitate extending the SCR

process to the field, would be required. For these and

other reasons, interim staff support at the Washington

Office level will be valuable as the SCR process is

introduced to the agency at large.

23

Summary

The responsible official should consider the SCR as an

optional tool to use when he or she thinks that the

scientific information contained in a draft analysis or

decision document could benefit from a review. When

conducted as suggested in this white paper, the SCR

will essentially serve as a report by reviewers to validate

the scientific information used in the draft decision or

analysis document, and it will enable the responsible

official to assert that a decision based on those documents

is consistent with available scientific information.

Acknowledgments

This paper is the final report of the ad hoc Work Group

on Science Consistency Reviews, which was chartered

by Thomas J. Mills and Harv Forsgren on behalf of the

Forest Service National Leadership Team. The authors

thank them and the team members for their leadership

and support during the development of this white paper.

Thanks also to Meredith Webster, Brian Kent, Lisa

Freedman, Jack Troyer, Gary Pierson, and Dick

Smythe for comments on the result.

25

Appendix 1—Frequently Asked Questions

1. Why do a science consistency review?

A science consistency review (SCR) helps the responsible

official to fully appreciate all the scientific information

relevant to a draft analysis or decision document, and

to make revisions in a draft document so it better

reflects the best available scientific information. The

responsible official can use the SCR process to obtain

valuable information on the use and interpretation of

scientific information in the draft document under

review, and he or she can include that information

when revising the draft document. If carried to the

point where the final SCR report documents that all

reviewed elements are consistent with the best available

scientific information, the responsible official can then

defensibly argue that the decision based on that reviewed

document is also consistent with available scientific

information. For certain decisions, the responsible

official may find it helpful to offer such an assertion.

2. To be consistent, does a decision have to conform

to the science?

No. Decisions are based on many factors: economic

conditions, political issues, legal concerns, ecological

interests, and so on. Science alone does not make the

decision. If the decisionmakers found the relevant 

scientific information, interpreted it correctly, recognized

the inherent error of it, and made reasonable predictions

using it, the decision will be consistent with the available

scientific information, but does not necessarily have to

conform to it.

3. Am I legally required to do an SCR?

Decisionmakers should check the latest Forest Service

directives, but at the time this document was written

there was no requirement to complete an SCR before

making a decision. In addition to SCRs, other methods

such as scientific peer review or science advisory boards

are available for evaluating the scientific information

used to support decisions. 

4. What is the distinction between a peer review 

and a science consistency review?

Scientific peer review refers to evaluation of the science

that forms a basis for analysis. An SCR refers to 

evaluation of whether the responsible official has

appropriately interpreted the scientific information 

in coming to a decision. A peer review is done to see 

if the science that addresses relevant issues is correct;

a consistency review evaluates whether the science is

used correctly.

A peer review can be defined more generally as any

review of documents that evaluates the quality and

validity of the information contained therein. A doctor’s

second opinion is an example of peer review. Scientific

peer review includes evaluation of the validity of the

information itself, and of whether the information is

applied, used, or acknowledged in a reasonable manner.

An SCR evaluates whether the information in a draft

decision document is interpreted properly, and whether

the interpretation is properly disclosed in a decision. 

It has to include some elements of a peer review, such

as whether the scientific information used in a planning

and decision process is properly interpreted and used,

but the process goes further in that the SCR evaluates

whether uncertainties and risks have been captured

in the draft document and whether conclusions drawn

in development of planning documents use scientific

information appropriately. A draft analysis or decision

document may propose alternatives involving actions

that scientific information shows are less than optimal,

but if interpreted and considered properly, would still

be consistent with science.

Interpreting and disclosing information and effects are

the important points. Both peer review and SCR include

a check of whether all relevant information has been

considered, but only the SCR provides a decisionmaker

with feedback that describes what information was

used and how it was interpreted, and discloses the

factors and risks involved in coming to the decision.

5. What is the product of the SCR?

The ultimate product is a final report issued by the

SCR review administrator that addresses whether a

draft decision document has considered, and is 

consistent with, available scientific information. Interim

reports offer a more structured way for technical experts

to respond to reviewer evaluations through iterative

review and revision of the draft document under review.

In both interim and final reports, the SCR is conducted

by evaluating specific, detailed elements according to

a set of standard criteria. 

No statements in the SCR should address the decision

itself. Rather, an SCR should address the question, 

"Is the technical information supporting the decision

consistent with science, and has the decision process

interpreted and disclosed appropriate information?"

The report may include dissenting views from reviewers.

6. Does the SCR certify the decision as 

scientifically credible?

The SCR does not certify a decision as scientifically

credible. When conducted as suggested in this white

paper, the SCR results in interim or final reports by

reviewers that validate the scientific information used

in the draft decision or analysis documents, and these

reports can be used to modify those draft analysis and

decision documents. 

If the analysis document is consistent with science, or

changed to become consistent with science, and if the
26

decision document adequately discloses that applicable

science was considered and interpreted correctly, and

if opportunities and risks were correctly analyzed, then

the responsible official can claim that the decision is

consistent with the best available science. Granted, the

decision may be influenced by social or economic factors

that could result in a different decision being made

than if science had been the only factor considered.

Nonetheless, the outcome of the SCR enables the

responsible official to claim that the decision is 

science-consistent.

7. Exactly what documents and analyses will be 

covered in the SCR?

An SCR may be requested for any analysis or decision

documents in which scientific information is used in

land management planning or environmental disclosures.

Examples may include but are not limited to forest

plans, environmental impact statements, environmental

assessments, records of decision, or decision notices.

Ideally, the SCR is conducted through an iterative

process whereby reviewers review early drafts of

analysis documents, technical experts revise them,

and this iterative process of review and revision 

continues throughout the preparation of final decision

documents.

8. Will it make me change my documents?

The SCR process is designed to give the responsible

official constructively critical feedback on draft analysis

and decision documents, so it is unlikely that a

responsible official would choose to follow the SCR

process without the expectation that some revision

would follow. The SCR report identifies weaknesses 

in the interpretation and use of scientific information

and offers suggestions for revision in order to achieve

consistency with available science. The decision on

whether to revise the draft document remains with 

the responsible official, but the constructively critical

feedback provided would foster revision to make the

revised document more accurately reflect the effects

and risks as they are understood in the scientific 

literature. Thus, if the responsible official embarks 

on the SCR process, he or she should expect to 

undertake revisions in the draft documents.

9. Is the SCR made part of the administrative record?

All official correspondence between the responsible

official and the review administrator will become part

of the administrative record. Draft or interim reports

exchanged between reviewers and technical experts

may not necessarily be part of the administrative record,

but retained written communications associated with

this process would be subject to the Freedom of

Information Act and would be discoverable. Similarly,

the responsible official decides whether the SCR report

should be included or appended to the Environmental

Assessment or Environment Impact Statement. 

10. Can I do this with less than a full-time 

dedicated team?

The amount of time required of the review team depends

on the scope of the decision and how much scientific

information relates to the decision. The SCR process

was designed to be scalable to different levels of analysis.

As such, having a team of three reviewers look at a

draft document for only one or two iterations could

complete the process. Decisions of greater scope and

complexity may require greater effort and greater 

organizational distance. The proposed structure was

developed, however, to use the reviewers’ time efficiently.

11. Who pays for the SCR? Do primary purpose 

financial concepts answer any questions 

about who pays for it?

According to the "primary purpose" financial management

strategy within the agency, costs for conducting the

SCR would be the responsibility of an appropriate

budget line item or combination of budget line items,

logically falling to the NFS side of the budget.

Widespread application of the SCR process could be

quite costly.

The informal or formal agreement between the 

responsible official and review administrator that

authorizes the SCR effort should specify how the 

work will be paid for. Insofar as the conduct of an SCR

is an elective process within the line authority of the

responsible official, the cost of conducting it would

logically come from some appropriate function under

the authority of the responsible official—in other

words, from planning-related budget line items within

the NFS budget. 

12. How much will the SCR cost?

Costs cannot be determined easily, but are an important

consideration in conducting an SCR. As a practical

and pragmatic matter, asking a reviewer to prepare

one or two reviews within a short period of time would

be done as an in-kind contribution by the reviewer, 

in a professional courtesy capacity similar to that of

providing a peer review of a scientific paper. The more

intensive the demands made on a reviewer above this

minimal level, though, the greater the likelihood that

the responsible official will need to support reviewer

expenses, including salary and any travel costs. 

13. What will the SCR buy me?

Successful completion of the SCR will provide the 

following:

1. more accurate estimates of an action’s effects and 

risks,

2. development of more, or different, options or 

management alternatives as draft documents are 

revised, and

3. increased public credibility and/or legal defensibility

for decisions.
27

14. Will an SCR keep me out of court or keep me 

from getting sued?

No, but an SCR may help the responsible official make

a more informed decision, and may provide a stronger

defense if the decision does wind up in court.

15. Can an SCR be an easy and quick process?

The SCR process can be done in a quick and easy

manner, especially for decisions of smaller scale and

complexity. Larger decisions will require greater effort.

16. When in the planning process should an SCR 

take place?

The earlier in the planning process that the SCR takes

place, the better. If conducted in ideal circumstances,

the first interim SCR report would be generated upon

review of the first draft of the analysis document. A

second interim SCR report would be based on the

final analysis document. The final SCR report would

be generated on the final draft of the decision document.

Experience suggests that if the SCR process is initiated

late in the game, revisions may be more disruptive to

the overall planning process than if the interim SCR

reviews had been done with early drafts of analysis

and decision documents.

17. What will trigger the SCR?

The SCR begins when a responsible official makes

some informal contact with the nearest appropriate

R&D staff about whether to conduct an SCR for a

given project. Motivations to request a review may

include the desire to make a considered resource

management decision, or to document the process of

making an honest effort to understand the issues that

affect a decision, as might be helpful in litigation.

This is not to say that R&D must be a part of the

process, but it seems logical to suggest that leadership

in NFS regional offices and R&D station headquarters

should be aware of any SCR efforts occurring within

their respective jurisdictions. That informal contact

will enable the responsible official to visit with R&D

personnel and others, if appropriate, to discuss

whether the SCR process or some other method is

most appropriate for the particular situation. If R&D

becomes substantively involved in the SCR through

assignment of a review administrator or reviewers,

R&D line authority must authorize that participation.

If the SCR is conducted using NFS staff or external

reviewers, however, such authorization or participation

would not be needed. 

18. Is there a minimum size requirement for an 

SCR?

No, there is no minimum size requirement. The process

as designed can be made to apply to small efforts such

as a project-level, site-specific decision or a forest plan

amendment.

19. How big should the scope of the analysis and 

decision be to do an SCR?

The process for conducting an SCR is designed to be

flexible and scalable, especially for small analysis or

decision documents at the project or plan amendment

level. It is intended to make efficient use of reviewer time

and effort. The scope of the analysis and decision is

perhaps less important than the implications of scientific

information for the analysis and decision. 

20. Do I need an SCR for a project or just for a 

plan amendment?

The responsible official decides when to do an SCR,

and that decision would encompass issues of scale.

21. Do SCRs work only in the Pacific Northwest?

Although the SCR process has been applied to a number

of large, high-profile efforts in the Pacific Northwest,

the process outlined in the attached white paper can

be and has been applied in other regions, and could

be modified to fit any agency land base in the Nation.

SCRs have been used in the Sierra Nevada Forest Plan

process, as well as in the Northern Great Plains

Assessment. Moreover, with some modification to the

process and to the agency affiliation of those filling

various SCR roles, the SCR process could be conducted

on management plans for forest industry lands or

even for forest land owned by nonindustrial private

forest landowners. 

22. Who initiates an SCR?

The responsible official has the responsibility to initiate

an SCR. Since the costs are borne by the appropriate

budget line under the responsible official’s direction,

the responsible official has line authority to decide

whether to engage the SCR process.

A station director or other R&D administrator with

concerns about the use of science in a draft decision

document can also initiate the SCR process. Ideally this

would occur with the responsible official’s cooperation

and support. One cannot rule out the possibility that

a scientist or administrator could conduct a unilateral

review of a draft analysis or decision document using

R&D funds; however, to move forward as outlined in

this white paper, the responsible official’s cooperation

and support are essential for any review to serve as the

basis for revising the draft document. Any such revisions

remain within the purview of the responsible official.

23. How can someone other than the responsible 

official handle concerns about the inappropriate

use of science in land management?

If science is being misinterpreted, the scientific 

community needs to set the record straight. A station

director, assistant director, or scientist may want to

retain control over how scientific information is used,

but no scientist has control over how his or her 

published information is interpreted, other than
28

accepts, he or she should not subsequently be asked

to review it. On the other hand, if a scientist has been

extensively cited in a draft analysis or decision document

that he or she did not prepare, that scientist may be

the best person to review the draft document.

Multiple iterations of review will probably be needed

after a draft document is revised in response to an 

initial SCR. The same reviewers are generally expected

to review any revised documents. Reviewers may be

replaced if the review administrator decides that their

contributions are not appropriate; examples would

include a reviewer who insists on making policy 

recommendations or proves unable to meet deadlines.

The responsible official or review administrator may

request a peer review of the final SCR report.

25. Can an SCR be done entirely within NFS?

Yes, an SCR can be done entirely within NFS, provided

the reviewers are not involved in drafting the document

reviewed and are knowledgeable about the scientific

information discussed in the document.

26. How can I find out about the experiences of 

others who have done an SCR?

A number of publications are available that discuss

experience gained in the use of Science Consistency

Evaluations in the Pacific Northwest and Pacific

Southwest. Those evaluations were similar to the SCR

process discussed here. This SCR process was designed

to be more streamlined, however, and to capture the

best elements of those previous efforts.

27. Is the Federal Advisory Committee Act (FACA)

an issue during an SCR?

The FACA does not appear to be an issue. The presumption

is that an SCR "contracts" with outside persons to

evaluate documents and provide reports. Even if an

outside review administrator coordinates the SCR, as

long as the report is simply something the responsible

official uses and it does not advocate a particular 

decision, FACA may not be relevant. Moreover, the

SCR report is considered information—a means of

providing technical expertise—rather than something

advocating or endorsing a particular action.

28. Must an SCR reviewer be a scientist?

No, the only attributes the reviewer must have are

expertise, credibility, and independence from the draft

decision document. Thus, a reviewer might be a Ph.D.

scientist, but he or she also could be a technical expert

or line officer who has these three attributes. In some

cases a manager may do the review independently,

especially on relatively small-scale issues. The goal of

the process is to ensure that decisions are consistent

with relevant scientific information; to accomplish this,

the review administrator should bring to the review any

person who has the required academic and scientific

credentials to address the issues.

through further publication and discussion of the

results and interpretations. 

A number of ways exist to resolve such concerns.

First, the concerned individual can ask the responsible

official to initiate the SCR process. If the concerned

individual has sufficient administrative or scientific

credibility, that request would be difficult to ignore. If

the responsible official decides not to initiate an SCR,

however, other avenues remain available to the concerned

individual. These include providing detailed comments

in a forest plan or in other public documents during

the public comment process, communicating directly

with the technical experts who developed the decision

document, or writing follow-up articles in research

publications that critique any misinterpretations.

Disagreements among agency personnel can be handled

through resolution at increasingly higher levels of line

authority.

Whenever scientists or research administrators have

questions about the scientific consistency of a decision,

however, they have a responsibility and an obligation to

act, such as through an official letter to the responsible

official.

24. Should Forest Service scientists ever be 

excluded for reasons of agency affiliation? 

Should different people be used for intermediate

or final reviews? If so, under what circumstances

should this occur?

Scientists within R&D are set apart administratively

from the NFS, and they have no authority for 

management of national forests. As such they can be

and often are viewed as independent from the agency’s

mission of national forest management. There may be

situations in which these scientists should be excluded

as reviewers, such as when a scientist provided direct

input to the analysis or decision document, but it is

not necessary to exclude them as reviewers simply

because of their affiliation with the Forest Service.

Overall, the affiliation of reviewers should be kept 

flexible for different situations. Selecting the review

team is the joint responsibility of the responsible official

and review administrator. Sometimes agency scientists

or NFS technical experts are the most appropriate

reviewers, but certainly others with relevant expertise

can fill this role. 

The selection of those who will conduct the review

should be based on their (1) expertise, (2) credibility,

and (3) independence from the decision. Therefore,

deciding who conducts the SCR review depends on

both the nature of the draft decision document and

the personnel who possess those three attributes.

Anyone asked to participate in drafting the decision

document should be disqualified from its review under

the SCR, under the axiom that a professional should

not review his or her own work. For example, a scientist

with certain expertise may be asked to participate in

the preparation of a draft document, and if that scientist
29

29. What is the importance of the administrative 

record in an SCR? What elements of the SCR 

are private between the review administrator 

and the responsible official?

An administrative record is the first thing looked at in

a judicial review. It documents the process as specified

by NEPA. 

30. Is the SCR subject to the Freedom of 

Information Act (FOIA)?

Yes, all "agency records" that exist at the time of an

FOIA request are subject to the Act. An "agency record"

is located in an official file and can be in the form of a

handwritten draft report, typed interim or final report,

video tape, audio tape, drawing, photo, computer disk,

email message, or sticky note. A personal document,

however, created by an employee that is not shared

with others, does not contain substantive agency

information, and is not located in the official files may

not be an "agency record." A complete discussion of

agency records vs. personal records can be found at 

the following Web site: http://www.usdoj.gov/oip/

foia_updates/Vol_V_4/page5.htm.

Depending on the circumstances involving each individual

FOIA request, however, all or portions of an SCR record

may be withheld, based on FOIA exemptions such as

Exemptions 3 and 5: 

• Exemption 3 protects information exempted by 

statute—i.e., the Archaeological Resources Protection

Act, or the National Historic Preservation Act; for 

example, the location of an archaeological site 

identified in an SCR might be withheld. 

• Exemption 5 protects inter- and intra-agency 

records that are part of an agency’s deliberative 

process; for example, a draft or interim SCR report 

might be withheld until the agency’s deliberative 

process is completed, by the regional forester’s 

issuance of a final resource management decision, 

if the release would cause "harm" to that deliberative

process. 

If you have questions about the treatment of draft

records, contact your regional FOIA coordinator. 

A complete discussion of the "foreseeable harm" 

standard can be found at the following Web site:

http://www.usdoj.gov/oip/foia_updates/Vol_XV_2/

page3.htm. 

The administrative record should include the final

SCR report to document the consideration of scientific

information in the agency’s resource management

decisionmaking process. Draft and interim SCR

reports may be made part of the agency record.

31. Must unanimity be reached among SCR team 

members?

Complete scientific unanimity will seldom be achieved.

If substantial disagreements exist among members of

the review team and have not been resolved, those

disagreements should be documented within the report.

The review administrator has the task of reconciling

contradictory reviews. This can best be done by 

establishing the nature of the reviewers’ disagreement

and resolving it objectively. A number of possibilities

for reconciling such reviews are proposed in the white

paper. 

32. Must the methods and process used to elicit 

the reviewers’ science consistency judgments 

be science-consistent?

Conceptually, the methods used in the review process

must themselves be consistent with the available science.

If there is concern about this aspect, the review

administrator or responsible official can request a peer

review of the SCR addressing the issue. This may be a

researchable opportunity as SCRs are conducted. 

33. Is there a need for each element to be 

completely consistent under every criterion?

The review administrator, in consultation with the

responsible official, decides which elements to include,

and the review must clearly identify those elements.

The available science is an important factor in this

decision. Decisions need to state how much available

scientific information has been considered, and the

responsible official must be flexible enough to live with

evaluations that include some elements that are not

"fully consistent." The review administrator is 

responsible for identifying the elements to review, 

but the responsible official decides whether to issue 

a decision when some elements might be listed as "not

consistent."

34. Is there a general preference of line officers 

for ratings: binary ratings, 5-point scales, text

narrative, or other?

The "yes or no" rating is used here because it requires

a reviewer to make a hard decision. But beyond that,

the narrative that the reviewer prepares to justify the

ranking is the most critical part of the review. It should

identify factors that influence ratings, with specific

recommendations for what needs to be strengthened

or what factors had the most influence on the rating.

Distinction needs to be made between whether "the

science is not conclusive" and "the science could be

clear but not enough information is presented."

Ideally, the reviewers should offer suggestions as to

how to achieve consistency on a particular element.

The most useful reviews are those that include a variety

of options for achieving science consistency, since

there can be multiple decisions that are all scienceconsistent.

A summary of which elements or criteria may be most

important with respect to the decision document

would be useful, especially if particular elements are

interrelated with or strongly influence a large number

of others. The responsible official, however, may

decide that some elements are important enough to

outweigh inconsistencies in other elements.
30

35. How "optional" is a revision by the responsible

official if the process is initiated? If an SCR 

report is delivered to the responsible official, 

does this mandate a revision of the document

reviewed?

The SCR report does not mandate revision. The

responsible official has discretion in deciding whether

to revise the draft documents in response to an SCR

report, unless agency policy essentially says that decisions

"must be consistent" with scientific information. 

Because the responsible official initiates the review,

the SCR report is intended for that responsible official,

who then must decide how to respond to it. Revisions

to the initial document may or may not be needed.

Existing NEPA policy requires disclosure of the rationale

for a decision; thus, acknowledgment of rreceiving the

report should be included in the decision document.

Once the responsible official initiates the process,

however, it seems logical to carry it to completion—to

revise the draft document, repeatedly if necessary, if

the SCR report suggests that certain elements are not

science-consistent.

36. When is the SCR finished?

An individual SCR report is finished when the review

administrator submits an interim or final SCR report

to the responsible official. The larger SCR process—

which may have multiple reviews and revisions—

comes to conclusion when the review administrator

and responsible official agree that it is finished.

Ideally, this consists of two steps: 1) The review

administrator issues a final SCR report indicating that

all elements in the draft document have been reviewed

and are consistent with the review criteria, and 2) the

responsible official accepts the final SCR report and

issues a decision based on that draft document.
31

Appendix 2—Lessons Learned

The following are some examples of lessons learned

from past science consistency reviews (SCRs). These

reviews include the Interior Columbia Basin Final

Environmental Impact Statement (ICB), Tongass Land

Management Plan Revision (TLMP), Sierra Nevada

Framework for Conservation and Collaboration (SNFCC),

and Northern Great Plains Land Management Plan

Revision (NGP).

These lessons learned should be viewed in the context

of adaptive management. They are taken from specific

situations, and may or may not relate to the subject

matter of the SCR, the composition of the SCR team,

and the amount of interaction between the team 

members and the resource managers. Most of these

teams were predominantly Forest Service scientists

(either R&D or National Forest System), Federal 

scientists, external academic (university) scientists,

independent consultants, or a combination of the above.

The reviews were characterized by the following kinds

of interaction: high levels of frequent interaction among

science and management teams; moderate levels of

interaction, perhaps constrained by distance or 

communication issues; and low levels of interaction

with very few and/or one-way interactions.

Scenario A

Predominantly R&D scientists; moderate level of 

interaction constrained by distance and 

communication (ICB):

• Reviewers should clearly articulate why the decision

document was found to be inconsistent with science.

Where possible, try to propose more than one 

solution to each inconsistency item. Try to leave 

some decision space for managers.

• Reviewers should stay neutral. When proposing 

changes, focus on strengthening the science 

consistency and stop short of advocating a position 

on the decision or editorial fixes to better conform 

to individual beliefs, values, or writing style.

• Technical experts can easily get defensive and argue

that the issue is adequately covered. A better question

might be to ask whether the science suggestion will

strengthen the document. It often takes less time to

fix inconsistencies than to debate their merit; in 

other words, do not waste time on the small issues.

If it will strengthen the document, just do it.

• Responsible officials should remind themselves that

they would rather be asked the tough questions now,

internally, than later in public meetings or in court.

• Technical experts need to be clear about the criteria

for selecting elements for consistency evaluation and

the criteria for judging consistency of those elements.

It is important to agree up front with the review 

administrator on what elements will be evaluated. 

Write them down. Discuss them with resource 

managers and technical experts so that everyone 

knows what the standards will be. Criteria are more

in the realm of the science team, but they should 

be discussed with the resource management team 

for their input, understanding, and ownership.

• All participants in an SCR need to be as careful 

about how something is said as what is said. Do not

let it become personal. When deadlines are short 

and frustration is high, tempers can fly. The review 

administrator and responsible official must intervene

before conflicts among staff escalate; a neutral liaison

can be useful to assist the interaction between science

and management. Consider focusing the discussion

between the science team and management team 

through a single spokesperson for each. Conduct 

facilitated meetings, if necessary. While individual, 

one-on-one discussions among members of the 

science and management teams can be useful and 

productive, they can also degenerate into friction 

and personality issues. Schedule face-to-face time 

and jointly discuss options to resolve; communicating

by email often escalates rather than solves the problem.

• Technical experts and resource managers should 

separate the most important elements from minor 

ones—there probably will be many more inconsistencies

than imagined. Stratify the "show stoppers" from 

the multitude of others needing improvement and 

concentrate on these issues first.

• Technical experts and reviewers should be open to 

more than one way to resolve an inconsistency. 

Often the issue is not black and white; try to find 

middle ground based on time constraints and the 

sensitivity of the issue.

• SCRs can be very time-consuming, and it can be 

tempting to take shortcuts. The rigor and time 

required for the evaluation depend on the scope of 

the management issues and the potential level of 

conflict arising from the decision. Time spent up 

front in designing the SCR will be time well spent. 

Decisions about the rigor and time required are 

more in the realm of the resource management 

team, but should be discussed and negotiated with 

the science team.

• Responsible officials should not attempt to negotiate

out of or sidestep science issues that may not reflect

well on the proposed decision document.

• All participants should stay within their appropriate

roles. Resource managers decide, disclose, and 

manage. Scientists evaluate, in a value-neutral way,

the use and disclosure of appropriate scientific 

information.
32

Scenario B

Predominantly external scientists: non-Forest Service

Federal, academic, State, private sector; low levels of

interaction constrained by time (SNFCC). 

• The time available and needed for an SCR are both 

crucial considerations that need to be settled up front.

• The responsible official should be considerate of how

much time can be expected from external reviewers

and be able to provide any needed support.

• The responsible official and review administrator 

should define the purposes and roles up front. They

need to be clear about how many iterations of 

interaction they expect will occur, and they should 

provide a structured setting for participation.

• The responsible official and the review administrator

should do their best to provide incentives (both 

monetary and nonmonetary) for outside reviewers 

to participate and to make their participation as 

smooth and easy as possible. Participants should 

not feel like this is a chore.

• For this scenario, an independent review 

administrator who is not affiliated with the Forest 

Service is recommended.

Scenario C

Predominantly Forest Service research scientists; high

levels of interaction, defined roles maintained, close

proximity, high iteration of review and interaction (TLMP).

• An understanding of the standards of evidence 

and documentation required by science is a key 

prerequisite.

• Participants in the SCR must understand the role 

of scientific debate.

• The executive leadership needs to be engaged and 

supportive.

• The partnership needs to be formed and formalized 

at the beginning.

• Scientists need to have an understanding of working

in the litigation environment.

• Timeframes are crucial.

• Focusing on decisionmaking versus science is a 

way to keep discussion away from personalities.

Scenario D

Predominantly external land grant university scientists;

low levels of interaction constrained by time (NGP).

• The responsible official should provide a structured

setting for the reviewers, with deadlines clearly stated.

• The technical experts and resource managers 

should clearly articulate what is to be done and the

evaluation criteria to be applied

Scenario E

Predominantly internal higher level NFS scientists or

experts; moderate to low levels of interaction constrained

by time. (Note: this scenario represents the use of

internal agency experts who have not been involved in

the development of a decision document as reviewers.

The Chugach National Forest has, as of January

2002, gone through several drafts of review in which

the review was conducted by regional office staff.)

• The coordination and protocol for conducting an SCR

with regional or similar-level NFS scientists or experts

should come from a high-level regional administrator.

• The responsible official and review administrator 

should set timelines for producing a consistency 

report, and provide adequate time for careful review

and written documentation.

• The SCR report should include documentation on 

the science literature or information considered by 

the reviewers.

• The responsible official and review administrator 

should ensure that the reviewers were not involved 

in preparing the decision document and that they 

have been insulated from internal pressure to 

generate findings of a particular nature.

Common Elements

• All participants in an SCR need to be clear on what

criteria should be used. 

• All participants should be clear about boundaries 

between management decisions and scientific 

information and the roles of managers/decisionmakers

and scientists.

• Especially in the case of reviewers, the responsible 

official and review administrator should be clear 

about time and personnel commitments and deadlines.