Use of Risk Assessment

Panels During Revision of the

Tongass Land and Resource 

Management Plan

Charles G. Shaw III

United States 

Department of

Agriculture

Forest Service

Pacific Northwest

Research Station

General Technical

Report

PNW-GTR-460

July 1999
Author CHARLES G. SHAW III is the Science Issues Coordinator for the Tongass National

Forest, Forest Service, Pacific Northwest Research Station, 2770 Sherwood Lane,

Suite 2A, Juneau, AK 99801-8545. During the revision process for the Tongass land

management plan, he was science manager for the revision and responsible for

development and implementation of the risk assessment panel process. 
Conservation and Resource 

Assessments for the Tongass Land 

Management Plan Revision

Charles G. Shaw III, Technical Coordinator

Use of Risk Assessment Panels During Revision of the Tongass

Land and Resource Management Plan

Charles G. Shaw III

Published by:

U.S. Department of Agriculture 

Forest Service

Pacific Northwest Research Station

Portland, Oregon

General Technical Report PNW-GTR-460

July 1999
Abstract Shaw, Charles G., III. 1999. Use of risk assessment panels during revision of the 

Tongass Land and Resource Management Plan. PNW-GTR-460. Portland, OR: 

U.S. Department of Agriculture, Forest Service, Pacific Northwest Research 

Station. 43 p. (Shaw, Charles, G., III, tech. coord.; Conservation and resource 

assessments for the Tongass land management plan revision).

This paper describes the process used to conduct the 16 risk assessment panels and

a subsistence working group held during revision of the Tongass land management

plan. It provides an overview of how results from the panels were used by forest 

managers in plan-related decisionmaking, discusses some reactions to the effort,

and identifies some opportunities to improve the process. The panel results provided

estimates of the relative risk that implementation of a range of alternative approaches

to management of the Tongass National Forest would pose to the continued persistence across the landscape of an array of species or resources and estimates of

potential socioeconomic effects on communities. As anticipated, results from these

risk assessment panels became an integral component of the effects analysis section

of the environmental impact statement that provided the context for successfully completing the Tongass Land and Resource Management Plan.

Keywords: Forest planning, effects analysis, National Forest Management Act, 

old-growth forest, forest policy, Tongass National Forest, socioeconomic issues,

southeast Alaska, subsistence, wildlife population viability, fisheries sustainability,

well-distributed populations. 
1

Introduction The Tongass National Forest (NF) in southeast Alaska recently revised its land and

resource management plan (USDA Forest Service 1996a, 1996b, 1997a, 1997b,

1997c). Several of the primary issues addressed during plan revision were similar to

those recently encountered in the Pacific Northwest (Forest Ecosystem Management

Assessment Team [FEMAT] 1993), the Columbia basin (Lehmkuhl and others 1997),

and elsewhere. These issues included maintaining enough habitat to support viable

and well-distributed populations of wildlife and sustainable production of fish over the

long term (e.g., a century or more) while addressing the likely socioeconomic effects

of various levels of resource use (e.g., timber harvesting).

The Tongass is unusual among National Forests in at least three aspects important

to planning: (1) at 16.9 million acres, it is considerably larger than any other single

National Forest (its size exceeds the combined area of all National Forests in

Oregon); (2) the acreage is scattered across a narrow strip of mainland and some

22,000 islands that form the Alexander Archipelago; and (3) it has congressionally

mandated obligations to provide for rather extensive subsistence use of natural

resources (U.S. Public Laws 1980) and to “seek to meet” timber demand (U.S. Public

Laws, Statutes 1990).

A principle component of the documentation for the development or revision of a

Forest plan is the section of the final environmental impact statement that compares

alternatives (USDA Forest Service 1997b). All reasonable alternatives are objectively

evaluated with substantial treatment given to each alternative so that comparative

merits can be evaluated. In describing the environment affected by each alternative,

this analysis discusses direct and indirect effects of implementation on an array of

forest resources.

To assist with development of the effects analysis and other components of the revision of the Forest plan for the Tongass NF, a new planning approach was applied by

the Forest Service. Research scientists from the Pacific Northwest Research Station

(PNW) were incorporated directly into the planning team (Everest and others 1997,

Mills and others 1998, Swanston and others 1996). Although full members of the

planning team, these PNW scientists had separate and distinct roles. One role was to

provide information for development of the effects analysis for the draft Forest plan

(USDA Forest Service 1996a, 1996b) by organizing and conducting a series of risk

assessment panels that followed an approach used previously in the Pacific Northwest (Cleaves 1993, 1995; FEMAT 1993; Starkey 1998). Besides its past use in the

Pacific Northwest, the approach of using professional judgment structured in risk

assessment panels was chosen because various wildlife habitat and community

relations were poorly understood and there was generally inadequate information 

on which to base predictive models. In addition, the desire was to complete the planning process quickly, leaving no time for data collection. 
Procedures

First Set of Risk 

Assessment Panels

2

Under this approach, structured risk assessment panels were conducted in fall and

winter 1995-96 to gather information from subject matter experts regarding the likely

effects on an array of resource issues of implementing each draft Forest plan alternative. The resource issues possibly affected included the viability of various oldgrowth-associated wildlife species, the sustainability of the fisheries resource, oldgrowth forest ecosystems, the social and economic values of forest resources to 

residents of southeast Alaska, and subsistence use of forest resources (primarily

Sitka black-tailed deer [Odocoileus hemionus sitkenis] and salmon).

The purpose of conducting these panels was to provide decisionmakers and the 

public with information on the relative risk from implementing each alternative to the

continued persistence across the landscape of the species or resource in question

and the potential socioeconomic effects on communities. Information generated from

these panel assessments (Everest and others 1997, Swanston and others 1996,

USDA Forest Service 1997d) was used in the effects analysis for the draft Forest

plan (USDA Forest Service 1996b) and a summary of results and implications was

incorporated into the draft environmental impact statement (USDA Forest Service

1996a). 

After time for public comment on these draft alternatives and their associated affects

analyses, including the risk assessment panels (app. 1; also USDA Forest Service

1996a, 1997d), the planning team began crafting the “final” alternative for the Forest

plan. Leadership for plan revision decided that results from a second set of risk

assessment panels convened to evaluate this emerging final alternative would be a

useful addition to the analysis of effects. A second set of risk assessment panels

therefore was conducted during spring 1997 to consider effects of the emerging final

alternative on the viability of certain old-growth-associated wildlife species and the

sustainability of the fisheries resource.

This paper discusses the procedures used in conducting these two sets of risk

assessment panels and clarifies the nature of the results obtained from them. Some

comments and insights on the utility and application of this methodology in the forest

planning process, and various reactions to it, also are provided. In addition, information is drawn together on these risk assessment panels1 that currently appears in

several documents (Everest and others 1997; Mills and others 1998; Swanston and

others 1996; USDA Forest Service 1996a, 1996b, 1997a, 1997b, 1997c, 1997d). 

Background—To provide scientifically credible information for revision of the

Tongass land management plan, panels of subject matter experts were used to evaluate nine Forest plan alternatives for their likely effects on forest resources and values.

The process was adapted from that used, tested, and judged effective in the Northwest Forest Plan (Cleaves 1993, 1995; FEMAT 1993; Starkey 1998). It is important 

1 A report was filed in the planning record (USDA Forest Service 

1997d) for each risk assessment panel. The full content of each 

panel report is not repeated here; however, they form an unpublished appendix (app. 1) to this paper that is accessible electronically on the internet at: <http://www.fs.fed.us/pnw/>; or in hard 

copy by request to the author or the Documents Coordinator for

the Tongass Land Management Planning Record, USDA Forest 

Service, Alaska Region, P.O. Box 21628, Juneau, Alaska 99802-

1628.
3

that these evaluations not be viewed as precise analyses of the likelihoods of habitat 

or population conditions, but rather as the informed professional judgments of knowledgeable experts gathered together in a structured format. The process was designed

to elicit judgments from these experts about relative risks and likely future outcomes.

The product of each panel consists of outcome rating sheets from each evaluator

and notes on the deliberations. These items were used to prepare a report for each

panel (app. 1; also USDA Forest Service 1997d) that provided input to the effects

analyses (USDA Forest Service 1996a) for the draft Forest plan (USDA Forest

Service 1996b). Thus the effort was intended to help guide planners and to provide

information for them to consider in further alternative design, but not to serve as a

rigid discriminator among alternatives.

Selection of panel topics—Considerable debate has transpired about the effects 

of management practices on habitat that supports populations of various old-growthassociated wildlife species in southeast Alaska (Kiester and Eckhardt 1994; Suring

and others 1993, 1994). Furthermore, wildlife viability was one of the five focus

issues for this plan revision (USDA Forest Service 1997a). To address these issues,

risk assessment panels evaluated northern goshawk (Accipiter gentilis atricapillus),

Alexander Archipelago wolf (Canis lupus ligoni), marbled murrelet (Brachyrampus

marmoratus), brown bear (Ursus arctos), American marten (Martes americana), 

Sitka black-tailed deer, and other terrestrial mammals. The latter included a group of

more widely distributed terrestrial mammals and a group of mostly endemic, small

mammals whose known distribution in southeast Alaska is somewhat restricted

(MacDonald and Cook 1996a, 1996b). Each of these old-growth-associated species

has an individual life history; yet, collectively their ecologies likely incorporate the

breadth of forest habitat features and other attributes of environmental variation 

represented across the Forest. 

Most of the previous work on wildlife habitats in southeast Alaska indicates a strong

association between high-quality habitats and old-growth forest condition (Iverson

and Rene 1997, Suring and others 1993). To look at the issue of wildlife habitat and

forest condition on a larger, landscape scale rather than focus exclusively on the specific, habitat-related needs of various individual species or guilds, there also was an

old-growth ecosystems panel.

Maintenance of fish habitat on the Tongass NF sufficient to sustain the productivity of

this world-class resource was the topic of a recent report to Congress (Anadromous

Fish Habitat Assessment Team [AFHA] 1995). It also was another of the five focus

issues for plan revision (USDA Forest Service 1997a). Accordingly, a fish-riparian risk

assessment panel evaluated the salmon habitat resource. 

The socioeconomic environment of southeast Alaska and its relation to the resources

and uses of the Tongass NF have experienced significant changes in recent years

(Allen and others 1998). Additional changes were anticipated, at least partially, in

response to direction provided in the Forest plan. Thus, socioeconomic considerations formed another of the focus issues for this plan revision (USDA Forest Service

1997a) and became the tenth panel topic.
4

Subsistence hunting, trapping, fishing, and gathering activities have long been important to many southeast Alaskan residents (Cultural Survival Inc. 1998). This importance is recognized in Federal law (U.S. Public Laws, Statutes 1980) that provides

for “the continuation of the opportunity for subsistence uses by rural residents of

Alaska, including both natives and non-natives, on the public lands.” The law also

indicates that customary and traditional subsistence uses of the renewable resources

“shall be the priority consumptive use of all such resources on the public lands of

Alaska.” Even though subsistence per se was not one of the five new focus issues

for this plan revision, its importance was recognized as being a continuing public

issue (USDA Forest Service 1997a) and the subject was included in this exercise—

albeit somewhat differently than in the other risk assessment panels.

Selection and duties of panel participants—Each panel had four (or more for 

the socioeconomic panel and subsistence working group) evaluators who performed

the risk ratings, a local subject matter expert available as a resource person, a facilitator, a scribe, and a silent observer. Panel evaluators were drawn from various

Federal agencies and from Alaska state government employees (including the state

university) who was appointed by the Governor per the amended Federal Advisory

Committee Act (FACA; U.S. Public Laws, Statutes 1972). This compliance with FACA

guidelines followed legal advice designed to avoid a potential issue on which the

Forest plan could be appealed. Evaluators all had recognized expertise in the area

under evaluation. For fish and wildlife, this included technical knowledge of the

species and an ecological understanding of general habitat requirements. For subsistence and socioeconomics, this included knowledge of personal use of forest

resources for food and fiber and an understanding of community dynamics. Each

evaluator served on only one panel topic. Particularly for the wildlife panels, effort

was made to select individuals who could set aside values of interested groups and

focus on the biological assessment task.

In addition, candidate evaluators, save one exception, were not substantively involved

in development of information specific to southeast Alaska that was relevant to the

panel’s evaluations. For example, the senior author of the wolf assessment (Person

and others 1996) was not considered as an evaluator for the wolf panel. He did, however, serve as the resource specialist during both wolf panels. In the overall makeup

of the panel, an attempt was made to obtain a range of expertise for both the species

and its geographic range, a mixture of research and management backgrounds, and

perspectives from habitat and population viewpoints. A limited number of true experts

are available in each subject area, and compliance with FACA guidelines further limited the pool of candidates. Thus, even though the effort was made, there were limits

to the diversity of participants’ backgrounds and the likely existence of particular bias

or preconceived ideas. 

Facilitators were planning team members who were trained and knowledgeable in the

risk assessment process. They conducted the panels and encouraged questions and

discussion as appropriate to clarify issues while striving to keep any one member

from dominating the group. Furthermore, facilitators attempted to stimulate thinking
5

and interchange, and identify opportunities for improving the process, probe for consistency, biases, and misunderstandings. In combination with the local resource specialist, each facilitator was encouraged to have evaluators comment on the amount

and quality of available information regarding the species and its habitat needs in

southeast Alaska. 

Scribes were Federal employees who had enough familiarity with the subject matter

and process to be able to capture the discussion. They recorded the likelihood ratings (app. 2) and captured in note form the discourse during the session. These

notes were used to clarify and track points cited by the evaluators and to support

later interpretations.

Identification of specific areas where information was lacking was considered valuable to development of the information needs and monitoring sections of the Forest

plan (USDA Forest Service 1997a). After the process was complete, the facilitator, in

coordination with the local resource specialists and the scribe, was responsible for

preparing the report on that panel for the planning record (app. 1; also USDA Forest

Service 1997d).

Silent observers also were Federal employees. They were not necessarily subject

matter experts, but they were considered to be capable of judging objectivity. The

silent observer’s role was to provide a short, written evaluation to the scribe at the

end of the panel deliberations on the merits of the process and its level of objectivity.

If silent observers or scribes had any concerns with the process during panel deliberations, they were directed to discuss these concerns in private with the facilitator. In

contrast to evaluators, scribes and silent observers often served on more than one

panel. Lists of duties for the facilitator, scribe, and silent observer are in appendix 3.

Rating mechanism and outcomes, wildlife species—Because of the inherent

uncertainty in these judgments, a “likelihood” approach was used to assess levels of

risk for most wildlife species. In this system, evaluators individually assigned 100

“likelihood” points by alternative to all projected outcomes. Allocation of all 100 points

to a single outcome was considered an expression of complete certainty in that particular outcome, whereas complete uncertainty was expressed by assigning 20 points

to each of the five outcomes. Degrees of uncertainty were expressed by spreading

points across the outcomes, as will be described later. These ratings were not done

in an information void because all available data and pertinent information were

made available to evaluators before points were assigned to outcomes. 

Results from this assignment of likelihood points do not represent probabilities in the

classic sense of frequencies; rather, they represent degrees of belief in future outcomes that are based on reasoned professional judgment and expressed in a probabilitylike scale. The nature of this approach is important to consider for several reasons, including the limited appropriateness of various statistical analysis procedures.

The form used for recording assignment of likelihood points appears in appendix 2.
6

The five outcomes for the wildlife panels were:2

I. Habitat is of sufficient quality, distribution, and abundance to allow the species to

maintain well-distributed breeding populations across the Tongass NF. The concept of well distributed must be based on knowledge of the species’ distributional

range and life history. 

II. Habitat is of sufficient quality, distribution, and abundance to allow the species to

maintain breeding populations distributed across the Tongass NF. However, some

local populations are more ephemeral because of reduced population levels and

increased susceptibility to environmental extremes and stochastic events associated with reduced habitat abundance and distribution. Vacated habitats may

become recolonized in the future.

III. Habitat is of sufficient quality, distribution, and abundance to allow the species to

maintain some breeding populations, but with significant gaps in the historic distribution in the Forest. These gaps are likely permanent and will result in some

limitation of interactions among local populations. The significance of gaps must

be judged relative to the species’ distributional range and life history.

IV. Habitat only allows continued species existence in refugia, with strong limitations

on interactions among local populations. The significance of extirpations across

islands or regional landscapes must be evaluated relative to the species’ distribution, range, and life history. 

V. Habitat conditions result in species extirpation from Federal land. 

In viewing these outcomes, evaluators were reminded that the archipelagic nature of

southeast Alaska and the extent of topographic variation were important components

to consider (see Burkey 1995). 

For each alternative, evaluators assigned their likelihood points across the five possible outcomes. They rated the risk to maintaining habitat sufficient to support a persistent, well-distributed population across the forest after 10 decades of implementation

of each management alternative. As such, individual evaluators independently compared each alternative to a preestablished set of outcomes rather than comparing

directly among alternatives. Note that the outcome scale is oriented to habitat rather

than population. For example, even if all evaluators were 100 percent certain that

outcome I would be achieved, this does not necessarily imply that the population

would achieve the associated level because of possible exogenous factors, as discussed later.

2 The assistance of Winston Paul Smith in developing these 

Tongass-specific outcomes for wildlife species from those used 

previously in the Pacific Northwest (FEMAT 1993) is gratefully 

acknowledged.
7

The panel on other terrestrial mammals followed this general procedure but evaluated two guilds of old-growth-associated mammals, one considered wide ranging and

the other endemic. Within each guild, evaluators selected what they considered to be

the most sensitive species or group of species to evaluate the effect of each alternative on the guild. For some alternatives, this group was comprised of a few or even

one species. This reflected, in part, differences in geographic distribution of many

island endemics and the geographic variation in human-caused disturbance represented among alternatives. In many circumstances, however, several species were

viewed as responding similarly to a selected alternative. Regardless, each guild was

treated similarly to the single-species panels by assigning only 100 points for each

alternative.

The 100-point rating mechanism was used for all wildlife panels except Sitka blacktailed deer. The assessment panel for Sitka black-tailed deer reevaluated a habitat

model designed to provide realistic estimates of deer habitat capability (a surrogate

for actual population levels) across the Forest (Suring 1992). Thus, the output of this

panel was unlike the other wildlife assessment panels. Rather than assigning likelihood points to outcomes, this panel identified variables important to high-quality deer

habitat, defined habitat types suitable for deer, and suggested habitat capability coefficients to use in a geographically based, computerized habitat model.

Rating mechanism and outcomes, fish-riparian panel—This panel had two 

sets of outcomes, one focusing on habitat condition and one on channel condition.

Accordingly, there were two sets of evaluators: four fisheries biologists assessing

habitat and two scientists with a geomorphology-hydrology background assessing

channel conditions. The fisheries biologists assessed the likelihood that implementation of each alternative would result in degradation of freshwater habitat for various

species of salmon. As with the wildlife panels, the focus was on likely changes in

habitat rather than in fish population levels. This panel used a different set of outcomes than those used by the wildlife panels (see below) and focused on habitat

sufficient to support sustainable populations rather than the viable and well-distributed criteria used in the wildlife panels. Overall conditions likely to occur over the

next 100 years were assessed; concerns regarding changes resulting from past management were considered in the discussion. This panel also evaluated effects after

10 years of alternative implementation, although these results were subsequently

used to a lesser extent. 

Risk assessments by alternative were made for anadromous and resident cutthroat

trout (Oncorhynchus clarki), steelhead (O. mykiss), anadromous and resident 

Dolly Varden char (Salvelinus malma), coho salmon (O. kisutch), pink salmon (O.

gorbuscha), chum salmon (O. keta), sockeye salmon (O. nerka), and chinook salmon

(O. tshawytscha). Each evaluator assessed likely habitat conditions, by alternative,

after 10 and 100 years of implementation and, as described above for the wildlife

panels, accordingly assigned likelihood points (100) to the following outcomes:3

3 The assistance of Gordon Reeves, Ronald Dunlap, Steve 

Kessler, and Fred Everest in developing these Tongass-specific 

outcomes for fish-riparian habitats from those used previously

in the Pacific Northwest (FEMAT 1993) is gratefully 

acknowledged.
8

I. New management activities will not cause additional degradation of freshwater

habitat for the species. Productive habitat will be well distributed across either the

Tongass NF or the historic range of the species within the Forest. Habitats that

are currently degraded will recover or will be moving toward recovery after 100

years.

II. New management activities will result in minor additional degradation of freshwater habitat for the species. Productive habitat will be adequately distributed

across the Tongass NF or the historic range of the species within the Forest.

Most habitats that are currently degraded will recover or will be moving toward

recovery after 100 years.

III. New management activities will result in moderate additional degradation of

freshwater habitat for the species. Distribution of productive habitat across the

Tongass NF, or the historic range of the species within the Forest, will contain

some gaps where the species will not occur or where populations will be

severely reduced. Many habitats that are currently degraded will not recover 

or will not be moving toward recovery after 100 years.

IV. New management activities will result in major additional degradation of freshwater habitat for the species. Distribution of productive habitat across the Tongass

NF, or the historic range of the species within the Forest, will contain large gaps

where the species will not occur or where populations will be severely reduced.

Most habitats that are currently degraded will not recover or will not be moving

toward recovery after 100 years.

V. New management activities will result in severe additional degradation of freshwater habitat for the species. The species will be extirpated or populations will be

decimated over much of its historic range on the Tongass NF. Habitats that are

currently degraded will not recover or will not be moving toward recovery after

100 years.

The physical scientists-hydrologists rated five possible outcomes for the potential

effects of land use on the natural conditions of streams. Natural conditions were

defined in terms of the following attributes:

• Large woody debris (pieces per 10,760 square feet of stream channel that are

greater than 4 inches in diameter and 3.3 feet long)

• Percentage of pool area

• Stream width-to-depth ratio

• Pools per reach

• Residual pool depth

• Stream bed grain size distribution 
9

Channel conditions were assessed (at 10 and 100 years of plan implementation) for

each alternative, and likelihood points (100) were assigned according to the following

outcomes: 

I. Riparian objectives will be met throughout the Tongass NF. There will be little or

no additional degradation from existing conditions due to new management activities. Areas not currently meeting riparian objectives will recover or will be moving

toward recovery in 10 or 100 years.

II. Riparian objectives will be met throughout most of the Tongass NF. There will be

minor additional degradation from existing conditions due to new management

activities. Areas currently not meeting riparian objectives will recover or will be

moving toward recovery in 10 or 100 years.

III. Riparian objectives will be met on much of the Tongass NF, but there will be substantial area where they are not met.

IV. Riparian objectives will be met on a small part of the Tongass NF, but they will not

be met over the majority of the Forest. There will be major additional degradation

from existing conditions due to new management activities. Most areas not currently meeting riparian objectives will not recover or will be not moving toward

recovery in 10 or 100 years. 

V. Riparian objectives will be met on a very small part of the Tongass NF. Almost all

areas will not meet riparian objectives. There will be severe additional degradation from existing conditions due to new management objectives. Areas not currently meeting riparian objectives will not recover or will not be moving toward

recovery in 10 or 100 years.

Rating mechanism and outcomes, old-growth ecosystems4—The old-growth

ecosystems panel also assigned likelihood points to outcomes, but they focused on

evaluating the persistence of interconnected and representative late-successional

ecosystems across southeast Alaska. The panel relied heavily on information contained in the forest’s scheme of 24 biogeographic provinces (USDA Forest Service

1991; fig.1) and closely followed the format and outcomes from FEMAT (1993). The

set of outcomes defined for this panel differed from those used for the wildlife species

assessments because evaluating ecosystems requires a broader perspective and different assessment criteria than examining habitat needs of individual species.

The ecosystem perspective emphasized the primary producers (e.g., plant communities), particularly the physical, chemical, and biological processes, including disturbance, that influence the dynamics of old-growth forest ecosystems. Projected effects

(over 100 years) of each Forest plan alternative on late-successional forests were

examined relative to variation in ecosystem quantity and quality and were expressed

as likelihood points distributed among four outcomes.

4 The assistance of Chris Iverson, Eugene DeGayner, and Tom 

Spies in developing these Tongass-specific outcomes for oldgrowth ecosystems from those used previously in the Pacific

Northwest (FEMAT 1993) is gratefully acknowledged.
This process identified three attributes that characterize the quality and quantity components of these ecosystems: (1) abundance and ecological diversity, (2) processes

and functions, and (3) connectivity. The panel evaluated each attribute, recognizing

that the features were probably not completely independent. Abundance refers to the

total area of forest that meets structural or functional criteria according to ecological

definitions and conditions within each biogeographical province. Old-growth community types were considered to be any assemblage of species that occurs within or

among old-growth forest types.

10

Figure 1—Biogeographical provinces used for the old-growth analysis during revision of the Tongass land

management plan.
11

In the Tongass NF, productive old growth is defined as an unharvested forested area

of trees more that 150 years old with more than 10 percent tree cover and an accumulated net saw-log volume of more than 8,000 board feet per acre (Julin and

Caouette 1997). The term “productive” refers to the utility of this forested area to provide wood products, although timber volume has been used in Forest planning as a

surrogate for wildlife habitat capability. The amount of productive old-growth forest

present in 1954, before the onset of large-scale industrial logging, was used as a

benchmark for comparisons of abundance in 2095; i.e., following implementation of

each alternative for 100 years. Ecological diversity reflected the spatial and temporal

distribution of late-successional assemblages across landscapes and the complexities

associated with varying local geographic, topographic, edaphic, or climatic features. 

Possible outcomes for maintaining abundance and ecological diversity of late-successional or old-growth ecosystems were:

I. Old growth is equal to or greater than long-term (i.e., 100 years) average and is

well distributed across environmental gradients, provinces, and community types.

II. Old growth is somewhat less than long-term average in forest types of some

provinces; represented in all major forest types, but underrepresented in some

types (may be within range of variability).

III. Old growth is below long-term average in most forest types; examples of a few

old-growth types are eliminated.

IV. Old growth is well below long-term average in all provinces; examples of several

old-growth types eliminated in some provinces.

The term “processes” refer to events and associated features or conditions influencing the development and maintenance of old-growth ecosystems at all spatial and

temporal scales. These include tree establishment, maturation, and death; canopy

gap formation and vegetation responses; understory development; small- and largescale disturbances, such as landslides and wind; decay and decomposition; nitrogen

fixation; canopy interception of energy or matter; and matter or energy transfer

among forest biota and between the forest and the biosphere. The term “functions”

refers to ecological values of a late-successional ecosystem or any of its components

that contribute to the maintenance of populations or species using these ecosystems

and contributing to the diversity or productivity of other ecosystems. Some examples

include habitat; climatic buffers; soil development, maintenance, or enrichment, such

as through inputs of coarse woody debris or nitrogen fixation; dispersion of biotic and

abiotic disturbance through landscapes; and nutrient cycles (e.g., production, storage, utilization, and decomposition).
12

Possible outcomes of the maintenance of ecological processes and functions of latesuccessional or old-growth ecosystems were as follows:

I. Full range of disturbance processes are represented; stand structure-dynamics

and landscape structure-dynamics-age attributes occur across all provinces.

II. Moderately wide range of disturbance processes; old-growth process and functions that are dependent on large, unaltered landscapes are limited, whereas oldgrowth process, structure, or functions dependent on a wide range of ages are

moderately limited.

III. Old-growth process, structure, or function limited in many provinces; many landscapes and stands too small or young to sustain old-growth processes, structure,

or functions or stand structure does not develop.

IV. Old-growth process, structure, or function is extremely limited or absent in some

provinces.

Connectivity is a measure of the extent to which the landscape pattern of the oldgrowth ecosystem provides for biological and ecological flows that sustain old-growthassociated animal and plant species across southeast Alaska. Connectivity does not

require that old-growth habitat be contiguous throughout a region. Rather, the spatial

distribution of old growth (at any one point in time) across landscapes should be such

that old-growth-associated species can readily move or be carried across areas that

are not in old-growth ecosystem conditions. Thus, landscape features affecting connectivity of old-growth ecosystems include distance among old-growth areas and forest conditions in the matrix between old-growth areas. The archipelagic nature of

southeast Alaska and the often dramatic topographic relief accentuates the implications of human disturbance and additional habitat fragmentation. For the panel

assessment, connectivity was compared to conditions in 1954 and emphasized connectivity within islands rather than among islands.

The four possible outcomes characterizing different levels of ecological connectivity

of old-growth forest communities and ecosystems evaluated in this assessment were:

I. Connectivity is as effective as it was prior to large-scale timber harvest.

II. Connectivity is effective and characterized by moderate distances between oldgrowth areas. Matrix contains high levels of old-growth elements and riparian 

corridors.

III. Connectivity is moderately effective and characterized by moderately wide 

distances between old growth and elements of old growth in the matrix (e.g.,

retention patch, riparian corridors, etc.).

IV. Connectivity is impotent. There are wide distances and limited presence of

connectivity elements in matrix.
Rating mechanism and indicators, socioeconomic panel5—The socioeconomic

panel used nine indicators of social and economic health rather than outcomes to

evaluate the impact of different Forest plan alternatives on 32 communities across

southeast Alaska. Instead of assigning likelihood points to represent the predicted

degree of effect of each Forest plan alternative, these evaluators independently

noted an overall direction of effect (positive, negative, or unchanged) for each indicator across every individual community (fig. 2 ). This process was similar to that used

previously by the Chequamegon and Nicolet National Forests in Wisconsin to identify

methods and principles for evaluating the social and economic impacts of implementing ecosystem management (Jakes and Harms 1995). Because of a concern that 

5 The assistance of Guy Cellier in developing Tongass-specific 

information for the socieoeconomic panel and subsistence 

working group is gratefully acknowledged.

13

Figure 2—An example of socioeconomic panel results by community, issue, and alternative.
14

factors exogenous to the Forest plan could have a possibly large but unknown effect

on these communities over extended time frames, this panel used a 10-year view of

the future rather than the 100-year view used by most other panels.

The list of socioeconomic indicators used in the evaluations was derived from previous experience with issues of importance to communities in southeast Alaska. For

each of the selected indicators, a brief written description of the issue and associated

implications were provided to panel evaluators, as described below:

1-4. Resource jobs: These indicators consider the type of job, how it may be affected by Forest management actions (both direct and indirect), seasonality, dollar

value of the job, etc. There were separate resource job categories for timber,

commercial fishing, tourism and recreation, and mining.

5. Economic structure and diversity: This indicator considers the structure of the

community and how it might change owing to the alternative. Is the community’s

economy dependent on one or many resources? Is the prevalent economic base

well diversified? and How is this likely to be affected by the alternative?

6. Community stability: How will the current stability within the community be

affected by this alternative, based on your knowledge of what this stability is

based on?

7. Quality of life: What constitutes the most important quality of life issues for the

community? and How could this alternative affect these issues?

8. Recreational opportunities: Is this alternative likely to affect recreational opportunities? Consider access, quality of the opportunity, travel time, increased road

density, and competition.

9. Access to traditional lifestyle: People in southeast Alaska have many different

requirements for maintaining their particular chosen lifestyle. Even though this

indicator category overlaps with some of the others above, are there additional

unique lifestyle aspects that may be enhanced (positive or up arrow) or diminished (negative or down arrow) by this alternative?

Rating mechanism, subsistence working group (see footnote 5)—The subsistence working group was structured more informally than were the panels. The rating

mechanism used for the subsistence evaluations therefore was less formal that those

described above. The group developed five indicators to help them focus on the likely

community-level, subsistence-related affects of each alternative. The focus was on

management actions that would affect subsistence use of deer and fish and to a lesser extent the gathering of berries, bark, and other items from the Forest. The indicators were: 

1. New urban access: Would new road access increase competition from nonsubsistence users (primarily from Juneau and Ketchikan)? 

2. New rural access: Would new road access increase competition for the 

specific community from other rural communities?
15

3. Fish streams crossed: An assumption was made that an increase in road density would result in some degradation to streams and subsequent reductions in fish

production and availability to subsistence users. The amount of timber harvest

and road density around each community (by alternative) were used as criteria to

evaluate the effect of this assumption.

4. Substantial part of remaining old growth cut in community home range:

Because old-growth forest in a community’s home range is important for a 

variety of subsistence resources besides deer (such as berries, bark, etc.), the

assumption was made that cutting a substantial part of a particular home range

would be detrimental to subsistence users.

5. Projected deer harvest >10 percent: An assumption was made that a deer harvest of more than 10 percent of the habitat capability in wildlife analysis areas6

(USDA Forest Service 1991) around communities could lead to a decline in deer

availability for subsistence users. To evaluate the impact of this assumption,

wildlife analysis areas exceeding a 10-percent harvest were highlighted on maps.

The habitat capability calculations were based on the model developed by the

Sitka black-tailed deer panel.

Primarily by responding to these criteria in a “yes or no” fashion and then aggregating

the results, the working group evaluated Forest plan alternatives by community or

groups of communities. Twenty years was used as an evaluation period to estimate

cumulative effects, the period considered necessary for the next generation to be

able to harvest deer.

Conducting the panels—Each risk assessment panel was comprised of a facilitator,

a local resource specialist, a scribe, a silent observer and several evaluators. Four

evaluators served on the wildlife, fish-riparian (only two for channel conditions), and

forest ecosystems panels, five on the socioeconomic panel, and seven on the subsistence working group. One to two weeks before each panel session, evaluators were

mailed brief descriptions of the alternatives to be evaluated, procedures for the panel

process, and information regarding the Tongass NF. Panel evaluators were asked to

familiarize themselves with this material before convening the panel. 

Except for the first American marten panel, which was held for logistical reasons in

Seattle, Washington, each panel met for 1-1/2 to 2 days in Juneau, Alaska. When 

it could be arranged, two or more panels were held concurrently. This approach

required less repetition in the presentation of general information provided to all 

panels. 

6 Flynn, R. June 5, 1989. Letter to TLMP Interdisciplinary 

Team explaining the new wildlife analysis areas. On file with: 

U.S. Department of Agriculture, Forest Service, Tongass National 

Forest Land and Resources Planning Record, P. O. Box 21628, 

Juneau, AK 99802-1628.
16

For about 4 hours at the start of the session, the entire group of evaluators, facilitators, scribes, silent observers, and local resource specialists for each set of panels

met together for an orientation session. The process began with introductory presentations regarding the environmental and cultural settings of southeast Alaska; the forest planning process, including regulations in the National Environmental Policy Act 

(U.S. Public Laws, Statutes 1970) and the National Forest Management Act (U.S.

Public Laws, Statutes 1976); and an overview of recent events affecting forest 

management in the Tongass NF (e.g., petitions to list as endangered the Queen

Charlotte goshawk and the Alexander Archipelago wolf) and the alternative development process used in this planning effort.

Following this general overview, a detailed description of the general assessment

procedures and panel rating process was presented with examples. This presentation

included a statement on the purpose of conducting the panels and an explanation of

how the information derived from them would be used by the planning team. It was

emphasized that this process was not designed to force consensus among panel

evaluators, which is somewhat in contrast to a true Delphi process (Linstone and

Turoff 1975). General questions regarding the process, the environment, and the

alternatives were answered.

The following information was referenced during the introductory presentations or

elsewhere in the panel process and made available for further review (this material is

all part of the planning record [USDA Forest Service 1997d]): the framework for

developing Forest plan alternatives; a map of land use designations (LUD) for each

alternative; a map of productive old growth across the Forest in 1954 and in 1994; a

map of value comparison unit (VCU)7 boundaries and total miles of existing roads;

Forest-wide standards and guidelines for each alternative; current harvest levels

stratified by VCU; and total acres and location map of projected harvest through 2095

for each Forest plan alternative. Medium (1,000 to 10,000 acres) and large (>10,000

acres) blocks of contiguous productive old growth were superimposed on each of the

alternative projection maps for 2095. Considerable additional information was provided to the fish-riparian panel, as indicated in appendix 3, which also lists additional

information provided to the various wildlife panels.

After this general meeting, each panel met separately. Each session started with a

focused introduction by the facilitator to review the role of all panelists, further review

the rating system and assignment of likelihood points among outcomes by evaluators, discuss outcomes with an objective of achieving a common understanding, provide further clarification on alternatives as needed, and introduce the local resource

specialist. The local resource specialist then gave a Tongass-oriented presentation

on the species or resource to be evaluated by the panel. For wildlife panels, habitat

issues specific to southeast Alaska were highlighted (e.g., island biogeography, natural fragmentation of habitat, land ownership patterns, past timber harvesting practices,

historical distribution of species).

7 Value comparison units (VCUs) are distinct geographic areas, 

roughly analogous to watersheds, each encompassing a drainage basin containing one or more large stream systems. The 

boundaries usually follow watershed divides. The Tongass NF 

has 926 VCUs averaging 18,000 acres in size.
17

This presentation included handout material, some of which had been made available

in advance to panel evaluators. Specific but neutral questions relative to the alternatives were addressed to the local resource specialists and answered, but at this

stage of the process there was no discussion allowed among evaluators on the attributes of alternatives. Discussions on the biology of the organism and its habitat requirements were allowed. This distinction was maintained to ensure to the extent

possible that the initial assignment of likelihood points to outcomes was done independently by each evaluator. The facilitator assured evaluators that discussion specific 

to each alternative would occur after the first assignment of likelihood points. 

Following the presentations and the question and answer session, each panel member individually made, without consultation, their assignment of likelihood points by

alternative. This process took an hour or more as evaluators consulted handout

material, their notes, and particularly the maps that projected distribution of old- and

young-growth forest after implementation of each alternative for 100 years.

After these initial, independent evaluations were completed and recorded by the

scribe (generally at the end of the first day), they were shared anonymously and the

facilitator began discussion. The purpose of this discussion was for evaluators to clarify why particular likelihood point assignments were given, clarify differing interpretations of available knowledge, identify knowledge gaps, and clarify how this lack of

information was handled by evaluators. Following the discussion, each panel member

could again assign likelihood points to each alternative. The facilitator emphasized

that the process was not designed to necessarily achieve a consensus rating for

each alternative; indeed, evaluators need not change any of their initial assignments

of likelihood points. As before, ratings of each evaluator remained anonymous; the

scribe recorded scores and computed a mean for each alternative. We thus had an

assignment of likelihood points by each evaluator before and after discussion. 

In the wildlife panels, the discussion also attempted to clarify which outcomes the

evaluators thought would represent habitat conditions suitable to maintaining welldistributed populations. Although no “viability minimum” for persistence of a well-distributed population across the Forest was presented, what particular changes could

be made to alternatives to concentrate assignment of likelihood points to either outcome I or II for each alternative was used as a guide in this discussion. The scribe

captured salient components of these discussions.

Somewhat in contrast to the wildlife panels, the socioeconomic panel began with a

discussion of the socioeconomic indicators, and the local resource expert presented

background information on each community to be evaluated. An initial independent

evaluation of each community occurred without discussion of Forest plan alternatives

or communities. Following evaluation of all communities in each area of the Forest

(Chatham, Stikine, and Ketchikan), some discussion occurred.

For all panels, discussions were structured to indicate specific alternative components that could be modified to improve the rating. For example, the discussion

attempted to clarify how each of the following components, as incorporated among

the various alternatives, influenced each panel member’s assignment of likelihood
Second Set of Risk

Assessment Panels

18

points: (1) amount of Forest land available for timber harvest, (2) width of riparian

buffers, (3) management activities within riparian buffers, (4) planned timber harvesting rotation (i.e., 100 years vs. 200 years), (5) size and distribution of old-growth

reserves, (6) width of estuary buffer, (7) width of beach fringe buffer, (8) watershedscale timber harvesting constraints, (9) constraints on harvesting timber in primary

winter habitat for Sitka black-tailed deer, (10) type of timber harvest (i.e., clearcutting

vs. single-tree or small-group selection), (11) additional and total miles of roads, and

(12) number of designated wild and scenic rivers. 

The facilitator also tried to ensure that assignment of likelihood points was not influenced by factors exogenous to the set of alternatives. For example, risk ratings for

murrelets should not be low because of a concern with loss of birds to oil spills or gill

netting. Such factors were outside the scope of the analysis and considered equally

influential on all alternatives. Discussions to be sure that this sort of information was

not used in evaluations also helped to elucidate how judgments were reached. 

The discussion attempted to clarify if there were specific biogeographic areas in the

Tongass NF where concern was extremely high and thus influenced the forest-wide

assignment of likelihood points. For example, if concerns about viability of the northern goshawk were particularly high for southern Baranof Island, then evaluators were

asked to characterize what, if any, area-specific mitigation procedures might improve

the situation. 

After the discussion and second evaluation were completed, panelists were thanked

for their participation and the session adjourned. The facilitator used the sets of evaluation forms (see apps. 2 and 5) and the scribe’s notes to prepare the panel report

(app. 1; also USDA Forest Service 1997d).

As the final alternative for the Tongass Forest plan was being developed, there was

considerable interest expressed by the public and within the Forest Service, the

Departments of Agriculture and the Interior, elsewhere in the Administration, and Congress regarding the risk that the alternative might pose to Forest resources, particularly wildlife and fish (neither alternative 10, the Forest Supervisors’ preferred alternative in the revised supplement to the draft environmental impact statement [RSDEIS;

USDA Forest Service 1996a], nor alternative 11, the then-emerging final alternative,

had been subjected to previous risk assessment panels). Recognizing and understanding this interest, the Director of the Pacific Northwest Research Station commissioned the PNW scientists on the planning team to conduct a second set of risk

assessment panels. They were conducted in March and April 1997 to provide further

information for the effects analysis in the final environmental impact statement for the

Forest plan. This set of six risk assessment panels evaluated the northern (Queen

Charlotte) goshawk, Alexander Archipelago wolf, brown bear, American marten, fisheries resources (anadromous and resident), and other terrestrial mammals. 
19

This second set of risk assessment panels was commissioned to subject alternatives

10 and 11 to this process and, in so doing, provide a context to help with interpretation of the results. This context was provided for by including some additional alternatives in the array evaluated by these panels. The additional alternatives included:

1. Alternatives 1 (current condition) from the RSDEIS (USDA Forest Service 1996a)

and 9* from the 1979 Forest plan (USDA Forest Service 1979): They provided

reference points to the alternatives previously evaluated by risk assessment panels. These reference points were important to the interpretation of results from

both sets of panels. Having panelists evaluate the historical condition (pre-1954,

before the onset of large-scale industrial logging) projected into the future as an

alternative, something not done in the first set of panels, also was done to aid

interpretation of results (Lehmkuhl and others 1997).

2. A draft of the paper by Everest and others (1997) was available to managers in

winter-spring 1997. In it, the authors inferred that if the then-emerging final alternative had been subjected to risk assessment panels for the northern goshawk

and the Alexander Archipelago wolf, then the results of those panels likely would

have fallen between those obtained for alternatives 1 and 5 in the first set of risk

assessment panels. Adding alternative 5 from RSDEIS to the array to be evaluated by this set of risk assessment panels allowed for testing that inference.

3. Alternative 9 from the 1979 Forest plan (USDA Forest Service 1979), as it existed

in winter-spring 1997, differed from the version paneled previously (9*) in that

updates to the database for lands suitable for timber harvest reduced the amount

of old-growth forest that would be harvested over 100 years by some 350,000

acres. Analysis by Everest and others (1997) suggested that area of old growth

harvested was an important habitat feature in the first set of risk assessment panels. Because the U.S. Fish and Wildlife Service was basing its then-in-progress

listing review of the northern goshawk and the Alexander Archipelago wolf under

the Endangered Species Act (U.S.Public Laws, Statutes 1973) on alternative 9 in

a mixed form (partially alternative 9 and partially 9* as described here), it was

considered desirable to evaluate both forms of alternative 9. 

4. There was considerable interest expressed during the public comment period

regarding alternative 2 in the RSDEIS (USDA Forest Service 1996b). In response

to this interest and to provide another reference point for analysis, this alternative

also was evaluated by the second set of panels.

Thus, this second set of panels assessed the risk of implementing alternatives 1, 2,

5, 9, 9*, 10, and 11 and the historical condition to maintaining well-distributed habitat

for the six species and groups listed above. It was not considered necessary to evaluate additional alternatives for benchmark information to aid in interpreting results for

alternatives 10 and 11. In addition, previous experience indicated possible impacts to

the utility of the process if panelists gathered in this format were asked to evaluate

more alternatives than was considered to be absolutely necessary. As indicated earlier, the process is not designed to be a fine-precision exercise, but rather one to provide decisionmakers and the public with informed professional judgments obtained

through a structured, objective process.
20

As with the primary process used in the first set of risk assessment panels, these

panels were designed to evaluate the likelihood that various Forest plan alternatives

would maintain enough well-distributed habitat to support viable, or sustainable in the

case of fish, populations of specifically identified, old-growth-associated species. As

with the previous wildlife panels, these evaluations were made under the assumption

that each management alternative being evaluated would be implemented for 100

years.

Evaluators for this second set of panels were the same individuals as previously for

the other terrestrial mammals and Alexander Archipelago wolf panels. The same

evaluators also were used for the fish-riparian panel, although only habitat conditions

were assessed and thus the physical scientists and hydrologists who participated in

the first panel were not involved in the second one. One evaluator changed on the

bear panel owing to a change in employer and thus his FACA status; two evaluators

changed on the American martin panel and one on the northern goshawk panel

owing to their unavailability. Newly selected evaluators met the criteria described previously. Facilitators, scribes, and silent observers were selected as described for the

first set of panels.

These panels were all held in Juneau, Alaska, and conducted under the same format

used for the first set of panels with two additions: an appraisal of alternative features

and a more structured discussion regarding viable and well-distributed wildlife populations.

Appraisal of alternative features—From reports from the first set of risk assessment panels (appendix I; also USDA Forest Service 1997d), a synthesis was prepared to clarify how panel evaluators considered that the various elements of draft

alternatives would contribute to maintaining wildlife viability (see table 3 in Everest

and others 1997). Because this synthesis was done after the fact, there was some

subjectivity in assigning values to the various features discussed by the panelists. To

avoid this postpanel process, each evaluator in the second set of panels was asked

to complete an alternative appraisal form (app. 5) as they did their assignment of likelihood points to alternatives. As described for the assignment of likelihood points during the first set of panels, this appraisal form was also filled in twice—before and

after alternative-specific discussion.

Indepth discussion of viable and well-distributed populations—During discussions in the first set of panels, the five outcomes were examined relative to maintaining habitat sufficient to support well-distributed and viable populations of various

species across the planning area. Notes from these discussions and some postpanel

phone calls to evaluators to help clarify their thoughts were used to derive for reporting purposes two new categories from the original five outcomes: (1) outcomes that

represented habitat conditions such that wildlife populations likely would remain well

distributed, and (2) outcomes that represented habitat conditions such that wildlife

populations likely would not remain well distributed.
Results and 

Discussion

21

Category 1 was generally perceived to represent outcomes I and II whereas outcomes III, IV, and V generally fell in category 2. Subsequent data presentations from

the first set of panels often used mean values from these derived outcomes to present the results (Everest and others 1997, Swanston and others 1996). This approach

was consistent with that used in FEMAT (1993).

The discussion on habitat sufficient to support “well-distributed” populations had limited structure in the first set of panels and did not address possible relations between

“well distributed ”and “viable.” To further aid understanding on these issues, the second set of panels had a rigorous, indepth discussion of these terms and how they

related to the habitats described in the five outcomes used in the assignment of likelihood points. These discussions were captured in the notes for this second set of

panels and are presented in the panel reports (app. 1; also USDA Forest Service

1997d).

With these additions and the availability of reports from the first set of panels, the

second set of panels proceeded in the same way as the first. The written reports

from these panels (app. 1; also USDA Forest Service 1997d) frequently reference the

earlier panel for the particular species and thus provide some further summary information from both panels.

During preparation for the second set of panels, concerns were raised about various

outside parties attempting to influence the process by contacting the evaluators

before their participation in these panels. To determine the extent of any such contact, all evaluators were asked during the second set of panels if they had received

any outside contact regarding their participation in this effort; none had. 

All 16 panels were completed in a timely manner under often hectic and contentious

circumstances.8 Results from all panels were prepared as individual reports that were

placed in the planning record (app. 1; also USDA Forest Service 1997d). From all

panels where likelihood points were assigned, all data were presented in the reports.

In the summary information, mean values from the second set of evaluations were

used for comparisons, as they were considered to be the best single indicator of the

results. Because of the nature of the data and the strong desire to not overstate the

degree of confidence in it, no statistical tests were conducted. 

As anticipated, these panel results became an integral component of the effects analysis section of the environmental impact statement (USDA Forest Service 1997b),

which provided the context for completing the Forest plan (USDA Forest Service

1997a). Besides the expressed desire to have risk assessments conducted on the

various management alternatives in the draft Forest plan, the need to use these

panel results in evaluating effects of alternatives on various fish and wildlife species

was heightened by the decision to not use outputs from habitat capability models

(Suring 1992) to compare alternatives, as had been done in the past (USDA Forest

8 For example, the original schedule for holding the risk assessment panels was interrupted twice by Federal government shutdowns, which were due in part to the controversy generated 

between Congress and the Administration by the planning effort 

for the Tongass NF.
22

Service 1991). The lack of spatial specificity in these models, as well as other concerns with their reliability, were primary components of the decision to not use them

in the effects analysis.

The exception to this approach was the habitat capability model for deer (Suring

1992), which was the focus of the deer panel. The panel discussed the model and

suggested certain changes to it. With these changes and some further postpanel

enhancements, it was used in projecting habitat capability for deer in the Forest plan

(USDA Forest Service 1997a). The importance of deer as the primary food source for

wolves was emphasized repeatedly in the wolf panel. The importance of deer as a

food resource to rural residents of southeast Alaska was also a primary feature of the

subsistence working group, which emphasized the need for habitat protection within

each community’s home range—factors considered in the final plan. 

Results from both fish-riparian panels were in close agreement with those in AFHA

(1995) which formed the base for development of the “AFHA consistent” (Everest and

others 1997) standard and guidelines in the Forest plan. Results from both goshawk

panels and both wolf panels were in general agreement with the information in the

assessments on these species in southeast Alaska (app. 1; also Iverson and others

1996, Person and others 1996), although some would suggest that the second wolf

panel led to a more liberal interpretation of conservation needs than did the assessment. 

The assessments and panels for these species, coupled with the more general

ecosystems panel, were integral to development of the conservation strategy in the

Forest plan (USDA Forest Service 1997a). This strategy was of heightened importance because of the deliberations by the U.S. Fish and Wildlife Service regarding

the possible threatened or endangered status for the wolf and goshawk in southeast

Alaska that were ongoing at the time.9 Results of the marbled murrelet panel were

also in concert with the assessment on that species (DeGange 1996) and contributed

to the overall conservation strategy.

Results from the first set of panel evaluations were available to the Forest Supervisors

as they prepared their preferred alternative (USDA Forest Service 1996a, 1996b).

Results from the second set of panels clearly were used by managers in writing the

final alternative in the Forest plan. This influence is evident in appendix N of the environmental impact statement (USDA Forest Service 1997b); Forest plan standards

and guidelines for goshawk, marten, and small mammals (USDA Forest Service

1997a); and language on these same species in the record of decision (USDA Forest

Service 1997c). Additional conservation measures were adopted by managers in part

to reduce the risk, as indicated by the panel results, of not maintaining habitat sufficient to support viable and well-distributed populations of these species. These items

were noted in the evaluation of the use of scientific information in the Forest plan

(Everest and others 1997). 

9 In August 1997, the U.S. Fish and Wildlife Service determined 

that listing of the Alexander Archipelago wolf and the northern 

goshawk in southeast Alaska as threatened or endangered was 

not warranted.
23

Another, perhaps indirect, use of the panel results in the Forest plan was in generating the “Information Needs” section (see app. B in USDA Forest Service 1997a).

Many of the priority information needs also were discussion topics during panel deliberations; for example, a priority information need deals with the relations between

socioeconomic conditions in rural communities and resource allocations in the

Tongass—a focus of the socioeconomic panel. In both of the other mammals panels,

the critical need for additional habitat use and distribution information for small,

endemic mammals was specifically noted and became a high-priority information

need. That the panel process contributed to defining information needs is in agreement with Cleaves’ (1995) suggestion that uncertainty assessments can help guide

information gathering and research. 

The second set of panels also specifically addressed the issue of habitat sufficient 

to support viable and well-distributed populations of various wildlife species. In the

panel evaluators’ view, the terms were not synonymous. The concept of well-distributed habitat was a more difficult condition to meet than was maintenance of habitat

sufficient to sustain viability. Well-distributed habitat dealt primarily with the likelihood

that modified habitat would no longer be capable of supporting a continuous territoryto-territory distribution of resident individuals or groups; i.e., some previously occupied territories might become “permanently” (within the 100-year time span) vacant.

For the concept of viability, evaluators for the second set of panels indicated that

populations could continue to interact and interbreed within their historical distribution

across the Tongass NF in the presence of some additional human-caused “permanent” (over the 100-year time span) gaps in habitat. The presence of additional gaps

did not necessarily mean that interaction among population segments would be

diminished.

The second wolf panel noted the ability of individual wolves to cross many miles of

habitat no longer occupied by wolves to interact and breed with other wolves. The

American marten panel understood that their concept of a habitat gap—being as

wide as a previously occupied home range—likely had little if any effect on species’

interactions or interbreeding. It would take the loss of several contiguous home

ranges (1 to 3 square miles each) to affect interaction enough to markedly influence

viability. 

The discussion on habitat sufficient to support viable and well-distributed populations

is interesting in its own right, particularly when one reflects on the viability direction

contained in the regulations (USDA Forest Service 1982) for implementing the National

Forest Management Act (Raphael and Marcot 1994; U.S. Public Laws, Statutes 1976).10

10 Federal regulations for implementing the National Forest Management Act (USDA Forest Service 1982 [36 CFR 219.19:43048])

state: “Fish and wildlife habitat shall be managed to maintain viable 

populations of existing native and desired non-native vertebrate 

species in the planning area.” A viable population is defined for 

planning purposes as “one which has the estimated numbers and 

distribution of reproductive individuals to insure its continued 

existence is well distributed in the planning area.” Furthermore, 

“habitat must be provided to support, at least, a minimum number 

of reproductive individuals and that habitat must be well distributed

so that individuals can interact with others in the planning area.”
24

This discussion also serves, however, to emphasize that interpretation of the panel

results is incomplete if the focus is only on the likelihood scores—as frequently happened with these results by many interested parties.

Because of the discussions regarding concepts of viable and well-distributed populations that occurred during the second set of panels, the results of these panels likely

reflect a more complete understanding of these concepts relative to the habitat outcomes than do the first set of panels. These discussions generally indicated that

habitat conditions associated with meeting the concepts fell somewhere within outcome III with well-distributed being a more stringent criterion to meet than viable. 

The likelihood of maintaining habitat sufficient to support viable and well-distributed

populations therefore was not fully captured by any one outcome and thus was summarized as the range in means between scores assigned to outcomes I and II, combined and outcomes I, II, and III combined (app. 1). This approach also portrays the

quantitative results of the panels as a range rather than perhaps causing a mistaken

perception of precision by providing only a single number.

The following tabulation shows the likelihood of maintaining habitat sufficient to support well-distributed and viable wildlife populations, as assigned by the second set of

panel evaluators to the then-emerging final alternative for the Tongass land management plan (alternative 11):

Likelihood of well

Species distributed and viable11

Northern goshawk 71 — 97 

Alexander Archepilago wolf 83 — 97 

Brown bear 68 — 93 

American marten 36 — 91 

Other mammals (endemic) 19 — 55 

Other mammals (widely distributed) 38 — 82 

For the second set of panels, a qualitative appraisal of the alternative features in

terms of their contributions to maintaining habitat was made by the evaluators (app.

5). Full interpretation of the panel information requires use of all these results, not

just the likelihood scores. The appraisal of alternative features and the panel notes

provide a credible context to track the assignment of likelihood points to outcomes.

Clearer direction for use and a modified format for this appraisal form likely would

have enhanced its utility; several evaluators expressed difficulty with it.

11 For the second set of risk assessment panels, the likelihood of 

maintaining habitat sufficient to support well-distributed and viable 

wildlife populations is appropriately presented as being within the 

mean of the scores that bracket outcome III, as this “zone” is where

the evaluators generally indicated that their concepts of viable and 

well distributed fell. Evaluators indicated that the likelihood of well

distributed equaled the sum of outcomes I and II (19) for endemic

mammals.
25

Some other items that could be considered to improve the process of conducting

such risk assessment panels include developing a checklist of specific items that the

silent observer needs to consider, making background information available to evaluators at least 2 weeks before convening the panel, enhancing clarity of how to deal

with the relations between the planning area and the portion of that area occupied by

National Forest or other Federal lands, and clearly addressing the possible anchoring

effect that may be generated by a first set of panels if the same evaluators are going

to participate in a second set of panels.

An often cited but erroneous interpretation of these panel results is that only a mean

value of 80 or more likelihood points assigned to outcomes I and II combined is an

acceptable score. This misconception likely developed from the FEMAT panel

process, where a summary statistic of 80 likelihood points assigned to their outcome

A (roughly comparable to our combined outcomes I and II) was used for comparison

purposes (Cleaves 1995, FEMAT 1993, Starkey 1998). In the Tongass plan revision

process, the PNW scientists on the team did not designate what level of risk was

acceptable for these panel results or for any other component of the manager’s decision (Everest and others 1997). Rather, the panel results served as documentation of

the levels of risk associated with an array of management alternatives about which

the managers made informed decisions. As indicated by Starkey (1998), the degree

of risk, as generated from such assessment panels, that is “acceptable” is a societal

decision, not one of science.

Although the sum of likelihood points assigned to outcomes I and II combined or outcomes I, II, and III combined was a commonly used parameter during discussions of

these panel results, another valid way to view the outcome scores is to focus on the

assignment of likelihood points to outcomes IV and V. These values represent the

likelihood that a species would exist only in refugia (outcome IV) or would be extirpated (outcome V) from the planning area in 100 years. Using American marten and the

emerging final alternative from the second set of panels as an example, there was a

mean score of 9 points assigned to outcome IV and none to outcome V. Thus, after

100 years of full implementation of alternative 11 across the Forest (e.g., the available sale quantity is harvested every year for 100 years), there was a high (91 points

out of 100) likelihood that habitat sufficient to maintain persistent populations of

American marten will occur in a spatial pattern considerably more dense than mere

refugia. There was zero likelihood that American marten would be extirpated from the

forest.

Another often misunderstood component of the panel evaluation process is that all

ratings represent risk levels under a maximum effects scenario; e.g., the maps for

each alternative portrayed effects of implementation at the full implementation level

(maximum allowable cut is harvested each year). In reality, this action is highly

unlikely, but the assumption was necessary to put comparisons among alternatives

on the same base. The 100-year timeframe was challenged by some as being far too

long because plans are revised every 10 years; it was criticized by others as being

far too short to properly capture the likely cumulative effects of management actions

on habitat modification. Because evaluations of the timber resource were made on a

150-year timeframe in this Forest plan (USDA Forest Service 1997a), 100 years

seems a reasonable timeframe within which to make the habitat evaluations.
26

Attention also was drawn to the different standard used in evaluating risk to various

wildlife species—viable and well-distributed populations—and that used for fish—sustainable levels. This choice was a management decision and not an inherent property

of the panel process, although it might have been difficult to conduct the fish-riparian

panel by using a concept of viable and well distributed and the result may have been

less useful to managers.

Like much of the effort associated with this revision of the Tongass land management

plan, the panel process for risk assessment and results derived from these received

intense public, political, and media scrutiny. Some insight into the nature of this

scrutiny can be gained by reading the proceedings from the April 18, 1996, Senate

Hearing on the Tongass Land Management Plan (U.S. Senate Hearing 104-670,

1996). For example, during this hearing, Alaska’s Senator Murkowski, chairman of

the Senate Committee on Energy and Natural Resources, likened the process

employed in the socioeconomic panel to use of a Ouija® board. His Alaskan colleague, Representative Young (in his role as chairman of the House Committee on

Resources), subsequently quoted rather factually from the results of this same panel

in a letter to the Alaska Regional Forester discussing impacts to local residents generated by the Tongass plan.12

The anxiety on the part of some regarding the risk assessment panels may have

resulted from the perception that a closed door process was used to conduct them 

in contrast to the perceived openness of the overall land management planning

process. Some would suggest that because it’s the public’s business, it should be

conducted in public. This argument also could be applied to other components of the

planning process not open to the public. Others indicated that by not having the risk

assessment panels open to the public, a tremendous educational opportunity was

lost. The other side of this argument is that the desire to have them open was really

a hidden attempt to try to influence the outcome.

Risk assessment panels clearly need to be conducted in an environment where the

assembled experts can focus on the task, a job made difficult in a public forum.

Cleaves (1995) suggests that anonymity and confidentiality of panel evaluators is

important, although these conditions were not maintained for the Tongass panelists.

In fact, the silent observer’s report from one panel was leaked to the press shortly

after completion of panel deliberations, which led to the false charge that because

one of the evaluators was the president of a local Sierra Club chapter (not true) the

panel was biased (Whitney 1996). Those desiring to use expert judgment in risk

assessment panels in the future to address management questions are encouraged

to consider mechanisms that make the process more available to the public without

sacrificing the integrity of the results.

12 Young, Don. 1998. Young wants Forest Service to increase 

assistance to Alaska communities harmed by Clinton’s antiTongass plan [Press release]. March 31. Washington DC: U.S. 

House of Representatives, Committee on Resources. 2 p. 
27

The expressed concern by certain individual and interest groups regarding the use of

risk assessment panels during the plan revision is perhaps a symptom of the larger

debate regarding application of risk assessment in whatever form to ecological problems. There is strong opinion about approaches and use of the resulting information

when uncertain outcomes are being estimated. Lackey (1996), for example, characterizes the spectrum of debate on use of risk assessment in an ecological context

from those who would like to mandate its use (including some in Congress and certain scientists) to those who characterize risk assessment as “scientifically indefensible, ethically repugnant and practically inefficient.” Todd and Burgman (1998) present

an interesting statistical treatment regarding realistic levels of uncertainty and reliability in such data. Lackey (1996) further comments that risk analyses of any significant

ecological problem will result in options that create both big winners and big losers—

a further reason why results from such risk assessment panels should be presented

as input to decisionmakers and not used as the sole basis for decisionmaking.

Another component of the closed versus open debate over these risk assessment

panels was the limitations that FACA (U.S. Public Laws, Statutes 1972) placed on the

selection of participants. Because of FACA direction, only Federal employees or

State of Alaska employees specifically appointed by the Governor of Alaska to represent him in the process could serve on the panels. This direction precluded selection

of any non-Alaska university scientists or various private sector scientists who likely

could have met the other necessary qualifications. With an apparent lack of appreciation of this legal limitation, several critics summarily dismissed the composition of the

panels as being inherently biased (toward a need for further conservation) and used

this as a basis to challenge the results (see Tongass land management plan appeals

97-13-00-0097 and 0103 and 0107 in USDA Forest Service 1997d). Interestingly,

these FACA guidelines were not followed in risk assessment panels for FEMAT

(FEMAT 1993) or the Columbia basin project (Lehmkuhl and others 1997).

The PNW scientists involved in the Tongass process stressed that their roles

involved production of value neutral information and their not advocating any particular outcome (Everest and others 1997, Mills and others 1998). In following this

approach, they used the panel results as an expression of risk based on the technical expertise of the evaluators. Overall, they found that the results provided a sound,

technically competent representation of the risks associated with the various management alternatives evaluated. 

There is, however, a particularly difficult problem when relying on the opinion of technical experts (instead of hard data): that of separating clearly their personal and

organizational values from their technical opinion (Lackey 1996, Philpot 1992). Our

mechanism for detecting such situations was use of the silent observer. It is not

known how successful this mechanism was, although only one instance of possible

bias was noted. One improvement might be to make available to the silent observer

a checklist of items to consider. Other ways of attempting to achieve objective results

when relying on professional judgment should be pursued.

Other than assuring that all three components of the process—the likelihood scores,

the discussion notes, and the appraisal of alternative features—were internally consistent, rankings were not “adjusted.” In FEMAT (1993) and the Columbia basin
Conclusion

28

assessment (Lehmkuhl and others 1997), certain rankings of risk assessment panels

were “changed” if the panel leader or other team members considered that errors or

misinterpretations had occurred. That such adjustments were not made with risk

assessment panel results during the Tongass revision was partially because the

PNW scientists assigned to the revision team had limited expertise in some of the

areas that were paneled. It also was not done because the scientists were concerned

that any changes might have been viewed as an attempt by them to achieve a particular outcome. However, in developing future risk assessment panels to address 

ecological problems, a mechanism to deal with this issue needs to be part of the

planning process.

This paper describes the process used to conduct the risk assessment panels for

revision of the Tongass land management plan, provides an overview of the results

and how they were used in decisionmaking, and discusses some reactions to the

effort. Although controversial, the results of the risk assessment panels did form a

basis for development of the environmental impact statement (USDA Forest Service

1997b), which provided the context for completing the Forest plan (USDA Forest

Service 1997a). Whatever the future holds for ecological risk assessment (Lackey

1994), I encourage those contemplating the use of this risk assessment technique to

learn from and build on the details and procedures presented here and elsewhere

about the Tongass experience. 

Based primarily on the FEMAT experience (Cleaves 1993, FEMAT 1993, Starkey

1998), Cleaves (1995) provides an excellent set of guidelines to consider when conducting risk assessment panels and communicating their results. In the absence of

“complete” (or that judged to be sufficiently comprehensive), data-based information,

this Tongass experience supports Cleaves’ (1995) view that assessments by subject

matter experts are most reliable when rigorous and systematic processes have been

employed to control and compensate for outside influences, inconsistencies, and

eccentricities inherent in human judgment. Furthermore, to be useful to decisionmakers, the assessment results must be timely and presented in a form that can be readily used. Followup procedures to evaluate how decisionmakes used results from risk

assessment panels and other information provided to them (Everest and others

1997) further enhances the value of the product.

1 inch = 2.54 centimeters

1 foot = 0.305 meter

1 square foot = 0.0929 square meter

1 acre = 0.405 hectare

1 mile = 1.61 kilometers

1 square mile = 2.60 square kilometers

Metric Equivalents
Literature Cited

29

Allen, S.D.; Robertson, G.; Schaefers, J. 1998. Economies in transition: an assessment of trends relevant to management of the Tongass National Forest. Gen.

Tech. Rep. PNW-GTR-417. Portland, OR: U.S. Department of Agriculture, Forest 

Service, Pacific Northwest Research Station. 101 p. (Shaw, Charles G., III, tech. 

coord.; Conservation and resource assessments for the Tongass land management plan revision).

Anadromous Fish Habitat Assessment Team [AFHA]. 1995. Report to Congress: 

anadromous fish habitat assessment. R10-MB-279. Juneau, AK: U.S.Department 

of Agriculture, Forest Service, Pacific Northwest Research Station; Alaska Region.

[Irregular pagination].

Burkey, T.V. 1995. Extinction rates in archipelagos: implications for populations in 

fragmented habitats. Conservation Biology. 9: 527-541.

Cleaves, D.A. 1993. Out of uncertainty into forest policy: learning from the FEMAT

risk assessments. In: Proceedings of the 1993 Society of American Foresters 

national convention; [1993] November 7-10; Indianapolis. [Place of publication 

unknown]: [publisher unknown]: 262-270.

Cleaves, D.A. 1995. Assessing and communicating uncertainty in decision supportsystems: lessons from the an ecosystem policy analysis. AI Applications. 9(3): 87-

102.

Cultural Survival Inc. 1998. Cultural Survival Quarterly [Crisis in the last frontier—

the Alaska subsistence debate]. Cambridge, MA; 22(3). 81 p.

DeGange, A.R. 1996. A conservation assessment of the marbled murrelet in southeast Alaska. Gen. Tech. Rep. PNW-GTR-388. Portland, OR: U.S. Department of 

Agriculture, Forest Service, Pacific Northwest Research Station. 72 p. (Shaw, 

Charles G., III, tech. coord.; Conservation and resource assessments for the 

Tongass land management plan revision).

Everest, F.H.; Swanston, D.N.; Shaw, C.G., III [and others]. 1997. Evaluation of 

the use of scientific information in developing the 1997 Forest plan for the Tongass

National Forest. Gen. Tech. Rep. PNW-GTR-415. Portland OR: U.S. Department 

of Agriculture, Forest Service, Pacific Northwest Research Station. 69 p. (Shaw, 

Charles G., III, tech. coord.; Conservation and resource assessments for the 

Tongass land management plan revision).

Forest Ecosystem Management Assessment Team [FEMAT]. 1993. Forest 

ecosystem management: an ecological, social, and economic assessment. 

Portland, OR: U.S. Department of Agriculture; U.S. Department of the Interior 

[and others]. [Irregular pagination].

Iverson, G.C.; Hayward, G.D.; Titus, K. [and others]. 1996. Conservation assessment for the northern goshawk in southeast Alaska. Gen. Tech. Rep. PNW-GTR387. Portland, OR: U.S. Department of Agriculture, Forest Service, Pacific 

Northwest Research Station. 101 p. (Shaw, Charles G., III, tech. coord.; 

Conservation and resource assessments for the Tongass land management plan 

revision).
30

Iverson, G.C.; Rene, B. 1997. Conceptual approaches for maintaining well-distributed, viable wildlife populations: a resource assessment. In: Julin, K.R., comp. 

Assessments of wildlife viability, old-growth timber volume estimates, forested wet

lands, and slope stability. Gen. Tech. Rep. PNW-GTR-392. Portland, OR: U.S. 

Department of Agriculture, Forest Service, Pacific Northwest Research Station: 

1-23. (Shaw, Charles G., III, tech. coord.; Conservation and resource assessments for the Tongass land management plan revision). 

Jakes, P.; Harms, J. 1995. Report on the socioeconomic roundtable convened by 

the Chequamegon and Nicolet National Forests. Gen. Tech. Rep. NC-177. St. 

Paul, MN: U.S. Department of Agriculture, Forest Service, North Central Forest 

Experiment Station. 62 p.

Julin, K.R.; Caouette, J.P. 1997. Options for defining old-growth timber volume strata: a resource assessment. In: Julin, K.R., comp. Assessments of wildlife viability, 

old-growth timber volume estimates, forested wetlands, and slope stability. Gen. 

Tech. Rep. PNW-GTR-392. Portland, OR: U.S. Department of Agriculture, Forest 

Service, Pacific Northwest Research Station: 24-37. (Shaw, Charles G., III, tech. 

coord.; Conservation and resource assessments for the Tongass land manage

ment plan revision).

Kiester, A.R.; Eckhardt, C., eds. 1994. Review of wildlife management and conservation biology on the Tongass National Forest: a synthesis with recommendations.

Corvallis, OR: U.S. Department of Agriculture, Forest Service, Pacific Northwest 

Research Station. 282 p.

Lackey, R.T. 1994. The future of ecological risk assessment. Northwest Science. 

69(2): 171-174.

Lackey, R.T. 1996. Is ecological risk assessment useful for resolving complex ecological problems? In: Stouder, D.J.; Bisson, P.A.; Naiman, R.J., eds. Pacific 

salmon and their ecosystems: status and future options. New York: Chapman and 

Hall: 525-540. 

Lehmkuhl, J.F.; Raphael, M.G.; Holthausen, R.S. [and others]. 1997. Historical 

and current status of terrestrial species and the effects of proposed alternatives. 

In: Quigley, T.M.; Lee, K.M.; Arbelbide, S.J., tech. eds. Evaluation of EIS alternatives by the Science Integration Team, Volume II. Gen. Tech. Rep. PNW-GTR-406.

Portland, OR: U.S. Department of Agriculture, Forest Service, Pacific Northwest 

Research Station: 537-730. Chap. 4.

Linstone, H.A.; Turoff, M., ed. 1975. The Delphi method: techniques and applications. Reading, MA: Addison-Wesley. 620 p.

MacDonald, S.O.; Cook, J.A. 1996a. The land mammal fauna of southeast Alaska. 

Canadian Field Naturalist. 110(4): 571-598.
31

MacDonald, S.O.; Cook, J.A. 1996b. The mammals of southeast Alaska. A distribution and taxonomic update. Fairbanks, AK: The University of Alaska Museum. 

150 p.

Mills, T.J., Everest, F.H.; Janik, P. [and others]. 1998. Science/management collaboration: lessons learned from the revision of the Tongass National Forest plan. 

Western Journal of Applied Forestry. 13: 90-96.

Murphy, M.L.; Koski, K.V. 1989. Input and depletion of woody debris in Alaska 

streams and implications from streamside management. North American Journal 

of Fish Management. 9: 427-436.

Paustian, S.J.; Murphy, M.L.; Kessler, S.J.; Starostka,V.J. 1990. Coho salmon and

Dolly Varden char habitat capability for the Tongass National Forest management 

plan revision: analysis of the management situation. R10-MB-89. Juneau, AK: 

Alaska Region. 13 p.

Person, D.K.; Kirchoff, M.; Van Ballenberghe, V. [and others]. 1996. The

Alexander Archipelago wolf: a conservation assessment. Gen. Tech. Rep. PNWGTR-384. Portland, OR: U.S. Department of Agriculture, Forest Service, Pacific 

Northwest Research Station. 42 p. (Shaw, Charles G., III, tech. coord.; Conservation and resource assessments for the Tongass land management plan 

revision).

Philpot, C. 1992. Institutional change: science, policy, and management. In: Adams, 

P.W.; Atkinson, W.A., comps. Watershed resources: balancing environmental, 

social, political, and economic factors in large basins. Corvallis, OR: Oregon State

University Press: 127-130.

Raphael, M.G.; Marcot, B.G. 1994. Species and ecosystem viability: key questions 

and issues. Journal of Forestry. 92(4): 45-47.

Starkey, E.E. 1998. Assessing effect of forest management on biodiversity in the 

Pacific Northwest, USA. In: Bachmann, P.; Kohl, M.; Paivnen, R., eds. Assessment of biodiversity for improved forest planning, conference proceedings; 

1996 Oct. 7-11; Monte Verita, Switzerland. Boston, MA: Kluwer Academic 

Publishers: 389-396.

Suring, L.H., ed. 1992. Habitat capability models for wildlife in southeast Alaska. 

Unnumbered Publ. [Place of publication unknown]: U.S . Department of 

Agriculture, Forest Service, Alaska Region. 42 p.

Suring, L.H.; Crocker-Bedford, D.C.; Flynn, R.W. [and others]. 1993. A proposed 

strategy for maintaining well-distributed, viable populations of wildlife associated 

with old-growth forests in southeast Alaska. [Place of publication unknown]: U.S. 

Department of Agriculture, Forest Service, Alaska Region; report of the interagency committee, Tongass Land Management Planning Team. 278 p. On file with: 

U.S. Department of Agriculture, Forest Service, P. O. Box 21628, Juneau, AK 

99802-1628.
32

Suring, L.H.; Crocker-Bedford, D.C.; Flynn, R.W. [and others]. 1994. Response 

to the peer review of: a proposed strategy for maintaining well-distributed, viable 

populations of wildlife associated with old-growth forests in southeast Alaska. 

[Place of publication unknown]: U.S. Department of Agriculture, Forest Service, 

Alaska Region; response of the interagency committee, Tongass Land Management Planning Team. 11 p. On file with: U.S. Department of Agriculture, 

Forest Service, P. O. Box 21628, Juneau, AK 99802-1628.

Swanston, D.N. 1997. Controlling stability characteristics of steep terrain with discussion of needed standardization for mass movement hazard indexing: a resource 

assessment. In: Julin, K.R., comp. Assessments of wildlife viability, old-growth timber volume estimates, forested wetlands, and slope stability. Gen. Tech. Rep. 

PNW-GTR-392. Portland, OR: U.S. Department of Agriculture, Forest Service, 

Pacific Northwest Research Station: 44-58. (Shaw, Charles G., III, tech. coord.; 

Conservation and resource assessments for the Tongass land management plan 

revision).

Swanston, D.N.; Shaw, C.G., III; Smith, W.P. [and others]. 1996. Scientific information and the Tongass land management plan: key findings derived from the 

scientific literature, species assessments, resources analyses, workshops, and 

risk assessment panels. Gen. Tech. Rep. PNW-GTR-386. Portland, OR: U.S. 

Department of Agriculture, Forest Service, Pacific Northwest Research Station. 

30 p. (Shaw, Charles G., III, tech. coord.; Conservation and resource assessments

for the Tongass land management plan revision).

Todd, C.R.; Burgman, M.A. 1998. Assessment of threat and conservation priorities 

under realistic levels of uncertainty and reliability. Conservation Biology. 12(5): 

966-974.

U.S. Department of Agriculture, Forest Service. 1979. Tongass land and resource 

management plan. Juneau, AK: Alaska Region.

U.S. Department of Agriculture, Forest Service. 1982. National Forest System 

land and resource planning, code of Federal regulations 36, part 219. Federal 

Register 47 (190). 43026-43052. 

U.S. Department of Agriculture, Forest Service. 1991. Tongass land management 

plan revision: supplement to the draft environmental impact statement. R10-MB149. Juneau, AK: U.S. Department of Agriculture, Forest Service, Alaska Region.

U.S. Department of Agriculture, Forest Service. 1992. Channel type user guide—

Tongass National Forest, southeast Alaska. Tech. Rep. 26. Juneau, AK: Alaska 

Region. 179 p.

U.S. Department of Agriculture, Forest Service. 1995. Watershed restoration strategy. Juneau, AK: Alaska Region. 26 p.
33

U.S. Department of Agriculture, Forest Service. 1996a. Tongass land management plan revision: revised supplement to the draft environmental impact statement (RSDEIS); environment and effects. R10-MB-314A. Juneau, AK: Alaska 

Region. Chap. 3. 

U.S. Department of Agriculture, Forest Service. 1996b. Tongass land management revision: proposed revised Forest plan. Juneau, AK: Alaska Region.

U.S. Department of Agriculture, Forest Service. 1996c. Soil and water conservation handbook (FSH2509.22-96-1). Juneau AK: Alaska Region.

U.S. Department of Agriculture, Forest Service. 1997a. Tongass National Forest 

land and resource management plan. Juneau, AK: Alaska Region.

U.S. Department of Agriculture, Forest Service. 1997b. Tongass National Forest 

land and resource management plan: final environmental impact statement 

(FEIS). Juneau, AK: Alaska Region.

U.S. Department of Agriculture, Forest Service 1997c. Tongass National Forest 

land and resource management plan: record of decision (ROD). Juneau, AK: 

Alaska Region.

U.S. Department of Agriculture, Forest Service. 1997d. Tongass National Forest 

land and resources planning record. Juneau, AK: Alaska Region. On file with: 

U.S. Department of Agriculture, Forest Service, Tongass Land Management 

Planning Office, P. O. Box 21628, Juneau, AK 99802-1628.

U.S. Public Laws, Statutes, etc. 1970. Public Law 91-190 as amended. National 

Environmental Policy Act of 1969. 42 U.S.C. 4321-4347.

U.S. Public Laws, Statutes, etc. 1972. Public Law 92-463 as amended. Federal 

Advisory Committee Act of 1972 [FACA]. 5 U.S.C. 552.

U.S. Public Laws, Statutes, etc. 1973. Public Law 93-205. Endangered Species Act

of 1973. 16 U.S.C. 1531-1536, 1538-1540.

U.S. Public Laws, Statutes, etc. 1976. Public Law 94-588. National Forest 

Management Act of 1976. 16 U.S.C. 1604, section 6 (e)(1).

U.S. Public Laws, Statutes, etc. 1980. Public Law 96-487. Alaska National Interest 

Lands Conservation Act [ANILCA] of 1980. Act of Dec. 2, 1980. 16 U.S.C.3210.

U.S. Public Laws, Statutes, etc. 1990. Public Law 101-626. Tongass Timber Reform

Act of 1990. 16 U.S.C. 539d (1990).

U.S. Senate Hearing 104-670. 1996. Tongass Land Management Plan. Senate 

Committee on Energy and Natural Resources, Frank H. Murkowski, chair. 406 p.

Whitney, D. 1997. Murkowski questions Forest Service view of Tongass plan. 

Anchorage Daily News. April. 6; Sect. B: 2.
Appendix 1

34

This appendix consists of the 16 individual risk assessment panel reports and the

report of the subsistence working group. The original documents are on file in the

planning record for the Tongass National Forest plan revision (USDA Forest Service

1997d). The items listed in this appendix are accessible at the web site for the USDA

Forest Service, Pacific Northwest Research Station: http://www.fs.fed.us/pnw. Hard

copies may be obtained from the author or from the Documents Coordinator, Records

of the Tongass Land Management Plan, USDA Forest Service, Alaska Region, P.O.

Box 21628, Juneau, Alaska 99802-1628. 

Authors and titles are as follows:

First set of panels—

Cellier, G. 1995. Subsistence working group results.

Cellier, G. 1996. Socioeconomic panel results.

DeGayner, G. 1996. TLMP revision old-growth assessment panel summary.

DeGayner, G. 1996. Sitka black-tailed deer panel.

Dunlap, R. 1996. Fish-riparian assessment panel summary.

Iverson, C. 1996. American marten viability assessment panel summary.

Iverson, C. 1996. Alexander Archipelago wolf viability assessment panel summary.

Iverson, C. 1996. Brown bear viability assessment panel summary.

Iverson, C. 1996. Northern goshawk viability assessment panel summary.

Julin, K. 1995. Other terrestrial mammals panel assessment summary.

Smith, W. 1996. Marbled murrelet panel—panel assessment meeting notes.

Second set of panels—

DeGayner, G. 1997. Summary of the 1997 American marten risk assessment panel.

Dunlap, R. 1997. Summary of the 1997 fish habitat risk assessment panel.

Iverson, C. 1997. Summary of the 1997 other terrestrial mammals assessment panel.

Iverson, C. 1997. Summary of the 1997 Alexander Archipelago wolf risk assessment 

panel.

Iverson, C. 1997. Summary of the 1997 northern goshawk risk assessment panel.

Meade, C. 1997. Brown bear risk assessment panel summary.
Appendix 2

Evaluator Form

35

This form was used to record the assignment of likelihood points in both sets of risk

assessment panels. 
Appendix 3

Specific Duties

36

The specific duties of panel facilitators, scribes, resource specialists, and silent

observers are outlined.

Facilitator—

• Explains roles of all participants in process.

• Explains rating system and assignment of likelihood points to outcomes.

• Discusses outcomes; makes sure they are clearly understood by each evaluator.

• Explains appraisal of alternative features (this occurred during the second set of

panels).

• Provides further clarification on alternatives as needed.

• Introduces resource specialist.

• Explains that after the presentation by resource specialist there will be a questionand-answer session, but at this stage it should not develop into a discussion

among evaluators on the alternatives themselves. Explains that discussion regarding the biology of the organism and its habitat requirements is allowed. Reminds

evaluators that discussion specific to each alternative will occur after the assignment of likelihood points.

• Leads question-and-answer session.

• After question-and-answer session, hands out likelihood scoring sheets and features appraisal sheet (second set of panels) and again asks if there are any questions about the rating system.

• Monitors ratings to maintain evaluator independence.

• Works with scribe to enter likelihood point assignments on the computer after all

ratings and appraisals have been completed (most likely this will end the first day

of the session).

• Presents data to panel and shows where their dispersion is the greatest in assignment of likelihood points and appraisal of features. Uses these differences and

those noted by the resource specialist to start discussion.

• Leads discussion; ensures that the following are covered, by alternative:

✦ Clarifies why there were differences among panelists, if there were differences.

✦ Clarifies why there were differences among evaluators’ ratings and the resource

specialist’s ideas, if there were differences. 
37

✦ Clarifies how lack of knowledge was handled.

✦ Clarifies how a concern about one part of the Forest translated to a Forestwide likelihood rating. Notes all area-specific locations of concern.

✦ Discusses each component on the matrix chart in terms of how changes to it

might increase the likelihood of obtaining higher level outcomes for specific

alternatives (the appraisal of alternative features was included in the second set

of panels).

✦ Discusses how adoption of other Forest plan standards and guidelines might

raise likelihood ratings.

✦ Clarifies what exogenous factors might exist and determines if they influenced

the ratings (goal is that they should not).

• After discussion, allows evaluators to again rate alternatives.

• Works with scribe and resource specialist to structure key points for the panel

report.

Scribe—

• Takes comprehensive but not verbatim notes of the panel session.

• Captures questions and answers from evaluators, facilitator, and resource 

specialist.

• Captures key points of presentation by resource specialists.

• Captures questions and answers among evaluators and resource specialists (does

not necessarily need to identify questions by person).

• Records likelihood scores and appraisal of alternative features directly into spreadsheets on computer; with appropriate help, calculates the summary statistics and

prints for subsequent discussion.

• Presents summary statistics to facilitator.

• Captures discussion in notes; tries to keep track of all salient points (does not need

to identify points by person).

• Stops discussion if necessary to clarify points for the notes.

• Ensures that facilitator covers all points highlighted for discussion (see above

description of facilitator duties).

• Enters the second set of likelihood scores and data from the appraisal of alternative features (second set of panels) into the computer.
38

• Provides copy of notes to the facilitator as soon as possible.

• Appends silent observer’s writeup to the notes.

• Revises panel notes after review by facilitator and resource specialists.

• Assists facilitator with use of notes in developing panel report.

Resource specialist—

• Describes input materials prepared for the panel.

• Presents what is known and not known about the species and its habitat needs in

the Tongass NF. Clarifies fact from opinion; notes if there are different interpretations in the literature (about 1 seems appropriate).

• After the presentation, answers individual questions raised by evaluators and participates in any discussion on the species and its habitat needs. This is not to be

an alternative-specific discussion; that will occur after evaluators have assigned

their likelihood points.

• Assists the facilitator with discussion as necessary.

• Reads scribe’s notes and, along with facilitator, helps edit draft.

• Assists facilitator with development of key points for panel report.

Silent observer—

• Observes the entire panel process without commenting (may discuss any observations or concerns with the facilitator during breaks).

• Takes notes on panel proceedings to use in developing an evaluation.

• Provides the scribe with a written evaluation of the process at the end of the panel

meeting. The evaluation should focus on the objectivity of the process (did we do

what we said we were going to do in a value neutral manner) and on all the individuals participating (e.g., note any “overdominance or meekness”). If any exogenous

factors affecting the assignment of likelihood scores were detected, then these

should be noted.

A suggested addition to these duties for future consideration is to provide the silent

observer with a specific checklist of items on which they are asked to comment.
Appendix 4

Additional Information 

Provided to Panelists

39

Fish-riparian panels—The fish-riparian risk assessment panels had a thorough

briefing to acquaint panelists with the fish and stream resources of the Tongass NF

(including past management activities), as well as information on strategies considered for future management that could affect aquatic resources. Although some panelists were quite familiar with the Tongass NF, they all needed to have a similar

knowledge base before assessing effects of land-use alternatives on fish habitats.

They therefore were provided with the following information (unless noted otherwise,

these items are available from the planning record, USDA Forest Service 1997d):

• The classification system for stream channel types and process groups in the

Tongass NF (USDA Forest Service 1992).

• The array of fish, riparian, soil, water, and wetlands standards and guidelines used

in the Forest plan alternatives (from the proposed Forest-wide standards and

guidelines; USDA Forest Service 1996b, 1997d).

• A list of management objectives for riparian areas in the Tongass NF.

• A description of how information on channel process groups (USDA Forest Service

1992) and riparian prescriptions were integrated and applied in the Forest plan

alternatives. Forest plan alternative maps and an alternative matrix were displayed.

• A map of current roadless areas in the Tongass NF.

• Existing roads per square mile by VCU (see text footnote 7).

• The miles of roads anticipated, by alternative, Forest-wide, in 10 years and 100

years. Both the total miles of roads and maintained miles of roads were shown.

• The number of acres of projected timber harvest on mass movement hazard index

soil classes 1, 2, 3, and 4 (Swanston 1997), by alternative, Forest-wide, in 100

years.

• The “best management practices” used to manage riparian and aquatic habitat

(USDA Forest Service 1996c).

• A fish habitat reserve strategy previously designed for the Tongass NF (see watershed reserve strategy in USDA Forest Service 1997d).

• A system that rates the importance of individual watersheds for fish production

(USDA Forest Service 1979; see the forest habitat integrity plan in USDA Forest

Service 1997d).

• A strategy for managing areas in the Tongass NF with unstable soils (Swanston

1997).

• A monitoring plan to assess the effectiveness of management strategies for protecting fish and riparian habitats in the Tongass NF (USDA Forest Service 1996b,

1997a).
40

• The Anadromous Fisheries Habitat Assessment (AFHA 1995).

• Distribution map of streams across the Tongass NF (see streams map in USDA

Forest Service 1997d).

• A watershed restoration strategy for the Tongass NF and Alaska Region (USDA

Forest Service 1995).

• Information regarding the delivery of woody debris to streams in southeast Alaska

(Murphy and Koski 1989). 

• Fish model outputs that project usage of habitat capability by management area for

coho salmon 120 years into the future (Paustain and others 1990, USDA Forest

Service 1997d).

• Acres of forest land by alternative to be managed for timber harvesting (aggregated within VCUs [see text footnote 7] by 10-percent increment categories).

• Miles of streams affected by timber harvest or roads by Forest plan alternative.

• Number of acres of land experiencing timber harvest within 100 feet of streams in

the past decade.

• Cumulative acres of timber previously harvested in riparian areas.

• Number of acres and miles of streams in Forest areas within legislated reserves.

• Acres across the Tongass NF classified as riparian.

• Fish harvest statistics over the past three decades.

• A list of forest streams considered impaired by the Alaska Department of

Environmental Conservation (USDA Forest Service 1997d).

• Distribution maps for resident and anadromous fish on the Tongass NF.

The following topics were explored during the discussion that followed the rating of

Forest plan alternatives by the panel evaluators:

• Adequacy of process group direction (USDA Forest Service 1992) by riparian management option.

• Effectiveness of each riparian management option, including tradeoffs between

risks to fish habitat and the potential for timber production.

• Basic differences in alternatives (i.e., partial cutting or clearcutting) and the resulting potential effects on fish.
41

• Potential effectiveness of “best management practices” (USDA Forest Service

1996c) in providing for fish habitat.

• Role of monitoring in meeting management objectives for fish and riparian habitats.

• Role of headwater (non-fish-bearing) streams in providing for downstream fish

habitat, including the role of unharvested buffers and potential risks associated with

the blowdown of trees in buffers.

• Potential cumulative effects within watersheds from treatment of watersheds with

different management prescriptions.

• Potential changes in streamflow over the short and long terms.

• Likelihood of meeting water quality standards.

• Assessment of anticipated effects of Forest plan alternatives on fish and riparian

habitats over the next 10 years (as it might differ from the 100-year evaluation).

• Viability of fish stocks and species and the likelihood of their being well distributed

across the Tongass NF.

• Listing in order of their importance the major factors associated with management

actions that affect fish habitat (these might include role of source areas for large

woody debris adjacent to fish habitat, role of source areas for large woody debris

upstream of fish habitat, role of cumulative activities in a watershed that may affect

stream timing or flow, role of source areas for stream sediment, role of specific

roads, temperature, dissolved oxygen levels, etc.).

• Consideration of how the “risk” of unplanned actions (actions that should not have

occurred; e.g., not providing fish passage in an anadromous fish stream) may

affect fish populations.

• Role of past actions in rating future effects.

Wildlife panels—In addition to the presentations made by each local resource specialist and the general information provided, the following additional visual or written

information was provided (unless noted otherwise, these items are available from the

planning record, USDA Forest Service 1997d): 

• Marbled murrelet 

✦ Distribution map of murrelets in southeast Alaska during the 1970s and 1980s.

✦ Boundary map of the 21 biogeographical provinces in southeast Alaska.

✦ Map of the habitat conservation strategy by Forest plan alternative. 

✦ Existing proportion of young growth in the Forest by VCU (see text footnote 7).

✦ Projected proportion of second growth in the Forest by VCU (see text footnote 

7) in 2095 for each alternative.

✦ Current distribution across the Forest of timber volumes by low, medium, and 

high volume classes (Julin and Caouette 1997). 
• Queen Charlotte goshawk

✦ Draft of the northern goshawk conservation assessment (Iverson and others

1997).

✦ Acres of productive old-growth forest below 500 feet, below 1,200 feet, and on

slopes of less than 35 percent that already have been harvested for timber.

✦ Acres of productive old-growth forest below 500 feet, below 1,200 feet, and on

slopes of less than 35 percent that are projected to be harvested for timber by

2095 for each Forest plan alternative.

✦ Percentages of productive old-growth forest that were harvested between 1954

and 1994, by VCU (see text footnote 7).

• Alexander Archipelago wolf

✦ Draft of the Alexander Archipelago wolf conservation assessment (Person and 

others 1997).

✦ Wolf hunter and trapper harvest statistics (1975-95).

✦ Existing road density by wildlife analysis area (see text footnote 6).

✦ Estimates of Sitka black-tailed deer habitat capability by wildlife analysis area

(see text footnote 6) and by biogeographical province.

✦ Map of the planned road system for Prince of Wales Island following full implementation of Forest plan alternative 9.

• Brown bear 

✦ Riparian buffer widths for each of the three riparian management options (USDA

Forest Service 1996b) and for the moderate gradient mixed control and floodplain process groups (USDA Forest Service 1992).

✦ Map of the planned road system for Prince of Wales Island following full implementation of Forest plan alternative 9.

• American marten 

✦ Map of the planned road system for Prince of Wales Island following full implementation of Forest plan alternative 9.

✦ Percentages of productive old-growth forest that were harvested between 1954

and 1994 by VCU (see text footnote 7).

• Other terrestrial mammals

✦ Publications about the mammals of southeast Alaska (MacDonald and Cook

1996a, 1996b).

✦ Map of the roads planned in the Ketchikan Area following full implementation of

Forest plan alternative 9.

✦ Article by Burkey (1995).

In addition, an assessment of the projected additional miles of roads per VCU (see

text footnote 7) through 2095 for each Forest plan alternative was provided to evaluators on all wildlife panels.

42
Appendix V 

Alternative Appraisal 

Form 

This form was used in the second set of risk assessment panels. The intent of the

form was to gain insight into how panel evaluators considered that the various elements of draft alternatives contributed to maintaining wildlife viability or fish sustainability. The “+/-” structure of the form caused some difficulty for evaluators in that

some ended up with the opposite interpretation of a positive or negative feature than

was intended. An arrow system similar to that used in the socioeconomic panel (see

fig. 2) is suggested for future use. 

43
This page has been left blank intentionally.

Document continues on next page.
This page has been left blank intentionally.

Document continues on next page.
This page has been left blank intentionally.

Document continues on next page.
Shaw, Charles G., III. 1999. Use of risk assessment panels during revision of the Tongass 

Land and Resource Management Plan. PNW-GTR-460. Portland, OR: U.S. Department of 

Agriculture, Forest Service, Pacific Northwest Research Station. 43 p. (Shaw, Charles, G., 

III, tech. coord.; Conservation and resource assessments for the Tongass land management plan revision).

This paper describes the process used to conduct the 16 risk assessment panels and a 

subsistence working group held during revision of the Tongass land management plan. It 

provides an overview of how results from the panels were used by forest managers in planrelated decisionmaking, discusses some reactions to the effort, and identifies some opportunities to improve the process. The panel results provided estimates of the relative risk that implementation of a range of alternative approaches to management of the Tongass National Forest

would pose to the continued persistence across the landscape of an array of species or

resources and estimates of potential socioeconomic effects on communities. As anticipated,

results from these risk assessment panels became an integral component of the effects analysis section of the environmental impact statement that provided the context for successfully

completing the Tongass Land and Resource Management Plan.

Keywords: Forest planning, effects analysis, National Forest Management Act, old-growth 

forest, forest policy, Tongass National Forest, socioeconomic issues, southeast Alaska, 

subsistence, wildlife population viability, fisheries sustainability, well-distributed populations. 

The Forest Service of the U.S. Department of Agriculture is

dedicated to the principle of multiple use management of the

Nation’s forest resources for sustained yields of wood, water,

forage, wildlife, and recreation. Through forestry research,

cooperation with the States and private forest owners, and

management of the National Forests and National Grasslands,

it strives—as directed by Congress—to provide increasingly

greater service to a growing Nation.

The U.S. Department of Agriculture (USDA) prohibits discrimination in all its programs and activities on the basis of race,

color, national origin, gender, religion, age, disability, political

beliefs, sexual orientation, or marital or family status. (Not all

prohibited bases apply to all programs.) Persons with disabilities who require alternative means for communication of program information (Braille, large print, audiotape, etc.) should

contact USDA's TARGET Center at (202) 720-2600 (voice

and TDD).

To file a complaint of discrimination, write USDA, Director,

Office of Civil Rights, Room 326-W, Whitten Building, 14th 

and Independence Avenue, SW, Washington, DC 20250-9410

or call (202) 720-5964 (voice and TDD). USDA is an equal

opportunity provider and employer.

Pacific Northwest Research Station

333 S.W. First Avenue

P.O. Box 3890

Portland, Oregon 97208-3890
U.S. Department of Agriculture

Pacific Northwest Research Station

333 S.W. First Avenue

P.O. Box 3890

Portland, Oregon 97208-3890

Official Business

Penalty for Private Use, $300

do NOT detach label