#### ECOLOGY CENTER, INC., Plaintiff-Appellant, v. Deborah AUSTIN, in her official capacity as Forest Supervisor for the Lolo National Forest; Bradley Powell, Regional Forester of Region One of the U.S. Forest service; united states forest Service, an agency of the U.S. Department of Agriculture, Defendants-Appellees, Mineral County; Town of Superior; St. Regis School District No. 6; Superior School District No. 3; Montana Coalition Of Forest Counties; Tricon Timber, Defendant-Intervenors-Appellees.

No. 03-35995.

United States Court of Appeals, Ninth Circuit.

Argued and Submitted Feb. 14, 2005.

Filed Dec. 8, 2005.

*1058See also 82 FedAppx. 570.

*1060Thomas J. Woodbury, Forest Defense, P.C., Missoula, MT, for the plaintiff-appellant.

Aaron P. Avila, Ronald M. Spritzer, Department of Justice, Washington, D.C., for the defendants-appellees.

*1061Before: B. FLETCHER, McKEOWN, and GOULD, Circuit Judges.

BETTY B. FLETCHER, Circuit Judge.

Ecology Center, Inc. (“Ecology Center”) challenges the United States Forest Service’s (“Forest Service” or “Service”) Lolo National Forest Post Burn Project (“Project”), which was designed in the aftermath of the 2000 wildfires on the Lolo National Forest (“LNF”). Ecology Center raises a number of procedural and substantive claims under the National Environmental Policy Act (“NEPA”) and the National Forest Management Act (“NFMA”). We have jurisdiction pursuant to 28 U.S.C. § 1291. Because we find that the Forest Service’s decision to permit logging in critical old-growth forest and post-fire habitats was arbitrary and capricious, we reverse the grant of summary judgment to the Service and remand.

I

In 2000, wildfires burned approximately 74,000 acres on the Lolo National Forest. While the fires caused considerable damage to the forest, they also created habitat for species that are dependent upon'post-fire habitats, such as the black-backed woodpecker.

In response to the 2000 fires, the Forest Service began developing the Lolo National Forest Post Burn Project and preparing the requisite Environmental Impact Statement (“EIS”). The Forest Service considered four alternatives .in detail, including a “no action alternative.” In July 2002, the Forest Service selected a slightly modified version of “Alternative Number Five” for the Project. This alternative involves, *inter alia,*commercial thinning of small diameter timber and prescribed burning in old-growth forest stands, as well as salvage logging of burned and insect killed timber in various areas of the forest.

On February 7, 2003, Ecology Center filed its complaint, raising several claims under NEPA and NFMA. Ecology Center objects to the Forest Service’s decision to permit commercial logging in old-growth forest stands, raising concerns about the impact of such logging on the viability of species that are dependent upon old-growth habitat, such as the pileated woodpecker and the northern goshawk. Similarly, Ecology Center questions the Service’s impact analysis of salvage logging in post-fire habitat, particularly with respect to the black-backed woodpecker, a sensitive species. Ecology Center also raises concerns-regarding the impact of the Project on soil conditions and questions the reliability , of the Service’s soil quality analysis.

Ecology Center moved for a temporary restraining order and preliminary injunction, both of which the district court denied. The parties then filed cross-motions for summary judgment. On November 4, 2003, the district court struck extra-record declarations that Ecology Center had included with its motion and granted summary judgment in favor of the Forest Service.1

*1062II

“Because this is a record review case, we may direct that summary judgment be granted to either party based upon our de novo review of the administrative record.” *Lands Council v. Powell,*379 F.3d 738, 743 (9th Cir.2004), *amended by*395 F.3d 1019 (9th Cir.2005). Our review of agency actions challenged under NFMA and NEPA is governed by the judicial review provisions of the Administrative Procedure Act, - 5 U.S.C. § 706(2)(A). *Neighbors of Cuddy Mountain v. U.S. Forest Serv.,*137 F.3d 1372, 1376 (9th Cir.1998); *Ctr. for Biological Diversity v. U.S. Forest Serv.,*349 F.3d 1157, 1165 (9th Cir.2003). Under the APA, we review to determine if the agency’s actions were “arbitrary, capricious, an abuse of discretion, or otherwise contrary to law.” *Lands Council,*379 F.3d at 743.

NFMA imposes both substantive and procedural requirements on the Forest Service. 16 U.S.C. §§ 1600-1687. Procedurally, it requires the Forest Service to develop a land and resource management plan (“forest plan”) for each forest that it manages. 16 U.S.C. § 1604(a). Subsequent agency actions must not only comply with NFMA but also be consistent with the governing forest plan. 16 U.S.C. § 1604(i); *Idaho Sporting Cong., Inc. v. Rittenhouse,*305 F.3d 957, 961-62 (9th Cir. 2002) (“[A]ll management activities undertaken by the Forest Service must comply with the forest plan, which in turn must comply _with the Forest Act....”). NFMA also requires that the Forest Ser-viee adopt regulations specifying guidelines for the achievement of NFMA’s substantive mandates. 16 U.S.C. § 1604(g)(3).

“In providing for multiple uses, the forest plan must comply with substantive requirements of the Forest Act designed to ensure continued diversity of plant and animal communities and the continued viability of wildlife in the forest....” *Idaho Sporting Congress,*305 F.3d at 961; 16 U.S.C. § 1604(g)(3)(B). In addition to the mandate to maintain wild-life viability, the Forest Service must maintain soil productivity. 16 U.S.C. § 1604(g)(3)(C).

Unlike NFMA, NEPA imposes no substantive requirements. Instead, it is designed “to force agencies to publicly consider the environmental impacts of their actions before going forward.” *Idaho Sporting Congress,*305 F.3d at 963. “Agencies must adequately consider the project’s potential impacts and the consideration given must amount to a ‘hard look’ at the environmental effects.” *Id.*Specifically, NEPA requires the preparation of a detailed Environmental Impact Statement. 42 U.S.C. § 4332(2)(C). An EIS must “provide full and fair discussion of significant environmental impacts and shall inform decisionmakers and the public of the reasonable alternatives which would avoid or minimize adverse impacts or enhance the quality of the human environment.” 40 C.F.R. § 1502.1.

*1063*A. “Treatment” of Old-Growth Habitat*

The Project involves what the Forest Service characterizes . as rehabilitative “treatment” of old-growth (and potential old-growth) forest stands; this treatment entails the thinning of old-growth stands via commercial logging and prescribed burning. The Forest Service cites a number of studies that indicate such treatment is necessary to correct uncharacteristic forest development resulting from years of fire suppression. The Service also points out that the treatment is designed to leave most of the desirable old-growth trees in place and to improve their health.

Ecology Center highlights the scientific uncertainty and debate regarding the necessity, design, and long-term effects of such old-growth treatment. In particular, Ecology Center alleges that the treatment of old-growth forest harms species that are dependent upon old-growth habitat.. For example, Ecology Center claims that, even if treatment leaves most old-growth trees in place, it removes or alters other essential elements within old-growth habitat and disturbs bird species currently nesting or foraging within targeted stands.2 Although treatment may be designed to restore old-growth to “historic conditions,” Ecology Center points out this can be a misleading concept: for example, information regarding historic conditions is incomplete; altering particular sections of forest in order to achieve “historic” conditions may not make sense when the forest as a whole has already been fundamentally changed; many variables can affect treatment outcomes; and the treatment process is qualitatively different from the “natural” or “historic” processes it is intended to mimic.

*1. NFMA*

While Ecology Center does not offer proof that the proposed treatment causes the harms it fears, the Service does not offer proof that the proposed treatment benefits — or at least does not harm — old-growth dependent species. Ecology Center argues that because the Forest Service has not assessed the effects of old-growth treatment on dependent species, the Service cannot be reasonably certain that treating old-growth is consistent with NFMA’s -substantive mandate to ensure species diversity and viability. As a result, especially given the scientific uncertainty surrounding the treatment of old-growth stands, the Forest Service’s decision to treat additional old-growth stands was arbitrary and capricious.

Although the Forest Service points to a report which notes that two species of woodpecker were observed foraging in treated old-growth forest, it does not otherwise dispute the charge that it has not directly monitored the impact of treating old-growth on dependent species. Instead, the Service maintains-that it need not do so because (1) it has observed the short-term effects of thinning old-growth stands via commercial logging and prescribed burning on forest composition, (2) it has reason to believe that certain old-growth dependent species would prefer the post-treatment composition of old-growth forest stands, and-(3) its assumption that treatment does not harm old-growth dependent species is therefore reasonable. The Service further argues that we must defer to *1064its methodological choices regarding what to monitor and how to assess the impact of old-growth treatment.

An agency’s choice of methodology is entitled to deference. *See, e.g., Salmon River Concerned Citizens v. Robertson,*32 F.3d 1346, 1359 (9th Cir.1994). However, there are circumstances under which an agency’s choice of methodology, and any decision predicated on that methodology, are arbitrary and capricious. For example, we have held that in order to comply with NFMA, the Forest Service must demonstrate the reliability of its scientific methodology. *Lands Council,*379 F.3d at 752 (holding NFMA “require[s] that the hypothesis and prediction of the model be verified with observation”). Here, as in *Lands Council,*the Forest Service’s conclusion that treating old-growth forest is beneficial to dependent species is predicated on an unverified hypothesis. While the Service’s predictions may be correct, the Service has not yet taken the time to test its theory with any “on the ground analysis,” *id.,*despite the fact that it has already treated old-growth forest elsewhere and therefore has had the opportunity to do so. Just as it would be arbitrary and capricious for a pharmaceutical company to market a drug to the general population without first conducting a clinical trial to verify that the drug is safe and effective, it is arbitrary and capricious for the Forest Service to irreversibly “treat” more and more old-growth forest without first determining that such treatment is safe and effective for dependent species. This is not a case in which the Forest Service is asking for the opportunity to verify its theory of the benefits of old-growth treatment. Rather, the Service is asking us to grant it the license to continue treating old-growth forests while excusing it from ever having to verify that such treatment is not harmful.

The Service argues that under *Inland Empire Public Lands Council v. U.S. Forest Serv.,*88 F.3d 754 (9th Cir.1996), we must defer to its decision to monitor only the effect of treatment on forest composition, instead of also monitoring the effect of treatment on dependent species. However, *Inland Empire*is inapposite here. In that case, we permitted the Service to determine that it was complying with its duty to maintain species viability by using a “proxy-on-proxy” method for monitoring species population. *Id.*at 761. *But see Idaho Sporting Congress,*305 F.3d at 972-73(finding use of proxy-on-proxy approach arbitrary and capricious where there is evidence that the “methodology does not reasonably ensure viable populations of the species at issue”). Under the proxy-on-proxy approach, the Service does not ensure that it is maintaining species viability by directly monitoring species populations. Instead, the Service designates certain “management indicator species” as proxies for other species with comparable habitat needs. It then designates certain kinds of habitat as proxies for the management indicator spécies. Finally, it assumes that a species’ viability is maintained so long as the requisite amount of the species’ habitat is maintained. Here, the Service is not simply maintaining the amount of old-growth habitat necessary to support old-growth dependent species — it is altering the composition of old-growth habitat through an invasive process.

Although the Service concedes that the opinions of well-qualified experts vary with respect to the appropriateness of management' activities in old-growth areas, it also argues that it must have the “discretion to rely on the reasonable opinions of its own qualified experts even if, as an original matter, a court might find contrary views more persuasive.” *Marsh v. Oregon Natural Resources Council,*490 U.S. 360, 378, 109 S.Ct. 1851, 104 L.Ed.2d 377 (1989). *1065However, this is not a case in which different experts have studied the effects of commercial thinning and prescribed burning in old-growth forests and reached different conclusions. Here, experts have differing hypotheses regarding the effects that treating old-growth has on dependent species, yet the Forest Service proposes to continue treating old-growth without first taking the time to observe what those effects actually are. In light of its responsibilities under NFMA, this is arbitrary and capricious.

2. *NEPA*

“The EIS did not address in any meaningful way the various uncertainties surrounding the scientific evidence” upon which the decision to treat the Lolo National Forest old-growth rests. *Seattle Audubon Soc’y v. Espy,*998 F.2d 699, 704 (9th Cir.1993). Although the EIS identifies the public’s concerns regarding the impact of treatment on dependent species as “key” or “driving” issues,3 the EIS does not actually explain in any detail the bases of those concerns, much less address them. *Cf. id.*(finding NEPA violation where Forest Service failed “to include a full discussion of the scientific uncertainty” surrounding its strategy for maintaining spotted owl viability). The EIS discusses in detail only the Service’s own reasons for proposing treatment, and it treats the prediction that treatment will benefit old-growth dependent species as a fact instead of an untested and debated hypothesis. Even if -the Service considered these issues but concluded that it need not or could not “undertake further scientific study” regarding the impact of treatment on dependent species, it should have “explain[ed] in the EIS why such an undertaking [wa]s not necessary or feasible.” *Id.*For these reasons, we also find that the Service’s analysis of the impact of treating old-growth to be inadequate under NEPA.4

*B. Salvage Harvesting of Blaclc-Backed Woodpecker Habitat*

The Forest Service has designated the black-backed woodpecker as a “sensitive species,” a species for which viability is a concern.' The black-backed woodpecker’s viability is sensitive because it is particularly dependent upon post-fire landscapes, and the combination of fire-suppression efforts and past salvage logging has drastically reduced the amount of post-fire habitat. According to the EIS, “[d]ue to this reduction in habitat, black-backed woodpeckers went from being relatively abundant to relatively rare.” To make matters *1066worse, not all fires create black-backed woodpecker habitat, and the burned stands that do qualify only serve as black-backed woodpecker habitat for a limited number of years. As a result, new post-fire habitat must be continuously generated.

Ecology Center and the Forest Service agree that prior to the 2000 fires, there was a critical shortage of black-backed woodpecker habitat. Specifically, the Service estimated that from 1993 to 1998, fires created a total of 11,045 acres of post-fire habitat (9160 of those acres were burned in 1998 alone). This total was estimated to be 6% of what would have been created historically over a comparable six-year period. At that level, the Service considered the black-backed woodpecker and other post-fire habitat dependent species to be “at extreme risk.”

Writing in November 2000 about the “effect [on] black-backed woodpeckers,” Forest Service scientists explained: “We concluded in 1998 that salvage of any post-fire habitat on the Lolo [National Forest] ... ‘will impact individuals or habitat and with a consequence that the action may contribute to a trend towards federal listing or cause a loss of viability to the population or species.’ ” They explained that a small amount (less than 400 acres) of salvage could be permitted, but only *if*there was “a commitment to creation of post-fire habitat of an equal amount” through prescribed fires. Under those two conditions, they believed that salvaging “would not likely ... cause a loss of viability to the population,” even though it “may impact individuals or habitat.”

From the EIS, it appears that at the time the Project was developed, there was a total of 19,219 acres of black-backed woodpecker habitat in the Lolo National Forest: there were 9349 acres that were created by pre-2000 fires and were less than five years old, plus 9870 acres of habitat that were created by the 2000 fries. Of those acres, 9100 are located in the Project area. The selected alternative originally proposed salvage harvesting 1020 of those acres. However, the Record of Decision reduced that amount to 815 acres.5

The Forest Service maintains that the Project complies with NFMA because the 2000 fries created a large amount of potential habitat, and only a small portion will be salvaged. The Service further points out that the Project includes mitigation measures designed to minimize the adverse impact of salvaging.6 However, “[Research has also indicated that salvage logging, even when large numbers of snags are left, is detrimental to this species.” For example, one study “found that black-backed woodpeckers did not nest in areas that were considered lightly salvaged.” In addition, the Project’s mitigation measures do not provide for the creation of an equal amount of post-fire habitat elsewhere.7

*1067*1. NEPA*

The EIS fails to adequately explain the basis for the Forest Service’s conclusion that eliminating a portion of the newly-created habitat will not adversely affect the black-backed woodpeckers’ viability. Prior to the 2000 fires, the amount of post-fire habitat was so low that the black-backed woodpecker and other post-fire habitat dependent species were deemed at “extreme risk,” and Forest Service experts concluded that “salvage of any post-fire habitat ... would ... impact individuals or habitat ... with a consequence that the action may contribute to a trend towards federal listing or cause a loss of viability to the population or species.”8 Yet, the Project EIS states — without meaningful explanation — that even though salvaging post-fire habitat may negatively impact individual black-backed woodpeckers, it will “not likely result in a trend towards federal listing.” Without more, this, general statement regarding the possible impact and risk involved “do[es] not constitute a ‘hard look’ absent a justification regarding why more definitive information could not be provided.” *Neighbors of Cuddy Mountain,*137 F.3d at 1380; 40 C.F.R. § 1502.1(providing that the EIS “shall be supported by evidence that the agency has made the necessary environmental analy-ses”).

We can discern from the EIS only one possible basis for the Service’s changed assessment of the impact of salvaging: the fact that the 2000 fires created additional post-fire habitat, which raised the total amount of post-fire habitat. However, we cannot conclude from this fact alone that the habitat level is no longer critically low and that the previously prescribed restrictions on salvaging should no longer be enforced.9 To be reasonably certain that the post-Project habitat levels would be sufficient to ensure species viability, one must know where the threshold between “critical” and “sufficient” levels of burned habitat lies. Because the EIS does not disclose what this threshold is, much less explain how the threshold was determined, we cannot evaluate the Service’s decision. Indeed, we cannot even be certain that the Service determined and considered this factor when making its assessment.

Similarly, the EIS does not indicate how much further habitat levels have to drop before the species is downgraded from sensitive to threatened or how the Service plans to generate additional black-backed woodpecker habitat to counter-balance losses from salvaging, the passage of time, and ongoing fire-suppression efforts. Where, as here, “the information in the ... EIS was so incomplete or misleading that the decisionmaker and the public could not make an informed comparison of the alternatives., revision of an EIS may be necessary to provide ‘a reasonable, good faith, and objective presentation of the subjects required by NEPA.’ ” *Animal Def. Council v. Hodel,*840 F.2d 1432, 1439 (9th Cir.1988) (quoting *Johnston v. Davis,*698 F.2d 1088, 1095 (10th Cir.1983)), *amended by*867 F.2d 1244 (9th Cir.1989); *see also Idaho Sporting Cong. v. Thomas,*137 F.3d 1146, 1150 (9th Cir.1998) (“[Allowing the Forest Service to rely on expert opinion without hard data either vitiates a plaintiffs ability to challenge an agency *1068action or results in the courts second guessing an agency’s scientific conclusions. As both of these results are unacceptable, we conclude that NEPA requires that the public receive the underlying environmental data from which a Forest Service expert derived her opinion.”).

Thus, we hold that the Forest Service failed to either adequately explain its impact assessment or provide the information that is necessary to understand and evaluate the Forest Service’s decision to permit salvaging of the black-backed woodpeckers’ rare habitat, in violation of NEPA.

*2. NFMA*

The designation of a species as sensitive arises from the Forest Service’s obligations under NFMA. Pursuant to 16 U.S.C. § 1604(g)(3)(B), the Forest Service is required to “provide for diversity of plant and animal communities.” The Forest Service’s duty to maintain viable populations “applies with special force to ‘sensitive’ species.” *Friends of the Clearwater v. Dombeck,*222 F.3d 552, 556 n. 2 (9th Cir.2000) (citation and quotation omitted). The Lolo National Forest Plan requires, more specifically, that the Forest Service “manage to maintain population viability” for plant and animal species “that are not threatened or endangered, but where viability is a concern (ie., sensitive species).”

Because the Forest Service failed to provide the factual basis for its analysis and failed to adequately explain its decision, we cannot be reasonably certain that the salvaging — which the Service- concedes may harm individual black-backed woodpeckers — will not jeopardize the black-backed woodpeckers’ viability. *Cf. Native Ecosystems Council v. U.S. Forest Serv.,*418 F.3d 953, 964 (9th Cir.2005) (finding NFMA violation where “we cannot reasonably determine that the Forest Service has complied with the [Forest] Plan”). We therefore hold that the decision to salvage harvest black-backed woodpecker habitat is also arbitrary and capricious under NFMA.

*C. Soil-Quality Analysis*

Ecology Center next challenges the Forest Service’s soil quality analysis. Under the applicable Regional Soil Quality Standard, the Forest Service cannot allow an activity that would create detrimental soil conditions in fifteen percent of the activity area.10 Ecology Center claims that the methodology the Forest Service used to determine what percentage of soil is in a detrimental state in each activity area was insufficiently reliable because the Service estimated soil conditions on the basis of maps, samples from throughout the Forest, aerial reconnaissance, and computer modeling, but did not verify those estimates by directly observing soil conditions in the activity areas.11

*1069We addressed a nearly identical claim, involving the same Regional Soil Quality Standard, in *Lands Council.**12**See Lands Council,*379 F.3d at 752-53. In that case, “based on samples from throughout the Forest, and aerial photographs, the Forest Service estimated the quality of the soils in the Project area using a spreadsheet model.” *Id.*at 752. We held that the “Forest Service’s reliance on the spreadsheet models, unaccompanied by on-site spot verification of the model’s predictions, violated NFMA.” *Id.*at 752-53. In so holding, we relied on the analysis of an earlier district court case, *Kettle Range Conservation Group v. U.S. Forest Serv.,*148 F.Supp.2d 1107 (E.D.Wash.2001), where the Forest Service, “based on assumptions [from general data from the Forest soils], geological maps, and aerial photographs, ... *estimated*the condition of each unit, tried to determine which units *might*exceed established standards, and *projected*potassium levels.” *Id.*at 1127 (emphasis in original).

Although the methodologies in this case, *Lands Council,*and *Kettle Range*all appear to suffer from the same critical defect, the Forest Service argues that *Lands Council*is not controlling here. Specifically, the Service argues that the Regional Soil Quality Standard is not binding with respect to projects in the Lolo National Forest. In addition, the Forest Service argues that this case is factually distinguishable from *Lands Council*because, notwithstanding the non-binding nature of the Standard, the Forest Service completed sufficient on-the-ground verification of its soil quality estimates. We address each argument in turn.13

*1. The Regional Soil Quality Standard*

The Forest Service argues that the Standard is not binding because it was never incorporated into the Lolo National Forest Plan, and although it was incorporated into the Forest Service Manual, this court has held that the Forest Service Manual does not have the independent force and effect of law. *Western Radio Servs. Co. v. Espy,*79 F.3d 896, 901 (9th Cir.1996).

Even assuming, *arguendo,*that the Standard does not have the independent force and effect of law with respect to the Lolo National Forest, it would nonetheless be arbitrary and capricious for the Forest Service to ignore it because both the draft EIS and final EIS discuss the Standard as if it is binding and claim that the Service developed the Project in compliance with its provisions. Thus, even if we were to agree that the Standard is merely advisory, we would then be compelled to find that the draft EIS and final EIS are misleading in violation of NEPA. *See Resources Ltd. v. Robertson,*35 F.3d 1300, *10701304 n. 3 (9th Cir.1994) (rejecting the argument that the Forest Service could treat the Interagency Grizzly Bear Guidelines as optional where the Fish and Wildlife Service made its “no jeopardy” conclusion contingent on adherence to the Guidelines); *Animal Def. Council,*840 F.2d at 1439 (noting that a misleading EIS violates NEPA). Moreover, the purpose of the Standard is to ensure compliance with the substantive mandates of NFMA. The Forest Service does not explain how it can be certain that the Project complies with NFMA if the Project was not developed in accordance with the Standard.

*2. On-Site Verification of Soil Conditions*

First, the Service points to the transects it conducted during the BAER assessment of the 2000 fires’ aftermath. However, during the Project development, one of the Service’s own experts pointed out that the transects targeted burned areas, not proposed harvest units; as a result, most of the activity areas were *not*transected.14 Thus, the BAER transects do not distinguish this case from *Lands Council,*where the Service similarly “did not test much of the activity area.” *Lands Council,*379 F.3d at 752, *as amended by*395 F.3d at 1034.

Second, the Forest Service argues that its soil analysis is nonetheless sufficiently reliable because it utilized data from areas with ecological characteristics similar to the proposed harvest units. *Lands Council*expressly rejected this same argument. *Id.*(“The Forest Service concedes that it did not test much of the activity area, but argues that because it tested similar soils within the Forest, and similar soils act the same way, then the methodology is sound.”). We note that comments on the Project by the Service’s own expert support this holding: Region Scientist Nesser argued that the Service’s conclusions regarding soil conditions in the activity areas were “not credible” because the Service did not test the activity areas themselves, as the Standard requires.15

Third, the Service points to some informal field reports contained in the record, some of which indicate that a small percentage of the activity areas were observed directly.16 The record provides little information that enables us to assess the reliability or significance of these reports; for example, we do not know the qualifications of the persons conducting the field review, the methodology utilized, or whether the field observations confirmed or contradicted the Service's estimates. Moreover, the final EIS repeatedly explains that the Service will conduct *1071field testing to verify its estimates prior to any ground-breaking activities,17 and there is no indication in the draft EIS or final EIS that the Forest Service actually consulted and relied upon any of the field reports it now cites when it was making its selection amongst the Project alternatives. Thus, we cannot conclude that these reports distinguish this case from *Lands Council.*

Finally, we consider whether the Service’s plan to verify soil conditions in the activity areas after authorizing the Project, but before actually commencing harvesting activities, satisfies NFMA and NEPA. The fact that the Service plans to conduct on-site verification prior to any harvesting implies that even the Service recognizes that its soil-quality estimates need to be verified. Thus, we conclude that the Forest Service’s decision to authorize the Project first and verify later does not distinguish this case from *Lands Council,*and that the Project violates NFMA. We also conclude that the Service’s plan to verify its estimates post-decision fails to satisfy NEPA, because “NEPA requires consideration of the potential impact of an action *before*the action takes place.” *Neighbors of Cuddy Mountain,*137 F.3d at 1380(quoting *City of Tenakee Springs v. Clough,*915 F.2d 1308, 1313(9th Cir.1990) (emphasis in original)); *Kettle Range,*148 F.Supp.2d at 1127(“They [the Forest Service] apparently planned to [field test] sometime in the future. They should have done it before they completed the FEIS. The failure to do so is a violation of NEPA because it shows that USFS did not give a ‘hard look’ at the effects of the Project on the soils in the analysis area.”).

Ill

The Forest Service’s selection of Modified Alternative Number 5 for the Lolo National Forest Post-Burn Project violates both NEPA and NFMA. We reverse the district court’s summary judgment in favor of the Forest Service, and direct the district court to enter summary judgment on behalf of Ecology Center. We remand this case to the Forest Service for further proceedings consistent, with this opinion.18

REVERSED and REMANDED.

1

. In separate proceedings, the Sierra Club challenged the same project at issue here. The district court ruled in favor of the Sierra Club’s claim that the Forest Service was required to determine how much sediment affected streams could assimilate before implementing the Project, but ruled in favor of the Forest Service with respect to its analysis of the Project’s impact on unroaded areas. In a memorandum disposition decided December 2, 2003, we reversed the district court on both claims: we found that the Forest Service’s assessment of the Project’s effect on water quality satisfied NEPA, but that its analysis of *1062the Project’s impact on unroaded areas was inadequate. *Sierra Club, Inc. v. Austin,*82 Fed.Appx. 570 (9th Cir.2003). The case was remanded to the district court, and the parties then settled.

The Forest Service moved for this court to take judicial notice of the *Sieira Club*settlement agreement, and for leave to file supplemental information and declarations explain-

ing the terms of the settlement; we granted that motion. After taking judicial notice of the settlement, we invited the parties to advise us as to whether the settlement moots any portion of this appeal or affects the relief, if any, that the district court should grant. Although the settlement significantly reduces the scope of the Project, neither party argues that Ecology Center's claims are moot.

2

. For example, the pileated woodpecker is dependent upon old-growth snags (standing dead trees), but treatment involves timber harvesting, which "creates the potential for snag loss.” Although the Service acknowledges this danger, it believes that proposed "snag mitigation measures,” combined with the potential for snag-creation during the prescribed burning process, will offset most of the snag loss caused by logging.

3

. In a section of the EIS entitled "Issue Development,” the Service describes ”[p]otential issues,” which are "defined as actual and perceived effects, risks, and hazards of the Proposed Action.” Issues were identified both internally and through “public involvement.” In relevant part, the EIS states:

> There is a concern that the reduction of canopy closure and stand density caused by fuels reduction activities and commercial thinning may adversely affect habitat for several bird species that are dependent upon late-successional dry forests with closed canopies and high basal area.

> There is a concern that the ... commercial thinning of unburned trees, and the removal of understory fuels, could adversely affect Proposed, Threatened, or Endangered Species and listed Sensitive, Threatened, or Endangered Species ... known to be present within or to be potential to the analysis area.

4

. Ecology Center also raises claims challenging the Service’s analysis of the Project’s impact on two specific old-growth dependent species, the pileated woodpecker and the northern goshawk. Because we find that the Service’s decision to treat old-growth violates both NFMA and NEPA, we do not reach these more narrow claims.

5

. The *Sierra Club*settlement further reduced that amount to 155 acres.

6

. For example, the Project provides that no harvesting will occur wherever black-backed woodpecker nests are located; each spring, before harvesting actually begins, targeted stands will be re-surveyed for black-baclced woodpecker nests, and loggers will be instructed to preserve any identified nest trees as well as reserve patches surrounding the nesting trees.

7

.Although other aspects of the Project involve prescribed burning, "the Lolo prescribed fire program has not demonstrated an ability to recruit large amounts of post-fire habitat, although there is an improving trend in 1997-98.” Most kinds of prescribed burning do not create the necessary post-fire habitat; only higher-intensity fires create post-fire habitat. There is no indication that the kind of prescribed burning that the Project entails is designed to create black-backed woodpecker habitat.

8

. As noted above, these experts concluded that some salvaging could be tolerated, but only if limited to a small amount and mitigated by the creation of an equal amount of post-fire habitat elsewhere.

9

. We note that the Service considered the creation of 11,045 acres of post-fire habitat over a six-year period to be critically low (6% of the historic rate); thus, the creation of 19,219 acres over a comparable period does not necessarily represent a sufficient increase.

10

. An "activity area” is a unit of forest in which an activity, in this case salvage harvesting, is to take place; this Project proposes harvesting in approximately 128 different activity areas or "cutting units.” The Regional Soil Quality Standard further provides that if fifteen percent or more of the activity area already has detrimental soil conditions, then the Project will not be permitted to make it worse. Rather, the Project should then aim to improve the soil conditions.

11

. Specifically, the Service looked at maps to determine where damaging past activities (e.gtimber harvesting), jammer roads, fire-lines, and skid trails, as well as areas of high soil burn severity overlap with proposed activity areas. In addition, the Service input data from the Land Systems Inventory ("LSI”) and the Burned Area Emergency Rehabilitation Report ("BAER Report”) into models to generate estimates of the project's possible effects. The BAER Report was based on field reviews and helicopter flyovers, as well as "transect surveys” (an approved method of field observations). The LSI and BAER Report data was not collected for the purpose *1069of determining compliance with the Regional Soil Quality Standards. Because the Project was developed after the BAER transect surveys were conducted, the transects did not cover the vast majority of the activity areas; only a few were crossed by coincidence.

12

. Because *Lands Council*was decided after briefing, we ordered supplemental briefing.

13

. The Forest Service also reiterates its argument that this court must defer to its choice of methodology and reliance on its own experts. This argument fails here for the same reason it did in *Lands Council:*although this court is required to defer to agency expertise, the agency is not permitted to adopt and rely upon a methodology without reasonably verifying its reliability. *Lands Council,*379 F.3d at 752("The Forest Service, granted appropriate deference, still does not demonstrate the required reliability of the spread sheet model.”) Moreover, in this case, one of the Forest Service's own experts, Region Scientist John Nesser, called the Forest Service's soil analysis into question because it failed to assess soil conditions by field testing the actual activity areas — the exact reason that *Lands Council*deemed the Forest Service’s methodology inadequate.

14

. This point is corroborated by the draft EIS, which states that “not all proposed harvest units were visited with line transects” and that “much of the [soil quality] determination was based on information from the transportation and timber shop with respect to past activities and regeneration level of the jamer [sic] roads.”

15

. Specifically, Region Scientist John Nesser commented:

> The statement on page 3-35 [of the draft EIS] *that ‘All proposed activity units are at 3 percent (detrimental disturbance) or less pri- or to any new activities'*is not credible, given fire effects and the effects of past timber harvest and road building activities. This is because transects were done in the various burned areas, rather than in proposed harvest or other activity units. The burned units are not the correct activity unit to use for post fire activities. ... statements about meeting the 15 percent limits for detrimental disturbance are both irrelevant and wrong. Without using the correct activity areas for analysis, the cumulative effects assessment is not correct.

16

.Some of the field reports cited by the Service actually say very little about soil conditions.

17

. The final EIS explains that a ''[w]alk through of the proposed harvest and burn units *will*be completed prior to the .implementation of any ground disturbing activities in the activity units. The walk through *will identify the occurrence and percent of the activity area*with displacement, rutting, compaction, high burn severity, depth of organic matter, pedestaling, and slumping.” (emphasis added). Similarly, throughout its discussion of the Project's impacts, the EIS repeatedly notes that it "will verify” the conclusions it reached regarding soil conditions by having "qualified soil specialists” conduct "field reviews” of the proposed activity units "to evaluate the percent detrimental condition prior to any ground disturbing activities.”

18

. We do not reach the evidentiary issue: whether-the district court abused its discretion.in excluding the extra-record expert declarations. We note that, in the course of the preparation of a new EIS on remand, the parties may submit the declarations and any other relevant information to be included as part of the administrative record.

McKEOWN, Circuit Judge,

dissenting:

The Ninth Circuit, like the other circuits, repeats frequently the legal mantras of administrative review in the context of environmental decisions: “arbitrary and capricious,” “hard look,” and “no second guessing.”1 These standards are easy to *1072articulate, but it is more difficult to know when we have crossed the line from reviewer to decisionmaker. In this case, we have gone too far.

As summarized in *Neighbors of Cuddy Mountain v. United States Forest Service,*137 F.3d 1372, 1376 (9th Cir.1998), we review agency actions “to determine if they were arbitrary or capricious, an abuse of discretion, or not in accordance with law.” The limited nature of this inquiry underscores the latitude in implementation and interpretation that Congress intended for its agents. This latitude does not relieve us of our duty to insure that the agency took a “hard look,” as required under the National Environmental Policy Act (“NEPA”), or' to insure compliance with forest plans under the National Forest Management Act (“NFMA”). Nor does record review mean that'we abdicate our obligation to scrutinize the agency’s underlying data, assumptions, and conclusions. As a reviewing court, we play a critical role in the review process and we have not been, shy in letting the U.S. Forest Service know where it has fallen short as a matter of law and process.2 However, the majority’s extension of *Lands Council v. Powell,*379 F.3d 738 (9th Cir.2004), *amended by*395 F.3d 1019, 1024 (9th Cir. 2005), represents an unprecedented incursion into the administrative process and ratchets up the scrutiny we apply to the scientific and administrative judgments of the Forest Service. Because the majority has, in effect, displaced “arbitrary and capricious” review for a more demanding standard, I respectfully dissent.

The project under review is designed to address the aftermath of the Lolo National Forest wildfires in Montana in 2000 through selected thinning of smaller timber, salvage of timber, prescribed burning and regeneration of thousands of acres. The administrative record in this case is huge — a 1900 + page Final Environmental Impact Statement (FEIS), 150 detailed maps and 20,000 pages of background information. The majority highlights only parts of this record, criticizes the qualifications of the Forest Service’s personnel, and questions various scientific judgments. In faulting the Forest Service’s soil quality analysis and concluding that old-growth forest will not be impaired, the majority changes our posture of review to one where we sit at the table with Forest Service scientists and second-guess the minutiae of the decisionmaking process.

My concern is perhaps best illustrated in the majority’s application of *Lands Council*to the soil quality analysis issue. In *Lands Council,*we addressed the Forest Service’s analysis of disturbed soil conditions in a watershed restoration project in the Idaho Panhandle National Forest. The Forest Service had taken soil samples from throughout the forest but not in the activity area. *Id.*at 1035. Based on these samples and aerial photographs, it calculated the impact of the watershed restoration project on soil quality in the activity area. *Id.*at 1034.

*1073In *Lands Council,*we rejected the Forest Service’s choice of scientific methodology because it did not walk or test soil in the activity area. *Id.*We concluded that the “Forest Service’s reliance on the spreadsheet models, unaccompanied by on-site spot verification of the model’s predictions, violated NFMA.” *Id.*at 1035. Given the specific record and circumstances in *Lands Council,*this resolution may have made perfect sense. But in the abstract, without the ability to compare the record in *Lands Council*with the record here, there is no legal basis to conclude that the NFMA requires an on-site analysis where there is a reasonable scientific basis to uphold the legitimacy of modeling. NFMA does not impose this substantive requirement, and it cannot be derived from the procedural parameters of NEPA.

Application of *Lands Council*makes even less sense in this specific case. Here, the Forest Service conducted exactly the kind of on-site analyses found lacking in *Lands Council*— soil samples in activity areas. The record lists five activity areas surveyed in soil sample transects and details the walkthrough of a number of activity areas. The majority concedes that these on-site analyses exist but criticizes them for being too few and of poor quality.3 Resting on this characterization, the majority- rejects the Forest Service’s soil analysis on the ground that it was *“nearly*identical” to the one we found lacking in *Lands Council.*

*Lands Council*makes compliance with NFMA and NEPA a moving target. If one reads *Lands Council*to require on-site analysis in every case, which I respectfully suggest is an over-reading of the case, the circumstances here comply with that dictate. Yet the majority now says, without support in the record and absent a challenge by Ecology Center at the administrative level, that the on-site analysis was insufficient and that we do not possess enough information to know if the inspectors were qualified. From this judgment, we are left to conclude that not only does the court of appeals set bright-line rules, such as requiring an on-site, walk the territory inspection, but it also assesses the detail and quality of that analysis — even in the absence of contrary scientific evidence in the record. Page after page of the record contains details on field analysis, such as the following representative sample of an evaluation form:

*1074

In my view, it is not our role to criticize this evaluation because, for example, the form does not list the evaluator’s qualifications. Nor are we in a position to un-

scramble and pass judgment on the investigator’s field notes, as in the following excerpt, which are summarized and integrated into the FEIS:

*1075

Had Ecology Center offered a credible scientific critique of this methodology or provided some other basis for us to determine whether the Forest Service’s judgment was arbitrary and capricious, we would be in a different position. But just as we are not supposed to substitute our judgment for that of the Forest Service, neither can we rely on-in the absence of evidence — Ecology Center’s arguments in its briefs or in its expert declaration filed *after*the administrative review process.

*Lands Council*does not direct us to assess the sufficiency of the Forest Service’s on-site soil quality analyses beyond the traditional arbitrary and capricious standard; it only asks us to verify that there is such an on-site sampling. That opinion stated four times that the Forest Service had failed to conduct *any*on-site analysis — it did not say that the Service failed to conduct *enough*of them.4 The majority now extends the “on-site soil analysis” doctrine, initiated by *Lands Council,*while abandoning its reasoning. Although on-site analysis could plausibly be conducted in such a manner as to be arbitrary and capricious, the majority’s *1076justification for finding *this*project to be arbitrary and capricious imports a bright-line rule where individualized analysis is appropriate.

More importantly, in *Lands Council*we did not purport to create a general rule requiring on-site verification for all scientific hypotheses adopted by the Forest Service regardless of context. We did not even adopt a rule requiring all soil analy-ses to be verified by on-site sampling. Our decision was specific to the facts of that case. *See Lands Council,*395 F.3d at 1035*(“Under the circumstances of this case,*the Forest Service’s ... methodology, to be reliable, required that the hypothesis and prediction of the model be verified with observation.”) (emphasis added).

Nevertheless, the majority generalizes the “unverified hypothesis” principle articulated in *Lands Council*beyond the soil analysis context to other scientific findings made by the Forest Service. In so doing, the majority demonstrates the dangers of extending a reference — abstracted from a single, technically detailed, fact-specific decision — to unrelated factual contexts. For example, regarding the old-growth claim, the majority reverses the Forest Service’s judgment because “[hjere, as in *Lands Council,*the Forest Service’s conclusion that treating old-growth forest is beneficial to dependent species is predicated on an unverified hypothesis.” Op. at 16039. The Forest Service hypothesis is, however, supported by observational data. There is record evidence, based on direct observation, that treatment produces and preserves habitat for old-growth dependent species.5

The majority applies *Lands Council*to prohibit the Forest Service from inferring simply that because treating old-growth preserves or creates habitat for dependent species does not mean that such treatment will not harm the species. Op. at 1064 (claiming that the Forest Service just “assumes that a species’ viability is maintained so long as the requisite amount of the species habitat is maintained”). In reaching this conclusion, the majority acknowledges and then dismisses without explanation the very record evidence it says the Forest Service failed to provide — here, direct observation of certain species seen foraging in old-growth areas after treatment.6

Applying *Lands Council*to the old-growth issue is inappropriate for yet another reason. The panel assumes that Alternative 5, which involves thinning, salvage, and regeneration, disrupts a stable status quo “through an invasive process.” Op. at 1064. Yet the status quo is anything but stable. The Forest Service presents uncontested evidence that the failure to treat old-growth areas risks the very harms feared by Ecology Center, even though it has provided no evidence to support such a claim. Op. at 1064 (“Ecology *1077Center does not offer proof that the proposed treatment causes the harms' it fears_”). In fact, the record reveals that the failure to treat old-growth areas could result in “considerable loss of old growth trees from bark beetle predation,” which will put “at risk ... specific habitat niches for many wildlife species that are adapted to the more open growth old-forest character.” Old-growth areas “are now at risk for major disturbances such as disease and insect epidemics and high'-severity stand replacing fires.” Inaction or delay threatens the species Ecology Center seeks to protect.

No one contests the Forest Service’s conclusion that treatment will “provide direct reduction of bark beetle infestation or risk of future infestation.” Faced with uncontroverted evidence that inaction may harm old-growth areas, the majority still requires the Forest Service to produce more evidence that treating old-growth will not harm dependent species before it will allow the Service to save old-growth areas from other dangers. Under these circumstances, I cannot agree that the Forest Service’s decision to treat old-growth areas was arbitrary and capricious. Indeed, had the Forest Service taken the majority’s approach, its decision may well have been arbitrary and capricious for “failing] to consider an important aspect of [the] problem.” *See Lands Council,*395 F.3d at 1026.

Nor do I think it appropriate to analogize the Forest Service process to the separate and wholly inapposite regime of the Food and Drug Administration (“FDA”). The majority writes that “it would be arbitrary and capricious for a pharmaceutical company to market a drug to the general population without first conducting a clinical trial” just as the Forest Service cannot be permitted to “treat more and more old-growth forest without first determining that such treatment is safe and effective for dependent species.” *See*Op. at 1064. Without commenting on the obvious differences between humans and trees (and in fact acknowledging the importance of both to our environment), this analogy underscores the degree to which the majority inserts itself into the internal judgments of the Forest Service. The FDA process dictates a substantive and specific administrative course of action in terms of clinical trials and other requirements as a prelude to the approval of drugs and medical devices. Neither NEPA nor NFMA serve that function in the environmental context. To import the notion of clinical trials from the FDA context to soil sampling in federal forests is a leap too far.

Apparently we no longer simply determine whether the Forest Service’s methodology involves a “hard look” through the use of “hard data,” but now are called upon to make fine-grained judgments of its worth. In reaching this conclusion, the majority takes aim at two firmly established lines of precedent in administrative law. First, this view is contrary to the basic principle that we reverse agency decisions only if they are arbitrary and capricious. This standard of review does not direct us to literally dig in the dirt (or soil, as it were), get our fingernails dirty and flyspeck the agency’s analysis. Yet the majority does exactly that by rejecting the Forest Service’s soil analysis field checks and its observations and historical data in treated old-growth forests. The majority’s rationale cannot be reconciled with our case law requiring “[d]eference to an agency’s technical expertise and experience,” particularly “with respect to questions involving engineering and scientific matters.” *United States v. Alpine Land & Reservoir Co.,*887 F.2d 207, 213 (9th Cir. 1989). *See also Westlands Water Dist. v. U.S. Dept. of Interior,*376 F.3d 853, *1078871(9th Cir.2004) (reversing a district court decision that certain alternatives violated NEPA because of inadequate deference to the agency’s expertise).

Because the majority’s analysis cannot be squared with the deferential review required of us, I respectfully dissent.

1

. *See, e.g., Neighbors of Cuddy Mountain v. U.S. Forest Serv.,*137 F.3d 1372, 1376 (9th Cir. 1998) (noting that we review Forest Service actions under the APA and the National Forest Management Act to determine if they were "arbitrary and capricious”); *Idaho Sporting Cong., Inc. v. Rittenhouse,*305 F.3d 957, 963 (9th Cir.2002) (requiring a "hard *1072look” by agencies at environmental effects of projects); *Idaho Sporting Cong., Inc. v. Thomas,*137 F.3d 1146, 1150 (9th Cir.1998) (noting that courts should not "second guess [] an agency’s scientific conclusions” and, for that reason, requiring hard data to discourage second guessing).

2

. *See e.g. Natural Res. Def. Council v. U.S. Forest Serv.,*421 F.3d 797, 816 (2005) (holding that the Forest Service’s timber sale plan was arbitrary and capricious, and violated NEPA, because it was based on a clear misinterpretation of data on market demand for the timber); *Idaho Sporting Cong. v. Thomas,*137 F.3d at 1154(holding that a Forest Service plan violated NEPA because it had failed to take a "hard look” at the environmental effects of the proposed timber sale).

3

. *See*Op. at 1069 n. 11 (finding the Forest Service's soil transects "did not cover the vast majority of the activity areas; only a few were crossed by coincidence”); *see, e.g., id.*at 1070 (“The record provides little information that enables us to assess the reliability or signifi-canee of these reports; ... we do not know the qualifications of the persons conducting the field review, the methodology utilized. ...”). To paraphrase an old joke, the majority found the food terrible and the portions small.

4

. *See id.*at 1034(''The Forest Service did not walk, *much less test, the*land in the activity area.”); *id.*at 1035 ("The predictions of the model ... were not verified with on the ground analysis.”); *id.*("[T]he spreadsheet models, unaccompanied by on-site spot verification ..., violated NFMA.”); *id.*(The "soils analysis was based *entirely*on the model with *no*on-site inspection ....”) (emphasis added).

5

. For example, a report by Forest Service scientists utilized historic and stand structure research to plan for treatment that would enhance habitat for plants and species. The scientists found that treated areas have nesting and foraging opportunities for cavity nesting species such as pileated woodpeckers.

6

. The majority fails to acknowledge that we “have, in appropriate cases, allowed the Forest Service to avoid studying the population trends of [species] by using ... habitat as a proxy for ... population trends.” *Lands Council,*395 F.3d at 1036. *In Lands Council,*we rejected a proxy-on-proxy approach for measuring population trends in old-growth forest because the habitat data the Forest Service offered was "fifteen years old, with inaccurate canopy closure estimates, and insufficient data on snags.” *Id.*The majority does not identify any such problems in using the proxy-on-proxy approach in this case. *See*Op. at 1064.