OCS Study 

•

MMS 2003-035 

Using Before-After-Control Impact in 

Environmental Assessment: 

Purpose, Theoretical Basis, and 

Practical Problems 

Final Technical Summary 

Final Study Report 

U.S. Department of the Interior 

Minerals Management Service

Pacific OCS Region 
OCS Study 

MMS 2003-035 

Using Before-After-Control Impact in 

Environmental Assessment: 

Purpose, Theoretical Basis, and 

Practical Problems 

Final Technical Summary 

Final Study Report 

Authors 

Allan Stewart-Oaten 

Principal Investigator 

Prepared under MMS Cooperative 

Agreement Nos. 14-35-0001-30471 & 14-35-0001-30761 

by 

Southern California Educational Initiative 

Marine Science Institute 

University of California 

Santa Barbara, CA 93106 

U.S. Department of the Interior 

Minerals Management Service Camarillo

Pacific OCS Region May 2003
Disclaimer 

This report has been reviewed by the Pacific Outer Continental Shelf Region, Minerals Management 

Service, U.S. Department of the Interior and approved for publication. The opinions, findings, 

conclusions, or recommendations in this report are those of the author, and do not necessarily reflect 

the views and policies of the Minerals Management Service. Mention of trade names or commercial 

products does not constitute an endorsement or recommendation for use. This report has not been 

edited for conformity with Minerals Management Service editorial standards. 

Availability of Report

Extra copies of the report may be obtained from: 

U.S. Dept. of the Interior 

Minerals Management Service 

Pacific OCS Region 

770 Paseo Camarillo 

Camarillo, CA 93010 

Phone: 805-389-7621 

A PDF file of this report is available at: 

http://www.coastalresearchcenter.ucsb.edu/SCEI/ 

Suggested Citation 

The suggested citation for this report is: 

Stewart-Oaten, Allan. Using Before-After-Control-Impact in Environmental Assessment: Purpose, 

Theoretical Basis, and Practical Problems. MMS OCS Study 2003-035. Coastal Research Center, 

Marine Science Institute, University of California, Santa Barbara, California. MMS Cooperative 

Agreement Numbers 14-35-0001-30471 & 14-35-0001-30761. 116 pages. 
Table of Contents 

FINAL TECHNICAL SUMMARY 1 

FINAL STUDY REPORT 5 

Chapter 2 of Detecting Ecological Impacts: 

Goals in Environmental Monitoring 5 

Chapter 7 of Detecting Ecological Impacts:

Problems in the Analysis of Environmental Monitoring Data 16 

Chapter 8 of Detecting Ecological Impacts:

Estimating the Size of an Effect from a Before-After-Control-Impact-Pairs 

Design: the Predictive Approach Applied to a Power Plant Study 39 

Sequential Estimation of Log (Abundance) 56 

Temporal and Spatial Variation in Environmental Impact Assessment 68 

Impact assessment 103 

Pseudo-replication 113 

 
Using Before-After-Control-Impact in Environmental Assessment 

1

FINAL TECHNICAL SUMMARY 

STUDY TITLES: 

Study I. a. Environmental Assessment: Statistical Description of Variable Effects on 

Fluctuating Populations 

 b. Environmental Assessment: Statistical Description of Variable Effects on 

Fluctuating Populations (Continuation)

Study II. Adding Biology to BACI: Exploring the Use of Functional Groups, Trophic 

Relationships and Multiple, Ecologically Similar Comparison Sites in Choosing 

Models and Estimating Effects Impacts Analysis

REPORT TITLE: Using Before-After-Control-Impact in Environmental Assessment: 

Purpose, Theoretical Basis, and Practical Problems 

CONTRACT NUMBERS: 14-35-0001-30471 & 14-35-0001-30761 

SPONSORING OCS REGION: Pacific

APPLICABLE PLANNING AREA: Southern California

FISCAL YEAR(S) OF PROJECT FUNDING: 

Study I: a. FY 91, FY 92, FY 93, FY 94 

b. FY 95, FY 96, FY 97 

Study II: FY 97, FY 98 

COMPLETION DATE OF THE REPORT: March 2001 

COST(S): 

Study I: a. FY 91 - $27,440, FY 92 - $36,163, FY 93 - $31,587, FY 94 - $16,650

b. FY 95 - $22,346, FY 96 - $83,980, FY 97 - $78,674 

Study II: FY 97 - $30,148, FY 98 – $47,400, FY 99 – no cost 

CUMULATIVE PROJECT COSTS: 

Study I: a. $111,840

b. $185,000 

Study II: $77,548 

PROJECT MANAGER: Russell J. Schmitt

AFFILIATION: University of California, Santa Barbara

ADDRESS: Coastal Research Center, Marine Science Institute, University of California, 

Santa Barbara, CA 93106-6150

PRINCIPAL INVESTIGATOR: 1

Allan Stewart-Oaten 
Final Technical Summary – Stewart-Oaten 

2

Co-PI (Study II): 2Stephen C. Schroeter 

ADDRESSES: 1Department of Biological Sciences and Department of Ecology, Evolution, 

and Marine Biology, University of California, Santa Barbara, CA, 93106-9610; 2Marine 

Science Institute, University of California, 2270 Camino Vida Roble, Suite L, Carlsbad, CA 

92009 

BACKGROUND: Three types of decision in the management of offshore mineral resources 

are: whether to accept, modify or reject a proposed development (e.g., an oil platform), 

whether to continue, modify or discontinue an ongoing operation, and whether an operation 

has caused such damage that penalties or mitigation should be required. The second and third 

types often depend on monitoring assessment: determining the effects of a development after 

a period of operation. Many environmental variables, like population abundances, fluctuate 

naturally over time even without anthropogenic intervention, so assessments need to deal with 

time series where both serial correlation and systematic (e.g., seasonal) variation are likely. 

OBJECTIVES: The main aims of these two programs have been: 

(1) to develop ways to estimate or describe effects of an "alteration" of the environment on 

naturally fluctuating biological variables, using one or more neighboring, similar "control" 

sites to reduce and estimate the effects of natural temporal variation on these estimates; 

(2) to test the methods on real data, from the annual surveys of the Channel Islands National 

Park Service: specifically, to see whether neighboring, similar sites can be used to predict 

each other's fluctuations over time, such that the temporal variation and serial correlation of 

the residuals from the prediction are smaller than those of the original data.

DESCRIPTION: Two related methods were studied. One uses the difference between 

values observed at the "Impact" site and those observed at the controls. The values at the sites 

might be transformed, and multiple values from a set of controls can be averaged or otherwise 

summarized. The other method uses control values as covariates; in effect, it finds equations 

for predicting Impact values from control values under "before alteration" conditions and 

compares their predictions with either the predictions of the corresponding equations under 

"after" conditions or with the actual values of the sites observed after the alteration is in place. 

In this project, our main aim has been to develop and explain the broad approach, dealing 

with objections and misunderstandings, and explaining why some other approaches, which do 

not account for variation over time, do not separate natural variation from human effects. 

The Channel Islands data are annual surveys of about 70 species at 13 sites since 1981; a 

further site was added in 1983 and two more in 1986. Some species have been added and 

others dropped over the years. There are data on size and recruitment, but we have worked 

mainly with the abundance data for three groups, "band" , "quad" and "rpc" (the names refer 

to the sampling method). At each site, all samples use a single 100-metre transect. The 

positions on this transect are chosen anew each year, by choosing a random point and spacing 

positions equally from it to each end. The "band" species are sampled by counting in bands 

(currently 3X20m but this has varied) across the transect at the chosen positions. The "quad" 

species are sampled by counts in quadrats (currently 2m2

 but this has varied), and the "rpc" 

species by the fractions of contacts with (currently) 40 points on the boundaries of two 
Using Before-After-Control-Impact in Environmental Assessment 

3

concentric ellipses. Our main efforts here have gone into finding ways to summarize the data 

to check the idea that data from "similar" sites might have similar fluctuations. We have used 

plots and correlations for combinations of species, pairs and triples of sites, and 

transformations (raw data, logs, reciprocals, etc.), but there are so many possibilities that we 

need ways to summarize these summaries. 

SIGNIFICANT CONCLUSIONS: 1. An explicit, unambiguous definition of an "effect" is 

needed for design and interpretation of assessments. A few approaches and critiques use an 

explicit definition which is flawed. An example is "Impacts are those disturbances that cause 

mean abundance in a site to change more than is found on average" (Underwood 1992: 

Journal of Experimental Marine Biology and Ecology, 161, p. 152). The "mean" and 

"average" are undefined: in fact, the "mean" is implicitly taken to be the mean over the study 

period (thus not allowing for natural fluctuation over a time period of this length), and the 

average is over a "population" of sites which must be chosen subjectively - in practice, 

usually implicitly. Many other studies use implicit definitions with similar flaws. The 

definition of an effect should refer only to the alteration site, since a given effect is not 

changed if unaffected sites are naturally very different from this site. If other sites are similar 

to the alteration site, they can be used to improve estimation of the effect, but not to define it. 

2. There may not be a single "best" use of "control" sites. The most direct use is by 

differences or as covariates (see DESCRIPTION), but this involves a tradeoff between 

reducing the effects of large, long-term natural fluctuations, assumed to be widespread, and 

increasing the effect of short term, local variation and sampling error. This can be beneficial 

even when the apparent error of effect estimates seems to increase, because the apparent error 

is likely to underestimate the variation due to long-term fluctuations. However, the control 

site that best reduces these fluctuations may be different for different species. There may be 

no control site similar enough to do it well. There may be no large, long-term fluctuations, or 

some that are local but too rare or irregular to be allowed for in a time series model. 

STUDY RESULTS: 1. In papers 2, 6 and especially 5 below, we have defined an effect as 

the difference between the average abundance at the Impact site over some long period, such 

as the life-length of the alteration, and the average that would have been obtained without the 

alteration. We then show that, in principle, time series methods can be used to estimate this 

difference and give an estimate of uncertainty. 

2. The Channel Islands data illustrate several of the difficulties in the use of control sites (see 

CONCLUSIONS). Some pairs of close sites have high correlations for some species, but the 

relation between correlation and distance is weak, for any obvious meaning of "distance". For 

example, maps with lines joining sites with correlations > x (for various x) show plenty of 

lines connecting distant sites, and sites facing north to sites facing south. Sampling error is 

part of the cause: most of the time series are best fitted as independent observations. 

Variation over the replicate quadrats, is usually much smaller than the variation over time, but 

it does not include the error in the fixed 100m transect as an estimate for the site. Another 

cause may be that the sites were deliberately chosen to give as broad a range of conditions as 

possible - and thus to be dissimilar from each other. The study continues by comparing series 

from the first 50m of the 100m transect to the series given by subtracting the second half from 
Using Before-After-Control-Impact in Environmental Assessment 

117

The Department of the Interior Mission 

As the Nation's principal conservation agency, the Department of the Interior has responsibility for most 

of our nationally owned public lands and natural resources. This includes fostering sound use of our 

land and water resources; protecting our fish, wildlife, and biological diversity; preserving the 

environmental and cultural values of our national parks and historical places; and providing for the 

enjoyment of life through outdoor recreation. The Department assesses our energy and mineral 

resources and works to ensure that their development is in the best interests of all our people by 

encouraging stewardship and citizen participation in their care. The Department also has a major 

responsibility for American Indian reservation communities and for people who live in island territories 

under U.S. administration. 

The Minerals Management Service Mission 

As a bureau of the Department of the Interior, the Minerals Management Service's (MMS) primary 

responsibilities are to manage the mineral resources located on the Nation's Outer Continental Shelf 

(OCS), collect revenue from the Federal OCS and onshore Federal and Indian lands, and distribute 

those revenues. 

Moreover, in working to meet its responsibilities, the Offshore Minerals Management Program

administers the OCS competitive leasing program and oversees the safe and environmentally sound 

exploration and production of our Nation's offshore natural gas, oil and other mineral resources. The 

MMS Royalty Management Program meets its responsibilities by ensuring the efficient, timely and 

accurate collection and disbursement of revenue from mineral leasing and production due to Indian 

tribes and allottees, States and the U.S. Treasury. 

The MMS strives to fulfill its responsibilities through the general guiding principles of: (1) being 

responsive to the public's concerns and interests by maintaining a dialogue with all potentially affected 

parties and (2) carrying out its programs with an emphasis on working to enhance the quality of life for 

all Americans by lending MMS assistance and expertise to economic development and environmental 

protection.