Data Quality Objectives and Statistical

Design Support for Development of a

Monitoring Protocol for Recreational Waters

Final Report

Submitted to

U.S. Environmental Protection Agency

Microbial Exposure Research Branch

Microbiological and Chemical Exposure Assessment Research Division

National Exposure Research Laboratory

Prepared by

Research Triangle Institute

U.S. EPA Contract 68-D4-0091

Work Assignment 99-17

RTI Task 91U-7249-417

August 24, 1999
i

Table of Contents 

INTRODUCTION ......................................................... . 1

STEP 1: STATE THE PROBLEM ............................................. . 5

STEP 2: IDENTIFICATION OF DECISIONS AND 

RESEARCH OBJECTIVES ............................................ . 8

STEP 3: IDENTIFY THE INPUTS TO THE DECISION ........................... . 11

STEP 4: DEFINE THE STUDY BOUNDARIES ................................. . 14

STEP 5: DECISION RULE FOR BEACH CLASSIFICATION DECISION ............. . 17

STEP 6: SPECIFY ALLOWABLE LIMITS ON DECISION ERRORS ................ . 19

STEP 7: OPTIMIZE THE DESIGN ........................................... . 22

ATTACHMENT A WORKSHOP PARTICIPANTS ............................... . 26

List of Figures 

Figure 1. EPA’s Quality System ............................................... . 4

Figure 2. Development and Use of Monitoring Guidance ........................... . 10

Figure 3. Plan View of a Beach Showing Elements of the Statistical

 Design from the WHO Report ....................................... . 25

List of Tables 

Table 1. Input Parameters for the Guidance Development Decision ................... . 11

Table 2. Research Questions for Guidance Development ............................ . 18
RTI/7249/417/2.4.3 Page 1 of 29 Final Report 9/24/99

Data Quality Objectives for Developing a Protocol for

Monitoring Recreational Waters

Comment Compendium Draft

INTRODUCTION

On June 14, 15, and 16, 1999 members of the Environmental Protection Agency (EPA) National

Exposure Research Laboratory (NERL-Cincinnati) and Office of Water (OW), Office of Science

and Technology, Washington, DC (OST) held a workshop in Cincinnati, Ohio with invited nonEPA experts in environmental microbiology and statistics to plan a research study that will be

used to develop guidance, to be used nationwide, for monitoring recreational waters for indicator

bacteria.

Purpose of this Document

The purpose of this report is to describe the outputs of the Data Quality Objectives (DQOs)

Process and discussions about developing a statistical design that will be used to implement the

research study of recreational beach waters.

Background on Recreational Water Quality Monitoring

Current Environmental Protection Agency (EPA) recommended monitoring practices for bathing

beach water quality were suggested in 1968, as a part of the fecal coliform guideline developed by

the Federal Water Pollution Control Administration. The guideline stated that five water samples

should be taken over a 30-day period and that the geometric mean of the fecal coliform count of

the five samples would be used to determine compliance with the guideline. Since that time, EPA

has developed improved health risk-based guidelines for bathing beach waters and improved

methods for monitoring using Escherichia coli (E. coli) and enterococci as the indicators of

recreational water quality. However, the recommendations to the States in 1986 used the old

monitoring protocol for bathing beach waters.

The existing guidance for sampling and interpreting monitoring data is not particularly clear, and

the data resulting from its use are not useful to regulatory authorities or the public. The current

guidance does not address the variety of beach environments, i.e., marine or estuarine versus fresh

water, lake versus river, or the dynamics and variation in pollution sources that can affect beach

water sanitary quality on a daily basis. Therefore, data obtained using the current sampling

protocol are not necessarily relevant with regard to the circumstances or conditions at the

sampling sites on the day of sampling.

To address these problems, EPA’s OW, OST and the Microbiological and Chemical Exposure

Assessment Research Division (MCEARD) of NERL-Cincinnati in collaboration with the
RTI/7249/417/2.4.3 Page 2 of 29 Final Report 9/24/99

Environmental Monitoring for Public Access and Community Tracking (EMPACT) Program are

sponsoring a research project. This research has two objectives:

C develop guidance (i.e., a set of procedures) that can be used at individual beaches to

develop protocols for monitoring the quality of bathing waters, and 

C develop a system for translating technical monitoring data into easily understood risk

information so that the public can make informed decisions on the use of recreational

waters.

The research project is comprised of six phases:

1. The purpose of Phase 1 is to define all the elements that might influence the performance

of an effective sampling plan for monitoring bathing beach water quality. This phase has

been completed. 

2. Phase 2, which has also been completed, is to conduct a pilot sampling study at selected,

but representative, sites to empirically determine the magnitude of variation in indicator

concentrations that would be encountered at bathing beaches under natural conditions.

3. Phase 3 (this workshop) involves developing DQOs and a statistical sampling design for a

research project monitoring five characteristic recreational beach waters selected for the

EMPACT study. 

4. Phase 4, which will take place in the summer of 2000, involves contractors or EMPACT

partner cities conducting sampling studies using the design developed in Phase 3. 

EMPACT partner cities will be selected based on the type of bathing beach site available,

sources of fecal contamination affecting the sites and the availability of the laboratory

facilities. Five sites representing five characteristic recreational water environments will be

selected for monitoring:

C ocean beach (east coast),

C ocean beach (west coast),

C freshwater beach (Great Lake),

C freshwater inland river “beach” or recreational area, and

C estuarine beach.

The five sites that are being monitored in Phase 4 will not be surrogates for all beaches,

but the data from these sites will be used to develop guidance that other cities could

follow.

5. The purpose of Phase 5 is to interpret the collected data to determine the minimum

number of samples that must be taken and locations for sampling each site to adequately

describe the sanitary quality of the water. 
RTI/7249/417/2.4.3 Page 3 of 29 Final Report 9/24/99

6. The final phase, Phase 6, will be a workshop to develop a means to translate technical data

gathered from bathing beach monitoring efforts into understandable information that can

be used by the general population to make risk-based decisions about using recreational

beach resources.

Summary of the Pilot Study

Phase 2 of the research project, which was conducted in the summer of 1998, estimated

concentrations of two indicators of fecal contamination, Escherichia coli and enterococci. The

purpose of collecting these data was to obtain estimates of the variability in the concentrations of

these indicators, which is an important input in Step 6 of the DQO Process. 

In Phase 2, sampling was done at an estuarine beach site (Tenean Beach, Boston Harbor) in

Boston, ocean beach sites in New York (D.A. Beach Club, East Chesterbay, Bronx) and St.

Petersburg Florida (Northshore Beach), river beach sites (Belleville Lake) in Belleville, Michigan,

and Pensacola Florida (Blackwater River, Blackwater River State Park), and a lake beach site

(Metro Beach, Lake St. Clair) in Detroit. At each sample site, samples were collected every other

day so that samples were collected on three separate days. Samples were collected in the

morning, around noon, and in the afternoon of each sample date. Samples were collected from

three transects (areas of the designated beach) established along each beach; one transect was at

the “end” of the beach, one in the “middle” of the beach, and one along a line where the greatest

number of bathers were found. Along each transect, samples were taken where water depths

were approximately 11⁄2 and 4 feet. At each of the two locations along each transect, samples

were collected 1 foot below the surface of the water. Results of the pilot study are presented in

DQO Process Step 7.

The EPA Quality System, DQO Process, and Quality Assurance Project Life Cycles

The EPA Quality System was established in 1984 by EPA Order 5360.1. The Order requires all

environmental programs conducted by, or on behalf of, EPA to be supported by mandatory

Quality System (previously called a Quality Assurance Program). In 1997, the Order was

significantly revised and reissued (EPA Order 5360.1 CHG 1) and requires EPA organizations 

and holders of extramural agreements to conform the requirements of the American National

Standard ANSI/ASQC E4-1994, Specifications and Guidelines for Quality Systems for

Environmental Data Collection and Environmental Technology Programs. By establishing its

own Quality System, EPA ensured that regulatory decisions were based on data of known quality.

The EPA Quality System, as shown in Figure 1, is comprised of three structural levels:

C policy level,

C organization/program level, and

C project level.

At the policy level, the Quality System addresses requirements for implementation of the System.
RTI/7249/417/2.4.3 Page 4 of 29 Final Report 9/24/99

Quality

Manual

POLICY ORGANIZATION/

PROGRAM

QAARWP Training

QMPs

(G-2, R-2)

PROJECT

DQOs

(G-4)

QAPPs

(G-5, R-5)

SOPs

(G-6)

TAs

(G-7)

DQA

(G-9)

Data

Collection

PLANNING IMPLEMENTATION ASSESSMENT

EPA

Order

MSRs

(G-3)

(R-1)

(G-10)

Data Ver./

Data Val.

(G-8)

Figure 1. EPA’s Quality System

At the organizational level, the Quality System addresses implementation of:

C quality management plans,

C management system reviews,

C training, and

C the Quality Assurance Annual Report and Work Plan.

At the project level EPA’s Quality System includes the following components that address

planning, implementation, and assessment:

C the DQO Process,

C quality assurance project plans (QAPPs), and

C implementation of Data Quality Assessment (DQA) guidance.

The DQO Process is a part of the planning phase because it defines the problem to be studied, the

limitations of the study, and the type,

quantity, and quality of data required. 

As a result of the DQO Process, a

statistical sampling design can be

developed and documented in the QAPP. 

Methods that will be used to collect and

analyze samples are documented in the

QAPP. Finally, in the assessment phase,

the data are analyzed and interpreted.

An important element of EPA’s Quality

System is that it follows a graded

approach so that the degree of quality

controls applied to a project is based on

the intended use of the results and the

amount of confidence needed in the

quality of the results. 

Organization of this Report

This report is organized around the seven steps of the DQO Process, because the workshop was

organized according to these steps. These seven Steps are: (1) State the Problem, (2) Identify

the Decision, (3) Identify the Inputs to the Decision, (4) Define the Study Boundaries, (5)

Develop a Decision Rule, (6) Specify Allowable Limits on Decision Errors, and (7) Optimize the

Design.
RTI/7249/417/2.4.3 Page 5 of 29 Final Report 9/24/99

STEP 1: STATE THE PROBLEM

The purpose of this step of the DQO Process is to describe the problem to be studied. This

includes identifying the planning team members, describing the problem at hand, and summarizing

the relevant deadlines.

Identify the Planning Team

The planning team is the group who will develop the DQOs for the study being designed.

The planning team for developing DQOs for the study that will provide data used to develop the

recreational beach water monitoring guidance is comprised of the participants of this workshop

(refer to Attachment A for the participant list and contact information).

The head of the planning team and the decision-maker for this project is Dr. Alfred P. Dufour of

the U.S. EPA, National Exposure Research Laboratory (NERL).

There is also a project leadership group that assists Dr. Dufour in deciding how to proceed with

the DQO Process, whose members include:

C Steve Schaub (EPA, OW, OST)

C Kristen Brenner (EPA, NERL)

C Larry Wymer (EPA, NERL)

C John Martinson (EPA, NERL)

C Walt Stutts (EPA, NERL)

Statement of the Problem

The current recommendation for monitoring the quality of recreational waters calls for the

collection of 5 samples over a 30-day period and the calculation of a running geometric mean to

determine if the water quality meets existing regulatory standards. This approach does not

provide timely, accurate information for risk managers or the public, nor does it provide data that

can be easily interpreted by any group, such as individual swimmers or decision makers. A

solution to this problem is to develop a statistically valid monitoring guidance that takes into

account elements that contribute to the uncertainty associated with sampling bathing beach waters

and measurements of the potential health effects of bathing beach waters.

Objectives of the Study, Phase 4 of the Research Project

The purpose of this workshop, Phase 3 of the research project, was convened to develop DQOs

and a statistical sampling design for the five characteristic beach environments. The objective of

studying the five beach environments is to collect data to develop guidance that can be used to
1Ambient Water Quality Criteria for Bacteria – 1986, Bacteriological Ambient Water

Quality Criteria for Marine and Fresh Recreational Waters, EPA A440/5-84-002, U.S.EPA,

Washington, D.C.

2Cabelli, V.J., 1983, Health Effects Criteria for Marine Recreational Waters, EPA-600/1-

80-031, U.S. EPA, Research Triangle Park, N.C.

3Dufour, A.P. 1984, Health Effects Criteria for Fresh Recreational Waters, EPA-600/1-

84-004, U.S. EPA, Cincinnati, OH.

RTI/7249/417/2.4.3 Page 6 of 29 Final Report 9/24/99

design and implement a site specific statistically valid and defensible sampling designs at beaches

nationwide.

The purpose of the study is not to provide national estimates of bacterial contamination in

recreational beach waters, but to characterize and quantify the variance in bacterial concentrations

and understand the key factors that influence how bacterial concentrations vary over space and

time. This variance is one of many sources of the total uncertainty affecting decision making at

beaches. This information will be used to develop guidance for designing sampling protocols for

use at specific sites (i.e., it will provide guidelines to determine where to sample, how many

samples to take, and how often to sample at a given beach site). The guidance will be applied to

specific beaches based on knowledge of local site-specific physical, environmental, and

recreational use factors.

In addition to the main objective, there are potential secondary uses of the data that will be

generated from monitoring at the five characteristic environments. The items listed below are

some of the potential additional uses for these data:

C provide data to support modeling of bacterial concentrations over time and space at

beaches,

C support data collection for estimates of bacterial concentrations for national regulation

development, 

C support analysis of variability across sites with respect to supporting epidemiological

studies, and

C identify the quantity and type of ancillary data that need to be collected.

Conceptual Model of Exposure

In 1987 the U.S. EPA published recommended guidelines1 for maintaining the quality of

recreational waters. The guidelines were based on studies which showed that the rate of

swimming-associated gastroenteritis increased as the quality of the recreational water degraded2,3. 

The quality of the water was measured using bacterial indicators to estimate the extent of fecal

contamination. E. coli levels showed the best relationship to the swimming-associated rate of
RTI/7249/417/2.4.3 Page 7 of 29 Final Report 9/24/99

gastroenteritis in freshwater, while enterococci densities were equally effective for relating illness

in swimmers to water quality in both fresh and marine waters.

Indicator organisms were used to measure the quality of the recreational water. The density of

indicator bacteria indicate the level of fecal contamination and, therefore, the possible presence

and extent of enteric pathogens. The organisms that cause illness in swimmers are unknown. The

health endpoints are based on symptomatology, such as diarrhea, vomiting, and nausea. 

The major route of exposure for swimmers with regard to microbial pathogens was ingestion,

although dermal and respiratory exposures were also recognized as being potentially associated

with pathogens.

Schedule

The schedule for the remaining phases, Phases 4, 5, and 6, of the overall research project is as

follows.

C Phase 4 of the research project, the sample collection at the five characteristic

environments, will be performed during the bathing season of year 2000. The five

EMPACT cities will be selected by mid-September 1999.

C Phase 5 of the EMPACT project will be initiated in fiscal year 2000 and completed in

fiscal year 2001. 

C Phase 6 of the project will be initiated in fiscal year 2000 and completed in fiscal year

2001.

Social and Political Factors

Proposed legislation that possibly could be influenced by this project include H.R. 999, also

known as the Bilbray bill, and S. 522, which is also called the new Lautenberg bill. Both of these

bills would amend the Clean Water Act and require implementation of regulations within 4-5

years. EPA would be required to publish a monitoring strategy, i.e., how and when to monitor a

beach. The data provided by the EMPACT project will support the latter requirement.

The EMPACT project is a high priority research program within the Agency, and the data from

the project are high priority needs in a majority of the States. In addition, the public has a great

concern in this area, mainly because of the high public profile of adverse information about the

quality of beach water that is frequently seen in the print and electronic media. The public has

high expectations about the quality of recreational water that they use for leisure time activities.
RTI/7249/417/2.4.3 Page 8 of 29 Final Report 9/24/99

STEP 2: IDENTIFICATION OF DECISIONS AND 

RESEARCH OBJECTIVES

The purpose of this step of the DQO Process is to define the problem that the study will attempt

to resolve. Figure 2 presents the general process for developing and using the beach water

monitoring guidance.

Two Levels of Decisions

In developing DQOs for the study of the five characteristic beach environments, two levels of

decision-making must be kept in mind. The first level relates to the decisions that must be made

in developing a guidance manual that local authorities can use to develop a sampling plan to

measure indicator bacteria concentrations at their local recreational beaches. This is referred to as

the guidance development decision. The second level is the decision that local authorities make,

with the help of the guidance and local regulations or national guidelines, about the daily status of

a given beach (e.g., closed, posted warnings, or open for use) based on the indicator bacteria

concentration and supporting data collected at that beach. This decision is referred to as the

beach decision. Where possible, the remaining steps of the DQO Process address each level of

decision-making. 

Guidance Development Decision

The purpose of this step is to support the development of guidance for monitoring beach waters

that will help users collect the correct type, quality, and quantity of data to make the beach

decision. This research study must determine the sources and magnitude of variability in

estimating indicator bacteria concentrations. In addition, the best statistic or combination of

statistics (e.g., arithmetic mean, geometric mean, or percentile of the indicator bacteria

concentration) to support a risk-based decision must be selected.

Potential sources of variability include:

C location of sampling,

C time and trends (timing of sampling),

C number and age of bathers in the water, 

C presence or absence of point or non-point sources of fecal contamination,

C point versus non point source of fecal contamination, 

C the number and magnitude of sources of fecal contamination (e.g., storm water and

sewage outfall at one site), 

C events (e.g., rain, discharges from boats), 

C physical and hydrological parameters, 

C sample collection and transport, and

C analytical method.
RTI/7249/417/2.4.3 Page 9 of 29 Final Report 9/24/99

Beach Decision

The decision to be made at the beach level is to determine, based on the results of the water

quality monitoring, how to classify the water quality within a locally determined risk-based

classification scheme.

The risk-based classification scheme will be developed in Phase 6 of this research project. That is,

a stakeholder process will be used to determine what type of index and how many categories will

be used to indicate the level of risk of illness to bathers from contact with the water. For

example, a color scheme could be used where red indicates beach closure, yellow is a warning to

bathers, and green means the water is safe. The intent of the categorization scheme is to allow

bathers to make their own decisions about going into the water or not based upon the degree of

risk of illness they are willing to accept.
RTI/7249/417/2.4.3 Page 10 of 29 Final Report 9/24/99

EMPACT Study

Planning and

Sampling Design

Analysis of EMPACT

Study Results

Research

Study Data

Collection

Development of

Guidance on the

Design of Protocols

for Sampling

Recreational Bathing

Waters

Assess Overall

Results and

Revise Guidance

Secondary Data Uses

Application of

Guidance to Specific

Bathing Water

Beaches

Final Guidance

(and periodic

revisions)

Draft Guidance

Posting of

Beach Status

Consequences

Data

Collection

at Specific

Beaches

 = Process Flow

 = Information Flow

Figure 2. Development and Use of Monitoring Guidance
RTI/7249/417/2.4.3 Page 11 of 29 Final Report 9/24/99

STEP 3: IDENTIFY THE INPUTS TO THE DECISION

The purpose of this step is to identify the informational inputs that will be required to resolve the

decisions identified above in Step 2.

Inputs to the Guidance Development Decision

The impact of the parameters listed in Table 1 on the concentration of indicator bacteria should be

estimated to support development of the monitoring guidance. The list of parameters in Table 1

is a modified version of a list developed at a World Health Organization (WHO) workshop on

recreational waters held in Annapolis, MD November 8 to 13, 1998. The requirements for

measuring each parameter at each of the characteristic environments are indicated in Table 1.

Table 1. Input Parameters for the Guidance Development Decision

CHARACTERISTIC STUDY SITE

INPUT

PARAMETER

Fresh

Water

West Coast

Beach

East Coast

Beach

River Estuarine

Salinity/

Conductivity

!

1 ! ! ! !

Temperature (Air

and Water)

! ! ! ! !

Total Suspended

Solids

! ! ! ! !

Time ! ! ! ! !

Beach Study

Location

! ! ! ! !

Rainfall ! ! ! ! !

Wave

Length/Height

! ! ! NA2 !

Current 3 ! ! ! ! !

Sunny/Cloudy 4 ! ! ! ! !

Tidal Stage NA ! ! NA !

Lake or River

Level

! NA NA ! NA

Wind Speed ! ! ! ! !
CHARACTERISTIC STUDY SITE

INPUT

PARAMETER

Fresh

Water

West Coast

Beach

East Coast

Beach

River Estuarine

RTI/7249/417/2.4.3 Page 12 of 29 Final Report 9/24/99

Number and

Type of Bathers

in the Water

! ! ! ! !

Number of

Animals

Potentially

Affecting the

Water

! ! ! ! !

Number of Boats

in the Water

! ! ! ! !

Debris ! ! ! ! !

1 ! = Required.

2 NA = Not Applicable.

3

 = Current measurements should capture direction, but it may be too expensive to measure

speed.

4

 = To measure light, an automated system that measures visible and long-wave ultraviolet light,

which are biocidal, should be used at the sampling site.

The workshop participants also considered pH, turbidity, nutrients, and sand temperature but they

were not considered the most important variables to measure.

Inputs to the Beach Decision

Using the monitoring guidance, which will be developed from estimates of the impact of the Table

1 parameters on the concentration of indicator bacteria at the five characteristic recreational water

environments, local authorities will develop a sampling plan to measure indicator bacteria

concentrations at their local recreational beaches. In addition to the measured concentrations of

the indicator organism in the beach water, the information necessary for the local decision-makers

to determine the status of the beach on a given day includes the factors listed below.

C the risk-based standard or regulatory level that is the threshold for taking action,

C status of known contamination sources (provides information on why bacteria

concentration may be high and how long it may remain high),

C point sources versus nonpoint sources of fecal contamination,

C events (e.g., rainfall, sewage system failure, boat discharges),

C past history of site (e.g., do bacterial concentrations tend to stay high) and at nearby
RTI/7249/417/2.4.3 Page 13 of 29 Final Report 9/24/99

beaches,

C weather conditions,

C bias and precision in the measurement method, and

C physical characteristics of the site.
RTI/7249/417/2.4.3 Page 14 of 29 Final Report 9/24/99

STEP 4: DEFINE THE STUDY BOUNDARIES

The purpose of this step is to define the spatial and temporal boundaries of the problem to be

addressed. This involves first identifying the target population from which samples will be

collected. The spatial boundaries define the physical area to be studied and the locations where

samples should be taken. The temporal boundaries describe the time frame the study data will

represent and when the samples should be taken. In this section, the sequence of presenting the

guidance development versus beach decisions has been reversed, because the boundaries of the

guidance development decision are driven by how the beach decision will be made.

Identify the Target Population: Beach Decision

The target population is all the water in the defined beach area, with the primary characteristic of

interest being the concentration of the indicator organism.

Geographic Boundaries

The geographic boundaries for the beach decision are the same as those of the target

population: the width and extent of the designated beach, from the surface of the water to

the sediment.

Temporal Boundaries

The question regarding the temporal boundaries of this study is how often does a decision

regarding the quality of the beach water and the associated risk to bathers need to be

made. For instance, if the indicator bacterial concentrations have been below the

threshold then how often does a decision about the beach have to be made. Not enough is

known about the effect of the input parameters over time to answer this question.

Temporal boundaries will be site specific. Once sites are selected there may be specific

temporal issues that are relevant. For instance there may be cyclic patterns (diurnal and

seasonal patterns of beach use) that could cause systematic variation in the concentration

of indicator bacteria.

Identify the Target Population: Guidance Development Decision

The target population for the guidance development decision is the same as for the beach

decision.

Geographic Boundaries

The geographic boundaries for the research study that will be used to develop the

guidance extend beyond the beach to include the watershed around the beach and the area

where significant contaminant sources may be located.
4Kay, D., Fleisher, J.M., Salmon, R., Jones, F., Wyer, M.D., Godfree, A., Zelanauch –

Jaquotte, Z. and Shore, R. 1994 “Predicting the Likelihood of Gastroenteritis from Sea Bathing:

Results from Randomized Exposure” The Lancet 34:905-909.

RTI/7249/417/2.4.3 Page 15 of 29 Final Report 9/24/99

Temporal Boundaries

The temporal boundary for the research study that will be used to develop the guidance is

to represent the bathing season. Currently, the time frame for data collection is during

June through August of 2000. The data collection period may be different for sites in

warmer climates where the bathing season isn’t restricted to the summer months.

Information on temporal variability from the research study will be used to develop

guidance on how to address temporal boundaries for the beach decision.

Stratified Sampling

Stratified sampling may be useful in some cases. The intent of identifying potential strata in

stratified sampling is to facilitate the development of more efficient sampling designs. In this case,

the intent of statistically stratifying a beach site is not to make beach-level decisions for each

stratum, but to improve the sampling design to make a better decision for the entire beach. 

Possible statistical strata could be based on water depth or distance from the shore, proximity to

point sources, areas of the beach water where contaminants may collect due to physical conditions

or weather patterns, physical stratification of fresh water/salt water, or the swash zone near the

shore line. The study to support the guidance decision will attempt to identify factors that might

define such strata. These factors will be addressed in the sampling guidance so that managers of

particular beaches will have clear guidance on how to specify statistical strata for their sampling

design.

Scale of Decision-Making

The scale of decision-making refers to the smallest, most appropriate subsets of the population for

which decisions will be made based on the temporal or spatial boundaries. 

Samples are collected one foot below the surface because it was felt that head immersion was a

key component to exposure and this depth would represent head immersion. In addition, it is

difficult to achieve replicability when sampling at the surface, and a study has shown that sampling

at one foot correlates well with health effects4.

Beach Decision

The scale of decision-making is the designated beach in question. That is, one decision

will be made about the status of the entire beach, not about parts of the beach over some

time frame.
RTI/7249/417/2.4.3 Page 16 of 29 Final Report 9/24/99

Guidance Development Decision

The scale of decision making does not apply in the same way to the research study

supporting the guidance decision. The research addresses a number of questions or

hypotheses, hence, the scale of decision making will vary depending on the question posed

or hypothesis to be tested. The scale of decision making will be incorporated into each

research question or hypothesis in DQO Process Steps 5 and 7 below.

Practical Constraints

Identify any constraints or obstacles that could potentially interfere with the full implementation of

the data collection design. The following constraints were identified.

C Consistently locating the sample location in the water (e.g., staying aligned with a

designated point on the beach, determining the correct water depth from which to

sample).

C The QA/QC requirement for analysis of water samples for indicator bacteria (samples

must be received by laboratory within 6 hours of collection and analysis should be

performed within 2 hours of sample receipt.).

C Other constraints could be really inclement weather, inability to simultaneously collect

multiple samples from a site, and sampling in itself will alter water quality.
RTI/7249/417/2.4.3 Page 17 of 29 Final Report 9/24/99

STEP 5: DECISION RULE FOR BEACH CLASSIFICATION DECISION

The purpose of this step is to define the parameter of interest, specify the action level, and

integrate previous DQO outputs into a single statement that describes a logical basis for choosing

among alternative actions. This decision rule is made of the statistical parameter that is chosen to

characterize the population, the action level, and an “if.., then...” statement that defines the

conditions that would cause the decision maker to choose among alternative actions.

Parameter

The parameter for the beach decision will be determined in Phase 5 of the project with

stakeholder input. The current method uses the sample geometric mean of the indicator density of

five samples collected over a 30-day period. The alternatives to be considered include the

arithmetic mean, geometric mean, frequency of a single value exceedance, upper confidence limit

for some statistic, and an upper percentile of the indicator organism distribution.

Action Level

The action level will be consistent with the parameter yet to be determined. The current action

level is based on the geometric mean.

If/Then Statement

The decision rule below includes the parameter of interest whose true value the decision maker

would like to know. However, decisions cannot be made about the true value of the parameter

because this can only be estimated by the data collected on the indicator bacteria. One possible

decision rule is presented below.

If the true parameter of the concentration of the indicator bacteria within the designated

beach waters at the time of sampling is above the health-based threshold then close the

beach or post a red flag and investigate and try to solve problem causing the

contamination.

If the true parameter of the concentration of the indicator bacteria is below the healthbased threshold but above a warning level, then post a warning at the beach. It may be

necessary to investigate the source of the problem.

Otherwise, the beach is open for normal use and posted accordingly.

Relationship of Research Questions to Guidance Development

To develop the guidance manual for monitoring recreational waters, the impact of the factors in

Table 2 on the concentration of indicator bacteria must be estimated. All factors listed apply to

each of the five characteristic sites except for tides, which apply only to beach and estuarine
RTI/7249/417/2.4.3 Page 18 of 29 Final Report 9/24/99

environments. Also, vertical stratification of water levels may be more important in the fresh

water lake and estuarine environments.

Table 2. Research Questions for Guidance Development

Parameter Temporal or Spatial Priority

Direction of current with respect

to sources of contamination and

distance from a point source.

Spatial and temporal High

High number of bathers versus a

low number or none

Temporal and spatial High

High number of children bathers

versus a low number or none

Temporal and spatial High

Sunny versus cloudy conditions Temporal Low

Rain event versus no rain Temporal High

High number of animals

potentially affecting the beach

versus a low number or none

Temporal and spatial Low

High versus low tide Temporal and spatial

because it is a current

High

Water depth/distance from shore: 

ankle versus knee depth

Spatial Very high

Water depth/distance from shore: 

knee depth versus chest depth

Spatial Very high

Windy versus calm conditions Temporal High

Turbulent versus non turbulent

conditions

Temporal High

Storm runoff versus none Temporal Low

Stratification of water due to

temperature, salinity, etc. 

Spatial (may be temporal) Low

High number of nearby

recreational water craft versus

low number or none

Spatial and temporal High
RTI/7249/417/2.4.3 Page 19 of 29 Final Report 9/24/99 OBSERVED STATUS

STEP 6: SPECIFY ALLOWABLE LIMITS ON DECISION ERRORS

Allowable Limits on Errors for Beach Decision

Addressing allowable limits on uncertainty for the beach decision will be done after getting the

data from this study as part of developing the guidance manual

The consequence of beach decision errors is manifested in the observation that the likelihood of

pathogen occurrence increases with higher levels of fecal indicators. Although gastroenteritis of

short duration is a common health result of sewage contamination, the possibility of contracting a

serious diseases such as hepatitis exists and must be a concern.

One approach to structuring the beach decision and associated decision errors is presented in the

following 3 x 3 matrix. The X axis of the matrix presents the “true” beach conditions at a given

time for a given beach site. That is, the level of risk to bathers if the indicator bacterial

concentrations could be perfectly measured. However, since all measurements contain error, the

true conditions can never be known, but only estimated based on the concentrations of indicator

bacteria measured at the beach site. These measurements allow a determination about the

observed status of the beach, which is represented by the Y axis of the matrix. The matrix

presents the possible decisions made based on the observed status and the resulting outcome

depending on the true beach status.

Red Major False

Alarm 

Err on Safe

Side

Good

Agreement

Yellow Minor False

Alarm 

Good

Agreement

Miss

Green Good

Agreement

Slight Miss Bad Miss

Green Yellow Red

TRUE BEACH STATUS

Research Issues for Guidance Decision

C The goal for the beach decision is to minimize the probability of misclassification, with

consideration of the costs of various kinds of misclassifications.

C The goal for the research study, upon which the guidance will be based, is to maximize the

precision of estimates of variance components.
RTI/7249/417/2.4.3 Page 20 of 29 Final Report 9/24/99

C Professional judgment can be used to set allowable Type I and II error rates (alpha and

beta) and the required minimum detectable difference (delta) for hypothesis tests

(formulated from a subset of research questions in step 5).

Examples of Research Hypotheses

Two examples of research hypotheses are presented below as possible models for formulating

others from the larger list of research questions in Step 5. Each example is presented as two

competing models for formulating hypothesis. The first, H0 vs. Ha reflects a common method of

stating a hypothesis in which there is no consideration for what level of effect or difference may

be regarded as negligible; this often leads to the sometimes erroneous conclusion that there is no

effect when, in fact, the only correct conclusion is that an effect could not be demonstrated. The

additional set of hypotheses, H0

'

 and Ha

’

, explicitly take into account the magnitude of difference

to be considered as unimportant, and lead to more defensible interpretations of the results.

Hypotheses for Testing the Effect of Bather Density on Concentration of Indicator

Bacteria

Ho: Bather density has no effect on the concentration of indicator bacteria in the beach

water.

Ha: Bather density increases the concentration of indicator bacteria in the beach water.

HNo: For each 100 bathers at the beach, the concentration of the indicator bacteria

increases by M colony forming units (CFUs) per 100 ml, where M = threshold of

importance.

HNa: For each 100 bathers at the beach, the concentration of the indicator bacteria

increases by less than M CFUs per 100 ml, where M = threshold of importance.

Hypotheses for Testing Effect of Distance from Shore on Concentration of Indicator

Bacteria

Ho: Distance from shore has no effect on the concentration of indicator bacteria in the

beach water.

Ha: The concentration of indicator bacteria in the beach water varies with the distance

from the shore.

HNo: The difference of the mean concentrations of indicator bacteria ([F]max ! [F]min)

between each of the distances from shore measured is greater than or equal to M

CFUs per 100 ml, where M = threshold of importance.

HNa: The difference of the mean concentrations of indicator bacteria ([F]max ! [F]min)

between each of the distances from shore measured is less than M CFUs per 100

ml, where M = threshold of importance.
5Anderson, S. and W. W. Hauck (1983). “A new procedure for testing equivalence in

comparative bioavailability and other clinical trials.” Communications in Statistical Theory and

Methods 12: 2663-2692.

6Anderson, S. and W. Hauck (1986). “A proposal for interpreting and reporting negative

studies.” Statistics in Medicine 5: 203-209.

7Parkhurst, D.F. (1998). “Logical Interpretation of Statistical Hypothesis Tests

(Abstract).” Ecological Society of America. 83rd Annual Meeting. Baltimore, MD. August 2-6,

1998. Abstracts, page 104.

8Parkhurst, D. (1990). “Statistical hypothesis tests and statistical power in pure and

applied science.” In Acting Under Uncertainty: Multidisciplinary Conceptions, GM von

Furstenberg, ed., Kluwer Academic Publishers, Boston pp. 181-201.

9Parkhurst, D. F. (1985). “Interpreting failure to reject a null hypothesis.” Bulletin of the

Ecological Society of America 66: 301-302.

RTI/7249/417/2.4.3 Page 21 of 29 Final Report 9/24/99

References related to this reverse testing procedure are provided below5,6,7,8,9.
RTI/7249/417/2.4.3 Page 22 of 29 Final Report 9/24/99

STEP 7: OPTIMIZE THE DESIGN

Prior Information

Prior information about developing a sampling design for recreational beach waters is available

and has been applied to the development of the design for this research project. This prior

information was obtained from the pilot study and a World Health Organization (WHO)

workshop on recreational waters held in November, 1998.

Data from the Pilot Study

Analysis of the pilot study results indicates that densities of indicator organisms can be

expected to vary greatly over time, within and between days, and between locations. 

However, there is little consistency to the variation, with no particular time of day or

location along the beach exhibiting consistently higher or lower concentrations than other

times or locations. The exception was distance from shore, where concentrations of the

indicator organisms were found to be generally higher in water closer to the shore. Of the

events and other covariates monitored in this pilot study, only rainfall was found to have a

significant effect, with indicator concentrations generally higher after rainfall. The scope

of the pilot study, however, was necessarily limited; the full scale study will be necessary

to determine any such effects with adequate precision.

The pilot data also provide estimates of components of variance due to transect, water

depth, time-of-day, and day-to-day differences in concentration. These will be utilized in

the main study design for determining numbers of sampling points and sampling

frequencies with respect to these various factors.

Results from the WHO Workshop on Sample Design

The WHO workshop proposed a sampling protocol for recreational beach waters. This

protocol, which was used for discussion of a possible sampling design, is depicted in

Figure 3. The design required a beach area at least 60 meters long with transect lines

spaced 20 meters apart, along which the samples would be collected. Three water depths

would be sampled: (1) 0.15 meters (ankle depth) where samples would be collected at

0.075 meters below the surface, (2) 0.5 meters (knee depth) where samples would be

collected at 0.3 meters below the surface, and (3) 1.3 meters (chest depth) where samples

would be collected at 0.3 meters below the surface.

Design Features of This Research Study

Various elements of the WHO design were discussed in relation to the design for this research

study. Many of these design elements were not decided upon and need to be finalized.
RTI/7249/417/2.4.3 Page 23 of 29 Final Report 9/24/99

Spatial Aspects

C The location of the transects within the beach area could be determined from a

random starting point as shown by X1 in Figure 3.

C Should samples be collected at the same water depths as in WHO design?

Temporal Aspects

The temporal aspect will have a systematic sampling component and an event-driven

component. The systematic component will consist of daily sampling and periods of

intensive sampling to detect short-scale variations over time. For example, the daily

systematic sampling could consist of sampling two or three times a day using a grid to

determine sampling locations. For the intensive systematic sampling, one week a month

could be selected at random. During this week, one weekend day and one week day could

be selected at random and samples collected every hour for ten hours (11 samples). The

specifics of the systematic design still need to be resolved, such as the number of days

selected for intensive sampling, the density of the grid for the routine daily sampling, and

use of discrete or composite sampling.

The event-driven sampling, which still needs to be developed, is intended to measure the

effects of events (such as rain [stormwater runoff], sewage system spills or failures,

discharge from ships) on the concentration of the indicator bacteria. The event-driven

sampling will be characterized by frequent sampling (e.g., every 1 to 2 hours) and greater

spatial density (especially with respect to sources of contamination such as stormwater

and sewer outfalls). 

Other issues regarding the design of the event driven sampling are:

C practical constraints on event-driven sample collection may be more extreme or

difficult,

C it may not be obvious when an event is occurring; therefore, the routine

monitoring should detect problem events and trigger more frequent sampling,

C there could be a time lag after the event before the indicator bacteria concentration

increases, and

C the goal is to determine when the beach should be closed, and subsequently can be

opened.

Another issue regarding event sampling is that a different decision rule may be needed

when certain events occur. For example, additional statistical testing for extreme values

may be necessary. Another possibility is that the distribution of indicator organisms could

change following an event, such as rain. A change in the form of the underlying

distribution could imply the need for a different sampling approach to characterize events.
10Crump, K. S. (1998). “On Summarizing Group Exposures in Risk Assessment: Is an

Arithmetic Mean or a Geometric Mean More Appropriate?” Risk analysis 18: 293--?

11Haas, C. N. (1996). “How to average microbial densities to characterize risk.” Water

Res. 30: 1036-1038.

12Parkhurst, D. F. (1998). “Arithmetic versus geometric means for environmental

concentration data.” Environmental Science and Technology 32: 92A-98A.

RTI/7249/417/2.4.3 Page 24 of 29 Final Report 9/24/99

Replication

For the research study, triplicate analysis of one sample will be performed in the laboratory

at the “normal” frequency. The use of double or triplicate sampling for field collection

needs to be determined as well as the frequency for replicate sample collection. Other

related issues are:

C There may be five different laboratories doing analyses; therefore, inter-laboratory

variability may be an issue.

C If the differences between sites are expected to be greater than differences within

sites, then fewer replicates should be taken at each site and relatively more taken

across the sites. Also, replicates should be spread out over time to capture

variability over time.

C There may be more variation between replicate samples than there will be between

sites.

Operational Decision Rule

C Geometric mean was used in setting the current standard and is the statistic used in the

current monitoring to compare to the standard.

C Different operational decision rules to be explored using EMPACT data.

Recent papers by Haas, Crump, and Parkhurst all suggest that use of the geometric mean should

be re-evaluated10,11,12.

General QA/QC

EPA’s general approach to documenting the QA/QC procedures for this study will be to prepare

an umbrella plan that covers general issues such as project management, assessment methods, and

training; these will apply to all five sites. Site-specific requirements, such as sample collection and

analysis, will be addressed in Standard Operating Procedures.
RTI/7249/417/2.4.3 Page 25 of 29 Final Report 9/24/99

 = Sample Locations

X1 = randomly selected distance

BEACH

WATER

20 meters 20 meters

X1

0.15 m

0.5 m

1.3 m

Depth of

Water

In Meters

At least 60 meters

Figure 3. Plan View of a Beach Showing Elements of the Sampling Design from the 

 WHO Report
RTI/7249/417/2.4.3 Page 26 of 29 Final Report 9/24/99

ATTACHMENT A

WORKSHOP PARTICIPANTS

PARTICIPANT

NAME AND AREA

OF EXPERTISE

CONTACT INFORMATION

EPA Participants

Kristen Brenner

Microbiologist

U.S. Environmental Protection Agency

National Exposure Research Laboratory

Microbiological and Chemical Exposure

Assessment Research Division (MD-314)

26 West Martin Luther King Drive

Cincinnati, OH 45268

Phone: (513) 569-7317

Fax: (513) 569-7170

E-Mail: brenner.kristen@epa.gov

Gary Collins

Biologist

U.S. Environmental Protection Agency

National Exposure Research Laboratory

Ecological Exposure Research Division (MD-642)

26 West Martin Luther King Drive

Cincinnati, OH 45268

Phone: (513) 569-7174

Fax: (513) 569-7438

E-Mail: collins.gary@epa.gov

Alfred Dufour

Microbiologist

U.S. Environmental Protection Agency

National Exposure Research Laboratory

Director, Microbiological and Chemical Exposure Assessment

Research Division (MD-593)

26 West Martin Luther King Drive

Cincinnati, OH 45268

Phone: (513) 569-7303

Fax: (513) 569-7464

E-Mail: dufour.alfred@epa.gov
PARTICIPANT

NAME AND AREA

OF EXPERTISE

CONTACT INFORMATION

RTI/7249/417/2.4.3 Page 27 of 29 Final Report 9/24/99

John Martinson

Quality Assurance

Officer

U.S. Environmental Protection Agency

National Exposure Research Laboratory

Microbiological and Chemical Exposure

Assessment Research Division (MD-587)

26 West Martin Luther King Drive

Cincinnati, OH 45268

Phone: (513) 569-7564

Fax: (513) 569-7424

E-Mail: martinson.john@epa.gov

Stephen Schaub

Microbiologist

U.S. Environmental Protection Agency

Office of Water

Office of Ground Water and Drinking Water

Engineering and Analysis Division (4303)

Waterside Mall Building

401 M Street, S.W.

Washington, DC 20460

Phone: (202) 260-7591

Fax: (202) 260-1036

E-Mail: schaub.stephen@epa.gov
PARTICIPANT

NAME AND AREA

OF EXPERTISE

CONTACT INFORMATION

RTI/7249/417/2.4.3 Page 28 of 29 Final Report 9/24/99

Larry Wymer

Statistician

U.S. Environmental Protection Agency

National Exposure Research Laboratory

Microbiological and Chemical Exposure

Assessment Research Division (MD-593)

26 West Martin Luther King Drive

Cincinnati, OH 45268

Phone: (513) 569-7252

Fax: (513) 569-7464

E-Mail: wymer.larry@epa.gov

Non-EPA Participants

Malcolm Bertoni

DQO

Advisor/Facilitator

Research Triangle Institute

Suite 740

1615 M Street, NW

Washington, DC 20036

Phone: (202) 728-2067

Fax: (202) 728-2095

E-Mail: mjb@rti.org

Michael Cross

Meeting and DQO

Support

Research Triangle Institute

Suite 740

1615 M Street, NW

Washington, DC 20036

Phone: (202) 728-2045

Fax: (202) 728-2095

E-Mail: myc@rti.org

Abdel El-Shaarawi 

Statistician

National Water Research Institute

867 Lakeshore Road

Burlington, Ontario, Canada

Phone: (905) 336-4584

Fax: (905) 336-4989

E-Mail: Abdel.El-Shaarawi@CCIW.ca
PARTICIPANT

NAME AND AREA

OF EXPERTISE

CONTACT INFORMATION

RTI/7249/417/2.4.3 Page 29 of 29 Final Report 9/24/99

Richard Gilbert

Statistician/DQOs

Battelle Washington Office

901 D Street, S.W.

Suite 900

Washington, DC 20850

Phone: (202) 646-5030

FAX: 

E-Mail: ro_gilbert@pnl.gov

Howard Kator

Microbiologist

c/o College of William & Mary

1208 Greate Road

Gloucester Point, VA 23062

Phone: (804) 684-7341

Fax: (804) 684-7186

E-Mail: kator@vims.edu

Molly Leecaster

Statistician

Southern California Coastal Research Program

7171 Fenwick Lane

Westminster, CA 92683-5218

Phone: (714) 894-2222

Fax: (714) 894-9699

E-Mail: mollyl@sccwrp.org

David Parkhurst

Statistician

SPEA - Indiana University

Bloomington, IN 47405

Phone: (812) 855-4556 (Voice)

Fax: (812) 855-7802

E-Mail: parkhurs@indiana.edu

William Robertson

Microbiologist

Microbiology Advisor, Environmental Health

Directorate, Health Canada

P.L. 1912A

Ottawa, Canada K1A0K9

Phone: (613) 957-1505

Fax: (613) 952-2574

E-Mail: will_robertson@hc-sc.gc.ca