Appendix A.5: Evaluation of the Suitability of Individual Combinations of 

Indicators and Methods for Different Clean Water Act Programs

COMPLETION NOTICE

Evaluate the suitability of individual combinations of indicators and methods for different 

CWA programs (P18)

Summary of the Study 

This study was conducted to evaluate the strengths and weaknesses of both rapid and culture 

indicator/method combinations to method performance criteria and their appropriateness for 

different Clean Water Act programs. EPA identified and prioritized key concerns related to 

indicator/method performance and identified eight performance criteria: 1) limit of detection; (2) 

sensitivity; (3) specificity; (4) precision; (5) percent false positives and false negatives; (6) 

ability to differentiate fecal sources; (7) performance in different waterbody types; and (8) 

indicator/illness health relationship established. The indicator/methods evaluated were methods 

that showed a correlation to health in the freshwater and/or marine water epidemiological studies 

conducted by EPA under the National Epidemiological and Environmental Assessment of 

Recreational (NEEAR) Water Study or used to support the development of the 1986 ambient 

water quality criteria. Four methods (Enterococcus quantitative polymerase chain reaction 

(qPCR), Bacteroidales qPCR, EPA Method 1600 and EPA Method 1603) were quantitatively 

evaluated with respect to the performance criteria. EPA evaluated the performance criteria based 

on published literature for the various indicator/method combinations. 

Additionally, qPCR and culture methods were evaluated qualitatively with respect to several 

method attributes/considerations for implementation purposes, and how well each method works 

for each Clean Water Act program. QPCR methods were evaluated relative to culture methods 

on the following evaluation criteria that were important to the CWA programs: (1) ease of use, 

(2) low cost, (3) time to results, (4) allowance for use of historical data for model development, 

(5) ability to demonstrate effectiveness of treatment from source to beach, (6) count/signal 

associated with human health risk pathogens, and (7) precision/accuracy. 

Summary of Findings 

The quantitative evaluation of performance criteria revealed: 

Enterococcus 

23S qPCR

(diluted crude 

extract)

Bacteroidales 

16S qPCR

(diluted crude 

extract)

Method 1600 Method 1603

50% Limit of 

detection in 

CCE[CSE]

65

[904]

99

[1,380]

Not reported Not reported

False positives 0%,19% 0% 6% 6%

False negatives 1.1-4.5%, 2% 1.1-4.5%, 2% 6.5% 5%

Specificity 100% 100% Not reported Not reported

United States Office of Water EPA 823-R-10-004

Environmental Protection (4303T) December 2010

Agency

Evaluation of the Suitability of Individual 

Combinations of Indicators and Methods 

for Different Clean Water Act Programs 
Office of Science and Technology

Office of Water

U.S. Environmental Protection Agency

Washington, DC
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

3

Table of Contents

List of Tables .................................................................................................................4

List of Acronyms and Abbreviations.............................................................................4

Section

1 Introduction/Project Description....................................................................................5

2 Background....................................................................................................................6

2.1 Indicator/Methods Evaluated in the 2003-2005 NEEAR Study..................7

2.2 Indicator/Methods Evaluated in the 2007/2009 Marine 

Epidemiological Studies ..............................................................................8

2.3 Other Indicator Methods..............................................................................9

3 Quantitative Evaluation of Indicator/Method Combinations Based 

on Performance Criteria...............................................................................................11

3.1 Established Health Relationship ................................................................12

3.2 Limit of Detection......................................................................................14

3.3 Precision.....................................................................................................14

3.4 Sensitivity ..................................................................................................15

3.5 Specificity ..................................................................................................15

3.6 Ability of Differentiate Fecal Sources.......................................................16

3.7 Performance in Different Waterbody Types..............................................16

4 Evaluation of Appropriateness of Indicator/Methods for CWA Purposes ..................17

5 Study Limitations.........................................................................................................18

6 References....................................................................................................................18
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

4

List of Tables

Table 1 Indicators/Methods Tested by EPA in NEEAR Epidemiological Studies...............9

Table 2 Evaluation of Rapid and Culture Methods ............................................................13

Table 3 Evaluation of Rapid and Culture Methods for Clean Water Act Programs 

Implementation Purposes.......................................................................................17

List of Acronyms & Abbreviations

BEACH Act Beaches Environmental Assessment and Coastal Health 

Act

CCE calibrator cell equivalents

CE cell equivalents

CFU colony forming unit

CLAT culture, latex agglutination, and typing 

CSE calibrator sequence equivalents

CT Cycle Threshold

CWA Clean Water Act

DNA Deoxyribonucleic acid

EPA Environmental Protection Agency

GI gastrointestinal 

Ml milliliter

MST Microbial Source Tracking

mTEC modified membrane-Thermotolerant Escherichia coli agar 

NEEAR National Epidemiological and Environmental Assessment 

of Recreational 

NPDES National Pollutant Discharge Elimination System

PBS Phosphate Buffered Saline

PCR Polymerase Chain Reaction

POTW Publicly-Owned Treatment Works

qPCR Quantitative Polymerase Chain Reaction

RNA Ribonucleic acid

RSD Relative Standard Deviation

SCCWRP Southern California Coastal Water Research Project

TMDL Total Maximum Daily Load

WQS Water Quality Standards
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

5

1.0 INTRODUCTION/ PROJECT DESCRIPTION

Project P18 as described in Environmental Protection Agency’s (EPA’s) Critical Path Science 

Plan is to “identify and evaluate indicator/method combinations for strengths and limitations 

with respect to fecal source identification (human, animal); and performance in different 

waterbody types. The important features of each indicator/method will be described and the 

strengths and weaknesses of those features will be explained and evaluated. The ideal set of 

features will be proposed so that the indicator/methods can be compared to a hypothetical ideal 

indicator/method. The indicators/methods under consideration will be ranked for each feature 

with respect to ability to differentiate fecal sources, performance in different water body types 

and appropriateness for different CWA purposes. EPA plans to use the results of this evaluation 

to inform the decisions regarding which indicator/methods will be included in the new or revised 

criteria and under what conditions those indicator/methods will be recommended.” The purpose 

of this white paper is to report on the result of the evaluation of indicator/methods assessed by 

EPA in support of the development of new or revised recreational water quality criteria for 

bacteria.

Section 304(a)(1) of the CWA directs EPA to publish criteria for water quality accurately 

reflecting the latest scientific knowledge. The criteria published by EPA under section 304(a) 

are intended to provide guidance to States in setting water quality standards (WQS) to protect 

public health as well as to maintain and restore water quality and ecosystem integrity. CWA 

304(a) criteria are typically expressed as numbers (i.e., concentrations of pollutants) or narratives 

that EPA recommends that states put into their WQS to protect waters for aquatic life, wildlife, 

consumption of aquatic organisms by humans, and primary contact recreation. CWA section 303 

requires each state to adopt WQS for all waters of the state and to review, and revise them as 

necessary, every 3 years. Once adopted and approved, WQS are binding CWA regulatory 

standards and effective for the following CWA purposes:

• Water Quality Assessments. Sections 303(d) and 305(b) provide that states are required 

to assess their waters on a regular basis to determine if they are meeting WQS. The 

states’ water quality criteria are an essential baseline against which states determine

whether particular waters are “impaired.”

• Total Maximum Daily Loads (TMDL). TMDL calculations are required for all waters 

that have been listed as "impaired" under section 303(d). A TMDL specifies the 

maximum amount of a pollutant that a waterbody can receive and still meet WQS, and 

"allocates" pollutant loadings among point and non-point pollutant sources. TMDL 

calculations for impaired waters must be written to implement the applicable State WQS. 

• National Pollutant Discharge Elimination System (NPDES). NPDES permits are 

required under section 402 for point source discharges of pollutants to waters of the 

United States. NPDES permits must include effluent limitations more stringent than 

required by technology regulations, if necessary, to meet water quality standards, which 

include state water quality criteria. 
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

6

• Non-point Source Program. Water quality standards (including criteria) play a similarly 

important role under the CWA section 319 non-point source program as part of the listing 

and TMDL processes to determine whether best management practices or other risk 

management control strategies are needed to address non-point source pollution. 

• Recreational Water Monitoring and Notification. A State’s recreational water quality 

criteria are used in beach monitoring and notification programs. States and beach 

managers typically make decisions about whether to issue advisories or closure notices 

by measuring the results of their monitoring against their WQS.

EPA’s current recommended criteria for bacteria are based on bacterial indicator organisms –

Escherichia coli (E. coli) and Enterococcus – which generally do not cause illness, but have 

characteristics that make them good indicators of fecal contamination, and thus by inference, of 

pathogens capable of causing human illness such as acute gastrointestinal illness. EPA’s 

recommended bacteria criteria are intended to be adopted by states into state water quality 

standards to protect waters designated for recreational use activities such as swimming.

2.0 BACKGROUND

Beginning with the freshwater National Epidemiological and Environmental Assessment of 

Recreational (NEEAR) Water Study in 2003, EPA has been evaluating recreational water quality 

using traditional culture methods and rapid genetic methods based on qPCR. EPA’s research 

goal is to determine the indicator/method combination with the best correlation to illness in fresh 

and marine recreational waters. EPA was also responding to a provision in the Beaches 

Assessment and Coastal Health Act (BEACH Act) of 2000 which required the Agency to 

provide additional information for use in developing “appropriate, accurate, and expeditious and 

cost-effective methods (including predictive models) for detecting in a timely manner in coastal 

recreation waters the presence of pathogens that are harmful to human health.”

A culture method for bacteria is one that allows direct propagation of the bacteria, and for the 

visible (to the naked eye) increase in the number of cells, which can allow for enumeration via 

solid (i.e., membrane filtration technique) or liquid media (i.e., multiple tube fermentation 

technique). The currently approved EPA culture methods involve culturing and enumerating 

fecal indicator bacteria (E. coli or Enterococcus spp.). Current culture-based methods require 24 

to 48 hours to obtain results (Wade et al., 2008).

Rapid methods allow for the direct measurement of genetic material such as DNA without the 

need for incubation and can provide results between 2 to 6 hours from time of receipt by the 

laboratory to time results are available. Rapid methods are being evaluated for the development 

of new or revised criteria because they provide more timely results than traditional culture 

methods, allowing beach managers to make beach notification decisions on the same day water 

samples are collected. 
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

7

2.1 Indicator/Methods Evaluated in the 2003-2005 NEEAR Study

The rapid indicator/methods evaluated in the 2003-2004 freshwater and 2005 (Biloxi, MS) 

marine NEEAR studies were commercially-available test methods or validated or experimental 

research protocols. The methods included in the NEEAR study met the following criteria: 

(1) results could be obtained within a few hours; 

(2) Enterococcus or the new potential indicator, Bacteroidales, was detected by the 

method; 

(3) the detection limit of the method was sufficiently low to allow detection of indicators 

in the majority of samples from recreational water environments;

(4) the method was sufficiently resistant to water sample inhibitory effects to allow 

detection of indicators in the majority of samples from recreational water 

environments.

The four tests included in the 2003-2005 freshwater and marine NEEAR Water Studies are 

described below. The following descriptions are based on the methods available at that time and 

may not reflect recent changes to these methods.

1. Quantitative Polymerase Chain Reaction (qPCR) Method, two new rapid gene probe 

methods developed by Dr. Richard Haugland of the EPA (Haugland et al., 2005)

(hereafter referred to as the EPA qPCR Assay), were used to detect Enterococcus and 

Bacteroidales in water samples based on the collection of these organisms on membrane 

filters, extraction of their total DNA, and qPCR amplification (i.e., a process whereby 

target DNA strands are doubled in each cycle of amplification) of a genus-specific 

(Enterococcus) or order–specific (Bacteroidales) rDNA sequence using the TaqManTM

system. 

2. RAPTOR Fiber optic Biosensor, a portable, automated fiber optic biosensor developed 

by Research International, Woodinville, Washington that can be used to detect 

microbiological and chemical analytes in water samples. The RAPTOR Biosensor, used 

in the summer of 2003, was removed from the study in 2004 because the capture and 

detection antibodies were not sensitive enough. 

3. Luminex 100 System, a compact flow cytometer, developed by the Luminex Corporation, 

Austin, Texas and MiraiBio, Alameda, California, that analyzes immunoassays, complex 

genetic analyses, and/or enzymatic assays through the use of optics, fluidics, and 

advanced signal processing. The Luminex method was removed from further study in 

2004 because it lacked adequate sensitivity.

4. EPA Method 1600 is the EPA-approved membrane filter method using mEI Agar for the 

detection of Enterococcus in recreational water. This method was included in the study 

as a reference method for Enterococcus.

Of the new methods described above, only the qPCR method produced results that were 

sufficient to be included for further evaluation in the 2007 marine epidemiological studies in 

Fairhope, Alabama and Goddard, Rhode Island. The antibodies used in the RAPTOR and 
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

8

Luminex 100 methods lacked sensitivity for Enterococcus and Bacteroidales but these methods 

may be useful if new antibodies can be produced to increase the sensitivity and specificity of the 

methods. Both the RAPTOR and Luminex methods were dropped from further evaluation.

In the 2003-2005 freshwater and marine studies E. coli methods were not tested. EPA did not 

include an E. coli qPCR method because at that time, a method was not available. Since no 

qPCR method was available for comparison with the culture method, the culture method was not 

included. 

2.2 Indicator/Methods Evaluated in the 2007/2009 Marine Epidemiological Studies

In 2007, EPA conducted two marine epidemiological studies in Fairhope, Alabama and Goddard, 

Rhode Island at beaches predominantly impacted by treated Publicly-Owned Treatment Works 

(POTW) effluent. In 2009, EPA initiated two additional epidemiological studies at a tropical 

beach impacted predominantly by human sources of fecal pollution, and a beach impacted by 

urban runoff using several rapid and cultural indicator/method combinations. The tropical 

epidemiological study was conducted at Boquerón Beach in Puerto Rico; and the urban runoff 

study at Surfside Beach in South Carolina. Additional information on the site selection process 

for the urban runoff epidemiological study can be found at http://www.epa.gov/nheerl/neear/.

Table 1 presents a comprehensive list of the methods tested in the EPA epidemiological studies 

conducted to support the development of new or revised recreational water quality criteria. 

Several methods were tested at multiple epidemiological sites dating back to the freshwater 

Great Lakes studies that were initiated in 2003 as part of the NEEAR Water Study. E. coli

culture was not included in the methods tested in marine waters because E. coli is not stable and 

cells lyse easily in saltwater environments.

The Fecal Bacteroides assay was evaluated because of its potential as a human associated 

marker. However, the evaluation of this indicator/method combination in the 2007 

epidemiological studies resulted in its elimination due to its lack of human source specificity and 

the relatively high proportion of sample where target organisms were not detectable. A similar 

problem was encountered with the E. coli qPCR assay. However, evaluation of this assay was 

continued in 2009 in conjunction with an added DNA concentration and purification procedure.

The male-specific F+ Coliphage methods were eliminated due to the low density of the indicator 

in surface water.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

9

1

Haughland et al., 2005 2

USEPA 2002

3

Dick and Field, 2004 4

Siefring et al., 2008 5

Converse et al., 2009 6 Shanks et al., 2009 7

Haugland et al., 2010. 8

Chern et al., 2009 9

Chern et al., nd 10 Love and Sobsey, 2007 11USEPA 2001

2.3 Other Indicator Methods

2.3.1 Microbial Source Tracking Methods

There is uncertainty about the risk to human health associated with non-human sources of fecal 

pollution. The ability to differentiate sources of fecal contamination site-specifically may be

important for an accurate assessment of the risks to human health from domestic, agricultural 

animals and wildlife. Source differentiation is also important with regard to CWA monitoring 

and assessment (§303[d] and §305[b]) and TMDL programs. Some states have expressed a 

desire to be able to adjust the applicable criteria based on data that the indicator levels present in 

the waterbody are not due to human sources of fecal pollution. The stated concern is that once 

waters are listed as impaired, states must then expend resources to develop a TMDL to restore 

waters that pose less of a risk to human health. In this context, microbial source tracking (MST) 

methods are useful to supplement sanitary survey investigations (i.e., to identify sources of 

contaminants or TMDL sources) and for risk analysis (human vs. non-human vs. domestic 

animals). Most MST methods attempt to identify specific fecal sources to assist with prioritizing 

polluted areas for restoration.

Table 1 – Indicators/ Methods Tested by EPA in NEEAR Epidemiological Studies

Methods EPA Epidemiological Studies Where Methods 

Were Used

Enterococcus spp. qPCR Freshwater: 2003, 20041

Marine: 2005, 2007, 20091

Enterococcus culture (EPA Method 1600) Freshwater: 2003, 20042

Marine: 2005, 2007, 20092

Total Bacteroidales spp. qPCR Freshwater: 2003, 20043

Marine: 2005, 2007, 20094

Fecal Bacteroides spp. qPCR Marine: 20075

Human-specific Bacteroidales markers Marine: 20096,7

E. coli qPCR Marine: 20078, 20098,9

Clostridium spp. qPCR Marine: 2007, 20098

F+ RNA Coliphage CLAT assay Marine: 200710

F+ DNA Coliphage CLAT assay Marine: 200710

F+ Coliphage 24-hr SPOT assay Marine: 200711
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

10

EPA has been conducting research on rapid MST methods and has evaluated qPCR and PCR 

genetic markers of human and bovine pollution. This research includes a performance evaluation 

of (1) seven PCR and qPCR assays targeting Bacteroidales genes reported to be associated with 

either ruminant (goat, sheep, deer and others) or bovine feces (Shanks et al., 2010a); and (2) five 

PCR and ten qPCR assays targeting Bacteroidales genes reported to be associated with human 

feces (Shanks et al., 2010b). The bovine assay study found large discrepancies in the 

performance of qPCR assays across different bovine populations; and recommended that the use 

of bovine-associated MST applications require an a priori characterization of each watershed due 

to variability in genetic marker abundance and prevalence between populations. Study results 

also suggest that some assays are more suitable for the characterization of fecal contamination 

than others, and that the assay of choice can vary from one bovine population to another. 

The human assay performance study included the evaluation of two qPCR assays for 

quantification of recently developed human-associated genetic markers targeting putative 

Bacteroidales-like cell surface-associated genes (Shanks 2009), seven qPCR assays for 

quantification of 16s rRNA gene markers from human-associated Bacteroidales species 

(Haugland et al., 2010), and several other published assays. Some assays showed human source 

specificity levels exceeding 97% when tested against a panel of reference fecal samples 

originating from cattle, poultry, swine, and various wildlife animal sources. Based on assay 

performance and the prevalence of DNA targets in a collection of reference untreated sewage 

samples collected from 54 different waste water treatment facilities in the United States, this 

research suggests a potential application for human-associated quantitative methods for 

monitoring fecal pollution in ambient waters.

EPA is also conducting research to identify genetic sequences that could form the basis of 

chicken and seagull-associated MST methods as specific fecal source assays. EPA has selected 

PCR-based assays that could uniquely identify avian sources (primarily chicken and seagull) of 

fecal pollution. Four assays are currently being evaluated and additional molecular data are being 

collected to further validate existing assays and determine if additional assays can be developed. 

If EPA determines that chicken and seagull specific assays can be developed, EPA will evaluate 

chicken and seagull-associated MST PCR-based assays for sensitivity and specificity using 

reference fecal samples and environmental water samples with known sources of fecal 

contamination.

2.3.2 Chemical Methods

In addition to the microbial indicators discussed in sections 2.2 and 2.3, EPA also tested water 

samples for chemical constituents. The chemical constituents tested included coprostanol, a 

product of cholesterol metabolism in feces; and urobilin, a bile pigment found in human feces 

and urine. Additionally, 48 different chemicals distinctly associated with humans that could be 

markers of human sewage or human fecal contamination (e.g., caffeine, cotininine, 

Acetaminophen, and codeine) were tested. Preliminary data suggest that chemical detections 

were not very frequent at freshwaters beaches, but were more frequent and occurred at higher 

concentrations at marine beaches tested in 2005 and 2007. Preliminary analyses did not show a 

consistent relationship with health outcomes, and the chemical measurements were discontinued 

in the 2009 epidemiological studies due to a lack of funding.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

11

2.3.3 Southern California Coastal Water Research Project Research

Southern California Coastal Water Research Project (SCCWRP) has been conducting research 

on indicator/methods with the goal of developing rapid methods that can augment or replace 

existing traditional culture methods for one or more types of indicator bacteria. SCCWRP is 

assessing water quality by measuring both traditional and non-traditional indicators. Traditional 

indicator methods include total coliform, fecal coliform, Enterococcus using membrane filtration

and Enterolert chromogenic substrate method, E. coli using Colilert chromogenic substrate 

method and membrane filtration, and Coliphage. Nontraditional measurements include rapid 

methods for quantifying Enterococcus and E. coli, Bacteroides, Bacteroides thetaiotamicron, 

adenovirus, norovirus, enterovirus and Coliphage (somatic and F+), among others. These 

indicator methods are being tested by SCCWRP as part of a 4-year project on Rapid Bacterial 

Indicator Development and then validated in three large-scale epidemiological studies conducted 

at Avalon Bay Beach, Malibu Surfrider Beach and Doheny State Beach in southern California.

As part of the epidemiological studies, SCCWRP has analyzed more than 4,000 water samples 

using 36 different analytical methods (SCCWRP, 2010). 

As part of its 2010/2011 Research Plan, SCCWRP is conducting a 3-year study to assess which 

source identification methods are optimal for differentiating fecal sources with the goal of 

bringing together a team of water quality experts experienced in source identification methods to 

create a source identification manual, implement selected protocols at several beaches of high 

interest to California, and then transition source identification capabilities to local laboratories. 

3.0 QUANTITATIVE EVALUATION OF INDICATOR/METHOD COMBINATIONS 

BASED ON PERFORMANCE CRITERIA

EPA’s approach to evaluate the suitability of individual combinations of indicators and methods 

for different CWA programs (P18) was to conduct two separate analyses – a quantitative analysis 

of indicator/method combinations based on a set of performance criteria; and a qualitative 

evaluation of the appropriateness of the qPCR and culture methods for each CWA program.

Enterococcus qPCR, Bacteroidales qPCR, Method 1600 (membrane filtration method for 

Enterococci,) and Method 1603 (membrane filtration method for E. coli using modified mTEC 

agar) were evaluated because they have an association with illness in swimmers (Fleisher et al., 

2010; Wade et al., 2008; Wade et al., 2006; and USEPA 1986) (Table 2). The performance

criteria used to evaluate these methods are as follows: (1) established health relationship, (2) 

limit of detection, (3) sensitivity, (4) specificity, (5) precision, (6) percent false positive and (7) 

percent false negative. Table 2 shows the results of the quantitative evaluation of the qPCR and 

culture methods. The methods were also evaluated on their ability to differentiate fecal sources 

and performance in different waterbody types.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

12

3.1 Established Health Relationship

3.1.1 Enterococcus Culture and qPCR

A demonstrated relationship between indicator concentration and gastrointestinal (GI) illness is 

the most important criterion for selection of the method for new or revised ambient water quality 

criteria. The method must demonstrate a positive relationship between illness in humans and 

with indicator levels (i.e., as indicator levels increase, risk of illness increases) in recreational 

waters.

Enterococci are commonly found in the feces of humans and other warm-blooded animals.

Although some strains are ubiquitous and not related to fecal pollution, the presence of

enterococci in water is an indication of fecal pollution and the possible presence of enteric

pathogens. Results from EPA’s recent epidemiological studies showed a correlation between 

qPCR measured Enterococcus levels to GI illness at freshwater beaches impacted by POTW 

sources (Wade et al., 2008). Additionally, the recent studies point to significantly increased 

illness rates among swimmers 10 years and younger exposed above 35 CFU of Enterococcus

using the culture method compared to non-swimmers in the Great Lakes study, but the trend is 

not as strong (or evident) at low exposures. QPCR cell equivalent (CE) levels were a stronger 

predictor of GI illness than the CFU measure in the NEEAR study (Wade et al., 2008). Other 

studies have shown enterococci levels measured by culture methods correlated with GI illness 

levels in marine and fresh waters (Au-Yeung et al., unpub; Zmirou et al., 2003). The metaanalysis of the Epibathe studies found an increased risk of gastroenteritis in both marine and 

freshwater water sites when bathers were exposed to enterococci concentrations higher than 100 

enterococci/100 ml in marine water and 200 enterococci/100 ml in fresh waters (Au-Yeung et 

al., unpub).

3.1.2 E. coli Culture and qPCR

Epidemiological studies conducted by EPA and SCCWRP separately found that there was no 

relationship between E. coli qPCR and GI illness in marine waters. There are no recent EPA 

data on the health relationship between E. coli culture and swimming-related illness. E. coli

culture was not included in the list of methods tested by EPA in the NEEAR epidemiological 

studies. However, in epidemiological studies conducted by EPA in the 1970s, there was an 

association between E. coli culture and GI illness in fresh water but not marine waters.

More recent data from randomized controlled trial epidemiological studies conducted in Europe

found a weak relationship between E. coli culture and GI illness in freshwater only. A metaanalysis of the (Epibathe) studies found an increased risk of gastroenteritis in both marine and 

fresh water sites in bathers exposed to E. coli concentrations higher than the level that represents 

the 2006/7/EC1 “excellent quality” criteria (< 500 E.coli/100 ml) (Au-Yeung et al., unpub).

 1 Directive 2006/7/EC of the European Parliament and of the Council of 15 February 2006 concerning the 

management of bathing water quality and repealing Directive 76/160/EEC. The purpose of this Directive is to 

preserve, protect and improve the quality of the environment and to protect human health by complementing 

Directive 2000/60/EC. This Directive lays down provisions for: (a) the monitoring and classification of 

bathing water quality; (b) the management of bathing water quality; and (c) the provision of information to 

the public on bathing water quality.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

13

Table 2 –Evaluation of Rapid and Culture Methods

Performance 

Evaluation Criteria 

Rapid Methods Culture Methods

Enterococcus 23S 

qPCR

Diluted crude extract

Bacteroidales 16S qPCR

Diluted crude extract

Method 1600

(Enterococcus by MF)

Method 1603

(E. coli by MF)

50% Limit of detection 

(CCE)

651 991 NR NR

50% Limit of detection 

(CSE)

9041 1,3801 − −

Sensitivity 100%2 NR NR NR

Specificity 100%3 100%4 NR NR

Precision (log 10 

standard deviation)

0.244 0.204 2.2% - 18.9%5 25.9%6

False positives 0%1, 19%7 0%1 6%5 6%6

False negatives 1.1-4.5%1, 2.0%7 1.1-4.5%1, 2.0%7 6.5%5 5%6

Health relationship 

established

Yes8,9 Yes (Marine)9 Yes Yes (Freshwater)

Notes:

Results presented in this table may be modified as additional information is collected and peer reviewed.

For Method 1600 (Enterococcus by MF) and Method 1603(E. coli by MF), performance criteria are based on the results of the inter-laboratory validation in 

disinfected wastewater matrices.

NR = Not reported.

 1 Chern et al., 2009 2 Ludwig and Schleifer, 2000

3 Frahm and Obst, 2003

4 Siefring et al., 2008 

5 USEPA, 2006a 6 USEPA 2006b

7 Haughland, et al., 2005 8 Wade et al., 2008 9 Wade et al., 2010
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

14

3.1.3 Bacteroidales qPCR

Bacteroidales is a group of anaerobic bacteria commonly found in the gut of humans. 

Bacteroidales densities, as measured by qPCR, are orders of magnitude higher in raw waste 

streams than enterococci or E. coli densities due in part to its greater abundance in feces. There 

are qPCR primers available that target the general Bacteriodales population and other primers 

that are thought to target human-specific strains. A positive relationship has been observed 

between general Bacteroidales and GI illness in marine waters in the NEEAR epidemiological 

studies (Wade et al., 2010). 

3.2 Limit of detection

3.2.1 qPCR

The limits of detection of the Enterococcus and Bacteroidales qPCR methods have been reported 

as the estimated number of target organism calibrator cell equivalents (CCE) or target organism 

calibrator sequence equivalents (CSE) that need to be present per total water sample filter extract 

in order to allow detection in 50% of analyses of that extract by the specified method (Chern et 

al., 2009). Chern et al. (2009) reported mean limits of detection of 65 CCE (904 CSE) for 

Enterococcus qPCR and 99 CCE (1,380 CSE) for Bacteroidales qPCR.

3.2.2 Culture

The limits of detection of the Enterococcus and E. coli culture methods (Method 1600 and 

Method 1603, respectively) have not been reported. 

3.3 Precision 

3.3.1 qPCR

Precision of a qPCR method is the estimate of total variability of Cycle Threshold (CT) 

measurements obtained in analyses of common water sample filter extracts by single or multiple 

laboratories. It can be expressed in terms of standard deviation, incorporating between lab, 

between run and random error estimates. 

At this time, EPA has completed a Single Laboratory Validation Study for Enterococcus qPCR 

and Bacteroidales qPCR. A Multi-Laboratory Validation Study is on-going; however, the results 

will not be available until 2012 and are therefore, not presented or discussed in this report.

Estimates of method precision (expressed as log10 standard deviations of CCE) from analyses of 

10,000 Enterococcus and Bacteroides cells spiked into 51 different fresh and marine water 

samples have been reported as 0.24 and 0.20, respectively (Siefring et al., 2008).

3.3.2 Culture

Precision for Method 1603 has been characterized by laboratory-specific relative standard 

deviations (RSDs) from disinfected wastewater samples spiked with laboratory-prepared. For 

Method 1603, the within-laboratory pooled RSD was 25.9% (USEPA 2006b). Precision 

estimates in the range of 2.2% - 18.9% has been reported for Method 1600 (USEPA 2006a).
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

15

3.4 Sensitivity 

Sensitivity can be expressed as a percentage of targeted species that are detected by a method. 

3.4.1 qPCR

It has been reported that the Enterococcus qPCR method detects all validly described species of

Enterococcus (Ludwig et al., 2000). For the Bacteroidales qPCR method, there are no publish 

data available on the sensitivity of the method.

3.4.2 Culture

For the Enterococcus and E. coli culture methods there are no published reports available on 

method sensitivity. However, as part of the formal QA process for each laboratory that uses 

Methods 1600 and 1603, the methods recommend that laboratories develop a statement of 

accuracy for method by calculating the average percent recovery and the standard deviation of 

the percent recovery (USEPA 2006a, b).

3.5 Specificity

Specificity is the ability of a method to select and or distinguish the target bacteria under test 

from other bacteria in the same water sample. The specificity characteristic of a method is 

usually reported as the percent of false positive and false negative results. In the following 

sections, data are provided on the specificity, false positive and false negative rates for 

Enterococcus qPCR, Bacteroidales qPCR, Method 1600, and Method 1603.

3.5.1 qPCR

The specificity of the Enterococcus qPCR method was estimated from experimental analyses of 

five closely related non-Enterococcus species showing at least 10,000 times higher limit of 

detection when compared to Enterococcus species (Frahm and Obst, 2003). For the 

Bacteroidales qPCR method specificity was experimentally assessed using representative species 

of the related bacterial classes, Flavobacteria and Sphingobacteria (Siefring et al., 2008).

The percent false negatives for the Enterococcus and Bacteroidales qPCR methods have been 

reported as the percentage of water sample filter extracts that fail salmon DNA sample 

processing control assay quality control criterion of > 3 CT units higher than the mean of 

associated calibrator samples in the analysis of NEEAR study samples (Haugland et al., 2005; 

Chern et al., 2009). Additionally, Griffith et al. (2007) reported false negative results of 58% for 

enteroccocci qPCR extracted using a multi-step purification and concentration process after bead 

beating and 29% for enterococci qPCR bead beaten method. The analysis for false negative rates 

was conducted relative to the State of California’s (AB411) standard of 104 cells/100 ml for 

enterococci (Griffith et al., 2007).

Percent false positives for the qPCR methods have been reported as the percentages of negative 

control samples, consisting of clean filters that were subjected to the entire method including 

DNA extraction and qPCR analysis, that gave positive detection of target sequences (i.e., a true 

logarithmic amplification trace) during analyses of NEEAR study samples (Haugland et al., 
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

16

2005; Chern et al., 2009). It should be noted that the mean CT value from the false positive 

analyses shown for the study cited in footnote 7 in Table 2 was 43.65 which would not be 

detected as positive in a more recent version of the method where only 40 thermal cycles are run. 

Additionally, Griffith et al. (2007) reported false positive results of 3% for enteroccocci qPCR 

extracted using a multi-step purification and concentration process after bead beating and 9% for 

enterococci qPCR bead beaten method. For E. coli qPCR, the false positive results were 0% 

before and 27% after adjustment for amplification efficiency. The analysis for false positive rates 

was conducted relative to the State of California’s (AB411) standard of 104 cells/100 ml for 

enterococci and 400 cells/100 ml for E. coli (Griffith et al., 2007).

3.5.2 Culture

For the culture methods, the percent false negative is the percentage of the samples that had a 

negative result but were actually positive. For Method 1600 and 1603, the percent false negatives 

are reported from analyses of various environmental water samples (USEPA 2000). Five percent 

of the E. coli colonies observed gave a false negative reaction (USEPA, 2006). In unspiked CSO 

samples, false negative rates of 37.5% and 1.7% have been reported for methods 1600 and 1603, 

respectively (USEPA, 2008). Additionally, Francy et al. (2000) reported false negative results of 

11% for the modified membrane-Thermotolerant Escherichia coli agar (mTEC) (Method 1603) 

when compared to the mTEC (EPA Method 1103.1). Griffith et al. (2007) reported false 

negative results of 10% for Method 1600 with respect to the AB411 standard of 104 

enterococci/100 ml.

For Method 1603, the percent false positives reported from analyses of various environmental 

water samples were <1% (USEPA, 2000) and averaged 6% for marine and fresh water samples 

(USEPA, 2006). False positive confirmation rates were 0% and 6.7% for Method 1600 and 1603, 

respectively (USEPA, 2008) were reported in unspiked CSO samples. Francy et al. (2000) 

reported false positive results of 0% for the modified mTEC (Method 1603) when compared to 

the mTEC (Method 1103.1). Griffith et al. (2007) reported false positive results of 4% for 

Method 1600 with respect to the AB411 standard of 104 enterococci/100 ml.

3.6 Ability to Differentiate Fecal Sources 

This criterion evaluates the ability of the indicator/method combination to identify the sources 

contributing to fecal pollution in a waterbody. None of the indicator/methods evaluated are able 

to differentiate between fecal contamination sources. However, current research by EPA on 

microbial source tracking assays for use in detecting bovine and human fecal pollution suggests 

that these assays maybe useful in monitoring fecal contamination in ambient waters (see Section 

2.4.1). 

3.7 Performance in Different Waterbody Types

This criterion evaluates how well the method performs (i.e., does it work or provide usable 

results) in different types of recreational waters (e.g., freshwater, marine water, temperate, 

tropical, and subtropical waters). Recent epidemiological studies conducted by EPA have shown 

that Enterococcus qPCR and Bacteroidales qPCR performed well in both marine and 

freshwaters.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

17

Results from earlier EPA epidemiological studies found that Enterococcus culture works well in 

both marine and freshwater, whereas E. coli only had good correlation (R2 ≥ 0.5) with GI illness 

in freshwater (Cabelli, 1983; Dufour, 1984).

4.0 EVALUATION OF APPROPRIATENESS OF INDICATOR/METHODS FOR 

CWA PURPOSES

The appropriateness of the method for the various CWA programs is important for state adoption 

and implementation of EPA’s new or revised criteria recommendations. To assess the potential 

use and applicability of new fecal indicators and methods to meet CWA program needs, EPA’s

Monitoring and Assessment, TMDLs, and NPDES programs evaluated the impact of new 

indicators/methods on their programs. Each program identified attributes necessary for indicatormethods to meet their programs’ CWA needs. These attributes are shown in Table 4 and ranked 

on whether they are important (Yes) or not important (No) for each CWA program.

Table 3 –Evaluation of Rapid and Culture Methods for

Clean Water Act Programs Implementation Purposes

Method Attributes Beach 

Program

NPDES TMDL Assessment

Low Cost Yes Yes Yes Yes

Ease of Use Yes Yes Yes Yes

Time to results (rapid/results within 4 hours of 

sample process/analysis)

Yes No No No

Allows for use of historical data for model 

development, etc.

Yes Yes Yes Yes

Demonstrates effectiveness of treatment from 

source to beach

No Yes No No

Count/signal associated with human health 

risk pathogens

Yes Yes Yes Yes

Precise/accurate Yes Yes Yes Yes

While beach advisory decisions and closures require same day results to protect public health, 

for the Assessment, TMDLs and the NPDES programs same day results are not necessary. 

Therefore, rapid methods are suggested as a key component of the criteria for the Beach Program 

only. Other important attributes for indicator/methods for the Beach Program include ease of use 

of the method and cost. Ease of use refers to how easily the method can be applied by those 

skilled in analysis. The cost associated with new indicator/method combination in criteria could 

include capital cost, training cost, per sample cost and additional sampling requirements. Capital 

costs include the upfront cost such as equipment purchase and space required to conduct the test. 

For example, when performing genetic testing, aside from the equipment needed (e.g., platform 

[i.e., specific machine]), laminar flow hoods, dedicated pipettes), space is needed, ideally in 

separate rooms, for reagent preparation (material not containing any genetic materials). Space is 

also needed for the two types of sample preparation, those containing high target sequence DNA 

concentrations such as DNA standards and calibrator samples, and those containing expected 

low target sequence DNA concentrations (e.g., filter blanks and water samples) – the latter of 

which should also be in separate laminar flow hoods (USEPA, 2007). 
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

18

Training costs are incurred prior to routine testing so that the user can perform the test within the 

performance criteria of the test (USEPA, 2007). Training cost would be dependent on the types 

of training available, i.e., whether workshop-type training with hands-on experience or 

completing a training module. 

Sample cost can vary and becomes an important cost consideration with a new method 

depending on the volume of tests needed to be completed on a routine basis. Additional 

sampling is generally an effort that results from rapid testing. For example, if an early morning 

sample yields a positive result resulting in beach closures, it may then lead to additional 

sampling to determine if the beach still needs to be closed in the mid-afternoon (USEPA, 2007). 

Therefore, depending on the number of designated bathing beaches that need to be monitored, 

the monitoring frequency, number of samples, and the need for additional sample analysis per 

beach, the sample costs associated with the use of rapid method could become cost prohibitive or 

result in beaches being sampled less frequently or not re-sampled to revise advisory or closure 

decision during the day. 

Ease of use and cost are also important factors for Assessment, TMDL and NPDES programs. 

For the TMDL and Assessment programs the ability to use prior historical data for modeling 

loadings is also important. Current historical data for E. coli, enterococci, and fecal coliform are 

derived from culture methods data. Criteria based on a rapid method without the necessary 

linkage between qPCR and the culture method would not allow for the use of historical data.

For the NPDES Program, the ability of the indicator/method to reflect the treatment efficacy

(e.g., chlorination) is crucial. Traditional culture methods for fecal indicator bacteria detect only 

a subset of the total viable population within any given water sample. The qPCR method will 

theoretically detect all intact cells in a water sample whether they are viable or not. 

Precision and accuracy of methods are also important to each program as well as the ability to 

associate indicator counts/concentrations to health risk. 

5.0 STUDY LIMITATIONS

There are several weaknesses in this evaluation with regard to how the project was defined in the 

Critical Path Science Plan. Due to limited data on various water body types, EPA is unable to 

evaluate method performance in different water body types based on the performance criteria in 

Table 2. Additionally, method performance with respect to fecal source identification (i.e., 

human vs. animal) could not be evaluated because none of the current indicator/method 

combinations can distinguish between sources.

6.0 REFERENCES

Au-Yeung, H.C. , D. Rh. Thomas, and R.L. Salmon. Nd. Appendix 2 Section 2 Risk of 

gastroenteritis from recreational bathing water: A meta-analysis of randomized exposure trials in 

four European countries. From http://www.ies.aber.ac.uk/en/epibathe. Accessed on July 2009.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

19

Cabelli, V.J. 1983. Health Effects Criteria for Marine Recreational Waters. EPA 600/1-80-031, 

U.S. Environmental Protection Agency, Cincinnati, OH.

Chern E.C., K. Brenner, L. Wymer, and R.A. Haugland. 2009. Comparison of Fecal Indicator 

Bacteria Densities in Marine Recreational Waters by QPCR. Water Qual. Expo. Health 1(3), 

203-214.

Chern E.C., S.C. Siefring, J. Paar, M. Doolittle, and R.A. Haugland. Comparison of quantitative 

PCR assays for Escherichia coli targeting ribosomal RNA and single copy genes. Lett. Appl. 

Microbiol. Manuscript Submitted.

Converse R.R., A.D. Blackwood, M. Kirs, J.F. Griffith, and R.T. Noble. 2009. Rapid QPCRbased assay for fecal Bacteroides spp. as a tool for assessing fecal contamination in recreational 

waters. Water Res 43:4828–4837. From http://dx.doi.org/10.1016/j.watres.2009.06.036.

Dick L.K. and K.G. Field. 2004. Rapid estimation of numbers of fecal Bacteroidetes by use of a 

quantitative PCR assay for 16SrRNA genes. Appl Environ Microbiol 70(9):5695–5697.

DuFour, A.P. 1984. Health Effects Criteria for Marine Recreational Waters. EPA 600/1-84-004. 

U.S. Environmental Protection Agency, Cincinnati, OH. 

Fleisher, J.M., L.E. Fleming, H.M. Solo-Gabriele, J.K. Kish, C.D. Sinigalliano et al. 2010. The 

Beaches Study: health effects and exposures from non-point source microbial contaminants in 

subtropical recreational marine waters. Int. J. Epidemiol. 1-8. From http://ije.oxfordjournals.org.

Frahm, E. and U. Obst. 2003 Application of the fluorogenic probe technique (TaqMan PCR) to 

the detection of Enterococcus spp. and Escherichia coli in water samples. Journal 

Microbiological Methods 52, 123-131.

Francy, D.S. and R.A. Darner. Comparison of methods for determining Esherichia coli

concentrations in recreational water. 2000. Water Res., 34:2770-2778.

Griffith, J.F., Y. Cao, C. D. McGee, S. B. Weisberg. 2009. Evaluation of rapid methods and 

novel indicators for assessing microbiological beach water quality. Water Research, 43: 4900-

4907.

Griffith, J.F., D. Moore, C. McGee, and S.B. Weisberg. 2007. Beta Testing of Rapid Methods for 

Measuring Beach Water Quality. Southern California Coast Water Research Project Technical 

Report 506. February.

Haugland, R.A., Siefring, S.C., Wymer, L.J., Brenner, K.P. and Dufour, A.P. 2005. 

Comparison of Enterococcus density measurements by quantitative polymerase chain reaction 

and membrane filter culture analysis at two freshwater recreational beaches. Water Research, 

39:559-568.

Haugland, R.A., M. Varma, M. Sivaganesan, C. Kelty, L. Peed, and O.C. Shanks. 2010. 

Evaluation of genetic markers from the 16S rRNA gene V2 region for use in quantitative 

detection of selected Bacteroidales species and human fecal waste by real time PCR. Syst. Appl. 

Microbiol. 33:348-357.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

20

Love D.C., and M.D. Sobsey M.D. Simple and rapid F+ Coliphage culture, latex agglutination, 

and typing assay to detect and source track fecal contamination. Appl. Environ. Microbiol. 

73:4110–8.

Ludwig, W. and K.-H. Schleifer. (2000) How quantitative is quantitative PCR with respect to 

cell counts? Systematic Applied Microbiology 23: 556-562.

Shanks, O.C., C. Kelty, M. Sivaganesan, M. Varma, and R.A. Haugland. 2009. Quantitative PCR 

for genetic markers of human fecal pollution. Applied and Environmental Microbiology, 

75:5507-5513

Shanks, O.C., K. White, C.A. Kelty, S. Hayes, M. Sivaganesan, M. Jenkins, M. Varma, and 

R.A. Haugland. 2010a. Performance assessment PCR-based assays targeting Bacteroidales

genetic markers of bovine fecal pollution. Appl. Environ Microbiol., Mar;76(5):1359-66.

Shanks, O.C., White, K., Kelty, C.A., Sivaganesan, M., Blannon, J., Meckes, M., Varma, M., 

Haugland, R.A. 2010b. Performance of PCR-based assays targeting Bacteroidales genetic 

markers of human fecal pollution in sewage and fecal samples. Environ Sci Technol. 44:6281-8.

Siefring, S., M. Varma, E. Atikovic, L. Wymer and R.A. Haugland. 2008. Improved real-time 

PCR assays for the detection of fecal indicator bacteria in surface waters with different 

instrument and reagent systems. J. Water and Health 6:225-237.

Southern California Coastal Water Research Project (SCCWRP). FY 2010/2011 Research Plan –

Commissioned Approved. June 2010.

USEPA. 1986. Ambient Water Quality Criteria for Bacteria – 1986. EPA 440/5-84-002. United 

States Environmental Protection Agency, Office of Water, Washington, DC.

USEPA. 2000. Improved Enumeration Methods for the Recreational Water Quality Indicators: 

Enterococci and Escherichia coli. EPA-821/R-97/004. U.S. Environmental Protection Agency, 

Office of Water (4303T), Washington, DC.

USEPA. 2001. Method 1601: Male-specific (F+) and somatic Coliphage in water by twostep enrichment procedure. Technical Report EPA 821-R-01-030. U.S. Environmental 

Protection Agency, Office of Water, Washington, DC.

USEPA. 2002. Method 1600: Enterococci in water by membrane filtration using membraneEnterococcus indoxyl-D-glucoside agar (mEI), EPA 821/R-02/022. U.S. Environmental 

Protection Agency, Office of Water, Washington, DC.

USEPA. 2006a. Method 1600: Enterococci in water by membrane filtration using membraneEnterococcus indoxyl-D-glucoside agar (mEI), EPA 821-R-06-009. U.S. Environmental 

Protection Agency, Office of Water, Washington, DC.
Evaluation of the Suitability of Individual Combinations of Indicators and Methods 

for Different Clean Water Act Programs

21

USEPA. 2006b. Method 1603: Escherichia coli (E. coli) in Water by Membrane Filtration Using 

Modified membrane-Thermotolerant Escherichia coli Agar (Modified mTEC). EPA-821-R-06-

011. U.S. Environmental Protection Agency, Office of Water, Washington, DC.

USEPA. 2007. Guidelines establishing test procedures for the analysis of pollutants; Analytical 

methods for biological pollutants in wastewater and sewage sludge: Final rule. Federal Register, 

Vol. 72, No. 57, March 26, 2007.

USEPA. 2008. Results of the inter-laboratory testing study for the comparison of methods for 

detection and enumeration of Enterococci and E. coli in combined sewer overflows. EPA 821-

R-08-006. U.S. Environmental Protection Agency, Office of Water, Washington, DC.

Wade, T.J., N. Pai, J.N.S. Eisenberg, and J.M. Colford. 2003. Do U.S. Environmental Protection 

Agency water quality guidelines for recreational waters prevent gastrointestinal illness? A 

systematic review and meta-analysis. Environmental Health Perspectives 111(8): 1102-1109.

Wade, T.J., R. L. Calderon, E. Sams, M. Beach, K.P. Brenner, A. H. Williams, and A P. Dufour. 

2006. Rapidly Measured Indicators of Recreational Water Quality Are Predictive

of Swimming-Associated Gastrointestinal Illness. Environmental Health Perspectives, 

114(1):24-28. 

Wade, T.J., R.L. Calderon, KP Brenner, E. Sams, M. Beach, R. Haugland, L. Wymer, and A. P. 

Dufour. 2008. High Sensitivity of Children to Swimming-Associated Gastrointestinal Illness 

Results Using a Rapid Assay of Recreational Water Quality. Epidemiology, 19: 375-383.

Wade, T.J., E. Sams , KP Brenner , R. Haugland , E. Chern, et al. 2010. Rapidly measured 

indicators of recreational water quality and swimming-associated illness at marine beaches: a 

prospective cohort study. Environmental Health, 9:66. Available online at 

http://www.ehjournal.net/content/9/1/66.

Zmirou, D., Pena, L., Ledrans, M., Letertre, A. 2003. Risks associated with the microbiological 

quality of bodies of fresh and marine water used for recreational purposes: summary estimates 

based on published epidemiological studies. Archives of Environmental Health 58(11): 703-711