Meeting Report for 

The Peer Review of EPA’s Draft Recreational 

Water Quality Criteria (RWQC) Document 

Submitted to: 

Office of Water 

U.S. Environmental Protection Agency (EPA) 

Washington, DC 20460 

Submitted by: 

Eastern Research Group, Inc. 

110 Hartwell Avenue 

Lexington, MA 02421 

Final Meeting Report: November 1, 2011 
Notice 

This report was prepared by Eastern Research Group, Inc. (ERG), a U.S. Environmental Protection Agency 

(EPA) contractor, as a record of discussion during the Peer Review Meeting on EPA’s draft document

Recreational Water Quality Criteria (RWQC) for the Protection of Human Health, held September 12-13, 

2011, in Arlington, Virginia. The peer review summary report was prepared by the peer reviewers, with 

ERG support, on the second day of the peer review meeting. This section captures the main points and 

highlights of reviewers’ discussions and recommendations. The appendices include the technical charge, 

peer reviewers post-meeting comments organized by charge question, their individual comments including 

biographies, the meeting agenda, and the list of reviewers and observers at the meeting. Statements 

represent the individual views of meeting participants. 
i 

Contents 

Introduction ................................................................................................................................... 1

Background ..........................................................................................................................................1

Peer Review Process .............................................................................................................................2

Opening Remarks .................................................................................................................................3

Meeting Discussion Process ..................................................................................................................3

Peer Reviewers’ Summary Report ................................................................................................ 5

1. Does the document adequately present the logic and scientific support behind its conclusions? 

Please comment on the document’s transparency, the completeness of the presentations, and 

any shortfalls in the material presented. Are uncertainties for technical conclusions 

transparent? Do you have any suggestions on how we could improve this document? Please 

explain your responses. .................................................................................................................. 5

2. Are you aware of any important peer reviewed literature or applicable data that have not been 

incorporated in this document and should be included? ................................................................... 8

3. Do the analyses comparing the NEEAR freshwater and marine culture-based indicator data 

aggregated by days of similar water quality corroborate that the NEEAR culture data are 

generally consistent with the 1986 regression models?.................................................................. 10

4. Do the characterization and explanation of the qPCR method reflect that of the scientific 

literature? If not, what additional information should be included? Is the derivation of the 

qPCR value using the EPA data scientifically supported and adequately explained? Please 

discuss why or why not. ............................................................................................................... 12

5. Please provide any other general or specific comments on the document, making sure to 

identify the section number/page number you are commenting on. ............................................... 12

Appendix A - Technical Charge to External Peer Reviewers ................................................... A-1

Appendix B - Reviewers’ Post-Meeting Comments Organized by Charge Question ............. B-1

Appendix C - Individual Reviewers’ Post-Meeting Comments ............................................... C-1

Samuel Dorevitch ............................................................................................................................. C-3

Erin K. Lipp .................................................................................................................................... C-19

Graham McBride ............................................................................................................................ C-27

Rachel Noble .................................................................................................................................. C-35

Gordon Yasvinski ........................................................................................................................... C-47

Appendix D - Meeting Agenda .................................................................................................. D-1

Appendix E - List of Reviewers and Observers ....................................................................... E-1

Introduction 

1 

Introduction 

EPA developed the draft Recreational Water Quality Criteria (RWQC) for the Protection of Human Health, 

laying out the underlying science and associated technical information related to RWQC and how scientific 

findings were used during the development of the RWQC. The water quality methods associated with the 

RWQC are also discussed. In addition, EPA’s policy decisions related to RWQC as of July 4, 2011, are 

presented throughout the document. Eastern Research Group, Inc. (ERG), under contract to EPA, conducted 

an independent external peer review and meeting of the draft criteria. 

Background 

An important goal of the Clean Water Act (CWA) is to protect and restore waters for swimming. Section 

304(a) of the Act directs EPA to publish “advisory water quality guidance on the effects of the presence of 

pollutants in water on health and welfare”. These recommendations are referred to as Section 304(a) criteria. 

Under Section 304(a), EPA is required to issue recommended criteria and update them from time to time. 

Under Section 304(a)(9) of the CWA, EPA is required to update or revise the current recommended 304(a) 

criteria related to pathogens and pathogen indicators to protect swimmers from illnesses associated with 

pathogenic microbes in coastal and Great Lakes water bodies. 

In adopting new or revised water quality standards, States must adopt criteria that are scientifically 

defensible and protective of the use, but they have flexibility to do so by adopting EPA’s recommended 

criteria, adopting criteria to reflect site-specific conditions, or adopting other criteria that are scientifically 

defensible. 

Once adopted into State standards, water quality criteria express the desired ambient condition of the water to 

protect a designated use. State water quality standards are used for various CWA purposes or programs that 

identify and address the sources of pollution with the goal of attainment of the criteria, including National 

Pollutant Discharge Elimination System (NPDES) permits, water body assessments to determine use 

attainment, and development of Total Maximum Daily Loads (TMDLs). Water quality standards are also 

used by States in beach monitoring and notification programs. 

Historically, the federal government’s recommended criteria for protecting people who recreate in water 

have been based on fecal matter in recreational waters. In the 1960’s, the Public Health Service 

recommended using the indicator bacteria, fecal coliforms, as the primary contact recreational1 water quality 

criterion. In the late 1970s and early 1980s, EPA conducted public health studies evaluating several 

organisms as possible indicators, including fecal coliforms, E. coli, and enterococci. The studies showed that 

Enterococcus was a good predictor of gastrointestinal illnesses in fresh and marine recreational waters, and 

E. coli was a good predictor of gastrointestinal illnesses in fresh waters. As a result, EPA published revised 

criteria in 1986 (EPA’s Ambient Water Quality Criteria for Bacteria – 1986) for primary contact recreation. 

The 1986 criteria recommended the use of E. coli for freshwaters (criteria set as a geometric mean of 126 

colony forming units [cfu]/100 milliliters [mL]) and enterococci for fresh and marine waters (criteria set as 

geometric means of 33 cfu/100mL in freshwater and 35 cfu/100mL in marine water). These 

recommendations replaced EPA’s previously recommended bacteria criteria for fecal coliform of 200 

cfu/100mL. Most strains of E. coli and Enterococcus do not cause human illness (that is, they are not human

pathogens); rather, in many settings they are thought to indicate fecal contamination, and the assumption is 

that pathogens co-occur with incidences of fecal contamination. 

 

1

 Primary contact recreation - EPA’s §304(a) RWQC recommendations are national recommendations for all surface 

waters of the United States designated for swimming, bathing, surfing, or similar water contact activities. 
Introduction 

2 

Since EPA issued its recreational criteria over 20 years ago, there have been significant scientific advances, 

particularly in the areas of molecular biology, microbiology, analytical chemistry, the application of 

predictive models, and improved sanitary survey techniques. EPA believes that these new scientific and 

technical advances need to be factored into the development of new or revised 304(a) criteria for recreation. 

To this end, EPA has been conducting research and assessing relevant scientific and technical information to 

provide the scientific foundation for the development of new or revised criteria. The enactment of the 

Beaches Environmental Assessment and Coastal Health (BEACH) Act of 2000 (which amended the CWA) 

required EPA to conduct new studies and issue new or revised criteria, specifically for Great Lakes and 

coastal marine waters. 

On March 21, 2007 and April 7, 2008, the United States District Court of Central California ruled that EPA 

failed to perform its mandatory duties to complete the required studies under Section 104(v) of the CWA, 33 

U.S.C § 1254(v), by October 10, 2003, and to publish water quality criteria under CWA § 304(a)(9)(A), 33 

U.S.C. § 1314(a)(9)(A), by October 10, 2005. A consent decree was issued and it was agreed by both the 

plaintiff and the defendant, EPA, that new or revised water quality criteria for pathogens and pathogen 

indicators pursuant to 33 U.S.C. § 1314(a)(9)(A) would be signed for publication in the Federal Register by 

October 15, 2012. Also included in the terms of the consent decree was that epidemiological studies be 

completed in marine waters impacted by urban runoff in a temperate region and in a tropical region. At the 

same time, EPA also signed a settlement agreement with the plaintiffs, agreeing to conduct additional 

research and science studies. For full documentation on how EPA met the terms of the settlement agreement 

and consent decree visit EPA’s Recreational Water Quality Criteria website. http://water.epa.gov/scitech/ 

swguidance/standards/criteria/health/recreation/index.cfm

Peer Review Process 

In summer/fall of 2011, Eastern Research Group, Inc. (ERG), an EPA contractor, organized an independent 

peer review of EPA’s draft document, “Recreational Water Quality Criteria (RWQC) for the Protection of 

Human Health” to assess its scientific soundness and utility. Five experts were secured as peer reviewers 

(Appendix E) to conduct this review: 

• Samuel Dorevitch, University of Illinois at Chicago, IL 

• Erin Lipp, University of Georgia, GA 

• Graham McBride, National Institute of Water & Atmospheric Research Ltd., New Zealand 

• Rachel Noble, University of North Carolina, Chapel Hill, NC 

• Gordon Yasvinski, Health Canada, Canada 

ERG provided the reviewers with the draft RWQC document, a charge (Appendix A), which asked for 

reviewer comments on the various aspects of the document, and a CD with all reference materials used in 

developing the criteria. 

In the first stage of the review, the experts worked individually to prepare individual written pre-meeting 

comments to address the charge questions. To kick off this stage of the review, ERG organized and 

facilitated a conference call to brief reviewers and offer them the opportunity to obtain clarifications from 

EPA on the peer review materials. In addition, ERG asked EPA to present background on the context for 

developing the criteria document and provide the context for the review and the charge questions. Reviewers 

then worked individually to prepare and submit their pre-meeting comments addressing the charge questions, 

which ERG then provided to all reviewers and EPA prior to the meeting. 

In the second stage, ERG convened a two-day peer review meeting, on September 12-13, 2011, in Arlington, 

Virginia. Appendix D provides the meeting agenda and Appendix E provides the list of meeting attendees. 
Introduction 

3 

On the second day of the meeting, ERG worked with reviewers to prepare a draft peer reviewer summary 

report highlighting the key issues and recommendations from their comments and discussions. After the 

meeting, reviewers were asked to provide post-meeting comments, amending their pre-meeting comments, as 

appropriate, in light of their comments and discussions at the meeting. Reviewer post-meeting comments are 

provided in Appendices B and C. 

This report summarizes the peer review as follows: 

• ERG and EPA Opening Remarks 

• Peer Reviewers’ Summary Report 

• Appendix A: Charge to Peer Reviewers 

• Appendix B: Reviewers’ Post-Meeting Comments Organized by Charge Question 

• Appendix C: Reviewers’ Individual Post-Meeting Comments 

• Appendix D: Meeting Agenda 

• Appendix E: List of Reviewers and Observers 

Opening Remarks 

Kate Schalk (ERG), the meeting facilitator, opened the meeting by welcoming the reviewers and asking the 

reviewers and EPA representatives to introduce themselves. Ms. Schalk reviewed the meeting agenda 

(Appendix D). She noted that the pre-meeting comments were developed by reviewers working individually 

prior to the meeting, and that reviewers may express different opinions during the meeting. Ms. Schalk 

briefly reviewed the ground rules for the meeting, making clear that all discussions were to be among 

reviewers only; however, reviewers could request, and EPA could offer, clarifications where relevant. 

Ms. Schalk then asked Beth Doyle, Branch Chief, Human Health Risk Assessment Branch, Health and 

Ecological Criteria Division (HECD), EPA Office of Science and Technology, Office of Water, to offer 

some brief welcoming remarks. Also present from EPA were Shamima Akhter, John Ravenscroft, and 

Sharon Nappier from HECD. Jeff Soller (Soller Environmental), a consultant working with EPA to develop 

the criteria document also attended. And present via teleconference from the EPA Office of Research and 

Development (ORD) were Tim Wade, Alfred Dufour, Kevin Oshima, and Robert Cantilli. Appendix E 

provides a list of meeting attendees. 

Meeting Discussion Process 

Ms. Schalk then turned the meeting over to Rachel Noble, the technical review panel chair, to begin the 

reviewer discussions. Dr. Noble stated that the discussion would proceed following the order of the charge 

questions, and that each reviewer would in turn discuss his/her highest priority issue related to that question.

For example, for question 1, reviewer 1 would provide his/her highest priority response to question 1, with 

enough background information to open the topic for discussion. Then, in turn, each reviewer would add to, 

debate, or highlight other important components to that specific topic. When that topic was completely 

discussed to the satisfaction of the reviewers, reviewer 2 would provide his/her highest priority issue in 

response to question 1, and so forth. The discussions proceeded until all charge questions were addressed, all 

points of view presented and discussed, and additional information or comments that lay outside of the 

technical charge were presented. As needed, reviewers asked EPA for clarifications if discussions could not 

move forward without further information or clarification. At the close of all discussions addressing the 

charge and other issues identified by the reviewers, Ms. Schalk adjourned the formal meeting. After the 

formal meeting was adjourned, ERG worked with the peer reviewers to develop the peer reviewers’ 

summary report. 
Introduction 

4 
Peer Reviewers’ Summary Report 

5 

Peer Reviewers’ Summary Report 

Introduction 

The peer reviewers acknowledged the monumental amount of work that EPA has completed to critically 

examine and synthesize the vast amount of water quality related science. They commended EPA’s efforts to 

develop new or revised criteria, and agreed that there is scientific support for many aspects of the proposed 

criteria. The reviewers provided the following comments to assist EPA strengthen the draft Recreational 

Water Quality Criteria. 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for 

technical conclusions transparent? Do you have any suggestions on how we could 

improve this document? Please explain your responses. 

General 

1.1. Add an executive summary to the criteria document that contains a "road map". Emphasize what is 

staying the same from the 1986 criteria and what is changing. 

1.2. It would be useful to the reader for EPA to explain the components of a criterion. Provide basic 

definitions/explanations. In doing so, avoid using “criteria” as the microbe being measured and the 

density of the microbe not to be exceeded. Have a glossary to define terms and define the term when 

first used in the document. 

1.3. Stress that fecal indicator bacteria (FIB) are health risk indicators, not necessarily pathogen indicators. 

The later portions of the criteria document focused on the link between FIB and health, so there is little 

need for emphasis in the earlier portion of the document on an FIB pathogen relationship. Statements 

regarding FIB pathogen relationship in the water body type section might be better placed in the 

quantitative microbial risk assessment (QMRA) section. 

1.4. Release the topic specific guidance documents at the same time as the RWQC. If these won’t be 

available, there should be some indication of the basic contents of the topic specific guidance (TSG) 

documents. 

1.5. Write the document from the perspective of the end user, who is more interested in implementation 

than in the process/history by which the criteria were developed. Rather than writing in a retrospective 

framework, be prospective about the findings and science. 

1.6. Add text to the Introduction indicating that water quality monitoring and notification programs are 

critical but are not the only elements to protecting public health. Water quality monitoring is one 

component in a beach management strategy. 

Application of Criteria to Inland Waters 

1.7. Reorganize the structure of the "Waterbody Type" section (3.4) to emphasize public health protection 

first. De-emphasize the suggested similarity of coastal and inland waters. As written, the RWQC 

suggests that the variability in FIB (or FIB-pathogen associations) in inland versus coastal water 

distinctions is smaller than the variability across coastal sites. This discussion is most appropriate in 

Section 5.2.2 under QMRA. 
Peer Reviewers’ Summary Report 

6 

1.8. The reviewers endorsed extending coastal water criteria to inland waters, based upon protection of 

public health. However, they recommend that EPA acknowledge the paucity of health data upon which 

to base the extension of coastal derived indicator criteria to inland waters. The information presented 

on this topic in the RWQC should consider the findings presented in the Inland Waters Workshop, 

including the conclusion of workshop experts that coastal-derived criteria should be protective of 

public health in inland waters impacted by publicly owned treatment works (POTWs). Proceed with 

using coastal water criteria in the interest of protecting public health based on the best available 

information available at this time. 

Tropical and Subtropical Waters 

1.9. Be transparent about the lack of usable data from the National Epidemiological and Environmental 

Assessment of Recreational Water (NEEAR) studies in tropical waters. 

1.10. Cite findings from the Tropical Indicator Workshop. Mention different season lengths that could 

impact sampling requirements. 

1.11. Have a separate section addressing specific issues facing tropical (and subtropical) waters. It would be 

appropriate to include studies from Hawaii on this issue. 

1.12. States that currently implement more stringent criteria should be encouraged to continue doing so. 

Given the lack of useable data from NEEAR tropical sites, EPA should partner with tropical and 

subtropical states to develop new indicators, particularly rapid ones, as potential criteria. This is 

important given the high level of beach usage in these states. 

Quiescent and High Energy Beaches 

1.13. Acknowledge that all NEEAR studies, as well as the Mission Bay study, have been done in enclosed 

beach or embayments, except for Surfside Beach, SC. The proposed criteria should be applicable at 

high energy beaches but the differences should be noted. 

1.14. Acknowledge the possibility that had NEEAR studies been conducted at high energy beaches, 

respiratory illness may have been more frequent. 

1.15. Work to ensure that Southern California Coastal Water Research Project (SCCWRP) studies are 

included in the RWQC because they include good information about high energy beaches. 

1.16. EPA may wish to explain its rationale for selecting beaches for NEEAR studies. Such an explanation - 

the requirement of a range of FIB levels and POTW proximity - may explain the absence of highenergy marine sites. 

Single Sample Maximum (SSM) vs. Statistical Threshold Value (STV) 

1.17. Needs to be more clearly presented. Highlight what has changed and what is staying the same. 

1.18. Justify the assumption of log normal distribution of FIB data (for both culture-based and quantitative 

polymerase chain reaction [qPCR] data) as related to calculation of STV. 

1.19. The discussion needs to be more clearly laid out for the loss of tiers. Address the impact resulting from 

the loss of tiers. 

1.20. The uses of geometric mean (GM) versus STV for beach notification vs. other CWA purposes need to 

be clarified. 
Peer Reviewers’ Summary Report 

7 

1.21. The change in sampling frequency for the GM from at least 5 per 30 days should translate to at least 15 

per 90 days, rather than 12 per 90 days. 

1.22. Specify acceptance of other water quality assessment approaches, such as the frequency of beach 

closure approach used by the State of California, dependent upon increased stringency. 

1.23. Provide guidance on sampling frameworks that promote accurate representation of beaches over wide 

ranging conditions (e.g., wet weather vs. dry weather). 

1.24. There was general agreement that the concept of STV – which a substantial percent of single samples 

would be expected to exceed the GM – is more appropriate than the term “single sample maximum”, 

which may be misconstrued to mean FIB densities that should never be violated. 

Minimum Sampling Requirements (Wet Weather) 

1.25. A poorly designed sampling framework can result in under-protection of public health and inaccurate 

303(d) listing. Strive for a common set of minimum sampling requirements across states for all CWA 

purposes. In other words, sampling requirements informed by site-specific variability in FIB values 

would be useful, though more challenging from an implementation standpoint. 

1.26. States should be directed to sample at locations and during periods of high use, such as weekends, 

regardless of antecedent weather conditions. Minimum spatial and temporal sampling frequencies 

should be specific enough so that it would not be possible for water samples to be collected (for GM 

calculation) selectively on days expected to have low FIB values (i.e., we do not want beach managers 

to avoid sampling the day or two following rainfall). 

E. coli 

1.27. Be transparent about the lack of NEEAR E. coli qPCR study data. 

1.28. Acknowledge the strength of health risk as related to E. coli by culture concentrations in fresh water 

epidemiology studies. 

1.29. More fully address the options for rapid method implementation for E. coli. 

1.30. Increase consistency of messages about the use of E. coli as an effective indicator in fresh waters. 

qPCR 

1.31. Increase the amount of information regarding strength of relationship between qPCR for Enterococcus

and health outcomes at NEEAR fresh water beaches. Include visual representation. 

1.32. Failure to propose mandatory qPCR criteria at Great Lakes beaches is surprising given the results of 

the NEEAR studies. Because prior day culture measures are poor predictors of illness, explain why 

culture based methods are being put forward as the first choice and qPCR as an option. 

1.33. If the proposed RWQC are not revised to mandate qPCR monitoring at Great Lakes beaches, 

justification for failing to promote the better and faster method is necessary. 

1.34. Reviewers like the inclusion of the indicator bacteria and methods section (Section 3.1), but the written 

material needs to be more detailed and in relation to culture-based methods versus rapid methods needs 

to be more balanced. Inhibition information is too detailed and presented too early in the document. 

Reverse the order of Sections 3.1 and 3.2. 
Peer Reviewers’ Summary Report 

8 

QMRA 

1.35. Explain “scientifically defensible” as related to QMRA in developing site-specific criteria. 

1.36. Acknowledge that EPA did not deliver a side-by-side QMRA with the NEEAR Boquerón study. 

1.37. Without a QMRA/epidemiology validation study, the accuracy of QMRA estimates remains to be 

determined. 

1.38. Need to justify QMRA as an alternative to epidemiology studies in developing site specific criteria 

rather than a water body management tool. QMRA is a good management tool, encourage states to use 

it as a way to evaluate options for water management, but the option to use QMRA to develop criteria 

should be backed up by validation studies. 

1.39. EPA has an opportunity to take the lead in QMRA science, particularly in the development of accurate 

quantification approaches for pathogens. 

1.40. Need to provide a conceptual framework figure and examples/case studies. 

1.41. Increase breadth of potential “tools” to include resuspension and regrowth studies, as presented in 

Section 5.2. 

1.42. Technical support documents will be critical for appropriate use of specific QMRA modeling 

approaches and inputs. 

1.43. Text about sources of fecal contamination as related to QMRA is missing. 

2. Are you aware of any important peer reviewed literature or applicable data that have not 

been incorporated in this document and should be included? 

Reviewers recommended adding the following references to the RWQC: 

2.1. WHO/EPA Annapolis protocol (WHO 1999, WHO 2003) 

• Explain why not being used, but note that extension to 90 days for GM calculation takes these 

criteria close toward that protocol's beach grading component. Also discuss consideration of a 

"traffic light" notification approach. 

2.2. Kay et al. (2004)—including discussion (Wymer et al., 2005) and reply (Kay et al., 2006) 

• Explain reasons for not adopting the "UK ecologic exposure model" for "guideline" development. 

2.3. Dufour et al. (2006) 

• Children more at risk than adults. 

2.4. Heaney (2009) 

• NEEAR study swimmers illness may be influenced by sand exposure. 

2.5. Pratap et al. (2011) 

• Need for effective communication also needed for disease prevention. 
Peer Reviewers’ Summary Report 

9 

2.6. Bambic et al. (2011) 

• Bacteroidales measured by qPCR as alternate indicator and fecal source identifier were highly 

correlated with Enteroccoccus and E. coli when either traditional (cultivation dependent) or qPCR 

methods were used. 

• Norovirus GI and GII were reasonable predictors of other pathogens like Adenovirus 40/41, 

Adenovirus C, Giardia and Cryptosporidium measured by qPCR. 

2.7. Fewtrell et al. (1992), Harrington et al. (1992) 

• Health effects (respiratory) of white-water canoeing and surfing. 

2.8. Sinton et al. (2002), Noble et al. (2004) 

• Differing interpretations on relative microbial inactivation rates in freshwater cf. coastal waters. 

2.9. French et al. (2011) 

• Wild bird feces (via Campylobacter) demonstrated to be less virulent to humans, cf. feces from 

farmed animals. 

2.10. SCCWRP epidemiology studies/Colford et al., submitted 

• SCCWRP results to be used as soon as available. 

2.11. Lavender & Kinzelman (2009) 

• Implementation of already available rapid methods, especially equivalence between culture based 

and rapid methods for E. coli. 

2.12. Viau et al. (2011) 

• Example of application of QMRA in tropical waters. 

2.13. Dorevitch et al. (2011) (Added to the list during the October review of the final draft.) 

• Epidemiologic study of limited contact water recreation at inland waters. 

References 

Bambic, D.; McBride, G.; Miller, W.; Stott, R.; Wuertz, S. (2011). Final Report: Quantification of Pathogens 

and Sources of Microbial Indicators for QMRA in Recreational waters. Report by University of 

California (Davis), AMEC Earth and Environmental and NIWA to WERF (USA Water Environment 

Research Foundation) for the project PATH2R08. 375 p. 

Dorevitch S, Pratap P, Wroblewski M, Hrohorczuk DO, Li H, Liu LC, Scheff PA: Health risks of limited 

contact water recreation. Environmental Health Perspectives, 2011, in press. 

Fewtrell, L.; Godfree, A.F.; Jones, F.; Kay, D.; Salmon, R.L.; Wyer, M.D. (1992). Health effects of whitewater canoeing. The Lancet 339: 1587–1589. Their results for Site B refer. 

French, N.P.; Marshall, J.; Mohan, V. (2011). Campylobacter in food and the environment: new and 

emerging data on typing of Campylobacter strains in animals, environmental matrices and humans. 

Final report 07-10436, Molecular Epidemiology and Public Health Laboratory, Massey University, 

Palmerston North, New Zealand. Prepared for the New Zealand Food Safety Authority and Ministry 

for the Environment. http://www.foodsafety.govt.nz/elibrary/industry/examining-link-with-publichealth/new-and-emerging-data-on-typing-of-campylobacter.pdf. 
Peer Reviewers’ Summary Report 

10 

Harrington, J.F.; Wilcox, D.N.; Giles, P.S.; Ashbolt, N.J.; Evans, J.C.; Kirton, H.C. (1993). The health of 

Sydney surfers: an epidemiological study. Water Science and Technology 27(3–4): 175–181. 

Heaney, C.D.; Sams, E.; Wing, S.; Marshall, S.; Brenner, K.; Dufour, A.P.; and Wade, T.J. (2009). Contact 

With Beach Sand Among Beachgoers and Risk of Illness. American Journal of Epidemiology 70(2): 

164–172. 

Kay, D.; Bartram, J.; Prüss, A.; Ashbolt, N.; Wyer, M.D.; Fleisher, J.M.; Fewtrell, L.; Rogers, A.; Rees, G.

(2004). Derivation of numerical values for the World Health Organization guidelines for recreational 

waters. Water Research 38: 1296–1304. 

Kay, D.; Ashbolt, N.; Wyer, M.D.; Fleisher, J.M.; Fewtrell, L.; Rogers, A.; Rees, G. (2006). Reply to 

comments on "Derivation of numerical values for the World Health Organization guidelines for 

recreational waters". Water Research 40: 1921–1925. 

Lavender, J.S.; Kinzelman, J.L. (2009). A cross comparison of QPCR to agar-based or defined substrate 

methods for determination of Escherichia coli and enterococci in municipal water quality monitoring 

programs. Water Research 43(19):4967–4979. 

Noble, R.T.; Lee, I.M.; Schiff, K.C. (2004). Inactivation of indicator micro-organisms from various sources 

of fecal contamination in seawater and freshwater. Journal of Applied Microbiology 96: 464–472 

Pratap, P.L.; Desai, P.; and Dorevitch S. (2011). Beach communications: a need for evaluation of current 

approaches. Journal of Water and Health 9(3): 556-568. 

Sinton, L.W.; Hall, C.H.; Lynch, P.A.; Davies-Colley, R.J. (2002). Sunlight inactivation of fecal indicator 

bacteria and bacteriophages from waste stabilization pond effluent in fresh and saline water. Applied 

and Environmental Microbiology 68(3): 1122–1131. 

Viau, E.J.; Lee, D.; Boehm, A.B. (2011). Swimmer Risk of Gastrointestinal Illness from Exposure to 

Tropical Coastal Waters Impacted by Terrestrial Dry-Weather Runoff. Environmental Science & 

Technology. dx.doi.org/10.1021/es200984b 

WHO (1999). Health-based monitoring of recreational waters: the feasibility of a new approach (the 

“Annapolis Protocol”). World Health Organization, Geneva (http://www.who.int/water_sanitation_ 

health/bathing/Annapolis.pdf) 

WHO (2003). Guidelines for safe recreational waters volume 1 - Coastal and fresh waters. World Health 

Organization, Geneva (http://www.who.int/water_sanitation_health/bathing/srwe1/en/). 

Wymer, L.J.; Dufour, A.P.; Calderon, R.L.; Wade, T.J.; Beach, M. (2005). Comment on "Derivation of 

numerical values for the World Health Organization guidelines for recreational waters". Water 

Research 39: 2774–2777. 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based indicator 

data aggregated by days of similar water quality corroborate that the NEEAR culture 

data are generally consistent with the 1986 regression models? 

3.1. Much needed clarity was provided during the peer review for comparing freshwater and marine 

culture data to 1986 data. Since the purpose is to establish the targeted level of risk used in 

developing the 1986 criteria to qPCR methods, the RWQC should be clarified/reorganized. Change 

the title of Section 3.2.4 to “Establishing a comparable target level of risk for defining culture and 

qPCR criteria values”. 
Peer Reviewers’ Summary Report 

11 

3.2. Peer reviewers found that some aspects of the section, especially Figures 1a and 1b, undermine 

EPA’s effort to arrive at a targeted level of risk that applies to both PCR and culture measures of 

water quality. Specifically, the information currently in the document suggests an inverse/negative 

relationship between culture measures of indicator bacteria and health risk when analyzed using the 

seasonal aggregation approach. 

3.3. EPA could acknowledge that the raw data used to establish the 1986 criteria are no longer available 

for analysis. For that reason the two data sets (1986 and NEEAR) cannot both be analyzed using 

NEEAR methods. This limits EPA’s ability to identify qPCR values associated with the same 

targeted level of risk defined by specific CFU values. 

3.4. Given the unavailability of data used to develop the 1986 criteria EPA faced two options: 1) to 

analyze the NEEAR culture data using the 1986 approach and compare it to the 1986 data analysis 

or 2) analyze culture data only from the NEEAR study using the same methods published by Wade 

et al. for the qPCR data. 

• Results of Option 1 are presented in the RWQC. The limitations of that analytic method were 

not clearly acknowledged. Specifically, these analyses do not include individual level 

covariates. Given that a variety of exposures, such as digging in sand, are associated with 

gastrointestinal (GI) illness, and that these are not accounted for in the analysis, confounding is 

a significant possibility. 

• ORD staff clarified that EPA has conducted the option 2 analyses, but has not reported them. 

Those analyses reportedly show weak relationships between culture measures of indicators and 

GI illness. Those results should be presented in the RWQC. 

3.5. Since the ORD analysis (Option 2) shows a weak association between culture measures of FIB and 

health outcome, EPA must acknowledge that any targeted level of risk selected is inherently 

imprecise (i.e., it is difficult to establish with confidence the FIB CFU or qPCR calibrator cell 

equivalent [CCE] values associated with a narrowly-defined targeted risk level). Figures presenting 

results of those analyses (analogous to Figure 1 of Wade 2008 and Figure 2 of Wade 2010) should 

be included in the document. 

3.6. Since a clear threshold between safe and unsafe levels of FIB is not apparent in the data, EPA may 

want to consider other graded approaches (e.g., red, yellow, green) for beach notification purposes. 

3.7. Since the criteria values of 33 and 35 CFU/100 ml are not changing, justification of these numbers 

is less important than if EPA had proposed alternate values. In Wade 2008 (Table 6) and Wade 

2010 (Table 6) data are presented that support these culture-based criteria values, including tables in 

the RWQC would bolster EPA’s decision to retain those thresholds. 

3.8. In order to establish threshold values of qPCR Enterococcus, EPA could define expected case of 

illness per 1,000 swimmers at 33 and 35 CFU using NEEAR culture data analyzed the same way as 

NEEAR qPCR data. Then apply those targets to establish qPCR thresholds. 

3.9. Figures 2 and 3 should present confidence intervals instead of prediction intervals. Figure 2 of 

Wade 2010 and Figure 1 of Wade 2008 present graphs of Enterococcus data in marine and 

freshwater, respectively. It would be useful to include them in the RWQC document. 
Peer Reviewers’ Summary Report 

12 

4. Do the characterization and explanation of the qPCR method reflect that of the scientific 

literature? If not, what additional information should be included? Is the derivation of 

the qPCR value using the EPA data scientifically supported and adequately explained? 

Please discuss why or why not. 

4.1. Be more transparent about the strength of the relationship between NEEAR-GI (NGI) risk and 

qPCR and health risk measures in NEEAR freshwater studies as compared to culture methods. 

4.2. Investigate possible incorporation of SCCWRP study results for marine waters. 

4.3. Even though qPCR in the RWQC is not specified as a “criteria”, EPA could specifically note that 

associations between qPCR and illness are stronger than those between culture methods and illness 

in freshwater. 

4.4. Present graphical representation of qPCR data in relation to illness for freshwater and marine beaches. 

4.5. Acknowledge the similarity between cell equivalent (CE) and CCE quantification approaches in 

predicting health outcomes. 

4.6. Acknowledge relationships observed with Bacteroidales qPCR. Specify potential use of 

Bacteroidales qPCR for future. 

4.7. Describe range of quantification approaches for qPCR briefly. Place CCE in the context of available 

quantification approaches. 

4.8. Since EPA did not mandate use of qPCR in the draft, EPA could promote use of qPCR by making 

the method simple, user friendly, and as rapid as possible.

5. Please provide any other general or specific comments on the document, making sure to 

identify the section number/page number you are commenting on. 

5.1. Be more transparent about the fact that Virtual Beach does not work well in marine waters. 

5.2. Move appropriate information from Section 3.4 about indicator pathogen relationship, sources of 

contamination, and differences in risk as a function of contamination source to the QMRA 

discussion in Section 5. 

5.3. Section 5.2.3 needs to give examples of novel microorganisms. 

5.4. The process for evaluating new methods needs to be more clearly defined. Lines 2026-2029 should 

suggest that alternate methods be at least as protective rather than generating the same beach 

notification result. 

5.5. Suggested additional TSG 

• Sampling guidance for compliance purposes 

 Minimum requirements 

 Consideration of composite samples 

 Spatial sampling frequency 

 Temporal sampling frequency 

 Sampling depth 

 How does wet weather influence sampling requirements 

 Location in relation to activity/use 

 Time of day 
A-1 

Appendix A 

Technical Charge to External Peer Reviewers 

Technical Charge to Peer Reviewers 

A-3 

Technical Charge to External Peer Reviewers 

Contract No. EP-C-07-059 

Task Order No. 3-11 

August 18, 2011 

Peer Review of the Recreational Water Quality Criteria (RWQC) 

Goal: The Recreational Water Quality Criteria (RWQC) meet the requirements of the BEACH Act, to 

maintain water quality pursuant of the CWA 304(a), protect public health, provide means for timely beach 

notification in advance of exposure by allowing rapid methods and predictive models, and to introduce new 

tools for application to water quality protection. 

Purpose: The RWQC in this peer review package provides EPA’s proposed recommended CWA §304(a) 

Recreational Water Quality Criteria (RWQC) for States, Tribes, and Territories. In this document, EPA lays 

out the underlying science and associated technical information related to RWQC and how scientific 

findings were used during the development of the RWQC. The water quality methods associated with the 

RWQC are also discussed. In addition, EPA’s policy decisions related to RWQC as of July 4, 2011, are 

presented throughout the document. 

The criteria document is organized as follows: 

• Section 1 provides introductory and background information. 

• Section 2 explains the scope of the 2012 RWQC. 

• Section 3 describes the scientific aspects that were considered during the development of the 2012 

RWQC, such as sources of fecal contamination, types of bodies of water, human health endpoints, 

target populations, indicators of fecal contamination, and the expression of the 2012 RWQC. 

• Section 4 presents the numerical values used for developing the 2012 RWQC. 

• Section 5 describes the tools that can be used to assess and manage recreational waters, derive sitespecific criteria, and provide flexibility to States. The tools listed in Section 5 will not only provide 

States with flexibility in their adoption of the 2012 RWQC, but will also help States gain a better 

understanding of their surrounding watersheds. 

A series of appendices will be sent to you on a CD: 

• Appendix A provides additional information on indicators and enumeration methods. 

• Appendix B describes data and information used to evaluate the linking of water quality and health. 

• Appendix C contains information on sources of fecal contamination. 

• Appendix D provides additional information on flexibility tools. 

A CD will be provided to you with the final draft document, the appendices, reference documents, and the 

technical charge to reviewers. 

Background: An important goal of the CWA is to protect and restore waters for swimming. Section 304(a) 

of the Act directs EPA to publish “advisory water quality guidance on the effects of the presence of 

pollutants in water on health and welfare”. These recommendations are referred to as Section 304(a) criteria. 

Under Section 304(a), EPA is required to issue recommended criteria and update them from time to time. 

Under Section 304(a)(9) of the CWA, EPA is required to update or revise one of the current recommended 

304(a) criteria related to pathogens and pathogen indicators to protect swimmers from illnesses associated 

with pathogenic microbes in coastal and Great Lakes water bodies. 
Technical Charge to Peer Reviewers 

A-4 

In adopting new or revised water quality standards, States must adopt criteria that are scientifically 

defensible and protective of the use, but they have flexibility to do so by adopting EPA’s recommended 

criteria, adopting criteria to reflect site-specific conditions, or adopting other criteria that are scientifically 

defensible. 

Once adopted into State standards, water quality criteria express the desired ambient condition of the water 

to protect a designated use. State water quality standards are used for various CWA purposes or programs 

that identify and address the sources of pollution with the goal of attainment of the criteria, including 

National Pollutant Discharge Elimination System (NPDES) permits, water body assessments to determine 

use attainment, and development of Total Maximum Daily Loads (TMDLs). In addition, these water quality 

standards are used by States in beach monitoring and notification programs. 

Historically, the federal government’s recommended criteria for protecting people who recreate in water 

have been based on fecal matter in recreational waters. In the 1960’s, the Public Health Service 

recommended using the indicator bacteria, fecal coliforms, as the primary contact recreational1 criterion. In 

the late 1970s and early 1980s, EPA conducted public health studies evaluating several organisms as 

possible indicators, including fecal coliforms, E. coli, and enterococci. The studies showed that 

Enterococcus is a good predictor of gastrointestinal illnesses in fresh and marine recreational waters, and E. 

coli is a good predictor of gastrointestinal illnesses in fresh waters. As a result, EPA published in 1986 

revised criteria (EPA’s Ambient Water Quality Criteria for Bacteria – 1986) for primary contact recreation 

recommending the use of E. coli for fresh recreational waters (criteria set as a geometric mean of 126 

cfu/100mL) and enterococci for fresh and marine recreation waters (criteria set as geometric means of 

33/100mL in freshwater and 35 cfu/100mL in marine water). These recommendations replaced EPA’s 

previously recommended bacteria criteria for fecal coliform of 200 cfu/100mL. Most strains of E. coli and 

all enterococci do not cause human illness (that is, they are not human pathogens); rather, they merely 

indicate fecal contamination, and the assumption is that pathogens co-occur with incidences of fecal 

contamination. 

Since EPA issued its recreational criteria over 20 years ago, there have been significant scientific advances, 

particularly in the areas of molecular biology, microbiology, analytical chemistry, the application of 

predictive models, and improved sanitary survey techniques. EPA believes that these new scientific and 

technical advances need to be factored into the development of new or revised 304(a) criteria for recreation. 

To this end, EPA has been conducting research and assessing relevant scientific and technical information to 

provide the scientific foundation for the development of new or revised criteria. The enactment of the 

BEACH Act of 2000 (which amended the CWA) required EPA to conduct new studies and issue new or 

revised criteria, specifically for Great Lakes and coastal marine waters. 

On March 21, 2007 and April 7, 2008, the United States District Court of Central California ruled that EPA 

failed to perform its mandatory duties to complete the required studies under Section 104(v) of the CWA, 33 

U.S.C § 1254(v), by October 10, 2003, and to publish water quality criteria under CWA § 304(a)(9)(A), 33 

U.S.C. § 1314(a)(9)(A), by October 10, 2005. A consent decree was issued and it was agreed by both the 

plaintiff and the defendant, EPA, that new or revised water quality criteria for pathogens and pathogen 

indicators pursuant to 33 U.S.C. § 1314(a)(9)(A) would be signed for publication in the Federal Register by 

October 15, 2012. Also included in the terms of the consent decree was that epidemiological studies be 

completed in marine waters impacted by urban runoff in a temperate region and in a tropical region. At the 

same time, EPA also signed a settlement agreement with the plaintiffs, agreeing to conduct additional 

research and science studies. For full documentation on how EPA met the terms of the settlement agreement 

 

1

 Primary contact recreation includes activities that could be expected to result in ingestion of water or immersion. 

These activities include swimming, water skiing, surfing, and other activities where contact and immersion in water 

is likely. 
Technical Charge to Peer Reviewers 

A-5 

and consent decree visit EPA’s Recreational: Water Quality Criteria website. http://water.epa.gov/scitech/ 

swguidance/standards/criteria/health/recreation/index.cfm ERG, under contract to EPA, is conducting this 

peer review under work assignment 3-11. ERG requests the reviewers to read the draft Recreational Water 

Quality Criteria (RWQC) document and refer to the extensive Appendices and reference materials as 

needed. Please respond to each charge questions below and provide any other feedback or recommendations 

that will improve the overall soundness of the science and the development of the criteria. 

When writing your comments, please cut and paste each question into your document, and write your 

response under each question and subquestion. Please put your name in a header and paginate your review 

comments. See the end of the charge for further guidance on the preparation and submission of your 

comments. 

Charge Questions for the Criteria Document

 

1. Does the document adequately present the logic and scientific support behind its conclusions? 

Please comment on the document’s transparency, the completeness of the presentations, and any 

shortfalls in the material presented. Are uncertainties for technical conclusions transparent? Do you 

have any suggestions on how we could improve this document? Please explain your responses. 

2. Are you aware of any important peer reviewed literature or applicable data that have not been 

incorporated in this document and should be included? 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based indicator data 

aggregated by days of similar water quality corroborate that the NEEAR culture data are generally 

consistent with the 1986 regression models? 

4. Do the characterization and explanation of the qPCR method reflect that of the scientific literature? 

If not, what additional information should be included? Is the derivation of the qPCR value using 

the EPA data scientifically supported and adequately explained? Please discuss why or why not. 

5. Please provide any other general or specific comments on the document, making sure to identify the 

section number/page number you are commenting on. 

B-1 

Appendix B 

Reviewers’ Post-Meeting Comments 

Organized by Charge Question 

Reviewers’ Comments by Charge Question 

B-3 

Introduction to Comments 

Reviewer Comments 

Dorevitch EPA scientists and policy-makers, as well as those that assisted in the development of the 

draft RWQC are to be commended. The task they faced was monumental, in large part 

because of the tremendous amount of information about microbial measures of water quality 

and associated health risks that EPA has developed in the past 10 years. The National 

Environmental and Epidemiological Assessment of water Recreation (NEEAR) was a very 

sophisticated study that presented numerous implementation and analytic challenges. 

Furthermore, EPA researchers developed, refined, and published details of molecular methods 

that allow the timely measurement of fecal indicator bacteria. EPA’s research team 

succeeded in completing this ambitious study, and findings pertinent to criteria development 

have been published in peer-reviewed journals. 

EPA staff and consultants have developed draft “Recreational Water Quality Criteria” 

(RWQC), the subject of this peer review. Efforts were made to summarize a wealth of data 

used to develop both the 1986 Ambient Water Quality Criteria, as well as the present draft 

RWQC. The peer reviewers commend the EPA team on their efforts to produce such a 

document in a timeframe that meets all that is required of EPA. The comments offered by 

the peer review panel are meant to assist EPA in its efforts to develop new or revised criteria 

that are grounded in science and can be implemented in order to protect public health while 

meeting all requirements of the Clean Water Act. 

Lipp In general, this document represents an extraordinary amount of work conducted by the 

EPA to revise the national water quality criteria. The authors have managed to synthesize a 

broad collection of results and provide analyses that support their recommendations. This was 

not an easy task and the document represents a critical body of work. The comments 

provided here are meant to strengthen the document, increase clarity and provide more 

transparency in how the revision of the RWQC was developed. 

McBride The EPA and its advisers have performed an admirable task of sifting through the literature 

on the health of recreational water users, including many aspects of aquatic microbiology, 

epidemiology and risk analysis. The end result of their analysis is a generally well-argued 

case for the proposed revised water quality criteria. 

I am in agreement with all the comments made by the Review Panel. The following 

responses should be viewed as elaborations of those findings. 

Noble Overall, EPA has successfully synthesized a document to convey their current directions in 

development of new criteria for ambient waters. The document was generally well written, 

and most of the scientific arguments within were made with appropriate scientific support 

and documentation. In particular, EPA is to be commended for the sheer amount of water 

quality related science that they have conducted in the past 5 years. They have gained much 

new information that is relevant to criteria development. One of the most important 

overarching critiques of the criteria document is that they sell their excellent recent science 

short in that they do not use it to its potential for criteria development. 

Yasvinski None provided. 
Reviewers’ Comments by Charge Questions 

 B-4 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for technical 

conclusions transparent? Do you have any suggestions on how we could improve this 

document? Please explain your responses. 

Reviewer Comments 

Dorevitch Conclusion 1: Coastal-derived criteria can be extended to inland waters 

This seems to be the most significant change proposed in the RWQC (since the qPCR 

method was not proposed as a requirement). If epidemiologic data were available to suggest 

that coastal-derived criteria afford swimmers a similar level of protection in inland waters, 

justifying this change would be simple. Data regarding the health risks of swimming in 

inland waters are very limited; the most relevant published information probably comes 

from Marion et al., 2010 and the Keystone Lake, OK study (conducted by EPA more than 30 

years ago). Given that this extension of RWQC to inland waters represents a change from 

the prior application of AWQC to Great Lakes/coastal waters, the reason(s) for this change 

should be clear and compelling. 

The Draft RWQC provides a justification for this change that includes a discussion of 

indicators, pathogens, sources, and cites several published studies, EPA reports, and other 

reports. This approached should be revised, as it did not seem particularly compelling, and 

some portions of the text did not seem to be consistent with other portions of the draft 

RWQC. 

Some specifics: 

The EPA 2010 (g) report on the applicability of the Great Lakes NEEAR data to inland 

waters does not include a “conclusion” and the “Executive Summary” does not include an 

assessment of whether or to what degree NEEAR data could be applied to inland waters. 

Given the many differences between inland and Great Lakes/coastal waters summarized in 

that report, it’s not clear what serves as the basis of EPA’s determination (in the AWQC 

document lines 1307-1308) that the 2012 AWQC would be applicable to inland waters. 

Appendix C.4, however does include conclusions. Among these is that “Sufficient data do 

not exist to extended NEEAR epidemiology findings to non-POTW impacted inland 

waters.” This is not consistent with the draft 2012 AWQC, which does not appear to 

differentiate between inland waters impacted vs. not impacted by POTW sources. 

The discussion of indicators, and whether site-specific characteristics are better predictors of 

fecal indicator bacteria (FIB) densities than is the distinction between inland vs. coastal 

locations is a bit beside the point. Indicator bacteria have been established as criteria 

pollutants and specific criteria values have been established based on EPA’s epidemiologic 

studies. Thus, E. coli and enterococci are direct indicators of health risk, rather than 

indicators of pathogen presence or density. Thus, the efforts to justify extension of RWQC 

to inland waters need not (and given the limited and inconsistent relationship between 

indicators and pathogens described in the RWQC probably should not) be included. 

Likewise discussion of FIB persistence need not be presented, as the criteria indicators and 

criteria values of those indicators have been established directly from studies of health risk. 

Instead of the “site specific differences are more important than inland vs. coastal 

differences” argument, EPA may wish to put forward one or more of the following 
Reviewers’ Comments by Charge Question 

B-5 

rationales for extending the criteria to inland waters. These are: 

A) The WERF Inland Water Workshop report and the subsequent “Meeting Report” 

publication suggest that the criteria should protect public health by preventing sporadic cases 

of gastrointestinal illness. Thus, the use of FIB as criteria pollutants in inland waters should 

useful in making “recreation in and on the water” less risky at inland settings. 

B) The “Meeting Report” (not listed in the references, Dorevitch et al., 2010) notes that 

outbreaks of severe recreational waterborne disease have been reported almost entirely in 

inland waters, perhaps because of limited dilution capacity of receiving waters and 

proximity of recreation (and exposure) to fecal sources. Thus there is a need for public 

health protections in such settings. Although epidemiologic studies have not addressed 

outbreak prevention through FIB monitoring, it seems prudent to use FIB monitoring as way 

of reducing the occurrence of outbreaks of severe illness, as well as the sporadic cases of 

illness that occur among coastal swimmers. 

C) Millions of people per year use inland waters for recreation – see Outdoor Recreation for 

21st Century America, 2004, Cordell et al., Venture Publishing, State College PA. 

Perhaps a better formulation would be to revise the language used in draft 2012 RWQC, 

lines 1449-1451 to: “The application by States of the AWQC to inland water would be 

expected to improve public health protection to the millions of people who use inland 

waters, where, unlike coastal waters, numerous outbreaks of severe illness have been 

reported.” 

Given the impact of the potential extension of RWQC to inland waters on the regulated 

community, EPA may wish to address potential criticisms of this change. These include: 

The WERF report raised the possibility of the criteria could be overprotective (i.e., a given 

level of a fecal indicator may imply a lesser degree of health risk in inland waters). Second, 

it is not known if the application of NEEAR-derived criteria would in fact prevent outbreaks 

of severe disease as these are (fortunately) rare events, which makes it very difficult study 

the impact of an FIB monitoring program on rates of severe disease (or outbreaks) in inland 

waters. Finally, particularly if enterococci are used as a criteria pollutant, many inland 

waters without obvious anthropogenic sources may abruptly be “listed” for 303(d) purposes. 

Clearly the options prevented in Section 5 are intended for this purpose, and having options 

is reasonable. 

Some information presented that is considered to be support for the extension of RWQC to 

inland waters is speculative. An example: 

EPA concluded (lines 1214-1216) that differences in FIB survival “could be different in 

different waters, but those differences could be due to site-specific characteristics and not 

because of fundamental differences between coastal and non-coastal waters.” That survival 

“could be different” and the differences “could be” due to factors other than the coastal/noncoastal distinction is quite different from demonstrating that there is no difference in survival 

in relation to waterbody type. Even if that were the cases, it’s still a leap to conclude that 

indicator-health risk relationships would be the same in inland vs. other coastal water. As 

noted previously, the state of the science may better support removing from this section all 

discussion of FIB survival (and indicator-pathogen relationships). 

If the discussion of indicator-pathogen relationships is not moved to the QMRA section, the 

RWQC could be updated regarding the conclusion that the co-occurrence or correlation 

between FIB and pathogens is neither predictable nor consistent, which is very different than 
Reviewers’ Comments by Charge Questions 

 B-6 

concluding that there is no difference in those relationships in coastal and inland waters. 

First, correlations between FIB and protozoan pathogens have been described in several 

studies with some consistency (Dorevitch Doi, et al., 2011, which was cited elsewhere 

within the RWQC). Furthermore, the literature review in that paper makes clear that the 

relationship between FIB and protozoa does depend on whether the waters are heavily 

impacted by wastewater or not. Thus, at least in the case of Giardia and Cryptosporidium, 

there are fairly consistent/predictable relationships with FIB, particularly in waters not 

dominated by wastewater. 

Conclusion 2: Optional qPCR criteria will be the same in freshwater and marine water

Wade et al., 2010 noted (discussion section) that associations were stronger between 

Enterococcus CCE and GI illness than at NEEAR freshwater beaches than at NEEAR 

marine beaches, consistent with the Dufour/Cabelli studies of FIB measured by culture. 

Overlap of the confidence intervals in the NEEAR qPCR marine and freshwater curves are 

noted in the 2011 paper. However the finding reported in the RWQC (line 716) is 

unpublished and – because of the policy implications - should be the subject of peer review, 

particularly because this reflects a change from the way EPA interpreted findings of the 

Dufour/ Cabelli studies. If such peer reviewed publication is not available in time, the 

RWQC should describe the methods and findings of the analyses in detail. 

Conclusion 3: No need to change the FIB criteria, in fresh or marine waters, measured by 

culture

See discussion below on “Charge Question 3” 

In general, it may be helpful to the readers of the RWQC if EPA distinguishes in an 

executive summary which elements of the 2012 RWQC are found in the 1986 AWQC (i.e., 

they haven’t changed) and which elements are new or different. Elements that have not 

changed need little if any discussion, while changes do. This approach should result in less 

discussion of how the 1986 criteria were developed, including the culture methods, criteria 

pollutants and criteria values for fresh and marine water. 

Conclusion 4: No need to develop source-specific criteria

The discussion makes clear that the evidence is insufficient to change from the 1986 

precedent of not differentiating among sources. Again, since this is not a change, the amount 

of discussion could be reduced. 

Conclusion 5: The translation of HCGI to NGI and the factor of 4.5 (HCGI 8/1000 ~ NGI 

36/1000)

This is reasonable and supported by the text. 

Conclusion 6: The development of the qPCR threshold of 475 CCE (pages 28-29)

See “Charge question 3” regarding EPA’s efforts to demonstrate that the level of protection 

afforded by qPCR is comparable (in terms of cases per illness expected at the threshold 

value of enterococci CCE) to that at the culture-based criteria value. Consider creating a 

subheading (something like “3.3 The development of criteria values for enterococci by 

qPCR”) for this section. 

Lines 993-998 require clarification. If this material is to be retained in the RWQC, describe 

the methods of each of the “additional analyses that were not used directly”, what were the 

results of those methods, and what were the limitations and interpretation of those results. 
Reviewers’ Comments by Charge Question 

B-7 

What does it mean that the additional results “support the underlying basis” (line 994) of the 

approach to deriving the number 475 CCE/100mL? The Wymer translation is not “support” 

for the number 475; it’s part of the methods used in calculating that number. 

Conclusion 7: The development of criteria using the study population overall, rather than 

potentially “sensitive subgroups” (section 3.3)

Consider renaming Section 3.3 “Potentially sensitive subgroups” 

The Parkin 2003 reference (line 1057) does not establish what the citation suggests: that 

immune system of children can leave them vulnerable to the effects of environmental agents. 

The paper presents a modeling approach to water management, in which it is assumed that 

children are sensitive, to assess the impact of that assumption on model outputs. It would be 

circular to think of this as evidence of the vulnerability of children. An alternative is to cite 

some of the primary literature noted in Parkin, namely Kee 1994 and D’Allesio 1981. 

However, those studies do not demonstrate (nor were they designed to evaluate) whether, for 

a given level of indicator bacteria, children are more likely than adults to develop HCGI (or 

NGI). Best to leave this out and discuss, based on NEEAR data, whether or not the microbehealth risk association (slope) is different for adults and children. 

Lines 1049-1052 seem to conflict with 1064-1075. Suggest: delete lines 1049-1052 and 

introduce the rest of that paragraph with a sentence saying something like “There are two 

reasons to be concerned that for a given level of indicator bacteria in recreational water 

children would be more likely than adults to develop gastrointestinal illness.” Then note: 1) 

a suggestion of greater water ingestion during swimming (Dufour 2006) and 2) the 

observation from the NEEAR study that children spend more time in the water. Despite the 

theoretical reasons which suggest children might be at increased risk, the NEEAR study did 

not find this to be the case. First, in the analyses that used microbe density as a continuous 

variable (fresh and marine) lacked power to identify subtle differences between adults and 

children. Second, indicator density dichotomized at 33 CFU/100mL seems to have been a 

comparable predictor for adults and kids in freshwater (Wade 2008, Table 6). It would be 

very informative if the statistical significance of interaction terms (child/adult * microbe 

level) or (child/adult * above/below threshold) were reported. 

Lines 1095-1096: The above information does not indicate that children are more 

susceptible. The above information suggests that despite theoretical concerns about ingestion 

rates, duration of recreation and perhaps naïve immune systems, the epidemiologic data do 

not support (given sample size) differences in risk between adults and children. Consider 

deleting this paragraph. 

Consider moving paragraph lines 1027-1037 to just prior to the paragraph that begins on line 

1109. As currently organized, the discussion of over-representation of children in NEEAR 

enrollment is broken up by the discussion of NEEAR results regarding children vs. adults. 

Lines 1112-1113: I’m not sure what is meant by “in a robust manner.” Consider revising the 

final two sentences of this paragraph to “In the NEEAR study, children are over-represented 

(compared to the general population) and therefore have a disproportionally large influence 

on the overall relationship between microbe density and health risk derived from NEEAR 

study data.” 

Lines 1116-1130: This is a very important point and could be emphasized better as its own 

section, perhaps something like “Disease prevention through rapid notification.” As noted 

under “Charge Question 5” below, it would important for the public and stakeholders to 
Reviewers’ Comments by Charge Questions 

 B-8 

know that the 2012 RWQC provide for the first time, guidance to afford the public the 

benefits of same-day notification. 

Conclusion 8: qPCR methods for monitoring recreational waters are optional

This is a surprise and a disappointment, as the NEEAR study suggests that public health 

protections could be improved at Great Lake settings through the use of qPCR monitoring. 

The method can provide same-day results – a huge advance toward protecting the public 

form gastrointestinal illness using prior-day measures of water quality to predict health 

outcomes. Futhermore, the NEEAR study demonstrated the superiority of same-day qPCR 

measures to culture measures as predictors of NGI. 

Page 44 of the draft RWQC notes that Method A can be used as a site-specific alternative, 

pending the release of the “topic-specific guidance.” For inland waters several reasons were 

proposed in Appendix B.6 to suggest that inland waters may have different qPCR-health risk 

relationships than Great Lakes/coastal waters. For that reason it seems that such sites would 

need evaluations of method performance. However, given the vast amount of health and 

qPCR data collected at Great Lakes sites (and to a lesser degree, from marine sites), there 

seems to be enough data to apply the NEEAR results (and the 475/1,207 CCE/100mL 

criteria values) to Great Lakes/coastal waters. Any “site specific performance 

characterization” (line 1617) would not include the collection of health data, which NEEAR 

did collect, it’s hard to see how such characterization of method performance will provide 

better qPCR-health data than did the NEEAR study. It not realistic to expect that the 

NEEAR study could have been conducted at a wide range of combinations of waterbody 

type and pollutant source. When establishing the 1986 criteria, data from a limited number 

of sites were used, even though not all water types had been studied. The same could be 

done for (mandatory) qPCR criteria, applying the best available information. 

The term “site-specific performance characterization” should be clarified. The intent may be 

to evaluate qPCR inhibition and to support the development of a site-specific plan for 

addressing inhibition (such as dilution or some sort of sample processing). However, it 

should be clear that the performance being characterized is not the relationship between 

qPCR results and health risk, which has been done in several types of settings by the 

NEEAR study. 

Conclusion 9: Statistical threshold values will be used rather than single-sample maximum 

values

This change is an improvement of the 1986 AWQC. The term “maximum” implies a value 

never to be exceeded, while it is expected that about 25% of individual measures will exceed 

the STV. This change should be beneficial to beach manager in the efforts to communicate 

the meaning of an exceedance. 

The RWQC could be improved by explaining the basis for sampling required for calculating 

a geometric mean. Ideally, the variability in water quality should be known before 

estimating the number of samples required to calculate a GM. Furthermore, by choosing to 

sample only on days for which relatively low values of indicator bacteria are expected (no 

rain for several days, perhaps in the afternoon of day with much sunlight, the calculated GM 

could be lower than if samples were collected at random or on a fixed schedule (i.e., every 

Friday, Saturday, Sunday, and holiday, reflecting recreational use). Provisions to prevent 

selection of sampling days based on expected low values would improve the public health 

protection of the criteria. 

Conclusion 10: Section 5.2.2: Flexibility available to states
Reviewers’ Comments by Charge Question 

B-9 

The draft RWQC offers states two types of tools: Those not part of state water quality 

standards but which may enhance assessment/management/notification, and those that are 

designed to develop alternative water quality criteria. That QMRA (section 5.2.2) has been 

categorized as a pathway to alternative criteria rather than as an assessment/management 

tool requires evidence-based justification. 

Unlike another innovative aspect of the RWQC – the use of qPCR – very little is known 

about how well QMRA would actually protect swimmers from GI illness. Through the 

painstaking efforts of EPA/the NEEAR study team, we now can say quite a lot about how 

qPCR should (and should not) be done, and more importantly, how well qPCR methods can 

be used to predict health risk and provide timely information to the public. By contrast, it is 

unknown how well QMRA predictions (model outputs) of health risk approximate observed 

(i.e., epi study) health risks – we just know the model outputs. Furthermore, we don’t know 

how precise QMRA estimates of health risks really are, regardless of the confidence 

intervals provide by simulations. If a QMRA model estimate that a given degree of water 

quality is expected to result in 30 cases of NGI, that might mean somewhere between 5 and 

75 cases, with 95% confidence. It’s very hard to know, given that QMRA models have not 

been validated against observational data of swimmers at beaches. The studies cited in the 

draft RWQC – Soller, Schoen and Ashbolt- are estimates of risk/illness that have not been 

confirmed through observational studies. Although EPA had hoped to have a QMRA 

conducted in parallel with one of the NEEAR sites, apparently no comparisons have been 

reported. Many assumptions of QMRA – notably dose-response – are very uncertain, having 

been based on very limited data. Thus, although the theory behind QMRA is sound, the 5.2.2 

provision is unvalidated. QMRA should instead be considered a method for evaluating 

management options – which would provide estimates of relative risk, rather than estimates 

of absolute risk which would be required for any method that would be an alternative to an 

epidemiologic study. 

The Inland Waters Workshop (Appendix B.6) report and the subsequent Meeting Report 

note that QMRA methods require “grounding” (validation against observational data). If any 

validation has been done since the Workshop, this should be reported in the RWQC. 

Because the application of QMRA could allow localities/states to potentially produce lower 

site-specific/region-specific standards, it is important to know with confidence that the 

health of the public is not being compromised (and that undue burdens are not placed on 

regulated entities. 

Lines 1905-1907 (if non-human sources predominate) and lines 1432-1436: If the data are 

insufficient to propose national criteria for waterbodies impacted by non-human sources, are 

the data sufficient to support the development of site-specific standards for such 

waterbodies? 

Conclusion 10: Section 5.2.3: Alternate criteria

The concept here is very good, in particular the emphasis on establishing a relationship 

between new methods and existing methods at the criterion density. 

Since Wade et al., 2008 demonstrated that at Great Lakes beaches the qPCR method of 

measuring enterococci performed better than the culture method in predicting NGI, why not 

specify that equivalency should be demonstrated between the new method and best available 

approved method (i.e., enterococci qPCR)? 

Lines 2026-2027 note “that a consistent and predictable relationship “should exist between 

established and new methods. Furthermore whether the new or established methods would 

be used “beach notification and attainment decisions....would be similar” (lines 2028-2029). 
Reviewers’ Comments by Charge Questions 

B-10 

In order to reduce confusion down the road, spelling out, it would be helpful to explicitly 

state what is meant by “consistent” “predictable” and “similar” . A 2x2 table could be 

constructed with rows “new method” and “established method” and columns, “exceeds 

criteria value” vs “does not exceed criteria value.” Does it mean a chi square p-value of 

p<0.05 even if the new method misses some percent of exceedances? If EPA were to specify 

the sensitivity and/or positive predictive value it would be clear what states/municipalities 

must demonstrate. Furthermore, how much data is needed? Perhaps a season (Memorial 

Day-Labor Day) of 5x/week measurements or some other general specification, otherwise, a 

“consistent, predictable relationship” could be claimed based on a much smaller set of 

observations. 

Last, the RWQC notes that this approach could be used to establish “regional or state-wide 

alternate criteria” (line 1993). If this is the case, again EPA should specify that the data 

collected to establish the equivalence of an alternate criteria must match the scale of data 

presented. In other words, data from one beach can’t establish state-wide criteria, and statewide data can’t establish regional criteria, etc. 

Other aspects of Charge Question 1:

Some specifics are noted in response to other charge questions. Otherwise, no significant 

issues regarding transparency, shortfalls, uncertainties, suggestions for improving the 

document. 

Lipp In all this was a thorough and well laid presentation of the justification for the recreational 

water quality criterion. The authors provide a well-organized framework to understand the 

justification and implementation of the RWQC and provide a much needed section on how 

States can proceed with their own alternate criteria. More emphasis, however, should be 

focused on the new criterion and not the history of the current AWQC. It would also be 

helpful (from a structural/organizational perspective) to better cross reference specific 

appendices in the document rather than broad groups (e.g., appendix A.2 rather than 

appendix A, etc.). The addition of an executive summary would also improve the document. 

While the uncertainties for the technical conclusions are provided, limitations in applying 

this criterion to “all” water bodies should be better acknowledged. The RWQC is meant to 

be for all water bodies in the US but the data that EPA presents does not support this call. 

Namely, the RWQC appear to be robust for temperate POTW-impacted marine and 

freshwater coastal areas, with possible application to POTW-impacted inland waters; 

however, there are no compelling data that support its use in tropical (or subtropical) waters 

or in non-POTW impacted waters (coastal or inland). As a general note I was struck that to 

date all NEEAR studies at marine beaches were conducted in embayments or estuaries (with 

the exception of Surfside, SC), that were relatively quiescent. This may not be a good 

representation of the bulk of marine beaches, which tend to be open water beaches and is a 

limitation of the sites selected for the health effects assessments. This should be better 

acknowledged and information presented on site selection. Such information would help 

states/locales judge whether they should look at their own site specific assessments for other 

types of beaches. 

The use of predictive models and the great amount of work that has gone into developing an 

easy to adapt format (e.g., Virtual Beach) is to be commended. The ability to use pre-emptive 

strategies to protect public health should be a goal for all beach managers. While Virtual 

Beach works quite well for Great Lakes Beaches, its performance for marine beaches was 

not impressive (which may be have been exacerbated by the few days of exceedance for the 
Reviewers’ Comments by Charge Question 

B-11 

beaches tested). This should be clarified in the document. 

While the final criterion uses culturable E. coli, culturable enterococci and qPCR 

enterococci (CCE)for freshwater and culturable enterococci and qPCR enterococci (CCE) 

for marine water, there was significant effort put into the confirmation and validation of the 

qPCR for Bacteriodales it appears that this will not be used as a criterion. Is this one of the 

tools that States can suggest as an effective alternate indicator? If so, it may be helpful to 

state that explicitly somewhere. While the final RWQC will continue to rely on the same 

FIB criterion as suggested in 1986, the improvement in risk determination (and confirmation 

that risk levels are equivalent between fresh and marine waters) is a great step forward as is 

the specific guidance on qPCR (even if is not a required element). 

Another strength of this document is that it allows states to update and find alternatives for 

their specific areas. The technical support guidelines will need to be complete to ensure good 

translation of information. While it is outside the scope of this document, states should be 

able to compete for grants to develop these site specific alternatives. 

Some specific comments on logic and support for conclusions in this document: 

Line 720. For clarification the reason that there is a focus in this analysis/review on 

temperate POTW-impacted beaches needs to be stated early (namely, that EPA dropped the 

epi studies from run-off impacted and tropical beaches because of lack of correlation 

between FIB measured and health effect end points) 

Line 759-769 (see above statement about placement of this text). These studies were 

dropped because the results were inconclusive. Given the that the temperate POTWimpacted beaches all fit well into a single analysis it makes some sense to exclude these 

studies, but the fact that they did not fit the trends noted for the other studies should suggest 

that: 1) the current RWQC will not be adequate for these areas, 2) some other as yet untested 

FIB may be better suited for a single FIB for all water bodies (although this scenario would 

require many more studies, time, effort and money) or that additional epi studies are warranted 

to better determine the applicability (or lack thereof) the suggested RWQC. While there is 

sufficient language to allow states to address this issue on their own, this seems like a 

potentially large problem that may need additional federal (EPA) assistance. [Line 784-788. 

Following up on the issue noted above, this review of studies also shows a paucity of epi 

studies focused in tropical areas. Additional work is clearly needed in this area.] To this end, 

a separate section on issues associated with tropical/subtropical waters may be warranted 

and should include studies completed in Hawaii on this issue. 

McBride The document presents a well-argued case. I agree with its general approach, in particular: 

• coalescing the risks associated with marine and freshwater criteria; 

• extending the compliance assessment period; 

• allowing for flexibility in implementation (e.g., QMRA, novel indicators, predictive 

models). 

By and large the text seems clear. In the following I have noted those occasions when it 

seems not to be so. However, I do suggest that some matters need to be addressed or 

clarified: 

• No guidance is given on the desirable features of sampling programs, especially with 

respect to wet weather. I note that this is to be included in follow-up technical support 
Reviewers’ Comments by Charge Questions 

B-12 

guidance (TSG). Nevertheless, it should be helpful for there to be at least some mention of 

this issue in the criteria document—after all, the 1986 criteria did do so.1 And if this 

former position is to be abandoned (i.e., restricting sampling to dry weather), it would 

seem appropriate to say so and give the rationale for it. In my view wet weather sampling 

(or sampling shortly after rainfall) may be appropriate if recreational contact occurs at 

that time. I don't see lot of point in sampling during conditions that are not conducive to 

recreation. 

• Likewise I didn't see any reference to guidance on sampling depths (knee depth, chest 

depth...). 

• I wonder if respiratory illnesses have been dismissed too easily. Yes, it is true that in 

some studies GI can be the stronger endpoint, but that may not be so for water skiers, 

canoeists or for beaches with an energetic wave climate such that aerosolisation is rather 

stronger. After all some studies have detected respiratory effects but not GI: England 

(Fewtrell et al., 1992);2 Australia (Corbett et al., 1993, Harrington et al., 1993); New 

Zealand (McBride et al., 1998). 

• Has the case really been made that feces of all animal groups should be treated as posing 

the same risk as human feces? For example, there is evidence—in the New Zealand 

setting at least—of substantial differential virulence between Campylobacter jejuni

carried by farmed animals (sheep, cattle, poultry) versus wild birds, and yet wild bird 

strains can predominate in recreational waters (French et al., 2011). 

• The distinction between SSM and STV is not sufficiently clear, especially because for 

beach notifications (lines 1542–1545) the STV can act as maxima not to be exceeded 

(i.e., acting as a SSM) but for WQS attainment "no more than 25% of the observations 

can exceed the STV" (acting as a percentile limit). I also have difficulty understanding 

how the STV (if it is interpreted as a SSM) can be used in devising permit limits, 

especially if used alone (as suggested on line 1548). In my view stating only a limit never 

to be exceeded is next-to-useless in designing wastewater infrastructure. In contrast a GM 

(or median) accompanied by a high percentile (but not a maximum) tells designers and 

operators of wastewater facilities what is expected under normal and under unusual 

circumstances. In that regard a 75%ile seems rather low; international practice 

increasingly uses 90%iles or 95%iles. 

• Lines 1494–1503. This is timely and well-stated. But the last sentence begs a "so what's 

the consequence of that?" question. I presume it means that EPA is prepared to tolerate a 

moderate degree of STV exceedances and hence notifications (interpreting STV as a SSM, 

as is proposed for Beach Notifications)—even when the GM is way below its limit. 

• Line 1559 is ambiguous: '...i.e., no "excursions" acceptable."' I think refers to the GM, but 

I first read it as applying to the STV. 

• Given that the WHO, Europe and Australasia have adopted the "Annapolis protocol", its 

non-appearance in this document seems surprising. I understand that the CWA is 

 

1

 At page 8 of EPA'S Ambient Water Quality Criteria for Bacteria –1986: EPA440/5-84-002 (cited as "U.S. EPA 1986" 

in the draft RWQC) we have: "In general, samples should be collected during dry weather periods to establish so-called 

"steady-state" conditions. Special studies may be necessary to evaluate the effects of wet weather conditions on waters 

of interest especially if sanitary surveys indicate the area may be subject to storm water effects". The criteria themselves 

(in EPA 44/5-86-001, Quality Criteria for Water 1986) contained a similar message: "These densities are for steady 

state dry weather conditions." 

2

 All references in this section are listed in Section 2 of the Review Panel's report. 
Reviewers’ Comments by Charge Question 

B-13 

considered to pose an impediment to its adoption in the USA. But it would be helpful to 

explain why that is the case. And I note that the proposed extension of the compliance 

assessment period (recommended to be 90 days) takes the USEPA criteria closer to the 

"beach grading" component of that protocol. 

Noble General comments: The criteria document as assembled successfully accomplishes the 

following: 

1. Presents the history behind previous criteria development.

2. Discusses a range of aspects of 304 (a) criteria development as related to a range of water 

quality standards/procedures (i.e., does not focus solely on beach warnings and postings). 

3. Presents the scope of the science. 

4. Presents the linkage b/w water quality and health and explains epidemiological studies. 

5. Attempts to outline a framework for connection of recent epidemiological studies to those 

conducted in the 70s and 80s. 

6. Presents results of epidemiological studies conducted recently and subsequent data 

analyses. 

7. Attempts to show the reader a transitive approach toward linkage of current studies to 

older studies, bringing in results from rapid methods. 

8. Outlines a few other ways that waterbodies can be managed in the USA. 

The way the document is written is generally a treatise on the “process” that EPA used to 

develop or revise criteria. The document is not written in a forward thinking tone 

(prospective), it is rather retrospective. It is this reviewer’s opinion that detailed explanation, 

use of case studies, and providing of examples where relevant could greatly help EPA in 

paving a path forward for adoption of new criteria. 

There are serious shortcomings in the document in the following areas: 

A) The existence of the FIB to pathogen relationship is specifically stated in a wide range of 

places in the document. It is also noted that there is a lack of pathogen relationships to FIB, 

especially viruses. Why continue with the notion that FIB predict pathogens? There is ample 

evidence at this time that the numerical relationships between pathogens and FIB in 

waterbodies - freshwater and marine - exists only in certain cases, and often times does not 

exist. Typically, strong relationships have been shown between viral pathogens and FIB in 

cases of studies of direct discharge of sewage. It is likely that the poor relationship is a 

function of methodologies, statistical variability, lack of consistent study approaches, and/or 

lack of data using like methods, but this is not important to the current thinking in criteria 

development. EPA has clearly shown, as well as others, that FIB (measured using either 

culture- or molecular-based methods) can be linked to human health outcomes using 

epidemiology studies. EPA has stated that they are using the FIB-health outcomes 

relationship wholly to establish and defend and develop new criteria. The addition of 

information, and attempt at literature review of pathogens only complicates this criteria 

document. If anything this information presented about FIB to pathogen relationships should 

be placed in the QMRA section of the document (or in Section 5.0 broadly), to demonstrate 

to the reader the paucity of good, scientifically relevant information. Therefore, paving the 

way for a consistent, statistically rigorous QMRA approach to be presented by EPA. 

B) There is a complete lack of guidance on subtropical and tropical waters criteria 
Reviewers’ Comments by Charge Questions 

B-14 

development. EPA attempted one epidemiological study in tropical waters, and has not 

conducted any others. Given the importance of beach visitorship in FL, HI, and PR alone, 

this issue remains important. Hawaii has been urging EPA for decades to examine and assist 

them with development of other standards, citing serious issues with attempting to quantify 

Enterococcus solely as an appropriate indicator. Lack of information about Tropical Areas 

means that these areas should either be able to develop their own approaches OR that they 

can use the 1986 criteria. However simple use of the 1986 criteria means that they are using 

science that is nearly 40 years old, and has not been updated. Tropical Workshop attendees 

in 2002 hoped that EPA would develop new indicators that worked in all areas. This clearly 

is not the case for subtropical and tropical waters. Therefore, EPA should be required to 

assist these areas in developing an approach that will be successful for them, even if it 

represents a departure from the national standard. Tropical waters need to be specifically 

called out in the criteria document. 

C) There is no stated requirement for QPCR, which is a good thing as it is not the answer 

for some State’s monitoring needs. However, EPA misses an opportunity to stress the 

importance of rapid methods in providing more accurate means to protect public health at 

beaches. There are serious issues with public perception of the costs and difficulties seen 

with rapid QPCR based methods. That is, most people think that QPCR is more cumbersome 

and expensive than it really is. EPA could do a better job of demonstrating the tractability of 

these methods to stakeholders. They use Griffith and Weisberg, 2011 several times as an 

example. However, it would be useful to explain the costs and advantages of rapid methods 

versus culture based methods, and to provide more specific examples of where it has been 

used successfully to manage recreational waters (southern California, Great Lakes). EPA is 

unlikely to revise criteria in the next decade, therefore, they should consider 

recommendations of the following for QPCR: a) implementation at high use beaches (this 

has been very successful in southern California), b) implementation in areas that have high 

numbers of beach going families including children. EPA is making the right decision in not 

requiring QPCR to be conducted nationwide, but there are merits to its use, particularly 

important is the speed of management decisions. The case could be made by EPA to 

strengthen its recommendations for the use of QPCR (possibly some sort of incentive 

program?). Finally, EPA’s Method A is very complete and detailed. Is it possible for EPA to 

create a shortened, Cliff’s Notes version of Method A to provide to stakeholders so that they 

can visualize the process? (i.e., collect water sample, filter, bead beat, create master mix, 

load machine). A simplified version of the method would assist EPA greatly in bridging the 

next steps of rapid methods implementation. 

D) The intended switch from SSM to STV is not clearly presented and does not permit 

coastal and non- coastal States that use other management approaches to incorporate any 

flexibility in their current ways of managing and listing impaired waters. Some states use 

frequency of beach closures to manage their impaired water status, implementing a 

comparison between reference beaches and contaminated beaches. EPA needs to clearly 

state the minimum sampling frequencies (this is done but the sampling frequencies are 

actually less stringent than before?), they need to provide guidance on sampling during wet 

versus dry periods and over seasons. The frequency of beach closures approach, which 

compares like beaches to like beaches, is an important interpretation of beach management 

framework. Flexibility in the use of other more stringent approaches for compliance 

purposes needs to be included in the document. Sampling variability across states will lead 

to inconsistency in the way that the new STV approach is utilized for TMDL listings. EPA 

should strive now to make this approach consistent across states and provide succinct 

guidance on the minimum approaches. Currently, States that do the most monitoring spend 

the most money on TMDL implementation and development. These states also suffer from 

the fact that more monitoring equals more listings. EPA needs to create equality in the 
Reviewers’ Comments by Charge Question 

B-15 

system by mandating specific levels of monitoring.

E) There are major shortcomings in QPCR/rapid methods as presented. There are major 

holes in the recommendations on rapid methods that deserve to be addressed in the 

document. EPA needs to more transparently state that they did not include rapid E. coli

QPCR nor culture-based measures of E. coli in the freshwater NEEARS Studies. E. coli is 

an important indicator in a range of different non-EPA epidemiology studies, and the lack of 

data for the NEEARS freshwater studies needs to be stressed. Second, EPA is suggesting an 

extrapolation of E. coli standards to inland waters based upon old science (1986 science), 

this needs to be clearly stated and transparently discussed in the document. EPA has either 

not done or has not presented the appropriate data analysis across quantification approaches 

for QPCR to demonstrate that the more variable CCE quantification approach is superior to 

the cell equivalent (CE) quantification approach. In southern California, CE based 

quantification approaches have been used for two consecutive summers to rapidly measure 

beach water quality via Enterococcus quantification with some success. The approaches 

have demonstrated very strong relationships to existing culture-based methods. SCCWRP 

epidemiology studies have also shown that both CE and CCE quantification approaches 

have yielded significant relationships to human health outcomes. EPA has also shown that 

this relationship is significant for both quantification methods, but they are not transparent 

about this in the criteria document. EPA has not included relevant rapid methods data from a 

range of methods for SCCWRP conducted southern CA studies (including QPCR but also 

highlighting other methods like IMS-ATP). Some of this data is already in publication 

format (submitted to journal) and easily interpretable. The rapid methods information that is 

presented can be related with some difficulty to the culture based methods, but there are 

locations where both equivalency (numerical correlation) and/or health relationships exist 

that are different from the criteria development stated by EPA. Clearly, EPA will pick only 

one quantification approach, but addressing both quantification approaches will be important 

for understanding how modifications of the EPA QPCR methods could be implemented by 

others in the future. 

F) EPA has not fully utilized all of the epidemiological data available for informing their 

new criteria. In particular, as stated above the SCCWRP conducted epidemiological studies 

provide important information on the importance of assessing inhibition, on the use of cell 

equivalent (CE) based quantification approaches, epidemiology at non POTW impacted high 

energy beaches, and comparisons of EPA methods to other available rapid methods. EPA 

needs to take advantage of this available body of work. 

G) In places it appears that the criteria document is difficult to read and vague. The 

following issues exist, a) sections could be reorganized to flow better, b) that specific 

sections have not had the benefit of consideration as totally as others (specifically the 

sections about waterbody type and criteria setting across coastal and non-coastal waters, and 

c) that EPA is presenting the “process” of how they arrived at the criteria, but they are 

providing very little foresight as to actual implementation. In this way, they miss out on an 

opportunity to provide specific guidance on a) successful implementation of QMRA, b) what 

constitutes enough side by side sampling between culture based methods and QPCR, c) what 

constitutes a similar enough method to Method A and how they go about presenting that 

information, d) examples of the use of STV in TMDL impairment listings, and how the new 

approach will change existing approaches. 

Yasvinski It is clear that a tremendous amount of effort has been put in to developing these 

recommendations and their supporting documentation. In terms of adequate and transparent 

presentation of the conclusions, I would say nearly. In general, the conclusions are rooted in 

logic the depth of the literature reviewed scientifically supported, and EPA has gathered/ 
Reviewers’ Comments by Charge Questions 

B-16 

reviewed and produced sufficient information to support its risk assessment decisions. 

1. There are some decisions that need to be better illustrated and supported or defended in 

the 2012 document. Chiefly, the ones I would highlight are: 

• The findings around the fresh/marine equivalency would benefit from further 

illustration (question #3) 

• Stronger supporting discussion should be provided as to why the stronger qPCR 

results were not used to drive the criteria as opposed to the culture-based results 

(question #4) 

• Supporting discussion as to why the qPCR criteria remain expressed as NGI illness, 

while the culture based criteria are expressed as HCGI, even though the discussion 

seemed to suggest an easy conversion or interchangeability between the two (question 

#4) 

• More specific discussion around why E. coli, though not included in the NEEAR 

analyses is retained and justified in its role as the primary 

2. Additionally on the subject of transparency and completeness, the review panel discussed 

at the meeting that on the topics of: 

• Coastal vs. Inland waters 

• Tropical/subtropical vs. Temperate waters 

and perhaps to a slightly lesser extent: 

• Human source fecal material vs. Non-human source fecal material 

The basic assessment seemed valid - that the data variability and gaps were sufficient so 

as to suggest that the health risks for each type were similar enough and could not be 

independently characterized. However taking the direction of similarity in the discussion 

was done at the expense of acknowledging that there are also some real apparent 

differences between each comparison pair, but that the impact of this differences is not 

known or it is not known how significant. Acknowledging these differences and the data 

limitations is required to convey to the reader that EPA is aware of and has considered all 

the available information in presenting their decisions. 

The review panel further discussed that a more intuitive approach would be to structure 

the basis of the argument that extension of criteria across recreational water types was 

done to be protective of public health in the lack of sufficient information. This decision 

would be rooted in logic and allow EPA to acknowledge the differences and the data 

gaps. 

3. Another issue discussed by the review panel that was not addressed in my draft 

comments, but was captured in my specific comments concerned the clarity of the 

guidance for applying the criteria. It is proposed that this section needs to be rewritten in 

a more clear and straightforward manner. Currently, the explanation of the statistics, the 

rationale, the calculation of the statistics and the guidance as to application are all 

intertwined. It is recommended that these be separated for easy of understanding, and 

specifically that the application guidance be specifically laid out in manner that is simple 

and easy for the intended recipient (responsible recreational water authority or beach 
Reviewers’ Comments by Charge Question 

B-17 

manager) to understand. 

The following editorial comments are also submitted: 

1. It is clear to me that the EPA has completed a tremendous amount of work in preparing 

the newly proposed 2012 criteria. So much so, that it appears that the EPA should be 

able to confidently defend its conclusions and decisions. In the criteria document there 

appears to be some instances where the text in presenting a point, follows up with what 

might be a response to what expected criticism might be. Or similarly, uses a small point 

to point out consistency with EPA findings, rather than having a more complete, 

organized discussion of additional material that supports EPA’s decisions. As examples: 

• In 3.1, it is introduced that EPA is allowing for a qPCR method, but then immediately 

following the text focuses on the issue of possible interferences and how they have 

been resolved. 

• In 3.2.3 in discussing the body of epidemiological literature, after mentioning the 

Marion, 2010 study, it states that the E. coli findings support EPAs 1986 criteria, 

rather than having a more coordinated discussion of how the outside body of 

evidence is also in line with EPA’s conclusions. 

 I understand this is an extremely complex document with many issues that need to be 

addressed, and doing so in a logical and easy-to-read format is challenging. However I 

believe with the major issues to be addressed (e.g., the criteria itself, water body type, 

source of fecal contamination), the work put into this endeavour is such that the EPA 

should be able to present its findings and recommendations and then follow up with short 

explanations for each major issue - yes, here are the known challenges or 

shortcomings...and we believe these are addressed/justified because we have done this or 

have the following information which supports it. 

2. Another suggestion is that it would be valuable to have a brief opening summary or 

executive summary of the key decisions or the “need-to-know” information for the 

states, tribes and readers. The criteria has a lot of components and a lot of questions to 

be answered and it is difficult to tease out the key conclusions from the individual 

sections. 
Reviewers’ Comments by Charge Questions 

B-18 

2. Are you aware of any important peer reviewed literature or applicable data that have not 

been incorporated in this document and should be included? 

Reviewer Comments 

Dorevitch Waterbody type (pages 33-36): See charge question 1. 

Support for idea that recreational criteria applied to inland waters should be protective: 

Inland Water Meeting report: Dorevitch, Ashbolt et al., 2010. 

Millions of people use inland waters for recreation: 

Cordell, H.K., Betz, C.J., Green, G.T., Mou, S., Leeworthy, V.R., Wiley, P.C., Barry, J.J., 

Hellerstein, D., 2004. Outdoor Recreation for 21st Century America: A Report to the 

Nation: The National Survey on Recreation and the Environment. Venture Publishing, Inc., 

State College, PA. 

Heaney 2009: Suggests that a substantial portion of illness among swimmers enrolled in the 

NEEAR study may be due to sand exposure. This is significant, in that it means that some of 

the illness incidence among NEEAR study swimmers may be due to sand, rather than to 

water exposure and water quality. If this study is not addressed in the RWQC, it’s possible 

that it could be considered by others to be an oversight that falsely inflated the importance of 

beach water quality on health risk (“the problem isn’t the water, it’s the sand.”) The study 

should be acknowledged, along with the fact that the NEEAR freshwater (Wade 2009) and 

marine study (Wade et al., 2011) did adjust for digging in the sand should be noted as well. 

The basis for thinking that children would be at greater risk than adults (lines 1052-1053): A 

citation that would be relevant here is Dufour et al., 2006. 

Pratap et al., J Water and Health 2011: Even if extensive efforts aimed at measuring FIB and 

identifying conditions of elevated risk are applied, disease will not be prevented if effective 

communications do not follow. Pratap et al. noted substantial variability among 

states/tribes/territories in their approaches toward notifying the public, which have not been 

systematically evaluated for effectiveness. While not an element of identifying an indicator 

or establishing criteria values, perhaps it could be noted (perhaps in section 5.1 or in the 

general introduction to indicators in section 1.1) that indicator measurement is a critical, but 

not the only critical element, in a chain of steps that is intended to result in disease 

prevention. Efforts to modify behavior (i.e., don’t go in the water when a notification/swim 

ban/beach closure or advisory is in effect). 

Lipp The authors have been thorough in their review of materials, especially when taken as a 

whole with all appendices. 

McBride A number of extra references have been identified and are all included in the Review Panel's 

responses to this question. With regard to the lack of a "road map" (Review Panel's 

comment 1.1), the recent WERF QMRA study (Bambic et al., 2011, not cited in the 

RWQC—but listed in the Review Panel's report) has a conceptual diagram of a sort that may 

be appropriate as a decision tree for RWQC implementation, viz. 
Reviewers’ Comments by Charge Question 

B-19 

Noble There are four major missing sets of documentation. First, the epidemiological studies 

conducted by SCCWRP on the west coast are mentioned. However, these studies are not 

incorporated at all into the discussion, except under the context of NPS contaminated 

beaches and the literature produced as part of the Mission Bay, CA study (e.g., Colford et 

al., 2005). The beaches studied in CA are 1) major beaches for recreation, 2) representative 

largely of beaches found in southern California, 3) well studied and have results at this point 

assembled into the framework of journal articles that have been submitted or are ready to 

submit, and 4) represent high energy beaches that are not widely represented by other 

published epidemiological studies. Therefore, the data from these studies is far enough along 

that EPA could more effectively make use of the results. SCCWRP has been active in 

sharing all of the data generated from these studies, and it is to EPA’s benefit to take 

advantage of this information. EPA has full access to this data, and the beaches studied in 

southern California represent a vital subset of beaches that have not been adequately 

addressed by EPA (i.e., beaches that are recipients of NPS pollution). That is, those beaches 

that are high use, high energy (wave energy), have significant contributions of diffuse 

pollution (specifically San Juan Creek and Malibu Creek are two major conduits of 

concern), and are not impacted by any direct POTW point source. These types of beaches 

dominate southern California and Florida waters, and while Florida waters are subtropical in 

nature, the data collected from these beaches is a vital part of the package of information that 

EPA should be examining. Additionally, more emphatic inclusion of the Epibathe data, 

including a serious examination of the E. coli data and a presentation to the reader for how 

that data relates or differs to other studies that have been conducted in the US would be 

useful, as the omission of E. coli from this document is a serious issue (as noted below). 

Second, EPA has been seriously remiss in their inclusion of E. coli in freshwater studies. 

The criteria document is full of broad level statements about E. coli being adequately 

addressed. On page 22, EPA states that the reason that the E. coli testing was not done was 

because EPA was taking an approach of a single indicator to be used for all purposes. If the 

approach is to link current criteria to historical criteria, measurements of E. coli were an 
Reviewers’ Comments by Charge Questions 

B-20 

obvious necessity. The omission of E. coli in both the culture based format and molecular 

format (qPCR) in the Great Lakes NEEAR studies is of major concern. In other parts of 

documents, EPA has written about the fact that their omission of E. coli measurements is 

warranted (see Appendix A, where the lack of E. coli based information is rationalized). The 

results of Epibathe, Marion et al., 2010, and other studies conducted in freshwaters recently 

indicates that this is clearly untrue. It is this reviewer’s opinion that the EPA must be 

transparent about the lack of new science in relation to E. coli, specifically relating to rapid 

E. coli QPCR methods. They must address, with transparency, their lack of foresight in not 

measuring E. coli during the NEEAR freshwater studies. In 2006, there were several offers 

to EPA by outside entities to analyze the existing archived filters from the NEEAR 

freshwater studies using E. coli qPCR. EPA originally indicated interest in conducting these 

analyses, and but never pursued them fully. The lack of culture based E. coli work 

conducted in the other studies is another major issue. At the very least, E. coli should have 

clearly been measured by MF methods for the NEEAR Great Lakes Studies. EPA could 

create a section in this criteria document that is specifically for freshwater systems, and 

clearly delineate for the reader what will be accepted, and also clearly delineate that there 

are E. coli qPCR assays that exist for use in recreational water quality. EPA must also 

clearly state that they have not conducted the science to update the E. coli criteria, therefore 

those criteria remain the same as previously suggested in the 1986 document. 

Third, the presented analysis of the relationships between FIB and sources of contamination 

in coastal and non coastal waters is poorly written. It is this reviewer’s opinion that there are 

three general sets of data a) marine, b) freshwater-great lakes, and c) inland lakes, rivers and 

streams. EPA does not have the inland waters data to compare at any significant level to the 

coastal and Great Lakes data, it simply does not exist. EPA is only confusing the issue with 

the weakly presented analysis. Furthermore, EPA does not describe what portion of the 

assembled literature was used for their assessments in this section, only refers to Appendices 

A and B broadly. Without presenting any of the assembled information, where it came from 

and how it was preferentially selected from the large body of literature, about waterbody 

type and sources, FIB occurrence, FIB survival, and FIB associations with pathogens, the 

reader is left with a poor description of the summary of this huge literature review. It is this 

reviewer’s opinion that EPA must either completely redo this section, or they must state 

clearly what articles are being used to summarize information and present it specifically 

and quantitatively. For example, there are vast differences simply between decay rates in 

seawater and freshwaters, how can EPA state that these differences are not due to the water 

matrix, but due to site-specific differences? 

Fourth, EPA has preferentially selected a few epidemiological studies to highlight in the 

criteria document, on the freshwater side Epibathe, Ferley et al., Weidenmann et al., and 

Marion 2010. They have also highlighted information about sources of fecal 

contamination in relation to human health outcomes (specifically stating that many epi 

studies of NPS contamination have found weak or inexistent relationships with FIB). EPA 

speaks to relationships between E. coli and human health, but this information is sporadic 

and not presented with any consistency or clarity. EPA must clearly state that the previously 

stated studies (listed above) have found significant relationships to E. coli in freshwater, and 

that EPA has not conducted the necessary science to change the E. coli criteria substantially 

from 1986. EPA need to clearly state the need for E. coli based water quality assessments 

and epidemiological studies to be conducted so as to update the science in a similar way to 

what has been accomplished with Enterococcus, including the implementation of already 

available rapid methods (Lavender and Kinzelman, 2009). 

Yasvinski No. The data analysis and the literature review conducted for the multiple components the 
Reviewers’ Comments by Charge Question 

B-21 

criteria document considers (e.g., epidemiological studies, QMRA and other publications in 

the appendices) are very comprehensive and appear to be sufficiently exhaustive. I feel 

comfortable that the body of information has been suitably reviewed. The odd paper could 

be included to provide reference for specific points, as other reviewers have indicated, but I 

believe that no crucial streams of evidence have been overlooked. Two general comments I 

would have are as follows: 

1. It is proposed that better linkage of individual points or conclusions to the respective 

Appendix subsections would be extremely valuable and help improve transparency. 

2. It is also felt that an important opportunity to address the concept of the move 

worldwide towards a harmonized approach to risk management and the move away 

from reliance on numerical values of fecal indicator bacteria as the sole criterion for 

determining recreational water safety was missed. Although it is recognized that this is a 

criteria document and not a beach management document, the concept is in-line with 

EPA thinking. It is therefore important to highlight and ingrain this idea in the public 

consciousness and to also use the opportunity to point out the interconnectiveness 

between related EPA documents (e.g., sanitary survey and predictive model documents) 

to authorities and managers. 

Suitable citations for the Harmonized concept include: 

Microbiological Water Quality Guidelines for Marine and Fresh Water Recreational Areas. 

New Zealand Ministry for the Environment. 2002 

EU (2006). Official Journal of the European Union. Directive 2006/7/EC of the European 

Parliament and of the Council of 15 February 2006 concerning the management of 

bathing water quality and repealing Directive 76/160/EEC 

NHMRC (2008). Guidelines for managing risks in recreational water. National Health and 

Medical Research Council of Australia, Government of Australia, Canberra. 

Health Canada (2009). Guidelines for Canadian recreational water quality. (Draft) 

Document for Public Comment. Water Quality and Health Bureau, Healthy 

Environments and Consumer Safety Branch, Health Canada, Ottawa, Ontario. 

Available at: http://www.hc-sc.gc.ca/ewh-semt/consult/_2009/water_rec-eau/draftebauche-eng.php 
Reviewers’ Comments by Charge Questions 

B-22 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based indicator 

data aggregated by days of similar water quality corroborate that the NEEAR culture 

data are generally consistent with the 1986 regression models? 

Reviewer Comments 

Dorevitch It’s unclear why this issue is so prominently discussed in the draft AWQC. The document 

(Section 3.2.4) seems to frame the discussion of the 1986 numeric values vs. the NEEAR 

numeric values for indicators by culture as a “confirmation” of the 1986 numbers. The 

rationale for the comparison is never made clear: lines 853-855 state that this was done “to 

provide a comparison” – but why? 

Perhaps the intention is to establish that the qPCR results of the NEEAR study were 

developed in settings and with participants that generated a similar culture-health risk 

relationship as the Cabelli/Dufour studies. If this is the case, that should be made clear. 

However, the methods used to make this point are poor substitutes for analyzing both data sets 

(Cabelli/Dufour and NEEAR) using the NEEAR approach (which addresses individual-level 

confounders). Support for the comparability of the Cabelli/Dufour studies and the NEEAR 

study can be found in Table 6 of Wade et al., 2008 and Table 6 of Wade et al., 2010. This 

information (or these tables) could be presented in the RWQC in support of utility of the 

1986 criteria values for enterococci; for justification of the continued use of those criteria 

values; and for demonstrating some comparability of findings of the NEEAR and 

Cabelli/Dufour studies. 

If EPA were to nevertheless decide to include the re-analysis of NEEAR data using the 1986 

approach: 

I have significant concerns with the interpretation of Figure 1 (page 26) provided in lines 

897-899. To say that the plotted datapoints from the NEEAR study are “generally fell within 

the predicted range” of the earlier studies is somewhat deceptive. The datapoints from the 

NEEAR study, particularly the freshwater (Figure 1a) study, suggest a relationship that is 

inverse (a line from about 20/1,000 swimmers on the Y axis through log10 entero=2 /100mL 

on the x-axis and that provides some fit for the NEEAR data). A less convincing case can be 

made for marine water (draw a line from about 5/1,000 swimmers on the Y axis and log10 

entero=2). NEEAR results are “in the predicted range” but are inverse, suggesting that the 

NEEAR results are NOT consistent with the Cabelli/Dufour results. The use of data this 

aggregated, essentially ecologic, is problematic because of the ecologic fallacy: that the 

individuals who got sick at beaches with a seasonal average of “good water quality” were 

swimming in water with good water quality – perhaps those who got sick used the beach on 

days of poor water quality. It may be best to avoid presenting this analysis. If it does remain 

in the RWQC, the statement (lines 898-899) that “the NEEAR health and water quality data 

are consistent with those statistical models” must be deleted, though the following sentence 

“this analysis proved to be insufficient” – should be retained. As it stands, it’s problematic 

that the approach is insufficient yet a conclusion has been drawn. 

The “binning” approach is certainly better than the “beach season” ecologic approach. The 

binning approach, however, does not produce results that, as noted in lines 921-924 

corroborate “the previous conclusion that the NEEAR data were generally consistent with the 

1986 regression models.” Again, like Figure 1, it is not especially meaningful that the 

datapoints fall “within the range” of the 1986 datapoints. If both the 1986 and the NEEAR 

data had similar slopes the findings would support what EPA wants to say. However, the 

1986 data do suggest a significant positive slope for both freshwater and marine water while 
Reviewers’ Comments by Charge Question 

B-23 

there does not appear to be a slope in the NEEAR data. Likewise, Table 2 does not suggest 

within-beach trends. Were tests for trend done on those results? Either way, the 1986 

method did not adjust for confounders as the NEEAR study did. It is possible that 

associations observed in the Cabelli/Dufour studies would be different if adjustment were 

made for recent exposures, digging in sand, etc. 

The discussion on page 28 and Figure 3, again, are problematic. Was a test of statistical 

significance done for the slopes of the NEEAR marine and freshwater regression lines? I 

suspect that such slopes may not be statistically significant, given the wide confidence intervals 

around the slope and the limited number of datapoints for a relatively small slopes (the 

freshwater slope is almost 0). 

Additionally, the statement (lines 947-948) that “...the recommended RWQC values are 

similarly protective of public health in both marine and freshwaters” is based on an 

assumption that the slopes and intercepts of the freshwater and marine indicator-HCGI 

relationships are similar. Since there seems to be no slope to the freshwater line, and perhaps 

no statistically significant slope to the marine line, this statement does not appear justified. 

Furthermore, the wide confidence intervals around the slope noted in association with the 

1986 estimates of risk (line 943-947) seem to be quite wide here (Figure 3). If there really is 

not slope, water quality measures are not needed to estimate the level of risk, which simply 

would be the mean level of risk at all levels of water quality. 

One of the intended purposes of this section may be to identify a comparable level of target 

risk for establishing qPCR criteria. If this is the case, the section should be re-written to 

make that clear. 

It may be worth emphasizing that: 

1. Analysis of the NEEAR culture data by the methods used to establish the 1986 criteria 

show comparability of the culture data-health risk association in the NEEAR and the 

Cabelli/ Dufour studies (at least when assessed using the dichotomous “FIB above the 

thresholds of 33 and 35 CFU/100mL) (see above-noted reference to Table 6 of Wade et 

al., 2008 and Wade et al., 2010. 

2. Presenting the relationships of enterococci CFU vs. health risk generated from the 

NEEAR study using the same approach to multivariate modeling that was done for 

qPCR would be better than using the 1986 approaches. If that relationship is weak, 

potential responses to that observation include: 

a. Conclude that the qPCR models are better predictors of illness and therefore the 

method should be mandatory, rather than optional. Enterococci CCE criteria values 

should be those associated with a targeted level of risk of 8/1,000, same as the 1986 

freshwater criteria. The need for harmonizing the targeted risk levels has been 

emphasized many times previously (Airlie workshop, Boehm 2009). The use of the 

more protective target (freshwater rather than marine water) would be preferred 

from a public health standpoint. 

b. Present data similar to that found in Figures 1-2 of Wade et al., 2008 and Figure 2 

(right panel) of Wade et al., 2010. If in fact the slopes of these lines are similar in 

marine and freshwater, and if the slopes are similar for adults and children, the 

information can be combined into a single graph. A enterococci CCE criteria value 

providing the same level of protection as the culture values can then be selected by 

using 8 cases of AGI (or the NGI equivalent)/1,000. 

c. Given the limited precision of expected risk for given microbe density (the wide 
Reviewers’ Comments by Charge Questions 

B-24 

confidence intervals and the small slope, the relationship between enterococci CCE 

and NGI may be too weak to reasonably differentiate qPCR CCE values to that are 

associated with, for example, 8 or 10 or 12 or 14 cases /1,000. This would suggest 

that the selection of a single targeted risk level cannot be supported. In this case, 

ordinal categories may be more appropriate. This could take the form of the CCE 

values corresponding to risk of <8 cases/1,000 could be considered “green”; 8 to 

(arbitrarily) 12 could be considered “yellow” and >12 could be considered “red” (as 

examples). The use of graded categories defined by ordinal ranges of risk is 

probably a better reflection of the science (i.e., the limited precision in predictions 

of health risk based on microbe density or vice versa) than the use of discrete 

categories dichotomized by an imprecise estimate water quality for a given level of 

targeted risk. 

Lipp When aggregated by days of similar water quality, the data do support consistency with the 

1986 criteria; however, given the wide variability among data points the following issues 

should be considered: 

Line 897-899. In attempting to develop a quantitative mechanism to compare NEEAR data 

with the 1986 criteria based on culturable enterococci, different methods of data binning 

were used. This sentence refers to the results when the seasonal GM is taken for the seven 

beaches used in the NEEAR study. The report states that the data points generally fell within 

the predicted range of the 1986 regressions. This is a rather optimistic view of these points 

(as shown in Figure 1). For freshwater, the NEEAR data points (4) actually look to be 

negatively correlated with HCGI rate and no relationship is noted in marine water (3 data 

points). While they do technically fall within the confidence intervals of the regression this 

is not a robust measure (which is also acknowledged in the report) especially in light of the 

few data points. Binning by water quality levels provided a better point for comparison (as 

shown in next paragraph in the report). 

Line 950. The trends lines for these data seem particularly weak, especially for freshwater 

beaches (which in previous figures seemed to have a stronger relationship than marine 

water). The data in general seem to show a very high degree of variability. 

Line 961 - 968. Report states that results from marine and freshwater studies are sufficiently 

similar to allow combining in the new criteria. I do wonder if they are sufficiently similar 

only because all of the data are so highly variable (there seems to be a large spread); 

however the regression using the combined freshwater and marine data is stronger than 

when the water types are separated. The marine data points (alone) show an almost vertical 

regression line (appendix B.5) where a very small change in CCE results in a large change 

in # GI/1000. 

It is also not entirely clear why the NEARR data must be corroborated by 1986 analyses, 

given the difference in study design. 

McBride • This is a crucial issue, because this analysis has enabled the adoption of the same 

numerical limits on culturable fecal indicator bacteria as currently apply (under the 1986 

criteria). The general approach taken seems sound, but I concur with the caveats made by 

the Review Panel's responses to this question. I presume that EPA staff will urgently 

promote a peer-reviewed publication of this important analysis? 

• A particular feature of this analysis is seen by contrasting Figures 2a and 2b. In Figure 2a 

the "1986 freshwater data" and the "NEEAR freshwater data" are positioned similarly 

(with respect to the X and Y axes). In contrast on Figure 2b we see that the "1986 marine 
Reviewers’ Comments by Charge Question 

B-25 

data" and the "NEEAR marine data" differ in that the former tend to exhibit higher HCGI 

rates. Some explanation of this contrast would be helpful, because it is the cause of the 

reduced estimate of HCGI risk associated with the marine criterion. 

• It may be appropriate here to recall some history of the development of the 1986 criteria, 

especially for marine waters. In 1984 USEPA proposed an "acceptable risk" of 6 HCGI 

cases per 1000 swimming events and advertised that notion (see Federal Register 

49(102), 21987–21988 May 1984). Furthermore, in the FR, EPA used the Cabelli HCGI 

relationship with that illness risk giving rise to a limit of 3 enterococci per 100 mL 

(interpreted as a GM limit). Later, after various submissions pointed out that such a low 

limit was not generally attainable, the idea was raised that since a GM limit of 200 fecal 

coliforms per 100 mL had served well, let us translate that limit back to enterococci 

(giving 35 per 100 mL) and then read the Cabelli graph in the reverse direction to 

calculate the associated risk (19 per 1000 swimming events), as explained in Federal 

Register 51(45), 8012–8016, March 1986. That is, "acceptable risk" was not selected, it 

was calculated. That stance has now changed in the proposed revisions, in that an 

acceptable risk level has now been found to be consistent with both the freshwater and 

the marine criteria limits. Presentation of this change in the RWQC might be helpful. 

Noble EPA has made an attempt to link across studies in a transitive argument, and they present 

one way of examining the information. However, issues exist with the following that 

deserve to be explained. 

1. Models used in 1986 are different from the models used for the NEEARS studies that 

include confounding factors. EPA should be transparent as to why the modeling 

approaches are different (assume that raw data from 1986 is not available?) and explain 

clearly why they are using the cross walking approach given this reason. EPA could be 

more proactive in stating that the criteria development is linked to protection of public 

health. 

2. Few data points make it difficult to assess the similarities across studies (Is it possible to 

remove the seasonal aggregation presentation?), there are other possible ways to present 

and analyze the data, why has EPA chosen only these examples? 

3. It is generally unclear to the reader what final data was combined data and what final data 

was specific to a study or beach? Also, it is unclear what modeling approaches were 

used to generate the health risk assessments for each specific study. 

4. Why no presentation of Bacteroidales information? Can it be used as a tool in Section 

5.0? This indicator may have particular relevance in tropical waters, where regrowth of 

E. coli and Enterococcus has been shown to be a serious issue (e.g. Hawaii). 

5. It would help the reader to see information from Wade et al., 2008 and 2010 showing 

relationships to human health above and below specific standards. 

6. No mention or incorporation of SCCWRP studies makes it difficult to understand what 

selection process was for the epidemiological information? How do the SCCWRP 

results relate to the marine NEEAR studies? Are there large differences in rapid method 

and culture method results? How about presenting a table, for transparency, that shows 

what studies individually showed a relationship to human health for what methods and 

what studies didn’t? In relation to EPA conducted studies, the SCCWRP studies are 

some of the most extensive, and they specifically followed EPA experimental design, so 
Reviewers’ Comments by Charge Questions 

B-26 

the lack of inclusion of this information is curious. 

7. EPA does a poor job of explaining why they have included certain studies in their 

considerations and not others, is it possible that these other studies generated 

information that make this aggregation of data and cross talk across criteria 

establishment confounded? It generally appears that the EPA studies only have been 

included in the final criteria development. While this is a decision for EPA to make, it 

needs to be justified and made clear throughout the document. There is a strong 

comparison between the Marion et al. study and EPA studies in the Appendices, but it is 

not clear why that kind of comparison is not conducted with an array of recent rigorous 

studies conducted by others. 

Yasvinski Not quite. My initial comments to this question were to propose that the findings that 

prompted the decisions of equivalency needs to be illustrated further, through a combination 

of graphical data showing the marine and freshwater data combined and an indication of the 

statistical calculations showing no statistical difference. Further to this, a valuable 

discussion would be as to how although the water quality data at the beaches was shown to 

be very good, it was felt that there was sufficient data captured at points where swimmers 

would be at elevated risk so as to make the data sets and observed relationships robust. 

EPA’s desire for pursuing unified illness rates for fresh and marine waters for public health 

protection and for application and implementation purposes is understood. At the meeting 

the review panel discussed that Figure 1. could be regarded as doing a disservice to the 

argument of marine/freshwater equivalency, but since it is addressed as being insufficient to 

refine the illness estimates, it’s inclusion was considered valuable for transparency purposes. 

The reviewers further addressed that in the absence of a suitably-illustrated supporting 

discussion, the apparent lack of relationships in Figure 2 questions whether the NEEAR data 

and the 1986 data are sufficiently similar. Ultimately the review panel suggested (and I 

support) that the NEEAR study data be used on its own be used to determine conclusions 

regarding fresh and marine water illness rate similarity. 

Guidelines or criteria can have several components beyond the straight scientific data 

component. These additional components can also impact or shape the way the final 

recommendations are presented: 

• Historical component 

• Public health decision component 

• Jurisdictional component 

• Practicality of Implementation component. 

In my experience on the work done in developing the (draft) recommendations for the 

Guidelines for Canadian Recreational Water Quality, the current fecal indicator 

recommendations were similarly based on assessment of the best available scientific data. 

However, because of the data limitations, and the lack of an exact parameter-to-illness 

causative relationship, the recommendations are not identified as illness thresholds, but 

rather as risk management values intended to be protective of public health. 

Whatever EPA’s decision regarding a tolerable illness rate and marine-freshwater rate 

equivalency – whether it be weighted heavier or data relationships or on policy/risk 

management/public health decisions – it is recommended that it provided that it appears 

logical, transparent, acknowledges it’s limitations and fits within EPA’s scope of authority, 

it will be capable of being defended. If not, it will give the appearance of potentially hiding 

something, and therefore will likely be subject to criticism, whether it is warranted or not. 
Reviewers’ Comments by Charge Question 

B-27 

4. Do the characterization and explanation of the qPCR method reflect that of the scientific 

literature? If not, what additional information should be included? Is the derivation of the 

qPCR value using the EPA data scientifically supported and adequately explained? 

Please discuss why or why not. 

Reviewer Comments 

Dorevitch Given that the qPCR laboratory and calculation methods utilized in the NEEAR study were 

the ones that established the enterococci CCE-NGI risk association, I would be reluctant to 

see that method revised. Revisions to the calculation approach of NEEAR study water 

quality data that were found to improve health predictions within the NEEAR dataset would 

be valuable. Revised laboratory methods in isolation (that is, not linked to health outcomes 

data) would be difficult to assess. 

Lipp More data are needed to confirm the use of the qPCR (CCE) between labs and among many 

different water types. This work appears to be on-going but results are critical to fully assess 

the utility of this alternate indicator. Given the specific need for developing rapid methods to 

assess water quality (per BEACH Act and lawsuit) data are needed to provide states with 

specific recommendations about implementing this tool, along with resources for set up and 

training. Specific questions/issues are noted below: 

Line 414-415 (and elsewhere). Given the variability in water types both within and between 

beaches (over space and time), it might be expected that PCR failures (inhibition) will 

happen with some regularity. There needs to be specific guidance for how to handle these 

occurrences (especially where some beaches routinely face this problem). 

Line 458-464. While qPCR methods (CCE) show excellent promise as a rapid tool for 

measuring water quality the limitations in validation of this method are cause for some 

concern. More work is needed to provide guidance to states on how best to implement and 

trouble shoot this approach 

Line 1116-1130. What happened to qPCR for Bacteroidales? Will this be an option for 

states with marine beaches if they choose to adopt another target? 

Line 1613. The guidelines for states on determining the performance characteristics of the 

rapid (qPCR) methods will be critical for successful implementation. Has any headway been 

made on drafting such a document? 

Apppendix A.2 Single lab validation of EPA method A (qPCR for enterococci) 

• Variability in the copy number of the 23S rRNA gene between species and over the 

course of a growth cycle is a potential problem in correct quantification using this 

method. Currently, EPA is using a copy number of 4 (reported for E. faecalis) as a 

reference point. Is any work anticipated to determine the broader range of copy numbers 

in other species (e.g., E. faecium has 6 copies)? If so, and average copy number could be

used for quantification. 

• In this single lab validation, the recovery was very consistent among samples in PBS but 

seems to have a fair amount of variation in both marine and freshwater. Is EPA 

evaluating methods to improve recovery consistency? 

• It will be helpful to see results from multi lab validation. 
Reviewers’ Comments by Charge Questions 

B-28 

Appendix A.4. Parameters that affect performance of qPCR method for enterococci

• Study temperatures tested here are reflective of seasonal conditions in temperate areas, 

not tropical or subtropical (i.e., spring 15 – 17 C, summer 20 – 23 C, winter 4 – 8 C). In 

all seasons, these temperatures are cooler than would be expected in tropical/subtropical 

waters. Especially at the higher temperature scale (e.g., 30 C in the summer) may have a 

strong effect on persistence/decay of enterococci which could inform the use of this 

indicator in warm (hot) tropical waters. 

Appendix A.6. Suitability of indicators and methods for different CWA programs 

• The limit of detection information present in Table 2 is somewhat confusing. For qPCR 

(CCE and CSE) a 50% limit of detection is used. This a term generally used for 

evaluation of presence/absence data, not quantifiable results. While the data are derived 

from Chern et al., 2009, it is not clear in the Chern paper how the 50% LOD was derived. 

In most studies, the actually LOD varies with processing parameters but in general a 

theoretical limit of detection can be derived for an ‘optimal scenario.’ It is not clear why 

that approach was not taken here (for both culture and qPCR methods). 

McBride As this is not my particular expertise I defer to Panel colleagues on this question. 

Noble EPA presents a method that is highly successful to quantifying Enterococcus in water. The 

information, however, is lacking in several specific areas. 

1. QPCR criteria has been shown, but the linkage between this criteria for actual studies 

conducted recently and illness has not been shown. EPA needs to specifically show that 

QPCR relationships to human health in freshwaters are stronger than those for culture 

based methods. The reader should be presented with the actual illness rates that would 

have resulted in a range of EPA and non EPA studies if this selected criteria was 

retrospectively applied. As a matter of fact, it would be useful to see this done for every 

epidemiological study conducted recently that utilized Enterococcus qPCR. 

2. The scientific literature is full of a range of quantification approaches for qPCR. The 

rationalization and explanation for the selection of the CCE or calibrated cell equivalent 

quantification approach for QPCR is not adequate. The lack of inclusion of the results of 

the SCCWRP epidemiological studies is vital in this rationalization, along with other 

work that has been conducted in the Great Lakes using E. coli QPCR assays. EPA is 

missing a valuable opportunity to explain this to the public/stakeholders. They also must 

present their epidemiology data in both the cell equivalent and the calibrated cell 

equivalent format, in the context of inhibition, and at least let the reader see that there 

are serious advantages and disadvantages to each approach. There are other general 

approaches (using the CE approach with inhibition flagged) to manage beach waters that 

have been presented by the State of CA, and while these are mentioned they are not 

explained. EPA should present this information and be transparent. EPA must address 

these issues and present a sequence of rational arguments for selecting the calibrated 

cell equivalent approach. 

3. EPA presents the method, but provides almost no guidance as to how others can use 

similar methods, like IMS ATP, are there possibilities for methods like IMS ATP? 

4. The presentation of method specific details is limited to Appendix A, where EPA 

quantifies cross laboratory variability, and 50% limit of detection. EPA must provide a 

better example of what type of variability they expect from rapid methods as compared 

to culture based methods. This is vital when considering posting a beach based upon a 
Reviewers’ Comments by Charge Question 

B-29 

single suggested value. Possibly a section that compares the methods and shows a 

shortened version of the methods comparisons that have been done and the variability 

observed. 

Yasvinski As this is not my specific area of expertise, and others on the review panel are more 

qualified than I to comment on the accuracy and appropriateness of the methods, I will 

withhold from commenting on this particular area. However, in terms of the derivation of 

the qPCR value, this does not quite feel adequately explained. Illustration showing 

comparison of NEEAR data to the calibration curves with explanation of how the 

relationships observed are felt to be valid would be helpful. 

Additionally, an expanded explanation to the reader as to why the qPCR criteria seem 

considerably elevated relative to the culture criteria, and how these are logical and are 

justified, would be useful. 
Reviewers’ Comments by Charge Questions 

B-30 

5. Please provide any other general or specific comments on the document, making sure to 

identify the section number/page number you are commenting on. 

Reviewer Comments 

Dorevitch Some suggestions for organization of the document: 

• Start with an executive summary that highlights: 

- The CWA and the background for reviewing/revising AWQC/RWQC 

- NEEAR study (# beaches, participants, years) 

- What stays the same (no changes to culture method criteria values) in relation to current 

requirements 

- What changes are proposed 

One huge change: inland waters, 

One small change: the option for data collection to support future use of qPCR 

The Section 5 provisions 

- How the 2012 RWQC can improve public health (rapid notification, more waters now 

covered/protected). 

Has EPA estimated the number (or percent) of days of beach use that, following application 

of rapid methods, will be appropriately closed when prior day culture methods would leave 

the beach inappropriately open (or without an advisory/notification)? 

Has EPA estimated to number or inland water recreators (or person-days of recreation) that 

for the first time, fall under the protection of the RWQC? 

Miscellaneous comments/edits/corrections: 

Lines 309-310: Consider revising “State of the science review on occurrence and crossinfectivity of specific pathogens associated with animals.” To “State of the science review 

on the occurrence and human health risks of specific waterborne zoonotic pathogens.” 

Lines 342-343: Consider revising “...considered all of the indicators of fecal 

contamination...” to “...considered indicators of fecal contamination” or better still, 

“...considered indicators of health risk...” 

Line 358: Consider revising “These particular FIB are not pathogenic...” to “These 

particular FIB are generally not pathogenic...” (since some E. coli are pathogenic). 

Line 359: Again, the previously noted suggestion that over-emphasizing FIB as pathogen 

indicators rather than health risk indicators, given that the document elsewhere notes that 

FIB are poor indicators of pathogen presence. 

Lines 366-373: Text is confusing. Consider first explaining that the membrane filtration 

method generates CFU/volume while the defined substrate method generates MPN/volume. 

The defined substrate method does not involve counting colonies, but rather relies upon 

probabilities. Neither method includes counting of microbes in a water sample, but rather, 

generate results following culturing those microbes for 18-24 hours. 
Reviewers’ Comments by Charge Question 

B-31 

Line 383: Change viable to “culturable” as bacteria in the VBNC (viable but not culturable) 

state would not be counted using culture methods. 

Line 390: Consider changing “methods in water” to “methods in a specific waterbody or 

beach” 

Line 401: Suggest that there’s an qPCR more recently developed (? since Method A was 

published?). Consider deleting the word “more.” 

Line 418: Change both uses of “incidences” to “instances” 

Line 419-422: This isn’t definite. Chern 2011 suggests that the SPC and target DNA have 

similar amplification efficiencies. 

Lines 440-447: Why not refer to this as the ddCT approach? 

Lines 471, 487: FIB “occurrence” sounds like FIB “presence.” Since FIB are essentially 

always present, albeit sometimes in low concentration/density, “occurrence of FIB” isn’t 

informative. Consider changing to “density” or “concentration.” 

Line 517: For the sake of completeness could add that the 1986 differed for marine and 

freshwater in part because of the different levels of “acceptable risk.” 

Line 578: Consider revising “actual increase in the burden of illness swimmers experience” 

to “actual increase in the incidence of illness among swimmers” (as “burden” is more 

general and could refer both to the incidence rate and the severity of illness). 

Line 596: consider changing “an eye infection” to “a self-reported eye infection”. Similarly, 

Line 598: consider changing “wound that became infected” to “wound that was reported to 

have become infected” In both cases, it may not be clear that “infection” was based on selfreport, rather than diagnosis by a physician or other health care provider. 

Lines 600-610: Not really. It would be true that that a larger percent of epi study participants 

experiences GI illness than other health outcomes. However, that’s no reason to assume that 

preventing GI illness would also prevent non-GI illness. It might, but “a similar level of 

protection” implies that a similar proportion of GI and non-GI illnesses would be prevented. 

We don’t know if that’s true. It might be better to state that NEEAR and previous epi studies 

found that the health outcome with the highest incidence was GI illness. Preventing GI 

illness may or may not result in a comparable reduction in health outcomes, but it will 

prevent the most type of recreational waterborne illness. 

Line 635-626: Consider revising to “...RWQC because neither possible relationships 

between these microbes and FIB, nor relationships between FIB and these health outcomes, 

have been the subject of observational studies.” 

Line 651: Change “incidences” to “incidence” 

Line 670-671: Revise “time issues” to “short-term (hours) variability” 

Line 689: Suggest changing “particular sensitive subpopulations” to either “potentially 

sensitive subpopulations” or “particular subpopulations” (since in the end, there wasn’t 

sufficient evidence to support that assumption that children are in fact “sensitive” 

Lines 715-716: change “rate of increase in risk per unit increase in enterococci” to “change 

in risk per unit change in enterococci”. Also: This analysis (p=0.44) is unpublished but has 

substantial policy implications. Consider elaborating on the source of this info and if 
Reviewers’ Comments by Charge Questions 

B-32 

possible, the status of the EPA report or the publication status (submitted, in revision, etc).

Lines 739-741: As written this suggests that Wade 2010 evaluated the use of Bacteroidales 

in marine and freshwater and found that indicator to be associated with GI illness in marine 

waters only. Because Wade 2010 only addressed marine waters, better to revise as “GI 

illness in swimmers at marine waters was also associated with exposure to levels of 

anaerobic bacteria of the order Bacteroidales enumerated with EPA’s Bacteroidales qPCR 

method (Wade 2010).” 

Lines 764-767: The very low indicator levels is likely an important (perhaps the most 

important) reason for the absence of a relationship between FIB and health at both Boqueron 

and Surfside. This information, rather than inhibition (which would not impact the 

enterococci culture data) should be noted. 

Lines 790-795: The end of this section states that E. coli levels “correlated with HCGI in a 

similar manner as in EPA’s 1970s” studies. First, the Ohio reservoir study reported an 

association with GI symptoms but not with HCGI. Second, the term “correlation” is used in 

several places in the RWQC to mean “association” or “statistically significant relationship”. 

This is one such example. The Ohio study reported the results of a multivariate regression, 

not a correlation analysis. 

Lines 810-813: Please justify that statement that a limitation of the RCTs is smaller sample 

size. From a practical standpoint, it’s actually an advantage that smaller sampler sizes are 

needed. The reason sample sizes can be smaller is that randomization should generate 2 

groups (exposed and unexposed) that are similar in all ways other than the exposure. 

Because the observational designs do not randomize, much larger sample sizes are needed to 

adjust for confounders. It could be noted instead that the most significant limitation of the 

RCTs is that participants who are randomized to “exposure” have been asked to immerse 

their head in water 3 times over a short duration (10 minutes). This is not swimming. It may 

result in rates of water ingestion that are greater or less than swimming, but it is not 

swimming. There is an assumption that the risks of GI illness in the RCTs are the risks 

associated with swimming; they are the risks associated with head immersion. 

Line 832: Change “swimmers” to “those randomized to head immersion.” 

Related: Lines 1275-1294 refer to Epibathe. This was a series of studies that evaluated the 

health risks of controlled exposure (head immersion), not swimming. Any conclusions about 

the relevance of Epibathe to EPA criteria development should note that Epibathe did not 

evaluate the health risks of swimming, but rather, head immersion (3 times). 

Lipp The text should discuss the notion that FIB are used in practice as indicators of health risk 

and not specifically correlates of pathogens. There are many papers showing no correlation 

between FIB and enteric viruses, or other pathogens. However, understanding the 

relationship of FIBs to pathogens will become more important if using a QMRA approach. 

This topic would be best discussed in the QMRA section. 

Line 675. The summer season focus makes sense for temperate waters but in tropical waters, 

where beaches are used year round, epi studies done at other seasons may be important. 

While it is speculative, high water temperatures and UV irradiation during the summer 

tropical beach epi study might help to explain the very low numbers of FIB observed. 

Line 1064. It should be made clear that this statement is referring only to freshwater studies. 

Line 1135. The first line of this paragraph states that this RWQC are national 
Reviewers’ Comments by Charge Question 

B-33 

recommendations for all surface waters in the US, but data reported here and in the appendix

does not support this. Ongoing questions remain about the use of this criterion for nonPOTW impacted beaches and inland waters and tropical waters. This should be 

acknowledged and special considerations given for these areas in section 5 (for 

implementing site specific criteria). 

Line 1189-1191. Report states that based on the review of literature and the overall site to 

site variability of FIB concentrations that the RWQC will be applicable to both coastal and 

non coastal sites. But, this is in contrast to the EPA study (Appendix C.4) that states that 

epidemiological studies to date do NOT support extending these criteria to inland waters 

NOT impacted by POTW discharge. 

Line 1307. This statement is not consistent with findings reported in Appendix C.4 (see 

above comment on same issue) 

Line 1428. While based on QMRA (Appendix C.3) and other studies, inputs from animal 

feces can result in similar risks as human fecal contamination, it should be pointed out that 

not all animals present the same risk levels (e.g., Appendix C.3 - pigs and poultry showing 

lower risks) 

Line 1747-1751. The development of these models and software for model development are 

a great addition to allow rapid assessments; however, while the Great Lakes Beaches models 

have been successful it is important to note in this document that the results for marine 

beaches are currently inadequate. This likely reflects the fact that the models are predicting 

FIB levels, which tend to be lower at marine beaches, even with health effects are noted. As 

any work been done to adjust these tools to predict health effects directly rather than the FIB 

levels? 

Line 1893-1900. Given the inconclusive results of the non-POTW impacted beach and 

tropical beach, is the EPA anticipating that states/regions address these on a short time 

frame? Does the EPA anticipate providing any assistance for these regions? This is a critical 

gap in knowledge. 

Line 1986-1997. Does the EPA plan to update or offer technical guidance to the states on 

this issue periodically? It seems that EPA might be in a better position to offer new 

methods/technology options with appropriate QC behind the method. 

Appendix B.4 is identical to Appendix B.1 

Typos: 

Line 257. There is a typo. WERF is Water Environment Research Foundation (not “experts” 

as written)

Line 316. The word ‘Water’ is missing from National Epidemiological and Environmental 

Assessment of Recreational ...

Line 848. Typo .. “if” should be “of” (“...terms OF indicator density ...”) 

Line 885. Typo, ‘trail’ should be ‘TRIAL’ 

Line 1165. Typo. There is a word missing from this sentence or the structure is awkward. 

“Loadings and hydrodynamics of FIB in human POTW coastal and non-coastal waters ...” 

Human POTW? 
Reviewers’ Comments by Charge Questions 

B-34 

Line 1205-1207. “salinity’ should be removed from this sentence given salinity would have 

a lot to do with whether a beach was coastal or non-coastal (e.g., coastal marine beaches will 

be saline and non coastal beaches will be fresh) 

Line 1251. Typo. Remove the word “was”. Sentence should read “...a retrospective study in 

the French Alps ...” 

McBride Line Comment 

183 "...recommended fecal coliform criteria..." should be "...recommended log 

mean fecal coliform criteria..." 

254–

255 

With regard to this text: "These projects included epidemiological studies to 

provide data correlating illness with indicators, site-characterization studies to 

facilitate QMRA...": To do that the epidemiological studies would need to assay 

for pathogens, not just indicators—QMRA uses dose-response relationships for 

pathogens. I understand that this has not been done. 

269 I note that these QMRA studies are for impacts of agricultural animals. So 

inferring from these that wildlife poses similar risks may lack sufficient 

foundation (but I do recognize that in the absence of studies on wildlife a 

precautionary approach may be very desirable from a public health viewpoint). 

498 While true that the 1976 criteria (in USEPA Quality Criteria for Water, July 

1976) recommended the same numerical limits as did NTAC, it may be helpful 

to note that the rationale for these limits changed (from the earlier USPHS 

studies of Stevenson 1953, as interpreted by NTAC), to an argument based on 

minimizing Salmonella infections (as is clearly stated in that text). 

524–538 What is meant by "the" observed GM enterococci and "the" observed fecal 

coliforms? Is it for the combined studies mentioned on line 514? 

557 The word "because" is misleading here. It should be "in that". The paragraph 

doesn't explain why the changed definition was adopted. Maybe it should? 

603–604 Maybe note that the McBride et al. study found respiratory illness rates were 

associated with the concentrations of enterococci but failed to detect an effect 

with gastro-intestinal rates. (So it shouldn't be cited in support of the opposite 

pattern, as has been done on line 603.) Also Fleisher et al. (1998) reported 

"Attributable proportions of illness" as 34.5% for GI but a higher value 

(40.0%) for AFRI. So is the text on these two lines correct? 

716 Hypothesis tests of no difference should not "accept" that hypothesis when the 

p-value exceeds the significance level. It suffices to say that differences were 

not statistically significant (there always is a difference between two 

populations, so it makes no sense to say that there isn't). Null hypothesis test 

procedures are devised by assuming that the null is true, and only reject that 

hypothesis if data are sufficiently convincing. On logical grounds that 

procedure doesn't admit an inference that the null is true (or even likely to be 
Reviewers’ Comments by Charge Question 

B-35 

true). 

837–843 The Corbett study did detect an association with respiratory illness however. 

969 Figure 4 doesn't say which GI category is being graphed; it should (i.e., NGI). 

977 Wymer et al., 2011 is Appendix B.3; that needs to be clarified. 

1355 Not sure why there's an emphasis on outbreaks here. The majority of the 

human disease burden manifests as endemic or sporadic patterns cf. outbreaks. 

And, in particular, campylobacteriosis seldom appears as outbreaks yet can 

have a high endemic rate (that's certainly the case in New Zealand, Till & 

McBride 2004). 

1411–

1412 

I don't understand this text: "The few studies conducted in non-POTW 

impacted waters that also report significant health effects (McBride, 1998; 

Cheung, 1990; and Wiedenmann 2006) have ... 2) combined epidemiological 

data from beaches with non-human fecal source impacts with data from 

beaches impacted by human fecal contamination sources." The New Zealand 

study did have beaches with human impacts and beaches with animal impacts, 

but the analysis did consider them separately (as well as combined). 

1428 Suggest replace "Thus, waterbodies with substantial animal inputs" with 

"Thus, waterbodies with substantial farmed animal inputs". Note too that 

"McBride, 1998" (lines 1409 & 1428) should be "McBride et al., 1998". 

1460–

1461 

"The GM is calculated as the arithmetic mean of the log-transformed 

densities" should be "The GM is calculated as the antilog of the arithmetic 

mean of the log-transformed densities". 

1471 Maybe elaborate a little more on what 40 CFR 122.45 is all about? 

1532 Is "log standard deviation" ambiguous? Some might interpret it to means what 

it says! Should it be the mean the standard deviation of the logarithms (to base 

10?) of the data? 

1835–

1838 

Is Colford's Mission Beach study an exception to this human-impacted 

category? 

1918–

1927 

Another use of QMRA is to assess "what if" scenarios. I argue that's an 

important use of it, one that epidemiological studies can't perform. 

1944–

1949 

Were any of the epidemiological studies conducted close to an outfall? If not, 

is there a case for advocating, say, QMRA in those circumstances (as is done 

in the New Zealand Guidelines (at pp. 3&4): 

http://www.mfe.govt.nz/publications/water/microbiological-quality-jun03/). 

Noble Sections 3.0, 3.1, and 3.2 are poorly written and need to be rewritten and reorganized. The 

sections on this history of criteria in the beginning of the document are much more clearly 

written and presented. In Appendix C: ASSESSMENT OF THE APPLICABILITY OF 
Reviewers’ Comments by Charge Questions 

B-36 

EXISTING EPIDEMIOLOGY DATA TO INLAND WATERS, there are statements that are 

contradictory to those written in Section 3.0+ of the criteria document. In particular, the data 

analysis is not possible given the range of methods that were used to conduct coastal versus 

inland waters assessments. The document clearly states the caveats associated with 

interpreting the data across sites and methods. EPA clearly states the variability observed 

from site to site at beaches, then there is a huge effort expended to show the linkage across 

coastal and non-coastal waters. It is simply not possible to state that coastal and non coastal 

waters behave similarly in relation to FIB dynamics. 

Specific comments: Use either Enterococcus sp. or enterococci throughout 

FIB = fecal indicator bacteria, typically in the literature, this refers to total coliforms, fecal 

coliforms, E. coli or Enterococcus. Where you mean FIB use FIB 

Fecal indicator and fecal indicator organisms (FIO) include all indicator organisms. Most 

typically use this as a catchall for all FIB, plus other “indicator” bacteria that can sometimes 

include other bacteria such as C. perfringens. It is this reviewer’s opinion that in the criteria 

document, in almost all instances seen the EPA should consistently be using FIB specifically 

If going to define acronyms for all, please use them. Specifically use FIB please, not “fecal 

indicators” which could be inclusive of enteric viruses, protozoa, chemical indicators of 

fecal pollution etc. 

EPA could please include: 

• A basic example of what constitutes an adequate side by side testing scheme for 

implementation of qPCR and if states have already done this, how they can “bypass” 

this step. 

• A QMRA case study, or at least an example of what would be adequate. 

• More information as to how spatial and temporal sampling frequency plays into GM 

and STV. There is little mention of how EPA selected to reduce (?) the required 

sampling frequency for STV (12 for 90 days as opposed to 5 for 30 days?), what 

information has been assembled to defend this? 

• EPA needs to at least discuss frequency of beach closures in a reference beach vs. 

contaminated beach context in terms of impairment listing, and they need to defend 

their approach, or specifically accept other approaches that have been scientifically 

and rigorously established, also need to defend the use of mixed information in 

impairment assessment, can an agency mix and match QPCR and culture based data 

for STV assessments? What is the reasoning for accepting this approach? Not 

accepting this approach? 

• Are all E. coli GM and STV assessments to be conducted the same as Enterococcus? 

• What about use attainability. There is very little information provided in this section. 

EPA has done a poor job in the past of describing how to establish tiered approaches. 

Now that they are doing away with this system, they need to address how this will 

impact states that are short on resources. States might find themselves in a situation 

where they are expending resources at a beach that has 1 user per month, and reducing 

resources at their higher use beaches. This is an undesired effect of the tiered system, 

but a reality in tight budget times. This reviewer is already aware of several states that 

have suggested that they will have to shift resources away from high use beaches to 

mostly unused beaches if the tiered system is abandoned. There needs to be a clearly 

thought out process and presentation of the information to stakeholders, as this will 
Reviewers’ Comments by Charge Question 

B-37 

impact those in the water quality agencies greatly. Are they expected to maintain all of 

their sampling sites that they currently sample with lower frequencies at the 90 day 12 

sample minimum? 

• EPA is to be commended for introducing new scientific tools such as QMRA in the 

possible arsenal of water quality managers. EPA could more adequately explain the 

QMRA process and provide examples (this was already included in other comments). 

One major problem is that the source identification and quantification science that 

needs to be done has not been done in an organized way. EPA should take the lead on 

this. With specific personnel at EPA (Orin Shanks) this work can be completed 

successfully, but there must be a plan toward doing it at a national level. EPA has not 

specified how agencies/stakeholders sift through available source identification and 

quantification approaches to select something that is rigorous and can be used 

successfully. Finally, the linkage to FIB is not clearly made in the criteria document or 

any guidance that EPA has published. Guidance on the linkage of source specific 

information specifically to either E. coli or Enterococcus has not been provided. This 

is a vital step toward the successful use of QMRA as an additive to a management 

schema for surface waters. 

• QPCR methods, EPA has not specified what is acceptable for equivalency of rapid 

methods (i.e., use of Method A with a different QPCR platform, how many side by 

side samples need to be run to demonstrate equivalency between methods?). This 

needs to be done now before other classes of methods arrive. Set the stage for other 

criteria development, be novel and thoughtful about this process. If want to use a 

different platform, specify the acceptable criteria. If want to use a different available 

tested method, define what is acceptable. It is assumed that much of this information 

will be in a Topic Specific Guidance, but some precursor information is desirable. 

Yasvinski File containing marked specific comments submitted with the draft comment file remains 

unchanged. 

C-1 

Appendix C 

Individual Reviewers’ Post-Meeting Comments 

C-3 

Samuel Dorevitch 

Visiting Associate Professor 

School of Public Health 

University of Illinois at Chicago 

Chicago, IL 
 C-4 

Samuel Dorevitch received his medical degree from the University of Chicago, completed 

residencies in Emergency Medicine and in Occupational Medicine, and received a Masters in 

Public Health Degree from the University of Illinois at Chicago (UIC). He is an Associate 

Professor in the UIC School of Public Health, Division of Occupational and Environmental 

Health Sciences. Dr. Dorevitch also has an appointment in the UIC Institute for Environmental 

Science and Policy. Dr. Dorevitch was the principal investigator of a cohort study of recreation 

on inland waters. One ongoing area of research is the use of molecular methods to rapidly 

identify and communicate the presence of hazardous conditions at beaches. 
Samuel Dorevitch 

C-5 

Peer Review of the Recreational Water Quality Criteria (RWQC) 

Comments Submitted by Samuel Dorevitch 

Introductory Comments 

EPA scientists and policy-makers, as well as those that assisted in the development of the draft RWQC 

are to be commended. The task they faced was monumental, in large part because of the tremendous 

amount of information about microbial measures of water quality and associated health risks that EPA has 

developed in the past 10 years. The National Environmental and Epidemiological Assessment of water 

Recreation (NEEAR) was a very sophisticated study that presented numerous implementation and 

analytic challenges. Furthermore, EPA researchers developed, refined, and published details of molecular 

methods that allow the timely measurement of fecal indicator bacteria. EPA’s research team succeeded in 

completing this ambitious study, and findings pertinent to criteria development have been published in 

peer-reviewed journals. 

EPA staff and consultants have developed draft “Recreational Water Quality Criteria” (RWQC), the 

subject of this peer review. Efforts were made to summarize a wealth of data used to develop both the 

1986 Ambient Water Quality Criteria, as well as the present draft RWQC. The peer reviewers commend 

the EPA team on their efforts to produce such a document in a timeframe that meets all that is required of 

EPA. The comments offered by the peer review panel are meant to assist EPA in its efforts to develop 

new or revised criteria that are grounded in science and can be implemented in order to protect public 

health while meeting all requirements of the Clean Water Act. 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? 

Conclusion 1: Coastal-derived criteria can be extended to inland waters 

This seems to be the most significant change proposed in the RWQC (since the qPCR method will not be 

a requirement). If epidemiologic data were available to suggest that coastal-derived criteria afford 

swimmers a similar level of protection in inland waters, justifying this change would be simple. Data 

regarding the health risks of swimming in inland waters are very limited; the most relevant published 

information probably comes from Marion et al., 2010 and the Keystone Lake, OK study (conducted by 

EPA more than 30 years ago). Given that this extension of RWQC to inland waters represents a change 

from the prior application of AWQC to Great Lakes/coastal waters, the reason(s) for this change should 

be clear and compelling. 

The Draft RWQC provides a justification for this change that includes a discussion of indicators, 

pathogens, sources, and cites several published studies, EPA reports, and other reports. This approached 

should be revised, as it did not seem particularly compelling, and some portions of the text did not seem 

to be consistent with other portions of the draft RWQC. 
Samuel Dorevitch 

C-6 

Some specifics: 

The EPA 2010 (g) report on the applicability of the Great Lakes NEEAR data to inland waters does not 

include a “conclusion” and the “Executive Summary” does not include an assessment of whether or to 

what degree NEEAR data could be applied to inland waters. Given the many differences between inland 

and Great Lakes/coastal waters summarized in that report, it’s not clear what serves as the basis of EPA’s

determination (in the AWQC document lines 1307-1308) that the 2012 AWQC would be applicable to 

inland waters. Appendix C.4, however does include conclusions. Among these is that “Sufficient data do 

not exist to extended NEEAR epidemiology findings to non-POTW impacted inland waters.” This is not 

consistent with the draft 2012 AWQC, which does not appear to differentiate between inland waters 

impacted vs. not impacted by POTW sources. 

The discussion of indicators, and whether site-specific characteristics are better predictors of fecal 

indicator bacteria (FIB) densities than is the distinction between inland vs. coastal locations is a bit beside 

the point. Indicator bacteria have been established as criteria pollutants and specific criteria values have 

been established based on EPA’s epidemiologic studies. Thus, E. coli and enterococci are direct 

indicators of health risk, rather than indicators of pathogen presence or density. Thus, the efforts to justify 

extension of RWQC to inland waters need not (and given the limited and inconsistent relationship 

described in the RWQC probably should not) be included. Likewise discussion of FIB persistence need 

not be presented, as the criteria indicators and criteria values of those indicators have been established 

directly from studies of health risk. 

Instead of the “site specific differences are more important than inland vs. coastal differences” argument, 

EPA may wish to put forward one or more of the following rationales for extending the criteria to inland 

waters. These are: 

A) The WERF Inland Water Workshop report and the subsequent “Meeting Report” publication suggest 

that the criteria should protect public health by preventing sporadic disease. Thus, the use of FIB as 

criteria pollutants in inland waters should useful in making “recreation in and on the water” less risky at 

inland settings. 

B) The “Meeting Report” (not listed in the references, Dorevitch et al., 2010) notes that outbreaks of 

severe recreational waterborne disease have been reported almost entirely in inland waters, perhaps 

because of limited dilution capacity of receiving waters and proximity of recreation (and exposure) to 

fecal sources. Thus there is a need for public health protections in such settings. Although epidemiologic 

studies have not addressed outbreak prevention through FIB monitoring, it seems prudent to use FIB 

monitoring as way of reducing the occurrence of outbreaks of severe illness, as well as the sporadic cases 

of illness that occur among coastal swimmers. 

C) Millions of people per year use inland waters for recreation – see “Outdoor Recreation for21st Century 

America, 2004, Cordell et al., Venture Publishing, State College PA. 

Perhaps a better formulation would be to revise the language used in draft 2012 RWQC, lines 1449-1451 

to: “The application by States of the AWQC to inland water would be expected to improve public health 
Samuel Dorevitch 

C-7 

protection to the millions of people who use inland waters, where, unlike coastal waters, numerous 

outbreaks of severe illness have been reported.” 

Given the impact of the potential extension of RWQC to inland waters on the regulated community, EPA 

may wish to address potential criticisms of this change. These include: 

The WERF report raised the possibility of the criteria could be overprotective (i.e., a given level of a fecal 

indicator may imply a lesser degree of health risk in inland waters). Second, it is not known if the 

application of NEEAR-derived criteria would in fact prevent outbreaks of severe disease as these are 

(fortunately) rare events, which makes it very difficult study the impact of an FIB monitoring program on 

rates of severe disease (or outbreaks) in inland waters. Finally, particularly if enterococci are used as a 

criteria pollutant, many inland waters without obvious anthropogenic sources may abruptly be “listed” for 

303(d) purposes. Clearly the options prevented in Section 5 are intended for this purpose, and having 

options is reasonable. 

Some information presented that is considered to be support for the extension of RWQC to inland waters 

is speculative. An example: 

EPA concluded (lines 1214-1216) that differences in FIB survival “could be different in different waters, 

but those differences could be due to site-specific characteristics and not because of fundamental 

differences between coastal and non-coastal waters.” That survival “could be different” and the 

differences “could be” due to factors other than the coastal/non-coastal distinction is quite different from 

demonstrating that there is no difference in survival in relation to waterbody type. Even if that were the 

cases, it’s still a leap to conclude that indicator-health risk relationships would be the same in inland vs. 

other coastal water. As noted previously, the state of the science may better support removing from this 

section all discussion of FIB survival (and indicator-pathogen relationships). 

If the discussion of indicator-pathogen relationships is not moved to the QMRA section, the RWQC could 

be updated regarding the conclusion that the co-occurrence or correlation between FIB and pathogens is 

neither predictable nor consistent is very different than concluding that there is no difference in those 

relationships in coastal and inland waters. First, correlations between FIB and protozoan pathogens have 

been described in several studies with some consistency (Dorevitch 2011, which was cited elsewhere 

within the RWQC). Furthermore, the literature review in that paper makes clear that the relationship 

between FIB and protozoa does depend on whether the waters are heavily impacted by wastewater or not. 

Thus, at least in the case of Giardia and Cryptosporidium, there are fairly consistent/predictable 

relationships with FIB, particularly in waters not dominated by wastewater. 

Conclusion 2: Optional qPCR criteria will be the same in freshwater and marine water

Wade et al., 2010 noted (discussion section) that associations were stronger between Enterococcus CCE 

and GI illness than at NEEAR freshwater beaches than at NEEAR marine beaches, consistent with the 

Dufour/Cabelli studies of FIB measured by culture. Overlap of the confidence intervals in the NEEAR 

qPCR marine and freshwater curves are noted in the 2011 paper. However the finding reported in the 

RWQC (line 716) is unpublished and – because of the policy implications - should be the subject of peer 

review, particularly because this reflects a change from the way EPA interpreted findings of the 
Samuel Dorevitch 

C-8 

Dufour/Cabelli studies. If such peer reviewed publication is not available in time, the RWQC should 

describe the methods and findings of the analyses in detail. 

Conclusion 3: No need to change the FIB criteria, in fresh or marine waters, measured by culture

See discussion below on “Charge Question 3” 

In general, it may be helpful to the readers of the RWQC if EPA distinguishes in an executive summary 

which elements of the 2012 RWQC are found in the 1986 AWQC (i.e., they haven’t changed) and which 

elements are new or different. Elements that have not changed need little if any discussion, while changes 

do. This approach should result in less discussion of how the 1986 criteria were developed, including the 

culture methods, criteria pollutants and criteria values for fresh and marine water. 

Conclusion 4: No need to develop source-specific criteria

The discussion makes clear that the evidence is insufficient to change from the 1986 precedent of not 

differentiating among sources. Again, since this is not a change, the amount of discussion could be 

reduced. 

Conclusion 5: The translation of HCGI to NGI and the factor of 4.5 (HCGI 8/1000 ~ NGI 36/1000)

This is reasonable and supported by the text. 

Conclusion 6: The development of the qPCR threshold of 475 CCE (pages 28-29)

See “Charge question 3” regarding EPA’s efforts to demonstrate that the level of protection afforded by 

qPCR is comparable (in terms of cases per illness expected at the threshold value of enterococci CCE) to 

that at the culture-based criteria value. Consider creating a subheading (something like “3.3 The 

development of criteria values for enterococci by qPCR”) for this section. 

I’m not sure what is meant by lines 993-998. If this material is to be retained in the RWQC, describe the 

methods of each of the “additional analyses that were not used directly”, what were the results of those 

methods, and what were the limitations and interpretation of those results. What does it mean that the 

additional results “support the underlying basis” (line 994) of the approach to deriving the number 475 

CCE/100mL? The Wymer translation is not “support” for the number 475; it’s part of the methods used in 

calculating that number. 

Conclusion 7: The development of criteria using the study population overall, rather than potentially 

“sensitive subgroups” (section 3.3)

Consider renaming Section 3.3 “Potentially sensitive subgroups” 

The Parkin 2003 reference (line 1057) does not establish what the citation suggests: that immune system 

of children can leave them vulnerable to the effects of environmental agents. The paper presents a 

modeling approach to water management, in which it is assumed that children are sensitive, to assess the 

impact of that assumption on model outputs. It would be circular to think of this as evidence of the 
Samuel Dorevitch 

C-9 

vulnerability of children. An n alternative is to cite some of the primary literature noted in Parkin , namely 

Kee 1994 and D’Allesio 1981. However, those studies do not demonstrate (nor were they designed to 

evaluate) whether, for a given level of indicator bacteria, children are more likely than adults to develop 

HCGI (or NGI). Best to leave this out and discuss, based on NEEAR data, whether or not the microbehealth risk association (slope) is different for adults and children. 

Lines 1049-1052 seems to conflict with 1064-1075. Suggest: delete lines 1049-1052 and introduce the 

rest of that paragraph with a sentence saying something like “There are two reasons to be concerned that 

for a given level of indicator bacteria in recreational water children would be more likely than adults to 

develop gastrointestinal illness.” Then note: 1) a suggestion of greater water ingestion during swimming 

(Dufour 2006) and 2) the observation from the NEEAR study that children spend more time in the water. 

Despite the theoretical reasons which suggest children might be at increased risk, the NEEAR study did 

not find this to be the case. First, in the analyses that used microbe density as a continuous variable (fresh 

and marine), it lacked power to identify subtle differences between adults and children. Second, indicator 

density dichotomized at 33 CFU/100mL seems to have been a comparable predictor for adults and kids in 

freshwater (Wade 2008 Table 6).. It would be very informative if the statistical significance of interaction 

terms (child/adult * microbe level) or (child/adult * above/below threshold) were reported. 

Lines 1095-1096: The above information does not indicate that children are more susceptible. The above 

information suggests that despite theoretical concerns about ingestion rates, duration of recreation and 

perhaps naïve immune systems, the epidemiologic data do not support (given sample size) differences in 

risk between adults and children. Consider deleting this paragraph. 

Consider moving paragraph lines 1027-1037 to just prior to the paragraph that begins on line 1109. As 

currently organized, the discussion of over-representation of children in NEEAR enrollment is broken up 

by the discussion of NEEAR results regarding children vs. adults. 

Lines 1112-1113: I’m not sure what is meant by “in a robust manner.” Consider revising the final two 

sentences of this paragraph to “In the NEEAR study, children are over-represented (compared to the 

general population) and therefore have a disproportionally large influence on the overall relationship 

between microbe density and health risk derived from NEEAR study data.” 

Lines 1116-1130: This is a very important point and could be emphasized better as its own section, 

perhaps something like “Disease prevention through rapid notification.” As noted under “Charge 

Question 5” below, it would important for the public and stakeholders to know that the 2012 RWQC 

provide for the first time, guidance to afford the public the benefits of same-day notification. 

Conclusion 8: qPCR methods for monitoring recreational waters are optional

This is a surprise, and a disappointment, as the NEEAR study suggests that public health protections 

could be improved at Great Lake settings through the use of qPCR monitoring. The method can provide 

same-day results – a huge advance toward protecting the public form gastrointestinal illness using priorday measures of water quality to predict health outcomes. Furthermore, the NEEAR study demonstrated 

the superiority of same-day qPCR measures to culture measures as predictors of NGI. 
Samuel Dorevitch 

C-10 

Page 44 of the draft RWQC notes that Method A can be used as a site-specific alternative, pending the 

release of the “topic-specific guidance.” For inland waters several reasons were proposed in Appendix 

B.6 to suggest that inland waters may have different qPCR-health risk relationships than Great 

Lakes/coastal waters. For that reason it seems that such sites would need evaluations of method 

performance. However, given the vast amount of health and qPCR data collected at Great Lakes sites (and 

to a lesser degree, from marine sites), there seems to be enough data to apply the NEEAR results (and the 

475/1,207 CCE/100mL criteria values) to Great Lakes/coastal waters. Any “site specific performance 

characterization” (line 1617) would not include the collection of health data, which NEEAR did collect, 

it’s hard to see how such characterization of method performance will provide better qPCR-health data 

than did the NEEAR study. It not realistic to expect that the NEEAR study could have been conducted at 

a wide range of combinations of waterbody type and pollutant source. When establishing the 1986 

criteria, data from a limited number of sites were used, even though not all water types had been studied. 

The same could be done for (mandatory) qPCR criteria. 

The term “site-specific performance characterization” should be clarified. The intent may be to evaluate 

qPCR inhibition and to support the development of a site-specific plan for addressing inhibition (such as 

dilution or some sort of sample processing). However, it should be clear that the performance being 

characterized is not the relationship between qPCR results and health risk, which has been done in several 

types of settings by the NEEAR study. 

Conclusion 9: Statistical threshold values will be used rather than single-sample maximum values.

This change is an improvement of the 1986 AWQC. The term “maximum” implies a value never to be 

exceeded, while it is expected that about 25% of individual measures will exceed the STV. This change 

should be beneficial to beach manager in the efforts to communicate the meaning of an exceedance. 

The RWQC could be improved by explaining the basis for sampling required for calculating a geometric 

mean. Ideally, the variability in water quality should be known before estimating the number of samples 

required to calculate a GM. Furthermore, by choosing to sample only on days for which relatively low 

values of indicator bacteria are expected (no rain for several days, perhaps in the afternoon of day with 

much sunlight, the calculated GM could be lower than if samples were collected at random or on a fixed 

schedule (i.e., every Friday, Saturday, Sunday, and holiday). Provisions to prevent selection of sampling 

days based on expected low values would improve the public health protection of the criteria. 

Conclusion 10: Section 5.2.2: Flexibility available to states

The draft RWQC offers states two types of tools: Those not part of state water quality standards but 

which may enhance assessment/management/notification, and those that are designed to develop 

alternative water quality criteria. That QMRA (section 5.2.2) has been categorized as a pathway to 

alternative criteria rather than as an assessment/management tool requires evidence-based justification. 

Unlike another innovative aspect of the RWQC – the use of qPCR – very little is known about how well 

QMRA would actually protect swimmers from GI illness. Through the painstaking efforts of EPA/the 

NEEAR study team, we now can say quite a lot about how qPCR should (and should not) be done, and 

more importantly, how well qPCR methods can be used to predict health risk and provide timely 
Samuel Dorevitch 

C-11 

information to the public. By contrast, it is unknown how well QMRA predictions (model outputs) of 

health risk approximate observed (i.e., epi study) health risks – we just know the model outputs. 

Furthermore, we don’t know how precise QMRA estimates of health risks really are, regardless of the 

confidence intervals provide by simulations. If a QMRA model estimate that a given degree of water 

quality is expected to result in 30 cases of NGI, that might mean somewhere between 5 and 75 cases, with 

95% confidence. It’s very hard to know, given that QMRA models have not been validated against 

observational data of swimmers at beaches. The studies cited in the draft RWQC – Soller, Schoen and 

Ashbolt – are estimates of risk/illness that have not been confirmed through observational studies. 

Although EPA had hoped to have a QMRA conducted in parallel with one of the NEEAR sites, 

apparently no comparisons have been reported. Many assumptions of QMRA – notable dose-response – 

are very uncertain, having been based on very little data. Thus, although the theory behind QMRA is 

sound, the 5.2.2 provision is unvalidated. QMRA should instead be considered a method for evaluating 

management options – which would provide estimates of relative risk,, rather than the absolute measures 

of risk which would be required for any method that would be an alternative to an epidemiologic study. 

The Inland Waters Workshop (Appendix B.6) report and the subsequent Meeting Report note that QMRA 

methods require “grounding” (validation against observational data). If any validation has been done 

since the Workshop, this should be reported in the RWQC. Because the application of QMRA could 

allow localities/states to potentially produce lower site-specific/region-specific standards, it is important 

to know with confidence that the health of the public is not being compromised (and that undue burdens 

are not placed on regulated entities. 

Lines 1905-1907 (if non-human sources predominate) and lines 1432-1436: If the data are insufficient to 

propose national criteria for waterbodies impacted by non-human sources, are the data sufficient to 

support the development of site-specific standards for such waterbodies? 

Conclusion 10: Section 5.2.3: Alternate criteria

The concept here is very good, in particular the emphasis on establishing a relationship between new 

methods and existing methods at the criterion density. 

Since Wade et al., 2008 demonstrated that at Great Lakes beaches the qPCR method of measuring 

enterococci performed better than the culture method in predicting NGI, why not specify that equivalency 

should be demonstrated between the new method and best available approved method (i.e., enterococci 

qPCR)? 

Lines 2026-2027 note “that a consistent and predictable relationship “should exist between established 

and new methods. Furthermore the whether the new or established methods would be used “beach 

notification and attainment decisions....would be similar” (lines 2028-2029). In order to reduce confusion 

down the road, spelling out, it would be helpful to explicitly state what is meant by “consistent” 

“predictable” and “similar”. A 2x2 table could be constructed with rows “new method” and “established 

method” and columns, “exceeds criteria value” vs “does not exceed criteria value.” Does it mean a chi 

square p-value of p<0.05 even if the new method misses some percent of exceedances? If EPA were to 

specify the sensitivity and/or positive predictive value it would be clear what states/municipalities must 

demonstrate. Furthermore, how much data is needed? Perhaps a season (Memorial Day-Labor Day) of 
Samuel Dorevitch 

C-12 

5x/week measurements or some other general specification, otherwise, a “consistent, predictable 

relationship” could be claimed based on a much smaller set of observations. 

Last, the RWQC notes that this approach could be used to establish “regional or state-wide alternate 

criteria” (line 1993). If this is the case, again EPA should specify that the data collected to establish the 

equivalence of an alternate criteria must match the scale of data presented. In other words, data from one 

beach can’t establish state-wide criteria, and state-wide data can’t establish regional criteria, etc. 

Other aspects of Charge question 1:

Some specifics are noted in response to other charge questions. Otherwise, no significant issues regarding 

transparency, shortfalls, uncertainties, suggestions for improving the document. 

2. Are you aware of any important peer reviewed literature or applicable data that have 

not been incorporated in this document and should be included? 

Waterbody type (pages 33-36): See charge question 1. 

Support for idea that recreational criteria applied to inland waters should be protective: Inland Water 

Meeting report: Dorevitch, Ashbolt et al., 2010. 

Millions of people use inland waters for recreation: 

Cordell, H.K., Betz, C.J., Green, G.T., Mou, S., Leeworthy, V.R., Wiley, P.C., Barry, J.J., Hellerstein, D., 

2004. Outdoor Recreation for 21st Century America: A Report to the Nation: The National 

Survey on Recreation and the Environment. Venture Publishing, Inc., State College, PA. 

Heaney 2009: Suggests that a substantial portion of illness among swimmers enrolled in the NEEAR 

study may be due to sand exposure. This is significant, in that it means that some of the illness incidence 

among NEEAR study swimmers may be due to sand, rather than to water exposure and water quality. If 

this study is not addressed in the RWQC, it’s possible that it could be considered by others to be an 

oversight that falsely inflated the importance of beach water quality on health risk (“the problem isn’t the 

water, it’s the sand.”) The study should be acknowledged, along with the fact that the NEEAR freshwater 

(Wade 2009) and marine study (Wade et al., 2011) did adjust for digging in the sand should be noted as 

well. 

The basis for thinking that children would be at greater risk than adults (lines 1052-1053): A citation that 

would be relevant here is Dufour et al., 2006. 

Pratap et al., J Water and Health 2011: Even if extensive efforts aimed at measuring FIB and identifying 

conditions of elevated risk are applied, disease will not be prevented if effective communications do not 

follow. Pratap et al. noted substantial variability among states/tribes/territories in their approaches toward 

notifying the public, which have not been systematically evaluated for effectiveness. While not an 

element of identifying an indicator or establishing criteria values, perhaps it could be noted (perhaps in 
Samuel Dorevitch 

C-13 

section 5.1 or in the general introduction to indicators in section 1.1) that indicator measurement is a 

critical, but not the only critical element, in a chain of steps that is intended to result in disease prevention. 

Efforts to modify behavior (i.e., don’t go in the water when a notification/swim ban/beach closure or 

advisory is in effect). 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based 

indicator data aggregated by days of similar water quality corroborate that the 

NEEAR culture data are generally consistent with the 1986 regression models? 

It’s unclear why this issue is so prominently discussed in the draft AWQC. The document (Section 3.2.4) 

seems to frame the discussion of the 1986 numeric values vs. the NEEAR numeric values for indicators 

by culture as a “confirmation” of the 1986 numbers. The rationale for the comparison is never made clear: 

lines 853-855 state that this was done “to provide a comparison” – but why? 

Perhaps the intention is to establish that the qPCR results of the NEEAR study were developed in settings 

and with participants that generated a similar culture-health risk relationship as the Cabelli/Dufour 

studies. If this is the case, that should be made clear. However, the methods used to make this point are 

poor substitutes for analyzing both data sets (Cabelli/Dufour and NEEAR) using the NEEAR approach 

(which addresses individual-level confounders). Support for the comparability of the Cabelli/Dofour 

studies and the NEEAR study can be found in Table 6 of Wade et al., 2008 and Table 6 of Wade et al., 

2010. This information (or these tables) could be presented in the RWQC in support of utility of the 1986 

criteria values for enterococci; for justification of the continued use of those criteria values; and for 

demonstrating some comparability of findings of the NEEAR and Cabelli/Dufour studies. 

If EPA were to nevertheless decide to include the re-analysis of NEEAR data using the 1986 approach: 

I have significant concerns with the interpretation of Figure 1 (page 26) provided in lines 897-899. To say 

that the plotted datapoints from the NEEAR study are “generally fell within the predicted range” of the 

earlier studies is somewhat deceptive. The datapoints from the NEEAR study, particularly the freshwater 

(Figure 1a) study, suggest a relationship that is inverse (a line from about 20/1,000 swimmers on the Y 

axis through log10 entero=2 /100mL on the x-axis and that provides some fit for the NEEAR data). A less 

convincing case can be made for marine water (draw a line from about 5/1,000 swimmers on the Y axis 

and log10 entero=2). NEEAR results are “in the predicted range” but are inverse, suggesting that the 

NEEAR results are NOT consistent with the Cabelli/Dufour results. The use of data this aggregated, 

essentially ecologic, is problematic because of the ecologic fallacy: that the individuals who got sick at 

beaches with a seasonal average of “good water quality” were swimming in water with good water quality 

– perhaps those who got sick used the beach on days of poor water quality. It may be best to avoid 

presenting this analysis. If it does remain in the RWQC, the statement (lines 898-899) that “the NEEAR 

health and water quality data are consistent with those statistical models” must be deleted, though the 

following sentence “this analysis proved to be insufficient” – should be retained. As it stands, it’s 

problematic that the approach is insufficient yet a conclusion has been drawn. 

The “binning” approach is certainly better than the “beach season” ecologic approach. The binning 

approach, however, does not produce results that, as noted in lines 921-924 corroborate “the previous 
Samuel Dorevitch 

C-14 

conclusion that the NEEAR data were generally consistent with the 1986 regression models.” Again, like 

Figure 1, it is not especially meaningful that the datapoints fall “within the range” of the 1986 datapoints. 

If both the 1986 and the NEEAR data had similar slopes the findings would support what EPA wants to 

say. However, the 1986 data do suggest a significant positive slope for both freshwater and marine water 

while there does not appear to be a slope in the NEEAR data. Likewise, Table 2 does not suggest withinbeach trends. Were tests for trend done on those results? Either way, the 1986 method did not adjust for 

confounders as the NEEAR study did. It is possible that associations observed in the Cabelli/Dufour 

studies would be different if adjustment were made for recent exposures, digging in sand, etc. 

The discussion on page 28 and Figure 3, again, are problematic. Was a test of statistical significance done 

for the slopes of the NEEAR marine and freshwater regression lines? I suspect that such slopes may not 

be statistically significant, given the wide confidence intervals around the slope and the limited number of 

datapoints for a relatively small slopes (the freshwater slope is almost 0). 

Additionally, the statement (lines 947-948) that “...the recommended RWQC values are similarly 

protective of public health in both marine and freshwaters” is based on an assumption that the slopes and 

intercepts of the freshwater and marine indicator-HCGI relationships are similar. Since there seems to be 

no slope to the freshwater line, and perhaps no statistically significant slope to the marine line, this 

statement does not appear justified. Furthermore, the wide confidence intervals around the slope noted in 

association with the 1986 estimates of risk (line 943-947) seem to be quite wide here (Figure 3). If there 

really is not slope, water quality measures are not needed to estimate the level of risk, which simply 

would be the mean level of risk at all levels of water quality. 

One of the intended purposes of this section may be to identify a comparable level of target risk for 

establishing qPCR criteria. If this is the case, the section should be re-written to make that clear. 

It may be worth emphasizing that: 

1. Analysis of the NEEAR culture data by the methods used to establish the 1986 criteria show 

comparability of the culture data-health risk association in the NEEAR and the Cabelli/Dufour studies 

(at least when assessed using the dichotomous “FIB above the thresholds of 33 and 35 CFU/100mL) 

(see above-noted reference to Table 6 of Wade et al., 2008 and Wade et al., 2010. 

2. Presenting the relationships of enterococci CFU vs. health risk generated from the NEEAR study 

using the same approach to multivariate modeling that was done for qPCR would be better than using 

the 1986 approaches. If that relationship is weak, potential responses to that observation include: 

3. Conclude that the qPCR models are better predictors of illness and therefore the method should be 

mandatory, rather than optional. Enterococci CCE criteria values should be those associated with a 

targeted level of risk of 8/1,000, same as the 1986 freshwater criteria. The need for harmonizing the 

targeted risk levels has been emphasized many times previously (Airlie workshop, Boehm 2009). The 

use of the more protective target (freshwater rather than marine water) would be preferred from a 

public health standpoint. 
Samuel Dorevitch 

C-15 

a. Present data similar to that found in Figures 1-2 of Wade et al., 2008 and Figure 2 (right panel) of 

Wade et al., 2010. If in fact the slopes of these lines are similar in marine and freshwater, and if 

the slopes are similar for adults and children, the information can be combined into a single 

graph. A enterococci CCE criteria value providing the same level of protection as the culture 

values can then be selected by using 8 cases of AGI (or the NGI equivalent)/1,000. 

b. Given the limited precision of expected risk for given microbe density (the wide confidence 

intervals and the small slope, the relationship between enterococci CCE and NGI may be too 

weak to reasonably differentiate qPCR CCE values to that are associated with, for example, 8 or 

10 or 12 or 14 cases /1,000. This would suggest that the selection of a single targeted risk level 

cannot be supported. In this case, ordinal categories may be more appropriate. This could take the 

form of the CCE values corresponding to risk of <8 cases/1,000 could be considered “green”; 8 to 

(arbitrarily) 12 could be considered “yellow” and >12 could be considered “red” (as examples). 

The use of graded categories defined by ordinal ranges of risk is probably a better reflection of 

the science (i.e., the limited precision in predictions of health risk based on microbe density or 

vice versa) than the use of discrete categories dichotomized by an imprecise estimate water 

quality for a given level of targeted risk. 

4. Do the characterization and exploration of the qPCR method reflect that of the 

scientific literature? If not, what additional information should be included? Is the 

derivation of the qPCR value using the EPA data scientifically supported and 

adequately explained? Please discuss why or why not. 

Given that the qPCR laboratory and calculation methods utilized in the NEEAR study were the ones that 

established the enterococci CCE-NGI risk association, I would be reluctant to see that method revised. 

Revisions to the calculation approach of NEEAR study water quality data that were found to improve 

health predictions within the NEEAR dataset would be valuable. Revised laboratory methods in isolation 

would be difficult to assess in relation to health outcome prediction. 

5. Please provide any other general or specific comments on the document, making sure 

to identify the section number/page number you are commenting on. 

Some suggestions for organization of the document. 

- Start with an executive summary that highlights: 

- The CWA and the background for reviewing/revising AWQC/RWQC 

- NEEAR study (# beaches, participants, years) 

- What stays the same (no changes to culture method criteria values) 

- What changes are proposed 

 One huge change: inland waters, 

 One small change: the option for data collection to support future use of qPCR 

 The Section 5 provisions 

- How the 2012 RWQC can improve public health (rapid notification, more waters now 

covered/protected). 
Samuel Dorevitch 

C-16 

Has EPA estimated the number (or percent) of days of beach use that, following application of rapid 

methods, will be appropriately closed when prior day culture methods would leave the beach 

inappropriately open (or without an advisory/notification)?

Has EPA estimated to number or inland water recreators (or person-days of recreation) that for the first 

time, fall under the protection of the RWQC? 

Miscellaneous comments/edits/corrections 

Lines 309-310: Consider revising “State of the science review on occurrence and cross-infectivity of 

specific pathogens associated with animals.” To “State of the science review on the occurrence and 

human health risks of specific waterborne zoonotic pathogens.”

Lines 342-343: Consider revising “...considered all of the indicators of fecal contamination...” to 

“...considered indicators of fecal contamination” or better still, “...considered indicators of health risk...” 

Line 358: Consider revising “These particular FIB are not pathogenic...” to “These particular FIB are 

generally not pathogenic...” (since some E. coli are pathogenic). 

Line 359: Again, the previously noted suggestion that over-emphasizing FIB as pathogen indicators rather 

than health risk indicators, given that the document elsewhere notes that FIB are poor indicators of 

pathogen presence. 

Lines 366-373: Text is confusing. Consider first explaining that the membrane filtration method generates 

CFU/volume while the defined substrate method generates MPN/volume. The defined substrate method 

does not involve counting colonies, but rather relies upon probabilities. Neither method includes counting 

of microbes in a water sample, but rather, generate results following culturing those microbes for 18-24 

hours. 

Line 383: Change viable to “culturable” as bacteria in the VBNC (viable but not culturable) state would 

not be counted using culture methods. 

Line 390: Consider changing “methods in water” to “methods in a specific waterbody or beach” 

Line 401: Suggest that there’s an qPCR more recently developed (? since Method A was published?). 

Consider deleting the word “more.” 

Line 418: Change both uses of “incidences” to “instances” 

Line 419-422: This isn’t definite. Chern 2011 suggests that the SPC and target DNA have similar 

amplification efficiencies. 

Lines 440-447: Why not refer to this as the ddCT approach? 
Samuel Dorevitch 

C-17 

Lines 471, 487: FIB “occurrence” sounds like FIB “presence.” Since FIB are essentially always present, 

albeit sometimes in low concentration/density, “occurrence of FIB” isn’t informative. Consider changing 

to “density” or “concentration.” 

Line 517: For the sake of completeness could add that the 1986 differed for marine and freshwater in part 

because of the different levels of “acceptable risk.” 

Line 578: Consider revising “actual increase in the burden of illness swimmers experience” to “actual 

increase in the incidence of illness among swimmers” (as “burden” is more general and could refer both 

to the incidence rate and the severity of illness). 

Line 596: consider changing “an eye infection” to “a self-reported eye infection”. Similarly, Line 598: 

consider changing “wound that became infected” to “wound that was reported to have become infected” 

In both cases, it may not be clear that “infection” was based on self-report, rather than diagnosis by a 

physician or other health care provider. 

Lines 600-610: Not really. It would be true that that a larger percent of epi study participants experiences 

GI illness than other health outcomes. However, that’s no reason to assume that preventing GI illness 

would also prevent non-GI illness. It might, but “a similar level of protection” implies that a similar 

proportion of GI and non-GI illnesses would be prevented. We don’t know if that’s true. It might be better 

to state that NEEAR and previous epi studies found that the health outcome with the highest incidence 

was GI illness. Preventing GI illness may or may not result in a comparable reduction in health outcomes, 

but it will prevent the most type of recreational waterborne illness. 

Line 635-626: Consider revising to “...RWQC because neither possible relationships between these 

microbes and FIB, nor relationships between FIB and these health outcomes, have been the subject of 

observational studies.” 

Line 651: Change “incidences” to “incidence” 

Line 670-671: Revise “time issues” to “short-term (hours) variability” 

Line 689: Suggest changing “particular sensitive subpopulations” to either “potentially sensitive 

subpopulations” or “particular subpopulations” (since in the end, there wasn’t sufficient evidence to 

support that assumption that children are in fact “sensitive” 

Lines 715-716: change “rate of increase in risk per unit increase in enterococci” to “change in risk per 

unit change in enterococci”. Also: This analysis (p=0.44) is unpublished but has substantial policy 

implications. Consider elaborating on the source of this info and if possible, the status of the EPA report 

or the publication status (submitted, in revision, etc). 

Lines 739-741: As written this suggests that Wade 2010 evaluated the use of Bacteroidales in marine and 

freshwater and found that indicator to be associated with GI illness in marine waters only. Because Wade 

2010 only addressed marine waters, better to revise as “GI illness in swimmers at marine waters was also 
Samuel Dorevitch 

C-18 

associated with exposure to levels of anaerobic bacteria of the order Bacteroidales enumerated with 

EPA’s Bacteroidales qPCR method (Wade 2010).” 

Lines 764-767: The very low indicator levels is likely an important (perhaps the most important) reason 

for the absence of a relationship between FIB and health at both Boqueron and Surfside. This information, 

rather than inhibition (which would not impact the enterococci culture data) should be noted. 

Lines 790-795: The end of this section states that E. coli levels “correlated with HCGI in a similar manner 

as in EPA’s 1970s” studies. First, the Ohio reservoir study reported an association with GI symptoms but 

not with HCGI. Second, the term “correlation” is used in several places in the RWQC to mean 

“association” or “statistically significant relationship”. This is one such example. The Ohio study reported 

the results of a multivariate regression, not a correlation analysis. 

Lines 810-813: Please justify that statement that a limitation of the RCTs is smaller sample size. From a 

practical standpoint, it’s actually an advantage that smaller sampler sizes are needed. The reason sample 

sizes can be smaller is that randomization should generate 2 groups (exposed and unexposed) that are 

similar in all ways other than the exposure. Because the observational designs do not randomize, much 

larger sample sizes are needed to adjust for confounders. It could be noted instead that the most 

significant limitation of the RCTs is that participants who are randomized to “exposure” have been asked 

to immerse their head in water 3 times over a short duration (10 minutes). This is not swimming. It may 

result in rates of water ingestion that are greater or less than swimming, but it is not swimming. There is 

an assumption that the risks of GI illness in the RCTs are the risks associated with swimming; they are the 

risks associated with head immersion. 

Line 832: Change “swimmers” to “those randomized to head immersion.” 

Related: Lines 1275-1294 refer to Epibathe. This was a series of studies that evaluated the health risks of 

controlled exposure (head immersion), not swimming. Any conclusions about the relevance of Epibathe 

to EPA criteria development should note that Epibathe did not evaluate the health risks of swimming, but 

rather, head immersion (3 times). 
C-19 

Erin K. Lipp 

Associate Professor 

Department of Environmental Health Science 

College of Public Health 

The University of Georgia 

Athens, GA 
C-20 

Erin Lipp is an associate professor in the Department of Environmental Health Science at the 

University of Georgia, where she is also the department Graduate Coordinator and Director of 

the Georgia Oceans and Health Initiative. Her research focus is the ecology of human pathogens 

in ambient waters and the role of environmental exposures in disease transmission. Her research 

incorporates molecular biology, microbial ecology, epidemiology and climate research to better 

understand the fate of bacteria and viruses introduced from wastewater to aquatic environments 

and their potential for transmission to humans and other hosts. She has been an invited 

participant and report author for a variety of workshops related to waterborne disease, marine 

water quality and climate change effects on health. 
Erin K. Lipp 

C-21 

Peer Review of the Recreational Water Quality Criteria (RWQC) 

Comments Submitted by Erin K. Lipp 

Introductory Comments 

In general, this document represents an extraordinary amount of work conducted by the EPA to revise the 

national water quality criteria. The authors have managed to synthesize a broad collection of results and 

provide analyses that support their recommendations. This was not an easy task and the document 

represents a critical body of work. The comments provided here are meant to strengthen the document, 

increase clarity and provide more transparency in how the revision of the RWQC was developed. 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for 

technical conclusions transparent? Do you have any suggestions on how we could 

improve this document? Please explain your responses. 

In all this was a thorough and well laid presentation of the justification for the recreational water quality 

criterion. The authors provide a well-organized framework to understand the justification and 

implementation of the RWQC and provide a much needed section on how States can proceed with their 

own alternate criteria. More emphasis, however, should be focused on the new criterion and not the 

history of the current AWQC. It would also be helpful (from a structural/organizational perspective) to 

better cross reference specific appendices in the document rather than broad groups (e.g., appendix A.2 

rather than appendix A, etc.). The addition of an executive summary would also improve the document. 

While the uncertainties for the technical conclusions are provided, limitations in applying this criterion to 

“all” water bodies should be better acknowledged. The RWQC is meant to be for all water bodies in the 

US but the data that EPA presents does not support this call. Namely, the RWQC appear to be robust for 

temperate POTW-impacted marine and freshwater coastal areas, with possible application to POTWimpacted inland waters; however, there are no compelling data that support its use in tropical (or 

subtropical) waters or in non-POTW impacted waters (coastal or inland). As a general note I was struck 

that to date all NEEAR studies at marine beaches were conducted in embayments or estuaries (with the 

exception of Surfside, SC), that were relatively quiescent. This may not be a good representation of the 

bulk of marine beaches, which tend to be open water beaches and is a limitation of the sites selected for 

the health effects assessments. This should be better acknowledged and information presented on site 

selection. Such information would help states/locales judge whether they should look at their own site 

specific assessments for other types of beaches. 

The use of predictive models and the great amount of work that has gone into developing an easy to adapt 

format (e.g., Virtual Beach) is to be commended. The ability to use pre-emptive strategies to protect 

public health should be a goal for all beach managers. While Virtual Beach works quite well for Great 

Lakes Beaches, its performance for marine beaches was not impressive (which may be have been 
Erin K. Lipp 

C-22 

exacerbated by the few days of exceedance for the beaches tested). This should be clarified in the 

document. 

While the final criterion uses culturable E. coli, culturable enterococci and qPCR enterococci (CCE)for 

freshwater and culturable enterococci and qPCR enterococci (CCE) for marine water, there was 

significant effort put into the confirmation and validation of the qPCR for Bacteriodales it appears that 

this will not be used as a criterion. Is this one of the tools that States can suggest as an effective alternate 

indicator? If so, it may be helpful to state that explicitly somewhere. While the final RWQC will continue 

to rely on the same FIB criterion as suggested in 1986, the improvement in risk determination (and 

confirmation that risk levels are equivalent between fresh and marine waters) is a great step forward as is 

the specific guidance on qPCR (even if is not a required element). 

Another strength of this document is that it allows states to update and find alternatives for their specific 

areas. The technical support guidelines will need to be complete to ensure good translation of 

information. While it is outside the scope of this document, states should be able to compete for grants to 

develop these site specific alternatives. 

Some specific comments on logic and support for conclusions in this document: 

Line 720. For clarification the reason that there is a focus in this analysis/review on temperate POTWimpacted beaches needs to be stated early (namely, that EPA dropped the epi studies from run-off 

impacted and tropical beaches because of lack of correlation between FIB measured and health effect end 

points) 

Line 759-769 (see above statement about placement of this text). These studies were dropped because the 

results were inconclusive. Given the that the temperate POTW-impacted beaches all fit well into a single 

analysis it makes some sense to exclude these studies, but the fact that they did not fit the trends noted for 

the other studies should suggest that: 1) the current RWQC will not be adequate for these areas, 2) some 

other as yet untested FIB may be better suited for a single FIB for all water bodies (although this scenario 

would require many more studies, time, effort and money) or that additional epi studies are warranted to 

better determine the applicability (or lack thereof) the suggested RWQC. While there is sufficient 

language to allow states to address this issue on their own, this seems like a potentially large problem that 

may need additional federal (EPA) assistance. [Line 784-788. Following up on the issue noted above, this 

review of studies also shows a paucity of epi studies focused in tropical areas. Additional work is clearly 

needed in this area.] To this end, a separate section on issues associated with tropical/subtropical waters 

may be warranted and should include studies completed in Hawaii on this issue. 

2. Are you aware of any important peer reviewed literature or applicable data that have 

not been incorporated in this document and should be included? 

The authors have been thorough in their review of materials, especially when taken as a whole with all 

appendices. 
Erin K. Lipp 

C-23 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based 

indicator data aggregated by days of similar water quality corroborate that the 

NEEAR culture data are generally consistent with the 1986 regression models? 

When aggregated by days of similar water quality, the data do support consistency with the 1986 criteria; 

however, given the wide variability among data points the following issues should be considered: 

Line 897-899. In attempting to develop a quantitative mechanism to compare NEEAR data with the 1986 

criteria based on culturable enterococci, different methods of data binning were used. This sentence refers 

to the results when the seasonal GM is taken for the seven beaches used in the NEEAR study. The report 

states that the data points generally fell within the predicted range of the 1986 regressions. This is a rather 

optimistic view of these points (as shown in Figure 1). For freshwater, the NEEAR data points (4) 

actually look to be negatively correlated with HCGI rate and no relationship is noted in marine water (3 

data points). While they do technically fall within the confidence intervals of the regression this is not a 

robust measure (which is also acknowledged in the report) especially in light of the few data points. 

Binning by water quality levels provided a better point for comparison (as shown in next paragraph in the 

report). 

Line 950. The trends lines for these data seem particularly weak, especially for freshwater beaches (which 

in previous figures seemed to have a stronger relationship than marine water). The data in general seem to 

show a very high degree of variability. 

Line 961 - 968. Report states that results from marine and freshwater studies are sufficiently similar to 

allow combining in the new criteria. I do wonder if they are sufficiently similar only because all of the 

data are so highly variable (there seems to be a large spread); however the regression using the combined 

freshwater and marine data is stronger than when the water types are separated. The marine data points 

(alone) show an almost vertical regression line (appendix B.5) where a very small change in CCE results 

in a large change in # GI/1000. 

It is also not entirely clear why the NEARR data must be corroborated by 1986 analyses, given the 

difference in study design. 

4. Do the characterization and explanation of the qPCR method reflect that of the 

scientific literature? If not, what additional information should be included? Is the 

derivation of the qPCR value using the EPA data scientifically supported and 

adequately explained? Please discuss why or why not. 

More data are needed to confirm the use of the qPCR (CCE) between labs and among many different 

water types. This work appears to be on-going but results are critical to fully assess the utility of this 

alternate indicator. Given the specific need for developing rapid methods to assess water quality (per 

BEACH Act and lawsuit) data are needed to provide states with specific recommendations about 

implementing this tool, along with resources for set up and training. 
Erin K. Lipp 

C-24 

Specific questions/issues are noted below: 

Line 414-415 (and elsewhere). Given the variability in water types both within and between beaches (over 

space and time), it might be expected that PCR failures (inhibition) will happen with some regularity. 

There needs to be specific guidance for how to handle these occurrences (especially where some beaches 

routinely face this problem). 

Line 458-464. While qPCR methods (CCE) show excellent promise as a rapid tool for measuring water 

quality the limitations in validation of this method are cause for some concern. More work is needed to 

provide guidance to states on how best to implement and trouble shoot this approach 

Line 1116-1130. What happened to qPCR for Bacteroidales? Will this be an option for states with marine 

beaches if they choose to adopt another target? 

Line 1613. The guidelines for states on determining the performance characteristics of the rapid (qPCR) 

methods will be critical for successful implementation. Has any headway been made on drafting such a 

document? 

Apppendix A.2 Single lab validation of EPA method A (qPCR for enterococci) 

• Variability in the copy number of the 23S rRNA gene between species and over the course of a 

growth cycle is a potential problem in correct quantification using this method. Currently, EPA is 

using a copy number of 4 (reported for E faecalis) as a reference point. Is any work anticipated to 

determine the broader range of copy numbers in other species (e.g., E faecium has 6 copies)? If so, 

and average copy number could be used for quantification.

• In this single lab validation, the recovery was very consistent among samples in PBS but seems to 

have a fair amount of variation in both marine and freshwater. Is EPA evaluating methods to 

improve recovery consistency? 

• It will be helpful to see results from multilab validation

Appendix A.4. Parameters that affect performance of qPCR method for enterococci 

• Study temperatures tested here are reflective of seasonal conditions in temperate areas, not tropical 

or subtropical (i.e., spring 15 – 17 C, summer 20 – 23 C, winter 4 – 8 C). In all seasons, these 

temperatures are cooler than would be expected in tropical/subtropical waters. Especially at the 

higher temperature scale (e.g., 30 C in the summer) may have a strong effect on persistence/decay 

of enterococci which could inform the use of this indicator in warm (hot) tropical waters. 

Appendix A.6. Suitability of indicators and methods for different CWA programs 

• The limit of detection information present in Table 2 is somewhat confusing. For qPCR (CCE and 

CSE) an 50% limit of detection is used. This a term generally used for evaluation of 

presence/absence data, not quantifiable results. While the data are derived from Chern et al., 2009, 

it is not clear in the Chern paper how the 50% LOD was derived. In most studies, the actually LOD 
Erin K. Lipp 

C-25 

varies with processing parameters but in general a theoretical limit of detection can be derived for 

an ‘optimal scenario.’ It is not clear why that approach was not taken here (for both culture and 

qPCR methods) 

5. Please provide any other general or specific comments on the document, making sure 

to identify the section number/page number you are commenting on. 

The text should discuss the notion that FIB are used in practice as indicators of health risk and not 

specifically correlates of pathogens. There are many papers showing no correlation between FIB and 

enteric viruses, or other pathogens. However, understanding the relationship of FIBs to pathogens will 

become more important if using a QMRA approach. This topic would be best discussed in the QMRA 

section. 

Line 675. The summer season focus makes sense for temperate waters but in tropical waters, where 

beaches are used year round, epi studies done at other seasons may be important. While it is speculative, 

high water temperatures and UV irradiation during the summer tropical beach epi study might help to 

explain the very low numbers of FIB observed. 

Line 1064. It should be made clear that this statement is referring only to freshwater studies. 

Line 1135. The first line of this paragraph states that this RWQC are national recommendations for all 

surface waters in the US, but data reported here and in the appendix does not support this. Ongoing 

questions remain about the use of this criterion for non-POTW impacted beaches and inland waters and 

tropical waters. This should be acknowledged and special considerations given for these areas in section 5 

(for implementing site specific criteria). 

Line 1189-1191. Report states that based on the review of literature and the overall site to site variability 

of FIB concentrations that the RWQC will be applicable to both coastal and non coastal sites. But, this is 

in contrast to the EPA study (appendix C.4) that states that epidemiological studies to date do NOT 

support extending these criteria to inland waters NOT impacted by POTW discharge. 

Line 1307. This statement is not consistent with findings reported in Appendix C.4 (see above comment 

on same issue) 

Line 1428. While based on QMRA (Appendix C.3) and other studies, inputs from animal feces can result 

in similar risks as human fecal contamination, it should be pointed out that not all animals present the 

same risk levels (e.g., appendix C.3 - pigs and poultry showing lower risks) 

Line 1747-1751. The development of these models and software for model development are a great 

addition to allow rapid assessments; however, while the Great Lakes Beaches models have been 

successful it is important to note in this document that the results for marine beaches are currently 

inadequate. This likely reflects the fact that the models are predicting FIB levels, which tend to be lower 

at marine beaches, even with health effects are noted. As any work been done to adjust these tools to 

predict health effects directly rather than the FIB levels? 
Erin K. Lipp 

C-26 

Line 1893-1900. Given the inconclusive results of the non-POTW impacted beach and tropical beach, is 

the EPA anticipating that states/regions address these on a short time frame? Does the EPA anticipate 

providing any assistance for these regions? This is a critical gap in knowledge. 

Line 1986-1997. Does the EPA plan to update or offer technical guidance to the states on this issue 

periodically? It seems that EPA might be in a better position to offer new methods/technology options 

with appropriate QC behind the method. 

Appendix B.4 is identical to appendix B.1. 

Typos: 

Line 257. There is a typo. WERF is Water Environment Research Foundation (not “experts” as written) 

Line316. The word ‘Water’ is missing from National Epidemiological and Environmental Assessment of 

Recreational ...

Line 848. Typo .. “if” should be “of” (“...terms OF indicator density ...”) 

Line 885. Typo, ‘trail’ should be ‘TRIAL’ 

Line 1165. Typo. There is a word missing from this sentence or the structure is awkward. “Loadings and 

hydrodynamics of FIB in human POTW coastal and non-coastal waters ...” Human POTW? 

Line 1205-1207. “salinity’ should be removed from this sentence given salinity would have a lot to do 

with whether a beach was coastal or non-coastal (e.g., coastal marine beaches will be saline and non 

coastal beaches will be fresh) 

Line 1251. Typo. Remove the word “was”. Sentence should read “...a retrospective study in the French 

Alps ...” 
C-27 

Graham McBride 

Principal Scientist 

National Institute of Water and Atmospheric Research Ltd. (NIWA) 

Hamilton, New Zealand 
C-28 

Graham McBride is a Principal Scientist in the National Institute of Water and Atmospheric 

Research of New Zealand. His specialist fields are water quality modeling and statistics, 

including their application to water resource management and the setting of environmental 

guidelines and standards. He has lead epidemiological studies of swimmers' health and was one 

of the authors of his country's drinking water standards and also the water quality guidelines for 

recreational waters. Graham has published a number of peer-reviewed papers, book chapters 

and a text on water resources science and management. 
Graham McBride 

C-29 

Peer Review of the Recreational Water Quality Criteria (RWQC) 

Comments Submitted by Graham McBride 

Introductory Comments 

The EPA and its advisers have performed an admirable task of sifting through the literature on the health 

of recreational water users, including many aspects of aquatic microbiology, epidemiology and risk 

analysis. The end result of their analysis is a generally well-argued case for the proposed revised water 

quality criteria. 

I am in agreement with all the comments made by the Review Panel. The following responses should be 

viewed as elaborations of those findings. 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for 

technical conclusions transparent? Do you have any suggestions on how we could 

improve this document? Please explain your responses. 

The document presents a well-argued case. I agree with its general approach, in particular: 

• coalescing the risks associated with marine and freshwater criteria; 

• extending the compliance assessment period; 

• allowing for flexibility in implementation (e.g., QMRA, novel indicators, predictive models). 

By and large the text seems clear. In the following I have noted those occasions when it seems not to be 

so. However, I do suggest that some matters need to be addressed or clarified: 

• No guidance is given on the desirable features of sampling programs, especially with respect to wet 

weather. I note that this is to be included in follow-up technical support guidance (TSG). 

Nevertheless, it should be helpful for there to be at least some mention of this issue in the criteria 

document—after all, the 1986 criteria did do so.1 And if this former position is to be abandoned 

(i.e., restricting sampling to dry weather), it would seem appropriate to say so and give the rationale 

for it. In my view wet weather sampling (or sampling shortly after rainfall) may be appropriate if 

recreational contact occurs at that time. I don't see lot of point in sampling during conditions that 

are not conducive to recreation. 

 

1

 At page 8 of EPA'S Ambient Water Quality Criteria for Bacteria –1986: EPA440/5-84-002 (cited as "U.S. EPA 

1986" in the draft RWQC) we have: "In general, samples should be collected during dry weather periods to 

establish so-called "steady-state" conditions. Special studies may be necessary to evaluate the effects of wet 

weather conditions on waters of interest especially if sanitary surveys indicate the area may be subject to storm 

water effects". The criteria themselves (in EPA 44/5-86-001, Quality Criteria for Water 1986) contained a similar 

message: "These densities are for steady state dry weather conditions." 
Graham McBride 

C-30 

• Likewise I didn't see any reference to guidance on sampling depths (knee depth, chest depth,...). 

• I wonder if respiratory illnesses have been dismissed too easily. Yes, it is true that in some studies 

GI can be the stronger endpoint, but that may not be so for water skiers, canoeists or for beaches 

with an energetic wave climate such that aerosolisation is rather stronger. After all some studies 

have detected respiratory effects but not GI: England (Fewtrell et al., 1992);2 Australia (Corbett et 

al., 1993, Harrington et al., 1993); New Zealand (McBride et al., 1998). 

• Has the case really been made that feces of all animal groups should be treated as posing the same 

risk as human feces? For example, there is evidence—in the New Zealand setting at least—of 

substantial differential virulence between Campylobacter jejuni carried by farmed animals (sheep, 

cattle, poultry) versus wild birds, and yet wild bird strains can predominate in recreational waters 

(French et al., 2011). 

• The distinction between SSM and STV is not sufficiently clear, especially because for beach 

notifications (lines 1542–1545) the STV can act as maxima not to be exceeded (i.e., acting as a 

SSM) but for WQS attainment "no more than 25% of the observations can exceed the STV" (acting 

as a percentile limit). I also have difficulty understanding how the STV (if it is interpreted as a 

SSM) can be used in devising permit limits, especially if used alone (as suggested on line 1548). In 

my view stating only a limit never to be exceeded is next-to-useless in designing wastewater 

infrastructure. In contrast a GM (or median) accompanied by a high percentile (but not a maximum) 

tells designers and operators of wastewater facilities what is expected under normal and under 

unusual circumstances. In that regard a 75%ile seems rather low; international practice increasingly 

uses 90%iles or 95%iles. 

• Lines 1494–1503. This is timely and well-stated. But the last sentence begs a "so what's the 

consequence of that?" question. I presume it means that EPA is prepared to tolerate a moderate 

degree of STV exceedances and hence notifications (interpreting STV as a SSM, as is proposed for 

Beach Notifications)—even when the GM is way below its limit. 

• Line 1559 is ambiguous: '...i.e., no "excursions" acceptable."' I think refers to the GM, but I first 

read it as applying to the STV. 

• Given that the WHO, Europe and Australasia have adopted the "Annapolis protocol", its nonappearance in this document seems surprising. I understand that the CWA is considered to pose an 

impediment to its adoption in the USA. But it would be helpful to explain why that is the case. And 

I note that the proposed extension of the compliance assessment period (recommended to be 90 

days) takes the USEPA criteria closer to the "beach grading" component of that protocol. 

2. Are you aware of any important peer reviewed literature or applicable data that have 

not been incorporated in this document and should be included? 

A number of extra references have been identified and are all included in the Review Panel's responses to 

this question. With regard to the lack of a "road map" (Review Panel's comment 1.1), the recent WERF 

 

2

 All references in this section are listed in Section 2 of the Review Panel's report. 
Graham McBride 

C-31 

QMRA study (Bambic et al., 2011, not cited in the RWQC—but listed in the Review Panel's report) has a 

conceptual diagram of a sort that may be appropriate as a decision tree for RWQC implementation, viz. 
Graham McBride 

C-32 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based 

indicator data aggregated by days of similar water quality corroborate that the 

NEEAR culture data are generally consistent with the 1986 regression models? 

• This is a crucial issue, because this analysis has enabled the adoption of the same numerical limits 

on culturable fecal indicator bacteria as currently apply (under the 1986 criteria). The general 

approach taken seems sound, but I concur with the caveats made by the Review Panel's responses 

to this question. I presume that EPA staff will urgently promote a peer-reviewed publication of 

this important analysis? 

• A particular feature of this analysis is seen by contrasting Figures 2a and 2b. In Figure 2a the 

"1986 freshwater data" and the "NEEAR freshwater data" are positioned similarly (with respect 

to the X and Y axes). In contrast on Figure 2b we see that the "1986 marine data" and the 

"NEEAR marine data" differ in that the former tend to exhibit higher HCGI rates. Some 

explanation of this contrast would be helpful, because it is the cause of the reduced estimate of 

HCGI risk associated with the marine criterion. 

• It may be appropriate here to recall some history of the development of the 1986 criteria, 

especially for marine waters. In 1984 USEPA proposed an "acceptable risk" of 6 HCGI cases per 

1000 swimming events and advertised that notion (see Federal Register 49(102), 21987–21988 

May 1984). Furthermore, in the FR, EPA used the Cabelli HCGI relationship with that illness risk 

giving rise to a limit of 3 enterococci per 100 mL (interpreted as a GM limit). Later, after various 

submissions pointed out that such a low limit was not generally attainable, the idea was raised 

that since a GM limit of 200 fecal coliforms per 100 mL had served well, let us translate that limit 

back to enterococci (giving 35 per 100 mL) and then read the Cabelli graph in the reverse 

direction to calculate the associated risk (19 per 1000 swimming events), as explained in Federal 

Register 51(45), 8012–8016, March 1986. That is, "acceptable risk" was not selected, it was 

calculated. That stance has now changed in the proposed revisions, in that an acceptable risk 

level has now been found to be consistent with both the freshwater and the marine criteria limits. 

Presentation of this change in the RWQC might be helpful. 

4. Do the characterization and explanation of the qPCR method reflect that of the 

scientific literature? If not, what additional information should be included? Is the 

derivation of the qPCR value using the EPA data scientifically supported and 

adequately explained? Please discuss why or why not. 

As this is not my particular expertise I defer to Panel colleagues on this question. 
Graham McBride 

C-33 

5. Please provide any other general or specific comments on the document, making sure 

to identify the section number/page number you are commenting on. 

Line Comment

183 "...recommended fecal coliform criteria..." should be "...recommended log 

mean fecal coliform criteria..."

254–255 With regard to this text: "These projects included epidemiological studies to 

provide data correlating illness with indicators, site-characterization studies to 

facilitate QMRA...": To do that the epidemiological studies would need to assay 

for pathogens, not just indicators—QMRA uses dose-response relationships for 

pathogens. I understand that this has not been done.

269 I note that these QMRA studies are for impacts of agricultural animals. So 

inferring from these that wildlife poses similar risks may lack sufficient 

foundation (but I do recognize that in the absence of studies on wildlife a 

precautionary approach may be very desirable from a public health viewpoint). 

498 While true that the 1976 criteria (in USEPA Quality Criteria for Water, July 

1976) recommended the same numerical limits as did NTAC, it may be helpful to 

note that the rationale for these limits changed (from the earlier USPHS studies of 

Stevenson 1953, as interpreted by NTAC), to an argument based on minimizing 

Salmonella infections (as is clearly stated in that text). 

524–538 What is meant by "the" observed GM enterococci and "the" observed fecal 

coliforms? Is it for the combined studies mentioned on line 514?

557 The word "because" is misleading here. It should be "in that". The paragraph 

doesn't explain why the changed definition was adopted. Maybe it should?

603–604 Maybe note that the McBride et al. study found respiratory illness rates were 

associated with the concentrations of enterococci but failed to detect an effect 

with gastro-intestinal rates. (So it shouldn't be cited in support of the opposite 

pattern, as has been done on line 603.) Also Fleisher et al. (1998) reported 

"Attributable proportions of illness" as 34.5% for GI but a higher value (40.0%) 

for AFRI. So is the text on these two lines correct?

716 Hypothesis tests of no difference should not "accept" that hypothesis when the pvalue exceeds the significance level. It suffices to say that differences were not 

statistically significant (there always is a difference between two populations, so 

it makes no sense to say that there isn't). Null hypothesis test procedures are 

devised by assuming that the null is true, and only reject that hypothesis if data 

are sufficiently convincing. On logical grounds that procedure doesn't admit an 

inference that the null is true (or even likely to be true). 

837–843 The Corbett study did detect an association with respiratory illness however.

969 Figure 4 doesn't say which GI category is being graphed; it should (i.e., NGI).

977 Wymer et al., 2011 is Appendix B.3; that needs to be clarified.
Graham McBride 

C-34 

1355 Not sure why there's an emphasis on outbreaks here. The majority of the human 

disease burden manifests as endemic or sporadic patterns cf. outbreaks. And, in 

particular, campylobacteriosis seldom appears as outbreaks yet can have a high 

endemic rate (that's certainly the case in New Zealand, Till & McBride 2004).

1411–1412 I don't understand this text: "The few studies conducted in non-POTW impacted 

waters that also report significant health effects (McBride, 1998; Cheung, 1990; 

and Wiedenmann 2006) have ... 2) combined epidemiological data from beaches 

with non-human fecal source impacts with data from beaches impacted by human 

fecal contamination sources." The New Zealand study did have beaches with 

human impacts and beaches with animal impacts, but the analysis did consider 

them separately (as well as combined).

1428 Suggest replace "Thus, waterbodies with substantial animal inputs" with "Thus, 

waterbodies with substantial farmed animal inputs". Note too that "McBride, 

1998" (lines 1409 & 1428) should be "McBride et al., 1998".

1460–1461 "The GM is calculated as the arithmetic mean of the log-transformed densities" 

should be "The GM is calculated as the antilog of the arithmetic mean of the logtransformed densities".

1471 Maybe elaborate a little more on what 40 CFR 122.45 is all about? 

1532 Is "log standard deviation" ambiguous? Some might interpret it to means what it 

says! Should it be the mean the standard deviation of the logarithms (to base 10?) 

of the data?

1835–1838 Is Colford's Mission Beach study an exception to this human-impacted category?

1918–1927 Another use of QMRA is to assess "what if" scenarios. I argue that's an important 

use of it, one that epidemiological studies can't perform.

1944–1949 Were any of the epidemiological studies conducted close to an outfall? If not, is 

there a case for advocating, say, QMRA in those circumstances (as is done in the 

New Zealand Guidelines (at pp. 3&4): 

http://www.mfe.govt.nz/publications/water/microbiological-quality-jun03/). 
C-35 

Rachel Noble 

Associate Professor 

UNC-Chapel Hill 

Institute of Marine Sciences 

Morehead City, NC 
C-36 

Dr. Noble is an Associate Professor at the University of North Carolina at Chapel Hill, Institute 

of Marine Sciences in Morehead City, North Carolina. She shares joint appointments with the 

Institute for the Environment, the Department of Marine Sciences (within the College of 

Letters, Arts and Sciences), and the Department of Environmental Sciences and Engineering 

(within the Gillings School of Global Public Health). Her research program currently bridges 

the connections between the fields of environmental microbiology and microbial ecology. She 

is currently studying the hydrological patterns of stormwater as related to microbial contaminant 

fate and transport, with a focus the microbial contaminant impact on North Carolina’s vital 

recreational and shellfish harvesting waters. She is also working in eastern North Carolina 

estuaries to study the ecology of pathogens such as Vibrio vulnificus and Vibrio 

parahaemolyticus. These bacteria are relatives of Vibrio cholera, and are native to estuarine 

environments. Noble’s work with other UNC investigators is to elucidate the role of climate 

change and extreme events on the incidence of these pathogens, and the relationship to human 

health risk. Other components of Dr. Noble’s research are to quantify enteric and bacterial 

human pathogens in a variety of environments including stormwater runoff, wastewater 

effluents, and nuisance flows with a focus on coastal areas. Dr. Noble has also been strongly 

interested in the development of rapid (<3 hours) real-time quantification methods for bacteria 

such as E. coli and Enterococcus in surface waters. She has developed a suite of Quantitative 

Polymerase Chain Reaction-based assays that are currently being evaluated for monitoring of 

coastal, estuarine, and inland waters. 

Other strong interests of Dr. Noble are the burgeoning field of the “Ecology of Marine 

Infectious Disease” in which she was recently convened an NSF-funded workshop, and the 

advancement of minority students to technically adept positions in science through her work 

with the HBCU-RISE NSF Program (Historically Black Colleges and Universities-Research 

Infrastructure in Science and Engineering) through which she has a current award with Florida 

A&M University. 
Rachel Noble 

C-37 

Peer Review of the Recreational Water Quality Criteria (RWQC) 

Comments Submitted by Rachel Noble 

Introductory Comments 

Overall, EPA has successfully synthesized a document to convey their current directions in development 

of new criteria for ambient waters. The document was generally well written, and most of the scientific 

arguments within were made with appropriate scientific support and documentation. In particular, EPA is 

to be commended for the sheer amount of water quality related science that they have conducted in the 

past 5 years. They have gained much new information that is relevant to criteria development. One of the 

most important overarching critiques of the criteria document is that they sell their excellent recent 

science short in that they do not use it to its potential for criteria development. Some of the comments in 

the following sections are “bold” formatted in order to highlight importance. 

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for 

technical conclusions transparent? Do you have any suggestions on how we could 

improve this document? Please explain your responses. 

General comments: The criteria document as assembled successfully accomplishes the following 

• presents the history behind previous criteria development 

• discusses a range of aspects of 304 (a) criteria development as related to a range of water quality 

standards/procedures (i.e., does not focus solely on beach warnings and postings) 

• presents the scope of the science 

• presents the linkage b/w water quality and health and explains epidemiological studies 

• attempts to outline a framework for connection of recent epidemiological studies to those 

conducted in the 70s and 80s 

• presents results of epidemiological studies conducted recently and subsequent data analyses 

• attempts to show the reader a transitive approach toward linkage of current studies to older studies, 

bringing in results from rapid methods 

• outlines a few other ways that waterbodies can be managed in the USA 

The way the document is written is generally a treatise on the “process” that EPA used to develop or 

revise criteria. The document is not written in a forward thinking tone (prospective), it is rather 

retrospective. It is this reviewer’s opinion that detailed explanation, use of case studies, and providing of 

examples where relevant could greatly help EPA in paving a path forward for adoption of new criteria. 
Rachel Noble 

C-38 

There are serious shortcomings in the document in the following areas: 

A. The existence of the FIB to pathogen relationship is specifically stated in a wide range of places in the 

document. It is also noted that there is a lack of pathogen relationships to FIB, especially viruses. Why 

continue with the notion that FIB predict pathogens? There is ample evidence at this time that the 

numerical relationships between pathogens and FIB in waterbodies, freshwater and marine, exists only in 

certain cases, and often times does not exist. Typically, strong relationships have been shown between 

viral pathogens and FIB in cases of studies of direct discharge of sewage. It is likely that the poor 

relationship is a function of methodologies, statistical variability, lack of consistent study approaches, 

and/or lack of data using like methods, but this is not important to the current thinking in criteria 

development. EPA has clearly shown, as well as others, that FIB (measured using either culture- or 

molecular-based methods) can be linked to human health outcomes using epidemiology studies. EPA has 

stated that they are using the FIB-health outcomes relationship wholly to establish and defend and 

develop new criteria. The addition of information, and attempt at literature review of pathogens only 

complicates this criteria setting. If anything this information presented about FIB to pathogen 

relationships should be placed in the QMRA section of the document (or in Section 5.0 broadly), to 

demonstrate to the reader the paucity of good, scientifically relevant information. Therefore, paving the 

way for a consistent, statistically rigorous QMRA approach to be presented by EPA. 

B. There is a complete lack of guidance on subtropical and tropical waters criteria development. EPA 

attempted one epidemiological study in tropical waters, and has not conducted any others. The one study 

they did conduct was conducted almost entirely with QPCR based methods, a mistake given the levels of 

inhibition that were identified in these tropical waters. Given the importance of beach visitorship in FL, 

HI, and PR alone, this issue remains important. Hawaii has been urging EPA for decades to examine and 

assist them with development of other standards, citing serious issues with attempting to quantify 

Enterococcus solely as an appropriate indicator. Lack of information about Tropical Areas means that 

these areas should either be able to develop their own approaches OR that they can use the 1986 criteria. 

However simple use of the 1986 criteria means that they are using science that is nearly 40 years old, and 

has not been updated. Tropical Workshop attendees in 2002 said that EPA would develop new indicators 

that worked in all areas. This clearly is not the case for subtropical and tropical waters. Therefore, EPA 

should be required to assist these areas in developing an approach that will be successful for them, even if 

it represents a departure from the national standard. Tropical waters need to be specifically called out in 

the criteria document. 

C. There is no stated requirement for QPCR, which is a good thing as it is not the answer for some State’s 

monitoring needs. However, EPA misses an opportunity to stress the importance of rapid methods in 

providing more accurate means to protect public health at beaches. There are serious issues with public 

perception of the costs and difficulties seen with rapid QPCR based methods. That is, most people think 

that QPCR is more cumbersome and expensive than it really is. EPA could do a better job of 

demonstrating the tractability of these methods to stakeholders. They use Griffith and Weisberg, 2011 

several times as an example. However, it would be useful to explain the costs and advantages of rapid 

methods versus culture based methods, and to provide more specific examples of where it has been used 

successfully to manage recreational waters (southern California, Great Lakes). EPA is unlikely to revise 

criteria in the next decade, therefore, they should consider recommendations of the following for QPCR: 
Rachel Noble 

C-39 

a) implementation at high use beaches (this has been very successful in southern California), b) 

implementation in areas that have high numbers of beach going families including children. EPA is 

making the right decision in not requiring QPCR to be conducted nationwide, but there are merits to its 

use, particularly important is the speed of management decisions. The case could be made by EPA to 

strengthen its recommendations for the use of QPCR (possibly some sort of incentive program?). Finally, 

EPA’s Method A is very complete and detailed. Is it possible for EPA to create a shortened, Cliff’s Notes 

version of Method A to provide to stakeholders so that they can visualize the process? (i.e., collect water 

sample, filter, bead beat, create master mix, load machine). A simplified version of the method would 

assist EPA greatly in bridging the next steps of rapid methods implementation. 

D. The intended switch from SSM to STV is not clearly presented and does not permit coastal and noncoastal States that use other management approaches to incorporate any flexibility in their current ways of 

managing and listing impaired waters. Some states use frequency of beach closures to manage their 

impaired water status, implementing a comparison between reference beaches and contaminated beaches. 

EPA needs to clearly state the minimum sampling frequencies (this is done but the sampling frequencies 

are actually less stringent than before?), they need to provide guidance on sampling during wet versus dry 

periods and over seasons. The frequency of beach closures approach, which compares like beaches to like 

beaches, is an important interpretation of beach management framework. Flexibility in the use of other 

more stringent approaches for compliance purposes needs to be included in the document. Sampling 

variability across states will lead to inconsistency in the way that the new STV approach is utilized for 

TMDL listings. EPA should strive now to make this approach consistent across states and provide 

succinct guidance on the minimum approaches. Currently, States that do the most monitoring spend the 

most money on TMDL implementation and development. These states also suffer from the fact that more 

monitoring equals more listings. EPA needs to create equality in the system by mandating specific levels 

of monitoring. 

E. There are major shortcomings in QPCR/rapid methods as presented. There are major holes in the 

recommendations on rapid methods that deserve to be addressed in the document. EPA needs to more 

transparently state that they did not include rapid E. coli QPCR nor culture-based measures of E. coli in 

the freshwater NEEARS Studies. E. coli is an important indicator in a range of different non-EPA 

epidemiology studies, and the lack of data for the NEEARS freshwater studies needs to be stressed. 

Second, EPA is suggesting an extrapolation of E. coli standards to inland waters based upon old science 

(1986 science), this needs to be clearly stated and transparently discussed in the document. EPA has either 

not done or has not presented the appropriate data analysis across quantification approaches for QPCR to 

demonstrate that the more variable CCE quantification approach is superior to the cell equivalent (CE) 

quantification approach. In southern California, CE based quantification approaches have been used for 

two consecutive summers to rapidly measure beach water quality via Enterococcus quantification with 

some success. The approaches have demonstrated very strong relationships to existing culture-based 

methods. SCCWRP epidemiology studies have also shown that both CE and CCE quantification 

approaches have yielded significant relationships to human health outcomes. EPA has also shown that 

this relationship is significant for both quantification methods, but they are not transparent about this in 

the criteria document. EPA has not included relevant rapid methods data from a range of methods for 

SCCWRP conducted southern CA studies (including QPCR but also highlighting other methods like 

IMS-ATP). Some of this data is already in publication format (submitted to journal) and easily 
Rachel Noble 

C-40 

interpretable. The rapid methods information that is presented can be related with some difficulty to the 

culture based methods, but there are locations where both equivalency (numerical correlation) and/or 

health relationships exist that are different from the criteria development stated by EPA. Clearly, EPA 

will pick only one quantification approach, but addressing both quantification approaches will be 

important for understanding how modifications of the EPA QPCR methods could be implemented by 

others in the future. 

F. EPA has not fully utilized all of the epidemiological data available for informing their new 

criteria. In particular, as stated above the SCCWRP conducted epidemiological studies provide 

important information on the importance of assessing inhibition, on the use of cell equivalent (CE) 

based quantification approaches, epidemiology at non POTW impacted high energy beaches, and 

comparisons of EPA methods to other available rapid methods. EPA needs to take advantage of 

this available body of work. 

G. In places it appears that the criteria document is difficult to read and vague. The following issues exist, 

a) sections could be reorganized to flow better, b) that specific sections have not had the benefit of 

consideration as totally as others (specifically the sections about waterbody type and criteria setting across 

coastal and non-coastal waters, and c) that EPA is presenting the “process” of how they arrived at the 

criteria, but they are providing very little foresight as to actual implementation. In this way, they miss out 

on an opportunity to provide specific guidance on a) successful implementation of QMRA, b) what 

constitutes enough side by side sampling between culture based methods and QPCR, c) what constitutes a 

similar enough method to Method A and how they go about presenting that information, d) examples of 

the use of STV in TMDL impairment listings, and how the new approach will change existing 

approaches. 

2. Are you aware of any important peer reviewed literature or applicable data that have 

not been incorporated in this document and should be included? 

There are four major missing sets of documentation. First, the epidemiological studies conducted by 

SCCWRP on the west coast are mentioned. However, these studies are not incorporated at all into the 

discussion, except under the context of NPS contaminated beaches and the literature produced as part of 

the Mission Bay, CA study (e.g., Colford et al., 2005). The beaches studied in CA are 1) major beaches 

for recreation, 2) representative largely of beaches found in southern California, and 3) well studied and 

have results at this point assembled into the framework of journal articles that have been submitted or are 

ready to submit. Therefore, the data from these studies is far enough along that EPA could more 

effectively make use of the results. SCCWRP has been active in sharing all of the data generated from 

these studies, and it is to EPA’s benefit to take advantage of this information. EPA has full access to this 

data, and the beaches studied in southern California represent a vital subset of beaches that have not been 

adequately addressed by EPA (i.e., beaches that are recipients of NPS pollution). That is, those beaches 

that are high use, high energy (wave energy), have significant contributions of diffuse pollution 

(specifically San Juan Creek and Malibu Creek are two major conduits of concern), and are not impacted 

by any direct POTW point source. These types of beaches dominate southern California and Florida 

waters, and while Florida waters are subtropical in nature, the data collected from these beaches is a vital 
Rachel Noble 

C-41 

part of the package of information that EPA should be examining. Additionally, more emphatic inclusion 

of the Epibathe data, including a serious examination of the E. coli data and a presentation to the reader 

for how that data relates or differs to other studies that have been conducted in the US would be useful, as 

the omission of E. coli from this document is a serious issue (as noted below). 

Second, EPA has been seriously remiss in their inclusion of E. coli in freshwater studies. The criteria 

document is full of broad level statements about E. coli being adequately addressed. On page 22, EPA 

states that the reason that the E. coli testing was not done was because EPA was taking an approach of a 

single indicator to be used for all purposes. If the approach is to link current criteria to historical criteria,

measurements of E. coli were an obvious necessity. The omission of E. coli in both the culture based 

format and molecular format (qPCR) in the Great Lakes NEEAR studies is of major concern. In other 

parts of documents, EPA has written about the fact that their omission of E. coli measurements is 

warranted (see Appendix A, where the lack of E. coli based information is rationalized). The results of 

Epibathe, Marion et al., 2010, and other studies conducted in freshwaters recently indicates that this is 

clearly untrue. It is this reviewer’s opinion that the EPA must be transparent about the lack of new science 

in relation to E. coli, specifically relating to rapid E. coli QPCR methods. They must address, with 

transparency, their lack of foresight in not measuring E. coli during the NEEAR freshwater studies. In 

2006, there were several offers to EPA by outside entities to analyze the existing archived filters from the 

NEEAR freshwater studies using E. coli qPCR. EPA originally indicated interest in conducting these 

analyses, and but never pursued them fully. The lack of culture based E. coli work conducted in the other 

studies is another major issue. At the very least, E. coli should have clearly been measured by MF 

methods for the NEEAR Great Lakes Studies. EPA could create a section in this criteria document that is 

specifically for freshwater systems, and clearly delineate for the reader what will be accepted, and also 

clearly delineate that there are E. coli qPCR assays that exist for use in recreational water quality. EPA 

must also clearly state that they have not conducted the science to update the E. coli criteria, therefore 

those criteria remain the same as previously suggested in the 1986 document. 

Third, the presented analysis of the relationships between FIB and sources of contamination in coastal and 

non coastal waters is poorly written. It is this reviewer’s opinion that there are three general sets of data a)

marine, b) freshwater-great lakes, and c) inland lakes, rivers and streams. EPA does not have the inland 

waters data to compare at any significant level to the coastal and Great Lakes data, it simply does not 

exist. The comparison between freshwater and marine waters is what needs to be made, as most studies 

are separated out by this delineation. EPA is only confusing the issue with the weakly presented analysis. 

Furthermore, EPA does not describe what portion of the assembled literature was used for their 

assessments in this section, only refers to Appendices A and B broadly. Without presenting any of the 

assembled information, where it came from and how it was preferentially selected from the large body of 

literature, about waterbody type and sources, FIB occurrence, FIB survival, and FIB associations with 

pathogens, the reader is left with a poor description of the summary of this huge literature review. It is 

this reviewer’s opinion that EPA must either completely redo this section, or they must state clearly what 

articles are being used to summarize information and present it specifically and quantitatively. For 

example, there are vast differences simply between decay rates in seawater and freshwaters, how can EPA 

state that these differences are not due to the water matrix, but due to site-specific differences? The line of 

reasoning in this section is completely unfounded, and is problematic to the FIB and pathogen occurrence 

section and the other sections. EPA has done a poor job in specifying exactly what patterns have been 
Rachel Noble 

C-42 

seen where, and a very poor job in stating what references have been used to make their statements and 

how any metadata analysis was done. 

Fourth, EPA has preferentially selected a few epidemiological studies to highlight in the criteria 

document, on the freshwater side Epibathe, Ferley et al., Weidenmann et al., and Marion 2010. They have 

also highlighted information about sources of fecal contamination in relation to human health outcomes 

(specifically stating that many epi studies of NPS contamination have found weak or inexistent 

relationships with FIB). EPA speaks to relationships between E. coli and human health, but this 

information is sporadic and not presented with any consistency or clarity. EPA must clearly state that the 

previously stated studies (listed above) have found significant relationships to E. coli in freshwater, and 

that EPA has not conducted the necessary science to change the E. coli criteria substantially from 1986. 

EPA need to clearly state the need for E. coli based water quality assessments and epidemiological studies 

to be conducted so as to update the science in a similar way to what has been accomplished with 

Enterococcus, including the implementation of already available rapid methods (Lavender and 

Kinzelman, 2010). 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based 

indicator data aggregated by days of similar water quality corroborate that the 

NEEAR culture data are generally consistent with the 1986 regression models? 

EPA has made an attempt to link across studies in a transitive argument, and they present one way of 

examining the information. However, issues exist with the following that deserve to be explained. 

• Models used in 1986 are different from the models used for the NEEARS studies that include 

confounding factors. EPA should be transparent as to why the modeling approaches are different 

(assume that raw data from 1986 is not available?) and explain clearly why they are using the cross 

walking approach given this reason. EPA could be more proactive in stating that the criteria 

development is linked to protection of public health 

• Few data points make it difficult to assess the similarities across studies (Is it possible to remove 

the seasonal aggregation presentation?), there are other possible ways to present and analyze the 

data, why has EPA chosen only these examples? 

• It is generally unclear to the reader what final data was combined data and what final data was 

specific to a study or beach? Also, it is unclear what modeling approaches were used to generate 

the health risk assessments for each specific study. 

• Why no presentation of Bacteroidales information? Can it be used as a tool in section 5.0? This 

indicator may have particular relevance in tropical waters, where regrowth of E. coli and 

Enterococcus has been shown to be a serious issue. 

• It would help the reader to see information from Wade et al., 2008 and 2010 showing relationships 

to human health above and below specific standards. 
Rachel Noble 

C-43 

• No mention or incorporation of SCCWRP studies makes it difficult to understand what selection 

process was for the epidemiological information? How do the SCCWRP results relate to the 

marine NEEAR studies? Are there large differences in rapid method and culture method results? 

How about presenting a table, for transparency, that shows what studies individually showed a 

relationship to human health for what methods and what studies didn’t? In relation to EPA 

conducted studies, the SCCWRP studies are some of the most extensive, and they specifically 

followed EPA experimental design, so the lack of inclusion of this information is curious. 

• EPA does a poor job of explaining why they have included certain studies in their considerations 

and not others, is it possible that these other studies generated information that make this 

aggregation of data and cross talk across criteria establishment confounded? It generally appears 

that the EPA studies only have been included in the final criteria development. While this is a 

decision for EPA to make, it needs to be justified and made clear throughout the document. There 

is a strong comparison between the Marion et al. study and EPA studies in the Appendices, but it 

is not clear why that kind of comparison is not conducted with an array of recent rigorous studies 

conducted by others. 

4. Do the characterization and explanation of the qPCR method reflect that of the 

scientific literature? If not, what additional information should be included? Is the 

derivation of the qPCR value using the EPA data scientifically supported and 

adequately explained? Please discuss why or why not. 

EPA presents a method that is highly successful to quantifying Enterococcus in water. The information, 

however is lacking in several specific areas. 

• QPCR criteria has been shown, but the linkage between this criteria for actual studies conducted 

recently and illness has not been shown. EPA needs to specifically show that QPCR relationships 

to human health in freshwaters are stronger than those for culture based methods. The reader should 

be presented with the actual illness rates that would have resulted in a range of EPA and non EPA 

studies if this selected criteria was retrospectively applied. As a matter of fact, it would be useful 

to see this done for every epidemiological study conducted recently that utilized Enterococcus qPCR. 

• The scientific literature is full of a range of quantification approaches for qPCR. The 

rationalization and explanation for the selection of the CCE or calibrated cell equivalent 

quantification approach for QPCR is not adequate. The lack of inclusion of the results of the 

SCCWRP epidemiological studies is vital in this rationalization, along with other work that has 

been conducted in the Great Lakes using E. coli QPCR assays. EPA is missing a valuable 

opportunity to explain this to the public/stakeholders. They also must present their epi data in both 

the cell equivalent and the calibrated cell equivalent format, in the context of inhibition, and at 

least let the reader see that there are serious advantages and disadvantages to each approach. There 

are other general approaches (using the CE approach with inhibition flagged) to manage beach 

waters that have been presented by the State of CA, and while these are mentioned they are not 

explained. EPA should present this information and be transparent. EPA must address these issues 

and present a sequence of rational arguments for selecting the calibrated cell equivalent approach. 
Rachel Noble 

C-44 

• EPA presents the method, but provides almost no guidance as to how others can use similar 

methods, like IMS ATP, are there possibilities for methods like IMS ATP? 

• The presentation of method specific details is limited to Appendix A, where EPA quantifies cross 

laboratory variability, and 50% limit of detection. EPA must provide a better example of what type 

of variability they expect from rapid methods as compared to culture based methods. This is vital 

when considering posting a beach based upon a single suggested value. Possibly a section that 

compares the methods and shows a shortened version of the methods comparisons that have been 

done and the variability observed. 

5. Please provide any other general or specific comments on the document, making sure 

to identify the section number/page number you are commenting on. 

Sections 3.0, 3.1, and 3.2 are poorly written and need to be rewritten and reorganized. The sections on this 

history of criteria in the beginning of the document are much more clearly written and presented. In 

Appendix C: ASSESSMENT OF THE APPLICABILITY OF EXISTING EPIDEMIOLOGY DATA TO 

INLAND WATERS, there are statements that are contradictory to those written in section 3.0+ of the 

criteria document. In particular, the data analysis is not possible given the range of methods that were 

used to conduct coastal versus inland waters assessments. The document clearly states the caveats 

associated with interpreting the data across sites and methods. EPA clearly states the variability observed 

from site to site at beaches, then there is a huge effort expended to show the linkage across coastal and 

non-coastal waters. It is simply not possible to state that coastal and non coastal waters behave similarly 

in relation to FIB dynamics. 

Specific comments: Use either Enterococcus sp. or enterococci throughout 

FIB = fecal indicator bacteria, typically in the literature, this refers to total coliforms, fecal coliforms, E. 

coli or Enterococcus. Where you mean FIB use FIB 

Fecal indicator and fecal indicator organisms (FIO) include all indicator organisms. Most typically use 

this as a catchall for all FIB, plus other “indicator” bacteria that can sometimes include other bacteria such 

as C. perfringens. It is this reviewer’s opinion that in the criteria document, in almost all instances seen 

the EPA should consistently be using FIB specifically. 

If going to define acronyms for all, use them! Specifically use FIB please, not “fecal indicators” which 

could be inclusive of enteric viruses, protozoa, chemical indicators of fecal pollution etc. 

EPA could please include: 

• A basic example of what constitutes an adequate side by side testing scheme for implementation of 

qPCR and if states have already done this, how they can “bypass” this step. 

• A QMRA case study, or at least an example of what would be adequate. 
Rachel Noble 

C-45 

• More information as to how spatial and temporal sampling frequency plays into GM and STV. 

There is little mention of how EPA selected to reduce (?) the required sampling frequency for STV 

(12 for 90 days as opposed to 5 for 30 days?), what information has been assembled to defend this? 

• EPA needs to at least discuss frequency of beach closures in a reference beach vs. contaminated 

beach context in terms of impairment listing, and they need to defend their approach, or specifically 

accept other approaches that have been scientifically and rigorously established, also need to defend 

the use of mixed information in impairment assessment, can an agency mix and match QPCR and 

culture based data for STV assessments? What is the reasoning for accepting this approach? Not 

accepting this approach? 

• Are all E. coli GM and STV assessments to be conducted the same as Enterococcus? 

• What about use attainability. There is very little information provided in this section. EPA has done 

a poor job in the past of describing how to establish tiered approaches. Now that they are doing 

away with this system, they need to address how this will impact states that are short on resources. 

States might find themselves in a situation where they are expending resources at a beach that has 1 

user per month, and reducing resources at their higher use beaches. This is an undesired effect of 

the tiered system, but a reality in tight budget times. This reviewer is already aware of several states 

that have suggested that they will have to shift resources away from high use beaches to mostly 

unused beaches if the tiered system is abandoned. There needs to be a clearly thought out process 

and presentation of the information to stakeholders, as this will impact those in the water quality 

agencies greatly. Are they expected to maintain all of their sampling sites that they currently sample 

with lower frequencies at the 90 day 12 sample minimum? 

• EPA is to be commended for introducing new scientific tools such as QMRA in the possible arsenal 

of water quality managers. EPA could more adequately explain the QMRA process and provide 

examples (this was already included in other comments). One major problem is that the source 

identification and quantification science that needs to be done has not been done in an organized 

way. EPA should take the lead on this. With specific personnel at EPA (Orin Shanks) this work can 

be completed successfully, but there must be a plan toward doing it at a national level. EPA has not 

specified how agencies/stakeholders sift through available source identification and quantification 

approaches to select something that is rigorous and can be used successfully. Finally, the linkage to 

FIB is not clearly made in the criteria document or any guidance that EPA has published. Guidance 

on the linkage of source specific information specifically to either E. coli or Enterococcus has not 

been provided. This is a vital step toward the successful use of QMRA as an additive to a 

management schema for surface waters. 

• QPCR methods, EPA has not specified what is acceptable for equivalency of rapid methods (i.e., 

use of Method A with a different QPCR platform, how many side by side samples need to be run to 

demonstrate equivalency between methods?). This needs to be done now before other classes of 

methods arrive. Set the stage for other criteria development, be novel and thoughtful about this 

process. If want to use a different platform, specify the acceptable criteria. If want to use a different 

available tested method, define what is acceptable. It is assumed that much of this information will 

be in a Topic Specific Guidance, but some precursor information is desirable. 

C-47 

Gordon Yasvinski 

Scientific Evaluator 

Health Canada 

Ottawa, Ontario 

Canada 
C-48 

Gordon Yasvinski graduated from the University of Guelph with an Honours degree in 

Microbiology and has been with the Water, Air and Climate Change Bureau since 2003. His 

responsibilities include the preparation of health risk assessments for microbiological 

contaminants specific to drinking water and recreational water. Publications he has contributed 

to the development or review of include the Guideline Technical Documents for: Bacterial 

waterborne pathogens, Enteric protozoa, E. coli, Total coliforms and Turbidity. Gordon also led 

the update of Health Canada’s Guidelines for Canadian Recreational Water Quality with a 

Federal-Provincial-Territorial Working Group on Recreational Water Quality made up of 

representatives from Health Canada, provincial and territorial departments responsible for water 

quality, Environment Canada and the U.S. EPA. 
Gordon Yasvinski 

C-49 

Peer Review of the Recreational Water Quality Criteria (RWQC)

Comments Submitted by Gordon Yasvinski

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for 

technical conclusions transparent? Do you have any suggestions on how we could 

improve this document? Please explain your responses. 

It is clear that a tremendous amount of effort has been put in to developing these recommendations and 

their supporting documentation. In terms of adequate and transparent presentation of the conclusions, I 

would say nearly. In general, the conclusions are rooted in logic the depth of the literature reviewed 

scientifically supported, and EPA has gathered/ reviewed and produced sufficient information to support 

its risk assessment decisions. 

1. There are some decisions that need to be better illustrated and supported or defended in the 2012 

document. Chiefly, the ones I would highlight are: 

• The findings around the fresh/marine equivalency would benefit from further illustration (question 

#3) 

• Stronger supporting discussion should be provided as to why the stronger qPCR results were not 

used to drive the criteria as opposed to the culture-based results (question #4) 

• Supporting discussion as to why the qPCR criteria remain expressed as NGI illness, while the 

culture based criteria are expressed as HCGI, even though the discussion seemed to suggest an 

easy conversion or interchangeability between the two (question #4) 

• More specific discussion around why E. coli, though not included in the NEEAR analyses is 

retained and justified in its role as the primary 

2. Additionally on the subject of transparency and completeness, the review panel discussed at the 

meeting that on the topics of: 

• Coastal vs. Inland waters 

• Tropical/subtropical vs. Temperate waters 

and perhaps to a slightly lesser extent: 

• Human source fecal material vs. Non-human source fecal material 

The basic assessment seemed valid — that the data variability and gaps were sufficient so as to suggest 

that the health risks for each type were similar enough and could not be independently characterized. 

However taking the direction of similarity in the discussion was done at the expense of acknowledging 

that there are also some real apparent differences between each comparison pair, but that the impact of 
Gordon Yasvinski 

C-50 

this differences is not known or it is not known how significant. Acknowledging these differences and the 

data limitations is required to convey to the reader that EPA is aware of and has considered all the 

available information in presenting their decisions. 

The review panel further discussed that a more intuitive approach would be to structure the basis of the 

argument that extension of criteria across recreational water types was done to be protective of public 

health in the lack of sufficient information. This decision would be rooted in logic and allow EPA to 

acknowledge the differences and the data gaps. 

3. Another issue discussed by the review panel that was not addressed in my draft comments, but was 

captured in my specific comments concerned the clarity of the guidance for applying the criteria. It is 

proposed that this section needs to be rewritten in a more clear and straightforward manner. 

Currently, the explanation of the statistics, the rationale, the calculation of the statistics and the 

guidance as to application are all intertwined. It is recommended that these be separated for easy of 

understanding, and specifically that the application guidance be specifically laid out in manner that is 

simple and easy for the intended recipient (responsible recreational water authority or beach manager) 

to understand. 

The following editorial comments are also submitted: 

1. It is clear to me that the EPA has completed a tremendous amount of work in preparing the newly 

proposed 2012 criteria. So much so, that it appears that the EPA should be able to confidently defend 

its conclusions and decisions. In the criteria document there appears to be some instances where the 

text in presenting a point, follows up with what might be a response to what expected criticism might 

be. Or similarly, uses a small point to point out consistency with EPA findings, rather than having a 

more complete, organized discussion of additional material that supports EPA’s decisions. As examples: 

• In 3.1, it is introduced that EPA is allowing for a qPCR method, but then immediately following 

the text focuses on the issue of possible interferences and how they have been resolved. 

• In 3.2.3 in discussing the body of epidemiological literature, after mentioning the Marion, 2010 

study, it states that the E. coli findings support EPA’s 1986 criteria, rather than having a more 

coordinated discussion of how the outside body of evidence is also in line with EPA’s conclusions. 

I understand this is an extremely complex document with many issues that need to be addressed, and 

doing so in a logical and easy-to-read format is challenging. However I believe with the major issues 

to be addressed (e.g., the criteria itself, water body type, source of fecal contamination), the work put 

into this endeavour is such that the EPA should be able to present its findings and recommendations 

and then follow up with short explanations for each major issue - yes, here are the known challenges 

or shortcomings...and we believe these are addressed/justified because we have done this or have the 

following information which supports it. 

2. Another suggestion is that it would be valuable to have a brief opening summary or executive 

summary of the key decisions or the “need-to-know” information for the states, tribes and readers. 

The criteria has a lot of components and a lot of questions to be answered and it is difficult to tease 

out the key conclusions from the individual sections. 
Gordon Yasvinski 

C-51 

2. Are you aware of any important peer reviewed literature or applicable data that have 

not been incorporated in this document and should be included? 

No. The data analysis and the literature review conducted for the multiple components the criteria 

document considers (e.g., epidemiological studies, QMRA and other publications in the appendices) are 

very comprehensive and appear to be sufficiently exhaustive. I feel comfortable that the body of 

information has been suitably reviewed. The odd paper could be included to provide reference for specific 

points, as other reviewers have indicated, but I believe that no crucial streams of evidence have been 

overlooked. Two general comments I would have are as follows:

1. It is proposed that better linkage of individual points or conclusions to the respective Appendix 

subsections would be extremely valuable and help improve transparency. 

2. It is also felt that an important opportunity to address the concept of the move worldwide towards a 

harmonized approach to risk management and the move away from reliance on numerical values of 

fecal indicator bacteria as the sole criterion for determining recreational water safety was missed. 

Although it is recognized that this is a criteria document and not a beach management document, the 

concept is in-line with EPA thinking. It is therefore important to highlight and ingrain this idea in the 

public consciousness and to also use the opportunity to point out the interconnectiveness between 

related EPA documents (e.g., sanitary survey and predictive model documents) to authorities and 

managers. 

Suitable citations for the Harmonized concept include: 

Microbiological Water Quality Guidelines for Marine and Fresh Water Recreational Areas. New Zealand 

Ministry for the Environment. 2002 

EU (2006). Official Journal of the European Union. Directive 2006/7/EC of the European Parliament and 

of the Council of 15 February 2006 concerning the management of bathing water quality and 

repealing Directive 76/160/EEC 

NHMRC (2008). Guidelines for managing risks in recreational water. National Health and Medical 

Research Council of Australia, Government of Australia, Canberra. 

Health Canada (2009). Guidelines for Canadian recreational water quality. (Draft) Document for Public 

Comment. Water Quality and Health Bureau, Healthy Environments and Consumer Safety Branch, 

Health Canada, Ottawa, Ontario. Available at: http://www.hc-sc.gc.ca/ewhsemt/consult/_2009/water_rec-eau/draft-ebauche-eng.php 
Gordon Yasvinski 

C-52 

3. Do the analyses comparing the NEEAR freshwater and marine culture-based 

indicator data aggregated by days of similar water quality corroborate that the 

NEEAR culture data are generally consistent with the 1986 regression models? 

Not quite. My initial comments to this question were to propose that the findings that prompted the 

decisions of equivalency needs to be illustrated further, through a combination of graphical data showing 

the marine and freshwater data combined and an indication of the statistical calculations showing no 

statistical difference. Further to this, a valuable discussion would be as to how although the water quality 

data at the beaches was shown to be very good, it was felt that there was sufficient data captured at points 

where swimmers would be at elevated risk so as to make the data sets and observed relationships robust. 

EPA’s desire for pursuing unified illness rates for fresh and marine waters for public health protection and 

for application and implementation purposes is understood. At the meeting the review panel discussed 

that Figure 1. could be regarded as doing a disservice to the argument of marine/freshwater equivalency, 

but since it is addressed as being insufficient to refine the illness estimates, it’s inclusion was considered 

valuable for transparency purposes. The reviewers further addressed that in the absence of a suitablyillustrated supporting discussion, the apparent lack of relationships in Figure 2 questions whether the 

NEEAR data and the 1986 data are sufficiently similar. Ultimately the review panel suggested (and I 

support) that the NEEAR study data be used on its own be used to determine conclusions regarding fresh 

and marine water illness rate similarity. 

Guidelines or criteria can have several components beyond the straight scientific data component. These 

additional components can also impact or shape the way the final recommendations are presented: 

• Historical component 

• Public health decision component 

• Jurisdictional component 

• Practicality of Implementation component. 

In my experience on the work done in developing the (draft) recommendations for the Guidelines for 

Canadian Recreational Water Quality, the current fecal indicator recommendations were similarly based 

on assessment of the best available scientific data. However, because of the data limitations, and the lack 

of an exact parameter-to-illness causative relationship, the recommendations are not identified as illness 

thresholds, but rather as risk management values intended to be protective of public health. 

Whatever EPA’s decision regarding a tolerable illness rate and marine-freshwater rate equivalency – 

whether it be weighted heavier or data relationships or on policy/risk management/public health decisions 

– it is recommended that it provided that it appears logical, transparent, acknowledges it’s limitations and 

fits within EPA’s scope of authority, it will be capable of being defended. If not, it will give the 

appearance of potentially hiding something, and therefore will likely be subject to criticism, whether it is 

warranted or not. 
Gordon Yasvinski 

C-53 

4. Do the characterization and explanation of the qPCR method reflect that of the 

scientific literature? If not, what additional information should be included? Is the 

derivation of the qPCR value using the EPA data scientifically supported and 

adequately explained? Please discuss why or why not. 

As this is not my specific area of expertise, and others on the review panel are more qualified than I to 

comment on the accuracy and appropriateness of the methods, I will withhold from commenting on this 

particular area. However, in terms of the derivation of the qPCR value, this does not quite feel adequately 

explained. Illustration showing comparison of NEEAR data to the calibration curves with explanation of 

how the relationships observed are felt to be valid would be helpful. 

Additionally, an expanded explanation to the reader as to why the qPCR criteria seem considerably 

elevated relative to the culture criteria, and how these are logical and are justified, would be useful 

5. Please provide any other general or specific comments on the document, making sure 

to identify the section number/page number you are commenting on. 

File containing marked specific comments submitted with the draft comment file remains unchanged. 

D-1 

Appendix D 

Meeting Agenda 

D-3 

United States 

Environmental Protection Agency 

Office of Water 

Peer Review Meeting for EPA’s Draft Recreational Water Quality 

Criteria for the Protection of Human Health 

September 12-13, 2011 

Arlington, VA 

Agenda 

Monday, September 12, 2011 

8:30 a.m. Registration/Check-in 

9:00 a.m. Welcome and Announcements ............................................................................. Kate Schalk, ERG

9:05 a.m. EPA Remarks ................................................................................... Elizabeth Doyle, EPA/OW/HHRA

9:15 a.m. Introductions, Meeting Format and Ground Rules .................................................. Kate Schalk, ERG 

9:30 a.m. Technical Chair’s Remarks .................................................................. Rachel Noble, UNC-Chapel Hill

9:35 a.m. BREAK 

10:00 a.m. Reviewer Discussions .................................................................... Rachel Noble (Chair) & Reviewers

1. Does the document adequately present the logic and scientific support behind its 

conclusions? Please comment on the document’s transparency, the completeness of the 

presentations, and any shortfalls in the material presented. Are uncertainties for technical 

conclusions transparent? Do you have any suggestions on how we could improve this 

document? 

2. Are you aware of any important peer reviewed literature or applicable data that have not 

been incorporated in this document and should be included? 

 

12:00 p.m. LUNCH 

1:15 p.m. 3. Do the analyses comparing the NEEAR freshwater and marine culture-based indicator data 

aggregated by days of similar water quality corroborate that the NEEAR culture data are 

generally consistent with the 1986 regression models? 

2:45 p.m. BREAK 
D-4 

Agenda 

Monday, September 12, 2011 (continued) 

3:00 p.m. 4. Do the characterization and explanation of the qPCR method reflect that of the scientific

literature? If not, what additional information should be included? Is the derivation of 

the qPCR value using the EPA data scientifically supported and adequately explained? 

4:15 p.m. 5. Other general or specific comments not addressed in the charge questions will be 

discussed. 

4:50 p.m. Wrap Up and Plans for Writing Session ......................................................................... Rachel Noble

5:15 p.m. ADJOURN (Formal meeting closes) 

Writing Session 

Tuesday, September 13, 2011 

8:30 a.m. Review of Yesterday’s Discussions and Plans for Report Summary ... Rachel Noble and Kate Schalk

8:45 a.m. Writing Session .................................................................................................... Reviewers and ERG

10:15 a.m. BREAK 

10:30 a.m. Writing Session .................................................................................................... Reviewers and ERG

12:00 p.m. LUNCH (working lunch if needed) 

1:30 p.m. Writing Session .................................................................................................... Reviewers and ERG

3:30 p.m. Wrap Up Writing Session and Instructions for Post Meeting Work ................................... Kate Schalk 

3:45 p.m. Writing Session Closes 
E-1 

Appendix E 

List of Reviewers and Observers 

E-3 

United States 

Environmental Protection Agency 

Office of Water 

Peer Review Meeting for EPA’s Draft Recreational 

Water Quality Criteria for Human Health 

September 12-13, 2011 

Peer Reviewers

Samuel Dorevitch 

Associate Professor 

Division of Environmental and Occupational Health 

Sciences 

School of Public Health 

Institute for Environmental Science and Policy 

University of Illinois at Chicago 

Chicago, IL 

Erin K. Lipp 

Associate Professor 

Department of Environmental Health Science 

College of Public Health 

The University of Georgia 

Athens, GA 

Graham McBride 

Principal Scientist 

National Institute of Water and 

 Atmospheric Research Ltd. (NIWA) 

Hamilton, New Zealand 

Rachel Noble (Technical Chair) 

Associate Professor 

University of North Carolina - Chapel Hill 

Institute of Marine Sciences 

Morehead City, NC 

Gordon Yasvinski 

Scientific Evaluator 

Health Canada 

Ottawa, Ontario 

Canada 
E-4 

United States 

Environmental Protection Agency 

Office of Water 

Observers 

Shamima Akhter 

U.S. Environmental Protection Agency 

Office of Water 

Health and Ecological Criteria Division 

Washington, DC 

Elizabeth Doyle 

U.S. Environmental Protection Agency 

Office of Water 

Health and Ecological Criteria Division 

Washington, DC 

Sharon Nappier

U.S. Environmental Protection Agency 

Office of Water 

Health 

and Ecological Criteria Division 

Washington, DC 

John Ravenscroft

U.S. Environmental Protection Agency 

Office of Water 

Health and Ecological Criteria Division 

Washington, DC 

Jeff Soller

Soller Environmental 

Berkeley, CA 

Contractor Staff

Kate Schalk 

Eastern Research Group, Inc. 

Lexington, MA 

Michelle Arbogast 

Eastern Research Group, Inc. 

Lexington, MA

Remote Observers

Alfred Dufour 

U.S. Environmental Protection Agency 

Office of Research and Development 

National Exposure Research Laboratory 

Cincinatti, OH

Robert Cantilli 

U.S. Environmental Protection Agency 

Office of Research and Development 

Office of Science Policy 

Washington, DC 

Kevin Oshima 

U.S. Environmental Protection Agency 

Office of Research and Development 

National Exposure Research Laboratory 

Cincinatti, OH

Tim Wade 

U.S. Environmental Protection Agency 

Office of Research and Development 

National Health and Environmental Effects 

 Research Laboratory 

Research Triangle Park, NC