# Results of the Inter

- laboratory Method
Validation Study using U.S. Environmental Protection Agency
Method 1623 .1: Cryptosporidium and
Giardia in Water by Filtration/IMS/FAOffice of Water ( MS 140 )
EPA 816

- R

- 12

- 002 February 2012 http://water. epa.gov/ drinkiii 

### Acknowledgments This report was prepared under the direction of Carrie Miller of the Technical Support Center, Standards and Risk Management Division within the U.S. Environmental Protection Agency (EPA) Office of

Water

. It was prepared by Leah Fohl Villegas, PhD and John H. Carson Jr, PhD

, Shaw Environmental
and Infrastructure, Inc. und er EPA Contract: EP

- C

- 06

- 031, Work Assignment: 4

- 02

.



### Disclaimer This method validation study report has been reviewed by U.S. EPA Office of Water and appro ved for publication. Mention of trade names, commercial products , or participating laboratories doe s not
constitute endorsement or recommendation for use.

Questions regarding this report, method or their application should be addressed to:

Carrie Miller Coordinator, Laboratory Quality Assurance Program for the Analysis of Cryptosporidium

U.S. Enviro nmental Protection Agency Office of Ground Water and Drinking Water Technical Support Center, MC140 26 West Martin Luther King Drive
Cincinnati, OH 45268

- 1320 (513) 569

- 7919 (513) 569

- 7191 (fax) miller.carrie@epa .gov Safe Drinking Water Hotline 1

- 800

- 426

- 4791 http://water.epa.gov/drink/hotline/index.cfm

You may obtain copies from: http://water.epa.gov/scitech/drinkingwater/labcert/analyticalmethods_ogwdw.cfmiv 

## Table of Contents

Executive Summary

................................ ................................ ................................ ........................ 1

1.0 Introduction

................................ ................................ ................................ ................................ .. 3
2.0 Study Design and Implementation

................................ ................................ ............................... 4
2.1 Laboratory Selection

................................ ................................ ................................ ................ 5
2.2 Preparation of Spiking Suspensions

................................ ................................ ......................... 5
2.3 Quality Control

................................ ................................ ................................ ......................... 6
3.0 Data Repor ting and Validation

................................ ................................ ................................ ..... 6

4.0 Results

................................ ................................ ................................ ................................ .......... 6
4.1 Reagent Water

................................ ................................ ................................ .......................... 6

4.2 Source Water

................................ ................................ ................................ ............................ 8
5.0 Comparison of Validation Study Results

................................ ................................ ..................... 9
6.0 Development of QC Ac ceptance Criteria

................................ ................................ ................... 10

7.0 Conclusion

................................ ................................ ................................ ................................ . 11

8.0 References

................................ ................................ ................................ ................................ . 12
Appendix A. Instructions to Participating Laboratories:

................................ ................................ . 13
Appendix B. Description of Statistical Analysis Used to Develop QC Acceptance Criteria for
Method 1623.1:

................................ ................................ ................................ ................................ 15v 

## Table of Tables

Table 1: Method 1623.1 Quality Control Acceptance Criteria

................................ ................................ ...................... 2
Table 2: Source Water Specifications

................................ ................................ ................................ ............................. 4 Table 3: Participating Laboratories

................................ ................................ ................................ ............................... 5 Table 4: Results of Quality Control Samples

................................ ................................ ................................ .................. 6 Table 5: Spiked Reagent Water Sample Analysis Results for Cryptosporidium

................................ ............................. 7 Table 6: Spiked Reagent Water Sample Analysis Results for Giardia

................................ ................................ ............ 7 Table 7: Spiked Source Water Sample Analysis Results for Cryptosporidium

................................ ................................ 8 Table 8: Spiked Source Water Sample Analysis Results for Giardia

................................ ................................ ............... 9 Table 9: Comparison of Method 1623.1 Study Results to Previous Validation of Method 1623

................................ . 10 Table 10: Comparison of Study Criteria to Previous Criteria

................................ ................................ ........................ 11
Table B

- 1: Comparison of Study Criteria to Previous Criteria U sing Different Statistical Approaches

......................... 25



## Table of Figures

Figure 1. Results of single laboratory side

- by

- side analysis

................................ ................................ ........................... 3 Figure 2. Results of side

- by

- side analysis in 4 laboratories

................................ ................................ ........................... 3 Figure 3. Distribution of Source Waters

................................ ................................ ................................ ........................ 4 Figure 4. Graphical Comparison of Cryptosporidium Recovery to Previous Validation

................................ .............. 10 Figure B

- 1. Standard deviation versus mean of reagent water spike recovery grouped by laboratory

...................... 16 Figure B

- 2. Standard deviation versus mean of logits of reagent water spike recovery grouped by laboratory

........ 171 

## Executive Summary This report presents the results of the U.S. Environm ental Protection Agency’s (EPA) inter laboratory
validation study of Method 1623.1 : Cryptosporidium and Giardia by Filtration/IMS/FA [1]

. The substantial change s from Method 1623 [2] are the addition of sodium hexametaphosphate and a rinse to
remove excess debris during immunomagnetic separation of the oocysts and cysts

. The interlaboratory validation study of Method 1623.1 was performed after t wo side

- by

- side studies that showed improved accuracy for detecting the concentratio n of Cryptosporidium oocysts in water when using Method 1 623.1 compared with Method 1623. These side

- by

- side studies were 1) a single laboratory comparison of samples processed from nine public water system sources which showed an average improve ment of 18 percentage points with Method 1623.1 (p = 0.0001); and 2) a multi

- laboratory comparison of samples processed from three source waters which resulted in an average improvement of 15 percentage points
with Method 1623.1 (p=0.0197). The more significant improvement in Cryptosporidium recovery during
the side

- by

- side studies was particularly associated with samples that had low initial recovery using Method 1623. For those samples where initial recovery (i.e., with Method 1623) was rel atively higher, the improvement in recovery using Method 1623.1 was generally less pronounced. Both studies demonstrated similar precision and accuracy for Giardia recovery with either m ethod

.

Fourteen volunteer laboratories participated in this validat ion study and were previously approved by the “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium [ 3 ]

. ” Each laboratory analyzed 4 reagent water samples and 4 water samples taken from a public water system

source using Method 1623.1 . Th e site used to collect source water s amples was different for each laboratory and included six lake/reservoirs, seven rivers and one ground water under t he influence of
surface water

. Cryptosporidium mean recoveries of 60 and 61% from reagent and source waters, respectively, represent a ≥ 20 percentage point increase compared to the previous validation of Method
1623 [ 4 ]

. The Giardia mean recoveries of approximately 40% were similar to t he previous m ethod
validation. The mean re lative standard deviation decreased for Cryptosporidium and Giardia in reagent water and for Cryptosporidium in the source water matrices compared with the Method 1623 validation

. The mean relative standard deviation for Giardia in source water remained the same for both validation
studies.

The data from this 1623.1 validation were used to develop quality control criteria f or laboratory performance (Table 1 ). For each criterion, the distribution of recovery was estimated by usi ng random
effects ANOVA

. The recovery limits were estimated as the 5 th percentile of the predictive distribution.
The lower limit acceptable for recovery of Cryptosporidium detected in reagent and source water increased by 11 and 19 percentage points

, respectively, over Method 1623 criteria that were updated in
2009 [ 3 ]. The lower limit for recovery fo r Giardia in reagent water increased by 8 percentage points; however, the recovery for Giardia in source water decreased by 7 percentage points. Laborat ories performing Method 1623.1 are expected to have better detection rates for Cryptosporidium oocysts than those laboratories following Method 1623. Method 1623.1 includes more stringent quality control

measures.2 Table 1 : Method 1623.1 Quality Control Acceptance Criteria

Performance Test Acceptance Criteria Cryptosporidium Giardia Initial precision and recovery (IPR)
Mean percent recovery 38

- 100 2 7

- 100 Precision (as maximum relative standard deviation) 37 39 Ongoing precision and recovery (OPR) 33

- 100 22

- 100 Matrix spike/matrix spike duplicate (MS/MSD)
Mean percent recovery 32

- 100 8

- 100 Precision (as maximum percent difference) 46 973 

## 1.0 Introduction

In an effort since 2009 to improve method accuracy and precision, EPA’s Technical Support Center (TSC) in the Office of Ground Water and Drinking Water, Standards and Risk Management Division, has been evaluating the efficacy of adding sodium hexametaphosphate (NaHMP) to artificial and
environmental raw water matrices. These studies led to the development of Method 1623.1: Cryptosporidium and Giardia by Filtration/IMS/FS [1]. The substantial change from Method 1623 [2] is
the addition of NaHMP p rior to the elution p rocedure for the Envirochek ® HV Sampling C apsule , which are presently used by approximately 80% of the laboratories approved for Cryptosporidium analysis.
NaHMP has been used as an additive during ultrafiltration to reduce filter fouling [ 5 ], and has been used to improve Cryptosporidium detection in waste and finished waters [ 6

, 7 ]. NaHMP improves the dispersion efficacy of surfactants by sequestering ions associated with water hardness, lowering the surface tension, and increasing the zeta potential of pa rticles [ 8 ]

. Method 1623.1 also incorporates a rinse
step to remove excess debris during immunomagnetic separation of the oocysts and cysts

.

In the spring and summer of 2010

, TSC conducted a single

- laboratory side

- by

- side analysis that compared samples from 9 public water systems each processed using both Methods 1623 and 1623.1

. The results demonstrated that recoveries of spiked Cryptosporidium oocysts improved on average by 18 percentage points (p = 0.0001 ) using Method 1623.1 (Figure 1 ). Improvement in recovery was particularly notable when initial recovery (i.e., using Method 1623) was lower (i.e., below 30%). In an April and May 2011 study with four laboratories, Method 1623.1 was compared side

- by

- side with Method 1623 in analysis of six samples collected from three public water system sources (Figure 2). Method 1623.1 recovered an average of 15 percentage points more oocysts over Method 1623 for all three of the waters tested (p=0.0197). Again, the improvement in recovery was much more pronounced for those samples with a low initial recovery.

Figure 1

. Results of single laboratory side

- by

- side analysis Cryptosporidium matrix spike recoveries from 9 sources comparing analyses using Methods 1623 and
1623.1 (n=4). Error bars are ± one standard error of the mean. Figure 2 . Results of side

- by

- side analysis in 4 laboratories Cryptosporidium m atr ix spike recoveries from 3 sources comparing analyses using Methods 1623 and
1623.1 (n=12). Error bars are ± one standard error of the mean.

In June 2011 w e designed this study to provide inter

- laboratory validation of Method 1623.1 and generate
data to calculate quality control criteria that reflect the updated method and current laboratory capability.
L aboratories approved by the “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium [ 3 ], ” as of July 18, 2011, were invited to participate.4 

## 2.0 Study Design and Implementation

On behalf of EPA, Shaw Environmental and Infrastructure Inc. under EPA Contract EP

- C

- 06

- 031 (the referee laboratory) was tasked to design and coordinate the study. T he laboratories participating in this study were required to process and analyze all samples following Method 1623.1 and the instructions listed in Appendix A. The participants were asked to select source water with recoveries representative of
the matric es tested in the ir laboratory , with the exception of one laboratory that was asked to select a ground water under surface water influence. In most instances the laboratory selected source water from a nearby utility to reduce shipping costs. The study in cluded 14 source s from across the United States, consisting of 6 lake/rese rvoirs, 7 rivers and one ground water under the influence of surface water (Figure
3 & Table 2 )

.

Figure 3 . Distribution of Source Waters Rivers are blue t riangles, lake/reservoirs are red circles, and ground water under the influence of surface water is a green square.

Table 2 : Source Water Specifications

Source Water Type Collection
Date Turbidity ( NTU ) Conductivity ( μ S) pH

River 8/15/11 5.6 529 7.89
River 8/10/11 53 527 7.98
Ground Under Influence 8/9/11 2.89 687 6.8
Reservoir 8/18/11 3.77 252 7.5
Reservoir 8/10/11 1.0 64 7.39

River 8/16/11 3.64 604 7.8
Reservoir 8/8/11 0.78 51 6.68

River 8/15/11 7.8 318 7.82
River 8/11/11 4.4 885 7.93
River 8/10/11 25.5 524 8.04
Lake 8/10/11 5.32 137 6.5
River 8/10/11 3.9 82 7.6
Reservoir 8/11/11 1.6 33 7.1
Lake 8/17/11 3.47 400 8.55 

### 2.1 Laboratory **Selection**
Interested laboratories completed applications to participate, which included h istorical matrix spike recovery information for the source water they planned to use for the study

. All the laboratories routinely

use d the Envirochek ® HV sampling capsule, and several had experience using NaHMP. Laboratory
selection was based on 1) the largest volume of samples completed for the Long Term 2 Enhanced Surface Water Treatment Rule (LT2ESWTR) ; 2) the amount of research performed on the source; and 3) the laboratory precision recorded for the last 10 proficiency testing (PT) rounds. Since the laboratories were asked to select source water with recoveries representative of the matric es tested i n their laboratory,

we anticipated the best representation from laboratories that have processed the most samples. We also expected records of physical and chemical water quality parameters (Table 2) to be most available when the laboratory had a working knowledge about the source. Fourteen laboratories (Table 3) completed the study. Based on the national Cryptosporidium recovery average for PT samples of 54%, 8 of the 14
laboratories were above the national mean during the last 10 PT rounds and 11 of th e 1 4 laboratories were below the national average for RSD of 18.7%.

Table 3 : Participating Laboratories

Referee lab: Shaw Environmental and Infrastructure

, Cincinnati, OH
American Water

, Belleville, IL Analytical Services, Inc.

, Williston, VT BioVir Laboratories, Inc.

, Benicia, CA CH Diagnostic & Consulting Service

, Berthoud, CO City of Phoenix Water Services Lab

, Phoenix, AZ Environmental & Molecular Micro Lab at Michigan State University

, East Lansing, MI

New York City Department of Environmental Protection Pathogen Laboratory

, Kingston, NY Pennsylvania Department of Environmental Protection

, Harrisburg, PA Scientific Methods, Inc.

, Granger, IN
Tetra Tech

- Clancy Environmental

, St. Albans, VT
Truckee Mea dows Water Authority

, Sparks, NV TTL, Inc.

, Tuscaloosa, AL Underwriters Laboratories, Inc. , South Bend, IN

No te: No endorsement of these laboratories is implied, or should be inferred.



### 2. **2 Preparation of Spiking Suspensions**  The spiking suspensions were provided by the Wiscon sin State Laboratory of Hygiene, Madison , WI (WSLH )

. The 1

- mL suspensions of live oocysts and cysts were counted by flow cytometry and confirmed by a minimum of 12 calibration verification standards performed by WSLH. The num ber of organisms in each spiking suspension was blind to the referee and volunteer laboratories throughout the course of the study. The oocysts were the Iowa isolate of Cryptosporidium parvum shed on 7/12/11, and provided by Sterling Parasitology Laboratory (Tucson, AZ). The cysts were the H3 isolate of Giardia
lamblia shed on 7/27/11, and provided by Waterborne, Inc. (New Orleans, LA). The spiking suspensions were prepared on 7/28/11; each suspension contained 1 30.31 ± 1.57 Cryptosporidium oocysts and 150.19

± 1.60 Giardia cysts. Immediately after the Giardia were shed, a ll of the spiking suspensions were sorted
on the same date to reduce fungal contamination, which had been observed in the stock samples of this organism over the warm months. No problem was reported with the preparation of any of the spiking suspensions.

WSLH supplied each laboratory with 10 spiking suspension tubes labeled 1

- 10. The laboratory was instructed to spike reagent water with tubes 1

– 5 and source water with tubes 6

– 10. Tubes 3 and 8 contained spiking suspension fluid, but no organisms

, to serve as the blank samples for reagent and
source water. The laboratories were not informed as to which tube was the blank sample for each w ater type.6 

### 2. **3 Quality Control** 
The referee laboratory was responsible for acquiring supplies from the same lot, assessing their quality, and distributing them to the volunteer laboratories. Each laboratory was provided NaHMP, phosphate buffered saline, 1 0 Envirochek ® HV Sampling Capsule s (Pall Corporation, Ann Arbor, MI), and 10 IMS
bead reactions ( Dynabeads ® GC

- Combo

, IDEXX, Westbrook, ME )

. The spiking suspensions were sent directly to the laboratories from the vendor. All laboratories followed Method 1623.1 quality control

measures. In addition WSLH shipped one set of 10 blind spiking suspensions , from the same lot of organisms to be used for the study, to the referee laboratory a week prior to shipment to the volunteer
laboratories

. The referee lab oratory verified quality control limits and evaluated the suspensions and the reagent lots by spiking 4 reagent water samples and 4 source wat er samples collected from the Ohio
River. One reagent water sample was rendered unusable during a laboratory acci dent. The referee laboratory completed the 10 quality control samples prior to the shipment of supplies to the volunteer
laboratories. The spike values were not released to the referee laboratory analysts until after the completion of all of the in

- house samples two weeks later.

Table 4 : Results of Quality Control Samples

Matrix Percent Recover y by Sample Mean SD RSD 1 2 3 4 Reagent Water

Cryptosporidium 72.1 78.3 unusable 89.0 79.8 8.5 10.7
Giardia 79.9 79.2 unusable 71.2 76.8 4.8 6.3

Source Water

Cryptosporidium 71.4 63.7 45.3 26.1 51.6 20.2 39.2
Giardia 45.9 51.9 35.3 8.7 35.5 19.1 54.0



## 3.0 Data Reporting and Validation

Fourteen laboratories submitted collection data, reagen t documents, bench sheets, and examination results
for review and validation

. Results we re submitted for a total of 70 reagent water and 70 source water samples. The referee laboratory used a double entry sys tem to check each data set for completeness
includ ing : holding times, QC data, and result calculations

. The data from three source water samples were considered invalid and unacceptable for inclusion due to incorrect performance of IMS or cross

contamina tion between two samples. The summary data include all valid results (Tables 5, 6, 7 and 8) ; no outliers were detected by Grubbs test (alpha = 0.05).



## 4.0 Results

**4. 1**  **Reagent Water** The laboratories used a variety of “reagent waters” from filtered tap water to ultra purified milli

- Q system water. Reagent water is defined in both Method 1623 and Method 1623.1 as “ Water demonstrated to be free from the analytes of interest and potentia lly interfering substances at the method detection limit for the analyte. ” The mean recovery of Cryptosporidium oocysts from four spiked reagent water samples in
each of 14 laboratories was 60% with an average within laboratory RSD of 16.2 % (Table 5 )

. Th e mean

re covery of Giardia cysts from four spi ked reagent water samples was 40.8 % with an average within laboratory RSD of 17.4 % (Table 6 )

. No oocysts or cysts were detected in any of the reagent water blank samples, and there were not any “ non

- detects ” in the spiked samples. All laboratories examined 100% of the sample for reagent water samples.7 Table 5 : Spiked Reagent Water Sample Analysis Results for Cryptosporidium Percent Recover y by Sample 1 Mean SD RSD 1 2 3 4
70.6 75.2 78.3 67.5 72.9 4.8 6.5
79.0 56.8 66.8 58.3 65.2 10.2 15.6
29.2 38.4 31.5 36.1 33.8 4.2 12.4
53.0 79.8 66.0 65.2 66.0 11.0 16.6
27.6 44.5 25.3 67.5 41.2 19.5 47.3
69.1 64.5 74.4 69.8 69.4 4.1 5.9
66.0 41.4 53.7 54.5 53.9 10.0 18.6
62.2 47.6 57.6 53.0 55.1 6.2 11.3
69.1 59.1 39.9 77.5 61.4 16.2 26.4
68.3 77.5 67.5 63.7 69.3 5.9 8.5
81.3 59.9 55.3 68.3 66.2 11.5 17.3
61.4 82.9 73.7 60.6 69.6 10.7 15.3
65.2 80.6 68.3 70.6 71.2 6.6 9.3
47.6 44.5 49.9 34.5 44.1 6.8 15.3 All reagent water samples 60.0 15.0 16.2

1 Cryptosporidium s pike levels for the reagent water samples were 130.31 ± 1.57 oocysts.

Table 6 : Spiked Reagent Water Sample Analysis Results for Giardia Percent Recover y by Sample 1 Mean SD RSD 1 2 3 4
48.6 43.9 65.3 47.9 51.4 9.4 18.3
38.0 34.0 34.0 31.3 34.3 2.7 8.0
36.0 47.3 47.3 40.6 42.8 5.5 12.9
39.9 35.3 35.3 28.0 34.6 5.0 14.3
30.0 32.0 30.6 46.6 34.8 7.9 22.8
45.9 51.3 64.6 49.3 52.8 8.2 15.5
43.9 33.3 32.6 27.3 34.3 7.0 20.3
26.6 36.0 54.6 23.3 35.1 14.0 40.0
59.3 46.6 41.9 53.9 50.4 7.7 15.2
55.9 60.6 45.3 49.3 52.8 6.8 12.9
45.9 31.3 34.6 43.9 39.0 7.1 18.2
30.0 36.6 27.3 34.6 32.1 4.3 13.3
32.0 45.9 38.0 34.0 37.5 6.2 16.5
47.9 36.0 39.9 34.0 39.5 6.2 15.7 All reagent water samples 40.8 10.0 17.4

1 Giardia s pike levels for the reagent water samples were 150.19 ± 1.60 cysts.8 

### 4. **2 Source**  **Water**
The mean Cryptosporidium recovery f rom spiked source water was 61.2% with an average within laboratory RSD of 12.7% ( Table 7 )

. The mean Giardia recovery for source water was 37.8% with an average within laboratory RSD of 23.1% ( Table 8 )

. No oocy sts were detected in any of the source water samples that had not been spiked , and no non

- detects occurred in the spiked samples. Giardi a cysts were detected in four of the source water samples that had not been spiked

. In these cases, the total number of cysts counted in the spiked source water samples were reduced by the number of organisms detected in the unspiked source water sample, per section 9. 6.2 .2 of Method 1623 .1 , and recoveries were calculated
based on this corrected count. All laboratories examined 100% of the sample for source water samples. Only five samples, from one source , required three IMS subsamples to complete the e xamination.

Table 7 : Spiked Source Water Sample Analysis Results for Cryptosporidium Percent Recover y by Sample 1 Mean SD RSD 1 2 3 4
74.4 68.3 77.5 78.3 74.6 4.5 6.1
44.5 61.4 59.9 59.1 56.2 7.9 14.0
60.6 55.3 65.2 61.4 60.6 4.1 6.8

76.7 70.6 67.5 69.1 71.0 4.0 5.7
58.3 58.3 72.1 80.6 67.3 11.0 16.3
88.3 79.8 79.8 75.2 80.8 5.4 6.7
30.7 59.9 65.2 54.5 52.6 15.2 29.0
Invalid a 40.7 60.6 62.9 54.7 12.2 22.4
Invalid a 73.7 Invalid a 59.1 66.4 10.3 15.5

72.1 71.4 70.6 80.6 73.7 4.6 6.3
38.4 46.0 38.4 32.2 38.8 5.7 14.6
62.9 58.3 54.5 48.3 56.0 6.2 11.0
21.5 24.6 31.5 27.6 26.3 4.3 16.2
81.3 82.9 82.1 70.6 79.2 5.8 7.3 All valid source water samples 61.2 16.8 12.7

1 Cryptosporidium s pike levels for the reagent water samples were 130.31 ± 1.57 oocysts.

a I nvalid due to incorrect procedure9 Table 8 : Spiked Source Water Sample Analysis Results for Giardia Percent Recover y by Sample 1 Mean SD RSD 1 2 3 4
46.6 44.6 52.6 54.6 49.6 4.8 9.6
14.0 9.3 12.7 30.6 16.6 9.5 57.2
65.9 55.9 64.6 49.9 59.1 7.5 12.8
28.6 16.6 38.0 21.3 26.1 9.3 35.6
65.9 44.6 54.6 64.6 57.4 9.9 17.3
71.9 b 62.6 b 61.3 b 61.9 b 64.4 5.0 7.8
22.0 c 25.3 c 28.6 c 19.3 c 23.8 4.0 17.0
Invalid a 20.6 37.3 41.3 33.1 10.9 33.1
Invalid a 48.6 c Invalid a 45.3 c 46.9 2.4 5.0

42.6 35.3 40.6 59.9 44.6 10.7 23.9
36.6 25.3 40.6 30.0 33.1 6.8 20.6
53.9 d 39.9 d 30.0 d 24.6 d 37.1 12.9 34.7

8.0 10.0 21.3 14.0 13.3 5.9 44.2
27.3 26.0 28.6 29.3 27.8 1.5 5.3
All valid source water samples 37.8 17.4 23.1

1 Giardia s pike levels for the reagent water samples were 150.19 ± 1.60 cysts.

a Invalid due to incorrect procedure

b,c,d N ormalized to account for 5 , 1, and 23 Giardia cysts detected in the non

- spiked source water

, respectively

.


## 5.0 **Comparison of Validation Study Results** A summary of the precision and recovery for Method 1623 .1 w as compare d to the previous validation study for Method 1623 (Table 9 )

. T he Cryptosporidium reagent water and source water mean percent

r ecoveries are ≥ 20 percentage points higher than the previous validation study (Figure 4)

. Giardia mean percent recoveries are similar to the previous validation study. The mean RSD has decreased for Cryptosporidium in both matrices with Method 1623.1

.
There is a statistically significant difference between the Cryptosporidium and Giardia recoveries in both reagent and source waters ( p < 0.0001). This difference was not observed in the previous Method validation conducted in 1999 [ 4 ]. It is possi ble that either the modifications in Method 1623 .1 are deleterious to Giardia recovery or that the organisms were of poor quality. Th e former explanation is beyond the scope of this study; however, the latter is not unlikely because the Giardia cysts are n ot consistent products. There is a large amount of variation in this product due to shipping in warm seasons and a regular fungal contamination that is often found in the organism stocks, decreasing their shelf life. There are very few vendors of this pr oduct. Many of the laboratories commented that the microscopic
characteristics of the recovered Giardia were of poor quality

.10 Table 9 : Comparison of Method 1623.1 Study Results to Previous Validation of Method 1623

Cryptosporidium Giardia

Matrix Statistic Method 1623 Method

1623.1 Method 1623 Method

1623.1

Reagent

Water Mean % Recovery 40 60 38 41

Standard

deviation 9 9 9 7

Mean RSD 24 16 29 17

n 29 56 31 56

Source

Water Mean % Recovery 38 61 42 38

Standard

deviation 9 7 6 7

Mean R SD 25 13 23 23

n 14 53 16 53

Figure 4 . Graphical C omparison of Cryptosporidium R ecovery to Previous Validation Cryptosporidium recoveries comparing the validation study results for Method 1623 and Method 1623.1 in reagent and source water

. Error bars are ± the mean of the within laboratory standard
deviation.



## 6.0 Development of QC Acceptance Criteria These data from the 1623.1 validation were used to develop quality control criteria for l aboratory performance. For each criterion, the distribution of recovery was estimated by using random effects ANOVA. The recovery limits were estimated as the 5 th percentile of the predictive distribution for each
criterion (the complete statistical anal ysis and results using additional models are described in Appendix B) . The lower limit for recovery of Cryptosporidium detected in reagent and source water increased by
11 and 19 percentage points, respectively, over the Method 1623 criteria (Table 10)

. The Giardia recovery criteria increase 8 percentage points for reagent water

, but decreased 7 points for sou rce water (Table 10). It is unknown if the lower Giardia recovery in source water is due to the Method 1623.1
modification s or if the organisms wer e of poor quality. The quality of the Giardia spiking material varies seasonally and may have been impacted by the warm conditions during the study.11 Table 10 : Comparison of Study Criteria to Previous Criteria Performance Test Cr yptosporidium Giardia

Method 1623 Method

1623.1 Method 1623 Method

1623.1 Initial precision and recovery Mean % recovery 24

- 100 3 8

- 100 24

- 100 27

- 100 Precision (as maximum RSD) 55 37 49 39 Ongoing precision and recovery 22

– 100 33

- 100 14

- 100 22

- 100 Matrix spike/matrix spike duplicate Mean % recovery 13

- 111 3 2

- 100 15

- 118 8

- 100 Precision (as maximum RPD) 61 46 30 97



## 7 **.0 Conclusion** This study has successfully validated Method 1623 .1 in fourteen laboratories and demonstrated a

mean Cryptosporidium recovery from source water of 61.2% with an average within laboratory RSD of
12.7%. The mean Giardia recovery from source water was 37.8% with an average within laboratory
RSD of 23.1%

. These recoveries are supported b y two side

- by

- side studies that showed improved

accuracy of ≥ 15 percentage points for detecting the concentration of Cryptosporidium oocysts when using Method 1 623.1 compared with Method 1623. T he improvement in recovery was much more pronounced for thos e samples with a low initial recovery. Similar precision and accuracy were observed for detecting Giardia recovery compared with Method 1623 in both studies

. The lower limit acceptable for recovery of Cryptosporidium detected in reagent and source water increased by 11 and 19 percentage points, respectively, over the updated Method 1623 criteria

. Laboratories performing Method 1623.1
should have more accurate detection and follow more stringent quality control criteria than those laboratories following M ethod 1623.12 

## 8 **.0 References**

1. USEPA . 201 2

. Method 1623.1: Cryptosporidium and Giardia in Water by Filtration/IMS/FA.
EPA 816

- R

- 12

- 001

. Office of Water, Washi ngton, DC

.

2. USEPA. 2005. Method 1623: Cryptosporidium and Giardia in Water by Filtration/IMS/FA.
EPA

- 815

- R

- 05

- 002. Offic e of Water, Washington, DC

.

3. “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium Under the Saf e Drinking Water Act (Renewal)” 74 Federal Register 36 (25 February 2009), pp. 8529

- 8534.

4. USEPA. 2001. Results of the Interlaboratory Method Validation Study for Determination of Cryptosporidium and Giardia Using USEPA Method 1623

. EPA

- 821

- R

- 01

- 028. Office of Water, Office of Science and Technology, Engineering and Analysis Division, Washington, DC

.

5. Hill, V. R., A. L. Polaczyk, D. Hahn, J. Narayanan, T. L. Cromeans, J. M. Roberts, and J. E. Amburgey. 2005. Development of a rapid method for simultaneous recovery of diverse microbe s in drinking water by ultrafiltration with sodium polyphosphate and surfactants. Appl Environ
Microbiol 71: 6878

- 84.

6. Clancy, J .L. and R.M. McCuin. 2005. Cryptosporidium in wastewater: Occurrence, removal and
inactivation

. Water Environment Research Foundation.

7. McCuin, R.M., et al. 2001. Improving methods for isolation of Cryptosporidium oocysts and
Giardia cysts from source and finished waters in : Proceedings of American Water Works
Association Water Quality Technology Conference

. Denver, CO

.

8. Lepesteur, M., S. Blasdall, and N.J. Ashbolt,. 2003. Particle dispersion for further Cryptosporidium and Giardia detection by flow cytometry

. Lett Appl M icrobiol 37 (3): p. 218

- 29.13 

## Appendix A. Instructions to **Participating Laboratories:**

1. Please review the 2011 Method 1623 revision draft [subsequently Method 1623.1] by Aug. 8th. To enhance program

- wide data quality and consistency, and guard against the use of sample processing shortcuts that could compromise data quality, the updated method provides laboratories with the flexibility to select from options for various procedural c omponents in the method. However, each option must be performed according to the procedures used during a multi

- laboratory validation studies. No procedural modifications will be permitted without demonstrating acceptability through a multi

laboratory va lidation study using multiple matrices, as per 40 CFR Part 141.27. Therefore, for this study, only those modifications specified in the text or in Table 1 of the Method draft will be allowed. Review these modifications along with your laboratory SOPs and select the process that will best reflect your future practices (These samples can serve as an IDC set for a Method 1623 revision). Refer any questions to Leah Villegas at 513

- 569

- 7886 or villegas.leah@epa.gov

. Major changes to the draft are listed on page v. Especially note the following sections:

**•** 11.3.3 Sample Spiking [subsequently 11.2.3] : Follow the procedure as written, but adjust for a spike volume of 1 mL.

**•** 12.2.6.2 NaHMP pre

- elution [subsequently 12.2.7 Dispersant addition] : Follow the procedure as written. The pre

- elution step is the beginning of elution and a one working day holding time is required for elution to slide application.

**•** 13.3.2.12 Microcentrifuge tube portion of IMS: The volumes listed for the quantitative
transfer from the flat

- sided tube to the microcentrifuge tube have changed in the draft or you can use manufacturer’s instructions. The MPC

- S magnet or equivalent is specified.

**•** 13.3.2.17 Required rinse of the mic rocentrifuge tube: Follow the procedure as written.

**•** 13.4 Options for complicated matrices: Review and use if needed or preferred for your

IDC.

2. Two packages will arrive at your facility on August 9, 2011. Wisconsin State Laboratory of Hygiene will ship 10 live spiking suspensions directly to your laboratory. Shaw Environmental will ship 10 Envirochek® HV filters, a box of G/C combo IMS beads for 10 reactions, sodium hexametaphosphate, and PBS to use for the study along with a UPS mailer to return the da ta sheets.

3. All Method 1623 holding times MUST be met. The spiking suspensions will not expire until 8/19/11. Therefore, laboratories have until 8/19/11 to spike and filter the samples.

4. The entire pellet MUST be processed through the examination step for this study. Please contact Leah Villegas (513

- 569

- 7886 or villegas.leah@epa.gov ) if you anticipate that your selected source water will require multiple IMS subsamples.

5. Prepare shipped reagents: Prepare the reagents shipped on 8/9/11 as follows:

**•** Sodium hexametaphosphate (NaHMP):

o Prepare a sufficient quantity of 5% sodium hexametaphosphate (NaHMP) solution to

pre

- treat all of the filters associated with the OPR/MB used for that batch of elution

buffer. Pr e

- treatment may require up to 150 mL of NaHMP per sample.

o 5% w/v NaHMP solution

— Dissolve 50g of sodium hexametaphosphate ((NaPO 3 )
n - Sigma 71600 or equivalent) in 600 mL of reagent water. Dilute to final volume of 1000 mL using reagent water. Store in a plastic or glass container at room temperature for up

to 3 months. Discard when expiration date is reached or sooner if microbia l growth is
apparent.

**•** Prepare 1X PBS

o Follow preparation instructions provided on the packet.

6. Use the Benchsheets and Reagent Batch forms supplied by Shaw Environmental.

7. 10 spiking suspension are provided. They are numbered with a letter corresponding to your laboratory plus the numbers 1

- 10. Please add spike samples 1

- 5 to the reagent water samples and

6

- 10 to the source water samples (MS). Each set of 5 spiking suspension includes a blank sample. Use the spiking suspension label as the sample ID. The same batch of elution buffer must be used for all 10 samples.14 8. Follow the revised method draft as written; spiking 5 reagent water samples and 5 samples from your selected source water. Perform the pre

- elution treatment on all samples.

9. Perform three oocys t and cyst characterizations for each sample.

10. When samples are completed send the bench sheets (Reagent Batch Sheet, Bench sheets and Microscopy results forms) to Shaw Environmental using the provided UPS mailer and return label.

11. Once the study is comple ted and validated Shaw Environmental will send you a results page to complete your IDC information.15 

## Appendix B. Description of Statistical Analysis Used to Develop QC **Acceptance Criteria for Method 1623.1:** The QC acceptance criteria for Method 1623.1 were developed by characterizing the performance of the laboratories with the following prediction intervals:

**•** The IPR recovery limit is estimated as the 5 th percentile of the predictive distribution of means of

4 future observations.

**•** The IPR precision l imit is estimated as the 95 th percentile of the predictive distribution of RSDs

from 4 future observations.

**•** The OPR recovery limit is estimated as the 5 th percentile of the predictive distribution for recovery in reagent water.

**•** The M S recovery limit is estimated as the 5 th percentile of the predictive distribution for recovery

in source water.

**•** The MS / MSD average recovery limit is estimated as the 5 th percentile of the predictive

distribution for the mean of 2 recoveries in source water.

**•** The MS / MS D preci sion limit is estimated as the 95 th percentile of the predictive distribution for the relative percent difference (RPD) of 2 recoveries in source water.



### Calculation of Recovery Recovery is generally estimated as p = x /n , where x is the count and n the spiking number. It is a direct estimate of the probability that a spiked organism will be identified and counted. The number of organisms counted in a spiked sample is often viewed as a binomial random variable, Bin( n ,p ). Each organism spiked is viewed as an independent Bernoulli trial with common probability of success (being counted) equal to p

.
Probability was modeled in the recovery scale and using the logit transformation of the probability. This

is defined as 

# ( )

1 ln p

p y

**−**

**=** , where p is a pro bability and ln is the natural logarithm. The quantity 1 p

p

**−** is

known as the odds ratio (or just “the odds”). The inverse of the logit transformation is given by 

# ( )

1 y y pe e

**= +**

.

In the case of source water analysis, the average of counts found during analysis of the unspiked sample, the matrix correction, must be subtracted from the counts. In the logit scale, the spiking numbers or the matrix correction are not required to be integers.



### Approaches to Estimating Criteria Previous studies have consistently shown significant random effects for laboratory and for source water. These random effects were shown to be normally distributed in the logit (log recovery odds ratio) scale.

In addition to the lab/source water random ef fect, a substantial sample level random effect is apparent consistently in Method 1623 and Method 1623.1 data. This is seen in the extra- binomial variance, the fact that the variances of recoveries from replicate spiked samples is always much larger than w ould be expected under the binomial model. The variance of a binomial proportion (which is what a recovery is)

is **( )** 1 p p

n **−** . Figure B-1 below illustrates this using the sample standard deviations and means of lab reagent
water spiked repli cates from the method validation study. The lines show the standard deviations

expected under the binomial model: **( )** 1 p p

n **−**

.16 In this study the laboratory and source water random effects are combined into a single random effect
call ed Lab. This results in a one

- way random effects (RE) ANOVA model [1 ] . Since there is no fixed
effect (other than a constant) in the model, this is a pure RE model rather than a mixed effects model. The model for the reagent water data uses Lab as a random effec t also. The sample level random effect (error variance) in the logit normal model for recoveries directly accommodates the extra

- binomial variance seen in Figure B

- 1

.

Figure B

- 1

. Standard deviation versus mean of reagent water spike recovery grouped by laboratory Each point shows the mean and standard deviation from four replicates analyzed by one laboratory.

Figure B

- 2 below shows the same data with the means and standard deviations calculated in the logit
scale. In the l ogit scale the data show more constant standard deviations and less relationship between mean and standard deviation. In fact, the logit transformation does make the variance of binomial proportions approximately constant (based on the delta method approxi mation).

Three approaches have been used in this study to estimate the QC acceptance criteria for recovery and their precision of estimation. The first two are based on normality and the one

- way RE ANOVA decomposition in the logit scale. These methods are based on computing a prediction variance that is based on a linear combination of heterogeneous sums of squares, between lab and within lab. The exact distribution of the prediction variance is unknown, but there are various approximations. This is known as the Behrens

- Fisher problem. 0% 2% 4% 6% 8% 10% 12% 14% 16% 18% 20%

0% 20% 40% 60% 80% 100%Std Dev of Recovery

Mean Recovery 

# Std Dev vs Mean of Recovery

- Reagent Water

Crypto

Giardia

Binomial SD Crypto

Binomial SD Giardia17 The first approach uses the well

- know n Satterthwaite approximation [ 2] to the Behrens

- Fisher distribution (of the prediction variance) in the estimation of percentiles. This method produces prediction limits for recovery of individual spiked samples and for the mean recovery of groups of spiked samples.

The second approach uses a different approximation to the Behrens

- Fisher distribution and allows construction not only of prediction limits but also of confidence limits for those prediction limits (and therefore of tolerance limits). It uses methodology presented by Krishnamoorthy and Mathew [ 3].

Figure B

- 2

. Standard deviation versus mean of logits of reagent water spike recovery grouped by l aboratory

The third approach is a hierarchal Bayesian approach. It uses Ma rkov Chain Monte Carlo (MCMC) [4, 5] simulation imp lemented through the OpenBUGS [ 6] software to estimate the predictive distribution for each of the QC Acceptance criteria statistic s. Percentiles are estimated from this distribution. Transformations are not used. The simulation is directly in the scale of the counts.

Two methods have been used to estimate QC acceptance criteria for precision. The first is based on an F statistic app lied to the empirical pooled RPDs and RSDs from the study. The second is based on constructing RSDs and RPDs from the predictive distribution samples generated by the Bayesian MCMC
simulation. 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

- 1

- 0.5 0 0.5 1 1.5Std Dev of Logits

Mean Logit 

# Std Dev vs Mean of Logits

-- Reagent Water

Crypto

Giardia18 

### ANOVA Calculations The spiking numbers used in this study were larger than the spiking number of 100 that is typically used for routine QC. This was done in order to improve the precision of the study data. However, in constructing a predictive distribution for future recoveries one must account for the fact that the future spiking number will be 100. T his is done by multiplying the error mean square (error variance) or the error sum of squares by the ratio of the spiking number in the study divided by 100.

The between Lab mean square is not affected by the change in s piking number.



### Balanced Case: Reagent Water Criteria (IPR and OPR)

Let 

# { } , 1, , , 1, , ij 

### p i Lj n **= =**

  be estimated recoveries for spiked samples, with i indexing the

laboratories and j indexing the replicates. In the balanced case, there are n replicates per lab. In the case

of the reagent waters, L = 14, and n = 4. Let 

# ( ) 

# ( )

1 

### logit ln ij

ij p ij ij p 

### y p

**−** 

### = = be the corresponding logits.

The model is

ij i ij 

### y **μλ ε =++** 

, (1)

where 

# ( ) 2 

### ~ 0, i L 

### N **λ σ** are Lab random effects, and 

# ( ) 2 

### ~ 0, ij 

### N **ε σ** , all mutually independent. The

variance of an observation is 

# ( ) 2 2 

### var ij L 

### y **σ σ = +**  , the sum of the two “variance compo nents.”

The Lab mean square is



## ( ) 2

Lab

. .. 1 

### MS 1 L

i

i 

### n y y L

**=** 

### =
**−**

**−** 

# ∑

, (2)

where

. 1 1 n

i ij j y y
n

**= =**  

# ∑ are the Lab means and 14
1

.. . 14
1 i

i y y

**= =**  

## ∑ is the grand mean. It has L - 1 = 13 degrees of

freedom (df). The error mean square is



## ( ) 

# ( ) 2

. 1 1 1 MS 100 1 L n

E ij i i j SN y y L n

**= = =** 
**−**

**−** 

# ∑∑

, (3)

where SN is the study spiking number. It has L ( n – 1) = 42 df. The significance of the model is tested using the ratio MS Lab over MS E compared to the F distribution with 13 and 42 df.

The error variance 2 

### σ is estimated by the MS E , and the Lab variance 2

L 

### σ is estimated as

2 Lab MS MS E
L s
n **−**

**=**

. (4)

For a full discussion of one- way RE ANOVA models, see Montgomery [1 ].



### Unbalanced Case: Source Water (MS/MSD)

In the unbalanced case, there are a variable number of replicates per lab. Because of the invalidation of

some data in the source water data set, it is unbalanced. Let 

# { } , 1, , , 1, , ij i 

### p i Lj n **= =**

  be estimated19 recoveries for spiked samples, with i indexing the laboratories and j indexing the replicates. The values of

the n i vary from 2 to 4. Let 

# ( ) 

# ( )

1 

### logit ln ij

ij p ij ij p 

### y p

**−** 

### = = be the corresponding logits.

The Lab mean square for the unbalanced case is



## ( ) 2

Lab

. .. 1 

### 1 MS 1 L

i i

i 

### ny y L

**=** 

### =
**−**

**−** 

# ∑

, (5)

where

. 1 

### 1 i n

i ij j i 

### y y
n

**=** 

### = 

# ∑ are the Lab means and 14
1

.. . 14
1 i

i 

### y y

**=** 

### = 

# ∑ is the grand mean. It has L - 1 = 13 df. The

error mean square for the unbalanced case is


# ( ) **( )** 2

. 1 1 

### 1 MS 1 100 i n L

E ij i i i j 

### SN yy n L

**= =** 

### = −− 

# ∑∑

. (6)

The one- way model for the source water has 

## ( )

1 

### 1 39 L

i i 

### n

**=** 

### − = 

# ∑ error df. The significance of the model is

tested using the ratio MSE Lab over MSE E compared to the F distribution with 13 and 39 df, respectively.

The error variance 2 

### σ is estimated by the MS E , and the Lab variance 2

L 

### σ is estimated as

2

2 Lab 1 0

1 0

1 

### MS MS 1 , where 1 L

i L
E i
L i L
i
i

i 

### n

s nn n L
n **=**

**=**

**=** 

###   ** **

**−**

**= = −**  ** **

**−  **  **   **  

# ∑ **∑ ∑**  . (7)



### Predictive Distribution Based on ANOVA Calculations The first approach to estimating percentiles of the predictive distribution of recovery uses the well -known Satterthwaite approximation [ 2] to the Behrens

- Fisher distribution. The second uses an a pproximation presented by Krishnamoorthy and Mathew [3] based on the non- central t -distribution.



### Using Satterthwaite’s Approximation The prediction variance for a single new observation is given by


## ( ) **( )** 2 22 1 1 

### 1 1 L nL pred L 

### σ σσ **=+ ++** (8)

and is estimated by


## ( ) **( )** 2 22 1 1 

### 1 1 L nL pred L 

### s ss **=+ ++**

. (9)

The prediction variance for the mean of k future observations is estimated by


## ( ) **( )** 2 22 1 11

, 1 L k nL pred k L s ss **=+ + +** (10)20 Since these are “synthesized” variances, their distributions do not follow a chi- squared distribution, as is
class ically required for inference using the t

- distribution.

One approach to approximating their distribution is to use Satterthwaite’s approximation [ 7]. This approximates their distributions as being chi-squared with degrees of freedom given by 

# ( )
**( ) ( )**  

## ( ) **( )** 

# ( )
**( ) ( )**  

## ( ) **( )** 2 2

2 2 2 2 1 1

2 2

, 2 2 2 2 1 1 1 , and 1 11 1

, 11 1 pred

L L nL

pred k

k

L L k nL s df
s L s Ln

s df
s L s Ln **=   **  **+ −+ + −   ** 

**=   **  **+ −+ + −   **  (11)

respectively.

The predictive distributions for a single new observation and for the mean of k future observations in the logit recovery scale are estimated by a t

- distribution as 

# ( )

**( )**  2

2

, 

### ˆ ˆ ~ , , , and

ˆ ˆ ~, ,, pred

k pred k k 

### y t df

y t df **μ σ**

**μ σ** (12)

respectively.

Then the 95% lower prediction limits (LPL) in the logit and recovery scales for a single new observation
and for the mean of k future observations are given by

,

, 2 0.05,

2

, , 0.05,

, 

### ˆ ˆ ; , and 1

ˆ ˆ ; , 1 LPL

LPL

LPL k

k LPL k y

LPL p red df LPL y

y

LPL k pred k df LPL k y 

### e y tp

e

e y tp

e **μ σ**

**μ σ =**  **+ = +** 

**= + =**  **+** (13)

For the unbalanced case, in all the formulae of this section, n is replaced by n 0
. 

### Using Method of Krishnamoorthy and Mathew Krishnamoorthy and Mathew [3] use a non- central t

- distribution to approximate the distribution of a t

statistic computed using a synthesized or heterogeneous standard deviation in the denomin ator. Their method allows not only the estimation of percentiles but also of confidence limits about those percentiles. This permits the construction of one- sided tolerance limits, specifically of guaranteed content tolerance intervals. In our case, a 95% lower confidence limit on the 5 th percentile is equivalent to a 95% confidence 95% coverage Lower Tolerance Limit.

Their formulae for the balanced case are21 

## ( ) **( )**

**( ) ( ) ( )**  

## ( )

**( )** 

# ( ) .. 1 1

1 2

1 1, 1

2 2

Lab

. .. . 1 1 1 , , where 1

1 1,

, . 100 Lab k

E

L L n

Lab

L L n

i E ij i

i i j SS Tyt LL n

SS zL L F SS

SN SS n y y SS y y **β δ γ** 

**δ γ −** 

**− −**

**= = =**  **= +**

**−  ** 

**= +−**

**−  **  ** **

**=− = −** 

## ∑ **∑∑** (14)

In these formulae, our desired percentile probability is β = 0.05. The symbol z **β** denotes the β -quantile

value of the standard normal distribution. Our confidence coefficient γ for a lower confidence limit is not

95% but 5%.

In the unbalanced case, the estimator is



## ( ) **( )**
**( ) ( )**

**( ) ( )** 

## ( )



## ( )

1 2

1 2

2 1 ,

2

. 1

. 1

1 , , where 1

1 1 1 ,

,

1

,

1 1

. x u k

E
L NL

x

L

x i

i

L

i

i

L

i i SS T yt L L

L L SS z L F N L SS

SS n y y

y y L

L n **β δ γ** 

**λ δ γ** 

**λ −** 

**− −**

**=**

**=**

**= = +** 

**−**

**− −  ** 

**= + −**  ** **

**−  ** 

**= −**

**=**

**=** 

## ∑

**∑**

**∑** (15)

The calculations presented above are for limits for single spike recoveries; that is, OPR or MS (not average of MS/MSD) limits. For IPR and MS/MSD limits one adjusts the calculation of SS E

in equation 14 by dividing it by the number of counts averaged : 4 for the OPR limit and 2 for the MS/MSD limit.



### Bayesian MCMC Predictive Distribution The Bayesian MCMC method constructs by a very special type of simulation an approximation to the joint distribution of the model parameters that is most consistent wit h the data and with the assumptions behind the model. These assumptions include both the structure of the model and the assumed prior
distribution of the parameters, which are viewed as random rather than fixed. The joint probability
distribution for the m odel parameters so generated is known as the posterior distribution . It is this posterior distribution that is used to generate a predictive distribution for future observations.

The model used for the Bayesian MCMC is as follows. The basic counts follow a binomial distribution.

The counts ij X are indexed in the same manner as the logits of the recoveries


# ( ) ~ Bin , ij ij ij X Np

The spiking numbers ij N are random for each sample. The average and standard deviation of the spiking
number are determined by the vendor. In this study, they were 130.31 and 150.19 with standard
deviations of 1.57 and 1.6 for Cryptosporidium and Giardia , respectively. They were modeled by22 generating normal variates and rounding to integers. To compensate for the variance reduction due to rounding, the variances were inflated by a factor of 1.09, determined by simulation. 

# ( ) 

# ( **)** 2 ~ round ,1.09 ij N N **μ σ**

The logits of the ij p are modeled by the one- way RE ANOVA model.


# ( )

0, 1, logit ij ij i p bb **α =+ +** 
,

where α is a fixed constant for each simulation of the experiment, b 0 ,ij is a sample level random effect and
b 1,i is a Lab level random effect. These parameters have the following distributions. 

# ( )

**( ) ( )**  1

1 0, 0

1 1, 1

~ 0, , one value for each simulation

~ 0, , one value for each sample in each s imulation

~ 0, , one value for each Lab in each simu lation ij

i N

b N

b N **α α τ** 

**τ**

**τ −** 

**−**

**−**

The hyperparameters 

## ( )

0 1 , , **α τττ**  are distributed according to the diffuse Gamma distribution with

parameters α = 0.01 and β = 0.01.

When this MCMC simulation has converged to produce an acceptable distribution for the hyperparameters, a large set of simulated values from later in the run (throwing away the “burn -in” values) is saved. This set of values is a large sample from the posteri or distribution of the hyperparameters. This sample from the posterior distribution is then used to simulate the predictive distribution for the QC criteria statistics.

In order to generate the predictive distribution of the routine recovery verification (OPR) quality sample,
simulate one X value (count) with a target spiking level of 100 (but random N, as above) for each 

## ( )

0 1 , , **α τττ**  value. Take the 5 th percentile from this set of X values as an estimate of the OPR limit.

To generate the predictive distribution of the initial precision and recovery (IPR) mean and RSD, simulate

four X values with a target spiking level of 100 (but random N, as above) for each 

## ( )

0 1 , , **α τττ**  value. They

will share a common value o f α and of b 1 . Four independent values of b 0 will be generated, one for each X
value. For each set of four simulated X values, a mean recovery and RSD will be calculated. The 5 th and
95 th percentiles of the sets of means and RSDs will be the estimates of t he IPR precision and recovery
limits.

To generate the predictive distribution of the MS/MSD mean and RPD, simulate two X values with a target spiking level of 100 (but random N, as above) for each 

## ( )

0 1 , , **α τττ**  value. They will share a

commo n value of α and of b 1 . Two independent values of b 0 will be generated, one for each X value. For
each set of two simulated X values, a mean recovery and RPD will be calculated. The 5 th and 95 th percentiles of the sets of means and RPDs will be the estimates of the MS/MSD precision and recovery
limits.

The posterior distributions for the parameters was generated using OpenBUGS version 3.2.1 [ 6]. Four runs were made for the two organisms and two matrices. In each run nine independent Markov chains were generated with different starting values for the parameters. Mixing was very good for all parameters except for α and τ α . After thinning by a factor of 10 to reduce the autocorrelation, these were suitable as23 well. After burn -in and thinning, between 5,000 and 20,000 values were retained per chain. The predictive distribution simulations were performed using the R s tatistical computation system [ 7] (version 2.13.2).



### Limits for RSD and RPD
The RSD is the relative standard deviation, standard deviation divided by the mean. The RPD (applicable only for pairs of measurements) is the relative percent difference, the absolute difference divided by the
mean. It is equal to the RSD multiplied by 

### 2

.
As an alternative to the Bayesian MCMC prediction limits for RSD and RPD described in the previous subsection, the following ‘classical’ methodology was used to estimate quality limits for RSD and RPD.

Since RSDs are only applicable to data with positi ve means, it is computed only in the recovery scale, not in the logit scale. The population RSD is estimated as a pooled RSD from the labs in the study. This is computed as the root mean square of the individual lab RSDs.



## ( ) **( )** 2

1

pool

2

1

1 

### 1 RSD , in the balanced case

RSD 1 1 RSD , in the unbalanced case

where . L

i

i

L

i i

i

L

i i 

### L

n N LL

N n **=**

**=**

**=** 

###  ** ** 

**= **  ****

**− ** 

**− ** 

**=** 

# ∑

**∑**

**∑** (1 6)

For independent RSDs with a common mean, the square of their ratio has approximately a n F distribution. It must be remembered that future MS/MSD will be spiked at 100 organisms rather than 130 or 150, as in this study. Therefore, one can determine an approx imate limit for RSDs of k future recoveries as a 95% prediction limit based on the equation:

2

max, 0.95, 1, max, 0.95, 1, pool 100 pool 

### RSD RSD RSD RSD k S k df k k df 

### F F

**− −** 

###  

**=⇒= ⋅  **  ** ** , (1 7)

where df is the degrees of freedom for the pooled RSD estimator ( N – L ), and S is the spiking number used in this study. Because the pooled RSD does not include between Lab variance, the ordinary degrees of freedom is used rather than Satterthwaite’s approximation.

The limit for RPD is 

### 2 times the limit for the RSD of two future observations.24 

### Summary **and Discussion of Statistical Analysis** The quality control acceptance criteria developed using these statistical models and procedures describe d
above are listed in Table B

- 1. C olumn 1 list s the current Meth od 1623 criteria developed using data from
three different source s. The IPR and Giardia OPR criteria were based on results from six laboratories analyzing 293 Cryptosporidium and 186 Giardia OPR samples collected during the Information Collection Rule Sup plemental Survey, while the matrix spike criteria is based on results from 8 laboratories analyzing a total of 14 Cryptosporidium and 16 Giardia samples from 8 source waters collected during the Met hod 1623 validation study. The Cryptosporidium OPR minimum recovery
criterion was updated in 2009 [8] ; this update was based on 333 reagent water PT samples from 58
laboratories

. C olumn 2 lists criteria developed from results from 51 laboratories during 5 reagent water
PT rounds and is based on 753 C ryptosporidium and 530 Giardia samples. The criteria developed from
th ese PT round data used a random effects model in the recovery scale and more accurately reflect s current laboratory capability and provide s a better assessment of laboratory performance in reagent water using Method 1623.

C olumns 3 through 7 were calculated using the statistical models and procedures describe d in this Appendix. The results in each column are calculated from the data collected during this Method 1623.1 validation study which consisted of 56 reagent water and 5 3 source water samples collected from 14 different public water system sources and analyzed in 14 laboratories. C olumn 3 highlights the prediction limits based on the normal random effects model in the recovery scale. These quality control criteria were confirmed by comparing the recovery scale to the results using the logit transformation ( C olumn 4 ). The results for the recovery and logit scale a re very similar, within one value for most of the criteria. The only exception is the Giardia MS/MSD mean recovery which is 8 in recovery scale and 12 in lo git. This difference is due to the presence of a higher frequency of low recoveries (<30%) for th e Giardia MS/MSD, these lower recoveries have less impact in the logit scale. The use of the 5 th percentile as the prediction limits was also evaluated by developing tolerance limits in both the recovery and logit scale (C olumns 5 and 6 ). The data were a lso evaluated using a Bayesian Markov Chain Monte Carlo simulation ( C olumn 7 ). The calculated qu ality assurance criteria are similar for various statistical procedures and models, therefore the 5 th percentile prediction limit based on the normal random effects model in recovery scale (the yellow C olumn 3 ) was selected for the final version of Method 1623.1. This
is the same procedure and model used to develop the quality assurance criteria for Meth od 1623. The criteria developed using the PT data ( C olumn 2 ) demonstrates that laboratories are currently able to consistently achieve Cryptosporidium OPR recoveries greater than 33% . The modifications required in
Method 1623.1 have not demonstrated an i mpact on the quality control acceptance criteria for reagent water recoveries as compared to Method 1623, but t he higher value for acceptance criteria in Method
1623.1 provide a better standard for source water recovery

.25 Table B

- 1 : Comparison of Study Criteria to Previous Criteria Using Different Statistical Approaches 

## Cryptosporidium

Method 1623 Method 1623.1

Column Number 1 2 3 4 5 6 7

Performance Test Criteria in
Method

1623 Prediction
limits based
on normal
RE model in

recovery

scale* Prediction
limits
based on
normal RE
model in

recovery

scale Prediction
l imits based
on normal
RE model

in logit Tolerance and prediction limits for
normal RE model in recovery scale Tolerance and prediction limits for
normal RE model in logit scale Bayesian
MCMC
estimates

5% Pred 95% 5% Pred 95%

Initial Precision and Recovery (IPR)

Mean Recovery (percent) 24

- 100 36 38

- 100 37

- 100 27 39 46 27 38 45 37

Precision as maximum
RSD 55 34 0

- 37 0

- 37 35

Ongoing Precision and Recovery (OPR)

Single Recovery (percent) 22

- 100 33 33

- 100 32

- 100 26 37 44 27 37 43 32

MS/MSD

Mean Recovery (percent) 13

- 111 32

- 100 31

- 100 20 34 43 21 33 42 32

Precision as maximum
RPD 61 0

- 46 0

- 46 46


## Giardia

Method 1623 Method 1623.1

Performance Test Criteria in
Method

1623 Prediction
limits based
on normal
RE model in

recovery

scale* Prediction
limits based
on normal
RE model in recovery
scale Prediction
limits
based on
normal RE
model in logit scale Tolerance and prediction limits for
normal RE model in

recovery scale Tolerance and prediction limits for
normal RE model in logit scale Bayesian
MCMC
estimates

5% Pred 95% 5% Pred 95%

Initial Precision and Recovery (IPR)

Mean Recovery (percent) 24

- 100 32 27

- 100 28

- 100 20 27 31 22 28 31 28

Precision as maximum
RSD 49 37 0

- 39 0

- 39 34

Ongoing Precision and Recovery (OPR)

Single Recovery (percent) 14

- 100 29 22

- 100 24

- 100 19 25 29 21 26 29 23

MS/MSD

Mean Recovery (percent) 15

- 118 8

- 100 12

- 100 0 10 19 7 13 19 11

Precision as maximum

RPD 30 0

- 97 0

- 97 85

*Values calculated using results of 5 reagent water PT rounds26 

### References

1. Montgomery, D. 1991. Design and Analysis of Experiments , 3 rd Edition. Wiley, New York.
Sections 3

– 5.

2. Sahai, H. and M. Ojeda. 2003. Analysis of Variance for Random Models: Unbalanced Data: Theory, Methods, Application and Data Analysis , Vol. II. Birkhauser, Boston. pp. 383

- 386.

3. Krishn amoorthy, K. and T. Mathew. 2004. One

- Sided Tolerance Limits in Balanced and
Unb alanced One

- Way Random Models Based on Ge neralized Confidence Intervals.
Technometrics 46(1), pp. 44

- 52.

4. D. J. Spiegelha lter, A . Thomas and N. G. Best. 1996. “Computation on Bayesian graphical
models” Bayesian Statistics 5 pp. 407

- 425.

5. Hastings, W.K. (197 0) “ Monte Carlo sampling methods using Markov chains and their applications ” Biometrika 57 (1), pp. 97

- 109

.

6. Spiegelhalter, D

. , A

. Thomas, N

. Best and D

. Lunn . 2011. OpenBUGS User Manual . URL http://www.openbugs.info/Manuals/Manual.html

.

7. R Development Core Team. 2011

. R: A language and environment for statistical computing . R Foundation for Statistical Computing, Vienna, Austria. ISBN 3

- 900051

- 07

- 0, URL http://www.R

- project.org/

.

8. “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium Under the Saf e Drinking Water Act (Renewal)” 74 Federal Register 36 (25 February 2009), pp. 8529

- 8534.