Results of the Inter-laboratory Method 

Validation Study using U.S. 

Environmental Protection Agency 

Method 1623.1: Cryptosporidium and 

Giardia in Water by Filtration/IMS/FA
Office of Water (MS 140) 

EPA 816-R-12-002 

February 2012 

http://water.epa.gov/drink
iii

Acknowledgments

This report was prepared under the direction of Carrie Miller of the Technical Support Center, Standards 

and Risk Management Division within the U.S. Environmental Protection Agency (EPA) Office of 

Water. It was prepared by Leah Fohl Villegas, PhD and John H. Carson Jr, PhD, Shaw Environmental 

and Infrastructure, Inc. under EPA Contract: EP-C-06-031, Work Assignment: 4-02.

Disclaimer

This method validation study report has been reviewed by U.S. EPA Office of Water and approved for 

publication. Mention of trade names, commercial products, or participating laboratories does not 

constitute endorsement or recommendation for use. 

Questions regarding this report, method or their application should be addressed to: 

Carrie Miller 

Coordinator, Laboratory Quality Assurance Program for the Analysis of Cryptosporidium 

U.S. Environmental Protection Agency

Office of Ground Water and Drinking Water

Technical Support Center, MC140

26 West Martin Luther King Drive

Cincinnati, OH 45268-1320

(513) 569-7919

(513) 569-7191 (fax)

miller.carrie@epa.gov

Safe Drinking Water Hotline 1-800-426-4791

http://water.epa.gov/drink/hotline/index.cfm

You may obtain copies from:

http://water.epa.gov/scitech/drinkingwater/labcert/analyticalmethods_ogwdw.cfm
iv

Table of Contents

Executive Summary........................................................................................................................1

1.0 Introduction ..................................................................................................................................3

2.0 Study Design and Implementation ...............................................................................................4

2.1 Laboratory Selection ................................................................................................................5

2.2 Preparation of Spiking Suspensions.........................................................................................5

2.3 Quality Control.........................................................................................................................6

3.0 Data Reporting and Validation.....................................................................................................6

4.0 Results..........................................................................................................................................6

4.1 Reagent Water..........................................................................................................................6

4.2 Source Water............................................................................................................................8

5.0 Comparison of Validation Study Results.....................................................................................9

6.0 Development of QC Acceptance Criteria...................................................................................10

7.0 Conclusion.................................................................................................................................11

8.0 References.................................................................................................................................12

Appendix A. Instructions to Participating Laboratories:.................................................................13

Appendix B. Description of Statistical Analysis Used to Develop QC Acceptance Criteria for 

Method 1623.1: ................................................................................................................................15
v

Table of Tables

Table 1: Method 1623.1 Quality Control Acceptance Criteria ......................................................................................2

Table 2: Source Water Specifications.............................................................................................................................4

Table 3: Participating Laboratories ...............................................................................................................................5

Table 4: Results of Quality Control Samples..................................................................................................................6

Table 5: Spiked Reagent Water Sample Analysis Results for Cryptosporidium .............................................................7

Table 6: Spiked Reagent Water Sample Analysis Results for Giardia ............................................................................7

Table 7: Spiked Source Water Sample Analysis Results for Cryptosporidium................................................................8

Table 8: Spiked Source Water Sample Analysis Results for Giardia...............................................................................9

Table 9: Comparison of Method 1623.1 Study Results to Previous Validation of Method 1623 .................................10

Table 10: Comparison of Study Criteria to Previous Criteria........................................................................................11

Table B-1: Comparison of Study Criteria to Previous Criteria Using Different Statistical Approaches.........................25

Table of Figures

Figure 1. Results of single laboratory side-by-side analysis...........................................................................................3

Figure 2. Results of side-by-side analysis in 4 laboratories...........................................................................................3

Figure 3. Distribution of Source Waters........................................................................................................................4

Figure 4. Graphical Comparison of Cryptosporidium Recovery to Previous Validation ..............................................10

Figure B-1. Standard deviation versus mean of reagent water spike recovery grouped by laboratory......................16

Figure B-2. Standard deviation versus mean of logits of reagent water spike recovery grouped by laboratory........17
1

Executive Summary

This report presents the results of the U.S. Environmental Protection Agency’s (EPA) interlaboratory 

validation study of Method 1623.1: Cryptosporidium and Giardia by Filtration/IMS/FA [1]. The 

substantial changes from Method 1623 [2] are the addition of sodium hexametaphosphate and a rinse to 

remove excess debris during immunomagnetic separation of the oocysts and cysts. The interlaboratory 

validation study of Method 1623.1 was performed after two side-by-side studies that showed improved 

accuracy for detecting the concentration of Cryptosporidium oocysts in water when using Method 1623.1 

compared with Method 1623. These side-by-side studies were 1) a single laboratory comparison of 

samples processed from nine public water system sources which showed an average improvement of 18 

percentage points with Method 1623.1 (p = 0.0001); and 2) a multi-laboratory comparison of samples 

processed from three source waters which resulted in an average improvement of 15 percentage points

with Method 1623.1 (p=0.0197). The more significant improvement in Cryptosporidium recovery during 

the side-by-side studies was particularly associated with samples that had low initial recovery using 

Method 1623. For those samples where initial recovery (i.e., with Method 1623) was relatively higher, the 

improvement in recovery using Method 1623.1 was generally less pronounced. Both studies demonstrated 

similar precision and accuracy for Giardia recovery with either method. 

Fourteen volunteer laboratories participated in this validation study and were previously approved by the 

“Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium [3].” Each 

laboratory analyzed 4 reagent water samples and 4 water samples taken from a public water system 

source using Method 1623.1. The site used to collect source water samples was different for each 

laboratory and included six lake/reservoirs, seven rivers and one ground water under the influence of 

surface water. Cryptosporidium mean recoveries of 60 and 61% from reagent and source waters, 

respectively, represent a ≥20 percentage point increase compared to the previous validation of Method 

1623 [4]. The Giardia mean recoveries of approximately 40% were similar to the previous method 

validation. The mean relative standard deviation decreased for Cryptosporidium and Giardia in reagent 

water and for Cryptosporidium in the source water matrices compared with the Method 1623 validation. 

The mean relative standard deviation for Giardia in source water remained the same for both validation 

studies.

The data from this 1623.1 validation were used to develop quality control criteria for laboratory 

performance (Table 1). For each criterion, the distribution of recovery was estimated by using random 

effects ANOVA. The recovery limits were estimated as the 5th percentile of the predictive distribution. 

The lower limit acceptable for recovery of Cryptosporidium detected in reagent and source water 

increased by 11 and 19 percentage points, respectively, over Method 1623 criteria that were updated in 

2009 [3]. The lower limit for recovery for Giardia in reagent water increased by 8 percentage points; 

however, the recovery for Giardia in source water decreased by 7 percentage points. Laboratories 

performing Method 1623.1 are expected to have better detection rates for Cryptosporidium oocysts than 

those laboratories following Method 1623. Method 1623.1 includes more stringent quality control 

measures.
2

Table 1: Method 1623.1 Quality Control Acceptance Criteria

Performance Test Acceptance Criteria

Cryptosporidium Giardia

Initial precision and recovery (IPR)

Mean percent recovery 38 - 100 27 - 100

Precision (as maximum relative standard deviation) 37 39

Ongoing precision and recovery (OPR) 33 - 100 22 - 100

Matrix spike/matrix spike duplicate (MS/MSD)

Mean percent recovery 32 - 100 8 - 100

Precision (as maximum percent difference) 46 97
3

1.0 Introduction

In an effort since 2009 to improve method accuracy and precision, EPA’s Technical Support Center

(TSC) in the Office of Ground Water and Drinking Water, Standards and Risk Management Division, has 

been evaluating the efficacy of adding sodium hexametaphosphate (NaHMP) to artificial and 

environmental raw water matrices. These studies led to the development of Method 1623.1: 

Cryptosporidium and Giardia by Filtration/IMS/FS [1]. The substantial change from Method 1623 [2] is 

the addition of NaHMP prior to the elution procedure for the Envirochek® HV Sampling Capsule, which 

are presently used by approximately 80% of the laboratories approved for Cryptosporidium analysis. 

NaHMP has been used as an additive during ultrafiltration to reduce filter fouling [5], and has been used 

to improve Cryptosporidium detection in waste and finished waters [6, 7]. NaHMP improves the 

dispersion efficacy of surfactants by sequestering ions associated with water hardness, lowering the 

surface tension, and increasing the zeta potential of particles [8]. Method 1623.1 also incorporates a rinse

step to remove excess debris during immunomagnetic separation of the oocysts and cysts. 

In the spring and summer of 2010, TSC conducted a single-laboratory side-by-side analysis that 

compared samples from 9 public water systems each processed using both Methods 1623 and 1623.1. 

The results demonstrated that recoveries of spiked Cryptosporidium oocysts improved on average by 18 

percentage points (p = 0.0001) using Method 1623.1(Figure 1). Improvement in recovery was 

particularly notable when initial recovery (i.e., using Method 1623) was lower (i.e., below 30%). In an 

April and May 2011 study with four laboratories, Method 1623.1 was compared side-by-side with 

Method 1623 in analysis of six samples collected from three public water system sources (Figure 2). 

Method 1623.1 recovered an average of 15 percentage points more oocysts over Method 1623 for all 

three of the waters tested (p=0.0197). Again, the improvement in recovery was much more pronounced 

for those samples with a low initial recovery. 

 

Figure 1. Results of single laboratory side-by-side analysis

Cryptosporidium matrix spike recoveries from 9 sources comparing analyses using Methods 1623 and 

1623.1 (n=4). Error bars are ± one standard error of the mean.

Figure 2. Results of side-by-side analysis in 4 laboratories 

Cryptosporidium matrix spike recoveries from 3 sources comparing analyses using Methods 1623 and 

1623.1 (n=12). Error bars are ± one standard error of the mean.

In June 2011 we designed this study to provide inter-laboratory validation of Method 1623.1 and generate 

data to calculate quality control criteria that reflect the updated method and current laboratory capability. 

Laboratories approved by the “Laboratory Quality Assurance Evaluation Program for Analysis of 

Cryptosporidium [3],” as of July 18, 2011, were invited to participate. 
4

2.0 Study Design and Implementation

On behalf of EPA, Shaw Environmental and Infrastructure Inc. under EPA Contract EP-C-06-031 (the 

referee laboratory) was tasked to design and coordinate the study. The laboratories participating in this 

study were required to process and analyze all samples following Method 1623.1 and the instructions 

listed in Appendix A. The participants were asked to select source water with recoveries representative of 

the matrices tested in their laboratory, with the exception of one laboratory that was asked to select a 

ground water under surface water influence. In most instances the laboratory selected source water from 

a nearby utility to reduce shipping costs. The study included 14 sources from across the United States, 

consisting of 6 lake/reservoirs, 7 rivers and one groundwater under the influence of surface water (Figure 

3 & Table 2). 

Figure 3. Distribution of Source Waters

Rivers are blue triangles, lake/reservoirs are red circles, and ground water under the influence of 

surface water is a green square.

Table 2: Source Water Specifications

Source Water Type Collection 

Date

Turbidity 

(NTU)

Conductivity

(μS) pH

River 8/15/11 5.6 529 7.89

River 8/10/11 53 527 7.98

Ground Under Influence 8/9/11 2.89 687 6.8

Reservoir 8/18/11 3.77 252 7.5

Reservoir 8/10/11 1.0 64 7.39

River 8/16/11 3.64 604 7.8

Reservoir 8/8/11 0.78 51 6.68

River 8/15/11 7.8 318 7.82

River 8/11/11 4.4 885 7.93

River 8/10/11 25.5 524 8.04

Lake 8/10/11 5.32 137 6.5

River 8/10/11 3.9 82 7.6

Reservoir 8/11/11 1.6 33 7.1

Lake 8/17/11 3.47 400 8.5
5

2.1 Laboratory Selection

Interested laboratories completed applications to participate, which included historical matrix spike 

recovery information for the source water they planned to use for the study. All the laboratories routinely 

used the Envirochek® HV sampling capsule, and several had experience using NaHMP. Laboratory

selection was based on 1) the largest volume of samples completed for the Long Term 2 Enhanced 

Surface Water Treatment Rule (LT2ESWTR); 2) the amount of research performed on the source; and 3) 

the laboratory precision recorded for the last 10 proficiency testing (PT) rounds. Since the laboratories 

were asked to select source water with recoveries representative of the matrices tested in their laboratory, 

we anticipated the best representation from laboratories that have processed the most samples. We also 

expected records of physical and chemical water quality parameters (Table 2) to be most available when

the laboratory had a working knowledge about the source. Fourteen laboratories (Table 3) completed the 

study. Based on the national Cryptosporidium recovery average for PT samples of 54%, 8 of the 14

laboratories were above the national mean during the last 10 PT rounds and 11 of the 14 laboratories were 

below the national average for RSD of 18.7%. 

Table 3: Participating Laboratories

Referee lab: Shaw Environmental and Infrastructure, Cincinnati, OH 

American Water, Belleville, IL 

Analytical Services, Inc., Williston, VT 

BioVir Laboratories, Inc., Benicia, CA 

CH Diagnostic & Consulting Service, Berthoud, CO 

City of Phoenix Water Services Lab, Phoenix, AZ 

Environmental & Molecular Micro Lab at Michigan State University, East Lansing, MI 

New York City Department of Environmental Protection Pathogen Laboratory, Kingston, NY 

Pennsylvania Department of Environmental Protection, Harrisburg, PA 

Scientific Methods, Inc., Granger, IN 

Tetra Tech-Clancy Environmental, St. Albans, VT 

Truckee Meadows Water Authority, Sparks, NV 

TTL, Inc., Tuscaloosa, AL 

Underwriters Laboratories, Inc., South Bend, IN 

Note: No endorsement of these laboratories is implied, or should be inferred. 

2.2 Preparation of Spiking Suspensions

The spiking suspensions were provided by the Wisconsin State Laboratory of Hygiene, Madison, WI 

(WSLH). The 1-mL suspensions of live oocysts and cysts were counted by flow cytometry and 

confirmed by a minimum of 12 calibration verification standards performed by WSLH. The number of 

organisms in each spiking suspension was blind to the referee and volunteer laboratories throughout the 

course of the study. The oocysts were the Iowa isolate of Cryptosporidium parvum shed on 7/12/11, and 

provided by Sterling Parasitology Laboratory (Tucson, AZ). The cysts were the H3 isolate of Giardia

lamblia shed on 7/27/11, and provided by Waterborne, Inc. (New Orleans, LA). The spiking suspensions 

were prepared on 7/28/11; each suspension contained 130.31 ± 1.57 Cryptosporidium oocysts and 150.19 

± 1.60 Giardia cysts. Immediately after the Giardia were shed, all of the spiking suspensions were sorted 

on the same date to reduce fungal contamination, which had been observed in the stock samples of this

organism over the warm months. No problem was reported with the preparation of any of the spiking 

suspensions. 

WSLH supplied each laboratory with 10 spiking suspension tubes labeled 1-10. The laboratory was 

instructed to spike reagent water with tubes 1 – 5 and source water with tubes 6 – 10. Tubes 3 and 8 

contained spiking suspension fluid, but no organisms, to serve as the blank samples for reagent and 

source water. The laboratories were not informed as to which tube was the blank sample for each water 

type.
6

2.3 Quality Control 

The referee laboratory was responsible for acquiring supplies from the same lot, assessing their quality, 

and distributing them to the volunteer laboratories. Each laboratory was provided NaHMP, phosphate 

buffered saline, 10 Envirochek® HV Sampling Capsules (Pall Corporation, Ann Arbor, MI), and 10 IMS 

bead reactions (Dynabeads® GC-Combo, IDEXX, Westbrook, ME). The spiking suspensions were sent 

directly to the laboratories from the vendor. All laboratories followed Method 1623.1 quality control 

measures. In addition WSLH shipped one set of 10 blind spiking suspensions, from the same lot of 

organisms to be used for the study, to the referee laboratory a week prior to shipment to the volunteer 

laboratories. The referee laboratory verified quality control limits and evaluated the suspensions and the 

reagent lots by spiking 4 reagent water samples and 4 source water samples collected from the Ohio

River. One reagent water sample was rendered unusable during a laboratory accident. The referee 

laboratory completed the 10 quality control samples prior to the shipment of supplies to the volunteer 

laboratories. The spike values were not released to the referee laboratory analysts until after the 

completion of all of the in-house samples two weeks later. 

Table 4: Results of Quality Control Samples

Matrix Percent Recovery by Sample Mean SD RSD 1 2 3 4

Reagent Water

Cryptosporidium 72.1 78.3 unusable 89.0 79.8 8.5 10.7

Giardia 79.9 79.2 unusable 71.2 76.8 4.8 6.3

Source Water

Cryptosporidium 71.4 63.7 45.3 26.1 51.6 20.2 39.2

Giardia 45.9 51.9 35.3 8.7 35.5 19.1 54.0

3.0 Data Reporting and Validation

Fourteen laboratories submitted collection data, reagent documents, bench sheets, and examination results 

for review and validation. Results were submitted for a total of 70 reagent water and 70 source water 

samples. The referee laboratory used a double entry system to check each data set for completeness 

including: holding times, QC data, and result calculations. The data from three source water samples 

were considered invalid and unacceptable for inclusion due to incorrect performance of IMS or crosscontamination between two samples. The summary data include all valid results (Tables 5, 6, 7 and 8);

no outliers were detected by Grubbs test (alpha = 0.05). 

4.0 Results

4.1 Reagent Water 

The laboratories used a variety of “reagent waters” from filtered tap water to ultra purified milli-Q system 

water. Reagent water is defined in both Method 1623 and Method 1623.1 as “Water demonstrated to be 

free from the analytes of interest and potentially interfering substances at the method detection limit for 

the analyte.” The mean recovery of Cryptosporidium oocysts from four spiked reagent water samples in 

each of 14 laboratories was 60% with an average within laboratory RSD of 16.2% (Table 5). The mean 

recovery of Giardia cysts from four spiked reagent water samples was 40.8% with an average within 

laboratory RSD of 17.4% (Table 6). No oocysts or cysts were detected in any of the reagent water blank 

samples, and there were not any “non-detects” in the spiked samples. All laboratories examined 100% of 

the sample for reagent water samples.
7

Table 5: Spiked Reagent Water Sample Analysis Results for Cryptosporidium

Percent Recovery by Sample1

Mean SD RSD 1 2 3 4

70.6 75.2 78.3 67.5 72.9 4.8 6.5

79.0 56.8 66.8 58.3 65.2 10.2 15.6

29.2 38.4 31.5 36.1 33.8 4.2 12.4

53.0 79.8 66.0 65.2 66.0 11.0 16.6

27.6 44.5 25.3 67.5 41.2 19.5 47.3

69.1 64.5 74.4 69.8 69.4 4.1 5.9

66.0 41.4 53.7 54.5 53.9 10.0 18.6

62.2 47.6 57.6 53.0 55.1 6.2 11.3

69.1 59.1 39.9 77.5 61.4 16.2 26.4

68.3 77.5 67.5 63.7 69.3 5.9 8.5

81.3 59.9 55.3 68.3 66.2 11.5 17.3

61.4 82.9 73.7 60.6 69.6 10.7 15.3

65.2 80.6 68.3 70.6 71.2 6.6 9.3

47.6 44.5 49.9 34.5 44.1 6.8 15.3

All reagent water samples 60.0 15.0 16.2

1

Cryptosporidium spike levels for the reagent water samples were 130.31 ± 1.57 oocysts.

Table 6: Spiked Reagent Water Sample Analysis Results for Giardia

Percent Recovery by Sample1

Mean SD RSD 1 2 3 4

48.6 43.9 65.3 47.9 51.4 9.4 18.3

38.0 34.0 34.0 31.3 34.3 2.7 8.0

36.0 47.3 47.3 40.6 42.8 5.5 12.9

39.9 35.3 35.3 28.0 34.6 5.0 14.3

30.0 32.0 30.6 46.6 34.8 7.9 22.8

45.9 51.3 64.6 49.3 52.8 8.2 15.5

43.9 33.3 32.6 27.3 34.3 7.0 20.3

26.6 36.0 54.6 23.3 35.1 14.0 40.0

59.3 46.6 41.9 53.9 50.4 7.7 15.2

55.9 60.6 45.3 49.3 52.8 6.8 12.9

45.9 31.3 34.6 43.9 39.0 7.1 18.2

30.0 36.6 27.3 34.6 32.1 4.3 13.3

32.0 45.9 38.0 34.0 37.5 6.2 16.5

47.9 36.0 39.9 34.0 39.5 6.2 15.7

All reagent water samples 40.8 10.0 17.4

1

Giardia spike levels for the reagent water samples were 150.19 ± 1.60 cysts.
8

4.2 Source Water 

The mean Cryptosporidium recovery from spiked source water was 61.2% with an average within 

laboratory RSD of 12.7% (Table 7). The mean Giardia recovery for source water was 37.8% with an 

average within laboratory RSD of 23.1% (Table 8). No oocysts were detected in any of the source water 

samples that had not been spiked, and no non-detects occurred in the spiked samples. Giardia cysts were 

detected in four of the source water samples that had not been spiked. In these cases, the total number of 

cysts counted in the spiked source water samples were reduced by the number of organisms detected in 

the unspiked source water sample, per section 9.6.2.2 of Method 1623.1, and recoveries were calculated 

based on this corrected count. All laboratories examined 100% of the sample for source water samples. 

Only five samples, from one source, required three IMS subsamples to complete the examination. 

Table 7: Spiked Source Water Sample Analysis Results for Cryptosporidium

Percent Recovery by Sample1

Mean SD RSD 1 2 3 4

74.4 68.3 77.5 78.3 74.6 4.5 6.1

44.5 61.4 59.9 59.1 56.2 7.9 14.0

60.6 55.3 65.2 61.4 60.6 4.1 6.8

76.7 70.6 67.5 69.1 71.0 4.0 5.7

58.3 58.3 72.1 80.6 67.3 11.0 16.3

88.3 79.8 79.8 75.2 80.8 5.4 6.7

30.7 59.9 65.2 54.5 52.6 15.2 29.0

Invalida 40.7 60.6 62.9 54.7 12.2 22.4

Invalida 73.7 Invalida 59.1 66.4 10.3 15.5

72.1 71.4 70.6 80.6 73.7 4.6 6.3

38.4 46.0 38.4 32.2 38.8 5.7 14.6

62.9 58.3 54.5 48.3 56.0 6.2 11.0

21.5 24.6 31.5 27.6 26.3 4.3 16.2

81.3 82.9 82.1 70.6 79.2 5.8 7.3

All valid source water samples 61.2 16.8 12.7

1

Cryptosporidium spike levels for the reagent water samples were 130.31 ± 1.57 oocysts. a Invalid due to incorrect procedure
9

Table 8: Spiked Source Water Sample Analysis Results for Giardia

Percent Recovery by Sample1

Mean SD RSD 1 2 3 4

46.6 44.6 52.6 54.6 49.6 4.8 9.6

14.0 9.3 12.7 30.6 16.6 9.5 57.2

65.9 55.9 64.6 49.9 59.1 7.5 12.8

28.6 16.6 38.0 21.3 26.1 9.3 35.6

65.9 44.6 54.6 64.6 57.4 9.9 17.3

71.9b 62.6b 61.3b 61.9b 64.4 5.0 7.8

22.0c 25.3c 28.6c 19.3c 23.8 4.0 17.0

Invalida 20.6 37.3 41.3 33.1 10.9 33.1

Invalida 48.6c Invalida 45.3c 46.9 2.4 5.0

42.6 35.3 40.6 59.9 44.6 10.7 23.9

36.6 25.3 40.6 30.0 33.1 6.8 20.6

53.9d 39.9d 30.0d 24.6d 37.1 12.9 34.7

8.0 10.0 21.3 14.0 13.3 5.9 44.2

27.3 26.0 28.6 29.3 27.8 1.5 5.3

All valid source water samples 37.8 17.4 23.1

1

Giardia spike levels for the reagent water samples were 150.19 ± 1.60 cysts. a Invalid due to incorrect procedure

b,c,d Normalized to account for 5, 1, and 23 Giardia cysts detected in the non-spiked source water, 

respectively.

5.0 Comparison of Validation Study Results

A summary of the precision and recovery for Method 1623.1 was compared to the previous validation 

study for Method 1623 (Table 9). The Cryptosporidium reagent water and source water mean percent 

recoveries are ≥20 percentage points higher than the previous validation study (Figure 4). Giardia mean 

percent recoveries are similar to the previous validation study. The mean RSD has decreased for

Cryptosporidium in both matrices with Method 1623.1. 

There is a statistically significant difference between the Cryptosporidium and Giardia recoveries in both 

reagent and source waters (p < 0.0001). This difference was not observed in the previous Method 

validation conducted in 1999 [4]. It is possible that either the modifications in Method 1623.1 are 

deleterious to Giardia recovery or that the organisms were of poor quality. The former explanation is 

beyond the scope of this study; however, the latter is not unlikely because the Giardia cysts are not 

consistent products. There is a large amount of variation in this product due to shipping in warm seasons 

and a regular fungal contamination that is often found in the organism stocks, decreasing their shelf life. 

There are very few vendors of this product. Many of the laboratories commented that the microscopic 

characteristics of the recovered Giardia were of poor quality. 
10

Table 9: Comparison of Method 1623.1 Study Results to Previous Validation of Method 1623

Cryptosporidium Giardia

Matrix Statistic Method 1623 Method 

1623.1 Method 1623 Method 1623.1

Reagent 

Water

Mean % 

Recovery 40 60 38 41

Standard 

deviation 9 9 9 7

Mean RSD 24 16 29 17

n 29 56 31 56

Source 

Water

Mean % 

Recovery 38 61 42 38

Standard 

deviation 9 7 6 7

Mean RSD 25 13 23 23

n 14 53 16 53

Figure 4. Graphical Comparison of Cryptosporidium Recovery to Previous Validation

Cryptosporidium recoveries comparing the validation study results for Method 1623 and Method 

1623.1 in reagent and source water. Error bars are ± the mean of the within laboratory standard 

deviation.

6.0 Development of QC Acceptance Criteria 

These data from the 1623.1 validation were used to develop quality control criteria for laboratory 

performance. For each criterion, the distribution of recovery was estimated by using random effects 

ANOVA. The recovery limits were estimated as the 5th percentile of the predictive distribution for each 

criterion (the complete statistical analysis and results using additional models are described in Appendix 

B). The lower limit for recovery of Cryptosporidium detected in reagent and source water increased by 

11 and 19 percentage points, respectively, over the Method 1623 criteria (Table 10). The Giardia

recovery criteria increase 8 percentage points for reagent water, but decreased 7 points for source water 

(Table 10). It is unknown if the lower Giardia recovery in source water is due to the Method 1623.1 

modifications or if the organisms were of poor quality. The quality of the Giardia spiking material varies 

seasonally and may have been impacted by the warm conditions during the study. 
11

Table 10: Comparison of Study Criteria to Previous Criteria

Performance Test Cryptosporidium Giardia

Method 1623 Method 

1623.1 Method 1623 Method 1623.1

Initial precision and recovery 

Mean % recovery 24 - 100 38 - 100 24 -100 27 - 100

Precision (as maximum RSD) 55 37 49 39

Ongoing precision and recovery 22 – 100 33 - 100 14 - 100 22 - 100

Matrix spike/matrix spike 

duplicate 

Mean % recovery 13 - 111 32 - 100 15 - 118 8 - 100

Precision (as maximum RPD) 61 46 30 97

7.0 Conclusion 

This study has successfully validated Method 1623.1 in fourteen laboratories and demonstrated a 

mean Cryptosporidium recovery from source water of 61.2% with an average within laboratory RSD of 

12.7%. The mean Giardia recovery from source water was 37.8% with an average within laboratory 

RSD of 23.1%. These recoveries are supported by two side-by-side studies that showed improved 

accuracy of ≥ 15 percentage points for detecting the concentration of Cryptosporidium oocysts when 

using Method 1623.1 compared with Method 1623. The improvement in recovery was much more 

pronounced for those samples with a low initial recovery. Similar precision and accuracy were observed 

for detecting Giardia recovery compared with Method 1623 in both studies. The lower limit acceptable 

for recovery of Cryptosporidium detected in reagent and source water increased by 11 and 19 percentage 

points, respectively, over the updated Method 1623 criteria. Laboratories performing Method 1623.1 

should have more accurate detection and follow more stringent quality control criteria than those 

laboratories following Method 1623. 
12

8.0 References 

1. USEPA. 2012. Method 1623.1: Cryptosporidium and Giardia in Water by Filtration/IMS/FA. 

EPA 816-R-12-001. Office of Water, Washington, DC.

2. USEPA. 2005. Method 1623: Cryptosporidium and Giardia in Water by Filtration/IMS/FA.

EPA-815-R-05-002. Office of Water, Washington, DC.

3. “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium Under the 

Safe Drinking Water Act (Renewal)” 74 Federal Register 36 (25 February 2009), pp. 8529 -

8534. 

4. USEPA. 2001. Results of the Interlaboratory Method Validation Study for Determination of

Cryptosporidium and Giardia Using USEPA Method 1623. EPA-821-R-01-028. Office of Water, 

Office of Science and Technology, Engineering and Analysis Division, Washington, DC.

5. Hill, V. R., A. L. Polaczyk, D. Hahn, J. Narayanan, T. L. Cromeans, J. M. Roberts, and J. E. 

Amburgey. 2005. Development of a rapid method for simultaneous recovery of diverse microbes 

in drinking water by ultrafiltration with sodium polyphosphate and surfactants. Appl Environ 

Microbiol 71:6878-84.

6. Clancy, J.L. and R.M. McCuin. 2005. Cryptosporidium in wastewater: Occurrence, removal and 

inactivation. Water Environment Research Foundation.

7. McCuin, R.M., et al. 2001. Improving methods for isolation of Cryptosporidium oocysts and

Giardia cysts from source and finished waters in: Proceedings of American Water Works 

Association Water Quality Technology Conference. Denver, CO.

8. Lepesteur, M., S. Blasdall, and N.J. Ashbolt,. 2003. Particle dispersion for further

Cryptosporidium and Giardia detection by flow cytometry. Lett Appl Microbiol 37(3): p. 218-29.
13

Appendix A. Instructions to Participating Laboratories: 

1. Please review the 2011 Method 1623 revision draft [subsequently Method 1623.1] by Aug. 8th. 

To enhance program-wide data quality and consistency, and guard against the use of sample processing 

shortcuts that could compromise data quality, the updated method provides laboratories with the 

flexibility to select from options for various procedural components in the method. However, each option 

must be performed according to the procedures used during a multi-laboratory validation studies. No 

procedural modifications will be permitted without demonstrating acceptability through a multilaboratory validation study using multiple matrices, as per 40 CFR Part 141.27.

Therefore, for this study, only those modifications specified in the text or in Table 1 of the 

Method draft will be allowed. Review these modifications along with your laboratory SOPs and 

select the process that will best reflect your future practices (These samples can serve as an IDC set 

for a Method 1623 revision). Refer any questions to Leah Villegas at 513-569-7886 or 

villegas.leah@epa.gov. 

Major changes to the draft are listed on page v. Especially note the following sections:

• 11.3.3 Sample Spiking [subsequently 11.2.3]: Follow the procedure as written, but adjust 

for a spike volume of 1 mL.

• 12.2.6.2 NaHMP pre-elution [subsequently 12.2.7 Dispersant addition]: Follow the 

procedure as written. The pre-elution step is the beginning of elution and a one working 

day holding time is required for elution to slide application.

• 13.3.2.12 Microcentrifuge tube portion of IMS: The volumes listed for the quantitative 

transfer from the flat-sided tube to the microcentrifuge tube have changed in the draft or 

you can use manufacturer’s instructions. The MPC-S magnet or equivalent is specified. 

• 13.3.2.17 Required rinse of the microcentrifuge tube: Follow the procedure as written. 

• 13.4 Options for complicated matrices: Review and use if needed or preferred for your 

IDC.

2. Two packages will arrive at your facility on August 9, 2011. Wisconsin State Laboratory of Hygiene 

will ship 10 live spiking suspensions directly to your laboratory. Shaw Environmental will ship 10 

Envirochek® HV filters, a box of G/C combo IMS beads for 10 reactions, sodium hexametaphosphate, 

and PBS to use for the study along with a UPS mailer to return the data sheets.

3. All Method 1623 holding times MUST be met. The spiking suspensions will not expire until 8/19/11. 

Therefore, laboratories have until 8/19/11 to spike and filter the samples. 

4. The entire pellet MUST be processed through the examination step for this study. Please contact 

Leah Villegas (513-569-7886 or villegas.leah@epa.gov) if you anticipate that your selected source 

water will require multiple IMS subsamples.

5. Prepare shipped reagents: Prepare the reagents shipped on 8/9/11 as follows: 

• Sodium hexametaphosphate (NaHMP): 

o Prepare a sufficient quantity of 5% sodium hexametaphosphate (NaHMP) solution to 

pre-treat all of the filters associated with the OPR/MB used for that batch of elution 

buffer. Pre-treatment may require up to 150 mL of NaHMP per sample.

o 5% w/v NaHMP solution—Dissolve 50g of sodium hexametaphosphate ((NaPO3)n -

Sigma 71600 or equivalent) in 600 mL of reagent water. Dilute to final volume of 1000 

mL using reagent water. Store in a plastic or glass container at room temperature for up 

to 3 months. Discard when expiration date is reached or sooner if microbial growth is 

apparent.

• Prepare 1X PBS

o Follow preparation instructions provided on the packet.

6. Use the Benchsheets and Reagent Batch forms supplied by Shaw Environmental.

7. 10 spiking suspension are provided. They are numbered with a letter corresponding to your 

laboratory plus the numbers 1-10. Please add spike samples 1-5 to the reagent water samples and 

6-10 to the source water samples (MS). Each set of 5 spiking suspension includes a blank sample. 

Use the spiking suspension label as the sample ID. The same batch of elution buffer must be used 

for all 10 samples.
14

8. Follow the revised method draft as written; spiking 5 reagent water samples and 5 samples from 

your selected source water. Perform the pre-elution treatment on all samples.

9. Perform three oocyst and cyst characterizations for each sample. 

10. When samples are completed send the bench sheets (Reagent Batch Sheet, Bench sheets and 

Microscopy results forms) to Shaw Environmental using the provided UPS mailer and return label.

11. Once the study is completed and validated Shaw Environmental will send you a results page to 

complete your IDC information.
15

Appendix B. Description of Statistical Analysis Used to Develop QC 

Acceptance Criteria for Method 1623.1: 

The QC acceptance criteria for Method 1623.1 were developed by characterizing the performance of the 

laboratories with the following prediction intervals: 

• The IPR recovery limit is estimated as the 5th percentile of the predictive distribution of means of 

4 future observations. 

• The IPR precision limit is estimated as the 95th percentile of the predictive distribution of RSDs 

from 4 future observations. 

• The OPR recovery limit is estimated as the 5th percentile of the predictive distribution for 

recovery in reagent water. 

• The MS recovery limit is estimated as the 5th percentile of the predictive distribution for recovery 

in source water. 

• The MS/MSD average recovery limit is estimated as the 5th percentile of the predictive 

distribution for the mean of 2 recoveries in source water. 

• The MS/MSD precision limit is estimated as the 95th percentile of the predictive distribution for 

the relative percent difference (RPD) of 2 recoveries in source water.

Calculation of Recovery

Recovery is generally estimated as p = x/n, where x is the count and n the spiking number. It is a direct 

estimate of the probability that a spiked organism will be identified and counted. The number of 

organisms counted in a spiked sample is often viewed as a binomial random variable, Bin(n,p). Each 

organism spiked is viewed as an independent Bernoulli trial with common probability of success (being 

counted) equal to p. 

Probability was modeled in the recovery scale and using the logit transformation of the probability. This 

is defined as ( ) 1 ln p

p y − = , where p is a probability and ln is the natural logarithm. The quantity 1

p

− p

is 

known as the odds ratio (or just “the odds”). The inverse of the logit transformation is given by 

( ) 1 y y pe e = + . 

In the case of source water analysis, the average of counts found during analysis of the unspiked sample, 

the matrix correction, must be subtracted from the counts. In the logit scale, the spiking numbers or the 

matrix correction are not required to be integers.

Approaches to Estimating Criteria

Previous studies have consistently shown significant random effects for laboratory and for source water. 

These random effects were shown to be normally distributed in the logit (log recovery odds ratio) scale. 

In addition to the lab/source water random effect, a substantial sample level random effect is apparent 

consistently in Method 1623 and Method 1623.1 data. This is seen in the extra-binomial variance, the fact 

that the variances of recoveries from replicate spiked samples is always much larger than would be 

expected under the binomial model. The variance of a binomial proportion (which is what a recovery is) 

is p p ( ) 1

n

− . Figure B-1 below illustrates this using the sample standard deviations and means of lab reagent 

water spiked replicates from the method validation study. The lines show the standard deviations 

expected under the binomial model: p p ( ) 1

n

− . 
16

In this study the laboratory and source water random effects are combined into a single random effect 

called Lab. This results in a one-way random effects (RE) ANOVA model [1]. Since there is no fixed 

effect (other than a constant) in the model, this is a pure RE model rather than a mixed effects model. 

The model for the reagent water data uses Lab as a random effect also. The sample level random effect 

(error variance) in the logit normal model for recoveries directly accommodates the extra-binomial 

variance seen in Figure B-1.

Figure B-1. Standard deviation versus mean of reagent water spike recovery grouped by laboratory

Each point shows the mean and standard deviation from four replicates analyzed by one laboratory.

Figure B-2 below shows the same data with the means and standard deviations calculated in the logit 

scale. In the logit scale the data show more constant standard deviations and less relationship between 

mean and standard deviation. In fact, the logit transformation does make the variance of binomial 

proportions approximately constant (based on the delta method approximation). 

Three approaches have been used in this study to estimate the QC acceptance criteria for recovery and 

their precision of estimation. The first two are based on normality and the one-way RE ANOVA 

decomposition in the logit scale. These methods are based on computing a prediction variance that is 

based on a linear combination of heterogeneous sums of squares, between lab and within lab. The exact 

distribution of the prediction variance is unknown, but there are various approximations. This is known as 

the Behrens-Fisher problem. 

0%

2%

4%

6%

8%

10%

12%

14%

16%

18%

20%

0% 20% 40% 60% 80% 100%

Std Dev of Recovery

Mean Recovery

Std Dev vs Mean of Recovery-Reagent Water

Crypto

Giardia

Binomial SD Crypto

Binomial SD Giardia
17

The first approach uses the well-known Satterthwaite approximation [2] to the Behrens-Fisher distribution 

(of the prediction variance) in the estimation of percentiles. This method produces prediction limits for 

recovery of individual spiked samples and for the mean recovery of groups of spiked samples. 

The second approach uses a different approximation to the Behrens-Fisher distribution and allows 

construction not only of prediction limits but also of confidence limits for those prediction limits (and 

therefore of tolerance limits). It uses methodology presented by Krishnamoorthy and Mathew [3]. 

Figure B-2. Standard deviation versus mean of logits of reagent water spike recovery grouped by laboratory

The third approach is a hierarchal Bayesian approach. It uses Markov Chain Monte Carlo (MCMC) [4,5] 

simulation implemented through the OpenBUGS [6] software to estimate the predictive distribution for 

each of the QC Acceptance criteria statistics. Percentiles are estimated from this distribution. 

Transformations are not used. The simulation is directly in the scale of the counts. 

Two methods have been used to estimate QC acceptance criteria for precision. The first is based on an F

statistic applied to the empirical pooled RPDs and RSDs from the study. The second is based on 

constructing RSDs and RPDs from the predictive distribution samples generated by the Bayesian MCMC 

simulation.

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

-1 -0.5 0 0.5 1 1.5

Std Dev of Logits

Mean Logit

Std Dev vs Mean of Logits--Reagent Water

Crypto

Giardia
18

ANOVA Calculations

The spiking numbers used in this study were larger than the spiking number of 100 that is typically used 

for routine QC. This was done in order to improve the precision of the study data. However, in 

constructing a predictive distribution for future recoveries one must account for the fact that the future 

spiking number will be 100. This is done by multiplying the error mean square (error variance) or the 

error sum of squares by the ratio of the spiking number in the study divided by 100. 

The between Lab mean square is not affected by the change in spiking number. 

Balanced Case: Reagent Water Criteria (IPR and OPR)

Let { }, 1, , , 1, , ij p i Lj n = =   be estimated recoveries for spiked samples, with i indexing the 

laboratories and j indexing the replicates. In the balanced case, there are n replicates per lab. In the case 

of the reagent waters, L = 14, and n = 4. Let ( ) ( ) 1 logit ln ij

ij

p

ij ij p y p = = − be the corresponding logits. 

The model is 

ij i ij y =++ μλ ε , (1) 

where ( ) 2 ~ 0, λ σ i L N are Lab random effects, and ( ) 2 ~ 0, ε σ ij N , all mutually independent. The 

variance of an observation is ( ) 2 2 var ij L y = + σ σ , the sum of the two “variance components.” 

The Lab mean square is 

( )2

Lab . ..

1

MS

1

L

i

i

n

y y L =

= − − ∑ , (2) 

where .

1

1 n

i ij

j

y y

n =

= ∑ are the Lab means and 

14

1

.. 14 .

1

i

i

y y == ∑ is the grand mean. It has L - 1 = 13 degrees of 

freedom (df). The error mean square is 

( ) ( )2

.

1 1

1 MS

100 1

L n

E ij i

i j

SN

y y L n = =

= − − ∑∑ , (3) 

where SN is the study spiking number. It has L(n – 1) = 42 df. The significance of the model is tested 

using the ratio MSLab over MSE compared to the F distribution with 13 and 42 df. 

The error variance 2 σ is estimated by the MSE, and the Lab variance 2 σ L is estimated as 

2 MS MS Lab E

L s

n

− = . (4) 

For a full discussion of one-way RE ANOVA models, see Montgomery [1].

Unbalanced Case: Source Water (MS/MSD)

In the unbalanced case, there are a variable number of replicates per lab. Because of the invalidation of 

some data in the source water data set, it is unbalanced. Let { }, 1, , , 1, , ij i p i Lj n = =   be estimated 
19

recoveries for spiked samples, with i indexing the laboratories and j indexing the replicates. The values of 

the ni vary from 2 to 4. Let ( ) ( ) 1 logit ln ij

ij

p

ij ij p y p = = − be the corresponding logits. 

The Lab mean square for the unbalanced case is 

( )2

Lab . ..

1

1 MS

1

L

i i

i

ny y L =

= − − ∑ , (5) 

where .

1

1 i n

i ij

i j

y y

n =

= ∑ are the Lab means and 

14

1

.. 14 .

1

i

i

y y == ∑ is the grand mean. It has L - 1 = 13 df. The 

error mean square for the unbalanced case is 

( ) ( ) 2

.

1 1

1 MS 1

100

i L n

E ij i i

i j

SN

yy n L = =

= −− ∑∑ . (6) 

The one-way model for the source water has ( ) 1

1 39 L

i i

n = ∑ − = error df. The significance of the model is 

tested using the ratio MSELab over MSEE compared to the F distribution with 13 and 39 df, respectively. 

The error variance 2 σ is estimated by the MSE, and the Lab variance 2 σ L is estimated as 

2

2 Lab 1

0

0 1

1

MS MS 1 , where 1

L

L i

E i

L i L

i

i

i

n

s nn

n L

n

=

=

=

 

  − = = −   −  

   

∑

∑

∑

. (7) 

Predictive Distribution Based on ANOVA Calculations

The first approach to estimating percentiles of the predictive distribution of recovery uses the well-known 

Satterthwaite approximation [2] to the Behrens-Fisher distribution. The second uses an approximation 

presented by Krishnamoorthy and Mathew [3] based on the non-central t-distribution. 

Using Satterthwaite’s Approximation 

The prediction variance for a single new observation is given by 

( ) ( ) 2 22 1 1 σ σσ pred =+ ++ 1 1 L L nL (8) 

and is estimated by 

( ) ( ) 2 22 1 1 s ss pred =+ ++ 1 1 L L nL . (9) 

The prediction variance for the mean of k future observations is estimated by

( ) ( ) 2 22 1 11 , s ss pred k =+ + + 1 L L k nL (10) 
20

Since these are “synthesized” variances, their distributions do not follow a chi-squared distribution, as is 

classically required for inference using the t-distribution. 

One approach to approximating their distribution is to use Satterthwaite’s approximation [7]. This 

approximates their distributions as being chi-squared with degrees of freedom given by 

( )

( ) ( ) ( ) ( )

( )

( ) ( ) ( ) ( )

2 2

2 2 1 1 2 2

2 2

,

2 2 1 2 2 1 1

, and

1 11 1

,

11 1

pred

L L nL

pred k

k

L L k nL

s

df

s L s Ln

s

df

s L s Ln

=

   + −+ + −   

=

   + −+ + −   

(11) 

respectively. 

The predictive distributions for a single new observation and for the mean of k future observations in the 

logit recovery scale are estimated by a t-distribution as

( )

( )

2

2

,

~ , , , and ˆ ˆ

~, ,, ˆ ˆ

pred

k pred k k

y t df

y t df

μ σ

μ σ

(12) 

respectively.

Then the 95% lower prediction limits (LPL) in the logit and recovery scales for a single new observation 

and for the mean of k future observations are given by 

,

,

2

0.05,

2

, , 0.05, ,

ˆ ˆ ; , and 1

ˆ ˆ ; , 1

LPL

LPL

LPL k

k LPL k

y

LPL pred df LPL y

y

LPL k pred k df LPL k y

e

y tp

e

e

y tp

e

μ σ

μ σ

= + =

+

= + =

+

(13) 

For the unbalanced case, in all the formulae of this section, n is replaced by n0. 

Using Method of Krishnamoorthy and Mathew 

Krishnamoorthy and Mathew [3] use a non-central t-distribution to approximate the distribution of a tstatistic computed using a synthesized or heterogeneous standard deviation in the denominator. Their 

method allows not only the estimation of percentiles but also of confidence limits about those percentiles. 

This permits the construction of one-sided tolerance limits, specifically of guaranteed content tolerance 

intervals. In our case, a 95% lower confidence limit on the 5th percentile is equivalent to a 95% 

confidence 95% coverage Lower Tolerance Limit. 

Their formulae for the balanced case are
21

( ) ( )

( ) ( ) ( ) ( )

( ) ( )

.. 1 1

1 2

1 1, 1

2 2

Lab . .. .

1 1 1

, , where 1

1 1,

, . 100

Lab

k

E

L L n

Lab

L L n

i E ij i

i i j

SS Tyt LL n

SS zL L F

SS

SN SS n y y SS y y

β

δ γ

δ γ

−

− −

= = =

= + −

 

= +− −    

=− = − ∑ ∑∑

(14) 

In these formulae, our desired percentile probability is β = 0.05. The symbol zβ denotes the β-quantile 

value of the standard normal distribution. Our confidence coefficient γ for a lower confidence limit is not 

95% but 5%. 

In the unbalanced case, the estimator is 

( ) ( )

( ) ( )

( ) ( ) ( )

( )

1 2

1 2

2 1 ,

2

.

1

.

1

1

, , where 1

1 1

1 ,

,

1 ,

1 1 .

x

u k

E

L NL

x

L

x i

i

L

i

i

L

i i

SS T yt L L

L L SS z L F

N L SS

SS n y y

y y L

L n

β

δ γ

λ δ γ

λ

−

− −

=

=

=

= + −

  − − = + −     −

= −

=

=

∑

∑

∑

(15) 

The calculations presented above are for limits for single spike recoveries; that is, OPR or MS (not 

average of MS/MSD) limits. For IPR and MS/MSD limits one adjusts the calculation of SSE in equation 

14 by dividing it by the number of counts averaged: 4 for the OPR limit and 2 for the MS/MSD limit. 

Bayesian MCMC Predictive Distribution

The Bayesian MCMC method constructs by a very special type of simulation an approximation to the 

joint distribution of the model parameters that is most consistent with the data and with the assumptions 

behind the model. These assumptions include both the structure of the model and the assumed prior 

distribution of the parameters, which are viewed as random rather than fixed. The joint probability 

distribution for the model parameters so generated is known as the posterior distribution. It is this 

posterior distribution that is used to generate a predictive distribution for future observations. 

The model used for the Bayesian MCMC is as follows. The basic counts follow a binomial distribution. 

The counts Xij are indexed in the same manner as the logits of the recoveries 

X Np ij ~ Bin , ( ) ij ij

The spiking numbers Nij are random for each sample. The average and standard deviation of the spiking 

number are determined by the vendor. In this study, they were 130.31 and 150.19 with standard 

deviations of 1.57 and 1.6 for Cryptosporidium and Giardia, respectively. They were modeled by 
22

generating normal variates and rounding to integers. To compensate for the variance reduction due to 

rounding, the variances were inflated by a factor of 1.09, determined by simulation. 

( ( )) 2 ~ round ,1.09 N N ij μ σ 

The logits of the ij p are modeled by the one-way RE ANOVA model.

( ) 0, 1, logit ij ij i p bb =+ + α , 

where α is a fixed constant for each simulation of the experiment, b0,ij is a sample level random effect and 

b1,i is a Lab level random effect. These parameters have the following distributions.

( )

( )

( )

1

1

0, 0

1

1, 1

~ 0, , one value for each simulation

~ 0, , one value for each sample in each simulation

~ 0, , one value for each Lab in each simulation

ij

i

N

b N

b N

α τ α

τ

τ

−

−

−

The hyperparameters ( ) 0 1 , , α τττ are distributed according to the diffuse Gamma distribution with 

parameters α = 0.01 and β = 0.01. 

When this MCMC simulation has converged to produce an acceptable distribution for the 

hyperparameters, a large set of simulated values from later in the run (throwing away the “burn-in” 

values) is saved. This set of values is a large sample from the posterior distribution of the 

hyperparameters. This sample from the posterior distribution is then used to simulate the predictive 

distribution for the QC criteria statistics. 

In order to generate the predictive distribution of the routine recovery verification (OPR) quality sample, 

simulate one X value (count) with a target spiking level of 100 (but random N, as above) for each 

( ) 0 1 , , α τττ value. Take the 5th percentile from this set of X values as an estimate of the OPR limit.

To generate the predictive distribution of the initial precision and recovery (IPR) mean and RSD, simulate 

four X values with a target spiking level of 100 (but random N, as above) for each ( ) 0 1 , , α τττ value. They 

will share a common value of α and of b1. Four independent values of b0 will be generated, one for each X

value. For each set of four simulated X values, a mean recovery and RSD will be calculated. The 5th and 

95th percentiles of the sets of means and RSDs will be the estimates of the IPR precision and recovery 

limits. 

To generate the predictive distribution of the MS/MSD mean and RPD, simulate two X values with a 

target spiking level of 100 (but random N, as above) for each ( ) 0 1 , , α τττ value. They will share a 

common value of α and of b1. Two independent values of b0 will be generated, one for each X value. For 

each set of two simulated X values, a mean recovery and RPD will be calculated. The 5th and 95th

percentiles of the sets of means and RPDs will be the estimates of the MS/MSD precision and recovery 

limits. 

The posterior distributions for the parameters was generated using OpenBUGS version 3.2.1 [6]. Four 

runs were made for the two organisms and two matrices. In each run nine independent Markov chains 

were generated with different starting values for the parameters. Mixing was very good for all parameters 

except for α and τα. After thinning by a factor of 10 to reduce the autocorrelation, these were suitable as 
23

well. After burn-in and thinning, between 5,000 and 20,000 values were retained per chain. The predictive 

distribution simulations were performed using the R statistical computation system [7] (version 2.13.2). 

Limits for RSD and RPD

The RSD is the relative standard deviation, standard deviation divided by the mean. The RPD (applicable 

only for pairs of measurements) is the relative percent difference, the absolute difference divided by the 

mean. It is equal to the RSD multiplied by 2 . 

As an alternative to the Bayesian MCMC prediction limits for RSD and RPD described in the previous 

subsection, the following ‘classical’ methodology was used to estimate quality limits for RSD and RPD. 

Since RSDs are only applicable to data with positive means, it is computed only in the recovery scale, not 

in the logit scale. The population RSD is estimated as a pooled RSD from the labs in the study. This is 

computed as the root mean square of the individual lab RSDs. 

( ) ( )

2

1

pool

2

1

1

1 RSD , in the balanced case

RSD

1 1 RSD , in the unbalanced case

where .

L

i

i

L

i i

i

L

i i

L

n

N LL

N n

=

=

=





 = 

 −  − 

=

∑

∑

∑

(16) 

For independent RSDs with a common mean, the square of their ratio has approximately an F

distribution. It must be remembered that future MS/MSD will be spiked at 100 organisms rather than 130 

or 150, as in this study. Therefore, one can determine an approximate limit for RSDs of k future 

recoveries as a 95% prediction limit based on the equation:

2

max,

0.95, 1, max, 0.95, 1, 100 pool

pool

RSD

RSD RSD

RSD

k S F F k df − − k k df

    =⇒= ⋅

 

, (17) 

where df is the degrees of freedom for the pooled RSD estimator (N – L), and S is the spiking number 

used in this study. Because the pooled RSD does not include between Lab variance, the ordinary degrees 

of freedom is used rather than Satterthwaite’s approximation.

The limit for RPD is 2 times the limit for the RSD of two future observations. 
24

Summary and Discussion of Statistical Analysis

The quality control acceptance criteria developed using these statistical models and procedures described

above are listed in Table B-1. Column 1 lists the current Method 1623 criteria developed using data from 

three different sources. The IPR and Giardia OPR criteria were based on results from six laboratories 

analyzing 293 Cryptosporidium and 186 Giardia OPR samples collected during the Information 

Collection Rule Supplemental Survey, while the matrix spike criteria is based on results from 8

laboratories analyzing a total of 14 Cryptosporidium and 16 Giardia samples from 8 source waters 

collected during the Method 1623 validation study. The Cryptosporidium OPR minimum recovery 

criterion was updated in 2009 [8]; this update was based on 333 reagent water PT samples from 58 

laboratories. Column 2 lists criteria developed from results from 51 laboratories during 5 reagent water 

PT rounds and is based on 753 Cryptosporidium and 530 Giardia samples. The criteria developed from 

these PT round data used a random effects model in the recovery scale and more accurately reflects

current laboratory capability and provides a better assessment of laboratory performance in reagent water 

using Method 1623. 

Columns 3 through 7 were calculated using the statistical models and procedures described in this 

Appendix. The results in each column are calculated from the data collected during this Method 1623.1 

validation study which consisted of 56 reagent water and 53 source water samples collected from 14 

different public water system sources and analyzed in 14 laboratories. Column 3 highlights the prediction

limits based on the normal random effects model in the recovery scale. These quality control criteria 

were confirmed by comparing the recovery scale to the results using the logit transformation (Column 4). 

The results for the recovery and logit scale are very similar, within one value for most of the criteria. 

The only exception is the Giardia MS/MSD mean recovery which is 8 in recovery scale and 12 in logit. 

This difference is due to the presence of a higher frequency of low recoveries (<30%) for the Giardia

MS/MSD, these lower recoveries have less impact in the logit scale. The use of the 5th percentile as the 

prediction limits was also evaluated by developing tolerance limits in both the recovery and logit scale 

(Columns 5 and 6). The data were also evaluated using a Bayesian Markov Chain Monte Carlo 

simulation (Column 7). The calculated quality assurance criteria are similar for various statistical 

procedures and models, therefore the 5th percentile prediction limit based on the normal random effects 

model in recovery scale (the yellow Column 3) was selected for the final version of Method 1623.1. This 

is the same procedure and model used to develop the quality assurance criteria for Method 1623. The 

criteria developed using the PT data (Column 2) demonstrates that laboratories are currently able to 

consistently achieve Cryptosporidium OPR recoveries greater than 33%. The modifications required in 

Method 1623.1 have not demonstrated an impact on the quality control acceptance criteria for reagent 

water recoveries as compared to Method 1623, but the higher value for acceptance criteria in Method 

1623.1 provide a better standard for source water recovery.
25

Table B-1: Comparison of Study Criteria to Previous Criteria Using Different Statistical Approaches

Cryptosporidium

Method 1623 Method 1623.1

Column Number 1 2 3 4 5 6 7

Performance Test

Criteria in 

Method 

1623

Prediction 

limits based 

on normal 

RE model in 

recovery 

scale*

Prediction 

limits 

based on 

normal RE 

model in 

recovery 

scale

Prediction 

limits based 

on normal 

RE model 

in logit

Tolerance and 

prediction limits for 

normal RE model in 

recovery scale

Tolerance and 

prediction limits for 

normal RE model in 

logit scale

Bayesian 

MCMC 

estimates

5% Pred 95% 5% Pred 95%

Initial Precision and 

Recovery (IPR)

Mean Recovery (percent) 24 - 100 36 38 - 100 37 - 100 27 39 46 27 38 45 37

Precision as maximum 

RSD 55 34 0 - 37 0 - 37 35

Ongoing Precision and 

Recovery (OPR)

Single Recovery (percent) 22 - 100 33 33 - 100 32 - 100 26 37 44 27 37 43 32

MS/MSD

Mean Recovery (percent) 13 - 111 32 - 100 31 - 100 20 34 43 21 33 42 32

Precision as maximum 

RPD 61 0 - 46 0 - 46 46

Giardia

Method 1623 Method 1623.1

Performance Test

Criteria in 

Method 

1623

Prediction 

limits based 

on normal 

RE model in 

recovery 

scale*

Prediction 

limits based 

on normal 

RE model 

in recovery 

scale

Prediction 

limits 

based on 

normal RE 

model in 

logit scale

Tolerance and 

prediction limits for 

normal RE model in 

recovery scale

Tolerance and 

prediction limits for 

normal RE model in 

logit scale

Bayesian 

MCMC 

estimates

5% Pred 95% 5% Pred 95%

Initial Precision and 

Recovery (IPR)

Mean Recovery (percent) 24 - 100 32 27 - 100 28 - 100 20 27 31 22 28 31 28

Precision as maximum 

RSD 49 37 0 - 39 0 - 39 34

Ongoing Precision and 

Recovery (OPR)

Single Recovery (percent) 14 - 100 29 22 - 100 24 -100 19 25 29 21 26 29 23

MS/MSD

Mean Recovery (percent) 15 - 118 8 - 100 12 - 100 0 10 19 7 13 19 11

Precision as maximum 

RPD 30 0 - 97 0 - 97 85

*Values calculated using results of 5 reagent water PT rounds
26

References

1. Montgomery, D. 1991. Design and Analysis of Experiments, 3rd Edition. Wiley, New York. 

Sections 3 – 5.

2. Sahai, H. and M. Ojeda. 2003. Analysis of Variance for Random Models: Unbalanced Data: 

Theory, Methods, Application and Data Analysis, Vol. II. Birkhauser, Boston. pp. 383-386.

3. Krishnamoorthy, K. and T. Mathew. 2004. One-Sided Tolerance Limits in Balanced and 

Unbalanced One-Way Random Models Based on Generalized Confidence Intervals.

Technometrics 46(1), pp. 44-52.

4. D. J. Spiegelhalter, A. Thomas and N.G. Best. 1996. “Computation on Bayesian graphical 

models” Bayesian Statistics 5 pp. 407-425.

5. Hastings, W.K. (1970) “Monte Carlo sampling methods using Markov chains and their 

applications” Biometrika 57 (1), pp. 97-109.

6. Spiegelhalter, D., A. Thomas, N. Best and D. Lunn . 2011. OpenBUGS User Manual. URL 

http://www.openbugs.info/Manuals/Manual.html.

7. R Development Core Team. 2011. R: A language and environment for statistical computing. R 

Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL 

http://www.R-project.org/.

8. “Laboratory Quality Assurance Evaluation Program for Analysis of Cryptosporidium Under the 

Safe Drinking Water Act (Renewal)” 74 Federal Register 36 (25 February 2009), pp. 8529 -

8534.