Document ID: EPA-HQ-OW-2002-0011-0035
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2009-02-25T05:00Z

Updated Procedures for Evaluating Ongoing Precision and Recovery Quality
Control 

Calculated from February and May 2007 Proficiency Testing Data  

February 23, 2009	

Introduction

The procedures for evaluating Ongoing Precision and Recovery (OPR)
results used by the “Laboratory Quality Assurance Evaluation Program
for Analysis of Cryptosporidium under the Safe Drinking Water Act,”
hereafter referred to as the “Lab QA Program,” were originally based
on data collected during the Information Collection Rule Supplemental
Surveys from March 1999 to February 2000.  Table 1 summarizes the data
set used to develop a criterion for evaluation.  The OPR mean recovery
was calculated from data performed by a limited number of laboratories,
using the earliest versions of Method 1623 and Method 1622 (1999), and a
filter that is no longer used by any of the laboratories currently
participating in the Lab QA Program.  Therefore, EPA expects that the
minimum recovery criterion developed using this old data underestimates
current method and laboratory capability. 

To develop a more appropriate criterion, updated limits for lower
recovery values were calculated using the data from the February and May
2007 proficiency test (PT) rounds.  The PT samples in these two studies
served as good surrogates for OPR samples, because they involved spiking
Cryptosporidium oocysts into reagent water rather than a matrix.  The
data set is summarized in Table 1.  This updated minimum criterion of
22% recovery provides a better assessment of routine laboratory
performance than the original value for the following reasons: 1) the
data set is more current and is based on more samples (a total of 333);
2) 52 more laboratories are included in the data set; 3) data were
generated using the 2005 version of Method 1623, which is the required
version for the Long Term 2 Enhanced Surface Water Treatment Rule
(LT2ESWTR); 4) data were generated using filters currently used to
analyze LT2ESWTR samples rather than those filters used originally; and
5) the number of oocysts spiked into the samples was unknown to the
laboratories.

Table 1.  Summary of Original and Updated OPR Values

	Original Value	Updated Value

Criterion	11%	22%

Data Set	ICRSS data generated in 1999 and 2000	PT data generated during
the February and May 2007 rounds

Number of Laboratories	6	58

Number of Samples	293	333

Method Version Used	1999 version of Method 1623 and Method 1622	2005
version of Method 1623

Filter currently in use	No	yes

Blind vs. Unblind	Unblind	Blind

Statistical analysis	Estimation of variance within & between labs
Estimation of variance within & between labs

Details of the Calculation of OPR Minimum Recovery Criterion

To calculate the OPR recovery criterion, estimates of variance
attributable to four different sources were calculated: variability
between laboratories, variability between PT rounds, variability between
laboratory-and-round (i.e., attributable to an interaction between round
and laboratory), and variability within round and laboratory (i.e.,
analytical variability).  There were a few laboratories that performed
PT analyses using multiple filters.  For the purpose of these
calculations, the two sets of analyses were two separate entities; in
other words, the two lab/filter combinations were treated as two
different laboratories.  The different variance components were
calculated using PROC MIXED from SAS version 8 using the maximum
likelihood method of estimation on recovery data.  Details on the
maximum likelihood estimation can be found in SAS/STAT User’s Guide.

Estimates of between laboratory variance, between round variance,
between laboratory-and-round variance, and within laboratory-and-round
variance were labeled s2L, s2R, s2LR and s2w, respectively.  The
combined standard deviation (sc) is:

 

	Where:

 	 = average number of laboratory/filter combinations per round

  =   average number of rounds per laboratory/filter combination

		n = 	number of replicates per laboratory/filter and round (3)

		C = 	total number of laboratories/filters/rounds 

		nT =	total number of replicates over all labs/filters/rounds

Upper and lower recovery limits for OPR samples were then calculated as:

 	

		Where: 

	 	Xmean =	the mean recovery of all samples

		df is calculated using Satterthwaite’s estimate as given below:

 

Comparison of OPR Minimum Recovery Limit to PT OPRs

OPR data that were submitted in conjunction with Cryptosporidium PT
samples were compared to the newly calculated OPR minimum recovery
limit.  When applying the minimum percent recovery criterion of 22% to
the 1125 OPR samples submitted between September 2002 and October 2008,
41 samples  (3.6%) did not meet the criterion.  This comparison
validates the use of the new criterion, because it demonstrates that
laboratories can routinely meet the 22% recovery criterion.

 SAS Institute Inc. 1994.  SAS/STAT User’s Guide, Volume 2,
GLM-VARCOMP. Version 6, 4th Edition, June 1994.

 PAGE   

 PAGE   3