Document ID: EPA-HQ-OW-2010-0192-0038
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2010-08-23T04:00Z

Peer Review of the Method 1668A Interlaboratory Validation Study 

		

April 2010



  SEQ CHAPTER \h \r 1 U.S. Environmental Protection Agency

Office of Water (4303T)

1200 Pennsylvania Avenue, NW

Washington, DC  20460

EPA 820-R-10-007

Table of Contents

  TOC \o "1-2" \h \z    HYPERLINK \l "_Toc258826776"  Disclaimer	 
PAGEREF _Toc258826776 \h  iv  

  HYPERLINK \l "_Toc258826780"  Introduction	  PAGEREF _Toc258826780 \h 
1  

  HYPERLINK \l "_Toc258826781"  EPA’s Charge to the first two Peer
Reviewers	  PAGEREF _Toc258826781 \h  2  

  HYPERLINK \l "_Toc258826782"  EPA’s Charge to the third Peer
Reviewer	  PAGEREF _Toc258826782 \h  3  

  HYPERLINK \l "_Toc258826784"  Peer Review	  PAGEREF _Toc258826784 \h 
5  

  HYPERLINK \l "_Toc258826785"  Issues raised by both reviewers and
EPA’s responses	  PAGEREF _Toc258826785 \h  5  

  HYPERLINK \l "_Toc258826786"  Peer review comments froM m ichael C.
Newman	  PAGEREF _Toc258826786 \h  6  

  HYPERLINK \l "_Toc258826791"  Peer review comments from CHARLES S,
WONG	  PAGEREF _Toc258826791 \h  17  

PEER REVIEW COMMENTS FROM   HYPERLINK \l "_Toc258826798"  Kurunthachalam
kannan	  PAGEREF _Toc258826798 \h  21  

ORIGINAL   HYPERLINK \l "_Toc258826799"  Peer Review Comments	  PAGEREF
_Toc258826799 \h  34  

 Disclaimer   

Mention of trade names or commercial products does not constitute
endorsement or recommendation for use.

Contacts

The first two peer reviews in this document were conducted under EPA
Contract # 68-C-02-091 by Versar, Inc., 6850 Versar Center, Springfield,
Virginia 22151. The third and final peer review in this document was
conducted under Contract No. EP-C-07-059 by Eastern Research Group, Inc.
110 Hartwell Avenue. Lexington, MA 02421. Please direct your questions
concerning this document to: 

Clean Water Act methods Group, Brian Englert or Richard Reding

Engineering & Analytical Support Branch

Engineering and Analysis Division (4303T) 

Office of Science and Technology, Office of Water

U.S. Environmental Protection Agency

1200 Pennsylvania Avenue NW

Washington, DC 20460

  HYPERLINK "mailto:OSTCWAMethods@epa.gov"  OSTCWAMethods@epa.gov ,  
HYPERLINK "mailto:englert.brian@epa.gov"  englert.brian@epa.gov  or  
HYPERLINK "reding.richard@epa.gov"  reding.richard@epa.gov 

Introduction

This document contains the comments and EPA’s responses to comments of
a peer review of the Method 1668A Interlaboratory Validation Study
Report. In 2007, a contractor obtained three independent peer reviewers
and conducted this review for EPA using EPA’s Science Policy Council
Peer Review Handbook (EPA/100/B-06/002, 3rd Edition, the “Peer Review
Handbook”).

For the review, the contractor sent the Study Report to each reviewer
with a “charge.” Two reviewers provided written comments and
suggestions for improvement of the Study Report on the Method 1668A
Interlaboratory Validation Study. After a third reviewer dropped out for
personal reasons, EPA solicited a third reviewer in April of 2010. 

In our responses to peer reviewer comments and suggestions, we either
accept a suggestion or provide an explanation why the suggestion was not
accepted.  Based on the suggestions and on further data gathered, EPA
has revised Method 1668A and the “Method 1668A Interlaboratory
Validation Study Report” (March 2010, EPA-820-R-10-004.)  This Study
Report and Method 1668C are posted on the CWA methods website at   
HYPERLINK "http://www.epa.gov/waterscience/methods/method/other.html" 
http://www.epa.gov/waterscience/methods/method/other.html .

EPA’s Charge to the first two Peer Reviewers

EPA developed the following charge for the first two peer reviewers:

You are requested to peer review “Report of the Method 1668A
Interlaboratory Validation Study”. Please prepare a summary of
comments based on the EPA specified Charge.  The Comments Summary should
include a cover page with the title of the document, your name and
affiliation and comments organized by the format described in the cover
letter.  

Note: You may use this draft to become familiar with Method 1668A
Interlaboratory Validation Study, but please do not distribute it.  We
shall provide you with a revised copy after changes resulting from
comments have been incorporated. 

EPA Method 1668A was developed in the late 1990s and a draft version was
peer reviewed in 1999.  Results of that peer review were published in
February, 2000, in Peer Review of Draft EPA Method 1668, Revision A:
Chlorinated Biphenyl Congeners in Water, Soil, Sediment, and Tissue by
HRGC/HRMS. Draft EPA Method 1668A was revised as a result of the peer
review and the revised version, dated December 1999 (EPA-821-R-00-002)
was published in January of 2000.

After publication of the December 1999 draft of EPA Method 1668A, EPA
distributed the method extensively and planned inter-laboratory
validation.  In preparation for the inter-laboratory validation, EPA
updated, revised, and released Method 1668A on August 20, 2003.  A copy
of that revision is included with this letter.  The interlaboratory
validation study was conducted in 2003 - 2004.  Results of the study
were analyzed and the enclosed Report of the Method 1668A
Interlaboratory Validation Study was drafted in 2006.  The major result
of the interlaboratory study is a revision to the quality control
acceptance criteria (QCAC) in EPA Method 1668A based on interlaboratory
study results.  These QCAC are performance specifications within which
Method 1668A must be operated and performance yardsticks against which a
method modification must measured.

EPA received comments as a result of the interlaboratory study and other
comments between release of the August 20, 2003, revision and the
present.  EPA has included with this letter a list of changes that EPA
plans to make to EPA Method 1668A based on the revised QCAC and comments
received.  These changes are listed in the document titled:  Changes to
be made to EPA Method 1668A from the 8/20/03 revision based on the
Method 1668A interlaboratory validation study and comments and
corrections received from laboratories, included with this letter. 
Because of changes to Method 1668A, primarily as a result of the revised
QCAC, and based on your peer review, EPA plans to revise EPA Method 1668
to Revision B (EPA Method 1668B).  In your review, please respond to the
following questions:

1. 	Please review EPA's Report of the Method 1668A Interlaboratory
Validation Study for the objectives of the study, the soundness of the
technical approach, and the results produced.

2. 	Please render an opinion as to suitability for use of the data from
the study for revision of the QCAC in Method 1668 to make it Method
1668B. 

3. 	If you have a suggestion for an improvement of the interlaboratory
study report, please provide suggested text by section number.

4. 	If you believe that a given item should be clarified, please provide
the clarification by section number or define the clarification needed
to a sufficient level of detail that we can make the necessary changes. 

5. 	Please point out errors or omissions.

EPA’s Charge to the third Peer Reviewer

EPA developed the following charge for the third and final peer
reviewer:

BACKGROUND

EPA Method 1668A was developed in the late 1990s and a draft version was
peer reviewed in 1999.  Results of that peer review were published in
February, 2000, in “Peer Review of Draft EPA Method 1668, Revision A:
Chlorinated Biphenyl Congeners in Water, Soil, Sediment, and Tissue by
HRGC/HRMS.”  A copy of the peer-review report is included.  Draft EPA
Method 1668A was revised as a result of the peer review and the revised
version, dated December 1999 (EPA-821-R-00-002) and was published in
January of 2000.

After publication of the December 1999 draft of EPA Method 1668A, EPA
distributed the method extensively and planned an inter-laboratory
validation.  In preparation for the inter-laboratory validation, EPA
updated, revised, and released Method 1668A on August 20, 2003.  The
interlaboratory validation study was conducted in 2003-2004 to validate
the performance of EPA Method 1668A in municipal wastewater, fish
tissue, and biosolids matrices.  EPA received comments as a result of
the interlaboratory study and other comments after the release of the
August 20, 2003 revision of 1668A.  Results of the study were analyzed
and the “Report of the Method 1668A Interlaboratory Validation Study
was drafted in 2006 and peer reviewed in 2008.”  The major result of
the interlaboratory study is a revision to the quality control
acceptance criteria (QAQC) in EPA Method 1668A based on interlaboratory
study results.  

EPA used the results of the study to evaluate and revise Method 1668A
quality control (QC) acceptance criteria for initial precision and
recovery, ongoing precision and recovery, and labeled compound recovery
from real world samples.  EPA subsequently revised 1668A to 1668B based
on the Interlaboratory study, and comments received on 1668A. EPA
published 1668B with a list of changes in November 2008. These interlab
criteria (Table 5-1 of the study report) replace the single-lab
criteria, and are published in Table 6 of the revised working draft
version of PCB-congener method, EPA Method 1668B (Method VIII).  

“Method 1668A Interlaboratory Validation Study Report,” presents the
results of EPA’s interlaboratory validation of EPA Method 1668A  
Chlorinated Biphenyl Congeners in Water, Soil, Sediment, and Tissue by
HRGC/HRMS.  

Objectives of this study were to 1) characterize the performance of
Method 1668A in multiple laboratories and matrices, and 2) evaluate and,
if appropriate, revise the QC acceptance criteria in the method. 

EPA designed the Method 1668A Interlaboratory Validation Study in
accordance with guidelines published by EPA and ASTM International
(ASTM). These guidelines recommend a minimum of six complete data sets
for evaluation of a method. To allow for some loss of data due to error,
lost samples, outlier removal, or other unforeseen causes, EPA included
14 participant laboratories in the study.  The study design and report
are detailed in the document, “Method 1668A Interlaboratory Validation
Study Report.”

The QC acceptance criteria developed in the interlaboratory method
validation study did not allow the upper recovery limit for some
congeners to be above 100 percent.  The QC acceptance criteria were
recently revised based on data from the interlaboratory study and data
from two laboratories with extensive experience in use of Method 1668A.
Data were provided by TestAmerica, Knoxville, Tennessee and AXYS
Analytical Services, Ltd., Sidney, BC, Canada and are included in the
attached excel file. Method 1668B was subsequently revised. A draft
version of that as yet unpublished revision to 1668B is attached (See
Method VIII). The revisions to that method are summarized in the
introduction of the method.  The addendum describing the revised QC
acceptance criteria, “Addendum to the Method 1668A Interlaboratory
Validation Study Report Draft, August 2009” is included in this
review. Data supporting the addendum can be found in the file,
“OPRresults_Labs1&2.”

EPA peer reviewed the “Method 1668A Interlaboratory Validation Study
Report” in late 2007 but was unable to obtain three peer reviews and
is attempting to obtain a third peer review now.

CHARGE QUESTIONS

Please refer to the Review Guidelines and Formatting Instructions found
on pages 4 and 5 of this Charge.

Please review the attached documents “Method 1668A Interlaboratory
Validation Study Report” and “Addendum to the Method 1668A
Interlaboratory Validation Study Report Draft, August 2009.” In your
review, please respond to the following questions:

1. 	Please review EPA's Report of the Method 1668A Interlaboratory
Validation Study and its addendum. Comment on the objectives of the
study, the soundness of the technical approach, and the results
produced.

2. 	Please render an opinion as to suitability for use of the data from
the study and the addendum for revision of the QCAC in Method 1668 to
make it Method 1668B. 

3. 	If you have a suggestion for an improvement of the interlaboratory
study report or addendum, please provide suggested text by section
number.

4. 	If you believe that a given item should be clarified, please provide
the clarification by section number or define the clarification needed
to a sufficient level of detail that we can make the necessary changes. 

5. 	Please point out errors or omissions.

6. 	Please provide any additional comments you would like to make
regarding the documents under review.

Peer Review

Dr. Michael Newman, Dr. Charles Wong and Dr. Kurunthachalam Kannan
peer-reviewed the Study Report.  This document has three parts.  The
first section is EPA’s responses to some common issues form more than
one reviewer.  The second section is EPA’s responses to specific
comments from each reviewer. The last section contains the complete
review of each reviewer. 

Section One:  Selected Issues raised by more than one reviewer and
EPA’s responses

Comment: 	It would have been useful if the laboratories were asked to
provide measurement data on QC Check samples (Method 1668A, Section
7.15) from standardized reference materials (e.g., NIST SRMs or Canadian
NRC SRMs).

Response:  	EPA asked laboratories to analyze spiked and typical Clean
Water Act matrices, i.e., wastewater, tissue, and biosolids (sewage
sludge.) These samples better represented the intended application of
this method for environmental monitoring. 

In addition, laboratories used to using the available QC check samples
prepared in reagent waters or solvents might be overly familiar with
their content.  Method 1668A has been available for use by laboratories
since 2000. Because it contains the suggestion for a QC Check sample in
Section 7.15, many labs had been using a NIST SRM or NRC CRM, and might
recognize the sample from having analyzed it previously. 

Comment:  	Laboratory participation was disappointing and this was the
source of numerous issues.  For example, six of 14 laboratories declined
to participate in the biosolids analysis.

Response:  	The high failure-to-report rate is attributable to (1) the
extensive effort needed to set up the extraction, concentration,
cleanup, and HRGC/HRMS systems for determination of 209 PCB congeners,
(2) a lack of familiarity with the biosolids matrix, and (3) the
voluntary nature of participation.

Comment: 	Outlier deletion, as discussed in Grubbs' paper and many
others, should be done very thoughtfully according to clear rules.  For
example, the "outliers" detected based on the assumption of a normal
distribution might come from a population more appropriately described
as lognormal.

Response:  	Outlier tests were applied to both blind sample data (as
presented in Table 4-1 of the report) and QC data.  In both cases,
Grubbs' test was applied conservatively, following guidelines for
maximum outlier removal as specified by ASTM D2777-98.

Comment:  	Round all numbers to 3 significant figures.

Response:  	We rounded all results to three significant figures maximum
in the final report.

Comment:  	On page 4, paragraph 2, please consider replacing "properly
homogenized" with "homogenized."

Response:  	We have removed "properly."

Comment:  	It is extremely important in judging the generalizability of
the report conclusions that the laboratory participation in generating
the final QC criteria is very clearly discussed.  At present, it is not
easy to follow.

Response:  	Where possible, we revised the report to clarify this
discussion. EPA discusses exclusion of a laboratory's data is in Section
3 of the report. EPA generally rejected a laboratory because the
laboratory submitted incomplete data sets, insufficient raw data to
verify a laboratory's results, or did not follow the method correctly.

Comments:  	Move the definition of "Youden pair" from Section 2.5.1 to
its first use in Section 2.4.

Response:   	We moved the definition to its first use.

 

Section Two:  Comments by each reviewer and EPA’s responses

Peer review comments froM michael C. Newman

 

Name:  Michael C. Newman, Ph.D.

Affiliation:  121 York Point Dr., Seaford, VA, also Professor at the
College of William and Mary, Virginia Institute of Marine Science

Date: 	September 18, 2007

Comments on:  Method 1668A Interlaboratory Validation Study

I.	GENERAL IMPRESSIONS

Comment 1:   	Overall, this Report of the Method 1668A Interlaboratory
Validation Study is a sufficiently accurate, clear, and thorough
analysis discussing the results of a very difficult task.  The study
objectives were to (1) characterize the performance of Method 1668A for
multiple laboratories and matrices (wastewater, biosolid, fish tissue),
and (2) evaluate and possibly revise the QC acceptance criteria. These
objectives were met with some qualification. It is important to
emphasize prior to discussing the specific qualifications that this
effort does appear to be generally well-intended, -planned and
-interpreted.

Response:   	No response needed.

Comment 2:   	Some challenges to applying the results do exist.  The
laboratory participation was disappointing and this was the source of
numerous issues.  For example, six of 14 laboratories declined to
participate in the biosolids analysis.  An additional four produced
unusable numbers, resulting in only four of 14 potential sets of results
being applicable to the task at hand. For wastewater and fish tissue,
only six laboratories produced the data used in this exercise.  Four
laboratories successfully analyzed all three sample types and,
therefore, have a very strong influence on the results.  The candidate
laboratories included government and private organizations which might
be expected to have differences in response; however, it is unclear from
the report if the final subset was biased toward one group or another.
The very high failure-to-report rate for this process suggests that
failure to achieve useful analyses with 1668A warrants more discussion.
Was the high failure rate normal for such voluntary validation exercises
or a feature to be anticipated for Method 1668A as currently written?

Response:   	The high failure-to-report rate is attributable to (1) the
extensive effort needed to set up the extraction, concentration,
cleanup, and HRGC/HRMS systems for determination of 209 PCB congeners,
(2) a lack of familiarity with the biosolids matrix, and (3) the
voluntary nature of participation. When participating voluntarily, a
laboratory has no financial incentive to complete the work and submit a
complete set of results. Thus, when  the laboratory encounters
difficulty, as with the biosolids matrix, the laboratory may simply stop
 the analysis rather than time-consuming corrective actions.

Comment 3:   	Some laboratories deviated from 1668A for some analyses.
Also, outlier tests that require extremely thoughtful application were
applied to cull away some data.  Together, these features do not permit
as high confidence in the study’s generalizations as might be desired.

Response:   	Within limits, laboratories were allowed to deviate from
prescribed procedures under the performance-based provisions of Method
1668A.  EPA applied the same outlier removal procedures that we have
applied to EPA wastewater methods for the past approximately 20 years.
These procedures are based on ASTM D2777-98, which ensures that outlier
removal be limited to a small percentage of the data, and therefore the
removal would have a small effect on the study results. These outlier
procedures remove true outliers yet retain valid data. Additionally,
some of the QC sample results removed as outliers were “better”
(i.e., closer to 100% recovery) than the remaining results, which means
that their removal would not portray method performance as
unrealistically good. 

Comment 4:  	The studies conclusions are not unequivocal at this point.
The resulting, central concern can be articulated with a quote from
Stuart’s Basic Ideas of Scientific Sampling (1968): “the credentials
of a sample are not only important for the interpretation of results,
but are in principle the only information of value we can have in this
respect.”

Response: 	Since EPA completed this study and peer review, EPA has
obtained additional data that demonstrate that laboratories can overcome
blank contamination, and otherwise perform Method 1668 under tighter
specifications than we observed in this study. 

Comment 5:  	My general conclusion is that the results are adequate for
meeting the study objectives but should be augmented in the near future
in order to enhance confidence in the general validity of the
conclusions.  Detailed below are specific examples from within the
report and associated recommendations.

Response: 	EPA has since obtained additional data that demonstrate that
laboratories can perform Method 1668 under tighter specifications than
we observed in this study. Subsequent to this study, EPA has also
obtained MDL data from several contract laboratories.  In a study of
data comparability between two laboratories on samples collected from
the Passaic River in New Jersey, in which 151 PCB congeners were
identified and measured, accuracy as measured by analysis of a NIST SRM
was 15% or better.  Recoveries of the PCB congeners ranged from 90% to
124% and averaged 105%; precision ranged from 4.2 to 23%.

II. 	RESPONSE TO CHARGE QUESTIONS

1. 	Please review the EPA's Report of the Method 1668A Interlaboratory
Validation Study for the objectives of the study, the soundness of the
technical approach, and the results produced.

Comment 6: 	The Interlaboratory Validation Study attempts to
characterize the performance of Method 1668A in multiple laboratories
and matrices (wastewater, biosolid, fish tissue), and to evaluate and
possibly revise the QC acceptance criteria.  As stated above, my
overarching opinion is that the study met these goals but further work
is required to enhance confidence in the generality of the conclusions. 
The goals for this complicated method were extremely challenging so the
implied incremental generation of sufficient characterization/validation
information is, in my opinion, reasonable.  My suggestion that more work
is required is not an oblique inference about the work quality of the
study leaders.  This was a very difficult task.

Response:   	We appreciate your understanding of the difficulty of
conducting the study and the analyses.

2. 	Please render an opinion as to suitability for use of the data from
the study for revision of the QCAC in Method 1668 to make it Method
1668B. 

Comment 7:  	Yes.  The study data are adequate, although lacking in some
ways, for revision of the QC in Method 1668.

Response:   	EPA has since obtained additional data and revised the QC
data in the 2009 version of Method 1668.

3. 	If you have a suggestion for an improvement, please provide
suggested text by section number.

Comment 8:  	A wide range of suggestions and insights will be addressed
below regarding the improvement of the study or existing study
presentation.

Response:   	Our responses to this group of comments follow.

Comment 9:   	Page 6, Top paragraph and first full paragraph  - This is
the first place in the report that the issue of sample homogeneity
emerged.  It is also discussed in Section 4.1.  Described here is the
process in which a “sample processing laboratory was required to
perform background and homogeneity analyses of both the biosolids and
tissue matrices.”  The text does not convince me that the samples were
“homogeneous” in a strict sense relative to the analytes.  For
example, some fatty tissues might contain more analyte than others and
similar-sized pieces of fatty tissues might not be homogeneously
distributed in a well-mixed homogenate.  The biosolid to which sand was
added might experience density-driven sample component segregation
(i.e., sinking sand) so that the sample aliquot analyzed by the
volunteer laboratory might not have been homogeneous (well-mixed?)
relative to the analyte.  There are very specific methods of determining
this sample quality (well- or poorly-mixed) yet they are not applied by
the sample processing laboratory.  It is not clear how this might or
might not have contributed to differences in results among laboratories.
 This concern is also inherently linked to sample aliquot size which
1668 prescribes versus and the actual sample sizes used by the volunteer
laboratories. (I assume that the recommendations of sample sizes were
based only on anticipated analyte concentrations in typical samples and
the mechanics of the sample processing.)

				The point being made above can be illustrated easily using the
conventional explanation as depicted in the figure to the left.  The
analyte (dot) is homogeneously distributed in this figure(s top panel;
therefore, sample aliquot size illustrated in the figure as different
size boxes will not influence the measured analyte concentration in an
analyzed aliquot. (Homogeneity is a material quality in which the
analyte is distributed in a completely homogeneous manner throughout the
material.)   In the bottom panel, a heterogeneous material is depicted
in which sample aliquot size becomes critical depending on whether the
heterogeneous material is well- or poorly-mixed.  If sample aliquot
sizes are too small for a poorly-mixed, heterogeneous material (small
squares in figure bottom panel), variation among reported analyte
concentrations will be higher than that derived with aliquot sizes of
sufficient size (large square in the bottom panel) (Newman 1995).  To
conclude that a heterogeneous sample is sufficiently well-mixed to
permit use of a specified aliquot size (e.g., 20 g of fish tissue)
requires some conventional calculations.  The homogeneity analyses
described in this report do not include the associated statistics
permitting such decisions.  This is especially relevant to the biosolid
and tissue samples, and with sufficiently high solids content, also to
wastewater samples.  The pertinent approach to this problem and
associated statistical computations are described in classic papers by
Visman (1969), Ingamells (1973) Ingamells and Switzer (1974), and
Wallace and Kratochil (1987).  

Response:   	We are confident that the samples were homogeneous.
although using the standard that you have developed, those that you
reference, and existing data, it is impossible to prove.  The
sample-processing laboratory has extensive experience in sample
homogenization, and homogenization procedures are provided in Section
11.7 of Method 1668A.

Comment 10:  	Page 8, Top Bullets  - The laboratories were told to
report all “>nm” values as quantified concentrations and the
statistical analysis of these measured values was treated such that all
“>nm” values possessed the same quality of measurement.  It is
incorrect to assume that all values above “nm” had the same
measurement quality (e.g., Keith et al. (1983)).  The relative
uncertainty of a measured value can be estimated relative to the
standard deviation of the analytical process baseline noise (See also
figure on page 6),

 

where Z= Z-statistic value for a confidence level of ( (conventionally
95%), and N is the measured signal expressed in units of multiples of
the standard deviation of the baseline noise, e.g., N=3 and 10 are
associated with the signal at the detection limit and limit of
quantification, respectively.  Interpretation of the study results is
very difficult if this fact is not addressed in the discussions and
shown in graphs of results for the various samples and sample types. An
example of this will be discussed below relative to the figures in the
report.

Response:   	An analysis of this type would require knowledge of the
error function for the analytical system in each laboratory in the
interlab study. Differences in sample amounts and volumes, and final
extract volumes, used by the laboratories, as these amounts and volumes
would shift the results upward or downward on the concentration-error
function  would be further complicate the suggested analysis. Therefore,
this type of analysis is impractical and beyond the scope of the study.

Comment 11:   Page 12, First paragraph after bullet list and also last
sentence  - Here and elsewhere, my assumption is that the study results
are intended to provide measurement quality characteristics anticipated
for “in-control” analyses of the specified sample types by a
laboratory represented by the initial 14 laboratories. It is critical
that the population for which these final QC criteria are generated be
clearly defined given the variations in reporting, method application,
and data selected for final QC calculations.   

Response:   	EPA intends that any single laboratory using Method 1668,
not a hypothetical population derived from the laboratories in the
study, will use the final QC acceptance criteria. Our experience is that
QC acceptance criteria generated from inter-laboratory data are
approximately a factor of 2 broader than the same criteria generated
from data from a single laboratory. Note that EPA derived the QC
acceptance criteria for the recovery of labeled compounds from results
from all three matrices in the study. And EPA derived the QC acceptance
criteria for IPR and OPR sample recovery from results of spiked water
and solid matrices. Therefore, although the reduced data set may narrow
the criteria somewhat, the QC acceptance criteria are sufficiently
generous that future laboratory will not have difficulty meeting them.

Comment 12:   Here and elsewhere, Grubbs’ single outlier test (Grubbs
1969) is applied to cull away apparent outliers. Outlier deletion, as
discussed in his paper and many others, should be done very thoughtfully
according to clear rules. For example, the “outliers” detected based
on the assumption of a normal distribution might come from a population
more appropriately described as lognormal.  The “outlier” would be
falsely removed based on the assumption of the wrong distribution.  The
rules for handling outliers in Grubbs (1969) and other papers should be
considered prior to applying any outlier tests.  The results of such
application should then be described in the report.  Classic papers that
might be useful are Anscombe (1960) and Rosner (1983).  As multiple
outliers might be masking each other in these data sets, Rosner’s
method (1983) for multiple outliers might be useful.  Personally, I
think that Woodworth’s (2004) classification of outliers and the
associated approach based on inter-quartile ranges is more relevant as
it is more open to intelligent application than the above
hypothesis-based, parametric tests.  This point will be repeated later
in discussing the absence of quartile (Q1 and Q3) values in tables of
summary statistics.

Response:   	Outlier tests were applied to both blind sample data (as
presented in Table 4-1 of the report) and QC data. In both cases, EPA
applied the Grubbs test conservatively, following guidelines for maximum
outlier removal as specified by ASTM D2777-98. In the case of sample
data, these results were only used to calculate summary statistics of
the sample results by chlorination level, and therefore outlier removal
would not strongly affect conclusions about method performance. For QC
data, outlier removal was limited to one result per congener. For
labeled compound samples, there were a greater number of results, and a
maximum of two results were removed per congener. As stated elsewhere,
EPA had no strong evidence that these data did not follow a normal
distribution, and therefore EPA’s use of the Grubbs test is
appropriate. We are familiar with outlier tests based on the
interquartile range, but these tests tend to classify an unrealistically
large percentage of data as outliers.

Comment 13:   Page 13, Section 4.2.1, line 4 - The “homogeneity” was
found acceptable. Respectfully, my guess would be that this statement is
correct; however, I disagree that enough information was extracted to
make a definitive statement.  I base this disagreement on the material
already presented above.

Response:   	We agree that the samples could have been homogeneous but
we lack sufficient data to prove it.  At this point, we must assume
homogeneity because further testing is not possible.

Comment 14:	Page 14, Table 4-1- Here and elsewhere, it would be very
helpful to have the first and third quartile values (Q1 and Q3) or,
minimally, the inter-quartile range (|Q3-Q1|) in the table. One
immediate use would be to apply inter-quartile range-based methods to
explore potential outliers or outlying laboratories.   The inclusion of
these statistics is highly recommended.

Response:   	Application of these ranges would alter the results or
conclusions significantly or appreciably alter the QC acceptance
criteria to justify the effort to do so.

Comment 15:   As an aside, the differences between means and medians for
congeners with 3 to 8 chlorination levels (biosolids and tissues)
suggest a skewed sample set. Are these data skewed for any reason or is
this simply a spurious trend in the tabulated data?

Response:   	We agree that the distribution of results presented in
Table 4-1 were often skewed. This was most likely due to the difference
in prevalence and coelution of the various congeners within each
chlorination level. This skewness was not observed with a single
congener in the QC sample data, and therefore this skewness would not
reflect a departure from the normality assumption used in the developing
QC acceptance criteria.

Comment 16:	Page 29, Last line of text  - I generally agree but feel
that a less definitive, and more conditional, conclusion can be reached.

Response:   	Because you did not provide an alternative conclusion, we
continue to conclude that QC acceptance criteria developed from
multi-laboratory data more accurately reflect the performance of an
analytical method than QC acceptance criteria developed from
single-laboratory data. The QC acceptance criteria in Method 1668A were
estimates resulting from testing the method in a single laboratory. The
acceptance criteria in revise Method 1668 are based on data from several
laboratories. 

Comment 17:	Appendix A. Section 1.0., 95% Confidence Intervals - The
section begins by describing conventional, parametric calculations for
all congeners in all matrices at all concentrations.  Was the assumption
of normality tested and met, or found to be unimportant?  Some
discussion seems warranted.

Response:   	While a statistical test of normality is not extremely
powerful for a small set of data, the spiked QC sample results for
nearly all congeners passed the Shapiro-Wilk test at alpha = 0.05. Those
that did not pass generally had one or two extreme results, or
systematic differences in recovery between matrices. A normal
distribution of QC spike recoveries is also consistent with experience
with past environmental studies. While results of environmental samples
tend to follow a skewed distribution, percent recoveries of QC samples
spiked at approximately the same concentration level tend to follow a
more symmetric distribution.

Comment 18:	Appendix A Section 1.0, Application of Grubbs’ outlier
test  - Again, outlier tests should be used very carefully and
interpretation of any consequent data sets based on truncated
distribution statistics (see Grubbs (1969) and other sources for
discussion).

Response:   	Outlier tests were applied to both blind sample data (as
presented in Table 4-1 of the report) and QC data.  In both cases, EPA
applied the Grubbs’ test conservatively, following guidelines for
maximum outlier removal as specified by ASTM D2777-98. In the case of
sample data, these results were only used to calculate summary
statistics of the sample results by chlorination level, and therefore
outlier removal would not strongly affect conclusions about method
performance. For IPR and OPR data, outlier removal was limited to one
result per congener. For labeled compound data, there were more results,
and a maximum of two results were removed per congener. As stated
elsewhere, EPA had no strong evidence that these data did not follow a
normal distribution, and therefore EPA’s use of the Grubbs’ test is
appropriate.   

Comment 19:  	As a simple example, what is the n in the equation given
in Section 2.0?  This is not as easy a question to answer as might be
suspected since some observations were thrown out.

Response:   	For most analytes, the n in the equation given in Section
2.0 was 24, as stated in Section 5.5 on page 27 of the report. Two
outliers each were removed for labeled congeners 111 and 178, so n would
be 22 for these two congeners. 

Comment 20:	Also, was the assumption of a normal distribution (upon
which Grubbs’ test depends) appropriate?

Response:   	The assumption of normality was appropriate because the
testing was performed in the center of the analytical range. See also
the response to the comment on Appendix A. Section 1.0., 95% Confidence
Intervals (Comment 17), above.

Comment 21:	 Finally, what Type I error rate was selected?  Likely, the
typical 0.05 was used yet one could argue for a more conservative error
rate.

Response:   	Yes, EPA selected a Type I error rate of 0.05. A more
conservative Type I error rate would yield QC inappropriately wide
acceptance criteria, and would not be useful in detecting laboratories
that are not performing the method appropriately or are out of control.

4. 	If you believe that a given item should be clarified, please provide
the clarification by section number or define the clarification needed
to a sufficient level of detail that we can make the necessary changes. 

Comment 22:  	A wide range of issues will be addressed below as
suggestions for clarifying presentation.

Response:   	Our responses to this group of comments follow.

Comment 23:	Section 1.2, Line 3 - Change “seen” to “seven”?

Response:   	We have corrected this.

Comment 24:	 Page 4, Paragraph 2 - Please consider replacing “properly
homogenized” with “homogenized.”  The reasons are given above for
removing the vague word from this sentence. One could easily argue that
the samples were not shown to be “properly homogenized.”

Response:   	We have removed "properly."

Comment 25: 	Page 5, Paragraph 2 of Section 2.4 - The term, “Youden
pairs” is first used here but not defined until Section 2.5.1.  Please
move the definition up to the first use of the term.

Response:   	We have moved the definition to its first use.

Comment 26:	Table 2-1 (page 4) and Table 3-1 (page 12) - It is extremely
important in judging the generalizability of the report conclusions that
the laboratory participation in generating the final QC criteria is very
clearly discussed.  At present, it is not easy to follow.  Generally,
one starts with 14 laboratories (government and private) and ends up
with data from 4 to 6 labs.  It isn’t easy to judge how (1) withdrawal
of the various laboratories from providing results, (2) the study
author’s judgment about reported data usefulness, and (3) the final
data inclusion influences the statistical population of laboratories for
which inferences about QC are being made.  Several issues to be
addressed include the following. Are the QC conclusions relevant to only
the best or best equipped laboratories?  Are they representative of
primarily government laboratories, private laboratories, or both?  A
very clear and concise summary addressing this linchpin issue is needed.
 In developing this summary, discussion of the influence of outlier
rejection is also warranted.

Response:   	Where possible, we revised the report to clarify this
discussion. EPA discusses exclusion of a laboratory's data is in Section
3 of the report. EPA generally rejected a laboratory because the
laboratory submitted incomplete data sets, insufficient raw data to
verify a laboratory's results, or did not follow the method correctly.
Exclusion of these laboratories does not mean that the conclusions are
relevant only to the best laboratories, but to laboratories that perform
the method correctly. 

Comment 27:	Page 8, Section 2.7, first paragraph  - The requirements
were given for calibration curves but the laboratories interpreted the
requirements differently.  I saw no obvious reason for the
misinterpretation; however, some thought should be put into rewriting
the associated text so it is even clearer.  Also, relative to Section
2.7.1. and 2.7.2, were the consequences of any of the various deviations
quantifiable?  For example, the differences due to different aliquot
weights used by the various laboratories might be quantified with the
methods cited above for Page 6, Top paragraph and first full paragraph.

Response:   	We were unable to establish any clear relationship between
the various deviations and the results. The primary reason is that
neither we nor the laboratories linked a specific deviation to specific
data. If a laboratory chose to calibrate its analytical system different
from the calibration suggested in Method 1668A, or chose not to follow a
procedure, we had no way to force the laboratory to re-calibrate or to
repeat the analysis.

Comment 28:  	Page 9, Section 2.7.2 - Different problems were
encountered by different laboratories.  Different weight normalizations
were done by the different laboratories.  It isn’t clear to me how
this directly influenced statistical estimates and associated
inferences.  Also, the numerous deviations suggest that even more
clarity needs to be placed into 1668A.  Perhaps more discussion is
needed about what deviations should not be considered.

Response:   	In principle, we agree.  Although Method 1668A is clear
about the amount of material to be extracted and the final extract
volume, in revising the method we have provided more clarity about where
flexibility is allowed and not allowed in conducting the method.  We
control laboratories under contract to us, so these deviations would not
occur with our routine data gathering nor should they with others who
contract for lab work. In this interlab study, although we asked the
participants to follow Method 1668A, we had no control over the
deviations that these volunteers eventually chose. Therefore, we used
the data as provided except where we could justify finding it lacking,
inconsistent with other data, or outlying.

Comment 29: 	Page 11, Section 3, first paragraph - This section begins
by describing three laboratories that did not submit data. It is
important to know if non-reporters are considered a biased or random
subset of the population of possible responders. Was there a risk of
biased sampling of possible laboratories using this method as a result
of biased non-response?

Response:   	No. The laboratories that did not submit data were
government laboratories, but we have no evidence that their skill level
differs from other private or commercial laboratories.

Comment 30: 	Page 11, Section 3, fourth bullet  - How was this issue
handled during the analysis of results?  Would omission of an outlier
laboratory create a biased sample of laboratories?

Response:   	Results from the laboratory were not considered in the
analysis for the reasons given in this bullet in the report.  The
omission of this laboratory would not bias the results. We could not
question the laboratory about poor recoveries or other problems. In
addition, because the laboratory is no longer in business, the
laboratory is no longer part of the population of laboratories
performing this method.  This laboratory is not an "outlier laboratory."
We did not have any such laboratories because there were not enough
laboratories in the study with which to perform an outlying laboratory
test.

Comment 31: 	Page 15, Table 4-2 and all other tables - Univariate
statistics are applied throughout these tables.  Some graphs suggest
skewed distributions for some data.  Was the assumption of normality or
the robustness of conclusions to violations of this assumption assessed?

Response:   	Many of the univariate statistics and graphs combine
congeners, and as such the distribution of results would be affected by
differences in the prevalence of congeners in the environment and in
coelution. While a statistical test of normality is not extremely
powerful for a small set of data, the spiked QC sample results of most
congeners passed the Shapiro-Wilk test at alpha = 0.05. Those that did
not pass generally had one or two extreme results, or systematic
differences in recovery between matrices. A normal distribution of QC
spike recoveries is also consistent with experience with past
environmental studies. While results of environmental samples tend to
follow a skewed distribution, percent recoveries of QC samples spiked at
approximately the same level tend to follow a more symmetric
distribution.

Comment 32:  	Page 16, first lines - This discussion is too speculative
in my opinion.

Response:   	The sample-preparation lab prepared a solution containing
all of the congeners. The laboratory then lined up all of the sample
bottles, filled each with POTW effluent, and spiked an aliquot of the
solution into each bottle. The lab then analyzed two of the samples and
found all congeners to be present in the concentrations spiked (within
experimental error). The lab shipped the bottles to the participant
labs. The mean of the concentrations found by the participant labs for
the mono- and di- CB congeners was 3 and 54 %, respectively, whereas the
mean for the tri- through deca- CB congeners ranged between 68 and 96 %.
The most logical explanation is that the congeners were lost during
transit or when the bottles were opened at the participant laboratories.
Remember, EPA recovery corrected all results using the labeled compounds
as reference, and labeled compound recoveries were within the normal
range, so the losses were real and occurred prior to the time the
samples were spiked with labeled compounds at the participant labs.

Comment 33:	Page 17, Figure 4-1 (and also 4-3, 4-5, and 4-7) - As
dictated by the generally-accepted equation given above, the relative
uncertainty of a measurement will rapidly increase as concentration
decreases.  The following figure gives the general shape of such a
relationship. Depending on the positions of the detection limit (LOD in
this figure) and limit of quantitation (LOQ in this figure) (from Newman
(1995)), the variation in the y direction as a function of measured
concentration would be totally expected based on this relationship. 
Would it be possible to insert vertical lines at the LOD and LOQ in
these figures so a reader could judge this issue?  As a good example of
how this would help, it is difficult to decide if the explanation for
the Figure 4-5 outliers (Page 20, first few lines) is reasonable or
whether the variations at low concentrations (in lower part of the
Region of Less-Certain Quantitation? are what you would expect for any
measurement process.

Response:   	It would not be possible to insert lines at the LOD and/or
LOQ in Figure 4-1 or the other figures, because different participant
labs used different sample volumes and different final extract volumes.
Further, and most unfortunate, the congeners at low spike concentration
were also those at low chlorine level. Therefore, the apparent
resemblance to the Newman model is misleading.

Comment 34:  		Page 18, Figure 4-2 (and also 4.4, 4.6) - These figures
do not convey any useful information.  Relative Standard Deviations
(e.g., 4-3) are much more useful for conveying the associated
information (that variation can change with concentration).

Response:   	These figures can be useful in determining where in the
concentration scale absolute variability begins to increase linearly
with concentration.

Comment 35:  	Page 21, Table 4-4 - A simple mistake was made here such
that the “# pairs” column has two digits after the decimal point,
e.g., 32.00 pairs.

Response:   	We have corrected the number of pairs to a whole number.

Comment 36:	Page 29 - The conventional “This page intentionally
blank” statement is made but, in fact, there is text on the page.

Response:   	We have corrected  the offset between this page and the
previous page.

5. 	Please point out errors or omissions.

Comment 37: 	Please see comments above.

Response:   	We responded to these in the preceding section.

III. 	Specific Observations

[none submitted]

IV.	ADDITIONAL REFERENCES

1.  	Anscombe, F.J. 1960. Rejection of  outliers. Technometrics 2:
123-147.

2.  	Grubbs, F.E. 1969. Procedures for detecting outlying observations
in samples. Technometrics 11: 1-21.

3.  	Ingamells, C.O. 1974. New approaches to geochemical analysis and
sampling. Talanta 21: 141-155.

4.  	Ingamells, C.O. and P. Switzer. 1973. A proposed sampling constant
for use in geochemical analysis. Talanta 20: 547-568.

5.	Keith, L.H., W. Crummett, J. Deegan, Jr., R.A. Libby, J.K. Taylor,
and G. Wentler. 1983. Principles of environmental analysis. Anal. Chem.
55: 2210-2218.

6.	Newman, M.C. 1995. Quantitative Methods in Aquatic Ecotoxicology.
Lewis/CRC Publishers, Boca Raton, FL, pp. 426. 

7.	Rosner, B. 1983. Percentage points for a generalized ESD many-outlier
procedure. Technometrics 25: 165-172.

8.	Stuart, A. 1968. Basic Ideas of Scientific Sampling, Hafner
Publishing Co., New York, pp. 99.

9.	Visman, J. 1969. A general sampling theory. Mat. Res. Stand. 9:9-64.

10.	Wallace, D. and B. Kratochvil. 1987. Visman equations in the design
of sampling plans for chemical analysis of segregated bulk materials.
Anal. Chem. 59: 226-232.

11.	Woodworth, G.G. 2004. Biostatistics. A Bayesian Introduction. John
Wiley & Sons/

Peer review comments from

Charles S, Wong

 

Name:  	Charles S. Wong, Ph.D.

Affiliation:  	University of Alberta, Dept. of Chemistry

Date: 	September 18, 2007

Comments on:  	Method 1668A Interlaboratory Validation Study

I.  	GENERAL IMPRESSIONS

Comment 1:	This document is a synopsis of the interlaboratory validation
study intended to improve EPA method 1668A, a protocol for
congener-specific measurement of polychlorinated biphenyls (PCBs) in
water, soil, sediment, biosolids, and tissue by high resolution gas
chromatography/high resolution mass spectrometry.  Fourteen laboratories
were invited to participate, and samples as well as instructions and
required reporting information was provided to them.  Reported data was
compiled and summarized.

Comment 2:	The document provides a useful but succinct summary of the
interlaboratory validations very large dataset.  Sufficient detail is
provided on the background on Method 1668A and the methodology used to
invite participating laboratories to ascertain the effectiveness of the
validation study in characterizing measurement of PCBs in the samples
chosen.  Methods for selection of samples for the laboratories are clear
and reasonable, including explicit discussion of limitations for
logistical and technical purposes.  A clear and detailed discussion of
deviations from the required protocol is evident, so that the reader can
clearly understand limitations in the reported data.  For the most part,
the discussion of the usable data is clear and pertinent to the intended
goal of determining possible changes to the protocol and to acceptance
criteria.

Response:   	No response needed.

II. 	RESPONSE TO CHARGE QUESTIONS

1.	Please review the EPA’s Report of the Method 1668 Interlaboratory
Validation Study for the objectives of the study, the soundness of the
technical approach, and the results produced.

Comment 3:  	The objectives of the study are sound.  Should Method 1668A
be revised, and if so, what protocol changes and/or acceptance criteria
should be modified?  This can only be done with an interlaboratory
validation of a standard set of samples.

Comment 4:   	Sample selection is reasonable in my opinion.  Biosolids
and tissues were prepared by EPA at two distinct concentrations,
properly homogenized, and shipped to each laboratory with appropriate
safeguards against decompositions to ensure that that the integrity of
the samples upon receipt by the participating laboratory was maintained.
 While it would have been ideal to have wastewater samples with PCBs
shipped to the laboratories, it is reasonable to assume that testing of
Method 1668 capabilities would not be feasible.  Thus, providing
instructions to spike the wastewater effluent shipped to the laboratory
was reasonable.

Response:   	The wastewater samples were spiked with PCBs at a
sample-preparation laboratory. The participant laboratories spiked the
13C-labeled PCBs in their laboratory.

Comment 5:   	As previously stated, deviations from the method were
characterized in a succinct but meaningful manner.  This allowed the
flagging of suspect data as unusable for further purposes.  Many of the
deviations noted had details on the underlying rationale, e.g., problems
with cleanup of samples encountered by some of the laboratories. 
However, not all such deviations were detailed in this manner, which I
would have found useful.  It is troubling that there were sufficient
problems in sample processing, measurement, and even data reporting
(e.g., no calibration reporting from one laboratory???) that less than
half the data from the participating laboratories was usable (Table
3-1), as part of the point of having a large number of independent
participants was to assess the robustness of both the overall Method as
well as the validation for improving the Method.  That having been said,
I feel that the most problematic deviations were flagged unusable, and
that the usable data is sufficient for the intended purpose.

Response:   	You have verified our conclusions concerning problems
encountered with the samples and data reporting.

Comment 6:   	It would have been useful if the laboratories were asked
to provide measurement data on QC Check samples (Method 1668A, Section
7.15) from standardized reference materials (e.g., NIST SRMs or Canadian
NRC SRMs) in addition to the other QC data, e.g., OPR samples in reagent
water, and blanks.  This is important because as noted in the report,
PCB concentrations in the shipped samples were unknown, and having this
check would be useful.  If this was actually done, this was not clear
from the report.

Response:   	EPA asked laboratories to analyze spiked and typical Clean
Water Act matrices, i.e., wastewater, tissue, and biosolids (sewage
sludge.) These samples better represented the intended application of
this method for environmental monitoring. 

In addition, laboratories used to using the available QC check samples
prepared in reagent waters or solvents might be overly familiar with
their content.  Method 1668A has been available for use by laboratories
since 2000. Because it contains the suggestion for a QC Check sample in
Section 7.15, many labs had been using a NIST SRM or NRC CRM, and might
recognize the sample from having analyzed it previously. 

Comment 7:   	The reported results provided a good summary of the
variability observed with PCB measurement in the various matrices
supplied to the participating laboratories.  This information is most
useful for resetting IPR/OPR guidelines.   The summary and
recommendations appear reasonable, e.g., using average relative
responses under specific circumstances, noting poorer and more variable
recoveries for lower chlorinated congeners, and somewhat tighter
recovery windows particularly for heavier congeners.

Response:   	No response needed.

2.	Please render an opinion as to suitability for use of the data from
the study for revision of the QCAC in Method 1668 to make it Method
1668B.

Comment 8:  	I feel that this data, despite limitations noted above and
in the report, are of sufficient quality to revise the method for the
next version of this Method.

Response:   	We agree that EPA should move forward with the revised QC
acceptance criteria and other changes to Method 1668A.

3.  	If you have a suggestion for an improvement, please provide
suggested text by section number.

Comment 9:  	In Method 1668, Section 23.0, page 61:  It is useful to
provide the PCB numbering system used, to avoid confusion.  I realize
that CAS registry numbers are provided, which eliminate ambiguity;
however, most discussion of PCBs is by the numbering system, so having
that clarified explicitly in the document is very useful.  See comments
below on references.

Response:   	It is important to link the PCB numbers to the widely-used
and internationally-recognized CAS Registry numbers to minimize
ambiguity about the identity of specific congeners.

4.  	If you believe that a given item should be clarified, please
provide the clarification by section number of define the clarification
needed to a sufficient level of detail that we can make the necessary
changes.

Comment 10:  	It would be useful to provide more details on the
instructions to the participating laboratories as how to spike the
shipped wastewaters with PCBs.

Response:   	The wastewater samples were spiked at the sample-processing
laboratory. See Section 2.5.2 on page 6 of the report.

Comment 11:	Also, as stated previously, additional QC data from the
laboratories should be given to help ensure that the usable data was of
high quality.

Response:   	None of the participant laboratories submitted data for a
QC Check sample. Because the participant laboratories were volunteers,
EPA could not require analysis of a QC Check sample.

5.  	Please point out errors or omissions:

Comment 12:	In the document “Changes to be made to EPA Method 1668A
from the 8/20/03 revision based on the Method 1668A interlaboratory
validation study and comments and corrections received from laboratories
“, Under Quality control acceptance criteria, Table 5-1 from the
interlaboratory report should replace Table 6 not Table 5 in Method
1668A.

Comment 13:	In Method 1668A, Section 13.5.1.4, page 48, typo: should
read If calibration is not verified not If calibration is not verified.

Comment 14:	In Method 1668A, Section 22.0, page 59:  the title for
reference 4 should be provided for consistency.

Comment 15:	In Method 1668A, Section 22.0, page 60:  the title for
reference 18 should be provided for consistency.

Comment 16:	In Method 1668A, Appendix A, Section 2.1, page 110:  typo,
should be individual congeners not individual congene4rs.

Response:   	We have made these corrections.

III.	SPECIFIC OBSERVATIONS

These comments refer to the report on the interlaboratory study.

Comment 17:	Page 14, Section 4.2, Table 4-1 - I do not feel that
reporting of concentrations to up to 5 significant figures is
justifiable.  The GC used must have either a splitless or on-column
injector (Method 1668A, Section 6.9).  The latter in particular does not
have sufficient precision to justify 5 significant figures.  Three or
perhaps four significant figures is the maximum that should be present. 

Response:   	We have rounded all results to three significant figures
maximum.

Comment 18:	Page 14, Section 4.2, Table 4-1 - In addition, while
providing a mean, median, and maximum for concentrations of usable data
is useful, it is difficult to assess the distribution of data.  This is
important, as the fairly large differences between means and medians
suggest skewed data distributions, which are suggested by the various
figures of variability.  It would be useful to provide a box plot of
this data to make assessment much easier for the reader.

Response:   	A box plot would be so busy that the reader would not be
able to discern useful information.

Comment 19:	Page 21, Section 4.6, Table 4-4 - The same comments
regarding significant figures from my comments on Section 4.2, page 14,
Table 4-1 hold here as well.

Response:   	We have rounded all results to three significant figures
maximum. 

Comment 20:	Inclusion of the interlaboratory reports Appendix A,
Statistical Procedures Used to Develop QC Acceptance Criteria in the
Method as an Appendix may be useful.

Response:   	This is not necessary because the interlaboratory study
report has been posted on the CWA methods website. It would serve no
useful purpose to detail how the QC acceptance criteria were developed
in an Appendix to the method. EPA plans to provide the report as a part
of the rulemaking package so the information would be available to
readers interested in how EPA developed the QC acceptance criteria.

Comment 21:	The following comment is suggested for improvement of Method
1688A, and not the interlaboratory report:

Comment 22:	In Section 5.0 on Safety, pages 5-7, some explicit mention
should be made about the use of appropriate biohazard containment for
samples which are biosolids, as these are likely to host pathogens which
are a potential danger until rendered harmless by disinfection.  This
would include preparation of raw samples in a biosafety cabinet until
pathogens are killed (e.g., by soaking in solvents), appropriate safety
garments and emergency procedures for accidental spills of pathogenic
material, and warning about covering all materials containing solvents
in biosafety cabinets to prevent solvent fumes that can pose an
explosion hazard in biosafety cabinets that do not vent to exterior air.

Response:   	We agree and have added a sub-section on biohazards to
Section 5.

IV.	ADDITIONAL REFERENCES

1.	Mills, S.A., III; Thal, D. A., Barney J. (2007) -A summary of the 209
PCB congener nomenclature, Chemosphere 60, 1603-1612.

Peer review comments froM 

Kurunthachalam kannan 

 

Name:  Kurunthachalam Kannan, Ph.D.

Affiliation:  Research Scientist. Wadsworth Center, New York State
Department of Health & Professor, Department of Environmental Health
Sciences. State University of New York at Albany. Empire State Plaza PO
Box 509. Albany, NY 12201-0509

Date: 	March 24, 2010

Comments on:  Method 1668A Interlaboratory Validation Study

Review of “Method 1668A Interlaboratory Validation Study Report” and
“Addendum to the Method 1668A Interlaboratory Validation Study Report
Draft, August 2009.

Method validation is an integral part of a good analytical laboratory
practice.  The method 1668A validation study was intended to provide the
Environmental Protection Agency (EPA) with sufficient amount of data to
evaluate the performance of the method.  The method 1668A was originally
validated in 2000, in a single laboratory study.  However, the
guidelines published by the EPA and American Society for Testing and
Materials (ASTM) recommend a minimum of six complete data sets for
validating a method.  The objectives of the validation study were to
characterize the performance of the Method 1668A in multiple
laboratories and matrices and to evaluate and revise the Quality Control
(QC) acceptance criteria.  The objectives of the study are sound and
logical.  The multi-laboratory validation study is a requirement to meet
the national and international criteria to ensure reliability,
consistency, and accuracy of the analytical data generated using the
Method, especially when the method is used in the EPA’s Clean Water
Act programs.  The major outcome of the validation study was a revision
to the quality control/quality acceptance (QA/QC) criteria in EPA Method
1668A.

It is agreed that method validation studies are time and resource
consuming and EPA’s effort to perform this validation is highly
commendable.  Coordination of activities among several participating,
volunteer, laboratories is a daunting task.  EPA’s efforts on this
method validation are significant; nevertheless, data generated from
this study are under-utilized and with the existing data it is possible
to generate additional information required for validation.

1. 	Please review EPA's Report of the Method 1668A Interlaboratory
Validation Study and its addendum. Comment on the objectives of the
study, the soundness of the technical approach, and the results
produced.

Comment:  	The objectives of the study are sound and logical.  The
multi-laboratory validation study is a requirement to meet the national
and international criteria to ensure reliability, consistency, and
accuracy of the analytical data generated using the Method, especially
when the method is used in the EPA’s Clean Water Act programs.  The
major outcome of the validation study was a revision to the quality
control/quality acceptance (QAQC) criteria in EPA Method 1668A.  

	The overall technical approach, ranging from selection of laboratories,
sample selection, sample analysis, data reporting, data review, and
validation, is good, although there are some potential areas for
improvements.  A major drawback with the report is on the biosolids data
set.  The objective of the study was to have 6 complete data sets, as
per the EPA/ASTM guidelines, to validate the method.  Only 4 complete
data sets have been available for biosolids (Table 4-1), although
validation for biosolids appears to be an important goal for the method,
as indicated on the title.  Furthermore, the data submitted by the 4
laboratories for biosolids are highly variable and no ‘in-depth’
statistical analysis of data set has been made to determine accuracy and
precision of the data submitted for biosolids.  In other words, the
Method has not adequately met the objectives to validate the analysis of
PCBs in biosolids.  Many international organizations such as the
International Organization for Standardization (ISO) require at least
eight complete data sets to validate the methods.  Because the data set
available for biosolids does not meet the EPA/ASTM guidelines,
validation of the Method 1668A for biosolids is incomplete/inadequate.

Response:  	Reasons for receiving only four data sets for biosolid
samples are in Section 3 of the Report.  EPA believes that the high
failure-to-report rate may be attributable to (1) the extensive effort
needed to set up the extraction, concentration, cleanup, and HRGC/HRMS
systems for determination of 209 PCB congeners, (2) the difficulty and
lack of familiarity with the biosolids matrix, and (3) the voluntary
nature of participation in the study.  Given that four labs were able to
analyze the biosolids samples successfully, EPA chose to use the data. 
Although the ASTM guideline, D2777, generally requires six complete data
sets for method validation, ASTM has approved a means for taking
exceptions to the requirement for six complete data sets.  The
Association for Official Analytical Chemists, AOAC, has similar
requirements and exceptions.

	Agency guidelines for validation of chemical methods were developed by
the Forum for Environmental Measurement, FEM, as guidance, not
requirements. These guidelines do not specify a minimum number of labs
for a validation, and refer to D2777 as an example protocol.  Each
program office (e.g., EPA’s Office of Water) may apply the guidelines
as appropriate to specific situations, and to the extent practicable,
our validations are consistent with the FEM guidelines and other
guidelines.

	In the United States, validation principles developed AOAC and ASTM are
more widely followed than the international ISO guidelines.  EPA did not
desire to validate Method 1668 internationally, as was done with EPA
Method 1613 for chlorinated dioxins and furans.

	Note:  The four lab comment is repeated as comment “m” in section
5, and comment “a” in section 6 of this review. 

Comment:  	The second major concern with the validation study results is
that the data analysis/review is focused heavily (or only) on precision,
by measuring variability in concentrations reported by the laboratories
for PCB congeners.  Little attention has been paid to the accuracy of
the results.  Only for wastewater samples, recovery values are presented
as a measure of accuracy (only for wastewater, the spiked concentrations
were known and therefore the recovery could be measured).  Accuracy is
an important criterion in method validation.  There are several ways to
measure accuracy, without increasing the burden on the laboratories. 
Accuracy could have been assessed by analyzing a sample of known
concentration, as a laboratory control material, and comparing the
measured value to the true value. National Institute of Standards and
Technology (NIST) reference materials are available for PCBs and
laboratories could have been supplied or asked to analyze NIST reference
materials, along with samples supplied by the EPA.  It has been stated
that ‘to minimize the burden on volunteer participants’,
laboratories were not required to analyze initial precision and recovery
(IPR) samples.  Nevertheless, accuracy is an important parameter for
validating accuracy of a method, a well-characterized matrix such as
NIST SRMs can be used.  Alternatively, the technique of standard
additions, which is used to determine recovery of spiked congeners from
on-going precision and recovery (OPR) samples (sand/oil and water) could
have been used (Results from Table 4-4) to validate the accuracy of the
method.

Response:   	Table 4-4 presented results for accuracy (as recovery) for
the 27 labeled compounds spiked into all matrices (water, biosolids, and
tissue) by level of chlorination (homolog).  Mean recoveries ranged from
57 to 85 percent.  We believe that these recoveries are reasonable for
these analyses given the number of analytes and matrices covered by
1668C.  To address your concern about recoveries for the dioxin-like
(Toxic) congeners, we have expanded Table 4-3 of the Report to list
precision and recovery for the 27 individual labeled congeners from
wastewater and have expanded Table 4-4 to list precision and recovery
for all 27 labeled congeners (including the Toxic congeners) spiked into
all wastewater, biosolids, and tissue samples.  Revised Tables 4-3 and
4-4 are at the end of these responses.

	Regarding use of a Standard Reference Material (SRM) from NIST or a
certified reference material (CRM) from another source, EPA did not use
such materials to address previous criticisms for not using
“real-world” sample matrices that the Agency normally encounters in
its data-gathering activities.  In addition, SRMs and CRMs are often
dried or do not contain the mixture of pollutants found in real world
samples.  Therefore, we tested real-world wastewater, biosolid, and
tissue samples, in the forms typically submitted to laboratories for
analysis (i.e., whole-volume water samples, wet biosolids, and tissues).

Comment:  	The third issue on results: The method 1668A is intended for
congener specific analysis of PCBs with an emphasis on toxic congeners
(twelve non- and mono-ortho substituted PCBs).  The scope of the method
1668A indicates that twelve PCB congeners designated as ‘toxic’ by
the World Health Organization (WHO) are determined.  Nevertheless, the
validation study is focused on ten homolog groups.  The data analyses
involved summation of all isomer data within a congener group. 
Precision and accuracy of the Method 1668A for the analysis of
individual toxic congeners, are not validated.  In other words, the
method is not validated for 12 toxic PCB congeners.

Response:   	Method 1668A was validated in wastewater for all 209
congeners, as individual congeners and coeluted congener groups, and in
all matrices for the labeled analogs of the dioxin-like (“Toxic”)
congeners and congeners at the earliest and latest retention time at
each level of chlorination (27 labeled congeners total).  Presenting
results by level-of-chlorination (homolog) was a means of condensing
results so that they could be understood more easily.  However, to
respond to your concern about presenting results for the Toxics, we have
produced a table of recovery and precision for the 27 labeled compounds
individually, and shall replace Tables 4-3 and 4-4 in our revision of
the Report to show recoveries of the labeled compounds spiked into
wastewater and into all matrices, respectively.  The tables are attached
for your use at the end of these responses.

Comment:  	Furthermore, the suitability of this method for the analysis
of mono- and di-chlorobiphenyls (CBs) is questionable for the reasons
given below.  First, the precision and accuracy for mono- and di-CBs are
very poor, if not dismal.  For example, mean recovery of mono- and
di-CBs in wastewater samples were 3 and 54%, respectively.  Furthermore,
the concentration mono-CB found in wastewater sample is similar to what
was found in blanks (mean for wastewater is 27 pg/L [Table 4-1] and that
for water blank is 26 pg/L [Table 4-2]).  Obviously, the QC criteria
have been relaxed to reflect the poor recoveries (Addendum to the method
validation study).  Overall, the method did not perform well for mono-
and di-CBs.  If these two congener groups are removed, then the QC
criteria for the method would be tighter and more reliable/reproducible.
  These two congeners are easily lost in the solvent evaporation steps,
as has been stated in several places of the interlaboratory study
report.

Response:   	As explained in the second paragraph of Section 4.4 of the
Report, low recoveries at levels of chlorination (LOCs) 1 and 2 for
wastewater (Table 4-3) may be due to loss during transport from the
sample preparation laboratory to the participant laboratories because
data reported by the sample preparation laboratory showed that the
congeners were present immediately after shipment in replicates of the
samples held by the sample preparation lab.  Because the labeled
compounds at LOCs 1 and 2 were recovered in the normal range by the
participant labs, losses during extraction and/or cleanup can be ruled
out.  Therefore, and as explained in Section 4.4, the only mechanism for
loss we could think of is loss to the headspace of the sample bottle
during shipment, with subsequent loss to the atmosphere when the bottle
was opened.  Resources were not and are not available to verify this
theory.  Also, we have added labeled compound recoveries for wastewater
by LOC in Table 4-3, so that it is clear that the labeled compounds were
recovered at LOCs 1 and 2, even though the native compounds were not.

We recognize that the performance of this method will vary among the 209
congeners, and in different matrices.  This is typical of multi-analyte
methods because not all chemicals respond identically to extraction and
clean up techniques, or have identical instrument responses. In a study
of data comparability between two laboratories on samples collected from
the Passaic River in New Jersey, in which 151 PCB congeners were
identified and measured, accuracy as measured by analysis of a NIST SRM
was 15% or better.  Recoveries of the PCB congeners ranged from 90% to
124% and averaged 105%; precision ranged from 4.2 to 23%.

Comment:  	A few other issues on the results, such as robustness,
ruggedness, and limit of detection have not been discussed or analyzed
in detail.  A method validation would require that the above mentioned
parameters are tested and validated.  Some of the information presented
on the validation report is not clear (e.g., section 4.2).  For example,
how were the outliers detected?  How many outliers were found in
samples?  Rationale for removal of the outliers from data set is
missing.  A column showing number of outlier values should be presented
on Table 4-1.  The reproducibility variation coefficients
(between-laboratory precision and within-laboratory precision) for the
method were not reported.  Overall, data have not been fully utilized to
describe several parameters that are required in validating a method
(based on the comment 2 above and this comment).

Response:   	Robustness is indicated by method performance on multiple
matrices, the results of which are presented in revised Table 4-4 for
the 27 labeled compounds spiked into all samples.  These results show
that the labeled congeners are recovered from wastewater, biosolids, and
tissue within ranges expected for these types of analyses.

	We did not examine ruggedness directly in the interlab study. 
Ruggedness testing is done in a single-laboratory study when a method is
developed.  As stated previously, we expanded the performance data
gathered during the validation study to show recovery and precision on
an individual congener basis for the 27 labeled congeners, and have
included an updated table in the next revision to the Report.  EPA
provides an updated table at the end of these responses.  In addition,
evaluation of ruggedness (the effect of changes to variables in the
sample preparation, extraction, cleanup, and instrument steps) for a
method such as Method 1668A, which is applicable to a variety of sample
matrices, can be counter productive.  EPA’s historical experience
suggests that the sample extraction procedures used for the various
matrices are likely to have the greatest effect on the results.  Thus,
both the method and the study-specific instructions sought to minimize
deviations from the extraction steps described in the method.

	Regarding limit of detection (LOD), evaluation of LOD was beyond the
scope of the study.  An LOD study would require 7 replicate analyses
minimum of spikes at low concentrations to produce method detection
limits (MDLs).  

	Regarding outliers, as stated in the final paragraph in Section 3 of
the Report, “Study samples were assessed for outlying results using
Grubbs’ outlier test, performed in accordance with Standard Practice
for Determination of Precision and Bias of Applicable Test Methods of
Committee D-19 on Water (ASTM D2777-98).  Details on the outlier
analyses are presented in Appendix A.”  In Appendix A, we give further
detail on removal of outliers.  As stated in Appendix A, one outlier was
removed for the initial and on-going precision and recovery test, and
two outliers were removed for the test of labeled compound recoveries
from samples.  Because only three outliers were removed, we did not list
these three values.

	Regarding presentation of between- and within-laboratory precision,
pooled within-pair RSDs were presented by LOC in Table 4-4 for the 27
labeled congeners spiked into all samples.  We have expanded the
presentation to all 27 labeled congeners in the next revision to the
Report, and have attached the revised table at the end of these
responses.

2. 	Please render an opinion as to suitability for use of the data from
the study and the addendum for revision of the QA/QC in Method 1668 to
make it Method 1668B.

Comment:  	I believe that considerable effort has been put forward in
validating the method 1668A.  Nevertheless, the data from the validation
study are under-utilized and more important validation parameters such
as accuracy of the method, reproducibility, ruggedness, etc should be
evaluated.  These issues and corrections should be corrected/clarified. 
Once these revisions are made, the method may be revised to 1668B. 
However, I believe that there is lot more work to be done.

Response:   	As stated above, accuracy (as recovery), and
reproducibility and ruggedness (as RSD) data were presented by LOC in 
Tables 4-3 and 4-4.  EPA's Quality System defines accuracy to include
precision and recovery (bias), but most use accuracy and recovery
synonymously.  Table 4-4 of the Report gave recoveries across all labs
and matrices.  Reproducibility is indicated by the RSDs presented in
Table 4-4 and, although we did not evaluate ruggedness, per se, in the
study, how rugged a method is indicated by the precision of the
measurements (if the method isn't rugged, differences in the way
different labs practice the method will cause great variability in
results).  Also as stated above, we have  expanded the data in Tables
4-3 and 4-4 to show recovery and precision on an individual congener
basis for the 27 labeled congeners, so that reproducibility (as
precision) can be viewed on a per-congener basis.  The revised tables
are at the end of these responses.

3. 	If you have a suggestion for an improvement of the interlaboratory
study report or addendum, please provide suggested text by section
number & 

4. 	If you believe that a given item should be clarified, please provide
the clarification by section number or define the clarification needed
to a sufficient level of detail that we can make the necessary changes &

5. 	Please point out errors or omissions.

I feel that the charge questions 3, 4 and 5 are inter-related.  The
following items would help improve the study report and addendum

Comment:

a. 	Page 3, Section 2.4 (Sample Selection):  Some description of samples
is needed.  For example, does the fish tissue represent wholebody
homogenate or fillet?  Also, please specify when these fish samples were
collected?  I assume that these are wet samples (i.e., not
freeze-dried).  Similarly for biosolids, some sample description is
needed.  After reading the entire document, I realized that these are
wet samples.  It would be useful to indicate on page 3 (section 2.4),
what biosolids really are and when they were collected; also state that
they are wet and if possible indicate the moisture content?

Response:   	As stated in Section 2.4 of the Report, the tissue and
biosolids samples were generated from excess sample collected during two
unrelated EPA studies.   The biosolids samples were collected in 2001 as
part of EPA’s 2001 National Sewage Sludge Survey.  The fish tissue
samples were collected in 1999 and 2000 during the early phases of
EPA’s multi-year National Lake Fish Tissue Study (NLFTS). 

	All of the tissues used in the Method 1668A validation study were from
bottom-dwelling fish species and were originally prepared as whole-fish
composite samples.  The samples for the NLFTS were prepared as finely
ground tissue by a single laboratory.  Excess sample beyond that shipped
to the laboratories during the NLFTS was archived in 500-mL jars and
stored frozen. (The NLFTS also collected samples of predator species,
from which fillets were taken and composited for analysis.  The
bottom-dweller samples provided more tissue than the predator samples,
and thus a greater excess than was available for other purposes such as
a Method 1668A validation study.)  The tissue samples used for the
validation study were prepared from the 500-mL archive jars.  We have
added this information to our revision to the Report.

	Regarding your concern about the fish tissue type (whole; fillet), the
fish tissue homogenate was from a mixture of ground whole fish and
ground fish fillets.  Results of analyses of hundreds of fish tissue
samples have demonstrated that the performance of Method 1668 is
independent of tissue type.  Results were reported on a wet-weight
(as-received) basis, not a lipid basis.

	Both the biosolids and tissue samples had been stored in freezers at an
EPA sample repository.  In order to have a sufficient amount of each
sample available to support the study, EPA identified several samples of
each matrix type that could be combined to produce large volumes of
Youden pairs with the desired congener distribution.  Data for the PCBs
from those original studies were used to select samples that contained
PCB congeners in the range of interest for the validation study (e.g.,
neither the cleanest nor the most contaminated samples were selected,
but rather samples that would contain PCBs in the general operating
range of Method 1688A).

	Because of the chance that biosolids samples could lose low-molecular
weight PCB congeners during any drying procedure, the biosolids samples
were not dried.  Thus, EPA submitted biosolids samples to the
laboratories in the form that EPA believes is typical for this matrix:
wet, with varying moisture contents that reflect the source of the
material.  The original biosolids samples used to prepare the study
samples were collected as solids, as opposed to the pourable liquid
sewage sludges that may be produced at some wastewater treatment
facilities.  Each laboratory that analyzed the biosolids samples
determined the percent solid contents.  The reported values were in the
range from 30 to 40% for the two samples, amounts that are typical for
many biosolids produced in the U.S.  Results for the biosolids samples
were reported on a dry-weight basis.

Comment:

b.	Did the wastewater effluent samples contain suspended solids?  When
aliquoting wastewater samples, how was the homogeneity in the
distribution of suspended solids between aliquots ensured?  Wastewater
samples have been spiked with PCB congeners.  It appears that there was
no PCB present in wastewater sample prior to spiking.  Was that true?  I
suspect that PCBs can be detected in wastewater samples at several tens
to hundreds of pg/L level.    The information on the spike
concentrations listed on Table 2-2 is not clear.  What was the solvent
that was spiked into water?

Response:   	The wastewater samples were prepared from a final effluent
sample collected from a publicly owned treatment works (POTW) near the
sample preparation laboratory.  NPDES permit limits typically restrict
the total suspended solids (TSS) content of final effluents to values
around 10 mg./L.  Thus, the parent wastewater did not contain
appreciable suspended solids; and there was no difficulty in aliquoting
the wastewater samples.  The unspiked wastewater from divided into 1-L
aliquot in suitable glass bottles and each aliquot was spiked separately
(rather than spiking a bulk volume of wastewater and then subdividing
the spiked sample into replicate aliquots).  The sample preparation
laboratory analyzed several spiked aliquots to ensure homogeneity.

	In addition, the study-specific instructions provided to each
participating laboratory required that the laboratory filter the
wastewater sample prior to extraction and treat both the filtrate and
any solids on the filter in the manner described in Method 1688A.  This
instruction was included to prevent problems in which some laboratories
followed the method as written, and others deciding to skip the
filtration step if the wastewater did not appear turbid, as detailed in
Method 1668A.  This information has been added to the Report.

	Regarding the presence of a PCB background in the wastewater, the
unspiked wastewater sample was analyzed by the sample preparation
laboratory using Method 1668A.  Out of the 209 PCB congeners, 39
congeners were detected in the sample.  All of those congeners were
between PCB 001 and PCB 168, and all of the concentrations were between
17 and 247 pg/L, well below the concentrations of the spikes of the
congeners into the wastewater (see Table 2-2 of the Report for the
spiking levels).  Results for the blanks were all below the calibration
range of Method 1668A as practiced by the sample preparation laboratory.
 The sample prep lab thus flagged the results for the blanks as
estimates.  In preparing the actual study samples, EPA decided that
these background levels were low enough that adjustments need not be
made to the amount of each analyte spiked into the samples.  The solvent
used for spiking the congeners into wastewater was acetone.  The amount
of acetone used to spike the Youden pair samples was either 0.5 or 0.6
mL per 1-L volume of wastewater.   This information has been added to
the Report.

Comment:

c.	Page 5, Section 2.6:  Did all the participating laboratories use
HRMS?  The Method 1668A calls for the use of HRMS and the method
validation requires the use HRMS.  The deviations from the Method 1668A
have been listed for certain parameters (section 2.7 on page 6), but not
sure if all of the laboratories used HRMS for the analysis.  Also, on
page 5, it has been stated that the laboratories were instructed to
follow the Method 1668A, it will be helpful to mention/confirm that on
Section 3 (Data Review and Validation) that all laboratories indeed
followed the Method 1668A (other than the deviations noted on section
2.7).

Response:   	All participating labs used HRMS, and all labs claimed that
they followed Method 1668A other than for the deviations noted in
Section 2.7 of the Report.

Comment:

	d.	Page 6, line 3:  Expand “SOW”; the abbreviation is used here for
the first time.

Response:   	We have spelled out “SOW” to indicate Statement of
Work.

Comment:

 		e.   Page 6; Laboratory reporting of rules:  One important item that
is missing is the conversion of data from ‘wet weight’ to ‘dry
weight’ and vice versa (for biosolids).

Response:  	Section 17 of Method 1668A states how to calculate the
concentration of a congener on a wet- or dry-weight basis, as
appropriate.  The study-specific instructions stipulated that each
laboratory determine the percent solids of the samples, using no more
than 2.5 g for that purpose, to ensure that sufficient material was
available for several analyses by Method 1668A.  The reporting
instructions also stipulated that the biosolids results be reported on a
dry-weight basis.  Other reporting requirements may be used when
required by State or Federal regulations and it is not EPA’s intent to
preclude such approaches in an analytical method.  We have added the
information on the amount used for percent solids determination and the
basis on which results were to be reported to the Report.

Comment:

	f. 	A major issue with the method is lies in the weight of samples used
for extraction (especially biosolids) and final extract volume.  I think
the method should be modified to incorporate some flexibility because
the weight used for extraction can vary depending on the concentrations,
presence of interferences, etc.  Also, concentration to a final volume
of 20 uL prior to injection can introduce large variability in
analytical results.  I would prefer to use a sample weight of 15 g for
biosolids and a final volume of 100 or 200 uL.  I hope that the Method
1668B incorporates these changes to reflect some amount of flexibility
in the weight of the samples used for extraction and the final extract
volume.

Response:   	We believe that the most significant problem the labs
encountered with the biosolids sample was the inability to separate the
PCB congeners from the biosolids matrix using the extraction and cleanup
procedures in the method.  In part, this problem is likely caused by
overload of the cleanup columns.

	The study-specific instructions directed the laboratories to determine
the percent solids content of the biosolids samples before any other
steps, and then to use an aliquot of wet sample sufficient to yield
about 10 g of dry biosolid material.  These instructions were consistent
with the instructions in Section 11.5.1 of the method and were based on
EPA’s experience during the 2001 National Sewage Sludge Survey.  The
use of a consistent dry-weight aliquot across all of the participating
laboratories simplified the comparison of results across laboratories
and between samples.

	Section 11.5.1 of the method instructs the laboratory to weigh out “ 
SEQ CHAPTER \h \r 1 a well-mixed aliquot of each sample (of the same
matrix type) sufficient to provide 10 g of dry solids,” but Section 11
of the method also states that “  SEQ CHAPTER \h \r 1 For samples
known or expected to contain high levels of the CBs, the smallest sample
size representative of the entire sample should be used.” 

	Regarding concentration of the sample extract to 20 µL, the addition
of an n-nonane “keeper” in Section 12.7.7 of Method 1668A reduces
the possibility of loss when the extract is concentrated to 20 µL. 
Moreover, EPA’s experience with Method 1613B, for PCDDs/PCDFs, does
not suggest that the HRGC/HRMS laboratory community has difficulty with
the 20 µL final extract volume.  Finally, any option for changes to the
final extract volume would necessitate other changes in the method to
ensure that the final sample extract and the calibration standards
contain similar amounts of the labeled compounds used for isotope
dilution quantitation. Without clear evidence of a problem with the
current final extract volume, EPA does not believe that such changes are
warranted at this time. 

Comment:

	g.	Page 11, First paragraph, line 9:  Expand “ML”

Response:   	We have spelled out “minimum level of quantitation” in
the Report.

Comment:

	h.	Table 4-1, Page 12, column #7 (Mean), third row from the bottom;
241-???  Something is missing here.  It looks like a typo.  Is there a
value after ‘241’?  Also, footnote 1 for the table:  specify whether
biosolid and fish data are given on wet weight or dry weight basis.  For
footnote 2; insert “sum of” between ‘represent’ and ‘all’.

Response:   	We have made your suggested changes to the Report.  

Comment:

	i.    Section 4.3.;  Were the blanks analyzed similar to samples in all
aspects, especially the concentration factor (final extract volume)? 
This should be mentioned in section 4.3.

Response:   	Yes, to the extent that the matrices could be simulated. 
For the soil/tissue blank, a mixture of playground sand and corn oil was
used to simulate soil and tissue.  This mixture was chosen to minimize
the number of blanks that would need to be analyzed.  The method
requires that blanks and other QC samples be treated in the same manner
as field samples, including concentration of extracts to 20 µL.  There
is no allowance in the method for using a different final extract
volume.  We have added a statement to Section 4.3 of the Report
clarifying this information. 

Comment:

	j.	Page 13, Table 4-2:  “# Labs” for sand/oil should be “6”;
“10” is a misleading number here because “10” is the number of
datasets available, not ‘# Labs”.

Response:     	We have revised the header to column 2 in Table 4-2 to
state #Blank samples.

Comment:

	k.	Section 4.4:  Low recoveries of  mono- and di-CBs have been reported
to be due to loss during transport  or as ‘unknown’.  I would expect
that it is mainly due to loss of these volatile congeners in the solvent
evaporation steps in Method 1668A.

Response:   	As stated in response to the issue on results under item 1
above, the loss could not have occurred in the solvent evaporation steps
because the mono- and di- labeled compounds were recovered in the normal
range.  We have revised Table 4-1 in the Report to show this information
and have attached revised Table 4-1 at the end of these responses for
your use.

Comment:

	l.	Section 5.5:  “(Table 5-1).”  Move the ‘.” Out of the
parentheses.

Response:  	We could not find a period within parentheses in Table 5-1.

Comment:

	m.	Section 6.  I am little concerned about the statement that "the
result of this interlaboratory study met our objectives to
characterize……”.  The results have not satisfied the criteria to
validate the method for biosolids, if we follow the guidelines that we
need dataset from 6 laboratories.

Response:  	This comment is the same as your comment in item one of
section one, technical approach.  

Comment:

	n.	Corrections on addendum:  Table 4-1, footnote Biosolids
concentrations in ng/kg, wet weight, not ‘ng/L’.  Please correct. 
Also, the labeled compound recoveries are too low for a good method. 
The low recoveries are only for mono and di-CBs and if these two
congeners are removed or reported separately, the recoveries are more
than 50% for all other congeners.  I suggest that recoveries for mono-
and di-CB congeners are reported separately and tri-deca CB are reported
separately.  A 15 or 20% recovery is too low to consider a method to be
a good method.

Response:   	We have made the correction to the biosolids
concentrations.

	Regarding low recoveries for the mono- and di-CB congeners, we
responded to this issue in the section on results under item 1 above.

6. 	Please provide any additional comments you would like to make
regarding the documents under review.

Comment:

	a.	Although 14 laboratories were invited to participate (on a voluntary
basis), only 6 labs have provided results for fish and wastewater and 4
labs for biosolids.   Participation of a minimum of six labs is required
for a method validation study.  It is disappointing to note that several
laboratories have expressed dissatisfaction with method and problems in
analysis.  Inviting some laboratories for participation with some
(minimum) payment, and use of  NIST SRM samples for accuracy analysis
would enhance the quality of method validation. Because the method is
intended for use in EPA’s Clean Water Act programs, I would strongly
recommend that contract laboratories analyzing clean water act samples
should participate in this validation study.

Response:   	As suggested, we have obtained data, including MDLs, from
contract laboratories experienced with this method.  This data is in the
current version of this method, 1668C. We answered your comment about
the number of labs and use of SRMs in response to your first two
comments in section 1 of this review.  Again, the final data sets of 6 ,
6 and 4, respectively, for the three sample types, water, tissue and
biosolids, resulted from review of laboratory results against our data
validation guidelines and from admissions and narratives from the labs.

Comment:

	b.	I would suggest that mono and di-CBs are reported separately or
removed from the method.  The QC criteria for these two congener groups
should be reported separately from the other congeners.

Response:   	As explained in the section on results under item 1 above,
we believe that the low recoveries for wastewater may be the result of
losses during sample transport and not the result of a method
deficiency.  

Comment:

	c.	Participating laboratories should report twelve toxic congeners
separately and QC criteria for these congeners should be established; in
other words, the method validation is needed for these 12 toxic
congeners.

Response:   	There is no prohibition in Method 1668 to reporting results
for the toxic congeners separately.  And QC acceptance criteria were
given for the toxic congeners, as shown in Table 5-1 of the Report and
Table A-1 of the Addendum

Comment:

	d.	A major issue with biosolids analysis lies in the use of 30 g of
sample weight for extraction.  Similarly, concentration of samples to a
final volume of 20 uL, can impose serious variability issues.  I would
recommend that the method be amended to incorporate some flexibility in
the weight of the samples taken for extraction.  For biosolids, it can
be from 10 to 30 g (wet weight).  Also, concentration of extract to 20
uL, will introduce large variability.  I would suggest that the final
volume can be from 20 uL to 200 uL.  I would recommend using sediments
for method validation.

Response:  	As noted in response to earlier comments, the method does
not require the use of a 30-g sample.  Section 11.5.1 of the method
instructs the laboratory to weigh out “  SEQ CHAPTER \h \r 1 a
well-mixed aliquot of each sample (of the same matrix type) sufficient
to provide 10 g of dry solids,” Section 11 of the method also states
that “  SEQ CHAPTER \h \r 1 For samples known or expected to contain
high levels of the CBs, the smallest sample size representative of the
entire sample should be used.” 

	Regarding concentration of an extract to a final volume of 20 to 200
uL, a lab may concentrate to a higher volume under the equivalency
provisions in Section 9 of Method 1668 and under 40 CFR 136.6, provided
that all QC requirements in the method are met.  However, and as noted
in an earlier response, any option for changes to the final extract
volume would necessitate other changes in the method to ensure that the
final sample extract and the calibration standards contain similar
amounts of the labeled compounds used for isotope dilution quantitation.
 Without clear evidence of a problem with the current final extract
volume, EPA does not believe that such changes are warranted at this
time.

	Regarding use of sediments for method validation, EPA may use sediments
in future validation studies depending on program needs and
requirements.  However, at the time of the validation study, EPA felt
that biosolids were a more likely candidate for PCB congener regulations
than sediments. 

	Revised tables 4-3 and 4-4, described in EPA’s responses above, are
shown below for your convenience.



Table 4-3   Wastewater Sample Recovery and Precision by Level of
Chlorination

LOC	Percent Recovery (%)	Within-pair Relative Standard Deviations (%)

	# Results	Mean	Median	Min.	Max.	# Pairs	Pooled*	Median	Min.	Max.

 Native congeners spiked by sample prep lab

1	25	3.15	2.71	0.49	11.8	11	29.7	17.5	5.8	80.8

2	118	54.2	44.6	2.63	162	57	12.2	4.42	0.17	62.4

3	223	89.5	82.8	34.2	164	111	7.62	5.06	0.02	24

4	356	95.6	91.4	38.1	201	178	7.36	6.16	0	20.6

5	344	81.4	72.2	30.6	182	170	10.8	7.99	0.06	40.5

6	362	75.3	68.8	8.14	196	178	12.1	8.64	0.16	46.8

7	235	72.3	64.4	10.4	155	114	9.63	5.24	0.14	39.3

8	116	68	59.3	18.1	135	56	11.3	7.26	0.12	32.3

9	35	70.8	57.8	44.4	126	17	8.91	6.66	0.57	18.5

10	12	70	59.1	49.3	118	6	5.67	4.23	0.37	9.05

 Labeled congeners spiked by participant labs

1	24	51.4	48.9	21.0	84	12	19.7	13.8	3.45	42.9

2	24	58.5	55.1	25.0	90	12	15.7	11.9	0	29.2

3	36	67.4	62.6	26.0	108	18	12.5	5.32	0.516	33.3

4	33	60.5	57.5	35.0	101	15	8.24	3.34	0.873	17.7

5	83	77.2	81.0	41.0	110	41	11.0	2.61	0	28.7

6	50	75.6	74.3	38.6	106	25	9.83	7.13	1.32	21.8

7	36	76.9	77.0	5.00	123	18	30.4	3.72	0	126

8	23	76.6	79.5	38.4	94	11	6.20	5.12	2.29	12.4

9	24	71.2	70.0	49.1	98	12	6.33	4.24	0	9.92

10	12	73.1	74.5	52.8	98.2	6	6.81	5.26	4.07	12.0

* Pooled RSD calculated as the square root of the mean of the squared
within-pair RSDs



Table 4-4  Recovery and Precision of Labeled Compounds Spiked into
Samples1

LOC2	Labeled Congener3	Recovery (%)	Within-pair RSD (%)

# Results	

Mean	

Median	

Min.	

Max.	# Pairs	

Pooled4	

Median	

Min.	

Max.

1	1L	32	52.9	48.5	13.0	95.0	16	22.0	17.4	0.81	43.4

1	3L	32	60.4	61.1	15.9	107	16	18.9	10.9	2.70	47.6

2	4L	32	60.8	56.1	32.0	120	16	16.7	11.9	1.30	31.6

2	15L	32	64.8	69.0	25.0	96.3	16	13.7	2.8	0	31.4

3	19L	32	59.0	59.0	4.5	112	16	12.0	5.4	0.69	26.5

3	28L	32	78.8	87.8	25.0	118	16	11.7	7.8	1.40	33.3

3	37L	32	77.2	77.6	35.0	110	16	10.0	2.4	0.52	23.6

4	54L	31	61.6	59.1	25.5	109	15	10.3	5.4	0.87	22.0

4	77L	31	72.7	67.3	43.0	106	15	9.1	5.7	1.10	19.8

4	81L	31	74.9	71.0	26.0	129	15	9.0	6.6	1.30	18.8

5	104L	32	76.5	80.0	41.0	102	16	10.1	4.8	0	24.4

5	105L	31	81.9	86.3	52.7	103	15	10.9	5.2	0.50	25.2

5	111L	29	85.2	86.3	63.0	110	13	5.4	3.7	0.29	12.7

5	114L	32	83.0	86.3	48.0	119	16	10.8	5.2	1.00	28.3

5	118L	32	83.9	87.5	53.8	120	16	10.8	6.4	0.36	26.6

5	123L	32	84.7	88.8	53.8	116	16	11.4	6.5	0.48	23.4

5	126L	32	81.0	80.8	59.0	123	16	11.8	4.5	0	28.7

6	155L	32	77.3	81.3	33.0	106	16	7.8	3.6	0	21.8

6	156L+157L	32	87.8	76.5	44.0	216	16	12.9	8.0	1.10	30.3

6	167L	31	84.3	82.0	57.3	110	15	9.9	8.6	1.30	22.6

6	169L	32	78.3	75.3	30.5	112	16	14.8	8.0	2.40	37.5

7	178L	30	89.1	93.8	5.0	126	14	34.6	4.7	0.69	126

7	188L	32	78.0	85.2	32.0	136	16	8.5	4.6	0.22	17.4

7	189L	32	82.8	82.0	54.0	118	16	8.1	4.8	0	18.2

8	202L	31	84.9	87.8	38.4	145	15	9.2	5.4	1.30	19.9

8	205L	32	82.3	81.5	51.0	118	16	7.9	4.8	0.25	17.3

9	206L	32	80.8	81.3	50.0	115	16	9.2	7.7	0	19.8

9	208L	32	80.4	84.7	49.0	129	16	7.6	4.9	0	17.2

10	209L	32	78.7	79.8	47.7	110	16	9.2	6.3	2.20	18.9

 Wastewater, biosolids, and tissue

2 Level of chlorination

3 Labeled analog of World Health Organization dioxin-like (Toxic)
congener shown in bold

4 Pooled RSD calculated as the square root of the mean of the squared
within-pair RSDs

Section Three:  Original Peer Review Comments on Draft EPA Method 1668A
Interlaboratory Validation Study

Prepared for:

U.S. Environmental Protection Agency

Office of Water

Office of Science and Technology

Health and Ecological Criteria Division

1301 Constitution Ave., N.W.

Washington, D.C.  20004

First Two Sets of Comments Prepared by:

Versar, Inc.

6850 Versar Center

Springfield, Virginia  22151

September 2007

Third Set of Comments Prepared by:

Eastern Research Group, Inc. 

110 Hartwell Avenue. 

Lexington, MA 02421

March 2010.Peer Review Comments From

Michael C. Newman

Peer Review Comments on:  Draft EPA Method 1668A Interlaboratory
Validation Study

NAME: Michael C. Newman, PhD

AFFILIATION:  121 York Point Dr., Seaford, VA, also Professor at the
College of William and Mary, Virginia Institute of Marine Science

DATE: Sept. 18, 2007

I.  GENERAL IMPRESSIONS  

Overall, this Report of the Method 1668A Interlaboratory Validation
Study is a sufficiently accurate, clear, and thorough analysis
discussing the results of a very difficult task.  The study objectives
were to (1) characterize the performance of Method 1668A for multiple
laboratories and matrices (wastewater, biosolid, fish tissue), and (2)
evaluate and possibly revise the QC acceptance criteria. These
objectives were met with some qualification. It is important to
emphasize prior to discussing the specific qualifications that this
effort does appear to be generally well-intended, -planned and
-interpreted. Some challenges to applying the results do exist. The
laboratory participation was disappointing and this was the source of
numerous issues. For example, six of 14 laboratories declined to
participate in the biosolids analysis. An additional four produced
“unusable” numbers, resulting in only four of 14 potential sets of
results being applicable to the task at hand.  For wastewater and fish
tissue, only six laboratories produced the data used in this exercise. 
Four laboratories successfully analyzed all three sample types and,
therefore, have a very strong influence on the results.  The candidate
laboratories included government and private organizations which might
be expected to have differences in response; however, it is unclear from
the report if the final subset was biased toward one group or another.
The very high failure-to-report rate for this process suggests that
failure to achieve useful analyses with 1668A warrants more discussion.
Was the high failure rate normal for such voluntary validation exercises
or a feature to be anticipated for Method 1668A as currently written?
Some laboratories deviated from 1668A for some analyses. Also outlier
tests that require extremely thoughtful application were applied to cull
away some data. Together, these features do not permit as high
confidence in the study’s generalizations as might be desired.  The
studies conclusions are not unequivocal at this point.  The resulting,
central concern can be articulated with a quote from Stuart’s Basic
Ideas of Scientific Sampling (1968): “… the credentials of a sample
are not only important for the interpretation of results, but are in
principle the only information of value we can have in this respect.” 

My general conclusion is that the results are adequate for meeting the
study objectives but should be augmented in the near future in order to
enhance confidence in the general validity of the conclusions.  Detailed
below are specific examples from within the report and associated
recommendations.

II. RESPONSE TO CHARGE QUESTIONS 

1. Please review the EPA's Report of the Method 1668A Interlaboratory
Validation Study for the objectives of the study, the soundness of the
technical approach, and the results produced.

COMMENT: The Interlaboratory Validation Study attempts to characterize
the performance of Method 1668A in multiple laboratories and matrices
(wastewater, biosolid, fish tissue), and to evaluate and possibly revise
the QC acceptance criteria. As stated above, my overarching opinion is
that the study met these goals but further work is required to enhance
confidence in the generality of the conclusions.  The goals for this
complicated method were extremely challenging so the implied incremental
generation of sufficient characterization/validation information is, in
my opinion, reasonable.  My suggestion that more work is required is not
an oblique inference about the work quality of the study leaders.  This
was a very difficult task.

2. Please render an opinion as to suitability for use of the data from
the study for revision of the QCAC in Method 1668 to make it Method
1668B. 

COMMENT: Yes. The study data are adequate, although lacking in some
ways, for revision of the QC in Method 1668.

3. If you have a suggestion for an improvement, please provide suggested
text by section number.

 COMMENT:  A wide range of suggestions and insights will be addressed
below regarding the improvement of the study or existing study
presentation.

Page 6, Top paragraph and first full paragraph:  This is the first place
in the report that the issue of sample homogeneity emerged. It is also
discussed in Section 4.1.  Described here is the process in which a
“sample processing laboratory was required to perform background and
homogeneity analyses of both the biosolids and tissue matrices.”  The
text does not convince me that the samples were “homogeneous” in a
strict sense relative to the analytes.  For example, some fatty tissues
might contain more analyte than others and similar-sized pieces of fatty
tissues might not be homogeneously distributed in a well-mixed
homogenate.  The biosolid to which sand was added might experience
density-driven sample component segregation (i.e., sinking sand) so that
the sample aliquot analyzed by the volunteer laboratory might not have
been homogeneous (well-mixed?) relative to the analyte. There are very
specific methods of determining this sample quality (well- or
poorly-mixed) yet they are not applied by the sample processing
laboratory.  It is not clear how this might or might not have
contributed to differences in results among laboratories.  This concern
is also inherently linked to sample aliquot size which 1668 prescribes
versus and the actual sample sizes used by the volunteer laboratories.
(I assume that the recommendations of sample sizes were based only on
anticipated analyte concentrations in typical samples and the mechanics
of the sample processing.)

	The point being made above can be illustrated easily using the
conventional explanation as depicted in the figure to the left.  The
analyte (dot) is homogeneously distributed in this figure’s top panel;
therefore, sample aliquot size illustrated in the figure as different
size boxes will not influence the measured analyte concentration in an
analyzed aliquot. (Homogeneity is a material quality in which the
analyte is distributed in a completely homogeneous manner throughout the
material.)   In the bottom panel, a heterogeneous material is depicted
in which sample aliquot size becomes critical depending on whether the
heterogeneous material is well- or poorly-mixed.  If sample aliquot
sizes are too small for a poorly-mixed, heterogeneous material (small
squares in figure bottom panel), variation among reported analyte
concentrations will be higher than that derived with aliquot sizes of
sufficient size (large square in the bottom panel) (Newman 1995).  To
conclude that a heterogeneous sample is sufficiently well-mixed to
permit use of a specified aliquot size (e.g., 20 g of fish tissue)
requires some conventional calculations.  The homogeneity analyses
described in this report do not include the associated statistics
permitting such decisions.  This is especially relevant to the biosolid
and tissue samples, and with sufficiently high solids content, also to
wastewater samples.  The pertinent approach to this problem and
associated statistical computations are described in classic papers by
Visman (1969), Ingamells (1973) Ingamells and Switzer (1974), and
Wallace and Kratochil (1987).  

 

Page 8, Top Bullets: The laboratories were told to report all “>nm”
values as quantified concentrations and the statistical analysis of
these measured values was treated such that all “>nm” values
possessed the same quality of measurement.  It is incorrect to assume
that all values above “nm” had the same measurement quality (e.g.,
Keith et al. (1983)). The relative uncertainty of a measured value can
be estimated relative to the standard deviation of the analytical
process baseline noise (See also figure on page 6),

 

 where Z= Z-statistic value for a confidence level of % (conventionally
95%), and N is the measured signal expressed in units of multiples of
the standard deviation of the baseline noise, e.g., N=3 and 10 are
associated with the signal at the detection limit and limit of
quantification, respectively. Interpretation of the study results is
very difficult if this fact is not addressed in the discussions and
shown in graphs of results for the various samples and sample types. An
example of this will be discussed below relative to the figures in the
report.

Page 12, First paragraph after bullet list and also last sentence:  Here
and elsewhere, my assumption is that the study results are intended to
provide measurement quality characteristics anticipated for
“in-control” analyses of the specified sample types by a laboratory
represented by the initial 14 laboratories.  It is critical that the
population for which these final QC criteria are generated be clearly
defined given the variations in reporting, method application, and data
selected for final QC calculations.   

	Here and elsewhere, Grubbs’ single outlier test (Grubbs 1969) is
applied to cull away apparent outliers.  Outlier deletion, as discussed
in Grubbs’ paper and many others, should be done very thoughtfully
according to clear rules. For example, the “outliers” detected based
on the assumption of a normal distribution might come from a population
more appropriately described as lognormal. The “outlier” would be
falsely removed based on the assumption of the wrong distribution. The
rules for handling outliers in Grubbs (1969) and other papers should be
considered prior to applying any outlier tests.  The results of such
application should then be described in the report.  Classic papers that
might be useful are Anscombe (1960) and Rosner (1983). As multiple
outliers might be masking each other in these data sets, Rosner’s
method (1983) for multiple outliers might be useful.  Personally, I
think that Woodworth’s (2004) classification of outliers and the
associated approach based on interquartile ranges is more relevant as it
is more open to intelligent application than the above hypothesis-based,
parametric tests.  This point will be repeated later in discussing the
absence of quartile (Q1 and Q3) values in tables of summary statistics.

Page 13, Section 4.2.1, line 4: The “homogeneity … was found
acceptable…”  Respectfully, my guess would be that this statement is
correct; however, I disagree that enough information was extracted to
make a definitive statement.  I base this disagreement on the material
already presented above.  

Page 14, Table 4-1:  Here and elsewhere, it would be very helpful to
have the first and third quartile values (Q1 and Q3) or, minimally, the
interquartile range (|Q3-Q1|) in the table. One immediate use would be
to apply interquartile range-based methods to explore potential outliers
or outlying laboratories.   The inclusion of these statistics is highly
recommended.  As an aside, the differences between means and medians for
congeners with 3 to 8 chlorination levels (biosolids and tissues)
suggest a skewed sample set. Are these data skewed for any reason or is
this simply a spurious trend in the tabulated data?

Page 29, Last line of text: I generally agree but feel that a less
definitive, and more conditional, conclusion can be reached.

Appendix A. Section 1.0., 95% Confidence Intervals:  The section begins
by describing conventional, parametric calculations for all congeners in
all matrices at all concentrations.  Was the assumption of normality
tested and met, or found to be unimportant?  Some discussion seems
warranted.

Appendix A Section 1.0, Application of Grubbs’ outlier test: Again,
outlier tests should be used very carefully and interpretation of any
consequent data sets based on truncated distribution statistics (see
Grubbs (1969) and other sources for discussion). As a simple example,
what is the n in the equation given in Section 2.0?  This is not as easy
a question to answer as might be suspected since some observations were
thrown out.  Also, was the assumption of a normal distribution (upon
which Grubbs’ test depends) appropriate?  Finally, what Type I error
rate was selected? Likely, the typical 0.05 was used yet one could argue
for a more conservative error rate.

4. If you believe that a given item should be clarified, please provide
the clarification by section number or define the clarification needed
to a sufficient level of detail that we can make the necessary changes. 

COMMENT:  A wide range of issues will be addressed below as suggestions
for clarifying presentation.

Section 1.2, Line 3: Change “seen” to “seven”?

Page 4, Paragraph 2: Please consider replacing “properly
homogenized” with “homogenized.”  The reasons are given above for
removing the vague word from this sentence. One could easily argue that
the samples were not shown to be “properly homogenized.”

Page 5, Paragraph 2 of Section 2.4: The term, “Youden pairs” is
first used here but not defined until Section 2.5.1. Please move the
definition up to the first use of the term.

Table 2-1 (page 4) and Table 3-1 (page 12): It is extremely important in
judging the generalizability of the report conclusions that the
laboratory participation in generating the final QC criteria is very
clearly discussed. At present, it is not easy to follow. Generally, one
starts with 14 laboratories (government and private) and ends up with
data from 4 to 6 labs. It isn’t easy to judge how (1) withdrawal of
the various laboratories from providing results, (2) the study
author’s judgment about reported data usefulness, and (3) the final
data inclusion influences the statistical population of laboratories for
which inferences about QC are being made.   Several issues to be
addressed include the following. Are the QC conclusions relevant to only
the best or best equipped laboratories? Are they representative of
primarily government laboratories, private laboratories, or both? A very
clear and concise summary addressing this linchpin issue is needed.  In
developing this summary, discussion of the influence of outlier
rejection is also warranted.

Page 8, Section 2.7, first paragraph: The requirements were given for
calibration curves but the laboratories interpreted the requirements
differently. I saw no obvious reason for the misinterpretation; however,
some thought should be put into rewriting the associated text so it is
even clearer.  Also, relative to Section 2.7.1. and 2.7.2, were the
consequences of any of the various deviations quantifiable? For example,
the differences due to different aliquot weights used by the various
laboratories might be quantified with the methods cited above for
“Page 6, Top paragraph and first full paragraph.”

Page 9, Section 2.7.2. Different problems were encountered by different
laboratories. Different weight normalizations were done by the different
laboratories. It isn’t clear to me how this directly influenced
statistical estimates and associated inferences.  Also, the numerous
deviations suggest that even more clarity needs to be placed into 1668A.
 Perhaps more discussion is needed about what deviations should not be
considered.

Page 11, Section 3, first paragraph: This section begins by describing
three laboratories that did not submit data.  It is important to know if
non-reporters are considered a biased or random subset of the population
of possible responders.  Was there a risk of biased sampling of possible
laboratories using this method as a result of biased non-response?

Page 11, Section 3, fourth bullet: How was this issue handled during the
analysis of results?  Would omission of an outlier laboratory create a
biased sample of laboratories?

Page 15, Table 4-2 and all other tables: Univariate statistics are
applied throughout these tables.  Some graphs suggest skewed
distributions for some data.  Was the assumption of normality or the
robustness of conclusions to violations of this assumption assessed?

Page 16, first lines: This discussion is too speculative in my opinion.

Page 17, Figure 4-1 (and also 4-3, 4-5, and 4-7): As dictated by the
generally-accepted equation given above, the relative uncertainty of a
measurement will rapidly increase as concentration decreases.  The
following figure gives the general shape of such a relationship.
Depending on the positions of the detection limit (LOD in this figure)
and limit of quantitation (LOQ in this figure) (from Newman (1995)), the
variation in the y direction as a function of measured concentration
would be totally expected based on this relationship.  Would it be
possible to insert vertical lines at the LOD and LOQ in these figures so
a reader could judge this issue? As a good example of how this would
help, it is difficult to decide if the explanation for the Figure 4-5
outliers (Page 20, first few lines) is reasonable or whether the
variations at low concentrations (in lower part of the Region of
Less-Certain Quantitation?) are what you would expect for any
measurement process.

Page 18, Figure 4-2 (and also 4.4, 4.6): These figures do not convey any
useful information. Relative Standard Deviations (e.g., 4-3) are much
more useful for conveying the associated information (that variation can
change with concentration).

Page 21, Table 4-4:  A simple mistake was made here such that the “#
pairs” column has two digits after the decimal point, e.g., 32.00
pairs.

Page 29: The conventional “This page intentionally blank” statement
is made but, in fact, there is text on the page.

5. Please point out errors or omissions.

COMMENT: Please see comments above.

III. Specific ObservatioNS

[none submitted]

IV.   ADDITIONAL REFERENCES

Anscombe, F.J. 1960. Rejection of outliers. Technometrics 2: 123-147.

Grubbs, F.E. 1969. Procedures for detecting outlying observations in
samples. Technometrics 11: 1-21.

Ingamells, C.O. 1974. New approaches to geochemical analysis and
sampling. Talanta 21: 141-155.

Ingamells, C.O. and P. Switzer. 1973. A proposed sampling constant for
use in geochemical analysis. Talanta 20: 547-568.

Keith, L.H., W. Crummett, J. Deegan, Jr., R.A. Libby, J.K. Taylor, and
G. Wentler. 1983. Principles of environmental analysis. Anal. Chem. 55:
2210-2218.

Newman, M.C. 1995. Quantitative Methods in Aquatic Ecotoxicology.
Lewis/CRC Publishers, Boca Raton, FL, pp. 426. 

Rosner, B. 1983. Percentage points for a generalized ESD many-outlier
procedure. Technometrics 25: 165-172.

Stuart, A. 1968. Basic Ideas of Scientific Sampling, Hafner Publishing
Co., New York, pp. 99.

Visman, J. 1969. A general sampling theory. Mat. Res. Stand. 9:9-64.

Wallace, D. and B. Kratochvil. 1987. Visman equations in the design of
sampling plans for chemical analysis of segregated bulk materials. Anal.
Chem. 59: 226-232.

Woodworth, G.G. 2004. Biostatistics. A Bayesian Introduction. John Wiley
& Sons/



Peer Review Comments From

CHARLES WONG

Peer Review Comments on:  Draft EPA Method 1668A Interlaboratory
Validation Study

NAME:  Charles S. Wong, Ph.D.

AFFILIATION:  University of Alberta, Department of Chemistry

DATE:  18 September 2007

I.  GENERAL IMPRESSIONS

This document is a synopsis of the interlaboratory validation study
intended to improve EPA method 1668A, a protocol for congener-specific
measurement of polychlorinated biphenyls (PCBs) in water, soil,
sediment, biosolids, and tissue by high resolution gas
chromatography/high resolution mass spectrometry.  Fourteen laboratories
were invited to participate, and samples as well as instructions and
required reporting information was provided to them.  Reported data was
compiled and summarized.

The document provides a useful but succinct summary of the
interlaboratory validation’s very large dataset.  Sufficient detail is
provided on the background on Method 1668A and the methodology used to
invite participating laboratories to ascertain the effectiveness of the
validation study in characterizing measurement of PCBs in the samples
chosen.  Methods for selection of samples for the laboratories are clear
and reasonable, including explicit discussion of limitations for
logistical and technical purposes.  A clear and detailed discussion of
deviations from the required protocol is evident, so that the reader can
clearly understand limitations in the reported data.  For the most part,
the discussion of the usable data is clear and pertinent to the intended
goal of determining possible changes to the protocol and to acceptance
criteria.  

1.  Please review the EPA’s Report of the Method 1668 Interlaboratory
Validation Study for the objectives of the study, the soundness of the
technical approach, and the results produced.

COMMENT:  The objectives of the study are sound.  Should Method 1668A be
revised, and if so, what protocol changes and/or acceptance criteria
should be modified?  This can only be done with an interlaboratory
validation of a standard set of samples.

Sample selection is reasonable in my opinion.  Biosolids and tissues
were prepared by EPA at two distinct concentrations, properly
homogenized, and shipped to each laboratory with appropriate safeguards
against decompositions to ensure that that the integrity of the samples
upon receipt by the participating laboratory was maintained.  While it
would have been ideal to have wastewater samples with PCBs shipped to
the laboratories, it is reasonable to assume that testing of Method 1668
capabilities would not be feasible.  Thus, providing instructions to
spike the wastewater effluent shipped to the laboratory was reasonable.

As previously stated, deviations from the method were characterized in a
succinct but meaningful manner.  This allowed the flagging of suspect
data as unusable for further purposes.  Many of the deviations noted had
details on the underlying rationale, e.g., problems with cleanup of
samples encountered by some of the laboratories.  However, not all such
deviations were detailed in this manner, which I would have found
useful.  It is troubling that there were sufficient problems in sample
processing, measurement, and even data reporting (e.g., no calibration
reporting from one laboratory???) that less than half the data from the
participating laboratories was usable (Table 3-1), as part of the point
of having a large number of independent participants was to assess the
robustness of both the overall Method as well as the validation for
improving the Method.  That having been said, I feel that the most
problematic deviations were flagged unusable, and that the usable data
is sufficient for the intended purpose.

It would have been useful if the laboratories were asked to provide
measurement data on QC Check samples (Method 1668A, Section 7.15) from
standardized reference materials (e.g., NIST SRMs or Canadian NRC SRMs)
in addition to the other QC data, e.g., OPR samples in reagent water,
and blanks.  This is important because as noted in the report, PCB
concentrations in the shipped samples were unknown, and having this
check would be useful.  If this was actually done, this was not clear
from the report.

The reported results provided a good summary of the variability observed
with PCB measurement in the various matrices supplied to the
participating laboratories.  This information is most useful for
resetting IPR/OPR guidelines.   The summary and recommendations appear
reasonable, e.g., using average relative responses under specific
circumstances, noting poorer and more variable recoveries for lower
chlorinated congeners, and somewhat tighter recovery windows
particularly for heavier congeners.

2.  Please render an opinion as to suitability for use of the data from
the study for revision of the QCAC in Method 1668 to make it Method
1668B.

COMMENT:  I feel that this data, despite limitations noted above and in
the report, are of sufficient quality to revise the method for the next
version of this Method.

3.  If you have a suggestion for an improvement, please provide
suggested text by section number.

COMMENT:  In Method 1668, Section 23.0, page 61:  It is useful to
provide the PCB numbering system used, to avoid confusion.  I realize
that CAS registry numbers are provided, which eliminate ambiguity;
however, most discussion of PCBs is by the numbering system, so having
that clarified explicitly in the document is very useful.  See comments
below on references.

4.  If you believe that a given item should be clarified, please provide
the clarification by section number of define the clarification needed
to a sufficient level of detail that we can make the necessary changes.

COMMENT:  It would be useful to provide more details on the instructions
to the participating laboratories as how to spike the shipped
wastewaters with PCBs.

Also, as stated previously, additional QC data from the laboratories
should be given to help ensure that the usable data was of high quality.

5.  Please point out errors or omissions:

COMMENT:

In the document “Changes to be made to EPA Method 1668A from the
8/20/03 revision based on the Method 1668A interlaboratory validation
study and comments and corrections received from laboratories”:  Under
Quality control acceptance criteria, Table 5-1 from the interlaboratory
report should replace Table 6 not Table 5 in Method 1668A.

In Method 1668A, Section 13.5.1.4, page 48, typo:  should read “If
calibration is not verified” not “If calibration I s not
verified”.

In Method 1668A, Section 22.0, page 59:  the title for reference 4
should be provided for consistency.

In Method 1668A, Section 22.0, page 60:  the title for reference 18
should be provided for consistency.

In Method 1668A, Appendix A, Section 2.1, page 110:  typo, should be
“individual congeners” not “individual congene4rs”.

III.  SPECIFIC OBSERVATIONS

These comments refer to the report on the interlaboratory study.

	Section 4.2, page 14, Table 4-1.  I do not feel that reporting of
concentrations to up to 5 significant figures is justifiable.  The GC
used must have either a splitless or on-column injector (Method 1668A,
Section 6.9).  The latter in particular does not have sufficient
precision to justify 5 significant figures.  Three or perhaps four
significant figures is the maximum that should be present. 

	Section 4.2, page 14, Table 4-1. In addition, while providing a mean,
median, and maximum for concentrations of usable data is useful, it is
difficult to assess the distribution of data.  This is important, as the
fairly large differences between means and medians suggest skewed data
distributions, which are suggested by the various figures of
variability.  It would be useful to provide a box plot of this data to
make assessment much easier for the reader.

	Section 4.6, page 21, Table 4-4.  The same comments regarding
significant figures from my comments on Section 4.2, page 14, Table 4-1
hold here as well.

	Inclusion of the interlaboratory report’s Appendix A, Statistical
Procedures Used to Develop QC Acceptance Criteria” in the Method as an
Appendix may be useful.

The following comment is suggested for improvement of Method 1688A, and
not the interlaboratory report:

	In Section 5.0 on Safety, pages 5-7, some explicit mention should be
made about the use of appropriate biohazard containment for samples
which are biosolids, as these are likely to host pathogens which are a
potential danger until rendered harmless by disinfection.  This would
include preparation of raw samples in a biosafety cabinet until
pathogens are killed (e.g., by soaking in solvents), appropriate safety
garments and emergency procedures for accidental spills of pathogenic
material, and warning about covering all materials containing solvents
in biosafety cabinets to prevent solvent fumes that can pose an
explosion hazard in biosafety cabinets that do not vent to exterior air.

IV.  ADDITIONAL REFERENCES

Mills, S.A., III; Thal, D. A., Barney J. (2007) “A summary of the 209
PCB congener nomenclature”, Chemosphere 60, 1603-1612.

#####

Peer Review Comments From

KURUNTHACHALAM KANNAN

Peer Review Comments on:  Draft EPA Method 1668A Interlaboratory
Validation Study

NAME:  Kurunthachalam Kannan, Ph.D.

AFFILIATION:  Wadsworth Center, New York State Department of Health &
Professor, Department of Environmental Health Sciences. State University
of New York at Albany. Empire State Plaza PO Box 509. Albany, NY
12201-0509.

DATE:  March 24, 2010

Review of “Method 1668A Interlaboratory Validation Study Report” and
“Addendum to the Method 1668A Interlaboratory Validation Study Report
Draft, August 2009.

Method validation is an integral part of a good analytical laboratory
practice.  The method 1668A validation study was intended to provide the
Environmental Protection Agency (EPA) with sufficient amount of data to
evaluate the performance of the method.  The method 1668A was originally
validated in 2000, in a single laboratory study.  However, the
guidelines published by the EPA and American Society for Testing and
Materials (ASTM) recommend a minimum of six complete datasets for
validating a method.  The objectives of the validation study were to
characterize the performance of the Method 1668A in multiple
laboratories and matrices and to evaluate and revise the Quality Control
(QC) acceptance criteria.  The objectives of the study are sound and
logical.  The multi-laboratory validation study is a requirement to meet
the national and international criteria to ensure reliability,
consistency, and accuracy of the analytical data generated using the
Method, especially when the method is used in the EPA’s Clean Water
Act programs.  The major outcome of the validation study was a revision
to the quality control/quality acceptance (QAQC) criteria in EPA Method
1668A.  

It is agreed that method validation studies are time and resource
consuming and EPA’s effort to perform this validation is highly
commendable.  Coordination of activities among several participating,
volunteer, laboratories is a daunting task.  EPA’s efforts on this
method validation are significant; nevertheless, data generated from
this study are under-utilized and with the existing data it is possible
to generate additional information required for validation.  

1. 	Please review EPA's Report of the Method 1668A Interlaboratory
Validation Study and its addendum. Comment on the objectives of the
study, the soundness of the technical approach, and the results
produced.

The objectives of the study are sound and logical.  The multi-laboratory
validation study is a requirement to meet the national and international
criteria to ensure reliability, consistency, and accuracy of the
analytical data generated using the Method, especially when the method
is used in the EPA’s Clean Water Act programs.  The major outcome of
the validation study was a revision to the quality control/quality
acceptance (QAQC) criteria in EPA Method 1668A.  

The overall technical approach, ranging from selection of laboratories,
sample selection, sample analysis, data reporting, data review, and
validation, is good, although there are some potential areas for
improvements.  A major drawback with the report is on the biosolids
dataset.  The objective of the study was to have 6 complete datasets, as
per the EPA/ASTM guidelines, to validate the method.  Only 4 complete
datasets have been available for biosolids (Table 4-1), although
validation for biosolids appears to be an important goal for the method,
as indicated on the title.  Furthermore, the data submitted by the 4
laboratories for biosolids are highly variable and no ‘in-depth’
statistical analysis of dataset has been made to determine accuracy and
precision of the data submitted for biosolids. In other words, the
Method has not adequately met the objectives to validate the analysis of
PCBs in biosolids.   Many international organizations such as the
International Organization for Standardization (ISO) require at least
eight complete datasets to validate the methods.  Because the dataset
available for biosolids does not meet the EPA/ASTM guidelines,
validation of the Method 1668A for biosolids is incomplete/inadequate.

The second major concern with the validation study results is that the
data analysis/review is focused heavily (or only) on precision, by
measuring variability in concentrations reported by the laboratories for
PCB congeners.  Little attention has been paid to the accuracy of the
results.  Only for wastewater samples, recovery values are presented as
a measure of accuracy (only for wastewater, the spiked concentrations
were known and therefore the recovery could be measured).  Accuracy is
an important criterion in method validation.  There are several ways to
measure accuracy, without increasing the burden on the laboratories. 
Accuracy could have been assessed by analyzing a sample of known
concentration, as a laboratory control material, and comparing the
measured value to the true value. National Institute of Standards and
Technology (NIST) reference materials are available for PCBs and
laboratories could have been supplied or asked to analyze NIST reference
materials, along with samples supplied by the EPA.  It has been stated
that ‘to minimize the burden on volunteer participants’,
laboratories were not required to analyze initial precision and recovery
(IPR) samples.  Nevertheless, accuracy is an important parameter for
validating accuracy of a method, a well-characterized matrix such as
NIST SRMs can be used.  Alternatively, the technique of standard
additions, which is used to determine recovery of spiked congeners from
on-going precision and recovery (OPR) samples (sand/oil and water) could
have been used (Results from Table 4-4) to validate the accuracy of the
method.   

The third issue on results: The method 1668A is intended for congener
specific analysis of PCBs with an emphasis on toxic congeners (twelve
non- and mono-ortho substituted PCBs).  The scope of the method 1668A
indicates that twelve PCB congeners designated as ‘toxic’ by the
World Health Organization (WHO) are determined.  Nevertheless, the
validation study is focused on ten homologue groups.  The data analyses
involved summation of all isomer data within a congener group. 
Precision and accuracy of the Method 1668A for the analysis of
individual toxic congeners, are not validated.  In other words, the
method is not validated for 12 toxic PCB congeners.  Furthermore, the
suitability of this method for the analysis of mono- and
di-chlorobiphenyls (CBs) is questionable for the reasons given below. 
First, the precision and accuracy for mono- and di-CBs are very poor, if
not dismal.  For example, mean recovery of mono- and di-CBs in
wastewater samples were 3 and 54%, respectively.  Furthermore, the
concentration mono-CB found in wastewater sample is similar to what was
found in blanks (mean for wastewater is 27 pg/L [Table 4-1] and that for
water blank is 26 pg/L [Table 4-2]).  Obviously, the QC criteria have
been relaxed to reflect the poor recoveries (Addendum to the method
validation study).  Overall, the method did not perform well for mono-
and di-CBs.  If these two congener groups are removed, then the QC
criteria for the method would be tighter and more reliable/reproducible.
  These two congeners are easily lost in the solvent evaporation steps,
as has been stated in several places of the method validation report. 

A few other issues on the results, such as robustness, ruggedness, and
limit of detection have not been discussed or analyzed in detail.  A
method validation would require that the above mentioned parameters are
tested and validated.  Some of the information presented on the
validation report is not clear (e.g., section 4.2).  For example, how
were the outliers detected?  How many outliers were found in samples? 
Rationale for removal of the outliers from dataset is missing.  A column
showing number of outlier values should be presented on Table 4-1.  The
reproducibility variation coefficients (between-laboratory precision and
within-laboratory precision) for the method were not reported.  Overall,
data have not been fully utilized to describe several parameters that
are required in validating a method (based on the comment 2 above and
this comment).

2. 	Please render an opinion as to suitability for use of the data from
the study and the addendum for revision of the QCAC in Method 1668 to
make it Method 1668B. 

I believe that considerable effort has been put forward in validating
the method 1668A.  Nevertheless, the data from the validation study are
under-utilized and more important validation parameters such as accuracy
of the method, reproducibility, ruggedness, etc should be evaluated. 
These issues and corrections should be corrected/clarified.  Once these
revisions are made, the method may be revised to 1668B.  However, I
believe that there is lot more work to be done.

3. 	If you have a suggestion for an improvement of the interlaboratory
study report or addendum, please provide suggested text by section
number & 

4.       If you believe that a given item should be clarified, please
provide the clarification by section number or define the clarification
needed to a sufficient level of detail that we can make the necessary
changes &

5.      Please point out errors or omissions.

I feel that the charge questions 3, 4 and 5 are inter-related.  The
following items would help improve the study report and addendum

Page 3, Section 2.4 (Sample Selection):  Some description of samples is
needed.  For example, does the fish tissue represent wholebody
homogenate or fillet?  Also, please specify when these fish samples were
collected?  I assume that these are wet samples (i.e., not
freeze-dried).  Similarly for biosolids, some sample description is
needed.  After reading the entire document, I realized that these are
wet samples.  It would be useful to indicate on page 3 (section 2.4),
what biosolids really are and when they were collected; also state that
they are wet and if possible indicate the moisture content?  

Did the wastewater effluent samples contain suspended solids?  When
aliquoting wastewater samples, how was the homogeneity in the
distribution of suspended solids between aliquots ensured?  Wastewater
samples have been spiked with PCB congeners.  It appears that there was
no PCB present in wastewater sample prior to spiking.  Was that true?  I
suspect that PCBs can be detected in wastewater samples at several tens
to hundreds of pg/L level.    The information on the spike
concentrations listed on Table 2-2 is not clear.  What was the solvent
that was spiked into water?

Page 5, Section 2.6:  Did all the participating laboratories use HRMS? 
The Method 1668A calls for the use of HRMS and the method validation
requires the use HRMS.  The deviations from the Method 1668A have been
listed for certain parameters (section 2.7 on page 6), but not sure if
all of the laboratories used HRMS for the analysis.  Also, on page 5, it
has been stated that the laboratories were instructed to follow the
Method 1668A, it will be helpful to mention/confirm that on Section 3
(Data Review and Validation) that all laboratories indeed followed the
Method 1668A (other than the deviations noted on section 2.7). 

Page 6, line 3:  Expand “SOW”; the abbreviation is used here for the
first time.

Page 6; Laboratory reporting of rules:  One important item that is
missing is the conversion of data from ‘wet weight’ to ‘dry
weight’ and vice versa (for biosolids).  

A major issue with the method is lies in the weight of samples used for
extraction (especially biosolids) and final extract volume.  I think the
method should be modified to incorporate some flexibility because the
weight used for extraction can vary depending on the concentrations,
presence of interferences, etc.  Also, concentration to a final volume
of 20 uL prior to injection can introduce large variability in
analytical results.  I would prefer to use a sample weight of 15 g for
biosolids and a final volume of 100 or 200 uL.  I hope that the Method
1668B incorporates these changes to reflect some amount of flexibility
in the weight of the samples used for extraction and the final extract
volume.

Page 11, First paragraph, line 9:  Expand “ML”

Table 4-1, Page 12, column #7 (Mean), third row from the bottom; 241-???
 Something is missing here.  It looks like a typo.  Is there a value
after ‘241’?  Also, footnote 1 for the table:  specify whether
biosolid and fish data are given on wet weight or dry weight basis.  For
footnote 2; insert “sum of” between ‘represent’ and ‘all’.

Section 4.3.;  Were the blanks analyzed similar to samples in all
aspects, especially the concentration factor (final extract volume)? 
This should be mentioned in section 4.3.

Page 13, Table 4-2:  “# Labs” for sand/oil should be “6”;
“10” is a misleading number here because “10” is the number of
datasets available, not ‘# Labs”.

Section 4.4:  Low recoveries of  mono- and di-CBs have been reported to
be due to loss during transport  or as ‘unknown’.  I would expect
that it is mainly due to loss of these volatile congeners in the solvent
evaporation steps in Method 1668A.

Section 5.5:  “(Table 5-1).”  Move the ‘.” Out of the
parentheses.

Section 6.  I am little concerned about the statement that ‘the result
of this interlab study met our objectives to characterize……”.  The
results have not satisfied the criteria to validate the method for
biosolids, if we follow the guidelines that we need dataset from 6
laboratories.

Corrections on addendum:  Table 4-1, footnote’ Biosolids
concentrations in ng/kg, wet weight, not ‘ng/L’.  Please correct. 
Also, the labeled compound recoveries are too low for a good method. 
The low recoveries are only for mono and di-CBs and if these two
congeners are removed or reported separately, the recoveries are more
than 50% for all other congeners.  I suggest that recoveries for mono-
and di-CB congeners are reported separately and tri-deca CB are reported
separately.  A 15 or 20% recovery is too low to consider a method to be
a good method. 

6. 	Please provide any additional comments you would like to make
regarding the documents under review.

 Although 14 laboratories were invited to participate (on a voluntary
basis), only 6 labs have provided results for fish and wastewater and 4
labs for biosolids.   Participation of a minimum of six labs is required
for a method validation study.  It is disappointing to note that several
laboratories have expressed dissatisfaction with method and problems in
analysis.  Inviting some laboratories for participation with some
(minimum) payment, and use of  NIST SRM samples for accuracy analysis
would enhance the quality of method validation. Because the method is
intended for use in EPA’s Clean Water Act programs, I would strongly
recommend that contract laboratories analyzing clean water act samples
should participate in this validation study.

I would suggest that mono and di-CBs are reported separately or removed
from the method.  The QC criteria for these two congener groups should
be reported separately from the other congeners.

Participating laboratories should report twelve toxic congeners
separately and QC criteria for these congeners should be established; in
other words, the method validation is needed for these 12 toxic
congeners.

A major issue with biosolids analysis lies in the use of 30 g of sample
weight for extraction.  Similarly, concentration of samples to a final
volume of 20 uL, can impose serious variability issues.  I would
recommend that the method be amended to incorporate some flexibility in
the weight of the samples taken for extraction.  For biosolids, it can
be from 10 to 30 g (wet weight).  Also, concentration of extract to 20
uL, will introduce large variability.  I would suggest that the final
volume can be from 20 uL to 200 uL.  I would recommend using sediments
for method validation.

 American Society for Testing and Materials.  1998.  Annual Book of ASTM
Standards, Vol. 11.01. Standard Practice for Determination of Precision
and Bias of Applicable Test Methods of Committee D-19 on Water, ASTM
D2777-98, October 1998.

United States	Office of Water  DRAFT Reviews & Responses 

Environmental Protection Agency	(4303)	April 2010

 PAGE   

 

  PAGE  ii 

United States	Office of Water

Environmental Protection Agency	(4303)	April 2010

Method 1668A Interlab Study Peer Review

April 2010	  PAGE  iv 

 PAGE   47 

April 2010	

Method 1668A Peer Review

October 2007		  PAGE  45