## Validation and Peer Review of U.S. Environmental Protection Agency Chemical Methods of Analysis

Prepared for: 

## The EPA Forum on Environmental Measurements (FEM) Prepared by: 

### The FEM Method Validation Team FEM Document Number 2005-01 October 14, 2005
Principal Authors:
Eliz ab eth A . Mis h alan ie, Ph .D . Barry Lesnik Roy Araki Robin Se gall National Enforcem ent Office of S o lid W a s t e US EPA Region 10 Office of Air Quality Planning and Investigations Center Washington, DC Seattle, W a shington Standards

Denver, Colorado Res earch Triangle P ark, North Carolina Contributors:
David J. M unc h M i ke Wasko Eliz abeth Flynt Office of Water US EPA Region 4 Office of P e s ticide P r ogram s Office of Groundwater Athens, Georgia S t ennis S p ace Center, M i s s i s s i ppi

and Drinking Water

Cincinnati, Ohio
S t even C . Wen d e lk en , Ph .D . Je nny Sc ifr e s Joh n R. Mario Office of Groundwater US EPA Region 4 Suffolk County Crime Laboratory

and Drinking Water Athens, Georgia Hauppauge, New York

Cincinnati, Ohio

Margo Hunt Diane Gregg Teri S c h a effer Office of Environmental Information US EPA Region 6 Versar, Inc. Washington, DC Houston, Texas Springfield, Virginia Mered ith Clarage M . Susan Lumpkin US EPA Region 6 National Exposure Research Laboratory

Houston, Texas Res earch Triangle P ark, North CarolinaValidation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### Disclaimer This docum ent contains direction developed so lely to provide internal guidance to U.S. Envi r onme n t a l Protection Agency (EPA) personnel. EPA retains the discretion to adopt approaches that differ from this guidance on a case-by-case basis. The guidance set forth in this docum ent does not create any rights, substantive or procedural, en forceable at law by a party to litigation with EPA or the United States. The use of m a ndatory language such as "must" and "require" in this guidance docum ent reflects sound scientific practice and should not be construed to create any legal rights or requirem e nts. The use of non-m a ndatory language such as "m ay," "can," or "should" in this guidance does not connote a requirem e nt but does i ndicate EPA’s strongly preferred approach for validating and peer reviewing EPA m e thods prior to publication for general use.

iValidation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### Forew ord The U.S. EPA Science Policy Council established the Forum on Environm ental Measurem ents (FEM) in April 2003. The FEM is a standing com m ittee of senior EPA m a nagers who provide EPA and the public w ith a focal point for addressing m easurem ent and m e thods issues with mu ltiprogram impact. Action team s are com m i ssioned by the FEM to address specif i c issues. The Method Validation Team was form ed in Oc tober 2003, and tasked with developing Agencywide, internal guidance for validating and peer reviewing EPA m e thods prior to publication for general use. This docum ent contains guidance for chem ical m e thods of analysis. It is the first in a series of Agency-wide, m e thod validation guidance docum ents. Subsequent guidance will be developed for other types of m easurem ent m e thods (e.g., m i crobiology, biology, toxicology, radiochem i stry, and field procedures).

iiValidation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### Executive Summary EPA program offices publish a wide variet y of m easurem ent m e thods for use by EPA personnel, other governm e nt agencies, and the priv ate sector. These m e thods m a y originate from ma ny s our c e s s u c ha s EPA l abor a t ori e s , EPA c ontr a c t or s , s c i e nt i f i c or ga ni z a t ions, other governme nt laboratories, and from the privat e sector. Since these m e thods m a y be published as regulations, incorporated by reference in regulations, or publis hed as guidance, they mu st be thoroughly tested and peer reviewed prior to publication as EPA m e thods. This docum ent provides Agency-wide guidan ce for EPA personnel who will evaluate the perform a nce and suitability of new chemical m e thods of analysis before EPA publication. The m e thod validation principles contained herein are based on current, international approaches and guidelines for intralaboratory (single laboratory) and interlaboratory (multiple laboratory) m e thod validation studies. Peer review is required for EP A chemical m e thods of analysis. The EPA Science Policy Council Peer Review Handbook provides Agen cy-wide requirem e nts and options for that process.

iiiValidation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### Table of Contents Disclaimer
............................................................................................................................... ........ i Forew ord
............................................................................................................................... ......... ii Executive Summary
..................................................................................................................... iii 1 Introduction
........................................................................................................................ 1 1.1 Purpose
....................................................................................................................... . 1 1.2 Intended Audience
...................................................................................................... 1 1.3 Scope of Guidance
...................................................................................................... 1 1.4 Term inology
................................................................................................................ 2 2 Planning and Initiating Method Validation Studies
....................................................... 3 3 Method Description
........................................................................................................... 4 3.1 Procedures Composing a Method
............................................................................... 4 3.2 Method Purpose, Scope, and Applicability
................................................................ 4 4 Method Performance Characteristics
.............................................................................. 5 4.1 Selectivity
................................................................................................................... 6 4.2 Instrum e nt Calibration
................................................................................................ 6 4.3 Bias/Trueness
.............................................................................................................. 7 4.4 Precision
...................................................................................................................... 8 4.5 Quantitation Limits and Range
................................................................................... 9 4.6 Detection Lim it(s)
....................................................................................................... 9 5 Ruggedness Testing
.......................................................................................................... 10 6 Interlaboratory Studies
................................................................................................... 11 7 Method Validation Reports
............................................................................................. 12 8 Peer Review
...................................................................................................................... 13 9 Terms and Definitions
..................................................................................................... 14 10 References
......................................................................................................................... 18 TABLE 1 Method Validation Report Topic Areas
............................................................................. 12

ivValidation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 1 Introduction Method validation is the process of dem onstrating that an analytical m e thod is suitable for its intended use, and involves conducting a variety of studies to evaluate m e thod perform a nce under defined conditions ( 1 ). Method validation studies m a y involve a single laboratory (intralaboratory) or m u ltiple laboratories (interlaboratory). The goal is to dem onstrate that an alytical results produced by the application of a particular m e thod are fit for an intended purpose ( 1 ). Properly designed and successful m e thod validation studies create confiden ce in the reliability of a test m e thod. Method validation is one of several important quality syst em components that are designed to ensure the production of scientifically valid and useful analytical data ( 2 ). EPA’s chem ical m e thods of analysis are de signed for a wide variety of m e asurem ent objectives, such as m onitoring comp liance with environm ental regulations, enforcem ent of environm ental regulations, and for inform ation-gath ering scientific studies. Because m easurem en t objectives and data quality requirem e nts vary, th e technical details and acceptance criteria for conducting m e thod validation studies are containe d in existing, program -specific EPA docum ents r a th e r th a n i n a n Age n c y - w i d e gui da nc e . The r e f ore, this guidance docum ent describes fundam e ntal m e thod validation principles and general areas to be addressed when validati ng chemical me thods of analysis. This inform ation is based on current, international approaches and guidelines for single and mu ltiple laboratory m e thod validation studies. Before publication, all chemical m e thods of analysis also m u st be peer reviewed. The reader is referred to the m o st current version of the EPA Science Policy Council Peer Review Handbook ( 3 ), an excellent source of inform ation for both internal and external peer review processes. 1.1 Purpose The purpose of this guidance is to (a) describe scientific principles that should be addressed during m e thod validation studies, and (b) harm onize term s and definitions related to analytical chem istry m e thod validation studies and m e thod perform a nce characteristics. 1.2 Intended Audience This guidance is intended for internal use by EPA personnel who are responsible for chem ical m e thods of analysis, as defined in sec tion 1.3 of this docum ent, that either are to be published as serially num bered EPA m e thods, or published as, or incorporated by reference in, regulations. 1.3 Scope of Guidance This guidance contains recomme ndations for validating new, rational, quantitative, chemical m e thods of analysis, intended for use in analytical laboratories. These term s are def i ned as f o llow s :

1Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005

• “ N e w ” me a n s th a tth e me thod is not currently published as an EPA m e thod, or it is an update of a published EPA m e thod that includes pro cedural changes that will affect m e thod perform a nce.

• “Rational,” as opposed to empirical, m eans that the analytical results are not intended to be m e thod dependent. A rational m e thod of analysis is a m e thod that determines one or m o re identifiable chem icals or analytes for which there m a y be several equivalent m e thods of analysis available. Empirical m e thods determin e a value that can be arrived at only in term s of the m e thod per se and serves, by definition, as the only m e thod for establishing the m easurand 1

. This guidance does not explicitly address em pirical m e thods, which are used to estim ate m e thod-defined param e ters. Exam ples of rational m e thods are Method 8260 (Volatile Organic Compounds by Gas Chro m a tography/Mass Spectrom etry, SW -846), Method 200.8 (Determ i nation of Trace Elem ents in W a ters and W a stes by Inductively Coupled Plasm a -Mass Spectrom e try), and Method 26 (Determi nation of Hydrogen Chloride Em issions from Stationary Sources, 40 CFR Pa rt 60, Appendix A). Examples of empirical m e thods are Method 1311 (Toxicity Characteris tic Leaching Procedure, SW -846), Method 1664 (Oil and Grease), and Method 5 (Determi nati on of Particulate Matter Emissions from Stationary Sources, 40 CFR Part 60, Appendix A).

• “Quantitative” m eans that the m e thod is intended to produce num erical results (analyte concentration or other quantity) and an estim at e of the associated m easurem ent uncertainty ( 4 ).

• “Chemi cal m e thods of analysis, intended for use in analytical laboratories” m eans that the m e thod prim arily is intended for use in a laboratory to m easure chem ical properties. Although som e m e thods m a y include samp ling procedures (e.g., air Methods TO-15 and 26, solid waste Method 5035A), validation of field pro cedures is not explicitly addressed in this guidance. Much of the inform ation provided m a y be usef ul for validating other types of analytical m e thods such as empirical, qualitative, or field pr ocedures. However, these types of m e thods will have unique considerations that will require sli ghtly dif f

e rent validation approaches f r om those presented in this docum ent. 1.4 Terminology Scientif ic term s and m eanings change with tim e. For m a ny years, national and international standards organizations have attemp ted to harm oni ze term inology within scien tif ic disciplines. F o r the purpose of this guidance, a glossary of term s and definitions is included in Section 9. The entries were comp iled from current, authoritative sources, and references are included.

Rational and empirical methods of analysis are defin ed and discussed in an IUPAC technical report by Thompson et al. The report was published in Pure & Applied Chemistry, Vol. 71, No. 2, pp. 337-348, 1999.
2 1Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 2 Planning and Initiating Method Validation Studies Proper planning is critical for successful m e thod validation studies. The developm ent of a chemical m e thod of analysis m a y require m onths or years, and the form al validation phases should not be pursued until substantial evidence exists th at a m e thod is a good candidate for publication as an EPA m e thod. EPA scientists are relied upon to determine which m e thods are good candidates, and it is not the intention of this guidance to provide acceptance criteria for new m e thods. Method developm ent and m e thod validation studies generally are not viewed as com p letely separate processes. Method developm ent, ofte n a comp lex iterative process, is not explicitly addressed in this guidance. However, there are m a ny sim ilarities between the types of experim e nts conducted during m e thod developm ent and the types of tests perform e d during m e thod validation. Typically, an intralaboratory m e thod validation study is initiated when a detailed pro c edu r e has been written, operational param e ters are believed to be optimized and f i nalized, and no additional changes are anticipated. On occasi on, difficulties are encountered after beginning an intralaboratory study, and it is necessary to tr oubleshoot and optimize a m e thod. If procedural c h a nge s ar e ma de , a nd th e c h a nges m a y affect the m e thod perform a nce characteristics, then som e or all of the perform a nce tests m a y need to be repeated. An interlaboratory m e thod validation study is initiated only after a succe ssful intralab o r ato ry study has been comp leted. Unfortunately, som e m e thod perform a nce difficulties m a y only be revealed after initiating a mu ltiple laboratory study, and it m a y be necessary to troubleshoot and optim ize a m e thod after conducting such a study. If procedural changes are required, the changes m a y affect the m e thod perform a nce characteristics a nd som e or all aspects of an interlaboratory study m a y need to be repeated.

3Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 3 Method Description 3.1 Procedures Composing a Method The m easurem ent process components subjected to validation should be clearly described. For example, a m e thod m a y include procedures fo r laboratory subsam pling, subsam ple preparation, and instrum e ntal analysis. In this case, the m e thod validation effort should include an evaluation of all three m easurem ent process components com b ine d. Som e chem ical m e thods of analysis also include procedures for samp ling (e.g., m e thods for air analysis and for volatile organic compounds in soils and other solid m a trices), in which case the samp ling procedures should be included in the validation tests. In contrast, m a ny SW -846 m e thods (EPA publication “Tes t Methods f o r Evaluating Solid W a ste: Physical/Chem i cal Methods”) are separate procedures for sam p le preparation, sam p le cleanup, or instrum e ntal analysis. The SW -846 m e thods m a y be validated and used in various com b inations to obtain the final analytical re sults. In any case, the m easurem ent process components validated, and the com b ination of pro cedures com posing a m e thod, m u st be clearly and comp letely stated. 3.2 Method Purpose, Scope, and Applicability The purpose of a m e thod (i.e., m easurem ent objectiv es) and the intended use of the data m u st be clearly defined. In addition, m e thod scope a nd applicability mu st be well defined and clearly described
. This helps m i nim i ze m i sapplications by the user. Method scope and applicability includes the f o llowing:

 The m easurem ent process components validated
 The nature of the analytes and m a trices studied
 The range of analyte levels for which the m e thod is claim e d to be suitable
 A description of any known limitations and a ny assump tions upon which a m e thod is based
 A description of how the m e thod and analytical param e ters chosen m e et the data quality objectives for the specific application

4Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 4 Method Performance Characteristics The purpose, scope, and applicability determ ine the m e thod perform a nce characteristics chosen for study. The accuracy of a m e thod, with respect to the m a terials and conditions studied during validation, m a y be evaluated from the perfor m a nce characteristics. T h e m eaning of the term “accuracy” has changed over the years, and accur acy should be viewed as a qualitative concept rather than a synonym for bias ( 4, 5 ). Accuracy is defined as the closeness of agreem ent between a m easured value and either a true or accepted refere nce value. The followin g ex cerp t f

r o m re f e ren c e 4 (pages 167-168) provides a clear explanation of the current use of the term accuracy. Part 1 of the international standard ISO 5725- 1 on the accuracy of m easurem e nt m e thods and results defines accuracy as the closeness of agreem ent between a test result and the accepted reference value. This definition is supplem ented by a note which states that the term accuracy when applied to a set of test results involves a com b ination of random components and a com m on sy stem atic error or bias component. Accuracy is thus viewed as a characteristic of a m easurem e nt process consisting of precision as well as bias components. A process is considered to be accurate only if it is precise as well as unbiased. A m easurem e nt process that generates repeat observations of a characteristic which are not close enough to one another cannot generate an accurate result. I n fact, a new word has been coined to replace the word bias. Bias is now characterized by the attribute “trueness.” Trueness indicates the positive aspect of a m easurem ent process, in contrast to bias, which indicated a negative aspect. The old concept of accuracy has been changed to m ean trueness. The greater the bias of a m easurem e nt process, the less true it is. The term accuracy is now used in a broader context which includes both precision and trueness. Many tim es a m easurem ent result is derived as an average of re peat m easurem ent observations rathe r than a s i ngle observation. When the result is an average, it is lik ely that it is close to the conventional true value b u t the repeat m easurem ents are not close enough to one another. Such a m easurem ent process m ay be considered to be true but not accurate, as the process is not precise. Based on current trends in classifying m e thod validation study components ( 2 ), the perform a nce characteristics 2 that should be evaluated include, but are not lim ited to:

 Selectivity
 Instrum e nt calibration
 Bias/Trueness
 Precision
 Quantitation lim its and range
 Detection lim its “Sensitivity” is not listed in this guidance as a m e thod perform a nce characteristic. The scientif ic literature ref l ects dif f

e rent m eanings and uses f o r this term
. It is used to describe a perform a nce characteristic of a detector, instrum e nt, or an analytical m e thod. However, it is possible to evaluate any of these sensitivity param e ters f r om data and inf o rm ation obtained by studying the perform a nce characteristics listed above.
Most performance characteristics listed in this sec tion are also studied during method development. However, experiments designed for method development purposes are typically not performed under final operating conditions.
5 2Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 Before validating an analytical m e thod, it is necessary to ensure that proper samp le preservation and storage stability studies were perform e d during m e thod developm ent. Storage stability mu st investigate the stab ility of the analyte(s) f r om the tim e of sam p ling through the tim e of analysis. If an extraction is performed, the extract stability m u st also be investigated. Analytes m a y be lost through volatilization, sorption, chem ical de gradation (abiotic reactions), and mi crobial degradation. Traditionally, preservation to prevent chem ical degradation is occasionally evaluated but preservation to prevent m i crobial degradation is generally ignored. Microorganism s have the potential to degrade target analytes and represent a significant pathway for the fate and destruction of organic compounds. 4.1 Selectivity Selectivity ref e rs to the ability to correctly iden tif y the analyte(s) of interest in the presence of expected chemical/physical interferences. Se lectivity m a y be evaluated by testing appropriate m a trix blanks and fortified (spiked) m a trix bla nks. Matrix blanks should contain the chem ical s p e c i es, other than the analyte(s) of interest, which are reasonably expected to be present in an actual samp le. A worst-case samp le m a trix should be te sted, as defined in the scope of a m e thod, during both intra- and interlaboratory validation studies. In som e cases, adequate representations of m a tr ix blanks cannot be obtai ned or prepared. In such situations, actual sam p les of m a terial cont aining an undetermined am ount of native analyte m ay need to be studied to evaluate m e thod selectivity. This usually requires the use of mu ltiple analytical procedures (e.g., samp le preparation, wet-chem istry, and/or instrum e ntal analysis procedures) during an intralaboratory validation study to substantiate the selectivity of the m e thod being evaluated. A com b ination of procedures or techniques that are based on different m easurem ent principles should be used. This appr oach increases the likelihood that interferences will be identif ied. Method selectivity is typically expressed qualitatively. A qualitative selectivity statem ent includes a description of known interferences, interfer ence effects, and the nature of the analytical data and inf o rm ation that substantiates the identity of the analyte(s) in the m a trix of concern (e.g., elem ental or m o lecular structure data, retention tim es from chrom a tographic separations, selective reaction chemistry, and results from reference standards, reference m a terials, m a trix blanks, other blanks, or m a trix fortifications). Quantitative m easures of selectivity m a y also be used, although there is no generally accepted approach for the quantitative treatm e nt of selectivity data ( 2, 6). Therefore, the basis for quantitative selectivity m easures should be thoroughly described. 4.2 Instrument Calibration I n s tru m e nt calibration refers to the procedures used for correlating instrum e nt response to an am ount of analyte (concentration or other quantity). The characteristics of a calibration f u n c tio nan d justification for a selected calibration m odel s hould be dem onstrated during an intralaboratory m e thod validation study. The perform a nce of a calibration technique and the choice of calibration m odel (e.g., firstorder linear, curvilinear, or nonlinear) are critical fo r mi nimi zing sources of instrum e nt bias and

6Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 optim izing precision. A calibration m odel is a m a th em atical function that relates composition to instrum e nt response. The param e ters of the m odel are usually estim ated from the responses of known, pure analytes. Calibration errors can result from failure to identify the best calibration m odel; inaccurate estim ates of the param e ters of the m odel; or inadequately studied, system atic effects from m a trix components. During m e thod developm ent and validation, calibration m odels are typically evaluated by analyzing mu ltiple levels of calibration standards over a selected working range. Measurem ents should be replicated at each calibrant level to eval uate the random error associated with instrum e nt response, and to evaluate goodness-of-fit for vari ous linear, curvilinear, or non-linear m odels. For example, if a m e thod states that an average re sponse factor m a y be used for calibration over a specified analyte range, then the instrum e nt res ponse should be dem onstrate d, during intralaboratory m e thod validation, to be a first-order linear equation, a nd that the y-intercept is zero within error. The calibration study results should be included in a m e thod validation report. Access to inform ation about calibration perform a nce charact eristics assists the user in implem enting new m e thods and verifying that a laboratory’s instrum e nt perform a nce is acceptable. As m e ntioned above, the calibration approach st ated in a m e thod shoul d be thoroughly tested and justified during an intralaboratory validation study, and tested according to the required/recom m e nded calibration procedure duri ng an interlaboratory validation study. For exam ple, an intralaboratory study m a y include severa l calibration standards and replicate readings of each standard. If the calibration perform a n ce characteristics are deem ed acceptable, the required/recomm e nded calibration approach m a y include the use of only three calibration standards, single readings of each standard, and the calculation of an average response factor. In this case, the interlaboratory study would involve the use of an average response factor as opposed to the full calibration approach used during the intralaboratory study. In any case, the instrum e nt calibration procedure used in the final EPA m e thod should be thoroughly studied, justified, and docum ented in a m e thod validation report. 4.3 Bias/Trueness Bi a s r e fe r s t o th e ove r a l l ma gni tu de of known syst em atic (determi nate) errors associated with the use of an analytical m e thod. The presence of system atic errors can only be determ ined by comp arison of the average of m a ny results with a reliable, accepted reference value. Method bias m a y be estim ated by m easuring m a terials whos e composition is reasonably well known, such as reference m a terials, by com p aring results to those fr om at l e a s t one a l t e rn at e me th od or procedure, or by analyzing spiked m a terials. The relationship between the term s bias and trueness is clearly described in ISO 5725-1 ( 5 ). Two excerpts from this docum ent are as follows: The “trueness” of a m easurem e nt m e thod is of inter est when it is possible to conceive of a true value for the property being m easured. Although, for som e m easurem e nt m e thods, the true value cannot be known exactly , it m a y be possible to ha ve an accepted reference value for the property being m easured; for example, if suitable refere nce m a terials are available, or if the accepted reference value can be established by reference to another m easurem e nt m e thod or by preparation

7Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005
of a known sample. The truene s s of the m e a s ur e m e n t m e thod c a n be inve s tigat e dby c o m p aring the accepted reference value with the level of the resu lts given by the m easurem e nt m e thod. Trueness is norm a lly expressed in term s of bias. Bias can ar ise, for example, in chem ical analy s is if the m easurem e nt m e thod fails to extract all of an elem ent, or if the presence of one elem ent interferes with the determ ination of another. The term bias has been in use for statistical m a tters for a very long tim e, but because it caused certain philosophical objections am ong m e m b ers of som e professions (such as m e dical and legal practitioners), the positive aspect has been emphasized by the invention of the term trueness. It should be noted that all approaches for ev aluating trueness have limitations. Examples of these lim itations are as f o llows:

• The ref e rence m a terials approach introduces the important uncertainty of m a trix m a tching, and the reliability of a bias estim ate depends upon the relationship betw een the composition of the reference m a terial and the samp les.

• W h en using the alternate m e thod approac h, the reliability of a bias estim ate is dependent upon how mu ch is known about th e perform a nce characteristics of the alternate m e thod.

• The m a trix spiking approach introduces uncertainty regarding the behavior of spiked m a terials, comp ared to m a terials containing native analyte. This m a y be particularly problem atic for solid m a terials. Rigorous evaluations of trueness should be included in EPA m e thod validation studies. Minim a lly, trueness should be evalua ted at the extrem es of the quantitation range stated in the scope of a m e thod, and at regulatory levels. 4.4 Precision The general term “precision” is used to desc ribe the m a gnitude of random (indeterminate) errors associated with the use of an analytical m e thod. The sources of random error evaluated depend upon the range of conditions over which the data are collected. The following definitions are recomm e nded ( 7 ).

 Repeatability (of results of m easurem ents) - closene ss of agreem ent betw een the results of successive m easurem ents of the sam e m easurand carried out under the sam e conditions of m easurem ent. These conditions are called re peatability conditions. R e peatability conditions include the sam e m easurem ent procedure, obs erver, m easuring instrum e nt (used under the sam e conditions), location, (and) repetition over a short period of tim e. Repeatability m ay be expressed quantitatively in term s of the dispersion characteristics of the results.

 Reproducibility (of r e sul t s of m easurem ents) - closeness of agreem ent between the results of m easurem ents of the sam e m easurand carried out under changed conditions of meas u r emen t. A valid statem ent of reproducibility requires sp ecification of the conditions changed. The

8Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 changed conditions m a y include principle of m easurem ent, m e thod of m e asurem ent, observer, m easuring instrum e nt, reference standard, location, conditions of use, (and) tim e. Reproducibility m a y be expressed quantitativ ely interm s o f th e d i s p ers i o nch aracteris tics o f the results. Minim a lly, m e thod repeatability and reproducibility should be evaluated at the extrem es of the quantitation range stated in the scope of a m e thod, and mu st be evaluated at regulatory levels. (The quantitation range mu st include regulatory levels; see section 4.5.) Com m on m e asures of di spersion are the standard deviation and relative standard deviation of repeated m easurem ents. The repeatability and reproducibility conditions should be clearly stated so that the m easures of dispersion can be properly interpreted and evaluated. Precision and trueness evaluations, known as accuracy studies, are often perform e d concurrently. A variety of test m a terials m a y be used for these studies, and the effects of laboratory subsampling should be included when appropriate. 4.5 Quantitation Limits and Range The term “quantitation range” is used to d e s c ri b e th e s p a no f a n a l y t e le v e ls , a s c o n t a i n e dina samp le m a trix, for which m e thod perform a nce has been tested, and data quality is deem ed acceptable for its intended use. A quantitation range mu st include either a regulatory or other type of action level. A lower lim it and an upper lim it bound a quantitation range. A quantitation range m a y be wider than an instrum e nt’s calibration range becau se of dilution or concentration steps perform e d during sam p le preparation. Dilution factors or concen tration factors are used to relate the calibration range to the quantitation range. Analyte concentration will have an effect on m o st m e thod perform a nce characteristics, including trueness a nd precision. At a m i nimu m , m e thod trueness and precision (as described in sections 4.3 and 4.4) should be evaluated at the extrem es of the quantitation range. The lower limit of the quantitation range is com m only ref e rred to as the “lim it of quantitation” or LOQ. There are num erous def i nitions in the scientif ic literature f o r this lim it, and no s i ngl e de fi ni t i on i s appr opr i a te fo r all EPA chemical m e thods of analysis. Defined limits, such as those based on specifications for precision, should be verified by testing m ate rials containing analyte at the claim e d LOQ. 4.6 Detection Limit(s) The term “detection limit” is usedto describe the lowest analyte level that can be conf idently identified. There are m a ny specific definitions for this term , and it is used to describe the detection capabilities of detectors, instrum e nts, and analyti cal m e thods. The term “detection limit” mu st be defined, and a description of how it was evalua ted during m e thod validation mu st be provided. Lim its derived from m a them atical definitions or statistical m odels mu st be verified by testing m a terials containing analyte at the claim e d detection level.

9Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 5 Ruggedness Testing Ruggedness refers to the extent to which an analytical m e thod rem a ins unaffected by m i nor variations in operating conditions. Ruggedness tes ting involves experim e ntal designs for examining m e thod perform a nce when mi nor changes are m a de in operating or environm ental conditions. The changes should reflect expected, reasonable variations that are likely to be encountered in different laboratories. A comm only used, efficient approach for a rugge dness test is based on a fractional factorial experim e ntal design 3 ( 8, 9, 10 ). This approach is m o re efficient than varying conditions one at a tim e. Various types of precision studies m a y also re veal the effects of som e influential factors. Analyst observations are also important, and m a y reveal critical control points that should be docum ented in a m e thod. Ruggedness testing m a y be conducted at the end of a m e thod developm ent effort, or during an intralaboratory study. Regardless of timing, ruggedness testing should be comp leted before initiating an interlaboratory m e thod validation study, and critical control points should be identif ied in the docum ented m e thod.

The fractional factorial experimental design referen ced herein is not intended for method optimization purposes.
10 3Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 6 Interlaboratory Studies Interlaboratory studies determine whether an an alytical m e thod can be transf erred f o r use in other laboratories and used for regulatory testi ng. A m e thod that proved rugged for use in one laboratory m a y lose that characteristic when tried in several other laboratories. Therefore, an interlaboratory m e thod validation study should be com p leted prior to publishing an EPA m e thod f o r ge ne r a l use . The e x t e ntof th e study will vary depending on available resources and the intended use and applicability of the m e thod. These studies ar e resource intensive, and should not be initiated until a successful intralaboratory study has been com p leted and the issuing entity has dem onstrated a pressing programm a tic need for publishing a national m e thod. An interlaboratory m e thod validation study is a practical testing of the written m e thod on i d e n t i c a l ma t e r i als, usually derived from split sam p les, by a num ber of laboratories. The study is not intended to evaluate laboratories; it is intended to evaluate m e thod reproducibility am ong laboratories. Deviations from the interlaboratory study protocol should be strongly discouraged, and any deviations that occur should be docum ented. There are several excellent references for the design and conduct of interlaboratory m e thod validation studies. Three such references are listed in section 10 of this docum ent ( 11, 12, 13 ).

11Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 7 Method Validation Reports A m e thod validation report, suitable for placing in the public docket, should be prepared. The report should address the m e thod validation topics outlined in this guidance docum ent ( s ummar i z e di n Tabl e 1) , a nd ( a ) background inform ation on m e thod developm ent, (b) a description of the actual m e thod validation techniques, (c) any changes m a de to the m e thod as a result of the validation studies, and (d) any recom m e ndations for future work Table 1 M e t hod Validation Re por tTopic Ar e a s Topic Explanation Pu rp os e The purpose of the method is a clear and concise descrip tion of the measurement objec tives and the intended use of the data.
Sc ope and Applicability The m e thod scope and applicability serve to define th e range of m e thod perform ance studies. The scope and applicability includes (a) the m easurem ent process compone nts validated, (b) the nature of the analy t es and matrices studied, (c) the range of analy t e levels for whic h the method is claimed to be suitable, (d) a description of any known limitations and any assumptions upon which a method is based, and (e) a description of how the m e thod and analy tical param e ters chosen m eet the data quality objectives for the specific application.
Selectivity Selectivity is a perform ance characteristic that dem onstrates the ability of the m e thod to y i eld useful data for the analy t es, analy t e levels, and matrices de fined within the scope of the method. Selectivity is especially im p o rtan t at action levels of concern, and mustbe demonstrated in the presence of species known or predicted to present analy tical challenges to the method. Ty pically , selec tivity is demonstrated by providing information that substantiates the identity of th e analy t es in the presence of expected m a trix constituents. Instr u me nt Calibration This perform ance characteris tic is s o m e tim es referred to as “ i ns trum ent linearity .” In a broader s e ns e, it addresses instrument calibra tion and the nature of the resulting calib ration model. Duri ng method validation, sufficient data should be generated to demonstrate and justify the method-recommended calibration approach.

Bias /Tru en es s Trueness is a performance characteristic that addresses sources of known sy stem atic error, and bias is a measure of trueness. The extent of bias in a m eas urem ent proces s ty pically varies with analy t e, analy t e level, and m a trix
. Therefore, trueness should be evaluate d using samples that span the degree of analy tical difficulty encompassed by the specified scope and applicability
. Examples of approaches us ed to assess trueness include the use of reference materials, alternate methods or techniques, and matrix fortification. Precis i on (Repeatability and Reproducibility) Precision is a performance characteristic that reflects sources of random error in a measurement process. There ar e se ve ra l diffe re nt ty pe s of pre c i sion studie s (i. e . , re pe a t ab ility a nd re produc ibility studies ). M e thods des i gned for dem onstrating comp liance with regulatory requirem e nts should be evaluated for both repeatability (withinlab) and reproducibility (am ong labs). Quantitation Range and Limits A quantitation range corresponds to the range of analy t e concentrations (or other quantity ) characterized for m easurem ent accuracy (trueness and precision) during m e thod validation. Analy tical m easurem ent uncertainty should be well characterized over a quantitation range.

Detection Limits This general term describes the lowest level of anal y t e that can be identified with confidence. A method validation study ty pically involves a specific detection lim it definition, and includes an explanation of how the defined detection lim it was estim ated. Detection lim it es tim ates derived from m a th ematical definitions or s t atis tics mu s tbe verified by anal y z ing samples containing analy t e at th e claimed detection level. This perform ance characteris tic m a y not be important for all an aly tical applications, and its relevance depends on the m e thod scope and applicability
. Ruggedness Ruggedness refers to the capacity of an analy tical method to remain unaffected by sm all variations in operating conditions. Ruggedness testing involve s experimental designs for examin ing method performance when small changes are made in operating or environmental conditi ons. The changes should reflect expected, reasonable variations that are likely to be encountered in different laboratories. Interlaboratory Study Interlaboratory studies determine whether an analy tical method can be transferred for use in other laboratories and used for regulatory testing. Th e design and results of the interlabor atory study should be included in the method validation summary report. Da ta from the interlaboratory study s hould be reported in tabular form. There should also be a discussion included describing the de tails of, and rationale for, any changes made to the m e thod resulting from the interlaboratory study
.

12Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 8 Peer Review Before publication, EPA m e thods of analysis are peer reviewed in accordance with the inform ation provided in the m o st current versi on of the EPA Science Policy Council Peer Review Handbook ( 3 ). The Peer Review Handbook provides Ag ency-wide guidance for the consistent implem entation of peer review, and Program Of f i ces have the f l exibility to design peer reviews to suit their specific needs. The Handbook is the sour ce of detailed inform ation about which Agency work products are subject to peer review. In a ddition, there is inform ation about selecting peer review m echanism s (internal and external), pl anning a peer review process, conducting a peer review, and preparing a peer review record. Each EPA office has a Peer Review Coordinator available for assistance.

13Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 9 Terms and Definitions Term s and definitions were obtained from the following authoritative sources:

 Am erican National Standards Specif i cation and Guidelines f o r Quality System s f o r Environm ental Technology Program s, Annex A (2004) (ANSI/ASQ)
 International Vocabulary of Basic and General Term s in Metrology, 2 ndEdition, International Organization for Standardization (1993) (VIM)
 Accuracy (trueness and precision) of m easurem ent m e thods and results – P a rt 1: G e neral principles and definitions, International Organization for Standardization, ISO 5725-1, 1994-12-15 (ISO 5725-1) For each term and def i nition, the specif i c ref e rence is indicated in parentheses. N u m b ers identif ying VIM term s and definitions are also included. Th e num bering of the notes from ISO 5725-1 is the sam e as the num bering in the original ISO docum ent. accepted reference value: A value that serves as an agreed up reference for com p arison, and which is derived as: a) a theoretical or established value, based on scientific principles; b) an assigned or certified value, based on experim e ntal work of som e na tional or international organization; c) a consensus or certified value, based on collabor ative experim e ntal wo rk under the auspices of a scientific or engineering group; d) when a), b) and c) are not available, the e xpectation of the (m easurable) quantity, i.e. the m ean of a specified population of m easurem ents. (ISO 5725-1) accuracy: The closeness of agreem ent between a test result and the accepted reference value. (IS O 5725-1) NOT E 2 The term accuracy , when applied to a set of test results, involves a com b ination of random components and a com m on sy stem atic error or bias component. accuracy of measurement: closeness of the agreem ent between the result of a m easurem ent and a true value of the m easurand (VIM 3.5) NOTES 1 “Accuracy ” is a qualitative concept. 2 The term precision should not be used for “accuracy ”. bias: The difference between the expectation of th e test results and an accepted reference value. (ISO 5725-1) NOTE 5 Bias is the total sy stem atic error as contrasted to random error. There m a y be one or m o re sy stem atic error components contributing to the bias. A larger sy stem atic difference from the accepted reference value is reflected by a larger bias value.

14Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005
calibration: set of operations that establishes, under specified conditions, th e relatio n s h i p b e tw een values of quantities indicate db y a m e a s u rin g in s tru m e n t o r m e a s u rin g s y s t e m , o r v a lu e s r e p r e s e n te d by a m a terial m easure or a reference m a terial, a nd the corresponding values realized by standards (VIM 6.11) NOTES 1 The result of a calibration perm its either the assignm ent of values of m easurands to the indications or the determ ination of corr ections with respect to indications. 2 A calibration m ay also determine other m e trological properties such as the effe c t o f i n f l u e n c e quantities. 3 The result of a calibration m a y be recorded in a docum ent, som e tim es called a calibration certificate or a calibration report
. measurand: particular quantity subject to m easurem ent (VIM 2.6) EXAMPLE vapour pressure of a given sample of water at 20 °C. NOTE The specification of a m easurand m a y require statem ents about quantities such as tim e, temp erature and pressure. measurement procedure: set of operations, described specifi cally, used in the perform a nce of particular m easurem ents according to a given m e thod (VIM 2.5) NOTE A m easurem e nt procedure is us ua lly re c o rde din a doc um e n t that is s o m e tim e s its e l f c a lle d a “ m e a s urem ent procedure” (or a measurement method ) and is usually in sufficient detail to enable an operator to carry out a m easurem e nt without additional inform ation. method: A body of procedures and techniques for pe rform i ng an activity (e.g., samp ling, chemical analysis, quantif ication) system atically presented in the order in which they are to be executed. (ANSI/ASQ) precision: The closeness of agreem ent between indepe ndent test results obtained under stipulated conditions. (ISO 5725-1) NOTES 9 Precision depends only on the distribution of random errors and does not relate to the true value or the specified value. 10 The m easure of precision is usually expressed in term s of imprecision and computed as a standard deviation of the test results. Less precision is reflected by a larger standard deviation. 1 1 “Independent test results” m eans results obtaine d in a m a nner not influenced by any previ o u s resu l t o n th e same or similar test object. Quan titative m easures of precision depend critically on the stipulated conditions. Repeatability and reproducibility conditions are parti c ul ar s e t s of extrem e conditions. procedure: A specified way to carry out an activity or process. (ANSI/ASQ)
random error: result of a m easurem ent mi nus the m ean that would result from an infinite num ber

15Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005
of m easurem ents of the sam e m easurand carried out under repeatability conditions (VIM 3.13) NOTES 1 Random error is equal to error m i nus sy stem atic error. 2 Because only a finite num ber of m easurem e nts can be m a de, it is possible to determ ine only an estim ate of random error. repeatability (of results of measurements): closeness of the agreem ent between the results of successive m easurem ents of the sam e m easurand carried out under the sam e conditions of m easurem ent (VIM 3.6) NOTES 1 These conditions are called repeatability conditions. 2 Repeatability conditions include:

- the sam e m easurem e nt procedure

- the sam e observer

- the sam e m easuring instrum e nt, used under the sam e conditions

- the sam e location

- repetition over a short period of tim e. 3 Repeatability m a y be expressed quantitatively in term s of the dispersion characteristics of the results. reproducibility (of results of measurements): closeness of the agreem ent between the results of m easurem ents of the sam e m easurand carried out under changed conditions of m easurem ent. (VIM 3.7) NOTES 1 A valid statem ent of reproducibility requires specification of the conditions changed. 2 The changed conditions m a y include:

- principal of m easurem e nt

- m e thod of m easurem e nt

- observer

- m easuring instrum e nt

- reference standard

- location

- conditions of use

- tim e. 3 Reproducibility m a y be expressed quantitatively in term s of th e di s p e r s i on c h ar a c t e ri s ti c s of the results. 4 Results are here usually understood to be corrected results. result of a measurement: value attributed to a m easurand, obtained by m easurem ent (VIM 3.1) NOTES 1 When a result is given, it should be m a de clear whether it refers to:

- the indication

- the uncorrected result

- the corrected result and whether several values are averaged.

16Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005
2 A co mp lete statemen t o f th e resu lt o f a measu r emen t in cludes inform ati on about the uncertainty of m easurem e nt. systematic error: m ean that would result from an infinite num ber of m easurem ents of the sam e m easurand carried out under repeatability conditions m i nus a true value of the m easurand (VIM 3.14) NOTES 1 Sy stem atic error is equal to error m i nus random error. 2 Like true value, sy stem atic error and its causes cannot be comp letely known. 3 For a m easuring instrum e nt, see “bias” (VIM 5.25). true value (of a quantity): value consistent with the definition of a particular quantity (VIM 1.19) NOTES 1 This is a value that would be obtained by a perfect m easurem e nt. 2 True values are by nature indeterm inate. 3 The indefinite article “a”, rather than definite article “the”, is used in conjunction with “true value” because there m a y be m a ny values consistent with the definition of a given particular quantity
. trueness: The closeness of agreem ent between the average value obtained from a large series of test results and an accepted reference value. (ISO 5725-1) NOTES 3 The m easure of trueness is usually expressed in term s of bias. 4 Trueness has been referred to as “accuracy of th e m ean”. This usage is not recom m e nded. uncertainty of measurement: param e ter, associated with th e result of a m easurem ent, that characterizes the dispersion of the values that could reasonably be attributed to the m easurand (VIM 3.9) NOTES 1 The param e ter m a y be, for example, a standard deviation (or a given mu ltiple of it), or the half-width of an interval having a stated level of confidence. 2 Uncertainty of m easurem e nt comp rises, in general, m a ny components. Som e of these components m a y be elevated from the statisti cal distribution of the results of series of m easurem e nts and can be characterized by experim e ntal standard deviations. The other components, which can be characteri zed by standard deviations, are evaluated from assum e d probability distributions based on experience or other inform ation. 3 It is understood that the results of the m easure m e nt is the best estim ate of the value of the m easurand, and that all components of uncertainty , including those arising from sy stem atic effects, such as components associated with co rrections and reference standards, contribute to the dispersion.

17Validation and Peer Review of U.S. EPA Chem ical Methods of Analysis October 14, 2005 

### 10 References 1. Eurachem
. The Fitness for Purpose of Analytical Methods: A Laboratory Guide to Method Validation and Related Topics ; LGC (Teddington) Ltd.: Middlesex, United Kingdom , 1998. (ISBN 0-94892-612-0) 2. Thompson, M.; Ellison, S.L.R.; W ood, R. Harm onized guidelines for single-laboratory va l i dation of m e thods of analysis, IUPAC Technical Report. Pure Appl. Chem. 2002 , 74 (5) , 835-855. 3. EPA Science Policy Council Peer Review Handbook, 2 nded
. ; EPA 100-B-00-001; US Environm ental Protection Agency, Office of Science Policy, Office of Research and Developm ent, W a shington, DC, Decem ber 2000. 4. Kim o thi, S.K. The Uncertainty of Measurements; ASQ Quality Press: Milwaukee, W I , 2002. (ISBN 0-87389-535-5) 5. International Organization for Standardization (ISO). Accuracy (trueness and precision) of measurement methods and resu lts - Part 1: General principles and definitions; ISO 5725-1: 1994(E), Geneva, Switzerland, 1994. 6. Vessm an, J.; Stefan, R.; van Standen, J.; Dan zer, K.; Lindner, W
. ; Burns, D.; Fajgelj, A.; Muller, H. Selectivity in Analytical Chem istry, Pure Appl. Chem. 2001 , 73(8) , 1381-1386. 7. International Organization for St andardi z at ion (ISO). International Vocabulary of Basic and General Terms in Metrology (VIM) , 2 nded., Geneva, Switzerland, 1993. 8. ASTM E 1169-02. Standard Guid e for Conducting Ruggedness Tests; Annu. Book of ASTM Stand. 2003 , (14.02) , 318-324. 9. Youden, W
. J.; Steiner, E.H. Statistical Manual of the AOAC; AOAC: Arlington, VA, 1975. (Fifth printing 1987; ISBN 0-935584-15-3) 10. W e rnim ont, G. Use Of Statistics To Develop And Evaluate Analytical Methods; AOAC: Arlington, VA, 1985. (Fourth printing 1993; ISBN 0-935584-31-5) 11. AOAC International. AOAC® Official Methods SM Program Manual (OMA Program Manual, Part 6, Appendix D: Guidelines for Collaborative Study Procedures to Validate Characteristics of a Method of Analysis ; Gaithersburg, MD, 2002. 12. Horwitz, W
. Protocol for the desi gn, conduct and interpretation of m e thod perform a nce studies: Revised 1994 (Technical Report), Pure Appl. Chem. 1995 , ( 67 ) , 332-343. 13. ASTM E 691-99. Standard Practice for Conduc ting an Interlaboratory Study to Determine the Precision of a Test Method, Annu. Book of ASTM Stand. 2003 , (14.02) , 203-224.

18