Page 1 of 43



### This document is a **review draft** . This information is distributed solely for the purpose of pre

dissemination review under applicable information quality guidelines. It has not been formally disseminated by

EPA. It does not represent and should not be construed to represent any Agency determination or policy . It is being circulated for review of its technical accuracy . EPA Classification No.: **DRA FT**  **F I**  **N A**  **L (2106**  **- S**  **- 0**  **2 .0)**  CIO Approval Da te: CIO Transmittal No.: Review Date: 12/01 / 1 2 Issued by the EPA Chief Information Officer, Pursuant to Delegation 1 - 19, dated 07/07/2005 

## U.S. Environmental Protection A**genc y**  

# QU **AL ITY STA**  **NDA RD FOR ENVIRONMENTA**  **L DA TA**  **COLLECTION , PRODUCTION,**  **AN D USE**  **BY NON**  **- EPA**  **(EXTERNA L)**  **ORG ANIZ** **AT ION**  **S 1.**  **PURPOSE** This Quality Standard for Environmental Data Collection, Production, and Use by Non -EPA ( External ) Organizations (2106 - S - 02.0), including its Annexes:

 defines and establishes the quality requirements and ma nagem ent controls for non - EPA organizations that collect, produce, and use environmental data;  defines how EPA quality requirements in CIO Standard 2106 - S - 01.0, Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations (EPA 2013a ), shall be applied to non - EPA organizations under authorized agreements governed by Federal regulations;  reaffirms the applicability of and conformance to ANSI/ASQ E4 (Current Edition) for the planning, im plem entation, documentation, and assessm ent of extramural programs that collect, produce, and use environmental data;  provides the basis for non - EPA organizations to define effective processes, procedures , and ma nagem ent controls for planning, implementing, and assessing quality activities defin ed in their approved Quality Management Plans (QMPs) for environmental data collection, production, and use;  requires the use of Quality Assurance Project Plans (QAPPs) or equivalent documents to establish quality specifications in individual project activ ities that collect, produce, and use environmental data; identifies expected quality responsibilities for managers and staff in external organizations for activities involving environmental data collection, production, and use ; and  replaces EPA QA/R- 2, EPA Requirements for Quality Management Plans, March 2001 (Reissued May 2006) and EPA QA/R- 5, EPA Requirements for Quality Assurance Project Plans, March 2001 (Reissued May 2006) in t heir entirety with exceptions as specified under Clause 2.3

. This Standa rd includes three annexes which contain detailed specifications and guidance to support the requirements presented in the Standard. Normative annexes are additional requirements beyond those requirements in the body of the Standard and shall be regarded as part of the requirements of thePage 2 of 43

EPA Classification No.: **DRA FT**  **F I**  **N A**  **L (2106**  **- S**  **- 0**  **2 .0)**  CIO Approval Da te: CIO Transmittal No.: Review Date: 12/01 / 1 2 Standard. Informative annexes (or parts thereof) may become requirements if they are cited and/or stated as such in agreements or if so decided by the external organization using the Standard and this decision is documented in the external organization’s QMP.

**2. SCOPE A ND A**  **PPLICABILITY 2.1**  **SCOPE** The scope of this Quality Standard is consistent with EPA Quality policies and procedures, and encompasses EPA expectations for all program activities conducted by external organizations that involve the collection, production, and use of environmental data. It applies to environmental data produced or used through appro ved external agreements. Environmental data may be:  collected from environment media directly in the form of physical samples or from existing electronic databases;  produced from analytical methods or instrumentation, and mathematical or electronic com putational models or programs;  collected from the literature or other existing data sources (e.g., web sites o n the Internet); and  used in a variety of analyses and evaluations to document environmental conditions and to support decisions pertaining to such conditions. Activities involving environmental data encompassed by this Standard include, but are not limited to:  direct and indirect field or laboratory m easurements;  evaluating the operation and performance of environmental technology (e.g. , pollution control ) ;  inspections;  survey development or application;  enforcement and compliance monitoring or assessments;  application of environmental management systems;  environmental safety and health monitoring;  scientific research;  regulatory development;  use of statistical or economic analyses using environmental data;  use of information technology (e.g., the developm ent and use of models such as pollutant transport and ground water migration, databases) supporting Agency programs;  use of information sources outside of direct EPA management controls or authority (e.g., academic institutions); and  use of data obtained from other sources (e.g., literature, Internet). **2.2 AP**  **PLICA BILITY**  **This Standard applies to all non -** **EPA organizations having external agreements with EPA and shall be explicitly applied to all external agreements involving the collection, production, and**  **use of environmental data** 

. The terms and conditions of applicable external agreements shall include this Standard by reference as the requirements for quality management processes, applications, and personnel responsibilities.Page 3 of 43

EPA Classification No.: **DRA FT**  **F I**  **N A**  **L (2106**  **- S**  **- 0**  **2 .0)**  CIO Approval Da te: CIO Transmittal No.: Review Date: 12/01 / 1 2

**2.3 QUA**  **LITY M AN**  **AG EM**  **ENT REQUIREMENTS A**  **PPLICA BLE TO NON**  **- EPA**  **ORGA NIZA** 

**TIONS** Prior to this Standard, the quality management requirements for non - EPA organizations were given in two extramural reference documents :  EPA QA/R- 2, EPA Requirements for Quality Management Plans (March 2001 Edition, re authorized in May 2006); and  EPA QA/R- 5, EPA Requirements for Quality Assurance Project Plans (March 2001 Edition, re authorized in May 2006). **These documents are deleted as requirements for use in new or revised external agreements effective**  **with this Standard, and are replaced by this Standard. 2.3.1 Existing External A** **greements** The use of QA/R- 2 and QA/R- 5 by non - EPA organizations in existing external agreements may be allowed by EPA organizations under one or more of the following circumstances:  the documents have been explicitly incorporated or referenced into an extramural agreement or enforcement/consent agreement and, as a result, are legal requirements;  current Federal and/or EPA regulations explicitly cite the documents;  other EPA policies and orders explicitly cite the documents; or  use of the Uniform Federal Policy for Quality Assurance Project Plans (UFP - QAPP) is explicitly required. **Under these conditions, the requirements of QA**  / **R -**  **2 and QA/R -**  **5 shall replace those of CIO St**  **andard 2106 -**  **S -**  **02 (EPA 2013b**  **) for the term of the particular external agreement, but shall be limited to the defined period of performance of the agreement** 

. **EPA will confirm the appropriate continued use of these deleted documents in existing external agree** **ments. 2.3.2 New and Revised External A** **greements. All new work performed through external agreements shall comply with this Standard.**  Wh en the use of QA/R- 2 and QA/R- 5 is allowed for an external agreement, it is the responsibility of EPA to monitor the im plem entation of the agreement to ensure that criteria for their continued use are met. Such monitoring will be accomplished in accordance with this Standard ( s ee Clause 7.8). If an existing external agreement (i.e., contract Statement of Wo rk , assistance terms and conditions, etc.) is modified by the Agency during its period of performance and the external agreement includes the use of EPA QA/R- 2 and EPA QA/R- 5, the Agenc y shall determine if it is beneficial to the Government to invoke the quality require me nts as included under CIO Standard 2106 - S - 02. If it is determined to be beneficial to the Government, the use of EPA QA/R- 2 and EPA QA/R- 5 shall cease and the requirements of CIO Standard 2106 - S - 02 shall apply and shall be invoked in the modification to the agreement. Existing external or other agreements may be revised by EPA before their end dates to require the elem ents of this Standard in lieu of QA/R- 2 and QA/R- 5. Other Federal/EPA regulations will be revised when practicable to replace QA/R- 2 and QA/R- 5 with the requirements of this Standard.Page 4 of 43

EPA Classification No.: **DRA FT**  **F I**  **N A**  **L (2106**  **- S**  **- 0**  **2 .0)**  CIO Approval Da te: CIO Transmittal No.: Review Date: 12/01 / 1 2

**3. AU**  **DIENCE** The principal audience for this external Quality Standard is all non - EPA organizations and their em ployees (managers and staff) responsible for planning, collecting, assessing, and/or using environmental data and personnel who are assigned specific quality management duties and responsibilities within the non - EPA organization. **4. BA** **CKGROUND** This Standard, along with other applicable EPA standards, and with other relevant regulations, statutes, and external agreement terms and conditions, provides the foundation for the implementation of the EPA Quality Policy among non - EPA organizations performing work through external agreements. Such external agreements include contracts, assistance agreements, and enforcement agreements. This Standard conforms to the current editions of EPA Quality Policy CIO Polic y 2106 (EPA 2006a ) , and Procedure for Quality Policy CIO Procedure 2106 - P - 01 (EPA 2006b )

. Since 1979, EPA has required all extramural org anizations performing work that collects, produces, and uses environmental data to comply with quality requirements defined by EPA Orders. Originally issued in 1984 and reauthorized in 2000, EPA Order 5360.1 A2, Policy and Program Requirements for the Mandatory Agency

- Wide Quality System , required application of the Quality Program to environmental data collected and/or used by EPA and the design, construction, and operation by EPA organizations of environmental technology systems such as pollution cont rol and abatement systems; treatment, storage and disposal systems; and remediation systems. EPA Order 5360.1 A2, which was later re - designated CIO Polic y 2105.0, also clarified that the scope of environmental data includes, in addition to information coll ected directly from measurements, data compiled from other sources (i.e., databases, scientific literature), environmental models, and the performance of environmental technology. EPA Order 5360 A1, EPA Quality Manual for Environmental Programs , was develo ped to provide instructions for im plem enting the requirements of this Policy. This order was subsequently re - designated CIO Procedure 2105 - P - 01.0. The relevant content of CIO Procedure 2105 - P - 01.0 for extramural agreement holders was communicated using two documents that when cited required quality documentation: EPA QA/R- 2 (for QMPs) and EPA/QA R- 5 (for QAPPs). These previous policy and procedure documents required non - EPA organizations having extramural agreements with EPA to develop, document, and imple me nt a Quality Management System (QMS) that conformed to all applicable EPA policies and procedures, with American National Standard ANSI/ASQC E4 - 1994. Specifications and Guidelines for Quality Systems for Environmental Data and Technology Programs and wit h applicable Federal regulations on quality in extramural agreements. ANSI/ASQ E4 is a national consensus standard authorized by the American National Standards Institute (ANSI) and was developed by the American Society for Quality (ASQ)

. 1 The requirement for QMS conformance with ANSI/ASQ E4 (Current Edition) is continued in this Standard

. **5. AUT**  **HORITY** Authorities applicable to this Standard include the current editions of :  Quality Policy, CIO Polic y 2106;  Procedure for Quality Policy, CIO Procedure 2106 - P - 01; and

1 Am erican Society for Quality Control changed to the current American Society for Quality in 1997.Page 5 of 43

EPA Classification No.: **DRA FT**  **F I**  **N A**  **L (2106**  **- S**  **- 0**  **2 .0)**  CIO Approval Da te: CIO Transmittal No.: Review Date: 12/01 / 1 2  Quality Standard for the Collection, Production, and Use of Environmental Data by EPA Organizations, CIO Standard 2106 - S - 01.Page 6 of 43

**6. REL**  **AT ED REGUL**  **AT IONS, POLICIES, A**  **ND PROCEDURES** In addition to the documents listed in Clause 5 above, the following documents may be helpful for the application of this Standard. They contain provisions that, through reference in this text, constitute provisions of this Standard when invoked as part of a statement of work or required by an applicable policy or directive. Since regulations, policy documents, and standards are subject to periodic revision, users of this Standard should apply the current editions of the documents listed below as well as al l citations provided throughout this Standard:

 40 CFR 30, “Grants and Agreements with Institutions of Higher Education, Hospitals, and Other Non - Profit Organizations”;  40 CFR 31, “Uniform Administrative Requirements for Grants and Cooperative Agreements to State and Local Governments”;  40 CFR 35, “State and Local Assistance”;  48 CFR 46, Federal Acquisition Regulations (FAR), “Quality Assurance”;  U.S. EPA, Office of Grants and Debarment, Implementation of Quality Assurance Requirements for Organizations Receiving EPA Financial Assistance ;  U.S. EPA, Office of Acquisition Management, Contracts Management Manual ;  EPA Manual 1610, Interagency Agreement Policies, Procedures and Guidance Manual ;  ANSI/ASQ E4, Quality Systems for Environmental Data and Technology Programs

– Requirements with Guidance for Use , American National Standard, American National Standards Institute and American Society for Quality ;  Circular A- 119, Federal Participation in the Development and Use of Voluntary Consensus Standards and in Conformity Assessment Activities , Office of Management and Budget, 1 998;  National Technology Transfer and Advancement Act of 1995, PL104 - 113, March 1996;  44 U.S.C. Chapter 31 - Records Management by Federal Agencies (Federal Records Act);  44 U.S.C. Chap ter 33 - Disposal of Records (Federal Records Disposal Act);  EPA Policy Peer Review and Peer Involvement at the U.S. Environmental Protection Agency ;  U.S. EPA Science Policy Council Peer Review Handbook ;  U.S. EPA Science Policy Council, A Summary of Ge neral Assessment Factors for Evaluating the Quality of Scientific and Technical Information , EPA 100/B - 03/001, June 2003 ;  U.S. EPA Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity, of Information Disseminated by the Environmental Protection Agency ;  CIO Polic y 2155.1, Records Management ;  U.S. EPA, Assuring the Competency of Environmental Protection Agency Laboratories , Agency Policy Directive, EPA Forum on Environmental Measurements (February 2004);  U.S. EPA, Ensuring the Validity of Agency Methods: Methods Validation and Peer Review Guidelines (chemical methods), Agency Policy Directive Number FEM - 2005 - 01, EPA Forum on Environmental Measurements;  U.S. EPA, Ensuring the Validity of Agency Methods: Methods Validation and Peer Review Guidelines (radiochemical methods), Agenc y Policy Directive Number FEM - 2006 - 01, EPA Forum on Environmental Measurements;  U.S. EPA, Ensuring the Validity of Agency Methods: Methods Validation and Peer Review Guidelines: Sampling Method s for Chemical and Radiochemical Parameters , Agency Polic y Directive Number FEM - 2007 - 02, EPA Forum on Environmental Measurements;  U.S. EPA, Ensuring the Validity of Agency Methods. Validation and Peer Review Guidelines: Microbiological Methods of Analysis, Agenc y Policy Directive Number FEM - 2009 - 01, EPA Forum on Environmental Measurements;  U.S. EPA, Ensuring the Validity of Agency Methods, Validation and Peer Review Guidelines: Methods of Analysis Developed for Emergency Response Situations , Agenc y Policy D irective Number FEM - 2010 - 01, EPA Forum on Environmental Measurements;  U.S. EPA, Polic y to Assure Competency of' Laboratories, Field Sampling, and Other Organizations Generating Environment Measurement Data Under Agency- Funded Acquisitions, Agency Policy Di rective FEM - 2011 - 01,Forum on Environmental Measurements;Page 7 of 43

 U.S. EPA, CIO Data Standards Policy (DSP), Delegation 1 - 19, Environmental Data Standards (EDS), Final Environmental Sampling, Analysis, and Results: Project, Exchange Network Leadership Council Standard EX000002.2;  U.S. EPA, CIO Data Standards Policy (DSP), Delegation 1 - 19, Environmental Data Standards (EDS) , Final Environmental Sampling, Analysis, and Results: Field Activity Data Standard, Exchange Network Leadership Council Standard EX000004.2;  U.S. EPA, CIO Data Standards Policy (DSP), Delegation 1 - 19, Environmental Data Standards (EDS) , F inal Environmental Sampling, Analysis, and Results: Analysis and Results , Exchange Network Leadership Council Standard EX000005.2; and  U.S. EPA, CIO Data Standards Policy (DSP), Delegation 1 - 19, Environmental Data Standards (EDS) , Final Quality Assurance and Quality Control Data Standard, Exchange Network Leadership Council Standard EX000012.2. **7. REQUIREMENTS**  Each organization shall establish and maintain an effective Quality Management System (QMS) for the environmental programs it conducts or directs under the applicable external agreement. The QMS shall include the quality management controls and management elem ents needed for planning, implementing, and assessing the effectiveness of quality processes and procedures applied to the collection, production, and use of environmental data. These elements shall be used in conjunction with the other applicable clau ses of this Standard to formulate a complete QMS. Many of the management elements contained in this Standard are consistent with requirements found in other consensus standards (e.g., ISO 9001 [Current Edition]). Each organization must document their QMS i n an approved QMP which shall:

 identify internal and external factors that are relevant to the organization’s purpose and that affect its ability to achieve the expected outcomes of the QMS;  describe how the organization plans to achieve the expected outcom es of work to be performed, including prevention of undesired results;  describe and affirm management commitment to the QMS;  describe and chart organizational roles, responsibilities, and authorities;  describe how the competence of persons doing work sha ll be determined and maintained;  describe the activities expected to be performed within the scope of the QMS, including documentation of the work and quality procedures to be applied in a QAPP or equivalent document, and the process for its approval and i mp lem entation;  describe the process for addressing QA issues and concerns;  describe how communications related to the QMS scope will be ensured;  describe the control of required quality documentation;  describe how performance will be measured and reported, including internal asse ssments and ma nagem ent reviews;  discuss how problems and nonconformities will be corrected and documented ; and  describe how work conducted through sub - agreements with other organizations will be evaluated

. The QMS shall be planned, established, documented, implemented, and assessed as an integral part of a ma nagem ent system for the organization’s environmental programs. The QMS shall be d ocumented in the organization’s QMP and shall be established and operational before the initiation of the work defined in the external agreement. In some circumstances, a combined QMP and QAPP may be utilized to document these requirements (see Clause 7. 5 .2 )

. The QMP shall be reviewed and approved for implementation by the external organizat ion and then shall be reviewed and approved by EPA.Page 8 of 43

**7.1 MA**  **NA GEMENT REPRESENT** **AT IVE FOR QUA**  **LITY** The organization’s senior management shall designate or appoint a management representative for quality such as Quality Assurance Manager (QA M anager) or Dire ctor of Quality with defined authority that includes determining that the approved QMS is implemented and maintained in accordance with the requirements of this Standard, and who reports to top management on the performance effectiveness of the QMS. This QA Manager functions independently of environmental data operations or model development and has sufficient technical and management expertise and authority to conduct independent oversight in the implementation of the organization’s quality system in the e nvironmental programs of the organization. The roles and responsibilities for all QA personnel must be documented in the organization’s QMP. Additional information about these roles can be found in Clause 8. **7.2 QUA**  **LITY M AN**  **AG EM**  **ENT PLA N (QMP)**  The QMP (or equivalent document) for the organization shall describe when, how, and by whom controls shall be applied to the specific technical or project efforts required in the external agreement, and shall outline how these efforts are planned, implemented, and ass essed. The QMP shall identify and describe activities which directly or indirectly affect quality and shall include as a minimum:

 an identification of policies and standards with which the QMS conforms;  a description of the QMS;  a description of processes to assess the effectiveness of the QMS;  an identification of all organizations and business lines and programs to which the QMP applies;  a description of the organization, management and staff responsibilities and authorities, and quality m anagem ent perso nnel responsibilities and authorities;  a description of processes to ensure personnel competency;  description of documents and records management processes and procedures ;  description of processes involving the development and use of information technology me thods;  description of general processes for planning, implementing, and assessing environmental data collection, production, and use activities ; and  description of processes for including appropriate quality requirements in sub - agreements

. The QMP shal l reflect the current procedures of the organization for its QMS and shall be reviewed and approved by the organization and by EPA for implementation. The QMP shall address all applicable parts of this Standard and shall identify why any parts are not appl icable. Organizations should apply a “graded approach” to identify and define other types of acceptab le quality documents to be used. A graded approach generally involves basing the level of managerial controls applied to an item or work according to the intended use of the results and the degree of confidence needed in the quality of the results. Organizations that use a graded approach should describe the basis for their approach in the organization’s QMP. QMPs may

be approved by EPA for no longer than the term of the agreement or five years. I f the QMS is mo dified as a result of a reorganization or assessment corrective action, the QMP must be revised and resubmitted for approval by the organization’s senior management or delegated representative and th e EPA CIO or delegated representative. The specific requirements for the content of QMPs are contained in Annex A of this Standard and must be addressed by the organization. Advice on how to satisfy the QMP requirements may be found in Handbook for Prepar

ing Quality Management Plans (EPA 201 3 c).Page 9 of 43

**7.3 MA**  **NA GEMENT REVIEW A**  **ND REPORTING** At regular intervals or as required by the terms and conditions of the external agreement, management shall evaluate the QMS to confirm its ongoing effectiveness for the work to be performed and its adherence to policies, procedures, and standards included or referenced in the external agreement. The QMS shall be revised to reflect organizational, operational, and polic y changes identified during this review or as otherwise determined by management. The changes shall be documented in a revised QMP and shall be reviewed and approved by EPA in accordance with this Standard. **7.4 PERSONNEL COMPETENCE**  Personnel shall have the competence (i.e., technical skills, demonstrated knowledge, and experience) needed to perform assigned duties. Objective evidence of personnel job proficiency shall be documented and maintained for the duration of the project or activity affected ( or longer if required by statute or polic y) and documented in the QMP or QAPP as appropriate

. **7. 5**  **QUA LITY A** **SSURA NCE PR** **OJECT PL AN**  **(QA PP)**  The QAPP or equivalent document shall document the project goals and objectives, and shall define the QA procedures, QC specifications, and other technical activities that must be implemented to ensure that the results of the project or task to be performe d will m eet project objectives. The organization shall prepare a QAPP, or equivalent quality document, for all applicable projects and tasks involving environmental data collection, production, and use as listed previously in Clause 2.1. **7.5 .1**  **General Req uirements**  All QAPPs and equivalent documents shall be reviewed and approved by the organization’s QA Manager and their management (as applicable) before submission to EPA. The EPA QA Manager , or authorized representative as defined in the QMP, must then r eview and approve the QAPPs

. Moreover, all QAPPs must be approved prior to any data collection, production, or use except under circumstances explicitly defined in the terms and conditions of the external agreement. Under these circumstances, the organiza tion will use or modify existing standard operating procedures (SOPs) and other quality- related documents available to support such activities. This complete collection of QA documents must be submitted to the EPA QA Manager together with a detailed explan ation of the circumstances under which this emergency quality documentation was produced . This explanation must also address potential limitations in the interpretation of resulting data and information. The organization shall ensure that QAPPs are kept c urrent for the term of the agreement . W hen mo difications are needed, the changes shall b e made in a timely manner and communicated to all appropriate personnel. For mu lti - year projects, the organization’s project officer shall confirm at least an nually the continued relevance and applicability of the QAPP(s) . If a revision is needed, the review and approval shall be conducted in the same manner as the original documents, including review and approval by EPA. Other forms of project- level quality planning documents which are equivalent to an EPA QAPP shall be described with their review and approval processes in the organization’s QMP. Other QAPP formats (e.g. , UFP - QAPP) which satisfy the requirements of this Standard and its Annex B , and are approved by EPA, should also be descr ibed in the organization’s QMP. The QAPP sh all describe the project- specific activities and proposed timeframe to accomplish the work defined in the terms and conditions of the agreement. The QAPP’s period of approval shall be defin ed by EPA but not exceed the life of the agreement or five years

. The specific requirements for QAPPs are contained in Annex B of this Standard and must be addressed by the organization in all individual QAPPs. Advice on how to satisfy the QAPP requirements in this Standard may be found in Handbook for Developing Quality Assurance Project Plans (EPA 2013 d).Page 10 of 43

**7. 5**  **.2 Combined QMP**  **- QA**  **PP** In some circumstances, EPA may recommend the preparation of a combined QMP and QAPP as a more appropriate means to document org anizational level and project- level environmental data activities in external agreements, particularly when a full QMP is not needed. The specific elements to be addressed are included in the terms and conditions of the external agreement and are typically taken from Clause 7.2, Clause 7. 5 .1 , and Annex B; however, the EPA organization shall determine the final list of elements to be addressed in the combined QMP - QAPP. As a combined QMP - QAPP is essentially an expanded QAPP, it is subject to the same review a s any other QAPP . As it contains QMP elements , it m ust also be approved by the organization ’ s QA M anager. The combined QMP - QAPP is subject to EPA review and approval prior to the start of work. A combined QMP - QAPP should not be used for external agreements containing multiple projects to be determined over the life of the agreement (e.g., mission support contracts or program grants, etc.). **7. 6**  **USE OF INFORM AT**  **ION TECHNOLOGY METHODS A ND SOURCES**  **7. 6**  **.1 General Requirements**  Environmental data encompassed by this Standard may include results from the use of information technology (IT) methods and sources, such as services that collect and provide access to data and information. Such sources of these environmental data may incl ude:

 com puter and mathematical models and the use of other electronic methods ;  information systems ; and  the I nternet. Organizations shall develop and document in their QMP the general processes, procedures, and personnel responsibilities for evaluating the suitability of environmental data obtained from information technology sources. Project- specific processes, procedures, and evaluation criteria shall be developed and documented in a QAPP or other equivalent quality documentation as defined in the orga nization’s QMP. Environmental data from the use of IT methods and sources shall be assessed by the user to confirm its suitability. Any limitations or restrictions on the use of such data shall be documented and reported with the data (see Clause 7.9)

. El ectronic media upgrades or use of new software shall further require that all historical information is carried forward to meet retention schedule requirements and that original entries are maintained (e.g., results of calculations are consistent ) , informa tion is not lost or corrupted , and that the information remains accessible. **7.6 .2**  **Computer Models, Mathematical Models, and Use of**  **Other Electronic Methods** The development, evaluation, and application of mathematical or computerized models of environment al processes and conditions require QAPPs. Environmental data produced from computer models, ma them atical models, and produced by using other electronic methods (or software) shall be reviewed, evaluated, verified, and validated in order to confirm their s uitability for use ( s ee Clause 7. 9 and Annex B). The specific process and procedure shall be documented in applicable quality documentation and shall be reviewed and approved for use. **7. 6.3**  **Other Information Sources, Systems, and the Internet** Quality principles applied to environmental data collected from other information sources (including the Internet) shall require special attention (e.g., due diligence) to ensure that the data can be used for their intended purpose. Project specific evaluation cri teria shall be developed and documented in the applicable QAPP. Data standards and metadata shall be used where applicable and documented to ensure that supplied data retain their meaning for appropriate interpretation by the end - user.Page 11 of 43

An y lim itations, suc h as inconsistenc y with EPA environmental data standards requirements, or restrictions on the use of such data shall be documented and reported with the data. **7. 6**  **. 4**  **Control of Computer Software and Hardware** The organization shall ensure that computer software and hardware relevant to the mission and objectives of the organization are controlled appropriately and satisfy the organization’s objectives. The organization shall describe in its QMP the processes an d the roles, responsibilities, and authorities of ma nagem ent and staff for the control of computer software and hardware. (See also Annex A Clause A3.9)

. **7. 7**  **SYSTEM AT**  **IC PL ANNING**  An y activities involving the collection, production, or use of environmental data shall be planned, im plem ented, controlled and documented using a s ys tematic planning process to develop acceptance or performance criteria for all work (a plan - do - check - act approach) . At a minimum, this process shall:

 identify all relevant c ustomers and their expectations;  identify the technical and quality goals that meet the needs and expectations of the customer ;  translate the technical and quality goals into specifications that will produce the desired result;  consider any cost and schedule const raints within which project activities are required to be performed; identify m easurable acceptance criteria or performance measures by which the results will be evaluated and the suitability of the environmental data for their intended use is determined; and  result in a documented plan for collection, production, or use of environmental data. The general planning processes (including applicable design steps) shall be described in the organization’s QMP. All planning documentation shall be reviewed and app roved for implementation by authorized personnel before the affected planned work commences. Such documentation includes confirmation of conformance with EPA environmental data standards, work plans, schedules, SOPs (or references to where they may be obta ined) , and QAPPs. In some cases, it may be appropriate to subject general planning documents to independent review (e.g., internal or external peer review) in order to confirm their suitability. The reviews of other planning documents do not replace the re quired reviews of quality documents (e.g., QMPs, QAPPs). Additional information about the s ys tematic planning process can be found in Annex C. **7.8 AS**  **SESSMENT ,**  **OVERSIGHT ,**  **AN D RESPONSE**  Management shall determine during planning the level of routine oversig ht appropriate for the particular project and shall assign oversight responsibilities to project personnel qualified to perform this oversight function. Personnel having authority to issue stop -work orders shall be identified and documented in the applicab le QAPP. During this oversight attention should be given to interim assessments of data quality and the QA/QC used to en sure the integrity of the data. Assessment findings shall be communicated to the QA Manager of the organization in a written report. No nconforming methods or instruments or unsatisfactory interim assessments of data quality shall be reported to the organization responsible for them and on a timely basis to ensure the implementation of effective corrective actions. Wh en warranted by findings of deficiency or nonconformity with requirements (e.g., deviation from approved protocols, poor performance of the measurement systems), appropriate corrective actions shall be taken promptly. The adequacy and effectiveness of the corrective actio ns shall be confirmed, verified, communicated to participants, and documented. Results obtained from nonconforming methods or instruments shall be evaluated to determine the impact of the nonconformity on the quality of the data. The adequacy and effective ness of corrective actions taken shall be confirmed, verified, and documented.Page 12 of 43

Clause 7.3 requires review of the QMS. Internal assessment and response should include any results of project- level oversight that affect programs covered by the QMS. **7.9 D AT**  **A REVIEW, VERIFICA**  **TION A ND VA**  **LIDA TION, A**  **ND D AT**  **A U**  **SA BILITY REPORTING**  **7.9.1 General Requirements** Results obtained from products or services involving environmental data shall be reviewed, verified, validated, and qualified according to their intended use. T he general processes and roles and responsibilities of personnel relative to the assessment and verification of data usability shall be described in the organization’s QMP. Specific processes and roles and responsibilities of personnel pertaining to assess me nt of individual projects shall be described in the applicable QAPPs. The data shall be evaluated according to approved and documented procedures and shall be qualified appropriately before use. Any limitations or restrictions on the use of the data sha ll be expressed quantitatively to the extent practicable and shall be documented in any reporting of the data. Environmental data shall satisfy applicable data reporting requirements for data definition and source reporting and for review and certificatio n (when necessary). An y project reports that contain data, or report the results of environmental data operations (e.g., data summaries), shall be reviewed independently (i.e., by others than those who produced the data or the reports) to confirm that the data or results are presented correctly. These project reports must have a readily identifiable section that describes the applied QA and QC and discu sses the usability of the data. The reports shall be reviewed internally by the responsible project manager and QA Manager (or authorized representative defined in the QMP) to confirm that the report’s conclusions are supported by sound science and that the data quality objectives have b een met. This review shall be completed and any issues resolved prior to submission to EPA or any applicable external peer review, release, publication, or dissemination. **7.9.2 Peer Review** Organizations may

use peer review as an assessment tool. Peer rev iew is a documented critical review of a specific scientific or technical work product. Wh en applied, peer review shall be planned, implemented, and documented in accordance with the organization’s peer review policies and procedures (EPA may apply its pee r review process at interim or final stages of the project). For modeling projects, peer review is an important tool. To maximize its value to modeling projects, peer review and advice should begin as early in the conceptual development, modification, or application phase for models as possible (EPA 2009 ). Peer review, and in particular external peer review, involves significant time and resources so allocations must be incorporated into components of the project planning and any contracts. Peer review in the early stages of model development , modification, or application helps to evaluate the conceptual basis of models and potentially save time by redirecting misguided initiatives, identifying alternative approaches, or providing strong technical support for EPA positions. Peer review in the later stages of model development is useful as an independent external review of model c ode (i.e., model verification). **7.9.3 Review of Data for Potential Use**  Environmental data collected from sources that did not use a QMS equivalent to this Standard or whose quality is unknown or undocumented shall be reviewed and evaluated to confirm their suitability for use for this particular project

. D ata collected for purposes other than that of the current project shall als o be reviewed and evaluated for their suitability for use.Page 13 of 43

These data, which sometimes have been called “data for secondary use” or simply (but inaccurately) “secondary data,” shall be assessed according to approved and documented procedures and shall be q ualified appropriately before use. Som e of the elements in the review of environmental data for potential use include:  the need to satisfy requirements for objectivity, utility, and integrity;  the principal traditional indicators of environmental quality (e.g., precision, bias, representativeness, comparability, completeness, and sensitivity); and  the general usability factors (e.g., relevance, temporal compatibility, spatial inclusivity, credibility, aggregation level, compatibility, reproducibility, and degree of augmentation). These elements shall be compared to the requirements established by the acceptance criteria defined in the approved QAPP ( s ee Annex B and EPA 2013d )

. The documentation of how well these elements met the acceptance plan then become s part of the metadata that would accompany the data under consideration if accepted for use. If the data are not acceptable, the information regarding that determination shall be documented . If the data are only marginally acceptable, the rationale for an y decisions derived from the use of such data shall be documented

. **7. 10**  **DOCUMENTS A ND RECORDS M**  **AN AG**  **EM ENT**  Sufficient records shall be specified, prepared, reviewed, authenticated, and maintained to reflect the achieved level of quality for completed work and/or to fulfill all records requirements as needed by EPA. The processes for maintenance of records shall be documented and shall include provisions for retention, protection, preservation, traceability, and retrieval. W here evidentiary records are invo lved, the ma intenance of records shall also include establishing and implementing appropriate management controls, tracking, chain of custody, and confidentiality procedures (including the handling of information identified as sensitive or confidential for the affected records). Retention times for records shall be determined based on external agreement and statutory requirements and protected from damage and deterioration. **8. ROLES A**  **ND RESPONSIBILITIES 8.1**  **GENERA L M**  **AN AG**  **EMENT R OLES A**  **ND RESPONSIBIL ITIES**  The leadership (or senior management) of the organization is responsible for implementing a QMS that covers all environmental programs for which the management is accountable. Senior ma nagem ent includes the principal executive responsible and accountable for m ission accomplishment and overall operations of the organization. The QMP must be approved by the senior ma nager of the organization and be approved and signed by the QA M anager of the organization. The senior ma nager may require the concurrence on the QMP by appropriate subordinate line managers to encourage the acceptance and implementation of the QMS. The QMP and the QAPP shall be approved by EPA before the start of work. The following clauses list specific roles and responsibilities for quality management personnel and ma nagers. **8.2 QU AL**  **ITY M AN**  **AG EM**  **E NT PERSONNEL**  Quality management personnel (e.g., QA Manager, Quality Assurance Officer , Quality Assurance Coordinator) refers to individuals within the organization who are assigned specific quality management duties and are delegated authority for quality management as defined in the organization’s QMP.Page 14 of 43

The organization may assign and deploy other quality management personnel to support the day- to - day quality functions listed below. In some situations, such quality staff may be delegated specific quality ma nagem ent responsibilities as described in this Standard, including approval of quality documentation such as QMPs, QAPPs, SOPs, etc. A description of the coordination and deployment of all quality ma nagem ent personnel, including their roles and responsibilities, shall be documented in the organization’s QMP. The functions of the quality management personnel may be totally related to quality managemen t system activities or be in conjunction with other functions and responsibilities within the organization. If these personnel have other concurrent functions to perform, there must be no conflict of interest. Specific duties and responsibilities of all qu ality management personnel shall be documented in the organization’s QMP. Quality management personnel shall:  review and approve QMPs, QAPPs, and other quality documents prior to submission to EPA;  provide oversight of QA and QC implementation in the environm ental programs conducted by or for the organization; and  perform assessments of implemented environmental programs when needed to confirm the effective application of QC and QA practices, report results to management, and confirm the effectiveness of co rrective actions. **8.3 TECHNICA L M**  **AN AG**  **E RS**  **AND ST AF**  **F** Technical managers at all levels shall:  ensure that all organizational components and programs comply fully with the requirements of this Standard;  ensure that all work specified in the applicable external agreement for which the manager is responsible complies fully with the requirements of this Standard; and  perform all other applicable quality management roles and responsibilities assigned to them in their organization’s QMP. Staff members shall :  ensure that all work specified in the applicable external agreement for which the staff member is responsible complies fully with the requirements of this Standard;  ensure that the results of environmental programs are of sufficient quantity and adequate qual ity for their intended use; and  perform all other quality management roles and responsibilities assigned to them in their organization’s QMPs. **9. DEFINITIONS**  **Ap proved**  : The documented determination that the proposed quality document is suitable for the intended purpose and meets the requirements specified in the applicable Quality Standard. **Assessment:** The evaluation process used to measure the performance or effectiveness of a system and its elements. As used here, assessment is an all - inclusive te rm used to denote any of the following: audit, performance evaluation, management review, peer review, inspection, or surveillance. **Chief Information Officer (CIO):**  The Assi stant Administrator for the OEI (AA/OEI) who is also the EPA Senior Management Official for Quality. **CIO Procedure:** The required steps, course of action, or processes needed to accomplish or satisfy a policy. **Competence:** The ability to apply knowledge, skills, and experience to an activity to achieve intended results.Page 15 of 43

**Conformity:** The fulfillment of a requirement. **Corrective A ction:**  An action to eliminate the cause of a nonconformity or undesirable situation and to prevent recurrence. **Data Standard:** A documented consensus - based agreement on the format and definition of data. **Data Usability:** The process of determining and ensuring that the quality of the data produced meets the intended use of the data. **Environmental Data :**  An y data or information pertaining to the environment that describe measured outputs from processes; environmental conditions in a specific location; ecological effects and consequences; health effects and consequences; biological, chemical, and radiologic al conditions; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as databases, information s ys tems, literature, o r the I nternet. **Environmental Data Collection:** The process of acquiring or gathering environmental data through various means including (but not limited to) sampling and analysis activities, retrieval from information systems , literature, and received fro m EPA partners and the regulated community. **Environmental Data Operations:** The work performed to collect, produce, or use environmental data. **Environmental Data Production:** The process of generating environmental data through various means including (but not limited to) the use of measurement instrumentation, information technology, computer mo dels, and data analysis tools (e.g., statistics, risk assessment methods). **Environmental Programs:** The activities involving the environment, including, but not lim ited to: characterization of environmental processes and conditions; environmental monitoring; environmental research and development; the design, construction, and operation of environmental technologies; and laboratory operations on environmental samples

. **Equivalent Document:** A set of documents that contains all the information and management controls (signatures) as the required documents used in the Standard. **Extramural A greement:**  A legal agreement between EPA and a non - EPA organization for the acquisition of items or services by EPA or financial assistance to a non - EPA organization. Such agreements include acquisition agreements (e.g., contracts, work assignments, delivery orders, task orders), assistance agreements (e.g., cooperative agreements, research grants, State and local grants), and EPA-funded Interagency Agreements (IAs) with other governmental entities. **Graded A pproach:**  The process of basing the level of application of ma nagerial controls applied to an item or work according to the intended use of the results and the degree of confidence needed in the quality of the results. **Guidance:** A non - ma ndatory compilation of advice and example procedures and documentation , based on past experience. Guidance may supplement CIO procedures. **Handbook:** A non - ma ndatory book of instruction , guidance, or information reference manual. **Information:** For purposes of this Standard, information means any communication or representation of knowledge such as facts or data, in any medium or form, including, but not limited to, textual, numerical, graphic, cartographic, narrative, or audiovisual forms.Page 16 of 43

**Information System:** An organized collection, storage, and presentation s ys tem of data for de cision ma king, progress reporting, and for planning and evaluation of programs. **Information Technology:** The study, design, development, implementation, support, or management of com puter- based information systems, particularly software applications and com puter hardware. **Integrity (Information):** Assurance that the information is protected from unauthorized access or change and is not compromised through corruption or falsification. **Life Cycle:** The life span of a product or service from its initial planning and development, to its use and ma intenance, and to its final closure or disposal. **Ma nagement Controls:**  A system of management functions to enable managers to determine that the operations of a n organization satisfy predetermined goals and object ives, that performance is in line with standards and specifications, and to implement any remedial actions needed to ensure that human and other resources are being used in the most effective and efficient way. **Ma nagement System**  **:** A system to establish policy and objectives and to achieve those objectives (ISO 9001). A management system may describe the policies, objectives, principles, authority, responsibilities, accountability, and implementation plan of an organization for cond ucting work and producing products and services. Management systems include ISO 9001 on quality management, ISO 14001 on environmental management, and OHSAS 18000 on occupational health and safety. **Ma terial Transfer A** **greement (MT A)**  **:** A contract that govern s the transfer of tangible research materials between two organizations when the recipient intends to use it for its own research purposes. **Me tadata:**  The information about data required to facilitate its use, understanding and management. For example, metadata should answer questions about data such as why they were collected, how they were collected and by whom, what was done to the data, what they were used for, what were their limitations, and what were the acceptance criteria. **Normative A nnex:**  A part of a Standard that gives provisions (requirements) additional to those in the body of the Standard. A normative annex is considered to be a part of the Standard. **Normative Element:** An element that describes the scope of a Standard and which sets o ut provisions (requirements) of the Standard. **Organization:** A company, corporation, firm, enterprise, or institution, or part thereof, incorporated or not, public or private, that has its own functions and administration. **Objectivity (Information):** The a ssurance that information is presented in an accurate, clear, complete, and unbiased manner, and, as a matter of substance, is accurate, reliable, and unbiased. **Policy:** A high - level statement about an Agenc y requirement designed to influence and determine decisions, actions, and other matters. **Prime Recipient:** An organization that directly receives funding, support, or authorization to implement a project or program. **Procedure:** The required steps, course of action, or processes needed to accomplish a tas k

. **Process:** A set of interrelated resources and activities which transforms inputs into outputs. Examples of processes include analysis, design, data collection, operation, fabrication, and calculation. **Product:** The intended result or final output of an activity or process that is disseminated or distributedPage 17 of 43

am ong EPA organizations or outside of EPA. **Quality:** The totality of features and characteristics of a product or service that bear on its ability to meet the stated or implied needs and expectations of the user. **Quality A ssurance (Q** **A) :**  A m anagem ent or oversight function that deals with setting polic y and running an administrative system of management controls that cover planning, implementation, review, and m aintenance to ensure products and services are meeting their intended use. **Quality A ssurance Manager (**  **QA M**  **anager ):**  The individual designated as the principal manager within the organization having management oversight and responsibilities for planning, documenting, coordinating, and assessing the effectiveness of the QMS for the organization. NOTE: Other personnel hav ing QA or QC duties may be referred to as QA Officer and QA Coordinator. **Quality A ssurance Project Plan (Q** **AP P):**  A document describing in comprehensive detail the necessary QA, QC and other technical activities that must be implemented to ensure that the results of the work performed will satisfy

the stated performance objectives and criteria. **Quality Control (QC):** The overall system of technical activities that measure the attributes and performance of a process, item, or service against defined standards to verify that they meet the stated requirements established by the customer; operational techniques and activities that are used to fulfill requirements for quality. **Quality Management:** That aspect of the overall quality management system of the organiz ation that determines and implements the quality policy. Quality management typically includes strategic planning, allocation of resources, and other s ys tematic quality activiti es for the organization. **Quality Management Plan (QMP):** A formal document or m anual that describes the QMS in terms of the organizational structure, functional responsibilities of management and staff, lines of authority, and required interfaces for those planning, implementing, and assessing all activities conducted. **Quality Manag ement System (QMS):**  A structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and im plem entation plan of an organization for ensuring quality. The QMS provides the framework for planning, implementing, documenting, and assessing work for carrying out required QA and QC activities. **Required Element:** An element whose presence is obligatory in a Standard. **Requirement:** An expression of the content of a Standard conveying a criterion to be fulfilled if com pliance is to be claimed and from which no deviation is permitted. **Standard:** An accepted, consensus - based specification which defines systems, processes, me thodologies , or practices. It provides a basis for assuring consistent and acceptable minimum levels of quality, performance, safety, and reliability. Standards usually are included in or accompany procedures. **Standard Operating Procedure (SOP):** A written document that details the method for an operation, analysis, or action with thoroughly prescribed techniques and steps, and that is officially approved as the me thod for performing certain routine or repetitive tasks. **Sub -** **award:** A legal instrum ent that provides support for the performance of any portion of the project or program award in an external agreement that the recipient awards to an eligible sub - recipient. A sub award may be provided through any legal external agreement. **Sub -**  **recipient ( Sub):**  Generally an external organization that receives a sub - award from a prime recipient and that is accountable to the prime for products and/or services under that agreement.Page 18 of 43

**Transparency (Information):** The assurance that information is supported by s uf ficient metadata. **Usability A ssessment:**  The evaluation of data based upon the results of data verification and validation for the decision being made. Reviewers assess whether the process execution and resulting data meet quality objectives based on the criteria given in the QAPP. **User:** The cu stomer or organization for whom the results or products were collected or created. **Utility (Information):** The assurance that information is useful for its intended purpose. **Validation (Information):** The confirmation by examination and provision of object ive evidence that the particular requirement for which the information is intended is fulfilled; the process of determining whether the specifications were appropriate and that the verified results will meet the data user’s needs. **Verification (Informatio n):**  The confirmation by examination and provision of objective evidence that validated information fulfills specified requirements; the process of checking whether the information met the project’s specifications. **10. WA** **IVERS** Statutory requirements for quality m ay supersede the specifications in this Standard or be more rigorous. In such cases, affected programs shall be exempt from certain requirements in this Standard. EPA organizations conducting exempted activities shall comply with this Standard in all other respects. The following exemptions from these requirements apply:

 The collection of environmental data under the authority of Good Laboratory Practices as defined by 40 CFR 792, for the Toxic Substances Control Act.  The collection of environmental data under the authority of Good Laboratory Practices as defined by 40 CFR 160, for the Federal Insecticide, Fungicide, and Rodenticide Act. Requests for exemptions should be directed to the Office of Environmental Information (OEI) Quality Staff. Exemptions shall be made by the CIO (EPA Senior Management Official for Quality). Approved waivers will be documented by the requesting organization and the waiver will be added to their Quality Management Plan. **11. REFERENCES**  **EPA 2002a.**  Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. EPA/260R - 02 - 008 (October 2002 with Addenda in 2004 and 2005). www. epa .gov/quality / informationguidelines **EPA 2002b.**  Guidance for Choosing a Sampling Design for Environmental Data Collection (EPA QA/G 5S), Office of Environmental Information, U.S. Environmental Protection Agenc y. http://www.epa.gov/quality **EPA 2002c.**  Environme ntal Data Verification and Validation (EPA QA/G - 8), Office of Environmental Information, U.S. Environmental Protection Agency. http://www.epa.gov/quality **EPA 2005.**  Uniform Federal Policy for Quality Assurance Projects Plan W orkbook, Intergovernmental Data Quality Task Force, U.S. Environmental Protection Agency and U.S. Department of Defense. h ttp://www.epa.gov/fedfac/documents/qualityassurance.htmPage 19 of 43

**EPA 2006a.**  EPA Quality Polic y, CIO Polic y 2106. U.S. Environmental Protection Agency, W ashington, D.C. http://www.epa.gov/irmpoli8/policies/21060.pdf **EPA 2006b.**  EPA Procedure for Quality Polic y, CIO Procedure 2106 - P - 01.0. U.S. Environmental Protection Agency, Wa shington, D.C. http://www.epa.gov/irmpoli8/policies/21060.pdf **EPA 2006c.**  Peer Review Handbook. 3rd ed. EPA- 100 - B - 06 - 002. Prepared for the U.S. Environmental Protection Agency by members of the Peer Review Advisory Group, for EPA’s Science Policy Council. Wa shington, D.C: U.S. Environmental Protection Agency

. http://epa.gov/peerreview/pdfs/Peer%20Review%20HandbookMay06.pdf **EPA 2006d.**  Data Quality Assessment: Statistical Methods for Practitioners, (EPA QA/G - 9S ), Office of Environmental Information, U.S. Environmental Protection Agency. http://www.epa.gov/quality **EPA 2006e.**  Guidance on Systematic Planning U sing the Data Quality Objectives Process, (EPA QA/G 4), Office of Environmental Information, U.S. Environmental Protection Agency. http://www.epa.gov/quality **EPA 2009.**  Guidance on the Development, Evaluation, and Application of Environmental Models. Council for Regulatory Environmental Modeling, EPA/100/K- 09/003, U.S. Environmental Protection Agenc y, Wa shington, DC. http://www.epa .gov/crem **EPA 2013a**  **.** Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations , CIO Standard 2106 - S - 0 1 U.S. Environmental Protection Agenc y, W ashington, DC. http://www.epa.gov/quality **EPA 2013b**  **.** Quality

Standard for Environmental Data Collection, Production, and Use by Non - EPA (External) Organizations , CIO Standard 2106 - S - 02 U.S. Environmental Protection Agency, Wa shington, DC. http://www.epa.gov/quality **EPA 201**  **3 c**  **.** Handbook for Preparing Quality Management Plans, U.S. Environmental Protection Agency, Wa shington, DC. http://www.epa.gov/quality **EP A**  **201 3**  **d .**  Handbook for Developing Quality Assurance Project Plans, U.S. Environmental Protection Agency, W ashington, DC. http://www.epa.gov/quality **Gilbert 1987**

. Statistical Methods for Environmental Pollution Monitoring , John W iley, New York. **Ginevan, Michael E. & Splitstone, Douglas E., 2003.** Statistical Tools for Environmental Quality Measurement, Chapman & Hall/CRC, Boca Raton, FL.

**Ma nly, Bryan F.J., 2001.**  Statistics for Environmental Science and Management. Chapman & Hall/CRC, Boca Raton, FL.

**12. MA**  **TERA L SUPERSEDED**  This Standard replaces EPA/QA R- 2 and EPA/QA R- 5 in their entirety.Page 20 of 43

**13. AD**  **DITIONA L INFORMA** **TION** Other information and documentation on the EPA Quality Program may be found at http://www.epa.gov/quality

. The following Annexes contain detailed specifications and information that clarify the require me nts and use of this Standard. These are:  Annex A (Normative)

– Requirements for EPA Quality Management Plans for Non - EPA Organizations  Annex B (Normative)

– Requirements for EPA Quality Assurance Project Plans in External Agreements  Annex C (Informative)

– Guidelines on Planning for Projects in External Agreements

**Malcolm D. Jackson, Assistant Administrator and Chief Information Officer, Office of Environmental** 

**Information**Page 21 of 43

EPA Classification No.: **DRA FT**  **FINA L**  **(2106 -** **S -**  **02.0 A nnex A** **)** CIO Approval Date: CIO Transmittal No.: Review Date: 0 9/10/12 **AN NEX A**  **(NORM AT**  **IVE) REQUIREMENTS FOR QUA**  **LITY M AN**  **AG EM**  **ENT PLA NS FOR NON**  **- EPA**  **ORGA NIZA** **TIONS A1.**  **Introduction** Clause 7.2 in CIO Standard 2106 - S - 02 (EPA 2013b ) requires external organizations that collect, produce, or use environmental data to describe and document their quality management system (QMS) in a Quality Management Plan (QMP) or equivalent document acceptable to EPA. The QMP is unique to an organization and shall describe the management and technical processes and procedures necessary to plan, implement, document, and assess the effectiveness of the organization’s QMS applied to the work prescribed by the external agreement. Since non - EPA organizations may participate in more than one external agreement with EPA, the organization’s QMP may contain common m anagem ent information that may be applied to several agreements and not require an individual QMP for each agreement. The specific quality assurance (QA) and quality control (QC) procedures will be found in the Quality Assurance Project Plan (QAPP) for eac h external agreement (See Annex B). The QMP defines an organization’s quality- related elements that represent the core of the QMS including:

 ma nagem ent controls, policies, and objectives;  processes for and areas of applying quality procedures ; and  roles, responsibilities, and authorities of managers and staff. The QMP, therefore, is a management document that should be appropriately tailored to the needs of the organization. The QMP must be sufficiently inclusive, explicit, and readable to enable managers, supervisors, and staff to understand the priority that senior management places on QA, the established QA policies and procedures, and their respective QA roles. The QMP must be constructed and written so that an assessment can be made of the QMS’s effe ctiveness following implementation. This enables ma nagers to determine whether the QMS is being implemented in a way that ensures successful results can be obtained. The QMP shall focus on the processes used to plan, implement, document, and assess the pro grams to which it is applied. The level of detail should be based on a common sense, graded approach that establishes QA and QC requirements commensurate with the importance of the work, the available resources, and the unique needs of the organization. Q MPs shall be modified as the requirements change. This Annex describes the quality management elem ents that are normally considered to be critical to an effective QMS. A non - EPA organization im plem enting an external agreement shall evaluate these core elem ents to determine their applicability to its QMS for environmental data collection, production, and use. If additional quality management elem ents are useful or necessary to adequately describe the QMS, these elements shall be developed and discussed in th e QMP. Guidelines for satisfying these requirements may be found in Handbook for Preparing Quality Management Plans (EPA 201 3 c).Page 22 of 43

**A2. Preparation, Submission, Review, and A** **pproval A2.1**  **Preparation Responsibility** The senior ma nagem ent of the non - EPA organization is responsible for the preparation of a QMP that covers all work specified by the applicable external agreement and for which the organization’s ma nagem ent is accountable. Senior ma nagem ent includes the senior execut ive responsible and accountable for mission accomplishment and overall operations. Senior ma nagement shall ensure that the QMS documented in the QMP conforms to the requirements of this Standard. **A2.2 Non**  **- EPA** 

**Organization QMP Submission and A pproval**  A QM P m ust be approved and signed by the senior ma nager in the organization prior to being submitted to EPA for review and approval. The QMP must also be approved and signed by the QA manager (or equivalent position) in the organization. The senior ma nager may require the concurrence on the QMP by appropriate subordinate line managers to encourage the acceptance and implementation of the QMS. **A2.3 EPA** 

**Review and A pproval of the QMP**  Following internal organization approval, the non - EPA QMP must be submitted to the EPA organization responsible for the external agreement for review and approval. Each QMP will be reviewed to determine com pliance with general EPA Quality requirements including conformance with this Standard. QMPs that adequately address all program - level elem ents described in Clause 7 of this Standard should be approved and be valid for a period not to exceed the term of the external agreement. **A2.4 QMP Revisions**  A non - EPA QMS must be reviewed at least annually by the organization to confirm its continued accuracy and relevance in describing the QMS. This review must include an evaluation of the effectiveness of the QMP. W hen necessary, the organization shall revise its QMP to incorporate changes to the QMS. Revisions to a QMP may becom e necessary

due to:

 a change in the scope (i.e., statement of work) of the external agreement,  significant changes in the non - EPA organization’s mission or structure, or  any m ajor change to the organization’s QMS. A revised QMP may

be submitted to EPA at any time and is subject to the same review and approval processes as the original QMP. All personnel performing work for the organization shall be notified of changes to the QMS and the QMP. **A3. QUA**  **LITY M AN**  **AG EM**  **ENT P LA**  **N REQUIREMENTS A3.1**  **General Requirements**  **The QMP shall address all of the requirements pertaining to environmental data collection, production, and use as given in Clause 7 of this Standard.**  The QMP shall document all quality- related terms and conditions specified in the external a greement and describe their implementation. The QMP shall contain descriptions of the processes by which the organization plans, implements, documents, and determines the effectiveness of its quality management procedures , including QA and QC activities, to help management to obtain results of its technical work that are of the type and quality needed for their intended use.Page 23 of 43

Information about the organization’s structure and mission should be included in the QMP to provid e context for the external agreements. A brief description of the organization’s background and history may be included in the QMP for completeness. The QMP should reflect the organization’s commitment to quality management principles, and may be tailored by

the organization to meet its needs. To reduce duplication between QMPs and QAPPs, the QMP shall include discussion of those activities, policies, and procedures that are common to projects governed by other external agreements (e.g., mu ltiple contracts, work assignments, assistance agreements). Such discussions provide an “umbrella” under which individual project activities may be performed. The remainder of Annex A, Clause A3, provides the elements that must be addressed in the organization’s QMP. If the organization believes that an element does not apply to its QMS and should be om itted, the QMP must contain an explan ation of why it does not apply. The QMP shall always include:

 title page;  table of contents;  evidence of the review, approvals page, and revision history of the QMP; and  applicable references. **A3.2 Conformance with Policies, Procedures, and Standards**  The QMP shall identify all EPA Policies, Procedures, and Standards (and any internal organization policies and procedures to which the QMS conforms )

. **A3.3 Organizational and Program Field of A** **pplication** The QMP shall identify and describe:

 all parts of the organization to which the QMP applies, and  all business lines, programs, or actions involving the collection, production, and use of environmental data to which the QMS applies. **A3.4 Organization, Responsibility, and A** **uthority** The QMP shall include the following:  a statement of the organization’s polic y on QA, including the importance of quality in its environmental data products a nd services to the organization and its mission, and why; the general objectives/goals of the QMS; and the policy for resource allocation for the QMS, including personnel, intramural and extramural funding, and QA- related travel funding for EPA- related wor k;  an organization chart that identifies all of the components of the organization and, in particular, the organizational position and lines of reporting for the QA Manager (and any QA staff) that confirm and document the independence of the QA Manager fro m groups collecting, generating, using, and evaluating environmental data;  a discussion of the organizational structure of the QMS and the responsibilities and authorities of the QA Manager and any other QA staff;  a discussion of the specific quality mana gem ent controls required by the QMS and the work to which the controls apply; where oversight of delegated, contracted, or other external programs is needed to ensure that objectives for data quality and usability are attained; and where internal coordinat ion of QA and QC activities among organizational units needs to occur;Page 24 of 43

 a discussion of the QA and QC roles and responsibilities of line management, technical staff, and any other staff, and how accountability is defined;  a discussion of the organization’s process for resolving disputes regarding QMS requirements, QA and QC procedures, assessments, or corrective actions;  a discussion of how management shall ensure that applicable elements of the QMS are understood and implemented in all progra ms involving environmental data; and  an approval page for the signatures of the senior ma nager, senior line management (as appropriate), and the QA manager of the organization. This approval page may be part of a title page or a separate sheet following th e title page. **A3.5 QMS Description**  The QMP shall provide a description of the QMS and its implementation in the organization. The QMP shall describe the principal components in the QMS and how they are used to implement the QMS. These components include , but are not limited to,  QMS documents (e.g., Division or subordinate organization QMPs) ;  ma nagem ent assessments (self and independent) ;  staff competence ;  systematic planning processes ;  project- specific quality documentation (e.g., QAPPs or equivalent documentation) ;  standard operating procedures (SOPs) ;  technical audits/assessments (self and independent) ; and  data assessments. This discussion shall also identify how and when the components of the QMS are to be applied to individual projects and tasks in the external agreements. **A3.6 Personnel Competence**  The QMP shall include a statement of the organization’s policy for ensuring the competence of managers and staff to implement the QMS effectively and to provide training to ensure contin ued effectiven ess by personnel. The QMP discussion shall include the processes and the staff responsible for:  identifyi ng statutory, regulatory, or professional certifications that may be required to perform certain operations; and  identifyi ng, designing, performing, a nd documenting technical, quality, and project management training. The QMP shall also describe how staff proficiency in critical technical disciplines is maintained and documented. **A3.7 External A** **greements** The QMP shall describe the organization’s proc ess for ensuring that appropriate quality requirements are included and implemented in all external agreements subject to this Standard. The QMP shall also discuss the organization’s process for sub - agreements with other entities (e.g., subcontractors) for support to the statement of work in applicable external agreements.Page 25 of 43

**A3.8 Documents and Records Management**  The QMP shall describe (or provide a reference to) the processes by which the requirements contained in Clause 7. 10 of this Standard are met, including the responsibilities and autho rities of management and staff. In general, the QMP shall address the documents and records processes:  for identifying quality- related documents and records requiring control;  for handling documents and records to ensure their accessibility, protection from damage and deterioration, and means of retention, including discussion of the roles and responsibilities for ma nagem ent and staff;  by which all technical guidance documents are prepared, reviewed, approved, issued, used, and revised;  by which all planning documents (e.g., QAPPs, Sampling and Analysis Plans) are prepared, reviewed, approved, issued, used, and revised;  that ensure compliance with all statutory, contractual, and assistance ag reement requirements for records from environmental programs and that provide adequate preservation of key records necessary to support the mission of the organization;  for formal and informal reviews of products (e.g., reports, peer reviewed journal artic les); and  for reviews to confirm that performance and acceptance criteria were established and documented. The QMP shall describe how documents and records, including revisions, are reviewed for conformance with QMS requirements and with the terms and con ditions of external agreements, and are approved by authorized personnel before general use. The QMP shall describe or provide a reference to the management process that ensures that records accurately reflect completed work and/or fulfill statutory and contractual requirements, including any specific record keeping requirements defined in applicable EPA policies and procedures. The ma intenance of records includes defining requirements and responsibilities for record transmittal, distribution, retention, p rotection, preservation, traceability, disposition, and retrieval. The QMP shall identify how the disposition of records, in accordance with regulatory requirements, schedules, or directives from senior management, is accomplished. **A3.9 Use of Informatio**  **n Technology Methods and Sources** The QMP shall describe how the organization satisfies the requirements of Clause 7 .6 of this Standard and shall include the responsibilities of management and staff (Clause 8). **A3.10 Planning**  The QMP shall describe how the organization satisfies the planning requirements given in Clause 7. 7 of this Standard, including the responsibilities and authorities of management and staff. The descriptions shall address:  planning and scoping;  design of environmental data collection or acquisition;  design of model development, evaluation, and application; and  documentation of environmental data collection or acquisition and model design. The QMP shall describe the process for communications between the organization and any suppliers to ensure that there is a clear understanding by all participants of the needs and expectations of the organization and the products or results to be provided by the supplier. The QMP shall describe how the results of planning are documented in a QAPP or equivalent document and are approved by authorized personnel for implementation.Page 26 of 43

The QMP shall also describe or discuss the process for how information or data obtained from sources outside EPA that did not use an EPA- approved QAPP (or equivalent planning document) for data collection shall be e valuated and qualified for use. The QMP shall describe the review process for qualifying such data, including the use of data for purposes other than for which the data were originally collected. **A3.11 Imp**  **lementation of Work** The QMP shall describe how the organization satisfies the implementation of work requirements given in Clause 7. 7 of this Standard, including the responsibilities and authorities of management and staff. This description shall address and include processes for:  identification of operations needing procedures;  documentation of implementation procedures (e.g., reference methods, SOPs);  testing and evaluation of procedures to confirm their acceptable performance;  ensuring that work is performed according to approved plans;  deviations and waivers from approved procedures;  use of measurement and testing equipment and models;  use of data obtained from other sources;  ensuring the integrity of samples and data; and  mo nitoring of performance

. The QMP shall also describe how appropriate management controls for the release, change, and use of planned procedures are implemented. Such management controls provide for the necessary approvals, specific times and points for implementing changes, removal of obsolete documentation from work areas, and verification that the changes are made as prescribed. The QMP shall contain the organization’s process for identifying the need for SOPs, the process for developing SOPs, and the policy for using SOPs . The QMP shall also describe the process by which SOPs are reviewed for initial and subsequent use. The QMP shall also describe the organization’s process for deviations and waivers from approved procedures and the necessary reviews and approvals. **A3.12 Assessment, Oversight, and Response**  The QMP shall describe how the organization satisfies the requirements for assessment, oversight, and response as given in C lause 7.3 of this Standard, including the responsibilities and authorities of ma nagem ent and s taff (Clause 8). This description shall address and include processes for:  routine oversight;  identifyi ng appropriate assessment methods for use;  ensuring the competence of personnel conducting oversight and assessments;  ensuring the objectivity, independ ence, and competence of assessment personnel;  assessment planning and implementation;  assessment reporting; and  corrective action and verification. The QMP shall describe how assessment methods are chosen and the expected frequency of their application to environmental programs. In particular, the QMP shall document how senior management assesses the adequacy of the QMS.Page 27 of 43

The QMP shall describe how the competence of personnel conducting assessments is determined. Personnel conducting assessments shall be competent, based on project- specific requirements, to p erform the assigned assessment. Management is responsible for choosing the assessors, defining acceptance criteria, approving audit procedures and checklists, and identifying goals prior to initiation of an assessment. Assessors shall be technically knowledgeable with no real or perceived conflict of interest. If the assessors are chosen from within the organization, they must have no direct involvement or responsibility for the work being assessed. Th e exception to this requirement is for self- assessments, which must be identified as such. The QMP should also discuss the authority for and conditions under which “stop work” orders may be needed. It should also discuss when and how such decisions shall be made, and with whom any coordination will be needed. The QMP shall describe how management will respond to the results (or findings) and recommendations from assessments. The QMP will indicate how follow- up action shall be taken and documented to confi rm the implementation and effectiveness of the response action. In addition, processes for identifying, trending, and correcting common non - conformances found in different parts of the organization will be described to ensure QMS improvement. The QMP shall also describe how disputes, if encountered, as a result of assessments are resolved and by whom. **A3.13 Quality Improvement**  The QMP will d escribe :  t he organization's process for ensuring continual quality improvement, including the management process for determining, planning, implementing, and evaluating the effectiveness of quality im provement activities including: o who (organizationally) is responsib le for quality improvement; and o the corrective action program to ensure that conditions adverse to quali ty are  identified promptly and  corrected as soon as practical.  h ow the organization shall detect and prevent quality problems, and  h ow staff at all levels are encouraged to identify and establish communications among customers and suppliers, identify process improvement opportunities, identify problems, and offer solutions to those problems. Corrective actions will include :  i dentificat ion of root causes of problems,  determination of whether the problem is unique or has mo re generic implications, and a recommendation of procedures to prevent recurrence.Page 28 of 43

EPA Classification No.: **DRA FT FINA**  **L (2106**  **- S**  **- 02.0 A** **nnex B)** CIO Approval Date: CIO Transmittal No.: Review Date: 09/10/12 **AN NEX B (NORM**  **AT IVE)**  **REQUIREMENTS FOR QUA LITY A** **SSURA NCE PROJECT PL**  **AN S IN**  **EXTERNA L A** **GREEMENTS B1**  **. INTRODUCTION**  This Quality Standard for Environmental Data Collection, Production, and Use by Non -EPA (External) Organizations ( s ee Clauses 7. and 7.5 ) requires that all work performed by non - EPA organizations on behalf of EPA involving the collection, production, or use of any environmental data shall be implemented in accordance with an Agency- approved Quality Assurance Project Plan or approved equivalent planning document (hereafter referred to as a QAPP)

. The QAPP defines and documents how specific activities involving environmental data shall be planned, implemented, and assessed during a particular project. Therefore, it is the “road map” of how to achieve satisfactory results in such external agreements. This Annex presents the minimum specifications for quality assurance (QA) and for quality control (QC) procedures that must be addressed in a QAPP prepared by non - EPA organizations for external agreements, and describes the proce s s for its preparation, review , and approval . Supplemental guidelines on how to meet these QAPP requirements can be found in Handbook for Developing Quality Assurance Project Plans (EPA 201 3 d). The QAPP is a critical planning document for any environmental data operation since it docume nts how environmental data operations are planned, implemented, documented, and assessed during the life cycle of a program, project, or task. The ultimate success of an environmental program or project depends on the adequacy and sufficiency of the quality of the environmental data collected , produced, and used in estimation or decision - ma king. This depend s significantly on the adequacy of the QAPP and its effective implementation. Systematic planning ( Clause 7. 7 ) is an absolutely essential component of p roject management and the QAPP provides the mechanism for documenting the results of the planning process. Additional guidance on planning for projects may be found in Annex C of this Standard. **B2 .**  **QA PP**  **RESPONSIBILITIES A ND**  **AP PLICA**  **TION B2.1**  **QA PP Preparat**  **ion Responsibilities and A pprovals**  The external (non - EPA) organizati on’s Quality Management Plan (QMP) ( s ee Clause 7 and Annex A of this Standard ) establishes how, when, and by whom d evelopment, review, approval, and effective oversight of QAPPs occur

. In some cases, it may be necessary to add special requirements to the QAPP. The EPA organization sponsoring the work shall define any specific requirements beyond those listed in this Annex

. **Each QAPP or similar document must be approved prior to any data c ollection, production, or**  **use, with the possible exception of circumstances requiring immediate action to protect human health and the environment or operations conducted under po**  **lice powers.** Under these circumstances, the organization will use or modify existing standard operating procedures (SOPs) and other quality- related documents available to support such activities. This complete collection of QA documents must be submitted to the QA Manager together with a detailed explanation of the circumstances u nder which this emergency quality documentation was produced . This explanation must also address potential limitations in the interpretation of resulting data and information.Page 29 of 43

As noted in Clause 7. 5 .2 of this Standard, EPA may elect to require a combined QMP and QAPP (e.g., for sm all environmental research projects). A combined QMP - QAPP incorporates some elements of the QMP and some elements of the QAPP. The general elements to be addressed are typically taken from Clause s 7.2 and 7. 5 .1 of this Standard; however, the EPA organization shall determine the final elements to be addressed in the combined QMP - QAPP. As such, all of these elements must be addressed by EPA organizations and non - EPA organizations holding external agreements. The combined QMPQAPP is subject to the same review and approval process (both by the external organization and by EPA) as any other QMP and QAPP. **B2.2 QA**  **PP Implementation and Revision** The organization performing the work shall implement the EPA approved QAPP and ensure that all personnel involved in the work have copies of the approved QAPP and all other necessary documents and procedures, and fully understand the project needs prior to the start of activities

. Due to the complex and diverse nature of many programs involving environmental data collection, production, and use , changes to original QAPPs are often needed. The organization’s project manager, with the assistance of the QA M anager as appropriate, must determine the impact of such changes on the technical and quality objectives of the project. There should be agreement as to what changes would have minimal impact on the work and can be authorized in the field by the project manager (e.g. , change in sample location) and what changes would require a revision to an approved QAPP (e.g. , change in chemical analysis method)

. Wh en a substantive change is warranted, the organization shall modify the QAPP or prepare an addendum to document the change and submit the revision for approval by the sam e authorities that performed t he original review. T he change shall be implemented o nly after the revision has been approved and received (at least verbally with written follow- up) by project personnel and EPA. It is essential that the QAPP be kept current and that all personnel involved in the work have eas y access to a current version of the QAPP. For programs or projects of long duration, such as multi - year m onitoring programs, the QAPPs should be reviewed at least annually by the organization. If revisions are necessary to reflect current needs, the QAPP shall be revised and resubmitted for review and approval by the same authorities that performed the original review. **B2.3 Ap**  **plicability of Q AP**  **Ps** T he QAPP requirements in this Annex apply to all activitie s for EPA that acquire, generate, or analyze environmental data that are performed on behalf of EPA through external agreements. Such agreements ma y inc lude :

 acquisition of services and items through contracts, task orders, and other acquisition agreements;  assista nce agreements such as grants and cooperative agreements;  interagency agreements with other Federal, State, local, or international governments or entities;  enforcement agreements (e.g., administrative orders, consent agreements); and  voluntary partner programs (e.g., voluntary monitoring). Wh ere specific Federal regulations require QA and QC activities, QAPPs shall be prepared, reviewed, and approved in accordance with the specifications contained in this document for the data collection activity unless superseded by the regulation. QA and QC requirements are negotiated into the applicable interagency and international agreements, including sub - agreements , since EPA cannot unilaterally im pose its QA and QC requirements in these agreements.Page 30 of 43

**B3 .**  **QA PP**  **ELEMENTS A ND REQUIRE**  **ME NTS**  **B3.1 Overview and General Requirements**  This Clause describes the mi nim um elem ents and i nformation that are generally required for a QAPP for mo st operations involving environmental data collection, production, and use by extern al organizations. Due to the diversity of Agency programs, some elements described in this Annex ma y not be applicable to all programs. The final decision on the applicability or use of any or all of these elements for QAPPs shall be made by EPA and may b e specified in the statement of work for the external agreement. The content and level of detail in each QAPP may vary according to the nature of the work being performed and the intended use of the data. This is called “the graded approach” and allows fo r a flexible design of the QAPP content. The intent of the “graded approach” is to achieve sufficient and adequate detail in the QAPP to satisfy the scope and objectives of the project and the intended use of the data. The final decision on QAPP content a nd level of detail belongs to EPA

. **B3. 2**  **General Content Requirements** Most activities involving environmental data require the coordinated efforts of many individuals, possibly including managers, engineers, scientists, statisticians, and others. The QAPP mu st integrate the contributions and requirements of everyone involved into a clear, concise statement of what needs to be accomplished, how it shall be done, and by whom. It must provide understandable instructions to those who must approve or im plement the QAPP, including the field sampling team, the analytical laboratory, and the data reviewers. The use of SOPs and , when possible, national standards are encouraged in all aspects of the QAPP. Wh en these are used, they must be cited in the QAPP. The QAP P must include standardized, recognizable ma nagem ent elem ents that cover the entire scope of the project from planning, implementation, verification and validation, assessment , and data management to final reporting

. The QAPP must also recognize that there are different types of projects that may involve environmental data including, but not limited to:

 mo nitoring and use of direct measurements and laboratory results by EPA or its extramural partners;  collection or use of environmental data from other sources (e.g., obtained from databases, Internet, literature) that were collected originally for other purposes;  sam pling and analytical methods development or for validation purposes; and  generating or producing environmental data from the use of mathematical or probabilistic models and computer software. In order to be effective, the QAPP must specify the level or degree of QA and QC activities needed for the particular environmental data or model operations. The QA and QC technical requirements of a project should be commensurate with:

 the purpose of the environmental data collection (e.g., enforcement action, research and development studies ) or model use;  the typ e of work to be done (e.g., monitoring, site characterization, mo del simulation, bench level proof of concept) ; and  how the results shall be used (e.g., regulatory enforcement, Total Maximum Daily Load [TMDL] development, permit approval , research publications and journal articles ). The QAPP elements that follow are presented in an order reflecting the logical flow (Plan, Do, Check, Act) of the project life cycle and have been arranged , for convenience , into four general groups. The groups and their intent are summarized as follows:Page 31 of 43

 Project Manageme n t ( **Plan** ) - These elements cover the basic area of project management, including the project history, project objectives, technical and quality objectives (e.g., Data Quality Objectives), and roles and responsibilities of the participants. These elements document that the project has a defined goal and that the participants understand the goal and the approach to be used.  Data Acquisition ( **Do** ) - These elements cover all aspects of measurement and data acquisition systems design and implementation . It inclu des the intended measurements, data collection, or acquisition methods appropriate for achieving project objectives. Elements are presented for the different types of projects discussed above (i.e., monitoring, acquisition from other sources, and mo deling)

.  Assessments ( **Check** ) - These elements address the activities for assessing the effectiveness of the implementation of the project and associated QA and QC activities. The purpose of assessment is to ensure that the QAPP is implemented as prescribed and th at project actions are im plem ented as expected

.  Review, Evaluation of Usability, and Reporting Requirements (**Ac t**  ) - These elements cover the QA activities that occur after the data collection (or use) phase of the project is completed. Implementation of t hese elements will help to determine that the data conform to the specified criteria, thus achieving the project objectives and ensuring that data usability is documented in a final project report

. Included are any limitations or restrictions on the use of the data that will have to be identified and documented. **All applicable**  **elements must be addressed in the QAPP. For some projects, only part of the**  **QAPP elements discussed may apply to work being performed. If so, the elements that do not**  **apply to the work should be noted in the QAPP.**  Documentation, such as an approved Wo rk Plan, SOPs , m odel calibration reports, etc., may be referenced in response to a particular required QAPP element to reduce the size of the QAPP and the tim e required for preparation and review. All referenced documents must be attached to the QAPP itself or be placed on file with EPA and available to users when needed. Such references must be kept current by the submitter. The QAPP shall also identify and describe related QA planning documentation (e.g., QMPs) from subcontractors or suppliers of services critical to the technical and quality objectives of the project or task. The format of the QAPP is not critical except that all required elements must be addressed. The organization should present the content of the QAPP in a logical, easily understood manner. The following items may

be helpful in making the QAPP more user-friendly and useful:  table of contents ;  document control format;  applicable references (e.g., methods, SOPs, hand books); and  em bedded I nternet links. **B3. 3**  **Project Ma**  **nagement** The QAPP mu st d escribe the project adequately. E lements that address the basic project management and objectives of the work includ e :  ti tle , version number, and a pproval /sign - off s heets;  document format and table of contents;  distribution list;  p roject o rganization and schedule;  project b ackground , overview, and intended use of data;  project quality o bjectives (Data Quality Objectives) and me asurement performance c riteria ;  special trainin g requirements/c ertification ; and  documentation and records requirements. Refer to EPA 2013d for specifics on satisfying requirements in these areas

.Page 32 of 43

**B3. 4**  **DA TA**  **A CQUISITION**  The collection, production, and use of environmental data may include:  m onitoring/direct measurements - traditional sampling and analysis activities, taxonomic observations, or testing (e.g., ecological mo nitoring, biological studies);  collection of data from other sources - data retrieved from data bases, literature, or other electronic sources (e.g., i nternet or provided by othe r organizations);  s am pling and analytical/testing methods - the development of analytical me thods or other testing me thods (e.g., biological, ecological, GIS), or used for validation purposes; and  d evelopment , modification, and use of mathematical and probabilistic mo dels and other software applications (information technology products)

. The QAPP shall include sufficient descriptions of planned activities in order to meet t he requirements given in Clauses 7. 5 and 7. 7

. In addition, A nnex C ( P lanning) may be particularly helpful in d efining project planning needs. **B3.4.1 Monitoring/Direct Measurements**  Th e following QAPP elements cover aspects of measurement and testing equipment (M&TE) systems design and implementation in the project. These elements reflect traditional sampling and analysis activities for the collect ion of new data or information. T he elem ents shall include:  s am pling collection d esign , exper im ental d esign , and sampling tasks;  s am pling procedures and requirements ;  s am ple handling, c ustody procedures, and documentation;  sam ple collection documentation;  a nalytical m ethods requirements and task description;  q uality control requirements ;  instrumen t/equipment testing, inspection, and maintenance requirements , QA/QC supplies and consumables; and  d ata ma nagement requirements. Refer to EPA 2013d for specifics on satisfying requirements in these areas. For assistance in s am pling p rocess d esign, the organization should refer to Guidance for Choosing a Sampling Design for Environmental Data Collection , EPA QA/G - 5S (EPA 2002 b )

. **B3.4.2 Collection of Data from Other Sources** For those circumstances in which environmental data being acquired from sources other than direct me asurements by EPA or its partners (for example, published literature, other federal agencies, the I nternet) the QAPP shall describe how the requirements of this Annex will be met. Similarly, when data from EPA or its partners are intended for a use other than for which it was collected originally, the QAPP shall document the acceptance criteria for the use of these data. The QAPP shall identify and describe t he data sources and ty pes of data needed that are to be obtained from other sources, and shall define acceptance criteria for the use of such data in the project. The acceptance criteria for evaluating existing data shall be sufficiently detailed to allow for both quantitative and qualita tive criteria to be developed.Page 33 of 43

Quantitative criteria will allow for the statistical consideration of data, while qualitative data or information will allow for contextual consideration of data and information. If data from other sources are to be integrated into new data or information , the acceptance criteria shall address how the integration will be achieved and verified for appropriateness. If the quality of the data cannot be determined, the QAPP shall document this l im itation and the outputs or products of the project include disclaimers or discussion reflecting this limitation. Ke y elem ents of the process of review ing environmental data for potential use may include :  the need to satisfy the major requirements of the Information Quality Act (2002) ( e. g. , objectivity, utility, and integrity ) ;  a description of the data collection methodology employed;  the principal indicators of environmental quality ( e. g. , precision, bias, representativeness, com parability, completeness , and sensitivity); and  the general usability factors (e.g., relevance, temporal compatibility, spatial inclusivity, credibility, aggregation level, compatibility, reproducibility, and degree of augmentation). Th e following QAPP elements cover aspects of the design and implementation for the col lection of data in the project. Wh ere applicable, the elements shall be addressed in the QAPP and include:  proposed data source originator and publication information;  data format and accessibility;  establ ishment of acceptance criteria;  sam ple data collection methodology;  quality program and quality assurance procedures used by data originator;  documentation of sample quality assurance procedures; and  data management requirements. Refer to EPA 2013d and to A nnex C for assistance in constructing qualitative and quantitative acceptance criteria for the investigation of data obtained from undocumented sources. **B3.4.3 Sampling and A** **nalytical/Testing Methods Development and Validation** Projects for the purpose of developing or validating sampling and analytical testing methods typically will involve the collection, production, and use of environmental data as part of the design, testing, and verification/validation of the method. In some situations, the requirements of Clause B3.4.1 may also apply to methods development and validation and they should be applied as applicable. The QAPP shall describe how the requirements of Clause 7. 5 and this Annex will be met. Refer to EPA 2013d for specifics on satisfy ing requirements in these areas. **B3.4.4 Development and Use of Mathematical and Electronic Models (Information Technology**  **Products)** The development and use of mathematical and electronic models (i.e., information technology products) shall be described i n a QAPP. The QAPP shall meet the requirements of Clauses 7. 5 and 7.6 of this Standard. Reference should be made to EPA 2013d fo r guidance on addressing QAPP needs for model development, modification, and application. The QAPP shall discuss requirements for evaluation of soundness of science underlying a model, quality and quantity of available data, degree of correspondence with observed conditions, and the appropriateness of a model for a given application.Page 34 of 43

**B3.4.4.1 Quality A** **ssurance Project Plan Requ irements for Model Development/Modification**  The QAPP shall include, but not be limited to, consideration , development, and documentation of the specifications for model development identified below

. Wh ile some of these aspects may not be available at the start of a model development project, the QAPP should be amended as soon as the information becomes available. Problem Specification and Identification of Model Purpose and Scope The QAPP should define :  the objectives (i.e., what questions the model need s to answer);  the mo st important functions that the software application mu st perform;  the model scope (e.g., spatial, temporal and process detail); and  how the q uantitative or qualitative mo del performance criteria will be determined. Model Development or Selection Process The QAPP should identify and define:  t he theory that form s the basis for the mo del , the mathematical algorithms , and approaches used in executing the model computations ;  the existing software to be used in development of a new model, how it will be used and why it was selected;  the computer hardware and operating system requirements for the software application;  any assumptions and limitations and effect on model applicability; and  the procedures for con trolling, documenting, and archiving all significant changes to the software and hardware

. Data Requirements The QAPP should define clearly:  the data needs/inputs/sources;  any boundary or initial condition specifications ;  data reporting requirements an d format for model inputs and outputs ; and  the data quality and quantity objectives for model input and data used for model development. Evaluation of the Model It is necessary for the QAPP to define:  how the science underlying each component of the model will be evaluated;  the model testing strategies including design and code verification, individual module tests, integration tests, systems testing, acceptance testing, and beta testing, as applicable

.  t he procedure for each test shall be provided and the process of confirming the t est results included (EPA 2009);  the use of uncertainty and sensitivity analysis ; and  the requirements for project documentation (e.g., design document, source code, and user guide). **B3.4.4.2 Quality A** **ssurance Project Plan Requirements for Model A pplication**  The QAPP shall include, but not be limited to, consideration and development and documentation of the detailed specifications for model appli cation identified below:Page 35 of 43

Problem Specification, Model Purpose, and Model Selection The QAPP should define:  the basis for the selection of model from other available models; and  the research or regulatory objectives and the purpose of model application and analytical work

. Model Application The QAPP needs to define:  the scenarios to be simulated; and  how the modeling analysis and its results will be documented.  the model scope (e.g., spatial, temporal and process detail); and  how the q uantitative or qualitative mo del performance criteria will be determined. Data Requirements The QAPP should define clearly the quality and quantity of available data and procedures for excluding data. Evaluation of the Model It is necessary for the QAPP to define:  the quality objectives to be used in the model evaluation;  the requirements for qualitative and/or quantitative model corroboration;  the use of sensitivity and uncertainty analysis;  any requirements for model post- auditing;  the requirements for peer review of the model appli cation;  how the model evaluation will be documented;  the model parameters and method of estimation; and  the model calibration methods. **B3. 5**  **AS SESSMENTS**  Th e QAPP shall describe how the requirements of Clause 7. 7 of this Standard, including the roles and responsibilities of project personnel, will be addressed in a project involving the collection of new data or inclusion of data from other sources

. The QAPP elements that address the processes for assessing the ef fectiveness of the implementation of the project , and associated QA and QC activities such as as sessments and response/corrective a ctions mu st also be addressed. Examples of assessments include:  technical systems assessments;  performance audits of me asurement and analytical s ystems;  surveillance operations;  audits of data quality;  qualitative comparisons to acceptance criteria;  quantitative comparisons to acceptance criteria;  assessments to other criteria;  interim assessments of data quality  evalua tion of unconventional measurements; and  evaluation of unconventional monitoring projects. It should be noted that when using existing data, only the last six QA/QC activities listed apply. Refer to EPA 2013d for guidance on satisfying requirements in these areas.Page 36 of 43

**B3.6 REVIEW, EVA**  **LUA TION O**  **F USA BILITY, A**  **ND REP ORTING REQUIREMENTS**  **B3.6.1 Implement and Document Review, Verification and Validation, and Evaluation of Usability**  Th e QAPP shall describe how the requirements of C lause 7.9, including the roles and responsibilities of project personnel, wi ll be addressed in the project. The QAPP elements that address the processes for the review, assessment, and verification of usability includ e :  data verification and validation targets and methods;  quantitative and qualitative evaluations of usability;  potential limitations on interpretation ;  reconciliation with project requirements ; and  reports to management. If the quality of the data cannot be determined, the QAPP shall note that any limitations impacting the work products shall include appropriate disclaimers on the use of the products. For specific advice in data verification and validation , refer to Guidance on Environmental Data Verification and Validation , EPA QA/G - 8 (EPA 2002 c ); for data usability , refer to Data Quality Assessment: Statistical Methods for Practitioners EPA QA/G - 9S (EPA 2006d), and also EPA 2013d

. **B3.6.2 Implement and Document Model Review**  The c ritical review of a model or its application shall be conducted by individuals who are independent of the work and who collectively have technical expertise equivalent to those who performed the work

. Internal and external p eer review may help to ensure that the model is technically adequate, competently perform ed, properly documented, and satisfies established quality requirements through the review of assumptions, calculations, extrapolations, alternate interpretations, methodology, acceptance criteria, and/or conclusions from a model or its application (mod ified from EPA 200 9 ). The process for reviewing a mo del sh all be documented in the QAPP.Page 37 of 43

EPA Classification No.: **DRA FT FINA**  **L (2106**  **- S**  **- 02.0 A** **nnex C)** CIO Approval Date: CIO Transmittal No.: Review Date: 09/12/12 **AN NEX C (INFORM**  **AT IVE)**  **GUIDELINES ON PL**  **AN NING FOR PROJECT**  **S INVOLVING**

**EXTERN AL** **A GREEMENTS**  A nnex B presented the minimum specifications to be addressed in a Quality Assurance Project Plan (QAPP) developed by non - EPA organizations as part of an external agreement. A nnex C gives guidelines for systematic planning in addressing projects involving the collection , analysis, and reporting for environmental data , including the development of conceptual models of the data collection activities

. Systematic planning involves defining and characterizing the project and desired outcomes, and designing quality assurance (QA) and quality control (QC) requirements and procedures that will ensure the data collected, generated, or used during the project will support expected and needed project outcomes. The elements of s ys tematic planning can be grouped into three process areas :  project planning (Clause C1);  design of data collection activities (Clause C2); and  design and use of conceptual models when appropriate (Clause C3 ). The elements in both planning and development are iterative in nature as information from one area may change or alter the results in other areas. For example, a change in the conceptual model that describes the scientific mechanism driving the project can change the design of data collection and even the scope of investigation. The combination and s ynthesis of all three areas lead to a data collection activity capable of withstanding robust scrutiny

. As an added aid to planning, Clause C4 discusses the re lationship between systematic planning and the QAPP, including the Data Quality Objectives (DQO) process and an example application of the DQO process to the Uniform Federal Policy QAPP W orkbook (EPA 2005). **C1 .**  **PROJECT PLA NNING**  Project planning should be coordinated among the participa nts and include, as applicable , the following steps:  a definition of the project;  the scope of the investigation;  the needs or demands of the project; and  the reporting requirements established in the external agreement fo r the project or investigation

. **C 1.1**  **Definition of the Project** The definition step creates an overall framework upon which the project will be developed and should consider:  the definition of project/task scope and objectives and the desired action or result from the work;  an i dentification of program requirements and regulations that w ill be supported by the project;  the d evelop me nt of project goals, task/activity lists , time fram e s and logistical needs ;  the development of a conceptual model that describes the physical, chemical, scientific, engineering principles or mechanism relating to the objectives of the project;  the ne ed to plan and conduct pilot or reconnaissance studies that could provide sufficient data to assist in the development of the conceptual model; and  the identification of the different environmental risk assessment pathways that will be evaluated by the project , if appropriate

.Page 38 of 43

**C1.2 Scope of**  **the Investigation**  The scope step defines the boundaries (i.e., temporal, physical, and logistical) of the project and may include:  an identification of special applicable regulatory requirements and key spatial and temporal constraints such as budget, timing, authorization, permits, and access;  the recognition of organizations (e.g., sampling groups or analytical laboratories) that need to participate in the project and their role in planning, implementation, and assessment activities;  an outline of the composition of the project management team ( i.e., the management, technical , and administrative support team including any contractors and subcontractors );  a listing of conditions under which suspension of work will be necessary;  an identification of necessary personnel, their needed skills, and required types of equipment;  the results of literature search es and background information collection about the project including industries, products and wastes generated by the industries , and previous ana lytical results ; and  a preliminary determination of assessment tools needed such as technical reviews, audits, statistical or mathematical analyses, or engineering capability studies. **C1.3 Needs or Demands of the Project**  In order to effectively support successful com plet ion of the project, this step gives attention to:  an i dentification of data gaps that should be filled by the new project ;  a consideration of accompanying metadata needed for development of the conceptual model;  a li sting of procedures to handle unavoidable deviations from the Agency- approved QAPP;  an i dentification of standard operating procedures (SOPs) for developing and implementing corrective actions ;  the identification of possible methods/procedures for characterization and disposal of contaminated sample material that may be accumulated during the project ; and  an outline of the documentation needed to adequately describe the quality of the results. **C1.4 Reporting Requirements Established for the Project**  **or Investigation** Successful projects demand good communication both during the progress of the project and in the reporting of results. The reporting step gives attention to:  the e stablish me nt of communication and authority am ong the project team parti cipants and i dentification of the roles and responsibilities of each member of the project team;  an i dentification of project reports that will need to be prepared and the frequenc y of submission to management ;  a working schedule for data reporting;  a determination of data reporting format ; and  an identification of methods/procedures for storing, retrieving, analyzing, and reporting the data produced (based on the intended use of the data). **C2 .**  **DESIGN OF ENVIRONMEN TA** **L DA TA**  **A ND INFORM**  **A**

**TION COLLECTION OR A** **CQUISITION** The design process should include , but not necessarily be lim ited to , consideration and development of detailed plans for th e collection or acquisition of data or information, the analysis or sy nthesis of data and information, and specific data or information reporting requirements.Page 39 of 43

**C2.1 Collection or A** **cquisition of Data or Information** Significant attention should be devoted to designing and documenting all aspects of how the environmental data and information are collected. These considerations should include:  documentation of the degree to which the data or information collected will be representative of the conditions established by the goals of the project;  documentation of the sampling strategy, rationale , and necessary procedu res for collecting the d ata or information required to achieve the desired action or result;  documentation of the s am pling design and rationale together with the linkage to the conceptual mo del established for this project;  t he determination of the quanti ty of data needed to achieve the proj ect goals or support a decision with the desired level of confidence;  an i ntegration of logistical needs, costs and schedule into the preferred sampling design ;  documentation on the sample locations , sample types and number of sub samples per matrix ;  an assessment of the necessity for o n - site field and laboratory assessments to be performed prior to and during project implementation ; and  the selection of field sampling or testing methodology (except w here specified by statute or regulation), including specific sampling or field analytical instrumentation requirements and other analytical testing requirements. Practical advice on how to address these considerations may be found in current guidance from the Quality Staff , including EPA 2002 b and from standard works on environmental statistics (e.g., Gilbert 1987, Manly

2001, Ginevan & Splitstone 2003). **C2.2 An**  **alysis or Synthesis of Data or Information** After collection, data should be analyzed and information s ynthesized before definitive conclusions can be drawn. In planning for the analysis or synthesis of data , attention should be given to:  the selection of analytical methods, test procedures (except where specified by statute or regulation), and their associated quality performance expectations and identification of me asureable data quality or QA/QC performance criteria against which results will be compared;  an identification of me asurement data quality criteria or QA /QC requirements necessary to establish the quality of the data collected or pro duced;  the selection of analytical facility (or laboratory) together with me asures for verification that the laboratory not only is certified to an appropriate and applicable standard, but is competent and has demonstrated capability in performing the analyses needed ;  establishment of measurement quality objectives for specific data quality indicators such as precision, bias, representativeness, completeness, comparability, and sensitivity;  i dentificati on of field and laboratory QA /QC requirements for s am ple handling, packaging, shipping , custody , and health and safety procedures;  the integrity of d ata management procedures in the field and laboratory (e.g., data retrieval, security, QA and QC, storage, and retention);  the documentation of how information will be summarized and synthesized;  the construction of acceptance criteria for data or information obtained from external sources;  an outline of how metadata (or ancillary information) will be used ;  documentation of proposed information privac y and integrity procedures; and  the integrity of information management procedures in both the field and information collection centers.Page 40 of 43

**C2.3 Specific Data or Information Reporting Requirements**  Wh en an external agreement requires release of data summaries or information reports, care should be given to the documentation needed to ensure the integrity, clarity, transparency, and utility of the data. Attention should be focused on:  the d etermination of the leve ls of data or information verification and validation needed to meet project goals ;  documentation of p rocedures used for data or information validation and assessment ;  a summary

of potential techniques for assessing limitations on data or caveats on information use;  the d ata reporting deliverables and electronic data formats to be used; and  the typ e of reports expected together with their frequency of submission , for example, o field sampling /information gathering report , o summar ies of anal ytical data results, o data validation reports, o interim information syntheses, and o draft and final project reports. **C3 .**  **DESIGN OF CONCEPTUA L**  **MODEL DEVELOPMENT, E VA**  **LUA TION, A**  **ND A PPLI**  **CA**

**TION** A “model” is something that can be used to make a prediction and , in the first instance , based on the conceptual model that describes the physical, chemical, scientific, engineering principles, or mechanism relating to the objectives of the project (see Clause C.1, Project Planning). As a conceptual model is only an app roximation of a complex environmental phenomenon, attention has to be given to the documentation of conceptual model development so that the context may be understood fully

. The documentation of a general conceptual model should include , as applicable:  th e requirements for qualitative and/or quantitative conceptual model corroboration;  a consideration of the requirements for application of multiple conceptual models;  the use of conceptual model parameters characterized from direct measurements ;  the integra tion of cost or schedule constraints into design ;  the documentation of data reporting requi rements and required formatting;  a listing of the requirements for sensitivity and uncertainty analysis; and  an outline of the requirements of documentation to ensure transparency of conceptual model development. **C4 .**  **RELA**

**TIONSHIP OF SYST EM**  **AT IC PL**  **AN NING TO T**  **HE QUA LITY A** **SSURANCE PROJECT PL**  **AN (QA**  **PP)** The Quality Standard requires the use of systematic planning for any activities involving the collection, production, and uses of environmental data ( Clause 7. 7 of this Standard ) and also the documentation of ty pe, quantity, and quality of environmental data for their intended use. Systematic planning is a process based on the widely- accepted scientific method and uses a comm on sense approach to ensure that the level of documentation and rigor of effort in planning is commensurate with the intended use of the information and the available resources for data collection and models shall be documented in a QAPP or equivalent docu me nt. The elements of systematic planning are outlined in Table C - 1.Page 41 of 43 **Table C -**  **1. Elements of Systematic Planning** 

**Elements Organization:**  Identification and involvement of the project manager, sponsoring organization and responsible official, project personnel, stakeholders, scientific experts, customers, suppliers , and Quality Manager

. **Schedule:** Identification of project schedule, resources (including budget), milestones, regulatory or contractual requirements , and circumstances that could adversely affect the progress of the project. **Project Goal:** Description of the project goal, objectives, study questions , and conceptual model t hat describes the environmental phenomenon, contamination pathways, and im pact on human health. **Quality A ssurance:**  Specification of needed quality assurance and quality control activities to ensure performance criteria (for new data) or acceptance criteri a (for existing data) are satisfied. **Data Needs:** Identification of the type of data needed, how the data will be used to support the project’s objectives, and identification of potential limitations in data interpretation. **Criteria:** Determination of the quantity of data needed and specification of performance criteria for me asuring quality for new data, or acceptance criteria for the use of existing data. **Data Collection:** Description of how and where the data will be obtained (including existing data or data intended for secondary use) and identification of any constraints on data collection. **An aly** **sis:** Description of how data will be analyzed, evaluated, and assessed against their intended use with respect to the established performance or acceptance criteria. **C4.1 Ty**  **pes of Systematic Planning** Various government agencies and scientific disciplines have established and adopted different variations to systematic planning tailored to their specific application areas . F or example, the Observational Method is a variation on systematic planning that is used by many engineering professions. However, it is the Data Quality Objectives (DQO) Process (EPA 2006 e ) that is the most commonly used application of systematic planning in the general environmental community and is EPA’s recommend ed systematic planning process when data are to be used for decision ma king (e.g., compliance with an environmental standard) or for estimation (e.g., ascertain the mean concentration level of a contaminant). **C4.2 The DQO Pro**  **cess** The DQO Process can be used to establish performance or acceptance criteria, which serve as the basis for designing a plan for collecting data of sufficient quality and quantity to support the goals of a study. The DQO Process may be applied to all programs involving the collection and use of environmental data and applies to programs with objectives that cover decision making, estimation, modeling in support of research studies, monitoring programs, regulation development, and compliance support activ ities. The DQO Process is grounded in the scientific method of hypothesis, experiment, conclusion, and has been broken into seven steps to facilitate its use

. The outputs of the DQO Process are used to develop a QAPP, and, after data collection, for use in data quality assessment or statistical analysis of the resulting data or information. The steps of the DQO Process are shown in Figure C - 1 and discussed in more detail in EPA 2006 e . The QAPP describes in comprehensive detail the necessary QA activities th at must be implemented to ensure the results of the work performed satisfy the specified performance or acceptance criteria established for the project. The details of the QAPP may be assembled into convenient groups of elem ents to facilitate successful im plantation. An example is the Uniform Federal Policy for Quality Assurance Project Plans (UFP - QAPP) Wo rk book, EPA 2005, which, although is intended for use at Federal facility hazardous waste sites primarily, draws on multiple Agency documents on QA. The e lem ents of the UFP - QAPP are shown in Table C - 2.Page 42 of 43

The planning outputs from the seven steps of the DQO Process fit into the requirements for im plem entation of a QAPP. The relationship s am ong systematic planning, the DQO Process, and , for example, the elements of a UFP - QAPP are shown below in Table C - 3

. Several EPA organizations have accepted the UFP - QAPP as an alternative quality document to the EPA QAPP described in Annex B. The use of the UFP - QAPP or any other alternative QAPP document is subject to the approval of the EPA organiza tion responsible for the work.

**Figure C**  **- 1**  **. Th e DQO Process**  **Step 1. State the Problem.** Define the problem that necessitates the study , identify the planning team, examine the budget, and set the schedule

. **Step 2. Identify the Goal of the Study.** State how environmental data will be used in meeting objectives and solving the problem, identify the study questions, and define alternative outcomes

.

**Step 3. Identify Information Inputs.** Identify data and information needed to answer study questions. **Step 4. Define the Boundaries of the Study .**  Specify the target population and characteristics of interest, define spatial and temporal limits, and the scale of inference

. **Step 5. Develop the An**  **alytic A pproach.**  Define the parameter of interest, specify the type of inference, and develop the logic for drawing conclusions from findings

. **Step 6 .**  **Specify Performance or A cceptance Criteria**  **. Estimation**  Develop performance criteria for new data being collected or acceptance criteria for existing data being considered for use. **Decision Making** Specify probability limits for false rejection and false acceptance decision errors

.

**Step 7. Develop the Plan for Obtaining Data.** Select the resource - effective sampling and analysis plan

that meets the performance criteria

.Page 43 of 43

**Table C -** **2 . The Elements of a**  **Uniform Federal Policy Quality A** **ssurance Project Plan ( UFP**  **- QA**  **PP) A.**  **Project Management and Objectives** A1 Title and Approval Page A2 Document Format and Table of Contents A3 Distribution List and Sign - off Sheet A4 Project Organization A5 Project Planning/Problem Definition A6 Project Quality Objectives and Measurement Performance Criteria A7 Secondary Data Evaluation A8 Project Overview and Schedule **B. Measurement/Data A cquisition**  B1 Sam pling Tasks B2 Analytical Tasks B3 Sam ple Collection Documentation B4 Quality Control (QC) Sam ples B5 Data Management Tasks **C. A ssessment/Oversight**  C1 Assessments and Response Actions C3 Final Project Report C2 QA Management Reports **D. Data Review** D1 Overview D2 Data Review Steps (Verification/Validation) D3 Streamline Data Review

**Table C -** **3 . The Relationship between Systematic Planning, the DQO Process, and a**  **UFP -**  **QA PP**  **Systematic Planning DQO Process**  **Elements of a UFP -**  **QA PP**  Organization Step 1: Define the problem A1 - A5 Schedule Step 1: Define the problem A4, A8, C3 Project Goal Step 2: Identify goals Step 3: identify information needed for the study A4, A6 Quality Assurance Step 3: Identify information needed for the study B2

– B 5 Data Needs Step 4: Define boundaries Step 5: Develop the analytic approach A6, B1 Criteria Step 6: Specify performance or acceptance criteria A7, B1, C1, D1 - D2 Data Collection Step 7: Develop the plan for obtaining data A7 - A8, B1 Analysis Step 7: Develop the plan for obtaining data B1, A7, C1 - C 2, D3