# Handbook for **Preparing Quality Management Plans** 

**Agency Rev**  **ie w**  **Draft December**  **, 2012** 

This document is a **review draft** . This information is distributed solely for the purpose of pre dissemination review under applicable information quality guidelines. It has not been formall y disseminated by

EPA. It does not represent and should not be construed to represent any Agency determination or policy . It is being circulated for review of its technical accuracy .Handbook QMP ii December 2012 Agency Review Draft **FOREWORD** The U.S. Environmental Protection Agency (EPA) uses Quality Management Plan s (QMP s ) to document how an organization will plan, implement, and assess the effectiveness of its Quality Management S ys tem (QMS) appli ed to environmental data collection, production, and use. The requirements and specifications are listed in Annex A of Quality Standard For Environmental Data Collection, Production, and Use By EPA Organizations (EPA CIO 2106- S - 01) and Quality Standard For Environmental Data Collection, Production, and Use By Non -EPA (External) Organizations (EPA CIO 2106- S - 02.0) (current versions). This document:  p rovides technical and policy advice ;  provides implementation information;  does not substitute for statutes EPA administers or their implementing regulations;  i s not a regulation;  does not impose legally- binding requirements on EPA, states, or the regulated community ;  i s not mandatory . EPA may :  apply this handbook to any

particular program or project to the extent appropriate in light of site - specific facts ;  not apply

this handbook to a particular situation based upon the specific circumstances ; and  update, improve, revise, replace or withdraw this h andbook at any time. This document is one of several guidance and h andbook documents from the U.S. Environmental Protection Agency, Office of Environmental Information (OEI) Quality Staff. EPA Quality Program documents may be downloaded from the Quality Home Page: http://www.epa.gov/quality /qa_docs.html In many instances, regional or program quality assurance personnel and project management personnel will be engaged for specific implementation information. These personnel are listed here: http://www.epa.gov/quality /contacts.html Questions regarding this document or other documents can be directed to the OEI Quality Staff: U.S. EPA OEI/ Quality

Staff (2811R) Washington, DC 20460 Phone: (202) 564 - 6830 FAX: (202) 565 - 2441 e - mail: quality@epa.govHandbook QMP iii December 2012 Agency Review Draft **TABLE OF CONTENTS CHAPTER 1 INTRODUCTION**  **................................ ................................**  **................................ 1**  1.1 Background and Purpose ................................ ................................ ......................... 1 1. 2 Quality

Management Plans in the EPA Quality Management S ys tem .................... 1 1.3 Intended Audience ................................ ................................ ................................ ... 4 1.4 Period of Applicability ................................ ................................ ............................. 4 1.5 Supersession ................................ ................................ ................................ ............. 4 **CHAPTER 2 Q UALITY MANAGEMENT PLAN PREPARATION, SUBMISSION,**  **REVIEW, AND APPROVAL ................................**  **................................ ..........................**  **5** 2.1 Quality

Management Plan Preparation ................................ ................................ .... 5 2.2 Quality

Management Plan Submissionand management Approval ......................... 5 2.3 Quality

Management Plan Review and approval for use ................................ ......... 6 2.4 Quality

Management Plan Revisions ................................ ................................ ....... 6 2.5 Combined Quality Management Plan with Quality Assurance Project Plan ........... 8 **CHAPTER 3 HANDBOOK ON QUALITY MANAGEMENT PLAN CONTENT**  **............ 10**  3.1 General Guidelines ................................ ................................ ................................ . 10 3.1.1 Overview of QMP Content ................................ ................................ ....... 11 3.1.2 Administrative Elements ................................ ................................ ........... 11 3.1.3 Application of Graded Approach in QMPs ................................ .............. 12 3.2 Quality

Management Plan Conformance to Requirements ................................ ... 13 3.3 Organizational and Program Field of Application ................................ ................. 14 3.4 Organization, Responsibility, and Authority ................................ ......................... 15 3.4.1 Organizational I ssues ................................ ................................ ................ 15 3.4.2 Roles and Responsibilities ................................ ................................ ........ 16 3.4.3 Dispute Resolution ................................ ................................ .................... 17 3.5 Quality Management S ys tem Description ................................ ............................. 17 3.6 Personnel Competence ................................ ................................ ........................... 19 3.7 External Agreements ................................ ................................ .............................. 20 3.7.1 EPA Organizations ................................ ................................ .................... 20 3.7. 2 Non - EPA Organizations ................................ ................................ ........... 21 3.8 Documents and Records Management ................................ ................................ ... 22 3.9 Use of Information Technology Methods and Sources ................................ ......... 23 3.10 Planning ................................ ................................ ................................ ................. 25 3.10.1 The QMP as the Foundation of QMS Planning ................................ ........ 25 3.10.2 Commitment to Project - Specific Planning ................................ ............... 25 3.11 Implementation of Work ................................ ................................ ........................ 28 3.11.1 General Considerations ................................ ................................ ............. 28 3.11.2 Documentation of Implementing Procedures ................................ ........... 28 3.11.3 Sample and Data Integrity ................................ ................................ ........ 29 3.11.4 Performance Monitoring ................................ ................................ ........... 29 3.11.5 Deviations from Approved Processes ................................ ....................... 29 3.12 Assessment, Oversight, and Response ................................ ................................ ... 30 3.12.1 Ty

pes of Assessments ................................ ................................ ............... 30 3.12.2 Assessment Pl anning ................................ ................................ ................ 31 3.12.3 Assessment Conduct and Follow - Up ................................ ........................ 32Handbook QMP iv December 2012 Agency Review Draft 3.13 Quality

Improvement ................................ ................................ ................................ . 34 3.14 Data Review, Validation and Verification, and Data Usability Reporting ............ 34 3.14.1 Planning for Data Review ................................ ................................ ......... 34 3.14.2 Special Considerations for Data Review ................................ .................. 35 3.14.3 Related Ty

pes of Reviews ................................ ................................ ........ 36 **REFERENCES ................................**  **................................ ................................**  **............................ 37**  **APPENDIX A TERMS AND DEFINITIONS ................................**  **................................ .........**  **39**Handbook QMP v December 2012 Agency Review Draft **LIST OF ACRONYMS** ANSI American National Standards Institute ASQ American Society for Quality CI O Chief Information Officer DQA Data Quality Audit DQA Director of Quality Assurance EAS EPA Acquisition System GI S Geographic Information Sy stem IA Inter- agency

Agreement IGMS Integrated Grants Management Sy stem IQ G In formation Quality Guidelines ISO International Organization for Standardization ( not an acronym ) IT In formation Technology OEI Office of Environmental Information PDR Pre - Dissemination Review PE Performance Evaluation QA Quality

Assurance QA M anager Quality

Assurance Manager QAPP Quality

Assurance Project Plan QARF Quality

Assurance Review Form QC Quality

Control QMP Quality

Management Plan QMS Quality

Management S ys tem QSA Quality

S ys tem Audit SI O Senior Information Offi cial SOP Standard Operating Procedure TSA Technical S ys tems Audit UFP Uniform Federal PolicyHandbook QMP 1 December 2012 Agency Review Draft **CHAPTER 1 INTRODUCTION**  **1.1 BACKGROUND AND PURPO**  **SE** This section relates to Standard Clauses 1 Purpose, 2 S cope and Applicability

, and Annex A , Introduction (

Clause A1). This h andbook assists users with creation of Quality Management Plans (QMPs) or equivalent documents to satisfy

the requirements of the EPA Quality Pro gram as defined by

the following two EPA Standards:  Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations, (CI O 2106 - S - 01), “Internal Standard” (EPA 2013 a); and  Quality Standard for Environmental Data Collection, Pro duction, and Use by Non -EPA (E xternal) Organizations, (CIO 2106 - S - 02), “External Standard” (EPA 2013 b). This h andbook is not a standalone document, but intended to be used in conjunction with the Standards for the preparation of QMPs that satisfy EPA requirements. It is offered as a helpful resource for those who need to prepare QMPs that comply with these EPA Standards. It can be used b y both EPA organizations and by organizations conducting activities through external agreements with EPA. This h andbook is intended to help its users prepare QMPs that summarize the overall management structure in which individual environmental data operations will take place and describes in general terms the quality tools that the organization will apply to individual proje cts and who is responsible for implementing them. By definition, h andbook is, not mandatory , and application is at the discretion of the user. This handbook offers guidelines, advice, and examples that will help users to develop QMPs that satisfy the requirements of the Standards. The QA/R- 2 document , EPA QA/R - 2, Requirements for Quality Management Plans, has been withdrawn except for limited use under special circumstances. The QA/R- 2 document is no longer used for developing QMPs except where permitted by the Standards. **1.2 Q**  **UALITY MANAGEMENT PL ANS IN THE EPA QUALI**  **TY MANAGEMENT SYSTEM**  This section relates to Standard Clause 7.2 Quality Management Plan, 7.5.2 Combined QMPQAPP and Annex A Introduction (Clause A1). The Standards identified in the previous section require EPA organizations ( CI O Standard 2106 - S - 01) as well as non - EPA organizations operating under an external agreement with EPA ( CI O Standard 2106 - S - 02) to develop, document, and implement a Q uality M anagement S ys tem (QMS) that conforms to applicable EPA policies and procedures. A QMS is a structured and documented management system describing the policies, objectives, principles, organizational authority , responsibilities, accountability, and implementation plan of an organization for ensuring quality in activities involving environmental data .Handbook QMP 2 December 2012 Agency Review Draft The QMS provides the framework for planning, implementing, documenting, and assessing work performed b y the organization and for carrying out quality assurance (Q A ) and quality control ( QC ) activities for environmental data operations and environmental technology performance activities . A key step in the development and documentation of an organization’s Q MS is the creation of a Quality Management Plan (QMP) . The QMP is unique to an organization and describes the management controls and technical processes and practices necessary to plan, implement, document, and assess the effectiveness of the organization ’s QMS. Ingeneral, a QMP provides a road map for carrying out the QMS , including its scope, the details and justification for any exclusion, a description of the QMS processes, and roles and responsibilities for QMS implementation . The QMP when sufficiently clear and thorough can serve as the basis for a Quality S ys tem Audit . That is, the QMP document s what quality activities are performed , what management controls are applied to the planning, implementation, and assessment of such activities, and who is responsible for the implementation and its management oversight. The QMP also document s the relationships among all parts of the organization for the application of quality to the organization’s internal and external work. Such documentation enable s the user to understand who does what and when. Organizations (or a group of organizations) may also choose to develop and implement QMPs for large, complex programs involving environmental data collection, production, and use. These QMPs are typicall y referred to as “Programmatic QMPs” and the same criteria as for organization QMPs appl ies to them. The preparation and approval of “Programmatic QMPs” is discussed in Chapter 2. The Standards state that QMPs may be prepared by the following organizations:  All EPA organizations involved with the production, collection, or use of environmental data, including Headquarters Offices, Program Offices, Regions, and National Research Laboratories and Centers . Their sub - organizations may have QMPs as well to document the details of the QMS applied to that sub - organization and its implementation.  All non - EPA organizations performing authorized activities that produce, collect, or use environmental data on behalf of EPA. These may include (but not be limited to): o State s, o Tribes, o l ocal governments, o regulated parties, o entities party

to EPA settlement or consent agreements resulting from enforcement actions, o v olunteer organizations, o c ontractors, o c ooperative agreement holders, o grantees, o o ther federal governmental department s and agencies, o n on - governmental organizations, o international governments/organizations, and o e ducational institutions.Handbook QMP 3 December 2012 Agency Review Draft S ome organizations, including some EPA Program Offices and Regions, may elect to require their sub - organizations (e.g., subordinate offic es, divisions within a Region) to develop individual QMPs. Insome cases, an organization may develop additional QMPs at a sub organizational unit (e.g., Divisions within a Region, L aboratories within an Office) in order to provide more implementation details. The lower - level QMPs are tiered to the organization - wide QMP and do not need to repeat the details contained in the latter document. Sub - organizational QMPs can be beneficial because they provide the flexibility

to document any

unique circumstances wi thin the sub - organization that may have bearing on the QMS (e.g., the functions of a Laboratory In formation Management System). The Standards also require that these organizations prepare Quality Assurance Project Plans (QAPPs) or equivalent documents as appropriate , in order to describe and document explicit project details . QAPPs are an output of project planning and provide a road map to project implementation. They are distinct from QMPs in that they focus on particular data - related projects rather than on the organizational level ; they are a project - specific view of QMS implementation . QAPPs describ e in comprehensive detail the QA, QC and other technical activities that can be implemented to ensure that the results of the work performed will satisfy th e stated performance objectives and criteria. They describe the activities of a project in the acquisition of environmental data or information from direct measurement activities, existing data, or generated b y models. The QMP can define and describe the o verall structure and framework for planning, implementing, and evaluating environmental data operations, which in turn are de scribed and documented b y QAPPs for individual projects or tasks. Figure 1 depicts the relationship between QMPs and QAPPs. More in formation on the preparation of QAPPs is available in Handbook on Quality Assurance Project Plans (EPA Handbook H - 05 QAPP ) [ EPA 2013 c ] . As discussed in more detail in the QAPP guidance, QAPPs include those for individual projects or tasks, general or generic QAPPs (which may also be called QA Program Plans) that may be applied to multiple activities, and may include combined QMP - QAPP s for some specific activities that address elements of both the QMP and the QAPP.

**Figure 1**  **. Functional**  **Relationship of QMP and QAPPs to the QMS for an Organization** Handbook QMP 4 December 2012 Agency Review Draft **1.3 INTENDED**  **AUDIENCE** This section relates to Standard Clause 2.1, Scope, and Annex A , Quality Management Plan Requirements (

Clause A3). This document is intended primarily for those responsible for writing , reviewing , approving , or using QMPs in accordance with EPA Standards. It is also useful for anyone who wants a detailed understanding of the role of QMPs in EPA’s QMS , or more generall y for those seeking an example of how a Q uality M anagement S ys tem can conform to national or international consensus standards. **1.4 PERIOD OF APPLICABIL**  **ITY** This section relates to Standard Clause 7.2 Quality Management plan, and Annex A Agency Review and Ap

proval of the QMP (

Clause A2.3

). This handbook is valid for five ye ars from publication. Then it will be subject to review , resulting in one of the following :  revis ion ,  reaffirm ation without change, or  w ithdraw al . If circumstances warrant, this handbook may also be revised during the course of the five - ye ar period. EPA QMP s developed in accordance with CI O Standard 2106 - S - 01 will ty pically have a valid approval for a period not to exceed five ye ars (as defined in the Standard) , though they can be revised at any time to reflect mission or scope change, reorganization, the emergence of new tools, or any

other pertinent changes to the QMS . QMP s prepared b y external parties in accordance with CI O Standard 2106 - S - 02 may have an EPA approval valid for no more than five ye ars for State, local, and Tribal governments or the length of the extramural agreement for all other extramural agreement holders. The period for which a Quality Management Plan is valid is defined in the Quality Management Plan of the EPA organization sponsoring the work . They also can be revised at any time to reflect changes in scope or other issues affecting the implementation of the QMS. **1. 5**  **SUPERSESSION** This document , EPA Handbookfor Quality Management Plans ( CI O Handbook 2106 - H - 02 QM P ) is the original issue of QMP handbook that conforms to the current editions of CI O Policy 2106 and CIO Procedure 2106 - P - 01 as well as CIO Standard 2106 - S - 01 and CI O Standard 2106 - S - 02 .Handbook QMP 5 December 2012 Agency Review Draft **CHAPTER 2 QUALITY MANAGEMENT PLAN PREPARATION**  **, SUBMISSION, REVIEW**  **, AND APPROVAL 2.1**  **Q UALITY MANAGEMENT PL**  **AN P**  **REPARATION** This section relates to Standard Clause 7.2 Quality Management Plan, and Annex A QMP Preparation Responsibility ( Clause A2.1). In EPA, the organization’s senior management ensures the preparation , approval, and implementation of a QMP that covers all environmental programs for which the management is accountable. In practice , these activities are often delegated to other managers and staff. For non - EPA organizations , the organization’s senior management ensures the preparation of a QMP (or equivalent acceptable document content) that covers all work specified by the external agreement and for which the organization’s management is accountable. (See Sections 2. 4 and 3. 3 of this handbook for examples of equivalent acceptable QMP content documents.) In both cases, senior management includes the senior executive responsible and accountable for mission accomplishment and overall operations. Senior management ensure s that the QMS documented in the QMP conforms to the requirements of th e applicable Standard. The same provisions apply to the preparation of “Programmatic QMPs . ” Management authority and quality- related responsibilities for the Program are clearl y established before the QMP is prep ared. Managers and staff involved in preparing QMPs can use this document in conjunction with the applicable Standard . **2. 2**  **Q UALITY MANAGEMENT PL**  **AN SUBMISSION** **AND MANAGEMENT**  **APPROVAL** This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A QMP Preparation Responsibility , QMP Submission and Approval , and Agency Review and Approval of the QMP (

Clauses A2.1 , A2.2, A2.3). For EPA organizations , the QMP is approved internally and signed b y the organization’s senior manager a s we ll as its Se nior Information Official (SI O). The senior manager and SIO indicate their approval of the QMP on the signature page of the document. The senior managers are listed in the Standards. In most instances, QMPs below the Regional or Program Office level can be signed b y designated representatives for the SIO. These delegations are normally documented at least in the QMP but may also be done through the official delegations s ys tem. For a “Programmatic QMP,” a responsible senior manager is designated as the signing official for the QMP. The QMP is also approved and signed b y the QA representative to the senior manager of the organization or Program . This is ty pically the Director of Quality Assurance (DQA) or Quality Assurance Manager ( QA M anager) for the organization.Handbook QMP 6 December 2012 Agency Review Draft The senior manager may also require concurrence on the QMP by appropriate subordinate line managers to document the acceptance of implementation responsibilities in the QMS . **2. 3**  **Q UALITY MANAGEMENT PL**  **AN REVIEW AND APPROV AL FOR USE**  F ollow ing internal approval, an EPA QMP is submitted to the Office of Environmental In formation ( OE I) Quality Staff for Agency- level review and approval in accordance with CIO Standard 2106 - S - 01. Each QMP is reviewed to determine compliance with Agency- wide Qual ity Program requirements , including conformance to th e Internal Standard. The review of the QMP focus es on the substance of the group’s QA management process and not on format or technical details. QMPs that adequately address all program elements describe d in the Standard are ty picall y approved for a period not to exceed five years. The next section describes s ituations that may cause a QMP to be revised. For non - EPA organizations , the QMP is reviewed and approved b y the organization’s senior management as well as the designated QA representative of the senior management prior to its being submitted to EPA. The senior manager may require concurrence on the QMP by appropriate subordinate line managers to encourage the acceptance and implementation of the QMS. Following organization approval, the QMP is submitted to the EPA organization responsible for the external agreement for review and approval in accordance with CIO Standard 2106 - S - 02 . Each EPA organization define s its process for reviewing QMPs submitted under external agreements. This process include s procedures for verifying that the QMP is consistent with the EPA organization’s goals and for notifying the organization of deficiencies to be addressed in the QMP prior to its acceptance b y EPA. QMPs prepared b y external parties may have an EPA approval valid for no more than five ye ars for State, local, and Tribal governments or the length of the extramural agreement but not to exceed five years for all other extramural agreement holders. The period for which a Quality Management Plan is valid is defined in the Quality Management Plan of the EPA organization sponsoring the work. **2. 4**  **Q UALITY MANAGEMENT PL**  **AN REVISIONS** This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A QMP Revisions (

Clause A2.4

). For EPA organizations , although the QMP approval by the Agency may

be valid for up to five ye ars, all Agency

QMSs are reviewed at least annually by their organizations (as required b y CI O Procedure 2106 - P - 01) . This review include s an evaluation of the effectiveness of the QMP and its implementation . The process of developing and annuall y updating the QMP provides an opportunity

for management and staff to review and clarify roles and responsibilities, to address problem areas, and to acknowledge successes. Having an accurate and relevant QMP at all times is an essential part of every QMS and keep s technical staff informed about the quality program . Changes in quality policy and procedures are documented in a timely fashion by

QM P revisions. In general, a cop y of any EPA QMP revision s made during the year are encouraged to be submitted to the OEIQuality Staff as an attachment to , or as part of , the organization’s QA Annual Report and Work Plan which is required by CIO Procedure 2106 - P - 01 .Handbook QMP 7 December 2012 Agency Review Draft A revised QMP may

be submitted to the Quality Staff for review at any

time . Revisions occur under any of the following conditions:  expiration of the five - ye ar approval duration of the QMP ;  major structural change to the organization (e.g., addition or deletion of a division or branch) that requires Agency review ;  significant change in the organization’s mission, such as may occur in the reauthorization or revision of enabling environmental legislation, or reduc tion in resources (e.g., budget allocation, reduction in personnel) that may adversely impact the QMS;  results of audits or assessments and corresponding corrective actions impacting the QMS ; or  any other significant change to the organization’s QMS ( such as the adoption of new quality tools or processes ). In general, the decision to revise a QMP is the responsibility of the organization. Insome cases, a formal revision of the QMP may not be necessary , but it is helpful if the QMP describe s how changes will be retained and documented until a formal revision is needed and how the changes will be disseminated to users. As required b y the Standards, all appropriate personnel performing work for the organization through external agreements (e.g. , active contractors, assistance agreement holders) are advised of any changes that may affect their work for EPA so that they may be aware of and appl y any new requirements. To enhance performance, the QMP can describe the process for such notifications and who is responsible. For non - EPA organizations , a QMS review at least annually by the organization include s an evaluation of the effectiveness of the QMP. When necessary , the organization revise s its QMP to incorporate changes to the QMS. Similar to the conditions cited above requiring revision and resubmission for EPA QMPs, revisions to a non - EPA organization’s QMP may become necessary

due to :  a change in the scope ( such as a modification to the contract statement of work) in the external agreement ;  significant changes in the non - EPA organization’s mission ;  an organizational change within the non - EPA organization or its subcontractors that require s notification to EPA and other affected parties; or  any other significant change to the organization’s QMS ( s uch as the adoption of new quality tools or processes ) . A revised QMP may

be submitted at any

time and is subject to the same review and approval processes as the original QMP. All personnel (or affected parties) performing work for the organization or program can be notified of significant changes to the QMS and the QMP as required b y C IO Standard 2106 - S - 02 .Handbook QMP 8 December 2012 Agency Review Draft **2. 5**  **COMBINED QUALITY MAN AGEMENT PLAN**  **WITH QUALITY ASSURAN CE**  **PROJECT PLAN** Th is section relates to Standards Clause 7. 5 .2 Combined QMP- QA P P , and Annex B QAPP Preparation Responsibilities and App rovals ( Clause B2 .1). In some circumstances, EPA may recommend the preparation of a combined QMP and QAPP as a more appropriate means to document organizational level and project- level environmental data activities , particularl y when there is no available QMP for the organization performing the work . Examples may

include (but are not limited to) research projects, inspection programs, and monitoring networks involving multiple participants, among others. In such c ases , a combined QMP and QAPP can be a more appropriate means to document organizational - level and project level environmental data activities, p articularl y when a full QMP is not needed. Combined QMP QAPP s are not used for external agreements containing multiple projects that may be determined over the life of the agreement (e.g., mission contracts, program grants). A combined QMP - QAPP incorporate s some elements of the QMP and some elements of the QAPP. In general, such documents are largel y expanded QAPPs. As with any QMP, it can document the management controls that will be applied to the planning, implementation, and assessment of quality activi ties for the program or project , and can describe the processes to be followed to carry out those activities. It is EPA’s responsibility to ensure that all elements of a combined QMP - QAPP , as defined b y the EPA organization, are addressed b y the organization performing the work. The combined QMP - QAPP is subject to the same EPA review and approval as a QAPP.

**HELP BOX –**  **WHEN SHOULD A QMP BE REVISED?** Situations that might make it advisable to revise an existing QMP include but are not limited to:  change in reporting relationships among Quality Managers or other quality management personnel;  a corrective action that prompts the need to revise the existing QMS;  expansion of a laboratory facility to incorporate new categories of methods and anal yt es; and  si gnificant decreases in budget allocations to the QMS. On the other hand, the following situations may not lead to a formal QMP revision , but could be addressed through using a letter of notification to the approving organization :  personnel changes (e.g., a new Quality Manager);  a corrective action that facilitates implementation of the existing QMS; and  revisions to existing Standard Operating Procedures.Handbook QMP 9 December 2012 Agency Review Draft As with any

quality document produced b y or for an EPA organization, the combined QMP QAPP performs as part of the Quality Management System (QMS) of the EPA organization sponsoring the work and will be expected to align with that QMS. Further information for a combined QMP - QAPP is provided in Handbook on Quality Assurance Project Plans (EPA CI O Handbook 2106 - H - 05 QAPP ) , Se ction 1.6 . **HELP BOX –**  **COMBINED QMP/QAPP** Some examples of situations in which a combined QMP/QAPP might be appropriate may include (but are not limited to):  routine inspection programs where inspections are conducted in the same manner for all visits and samples of opportunity can be expected (e.g., EPA lead and/or radon programs);  small organizations that do infrequent work with EPA funds (e.g., competitively awarded one - time grants, environmental education grants);  one - time mobilization to a previousl y unvisited field location, such as for monitoring assistance to a Tribal location;  necessary

quick- turnaround data anal ys is that may involve new laboratory facilities, methods, or equipment;  environmental measurements related to technology performance, wher e QMS planning for the operation of the technology is already

addressed b y another document;  academic research activities of a one - time nature wherein a formal QMP is more detail than needed; or  a monitoring program conducted b y a volunteer organization wh ere the size and structure of the organization is limited.Handbook QMP 10 December 2012 Agency Review Draft **CHAPTER 3 HANDBOOK**  **ON Q UALITY**  **M ANAGEMENT**  **P LAN**  **CONTENT** This introduction does not correlate directly to a clause in the Standards . Annex A of CI O Standard 2106 - S - 01 and Standard 2106 - S - 02 list the elements that must be addressed in a QMP. However, the Standard also indicates that all elements may not appl y to an organization or program. If that is the case, then the QMP provides explanation and rationale to document why the QMP element does not appl y. (It is likel y that this will occur when “ Programmatic QMPs ” are being developed.) The QMP elements listed in the Standards are as follows:  g eneral guidelines ;  c onformance with policies, procedures, and standards;  o rganizational and p rogram f ield of a pplication;  o rganization, responsibility , and a uthority ;  QMS description;  p ersonnel competence;  e xternal agreements;  documents and records management;  u se of information technology methods and sources;  p lanning;  implementation of work;  a ssessment, oversight, and response;  data review, validation and verification, and data usability reporting ; and  quality

improvement. In the sections to follow, suggestions will be provided for each of the QMP elements. Since the text in the Standards is very detailed, the discussions here will refer to the Standards often. It will be beneficial for developers of QMPs to use this document in conjunction with the applicable S tandard. **3. 1**  **G ENERAL GUIDELINES**  This section relates to Standard Clauses 1, 2, 7. 2, Purpose, Scope and Applicability, Requirements, Quality Management Plan, and Annex A General Requirements ( Clause A3 ).Handbook QMP 11 December 2012 Agency Review Draft 3. 1 .1 Overview of QMP Content This section relates to Standard Clauses 7..2, Quality Management Plan, and Annex A General Requirements (

Clause A3.1). A QMP address es the requirements of CI O Standard 2106 - S - 01 (for EPA organizations) or CI O Standard 2106 - S - 02 (for non - EPA external organizations). It contain s descriptions of the processes b y which the organization plans, implements, documents, and determines the effectiveness of its quality management practices, including QA and QC activities, to help management to obtain results of its technical work that are of the ty pe and quality

needed for their intended use. The QMP also reflect s the organization’s commitment to quality management principles and practices. Including discussion of those activities, policies, and procedures that are common to all projects can reduce duplication between QMPs and QAPPs. Such discussions in the QMP provide an “umbrella” under which individual project activities may be perf ormed, and there is then no need to repeat them in each of the organization’s QAPPs. Documents with a different title, such as a Quality Manual or a Program Management Plan , if they fulfill the purpose of a QMP and contain the content as detailed in this chapter are accep table. For EPA organizations, the QMP can be prepared in accordance with the Agency ’s Open Government Plan, reflecting the principles of transparency , participation, and collaboration. 3. 1 .2 Administrative Elements This section relates to Standard Clauses 7. 2, Quality Management Plan, and Annex A General Requirements, and Organization, Responsibility, and Authority (Clauses A3.1 and 3.4 ). The format of a QMP may be tailored by

the organization, but in every case the QMP include s the following administrative elements :  title page ;  table of contents ;  approvals page with appropriate signatures;  revision history ; and  applicable references. A document control format may be helpful to organize the QMP and facilitate future revisions. The completed QMP can reflect that it has received the approvals described in Section 2.2. Figure 3 is an excerpt of a hy pothetical QMP approval page incorporating document control format.Handbook QMP 12 December 2012 Agency Review Draft **F igure 3**  **. Example of Document Control Format**  3. 1 .3 Application of Graded Approach in QMPs This section relates to Standard Clause 7.2 Quality Management Plan . The QMP may

incorp orate , where applicable, the concept of the graded approach, which is the process of applying quality and management controls to an item or work activity according to the intended use and degree of confidence needed in the results. For instance, the QMP may state that more comple x or significant projects carried out by the organization w ill have a more rigorous QAPP , whereas simpler or less significant projects may have a streamlined QAPP. It may also embody

the graded approach by

specifying that required approvals and reviews can be delegated under defined circumstances. It is important to remember that the graded approach is not an opportunity

to “cut corners” b y by

passing critical management controls. All EPA QMPs can certainl y include the graded approach, but non - EPA organizati ons may not need it. The application of graded approach to specific work is more commonly found at the project level . A more detailed discussion of the concept and its application to projects can be found in Handbook on Quality Assurance Project Plans (EPA Handbook H - 05 QAPP , Section 1.3 ) [ EPA 2013 c ] . For both the QMP and the QAPP, the level of detail can be commensurate with the stated goals of the organization or project and the intended use of the results of the environmental data collection or productio n. **DCN: QMP -**  **01 -**  **00 Revision: 0**  **Date: July 2010 Page**  **12 of**  **49** 

### APPROVAL PAGE **Deputy Assistant Administrator:**

**Printed Name Signature**  **Date Director of QA:** 

**Printed Name Signature**  **Date**Handbook QMP 13 December 2012 Agency Review Draft **3. 2**  **QUALITY MANAGEMENT P LAN CONFORMANCE**  **TO REQUIREMENTS**  This section relates to Standard Clause 6 R elated Regulations, Policies, and Procedures, and Annex A Conformance with Policies, Procedures, and Standards ( Clause A3.2). For EPA organizations , a QMP can c ite the external and internal policies , procedures , standards, and guidelines to which the QMP conforms. This help s users to under stand the scope and extent of the QMS more clearl y. These may include the documents listed in Figure 1 previousl y. In addition, see Section 6 of CI O Standard 2106 - S - 01 for a list of other potentially applicable guidelines. In addition, Clause 1 of the same Standard s tates that Informative Annexes (or parts thereof) may

become requirements if so decided by the organization . Documenting this commitment in the organization’s QMP makes it clear when and to whom the organization expects it to apply . N on - EPA organizations may be required to demonstrate and document that the QMP conforms to CI O Standard 2106 - S - 02 as part of a procurement solicitation, assistance negotiation, or other pre - award condition, and for any relevant terms of an applicable external agre ement. In addition, the QMP can cite any other requirements that are pertinent to the organization . Examples of other standards and policy documents include (but are not limited to) :  ISO 9001 (Current Edition), Quality M anagement S

ystems

– Requirements [I S O 2008 ]  Uniform Federal Policy for Implementing Environmental Quality Systems (EPA - 505 - F - 03 - 001) (Current Edition) [EPA 200 5 ] These sources could be used to produce quality management documents that are equivalent to an EPA QMP. For example, under ISO 9001 the equivalent quality management document is usually called the “quality

manual” but other titles may be used. **HELP BOX –**  **GRADED APPROACH EXAMPLES** The application of the graded approach concept to the operation of a QMS will vary from one organization to another. Here are a few examples of how the graded approach may be reflected in a QMP:  a categorization scheme for ranking projects conducted b y the organization by their significance and complexity , with higher- ranking projects receiving more thorough QA planning and review;  establishment of a cost threshold for external agreements, with agreements above the threshold being subject to on - site audits;  streamlined QAPP review and approval for short - duration projects;  records retention schedules tiered to the long - term importance of a given project; and  designation of criteria for determining which of an organi zation’s projects will be subject to formal peer review.Handbook QMP 14 December 2012 Agency Review Draft Conceptually , these documents will address the same general management controls as well as roles and responsibilities that are expected in an E PA QMP. Inaddition, the use of the Uniform Federal Policy

(UFP) format for QAPPs has been accepted by several EPA organizations as equivalent to the requirements given in the CIO Standard 2106 - S - 01 and CIO Standard 2106 - S 02. The UFP reference given above describes the Uniform Federal Policy for the creation of a QMS and related quality

management documents. It is analogous to CIO Standard 2106 - S - 02 and the QMP requirements in the Standard. Users can confirm acceptability of this format with the accepting EPA organization before using it to prepare QMPs or QAPPs .

**3. 3**  **O RGANIZATIONAL AND PR** **OGRAM FIELD OF APPLI CATION**  This section relates to Standard Clauses 2.1 and 2.2, Scope and Applicability , and Annex A Organizational and Program Field of Application (Clause A3.3). QMPs prepared using this handbook can focus on the programs and processes necessary to assure quality in environmental data collection, production, and use . EPA Regions may have Division - and Branch - level QMPs for decentralized Q uality M anagement S ys tems. Such QMPs may be approved at the Regional level (i.e., not by the OEI Quality Staff). Likewise, states may have Office or Division QMPs under their parent state QMP (or independent QMS) that are not reviewed or approved b y EPA when they are outside the scope of the external agreement with EPA. The QMP can c learl y identify the organization s or programs and processes to which i t applies .  For EPA organizations , the QMP can address quality management (i.e., management controls, roles and responsibilities of managers and staf f, quality processes to be applied, etc.) for a specific organizational unit , such as a Region, Headquarters Program Office, or Office of Res e arch and Development Laboratory or Center , or a subordinate organization within those units . The same criteria can also apply to any “Programmatic QMPs” or sub organization QMPs developed by the organization. **HELP BOX –**  **WHAT IS CONFORMANCE?** Conformance (also referred to as conformity) means fulfillment of specific requirements. In particular, a QMP conforms to a Standard or similar document if it addresses all applicable requirements provided in the Standard. Conformance with a Standard is distinct from voluntary use of helpful suggestions found in a Standard. For instance, QMPs created per EPA CIO Standard 2106 - S - 01 will conform to ANSI/ASQ E4, but they

may also use elements of I SO 9001 without the need to conform fully to the ISO standard. Conformance is also distinct from compliance , which is a legal term for meeting the requirements of specific laws and regulations.Handbook QMP 15 December 2012 Agency Review Draft  For non - EPA organizations , the QMP can address quality management for the organization that has an ext ernal agreement with EPA. As indicated above, the same criteria can also appl y to any “Programmatic QMPs” or sub - organization QMPs developed b y the organization. Examples of sub - organizations are listed in Section 1.2 . **3. 4**  **O RGANIZATION**  **, R ESPONSIBILITY**  **, AND AUTHORITY**  This section relates to Standard Clause 7, Requirements, and Annex A Organization, Responsibility, and Authority ( Clause A 3.4). 3. 4 .1 Organizational I ssues This section relates to Standard Clause 7 Requirements, and Annex A Organization, Responsibility, and Authority ( Clause A3.4). The QMP can p resent the organization’s Quality Policy , which is a summary statement of the organization’s commitment to quality in its environmental data operations. This Policy can vary in length and degree of specificity. M inim ally , the Quality Policy can reflect:  senior management’s commitment to and engagement with the QMS;  accountability

for quality at all management and staff levels; and  a commitment to the allocation of adequate resources to the QMS . L eadership on the part of senior management is crucial to the success of the QMS. The QMP can describe the specific steps management takes to ensure that the QMS is understood and implemented throughout the organization’s environmental data functions. These steps c ould include:  serving as a vocal champion of the QMS;  ensuring that quality

management positions are filled by qualified staff who are supported by their management chain;  activel y participating in specific QMS functions such as training and assessments; and  building quality- related commitments into the organization’s employee performance appraisal s ys tem. It is useful to include in the QMP an organizational chart depicting the main components of the organization , and highlighting the locations and reporting relationships of all staff with specific QMS respo nsibilities . The hypothetical EPA organizational chart in Figure 2 presents an example of how QMS - related relationship s can be displayed .Handbook QMP 16 December 2012 Agency Review Draft **F igure**  **2 .**  **Hypothetical organizational c hart depicting QA relationships**  If sub - unit QMPs are tiered under an organization - wide QMP, they can be listed in the main organization’s QMP, and the relationships between the documents can be defined, along with a description of the review and approval procedures for sub - unit QMPs. 3. 4 .2 Roles and Responsibilities This section relates to Standard Clause 8 Roles and Responsibilities, and Annex A Organization, Responsibility, and Authority ( Clause A3.4). The QMP can include discussion of all roles and responsibilities applicable to the QMS, such as :  s enior management ;  l ine management/supervisors ;  quality

management personnel (Director of Quality , Quality Managers, etc.) ;  p roject managers ;  t echnical staff

; and  administrative staff

. It is particularl y important that the roles and responsibilities of specifically designated quality management position s (such as Q uality Manager, Quality Assurance Coordinator, or Quality Assurance Officer) can have detailed explanation in the QMP. For example, representatives authorized to sign QAPPs and/or review technical reports on behalf of the QA Manager can be defined in the QMP.Handbook QMP 17 December 2012 Agency Review Draft Critical to the success of QMS implementation is a full description of and commitment to how quality management personnel are permitted functional independence from l ine management , a s well as direct access to senior management as necessary to address quality

issues. This functional independence can be documented in the QMP and delineated to ensure control of organizational risks that can arise through conflict of interest. 3. 4 .3 Dispute Resolution This section relates to Standard Clause 7.4 Quality System Assessments, and Annex A Organization, Responsibility, and Authority ( Clause A3.4). The QMP can describe the process for resolution of any quality- related dispute s that are not resolved through normal channels and can document those responsible for its implementation . This process can be designed , in general, to ensure that differences in opinion regarding quality sy stem implementation are:  fully documented by

the concerned parties;  resolved at the lowest level of management , if possible;  efficientl y brou ght to the attention of senior management if required to facilitate resolution ; and  subsequently

tracked to ensure that the agreed - upon resolution is accomplished in practice and is documented appropriatel y . Dispute resolution is an important process and it can be carefully designed. A ll managers and staff who could be affected by or participate in such a process can agree to conform to the process. An example of a quality- related dispute would be a disagreement between a line manager and a quality m anager regarding corrective actions that are necessary to resolve an audit finding of non - conformance . The dispute resolution process can ensure that disputes can be addressed and resolved with due process and in a timely manner so that work schedule and performance are not impeded. **3. 5**  **QUALITY MANAGEMENT S YSTEM D**  **ESCRIPTION** This section relates to Standard Clause 7 Requirements, and Annex A QMS Description ( Clause A 3 . 5). The principal purpose of the QMP is to describe the QMS that is being implemented b y an organization or program and to establish the roles and responsibilities for managers and staff for its implementation. Accordingl y, the QMP can summarize the content of the QMS and describe how it is structured . The QMS “big picture ” can be briefly described up front , with the remainder of the QMP then providing details of the QMS.Handbook QMP 18 December 2012 Agency Review Draft The document Guidance for Developing a Quality System for Environmental Programs (EPA QA/G- 1) [ EPA 20 0 2 b ] provides guidance on developing a QMS that conforms to the requirements of the Quality Standards. In addition to this document , users may find it helpful to examine the following documents for their discussions of QMS elements:  ANSI/ASQ E 4, Quality Systems for Environmental Data and Technology Programs

– Requirements with Guidance for Use . [ANSI 2004]  ISO 9001, Quality management systems

– Requirements . [I SO 2008] In general, QMS in the EPA - wide Quality Program embody a Plan - Do - Check- Act quality model. A QMS consists of the quality management controls and processes needed for planning, implementing, assessing, and improving the effectiveness of quality processes and practices applied to the collection, production, and use of environmental data. Management controls are a sy stem of management functions to enable managers to (1) determine that the operations of a program or organization satisfy predetermined goals and objectives and that performance is in line with standards and specifications; and (2) implement any remedial actions needed to ensure that human and other resources are being used in the most effective and efficient way possible in achieving the organization’s mission .

**HELP BOX –**  **QUESTIONS FOR QMP DEVELOPERS** Developers of QMPs for environmental data collection, production, and use should be able to answer questions such as the fol lowing:  Who are the principal stakeholders for our environmental data operations (they may include other organizational units, other levels of government, members of the public, etc.)?  What is our process for identifying, reconciling, and prioritizing thes e stakeholders’ requirements and expectations?  What specific environmental data will be collected, produced, or used to meet these requirements and expectations?  What processes and management controls will be carried out?  What roles and responsibilities wi ll be assigned for the execution of these processes and management controls?  How will we evaluate the effectiveness of our processes and their outputs on an ongoing basis?  How will we know the extent to which stakeholders’ requirements and expectations are being met?Handbook QMP 19 December 2012 Agency Review Draft **3. 6**  **PERSONNEL COMPETENCE** This section relates to Standard Clause . 6 Personnel Competence , and Annex A Personnel Competence (

Clause A3.6). This part of the QMP might b egin with the organization’s policy statement on ensuring the competence of personnel to perform the technical aspects of project work as well as the QA and QC functions that are designed to meet the organization’s project and mission objectives . It would also be benefi cial to describe management responsibility for support and/or commitment to ongoing training in technical fields both currently relevant as well as in advance of initiatives . It is important that the personnel responsible for the implementation of the QMS elements possess sufficient knowledge, experience, and understanding of quality principles, processes , and practices. Personnel competence is generall y composed of the requisite technical knowledge, relevant education, work experience, and specialized training (as required). Personnel competence can be maintained and demonstrated through means such as the following or in combination thereof :  **Training:** Technical and management training, which may include classroom, electronic, and on - the -

job, can be provide d by

the organization as needed.  **Certification:** Statutory , regulatory , or professional certifications may be required to perform certain operations (e.g., laboratory an alyst competency) .  **Observation:** Formal supervisory observation may

be required before a staff member is authorized for independent performance of an activity such as operating measurement and testing equipment. The process es for carrying out these or other measures to ensure personnel competence can be documented in the QMP. Inaddition, the QMP can describe how objective evidence of personnel job proficiency will be documented and maintained for the duration of the project or activity affected, or longer if required by statute or policy. Accurate and up - to - date records on personnel competence are valuable for numerous QMS - related purposes, such as QAPP and SOP development, audit preparation, and design of corrective actions. Such records provide management with documentation that staff members are competent to perform the work assign ed. Workers without adequate competence pose a risk to the successful completion of the work and possibly

to the organization’s liability in the case of an accident. Finally , the QMP can describe how and when any need to implement new or revised training, implement certification of personnel , or define additional personnel qualification s will be reevaluated b y the organization . This is advisable because the organization’s evolving mission may lead to new training requirements.Handbook QMP 20 December 2012 Agency Review Draft **3. 7**  **EXTERNAL AGREEMENTS** This section relates to Standard 01 Clause 7. 11 Management of Quality Requirements for External Agreements, Standard 02 Clause 7.11 Management of Quality Requirements for Subagreements, and Annex A External Agreements ( Clause A3.7) . 3. 7 .1 EPA Organizations Extramural agreements provide EPA with a critical means of accomplishing work related to the Agency ’s mission. Accordingl y, it is very important for the organization to ensure that appropriate quality

requirements are applied to its suppliers, subcontracto rs, and other external agreement holder s. This is vital to the success of the QMS, because many projects involving environmental data collection, production, and use may utilize inputs from outside the primary organization. These may

be focused contributio ns , such as those of testing laboratories; or in other cases, a sub - recipient may be assigned overall responsibility for a major activity such as a large, complex field sampling activity . For EPA organizations , e xternal agreements may include:  acquisition of services and items through contracts, task orders, and other acquisition agreements ;  assistance agreements such as grants and cooperative agreements ;  interagency

agreements negotiated with other Federal, State, Tribal, or local governments ;  international agreements negotiated with governmental, non - governmental organizations, or other entities ;  settlements and agreements (e.g., administrative orders, consent agreements) resulting from enforcement actions ;  sub - awards whose recipients are accountable to th e prime recipient of an external agreement for environmental data activities provided under the agreement ; and  voluntary

partner programs (e.g., voluntary monitoring) . The QMP can describe how the organization will:  determine whether the scope of a propo s ed or implemented external agre e ment includes the collection, production, or use of environmental data; and  ensure that appropriate quality practices and procedures are defined and applied to external agreements as appropriate. An EPA organization can identify , develop, and document the processes and procedures (including management and staff responsibilities) for ensuring that all external agreement holders appl y appropriate quality requirements. EPA o rganizations can identify and document how recipients of external agreements require their own suppliers ( e.g. , contractors, subcontractors, or financial assistance recipients) to maintain a quality s ys tem consistent with EPA requirements. This provision applies only to those suppliers providing services or items directly affecting the quality

of results or products from environmental data activities .Handbook QMP 21 December 2012 Agency Review Draft EPA organizations have considerable flexibility in requiring documentation on the scope of quality practices in external agreements for environment al data. For example, under acquisitions, an EPA organization can provide in the QMP a summary o f the organization’s process for implementing the EPA Contracts Management Manual [EPA 2009a] specification for using a QA Review Form (QARF). In addition, the QMP may describe how the QARF is integrated in the EPA Acquisition System (EAS). Similarly, the QMP may

include a description of the process for assistance agreements which is documented through the Integrated Grants Management System (IGMS). Moreover , the QMP may describe the processes for Inter- A gency A greements ( IA) and other assistance agreements . For further information, EPA organizations may refer to EPA Order 5700 Change 6, Assistance Administration Manual [EPA 1984], and to EPA Order 1610, Interagen cy Agreement Policies, Procedures, and Guidance Manual [EPA 2008a]. Each EPA organization’s Quality Manager (or designee) can be authorized to review documentation for pending external actions (e.g., Statements of Work and contractor proposals) to ensure that appropriate quality- related requirements are being addressed and incorporated. This authority can be clearly defined in the QMP. Moreover, for projects involv ing multiple EPA organizations with shared responsibilities , the QMP can describe h ow the lead responsibility for QA will be determined and assigned. In the case of IAs, when EPA provides the funding or a substantive majority

of resources expended , the application of QA and oversight responsibility remaining with EPA is negotiated into the agreement . When EPA does work for another agency or department, the EPA organization’s quality management s ys tem and QA oversight is applied, using a graded approach, by default unless otherwise negotiated in the agreement. 3. 7 .2 Non - EPA Organizations N on - EPA organizations that have external agreements with EPA can discuss how the quality requirements in their QMS will flow down to their suppliers and sub - recip i ent s associated with environmental data operations . However, any requirements on sub - recipient s (e.g., sub contractors ) are made through the prime recipient (e.g., contractor) , as there is no direct contractual relationship between EPA and external agreement sub - recipients . The QMP can l ist the functions that are likel y to be implemented by suppliers or sub - recip i ent s (e.g., field sampling services, laboratory testing, and information technology support). For each of those areas, the QMP would then describe how the QMS will be applied to sub - recipients . For example, non - EPA organizations may :  require lower- tier sub - recip i ents to prepare their own QMPs or QAPPs;  review procurement documents for sub - recipients;  conduct audits or on - site inspections of supplier facilities and operations;  conduct validation of data supplied by contract laboratorie s;  maintain acceptance criteria for lower- tier sub - recipients; or  issue stop - work orders or requests for corrective actions to sub - recipients.Handbook QMP 22 December 2012 Agency Review Draft **3. 8**  **D OCUMENTS AND**  **R ECORDS MANAGEMENT**  This section relates to Standard Clause 7.1 0 Documents and Records Management , and Annex A Documents and Records Management ( Clause A3.8). An organization’s process for management of quality- related documents and records is often crucial to the success of a QMS, which is ultimately not just about achieving quality performance, but also about demonstrating (that is, documenting) that quality performance has been achieved. Accordingl y, i t is very important the documents and records affecting the quality of the organization’s environmental data operations be identified and documented in the QMP. In general, the QMP can :  describe the maintenance and management of major quality planning documents, including the organization’s QMP and QAPPs ;  define other documents that will be considered quality- related and managed accordingl y;  describe characteristics (e.g. of function or content) to be used in identifying qualityrelated documents not anticipated by or named in the QMP ; **HELP BOX –**  **EXAMPLES OF QUALITY REQUIREMENTS FOR SUPPLIERS** Suppliers of services and indirectly through the su

ppliers, their sub - recipients involved in environmental data collection, production, and use may be required to:  develop QMPs describing their quality s ys tem;  develop QAPPs describing and documenting explicit project details;  develop SOPs for certain proce sses and practices;  commit to fulfilling specific QA and QC requirements established by the sponsoring organization;  submit to audits to confirm their capability to satisfy quality criteria (e.g., performance evaluations, technical sy stems audits);  obtain review and approval from the sponsoring organization for any mid - course changes ;  demonstrate that all QA and QC requirements have been carried out in practice ;  provide objective evidence of quality in products and services provided by their own suppliers and sub - recipient s, including source selection, source inspections, supplier audits, and examination of deliverables; and  review all deliverables against the acceptance criteria established b y the sponsoring organization.Handbook QMP 23 December 2012 Agency Review Draft  document how all technical guidance documents are prepared, reviewed, approved, issued, used, and revised (this can appl y to internally developed and externally acquired documents ) ;  define what records will be generated and ho w they will be maintained;  describe how the organization will ensure that documents and records accurately reflect completed work;  discuss retention times, archiving, retrieval, and disposition of quality- related documents and records;  describe processes f

or handling documents and records to assure their accessibility, protection from damage and deterioration, and means of retention, including discussion of the roles and responsibilities for management and staff;  document chain of custody and confidentiality procedures for evidentiary records; and  commit to compliance with all statutory , contractual, and assistance agreement requirements for records from environmental programs , including providing adequate preservation of key

records necessary

to support the mission of the organization . EPA organizations are required by C IO Standard 2106 - S - 01 to comply with Agency

policy on documents and records. The QMP developers can refer to the provisions of Se ction 6 of the EPA Records Management Policy [EPA 2009 b ] to define this process for their QMP . This section lists the minimum requirements for an EPA records management program, including creation, management, maintenance, protection, and disposal of records. In addition, QMP developers can document any additional organization - specific processes for management of documents and records as described above . For non - EPA organizations , the QMP can document or refer to the organization’s own records management po licy

and procedures and to the applicable terms and conditions of the external agreement . In general, Federal regulations governing external agreements will specify documents and records management requirements. Adherence to such requirements can be affirmed in the organization’s QMP. Any differences for specific projects can be documented in the QAPP; see Handbook on Quality Assurance Project Plans (EPA CI O Handbook 2106 - H - 05 QAPP ) for more information . **3. 9**  **U SE OF INFORMATION TE**  **CHNOLOGY METHODS A ND SOURCES**  This section relates to Standard Annex A Use of Information Technology Methods and Sources ( Clause A3.9). Especiall y with recent advances, information technology (IT) plays an increasingl y important role in the collection, production, and use of environmental data. As noted in CI O Standards 2106 - S - 01 and 2106 - S - 02, IT encompasses a wide range of activities, functions, and sources for environmental data collection, production, and use , including general computer software programs, computer models, information sy stems, and web - based applications.Handbook QMP 24 December 2012 Agency Review Draft The QMP can p resent the organization’s general approach to appl ying QA and QC processes and practices to applications involving information s ys tems management. A ctivities addressed by the QMP can include c onsideration of acquired results from IT methods and sources, such as services that collect and provide access to data and information. Organizations can develop and document the general processes, procedures, and personnel responsibilities for determining when QA planning documents (e.g., QAPPs) are needed and for evaluating the suitability

of environmental data obtained from information technology sources . The QMP can describe and document the processes related to the preparation, review, appro val, and use of QA planning documents for all IT - related applications. Developers of QMPs may find the discussions in the Standards and in their Annexes A and B to be useful in defining the organization’s QMP content for IT activ ities. Environmental data p roducts generated using IT applications and sources include (but may not be limited to) the following:  **C om** **puter mathematical models and other electronic methods :**  Environmental data produced from computer programs and models, mathematical models, and other electronic methods and sources [ e.g., Geographic Information Sy stems ( GIS) ] can be reviewed, evaluated, verified, and validated in order to confirm their suitability for use. For more information, see the Web site of EPA’s Council for Regulatory Environmental Modeling (http://www.epa.gov/crem/), which is intended to promote consistency and consensus among environmental model developers and users.  **EPA -**  **owned information systems :**  Q uality principles can be applied to the development, modification, operation, and management of EPA information systems to ensure that the sy stem is designed to satisf y the intended uses and user requirements and that data maintained in the system can be a udited and verified from entry to retrieval. Examples of EPA information sy stems include CERCLI S (Superfund) , TRI (toxics) , and SDWI S (drinking water); see http://www.epa.gov/enviro/index.html which is a centralized repository

of these s ys t e ms.  **O ther information systems and**  **the Internet :**  Environmental data collected from other information sources (including the Internet) can receive special attention to ensure that the data can be used for their intended purpose. The QMP can commit to the development and documentation of p roject - specific evaluation criteria in applicable QAPP s .  **Computer software and hardware :**  C omputer software and hardware relevant to the mission and objectives of the organizati on can be controlled appropriatel y to satisfy the organization’s objectives. QMP developers can refer to applicable OEI procedures and guidance for information on software and hardware controls ; these resources are compiled at http://www.epa.gov/irmpoli8/p olicies/index.html . For EPA organizations , the QMP can discuss the organization’s commitment to complying with all applicable CIO policies, procedures, standards, and guidelines, as well as organization specific policies and procedures. It can describe the process for ensuring that data and information produced b y or on behalf of the Agency meet applicable requirements and standards.Handbook QMP 25 December 2012 Agency Review Draft For non - EPA organizations , the QMP can de sc ribe organization - specific sy stem s, policies , and approaches. If the applicable external agreement provides for compliance with specific EPA CIO polices, the QMP can also reflect that commitment. In addition, a QMP for ei ther an EPA or an external organization can discuss any particular information sy stems issues that could be applicable across the range of environmental data operations covered in th e QMP. Examples may include (but are not limited to) :  f ield data logging processes ;  Laboratory

Information Management S ys tems ;  m odeling or other uses of data ;  e lectronic data bases ;  c omputer programs used to analyze environmental data ; and  sy stems for posting and sharing data (such as the Environmental Information Exchange Netw ork) . Project - specific processes, procedures, and evaluation criteria when developed are documented in the corresponding QAPPs. **3.1 0**  **PLANNING** This section relates to Standard Clause: Systematic Planning (7. 7 ), and Annex A Planning ( Clause A3.10). 3. 10 .1 The QMP as the Foundation of QMS Planning This section relates to Standard Clause: Systematic Planning (7. 7 ), and Annex A Planning (Clause A3.10). The QMP can describe how the organization satisfies the planning requirements of the QMS. Planning is generally

regarded as the most important phase of a quality s ys tem; in the absence of a sy stematic focus on planning along with an appropriate set of planning tools, a QMS can devolve into a mere inspection program for detecting flawed environm ental data . The foundation of the planning approach built into the QMS is provided by the QMP itself. The specific QMP elements described in this handbook beginning with Section 3. 7 ( en suring personnel competence, promoting quality performance in external agreements, and so on) constitute the building blocks of the organization’s overall plan for ensuring effective delivery of environmental data that meet the needs of the organization’s stakeholders and customers. 3. 10 .2 Commitment to Project - Specific Planning This section relates to Standard Clause, Systematic Planning (7. 7 ), and Annex A Planning (Clause A3.10).Handbook QMP 26 December 2012 Agency Review Draft In addition to this overall planning framework, the QMP can make a specific commitment to the use of project - level sy stematic planning for any activities involving the collection, production, and use of environmental data. Sy stematic planning for a project entails the following aspects at a minimum:  defining the project, or creating an overall framework upon which the project will be developed;  e stablishing the scope of the investigation, including its temporal, physical, and logistical boundaries;  documenting the needs or demands of the project, which may include conceptual models or sampling and anal ytical methods and procedures; and  defining protocols for tracking and reporting on project progress. The particulars of the s ys tematic planning approach depend on the focus of the project

– whether it involves sampling and anal ys is, data analysis, a modeling application, or another type of environmental data operation. But in all cases, the organization can establish and document in the QMP a sy stematic planning process that :  identif

ies all relevant customers , stakeholders, and interested parties (as applicable) , as well as their needs and expectatio ns, for the results of the work to be performed ;  identifies any internal or external resource, equipment, or technical expertise necessary for successful performance;  identif

ies the technical and quality goals that meet the needs and expectations of the cu stomer ;  translate s the technical and quality goals into specifications that will produce the desired result ;  consider s any cost and schedule constraints within which project activities are required to be performed ;  specif ies the QA and QC activities required for tracking project performance; and  identif

ies measurable acceptance criteria or performance measures b y which the results will be evaluated and the suitability of the environmental data for their intended use will be determined. The general planning processes (including applicable design steps) can be described. All planning documentation can be reviewed and approved for implementation by authorized personnel before the affected planned work commences. Such documentatio n includes, but is not limited to, work plans, schedules, standard operating procedures (SOPs), and QAPPs. The QMP can describe in specific terms the organization’s process for developing, reviewing, approving, implementing, and revising QAPPs. Detailed information i s available in Handbook on Quality Assurance Project Plans (EPA Handbook H - 05 QAPP , Section 1.7 ) [ EPA 2013 c ] .Handbook QMP 27 December 2012 Agency Review Draft The QMP can also document the process for ensuring that all versions of planning documents, procedures, and related documents will be retained in accordance with EPA policy and procedures for documents and records, and in accordance with extramural agreement regulations. **HELP BOX --**  **OVERVIEW OF SYSTEMATIC PLANNING ELEMENTS** The particulars of the s ys tematic planning approach depend on the focus of a given project , but the QMP should describe the organization’s overall approach to addressing and documenting the following elements: **Organization:** Id entification and involvement o f the project manager, sponsoring organization and responsible official, project personnel, stakeholders, scientific experts, customers, suppliers, and Quality Manager. **Schedule:** Identification of project schedule, resources (including budget), milestones, regulatory

or contractual requirements, and circumstances that could adversely affect the progress of the project. **Project Goal:** Description of the project goal s , objectives, study questions and conceptual model that describe the environmental issue or co ncern, contamination pathway s, and general impact on human health. **Quality Assurance:** Specification of needed quality assurance and quality control activities to ensure performance criteria (for new data) or acceptance criteria (for existing data) are sati sfied. **Data Needs:** Identification of the type of data needed, how the data will be used to support the project’s objectives, and identification of potential limitations in data interpretation. **Criteria:** Determination of the quantity of data needed and spe cification of performance criteria for measuring quality for new data, or acceptance criteria for the use of existing data. **Data Collection:** Description of how and where the data will be obtained (including existing data or data from other sources where th e quality may be unknown or undocumented) and identification of any constraints on data collection. **Analysis:** Description of how data will be analyzed, evaluated, and assessed against its intended use with respect to the established performance or acceptance criteria.Handbook QMP 28 December 2012 Agency Review Draft **3. 11**  **IMPLEMENTATION OF WO RK**  This section relates to Annex A , Implementation of W ork ( Clause A3.11). 3. 11 .1 General Considerations The QMP can describe the general processes designed to ensure that all work involving environmental data collection, production, or use is performed according to approved planning and technical documents (e.g., QAPPs, SOPs) and in the prescribed sequence described in them. The benefits of systemati c planning can be lost without careful project implementation aimed at producing the desired results. Although the details of project implementations are to be found in QAPPs, the QMP can address the organization’s overall approach to documentation of proc edures, sample and data integrity, performance monitoring, and approval of deviations from established processes. 3. 11 .2 Documentation of Implementing Procedures SOPs (or similar tools such as Reference Methods) routinely play a central role in promoting effective and consistent implementation of repetitive data acquisition tasks. An SOP is a document that details the method or procedure for implementing an operation, anal ys is, or action with thoroughl y prescribed techniques and steps . SOPs are ty pically approved as the “official” method for performing certain routine or repetitive tasks. The process for the development and implementation of SOPs along with the roles and responsibilities of managers and staff can be documented in the QMP. S OPs can be written in a format that can be readily understood by the user (i.e., using plain English to the extent possible) , with sufficient detail and clarity to ensure that effective results can be expected and achieved consistently . The process for the documentation of SOPs in the QMP may include (but may not be limited to) :  identification of operations needing procedures;  preparation of procedures, including form, content, and applicability ;  testing and evaluation of procedures (as necessary) to confirm their acceptable performance;  review and approval of documented procedures ; and  assurance that up - to - date SOPs are available and those out - of - date SOPs are revised or withdrawn as needed .  SOPs no longer in use are maintained b y the organization to support documentation of the projects using that version according to program and records management for that organization. Procedures that specify

technical requirements can be reviewed for adequacy and approved before use b y qualified personnel who are independent of the entity developing the procedure. For more information on the development of SOPs, developers of a QMP may find it helpful to examine EPA QA/G - 6, Guidance for Preparing Standard Operating Procedures ( EPA 2007 ) .Handbook QMP 29 December 2012 Agency Review Draft 3. 11 .3 Sample and Data Integrity This section relates to Standard Annex A , Implementation of W ork (Clause A3.1 1

). The general processes and roles and responsibilities of personnel relative to sample and data handling and custody can be described in the QMP. For instance, if the organization will maintain a chain of custody within or across projects, its approach can be explained in the QMP. Specific processes and roles and responsibilities of personnel pertaining to sample and data integrity for individual project s are presented in the applicable QAPPs. 3. 11 . 4 Performance Monitoring This section relates to Standard Annex A , Implementation of W ork ( Clause A3.11). L evel of management oversight and verification for the implementation of work is normally determined based on the importance of the particular project and the intended use of the project results. The QMP can describe how the implementation of work processes will be monitored by the organization , including the routine measurement of performance against established technical and quality

specifications to ensure continued satisfactory performance. It is important that the independence of personnel monitoring the work performance be ensured and that this independence be commensurate with the nature and importance of the activity . The QMP may need to address how to handle situations wherein complete independence of personnel cannot be achieved, such as within a small organization. Developers of the QMP may consider possible safeguards that could be utilized to provide for a practicable level of independence. 3. 11 .5 Deviations from Approved Processes This section relates to Standard Annex A Implementation of W ork (Clause A3.11). The QMP can describe the proc ess for how d eviations from approved processes and procedures will be resolved (if possible), documented , and reported to management. The process and roles and responsibilities of managers and staff for addressing these deviations and any waivers from esta blished policies, procedures, and standards can be described in the QMP . If the deviation necessitates a revised version of a planning document such as a QAPP or SOP, it is important to the reproducibility , defensibility, and utility of results that the current and subsequent versions of the document are indicate d using a document control format as described in Section 3.2.2.Handbook QMP 30 December 2012 Agency Review Draft **3. 1**  **2 ASSESSMENT**  **, OVERSIGHT, AND RES PONSE**  This section relates to Standard Clause: Assessment Oversight, and Response (7. 8 ) , and Annex A Assessment, Oversight, and Response ( Clause A.3.12). 3. 12 .1 Ty

pes of Assessments CI O Standards 2106 - S - 01 and 2106 - S - 02 require organizations to develop and implement processes for assessment and oversight, and for responses to them. Assessments ty picall y have specific criteria or measures of success associated with them. Ingeneral, an assessment is an evaluation process used to measure the performance or effectiveness of a sy stem and its elements. As used here, assessment is an all - inclusive term used to denote activities such as the following:  **D ata**  **Q uality**  **Audit (DQA)** 

– a scientific and statistical evaluation of validated data to determine if the data are of the right ty pe, quality , and quantity

to support their intended use.  **P erformance**  **E valuation (PE)** 

– a typ e of audit in which the quantitative data generated in a measurement sy stem are obtained independently and compared with routinely obtained data to evaluate the proficiency of an analyst or laboratory .  **Q uality**  **S ystem**  **A udit**  **(QSA)**

– a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the quality management sy stem are suitable and have been developed, documented, and effectivel y implemented in accordance with specified requirements. **HELP BOX –**  **MANAGEMENT -**  **ORIENTED SOP TOPICS** Many Standard Operating Procedures relate to the details of data collection, production, and use, and are typicall y addressed in a QAPP. Examples of management - oriented SOPs that may be provided or referenced in a QMP include the following:  Standard Operating Procedure Development  Quality

Assurance Project Plan Development  Quality

Training S ystem  External Agreement Review Process  Management of Quality Records  Quality

S ys tem Assessment Process  Corrective Action Process  Dispute Resolution ProcessHandbook QMP 31 December 2012 Agency Review Draft  **R eadiness**  **R eview** 

– a systematic, documented review of the readiness of the start - up or continued use of a facility , process, or activity . Readiness review s are ty pically conducted before proceeding beyond project milestones and prior to initiation of a major phase of work.  **S urveillance** 

– continual or frequent monitoring and verification of the status of an entity and the anal ys is of records to ensure that specified requirements are being fulfilled.  **T echnical**  **S ystems**  **A udit (TSA)** 

– a thorough, s ystematic, on - site, qualitative audit of a project or operation that may include facilities, equipment, personnel, training, procedures, recordkeeping, data validation, data management, and reporting aspects of a technical sy stem. In the general context of assessment s , the related concept of oversight pertains to monitoring activities or operations to confirm that they conform to approved plans. That is, are those implem enting the work doing what they said they would do? Inthe list above, “surveillance” encompasses aspects of both assessment and oversight. Users of this handbook may find that the term “audit” is used more commonly than “assessment” . 3 . 12 .2 Assessment Planning Assessments offer the assurance that projects and programs are being implemented in accordance with established plans. Inaddition, assessment findings provide the content that lead s from “check” to “act”

– that is, they identify opportunities for preventive actions, corrective actions , and continuous improvement. Accordingl y, when assessments of QMS are planned and scheduled b y the organization regularl y. Their evaluated results can determine the suitability and effectiveness of the implemented QM S and of the quality performance of the environmental programs to which it applies. The QMP can describe the process u sed by m anagement to determine during the planning stage the ty pe(s) of assessment activities appropriate for a particular project , and the assessment tool to be used. The types and frequency of assessments can be selected based on the particular needs of the organization and on the nature of its environmental data operations . S ome projects might benefit from periodic technical system s audits in the field to ensure that samples are collected in accordance with plans, while other projects might be best served b y rigorous data quality audits . A similar approach may

be applied to determining the depth and intensity of an assessment ( i. e., how many

issues will be addressed, how many assessors will be assigned, and how many day s the assessment will last) . The optimum scope and balance of assessments is a program - specific determination. For non - EPA organizations, specific requirements for ass essments may be included in the external agreement. In some cases, assessments may be explicitly

required in the scope of the external agreement such as in field monitoring activities or laboratory confirmation of sampling results .Handbook QMP 32 December 2012 Agency Review Draft Documenting the scope an d objectives of every assessment in an assessment plan aids in the planning process, permits consistency of approach over time, and provides the basis against which to document and evaluate results . A QMP can description of how such plans are generall y rev iewed and approved, and by whom reinforces the use of this valuable preparation tool and assists the assessment team in engaging with the appropriate parties . In this manner, a ssessments (or audits) can be performed according to approved written procedures (e.g., audit plan) , based on careful planning of the scope of the assessment, the information needed and expected from the assessment, and any limits or constraints encountered (e.g., seasonal considerations when collecting water or soil samples) . 3. 12 .3 Assessment Conduct and Follow - Up D escription in the QMP of the organization’s approach for conducting and using the results of assessments and oversight activities , including the roles and responsibilities of managers and staff in this process increases th e probability of these activities being accomplished . The general processes and roles and responsibilities of personnel relative to conduct of assessments, implementation of response actions, and verification of responses can be also described in the QMP . This includes procedures for ensuring the objectivity , independence, and competence of assessment personnel. Specific needs can be defined in individual QAPPs for the activity . The organization can have a s ys tem for selecting assessors who have the appropriate level of competence (i.e., capability , experience, and training) to meet the needs of the organization’s assessment program. Again, the specific competence needed for a particular assessment can be addressed in the QAPP, but the QMP can address the mo re general, overarching expectations. Assessors (or auditors) can also have demonstrated independence from the work being assessed and can have no real or perceived conflict of interest. Again, specific processes and roles and responsibilities of personnel pertaining to assessment of individual projects can be described in the applicable QAPPs , as discussed in CI O Handbook 2106 - H - 05 QAPP, Sections 2.4, 3.4, and 4.4. Assessment results can be documented, reported to, and reviewed by management. This process can be described in the QMP along with the roles and responsibilities of involved managers and staff. The QMP can describe the process for identifying, conducting, and tracking corrective actions in response to assessments. The organization can ensure that corrective actions are made on a timely

basis and that the effectiveness of their implementation is confirmed and documented. The OEI Quality

Staff typically conducts an assessment (i.e., Quality S ys tem Audit) of an EPA organization’s Quality

Management S ys tem at least every five ye ars, but may

do so more frequentl y if needed. Assessments of a non - EPA organization’s Quality Management System are normally conducted as provided for in the external agreement, or at least every five ye ars if the agreement is for an extended period. It is recommended that external organizations conduct selfassessments in accordance with a schedule defined in their QMP and agreed to by EPA.Handbook QMP 33 December 2012 Agency Review Draft For more information on planning, conducting, and reporting audits, EPA QA/G - 3, Guidance on Assessing Quality Systems [EPA 2003] and I SO 19011, Guidelines for Quality and/or Enviro nmental Management Systems Auditing [I SO 2002] m ay be helpful . Other guidance on the planning, implementation, and reporting of audits and assessments that a user may find useful in developing a QMP is available in the following documents:  Guidance on Technical Audits and Related Assessments for Environmental Data O perations (EPA QA/G - 7) [EPA 2000]  Data Quality Assessment: A Reviewer’s Guide (EPA QA/G - 9R) [EPA 2006a]  Data Quality Assessment: Statistical Tools for Practitioners (EPA QA/G - 9S) [EPA 2006b] **HELP BOX –**  **CORRECTIVE ACTION PROCESS** The QMP should describe the organization’s overall approach for designing, implementing, and tracking corrective actions taken in response to audit findings. The corrective action process should be the responsibility of the organizational unit to which the finding applies, and sh ould be sufficiently formal and transparent that management can confirm that the finding has been resolved. The following outline describes one formal approach that may be used to plan and conduct a corrective action process. **Step 1:** Brainstorm to identify the possible causes of the finding. **Step 2:** Agree on likely

root causes. (A number of standard quality tools are available to help identify root causes of a problem. ) **Step 3:** Develop a list of potential solutions. **Step 4:** Decide what so lution to evaluate more fully . **Step 5:** Develop a plan for implementing the solution and tracking its effectiveness. Develop a plan that breaks down the corrective action into discrete steps with an associated schedule. **Step 6:** Implement and monitor the cor rective action based on the corrective action plan. **Step 7:** Develop a process for evaluating the effectiveness of the corrective action once it has been implemented. **Step 8 :**  Determine the effectiveness of the implemented actions. Consider need to improve the solution or rethink root causes and start again.Handbook QMP 34 December 2012 Agency Review Draft **3.13 QUALITY IMPROVE MENT**  T o facilitate continuous improvement within the organizations Quality Sy stem describe in the QMP :  the organization's process for ensuring continual quality improvement, including the management process for determining, planning, implementing, and evaluating the effectiveness of quality

improvement activities including: o who (organizationally) is responsibl e for quality i mprovement; and o the corrective action program to ensure that conditions adverse to quality are  identified promptly , and  corrected as soon as practical.  how the organization shall detect and prevent quality problems, and  how staff at all lev els are encouraged to identify and establish communications among customers and suppliers, identify process improvement opportunities, identify problems, and offer solutions to those problems. In order to enhance the effectiveness of c orrective actions to be effective, it is strongl y recommended that they include:  identification of root causes of problems,  determination of whether the problem is unique or has more generic implications, and a recommendation of procedures to prevent recurrence. **3.1 4**  **DATA REVIEW, VALIDATION A** **ND VERIFICATION, AND DATA**  **U SABILITY**  **REPORTING** This section relates to Standard Clause: 7.9 Data Review, Validation and Verification, and Data Usability Reporting

, and Annex A Assessment, Oversight, and Response (Clause A.3.1 2) and Quali ty Improvement (Clause A.3.13 ). 3.14.1 Planning for Data Review The use of any environmental data for decisions follow s the review of such data to determine the extent to which the technical and quality ob jectives for the data were met. Both Quality Standards require that organizations develop and implement appropriate review processes to confirm the suitability

and adequacy

of data for their intended use. The primary purpose of a QMS for environmental data collection, production, and use is to ensure that the resulting data have the necessary

integrity , are defensible, and have adequate documentation to allow for other possible uses. The criteria for the adequacy of such data will largely

be specific to the project and can be addressed b y the QAPP; ho wever, it is appropriate for the QMP to document the general approaches and array

of evaluation tools and to describe the general roles and responsibilities of managers and staff in their use. The QMP can describe the possible or routine processes b y which r esults obtained from environmental data collection or production are reviewed, verified, validated, and qualified prior to their intended use.Handbook QMP 35 December 2012 Agency Review Draft The QMP should also commit to documenting any necessary deviation from routine processes and/or the development or use of alternative processes . Where possible, this discussion include s identification of the roles and responsibilities of management and staff relative to the se functions . S pecific processes and the roles and responsibilities of personnel pertaining to the evaluation of individual projects can be described in the applicable QAPPs. The Quality

Standards require that any project reports that contain data, or report the results of environmental data operations (e.g., data summaries) are to be reviewed independentl y (i.e., b y others than those who produced the data or the reports) to confirm that the data or results are presented correctl y . The QMP can describe the general process for the review and approval of these reports and can document the roles of managers and staff needing to participate . The Standards require that any limitations or restrictions on the use of the data shall be expressed quantitatively

to the extent practicable and shall be documented in any reporting of the data . When adequate do cumentation of the data accompanies its reporting it will reduce, but not completely

eliminate, the risk of the data being misinterpreted or misused for decisions. 3.14.2 Special Considerations for Data Review If environmental data are collected from sourc es that did not use a QMS or whose quality is unknown or undocumented, the organization can review and evaluate these data for their suitability for use. Examples of such sources may include:  academic research and journal articles;  previous studies of cont amination at a particular waste site; and  reports from volunteer groups such as a watershed association. In addition, when data collected for purposes other than that of the current project are to be used, the QMP can describe the general process to be uti lized for determining the suitability of the data for the new use. Details pertaining to a specific project can be described in the applicable QAPP. Key elements of a review of environmental data for potential use include:  the need to satisfy

the major requirements of the Information Quality Guidelines (IQG) , e.g., objectivity , utility , and integrity – see EPA/260R - 02 - 008, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency [EPA 2002 a] ;  the principal indicators of environmental quality, e.g., precision, bias, representativeness, comparability , completeness, and sensitivity

– see Appendix B of EPA CI O Handbook 2106 - H - 05 QAPP ; and  the general usability

factors, e.g., relevance, temporal compatibility , spatial inclusivity, credibility , aggregation level, compatibility , reproducibility, and degree of augmentation

– see Sections 2 .5, 3 .5, and 4 .5 of EPA CI O Handbook 2106 - H - 05 QAPP . The QM P may

n ote that s ome use s of environmental data may not require a QAPP , but the data can still be appropriately

vetted for their use. For example, the use of data for public information products may

not need the rigor of a QAPP , but would at least need som e review to confirm their adequacy for use.Handbook QMP 36 December 2012 Agency Review Draft In contrast, a technical decision or report from a large field sampling activity would be expected to have complete and comprehensive quality documentation to support the results. If possible, the QMP can specify the general circumstances under which a QAPP for such products will or will not be needed. (See CI O Handbook 2106 - H - 05 QAPP for details.) 3.1 4.3 Related Ty

pes of Reviews Contentious or significant activities may be subjected to review during the project proposal and/or planning and design stage. In some instances, it is also appropriate to conduct various reviews over the lifecycle of a project. If the organization applies a peer review process to information products , the QMP can describe the general process to be used, including selecting peer reviewers, interacting with them , and using their review comments. For EPA organizations, the current edition of the EPA Peer Review Handbook [EPA 200 6 c ] provides further information on this topic. Non - EPA organizations may have their own peer review process which can be documented in the QMP as applicable . To maximize its value, peer review can begin as early in the planning phase as possible. For EPA organizations, disseminated information products, including those that contain environmental data or conclusions derived from environmental data , may be subject to the EPA IQ G [EPA 2002a] requirements . As part of compliance with the EPA IQG, organizations can incorporate the principles of objectivity , utility and integrity into their existing review procedures, as appropriate. For additional information on project - level data review and evaluation of usability, users can refer to Handbook on Quality Assurance Project Plans (EPA Handbook H - 05 QAPP , Sections 2.5, 3.5, and 4.5 ) [ EPA 2013 c ] .Handbook QMP 37 December 2012 Agency Review Draft **REFERENCES ANSI**  **2004** American National Standards Institute and American Society for Quality, Quali ty Systems for Environmental Data and Technology Programs

– Requirements with Guidance for Use (ANSI/ASQ E4 - 2004 . http://asq.org/quality-press/display **EPA 1984** U.S. Environmental Protection Agency , EPA Order 5700 Change 6, Assistanc e Administration Manual. http://www.epa.gov/oarm **EPA 2000**  U.S. Environmental Protection Agency , Guidance on Technical Audits and Related Assessments for Environmental Data Operations (EPA QA/G - 7) . http://www.epa.gov/quality /qs - docs/g7 - final.pdf **E PA 2002**  **a** U.S. Environmental Protection Agency , Guidelines for Ensuring and Maximizing the Quality, Objectivity

, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency. http://www.epa.gov/quality /informationguidelines/documents/EPA_InfoQualityGuidelines.pdf **EPA 2002b** U.S. Environmental Protection Agency , Guidance for Developing Qu ality Systems for Environmental Programs (EPA QA/G - 1) http://www.epa.gov/quality **EPA 2003**  U.S. Environmental Protection Agency , Guidance on Assessing Quality Systems (EPA QA/G - 3) . http://www.epa.gov/quality /qs - docs/g3 - final.pdf **EPA 2005**  U.S. Environmental Protection Agency , Uniform Federal Policy for Implementing Environmental Quality Systems (EPA - 505 - F - 03 - 00 1) . http://www.epa.gov/fedfac/pdf/ufp_v2_final.pdf **EPA 2006a**  U.S. Environmental Protection Agency , Data Quality Assessment: A Reviewer’s Guide (EPA QA/G - 9R) . http://www.epa.gov/quality /qs - docs/g9r - final.pdf **EPA 2006b**  U.S. Environmental Protection Agency , Data Quality Assessment: Statistical Tools for Practitioners (EPA QA/G - 9S) . http://www.epa.gov/quality /qs - docs/g9s - final.pdf **EPA 2006c** U.S. Environmental Protection Agency, Peer Review Handbook 3 rd Edition. http://www.epa.gov/peerreview/pdfs/Peer%20Review%20HandbookMay06.pdf **EPA 2007**  U .S. Environmental Protection Agency , EPA QA/G - 6, Guidance for Preparing Standard Operating Procedures . http://www.epa.gov/quality /qs - docs/g6 - final.pdfHandbook QMP 38 December 2012 Agency Review Draft **EPA 2008a**  U.S. Environmental Protection Agency , EPA Order 1610, Interagency Agreement Policies, Procedures, and Guidance Manual. http://www.epa.gov/oarm **EPA 2008 b**  U.S. Environmental Protection Agency , Procedure for Quality Policy ( CI O Procedure 2106 - P - 01 ) . http://www.epa.gov/irmpoli8/policies/2106p01.pdf **EPA 2008**  **c** U.S. Environmental Protection Agency , Quality Policy , (CIO Policy 2106) . http://www.epa.gov/irmpoli8/policies/21060.pdf **EPA 2009a** U. S . Environmental Protection Agency , Contracts Management Manual. http://www.epa.gov/oarm **EPA 2009**  **b** U.S. Environmental Protection Agency , EPA Records Management Policy . (CIO Policy

2155.1). http://www .epa.gov/records/policy /2155/rm_policy

_cio_2155_1_2.pdf **EPA 2013**  **a** U.S. Environmental Protection Agency , Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations . ( CI O Standard 2106 - S - 01) . http://www.epa.gov/quality **EPA 2013**  **b** U.S. Environmental Protection Agency , Quality Standard for Environmental Data Collection, Production, and Use by Non -EPA (External) Organizations . ( CI O Standard 2106 - S - 02) . http://www.epa.gov/quality **EPA 2013**  **c** U.S. Environmental Protection Agency, Handbook on Quality Assurance Project Plans . ( CI O Handbook 2106 - H - 05 QAPP ). http://www.epa.gov/quality **ISO 2002**  International Organization for Standardization 19011, Guidelines for Quality and/or E nvironmental M

anagement S

ystems A uditing. http://www.iso.org **ISO 2005**  International Organization f or Standardization 17 025, General R equirements for the C ompetence of T

esting and C

alibration L aboratories . http://www.iso.org **ISO 2008**  International Organization for Standardization 9001, Quality M anagement S

ystems

– Requirements . http://www.iso.orgHandbook QMP 39 December 2012 Agency Review Draft **APPENDIX A TERMS AND DEFINITIONS**  **NOTE:** The following terms and definitions are taken from Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations ( C IO Standard 2106 - S - 01 .0 ) as well as Quality Standard for Environmental Data Collection, Production, and Use by Non -EPA (E xternal) Organizations (CIO Standard 2106 - S - 02) . **Assessment:** The evaluation process used to measure the performance or effectiveness of a sy stem and its elements. As used here, assessment is an all - inclusive term used to denote any of the following: audit, performance evaluation, management review, peer review, inspection, or surveillance. **Competence:** The ability to apply knowledge, skills, and experience to an activity to achieve intended results. **Conformity:** The fulfillment of a requirement. **Corrective Action:** An action to eliminate the cause of a nonconformity o r undesirable situation and to prevent recurrence. **Data:** A collection of facts and estimates from which conclusions may be drawn. **Data Usability:** The process of determining and ensuring that the quality of the data produced meets the intended use of the data. **Dissemination:** Process of distributing information to the public that represents an official EPA endorsed opinion or decision. [Examples of information not considered a dissemination are information intended only for government employees; EPA respons es to requests for Agency records under the Freedom of Information Act (FOIA), the Privacy Act, The Federal Advisory Committee Act (FACA) or other similar laws; correspondence directed to individuals or persons; ephemeral information; and distribution of information in documents filed in or prepared specificall y for a judicial case or an administrative adjudication.] (Source: Section 5.3 & 5.4, EPA Information Quality Guidelines) **Document:** Recorded information regardless of phy sical form or characteristics including individual records or items of non - record materials. **Environmental Data** : Any data or information pertaining to the environment that describe measured outputs from processes, environmental conditions in a specific location; ecological effects and consequences; health effects and consequences; biological, chemical, and radiological conditions; or the performance of environmental technology . For EPA, environmental data include information collected directly from measurements, produced from models, an d compiled from other sources such as databases, information system s, literature, or the Internet.Handbook QMP 40 December 2012 Agency Review Draft **Environmental Data Collection:** Process of acquiring or gathering environmental data through various means including (but not limited to) sampling and anal ys is activities, retrieval from information sy stems and the literature, and receipt from EPA partners and the regulated community . **Environmental Data Operations:** Work performed to collect, produce, use , or report environmental data. **Environmental Data Production:**  Process of generating environmental data through various means including (but not limited to) the use of measurement instrumentation, information technology , computer models, and data anal ys is tools (e.g., statistics, risk assessment methods). **Environmental Programs:** Activities involving the environment, including but not limited to: characterization of environmental processes and conditions; environmental monitoring; environmental research and development; the design, construction, and operatio n of environmental technologies; and laboratory operations on environmental samples. **Environmental Technology:** An all - inclusive term used to describe pollution monitoring, measurement and control devices and s ys tems, waste treatment processes and storage f acilities, and site remediation technologies and their components that may be utilized to remove pollutants or contaminants from or prevent them from entering the environment. **Extramural Agreement:** Legal agreement between EPA and a non - EPA organization fo r the acquisition of items or services b y EPA or financial assistance to a non - EPA organization. Such agreements include acquisition agreements (e.g., contracts, work assignments, delivery orders, task orders), assistance agreements (e.g., cooperative agre ements, research grants, state and local grants), and EPA - funded Interagency Agreements (IAs) with other governmental entities. **Graded Approach:** The process of basing the level of application of managerial controls applied to an item or work according to the intended use of the results and the degree of confidence needed in the quality of the results. **Guidance:** A non - mandatory compilation of advice, examples, best practices, or past experience. Guidance may

supplement procedures. **Handbook:** A non - mandatory c ompilation of advice, examples, best practices, or past experiences, may

be revised according to the issuing Office’s Peer Review Policy . **Information:** For purposes of this Standard, information means any communication or representation of knowledge such as facts or data, in any medium or form, including, but not limited to, textual, numerical, graphic, cartographic, narrative, or audiovisual forms (OMB In formation Quality Guidelines) . **Information Product** : Any book, paper, map, machine - readable material, audiovisual production, or other documentary material, regardless of phys ical form or characteristic (OMB Circular A - 130) .Handbook QMP 41 December 2012 Agency Review Draft **Information Quality Guidelines (IQG):** An Agency document that defines a basic standard of quality (including objectivity , utility , and integrity) for information products disseminated by EPA. For influential information products, the basic standard of quality also includes reproducibility

and transparency . **Information System:** An organized collection, storage, and presentation sy stem of d ata for decision making, progress reporting, and for planning and evaluation of programs. It can be either manual or computerized, or a combination of both. **Information Technology:** The study, design, development, implementation, support, or management of computer - based information sy stems, particularl y software applications and computer hardware. **Informative Annex:** A part of a Standard that gives additional information as guidance which is intended to assist the understanding or use of the Standard. An inf ormative annex is not considered to be part of the Standard when setting audit criteria. **Integrity (information):** Assurance that the information is protected from unauthorized access or change and i s not compromised through corruption or falsification. **Man agement Controls:**  A sy stem of management functions to enable managers to (1) determine that the operations of a program or organization satisfy predetermined goals and objectives, and that performance is in line with standards and specifications; and (2) implement any remedial actions needed to ensure that human and other resources are being used in the most effective and efficient way

possible in achieving the organization’s mission. **Management System** : A sy stem to establish policy and objectives and to ach ieve those objectives (ISO 9001). A management s ys tem may describe the policies, objectives, principles, authority , responsibilities, accountability, and implementation plan of an organization for conducting work and producing environmental data . Management sy stems include I SO 9001 on quality management, ISO 14001 on environmental management, and OHSAS 18000 on occupational health and safety. **Normative Annex:** A part of a Standard that gives provisions (requirements) additional to those requirements in the b ody

of the Standard. A normative annex is considered to a part of the Standard when setting audit criteria. **Organization:** Company, corporation, firm, enterprise, or institution, or part thereof, whether incorporated or not, public or private, that has its own functions and administration. Inthe context of this Standard, an EPA organization may be an Office, Region, National Research Center or Laboratory , or a sub - unit such as a division, branch, section, or team. **Policy:** A high - level statement about an Agency requirement designed to influence and determine decisions, actions, and other matters. It is usually driven b y statute, executive order, the mandate of an oversight agency or Congress, or the head of the organization. **Prime Recipient:** Organization that directly receives funding, support, or authorization to implement a project or program.Handbook QMP 42 December 2012 Agency Review Draft **Procedure (CIO Procedure)**  **:** The required steps, course of action, or processes needed to accomplish or satisfy

a policy . **Process:** A set of interrelated resources and activities which transforms inputs into outputs. Examples of processes include anal ys is, design, data collection, operation, fabrication, and calculation. **Product:** The intended result or final output of an activity or process that is disseminated or distributed among EPA organizations or outside of EPA. **Programmatic QMP:** a management plan that defines the structure and framework for planning, implementing, and evaluating quality for a specific policy , or program. **Quality:** The totality

of features and characteristics of a product or service that bear on its ability to meet the stated or implied needs and expectations of the user. **Quality Assurance (QA):** A management or oversight function that deals with setting policy and running an administrative system of management controls that cover planning, implementation, review and maintenance to ensure environmental data are meeting their intended use. **Quality Assurance Manager ( QA Manager** **):** The individual designated as the principal manager within the organization having management oversight and responsibilities for planning, documenting, coordinating, and assessing the effectiveness of the QMS for the organization. NOTE: Other personnel having QA or QC duties may be referred to as QA Officer and QA Coordinator . **Quality Assurance Project Plan (QAPP):** A document describing in comprehensive detail the necessary QA, QC and other technical activities that must be implemented to ensure that the results of the work performed will satisfy the stated performance objectives and criteria. **Quality Control (QC):** The overall sy stem of technical activities that measure the attributes and performance of a process, item, or service against defined standards to verify that they meet the stated requirements established by the customer; operational techniques and activities that are used to fulfill requirements for quality . **Quality Management:** That aspect of the overall quality management s ys tem of the organization that determines and implements the quality policy . Quality management typicall y includes strategic planning, allocation of resources, and other s ys tematic activities (e.g., planning, implementation, and assessment) pertaining to the application of quality practices to the organization’s programs. **Quality Management Plan (Q MP):**  A formal document or manual that describes the QMS in terms of the organizational structure, functional responsibilities of management and staff, lines of authority , and required interfaces for those planning, implementing and assessing all activities conducted.Handbook QMP 43 December 2012 Agency Review Draft **Quality Management System (QMS):** A structured and documented management s ys tem describing the policies, objectives, principles, organizational authority , responsibilities, accountability , and implementation plan of an organization for ensuring quality in its work processes, products (items), and services. The quality management sy stem provides the framework for planning, implementing, documenting, and assessing work performed b y the organization and for carrying out required QA and QC activitie s. **Quality Program** : The totality of management controls, processes, and documentation in EPA’s planning, implementation, and assessment of applying quality to the creation of Agency environmental data . **Record:** A document stating results retrieved or providing evidence of activities (I SO 9000:2005). NOTE: A federal record is an information resource in any format that is needed to describe Agency

activities (44 U.S.C. § 3301). **Requirement:** An expression of the content of a Standard that conveys criteria to be fulfilled is compliance is to be claimed and from which no deviation is permitted. **Service:** A discrete function that performs one or more operations and returns a set of results to an external requester. **Standard:** An accepted, consensus - based specif ication which defines s ys tems, processes, methodologies, or practices. Itprovides a basis for assuring consistent and acceptable minimum levels quality , performance, safety, and reliability. Standards usually are included in or accompany procedures. **Stand ard Operating Procedure (SOP):**  A written document that details the method for an operation, anal ys is, or action with thoroughly prescribed techniques and steps, and that is officiall y approved as the method for performing certain routine or repetitive task s. **Transparency (Information):** Assurance that information is supported by sufficient metadata when disseminated. **Usability Assessment:** Evaluation of data based upon the results of data validation and verification for the decision(s) being made. Reviewers a ssess whether the process execution and resulting data meet quality objectives based on the criteria given in the QAPP. **User:** An organization, group, or individual that utilizes the results or products from environmental programs or the customer for whom the results or products were collected or created. **Utility (Information):** Assurance that information is useful for its intended purpose. **Validation (Information):** Confirmation by examination and provision of objective evidence that the particular requirement for which the information is intended are fulfilled; the process of determining whether the specifications were appropriate and that the verified results will meet the data user’s needs.Handbook QMP 44 December 2012 Agency Review Draft **Verification (Information):** Confirmation by examination and provisi on of objective evidence that validated information fulfills specified requirements; the process of checking whether the information met the project’s specifications.