Handbook for Preparing

Quality Management Plans

Agency Review Draft

December, 2012

This document is a review draft. This information is distributed solely for the purpose of predissemination review under applicable information quality guidelines. It has not been formally 

disseminated by EPA. It does not represent and should not be construed to represent any Agency

determination or policy. It is being circulated for review of its technical accuracy.
Handbook QMP ii December 2012 Agency Review Draft

FOREWORD

The U.S. Environmental Protection Agency (EPA) uses Quality Management Plans (QMPs) to 

document how an organization will plan, implement, and assess the effectiveness of its Quality

Management System (QMS) applied to environmental data collection, production, and use. The 

requirements and specifications are listed in Annex A of Quality Standard For Environmental 

Data Collection, Production, and Use By EPA Organizations (EPA CIO 2106-S-01) and Quality 

Standard For Environmental Data Collection, Production, and Use By Non-EPA (External) 

Organizations (EPA CIO 2106-S-02.0) (current versions).

This document:

 provides technical and policy advice;

 provides implementation information;

 does not substitute for statutes EPA administers or their implementing regulations;

 is not a regulation;

 does not impose legally-binding requirements on EPA, states, or the regulated 

community;

 is not mandatory.

EPA may:

 apply this handbook to any particular program or project to the extent appropriate in light 

of site-specific facts;

 not apply this handbook to a particular situation based upon the specific circumstances;

and

 update, improve, revise, replace or withdraw this handbook at any time.

This document is one of several guidance and handbook documents from the U.S. Environmental 

Protection Agency, Office of Environmental Information (OEI) Quality Staff. EPA Quality

Program documents may be downloaded from the Quality Home Page:

http://www.epa.gov/quality/qa_docs.html

In many instances, regional or program quality assurance personnel and project management 

personnel will be engaged for specific implementation information. These personnel are listed 

here:

http://www.epa.gov/quality/contacts.html

Questions regarding this document or other documents can be directed to the OEI Quality Staff:

U.S. EPA

OEI/Quality Staff (2811R) 

Washington, DC 20460

Phone: (202) 564-6830

FAX: (202) 565-2441

e-mail: quality@epa.gov
Handbook QMP iii December 2012 Agency Review Draft

TABLE OF CONTENTS

CHAPTER 1 INTRODUCTION................................................................................................ 1

1.1 Background and Purpose .........................................................................................1

1.2 Quality Management Plans in the EPA Quality Management System....................1

1.3 Intended Audience ...................................................................................................4

1.4 Period of Applicability.............................................................................................4

1.5 Supersession.............................................................................................................4

CHAPTER 2 QUALITY MANAGEMENT PLAN PREPARATION, SUBMISSION, 

REVIEW, AND APPROVAL.......................................................................................... 5

2.1 Quality Management Plan Preparation ....................................................................5

2.2 Quality Management Plan Submissionand management Approval.........................5

2.3 Quality Management Plan Review and approval for use.........................................6

2.4 Quality Management Plan Revisions.......................................................................6

2.5 Combined Quality Management Plan with Quality Assurance Project Plan...........8

CHAPTER 3 HANDBOOK ON QUALITY MANAGEMENT PLAN CONTENT............ 10

3.1 General Guidelines.................................................................................................10

3.1.1 Overview of QMP Content ....................................................................... 11

3.1.2 Administrative Elements........................................................................... 11

3.1.3 Application of Graded Approach in QMPs .............................................. 12

3.2 Quality Management Plan Conformance to Requirements ...................................13

3.3 Organizational and Program Field of Application.................................................14

3.4 Organization, Responsibility, and Authority .........................................................15

3.4.1 Organizational Issues................................................................................ 15

3.4.2 Roles and Responsibilities........................................................................ 16

3.4.3 Dispute Resolution.................................................................................... 17

3.5 Quality Management System Description .............................................................17

3.6 Personnel Competence...........................................................................................19

3.7 External Agreements..............................................................................................20

3.7.1 EPA Organizations.................................................................................... 20

3.7.2 Non-EPA Organizations ........................................................................... 21

3.8 Documents and Records Management...................................................................22

3.9 Use of Information Technology Methods and Sources .........................................23

3.10 Planning .................................................................................................................25

3.10.1 The QMP as the Foundation of QMS Planning........................................ 25

3.10.2 Commitment to Project-Specific Planning ............................................... 25

3.11 Implementation of Work........................................................................................28

3.11.1 General Considerations............................................................................. 28

3.11.2 Documentation of Implementing Procedures ........................................... 28

3.11.3 Sample and Data Integrity ........................................................................ 29

3.11.4 Performance Monitoring........................................................................... 29

3.11.5 Deviations from Approved Processes....................................................... 29

3.12 Assessment, Oversight, and Response...................................................................30

3.12.1 Types of Assessments............................................................................... 30

3.12.2 Assessment Planning ................................................................................ 31

3.12.3 Assessment Conduct and Follow-Up........................................................ 32
Handbook QMP iv December 2012 Agency Review Draft

3.13 Quality Improvement.................................................................................................34

3.14 Data Review, Validation and Verification, and Data Usability Reporting............34

3.14.1 Planning for Data Review......................................................................... 34

3.14.2 Special Considerations for Data Review .................................................. 35

3.14.3 Related Types of Reviews ........................................................................ 36

REFERENCES............................................................................................................................ 37

APPENDIX A TERMS AND DEFINITIONS......................................................................... 39
Handbook QMP v December 2012 Agency Review Draft

LIST OF ACRONYMS

ANSI American National Standards Institute

ASQ American Society for Quality

CIO Chief Information Officer

DQA Data Quality Audit

DQA Director of Quality Assurance

EAS EPA Acquisition System

GIS Geographic Information System

IA Inter-agency Agreement

IGMS Integrated Grants Management System

IQG Information Quality Guidelines

ISO International Organization for Standardization (not an acronym)

IT Information Technology

OEI Office of Environmental Information

PDR Pre-Dissemination Review

PE Performance Evaluation

QA Quality Assurance

QA Manager Quality Assurance Manager

QAPP Quality Assurance Project Plan

QARF Quality Assurance Review Form

QC Quality Control

QMP Quality Management Plan

QMS Quality Management System

QSA Quality System Audit

SIO Senior Information Official

SOP Standard Operating Procedure

TSA Technical Systems Audit

UFP Uniform Federal Policy
Handbook QMP 1 December 2012 Agency Review Draft

CHAPTER 1

INTRODUCTION

1.1 BACKGROUND AND PURPOSE

This section relates to Standard Clauses 1 Purpose, 2 Scope and Applicability, and Annex A, 

Introduction (Clause A1).

This handbook assists users with creation of Quality Management Plans (QMPs) or equivalent 

documents to satisfy the requirements of the EPA Quality Program as defined by the following 

two EPA Standards:

 Quality Standard for Environmental Data Collection, Production, and Use by EPA 

Organizations, (CIO 2106-S-01), “Internal Standard” (EPA 2013a); and

 Quality Standard for Environmental Data Collection, Production, and Use by Non-EPA 

(External) Organizations, (CIO 2106-S-02), “External Standard” (EPA 2013b).

This handbook is not a standalone document, but intended to be used in conjunction with the 

Standards for the preparation of QMPs that satisfy EPA requirements. It is offered as a helpful 

resource for those who need to prepare QMPs that comply with these EPA Standards.

It can be used by both EPA organizations and by organizations conducting activities through 

external agreements with EPA. This handbook is intended to help its users prepare QMPs that 

summarize the overall management structure in which individual environmental data operations 

will take place and describes in general terms the quality tools that the organization will apply to 

individual projects and who is responsible for implementing them.

By definition, handbook is, not mandatory, and application is at the discretion of the user. This 

handbook offers guidelines, advice, and examples that will help users to develop QMPs that 

satisfy the requirements of the Standards.

The QA/R-2 document, EPA QA/R-2, Requirements for Quality Management Plans, has been 

withdrawn except for limited use under special circumstances. The QA/R-2 document is no 

longer used for developing QMPs except where permitted by the Standards.

1.2 QUALITY MANAGEMENT PLANS IN THE EPA QUALITY MANAGEMENT 

SYSTEM

This section relates to Standard Clause 7.2 Quality Management Plan, 7.5.2 Combined QMPQAPP and Annex A Introduction (Clause A1).

The Standards identified in the previous section require EPA organizations (CIO Standard 

2106-S-01) as well as non-EPA organizations operating under an external agreement with EPA 

(CIO Standard 2106-S-02) to develop, document, and implement a Quality Management System 

(QMS) that conforms to applicable EPA policies and procedures. A QMS is a structured and 

documented management system describing the policies, objectives, principles, organizational

authority, responsibilities, accountability, and implementation plan of an organization for 

ensuring quality in activities involving environmental data.
Handbook QMP 2 December 2012 Agency Review Draft

The QMS provides the framework for planning, implementing, documenting, and assessing work 

performed by the organization and for carrying out quality assurance (QA) and quality control 

(QC) activities for environmental data operations and environmental technology performance 

activities.

A key step in the development and documentation of an organization’s QMS is the creation of a 

Quality Management Plan (QMP). The QMP is unique to an organization and describes the 

management controls and technical processes and practices necessary to plan, implement, 

document, and assess the effectiveness of the organization’s QMS. In general, a QMP provides a 

road map for carrying out the QMS, including its scope, the details and justification for any 

exclusion, a description of the QMS processes, and roles and responsibilities for QMS 

implementation. The QMP when sufficiently clear and thorough can serve as the basis for a 

Quality System Audit. That is, the QMP documents what quality activities are performed, what 

management controls are applied to the planning, implementation, and assessment of such 

activities, and who is responsible for the implementation and its management oversight. The 

QMP also documents the relationships among all parts of the organization for the application of 

quality to the organization’s internal and external work. Such documentation enables the user to 

understand who does what and when.

Organizations (or a group of organizations) may also choose to develop and implement QMPs 

for large, complex programs involving environmental data collection, production, and use. These 

QMPs are typically referred to as “Programmatic QMPs” and the same criteria as for 

organization QMPs applies to them. The preparation and approval of “Programmatic QMPs” is 

discussed in Chapter 2.

The Standards state that QMPs may be prepared by the following organizations:

 All EPA organizations involved with the production, collection, or use of environmental 

data, including Headquarters Offices, Program Offices, Regions, and National Research 

Laboratories and Centers. Their sub-organizations may have QMPs as well to document 

the details of the QMS applied to that sub-organization and its implementation.

 All non-EPA organizations performing authorized activities that produce, collect, or use 

environmental data on behalf of EPA. These may include (but not be limited to):

o States,

o Tribes,

o local governments,

o regulated parties,

o entities party to EPA settlement or consent agreements resulting from 

enforcement actions,

o volunteer organizations,

o contractors,

o cooperative agreement holders,

o grantees,

o other federal governmental departments and agencies,

o non-governmental organizations,

o international governments/organizations, and

o educational institutions.
Handbook QMP 3 December 2012 Agency Review Draft

Some organizations, including some EPA Program Offices and Regions, may elect to require 

their sub-organizations (e.g., subordinate offices, divisions within a Region) to develop 

individual QMPs. In some cases, an organization may develop additional QMPs at a suborganizational unit (e.g., Divisions within a Region, Laboratories within an Office) in order to 

provide more implementation details. The lower-level QMPs are tiered to the organization-wide 

QMP and do not need to repeat the details contained in the latter document. Sub-organizational 

QMPs can be beneficial because they provide the flexibility to document any unique 

circumstances within the sub-organization that may have bearing on the QMS (e.g., the functions 

of a Laboratory Information Management System).

The Standards also require that these organizations prepare Quality Assurance Project Plans 

(QAPPs) or equivalent documents as appropriate, in order to describe and document explicit 

project details. QAPPs are an output of project planning and provide a road map to project 

implementation. They are distinct from QMPs in that they focus on particular data-related 

projects rather than on the organizational level; they are a project-specific view of QMS 

implementation. QAPPs describe in comprehensive detail the QA, QC and other technical 

activities that can be implemented to ensure that the results of the work performed will satisfy

the stated performance objectives and criteria. They describe the activities of a project in the 

acquisition of environmental data or information from direct measurement activities, existing 

data, or generated by models.

The QMP can define and describe the overall structure and framework for planning, 

implementing, and evaluating environmental data operations, which in turn are described and 

documented by QAPPs for individual projects or tasks.

Figure 1 depicts the relationship between QMPs and QAPPs. More information on the 

preparation of QAPPs is available in Handbook on Quality Assurance Project Plans (EPA 

Handbook H-05 QAPP) [EPA 2013c]. As discussed in more detail in the QAPP guidance, 

QAPPs include those for individual projects or tasks, general or generic QAPPs (which may also 

be called QA Program Plans) that may be applied to multiple activities, and may include 

combined QMP-QAPPs for some specific activities that address elements of both the QMP and 

the QAPP.

Figure 1. Functional Relationship of QMP and QAPPs to the QMS for an Organization
Handbook QMP 4 December 2012 Agency Review Draft

1.3 INTENDED AUDIENCE

This section relates to Standard Clause 2.1, Scope, and Annex A, Quality Management Plan 

Requirements (Clause A3).

This document is intended primarily for those responsible for writing, reviewing, approving, or 

using QMPs in accordance with EPA Standards. It is also useful for anyone who wants a detailed 

understanding of the role of QMPs in EPA’s QMS, or more generally for those seeking an 

example of how a Quality Management System can conform to national or international 

consensus standards.

1.4 PERIOD OF APPLICABILITY

This section relates to Standard Clause 7.2 Quality Management plan, and Annex A Agency

Review and Approval of the QMP (Clause A2.3).

This handbook is valid for five years from publication. Then it will be subject to review, 

resulting in one of the following:

 revision,

 reaffirmation without change, or

 withdrawal.

If circumstances warrant, this handbook may also be revised during the course of the five-year 

period.

EPA QMPs developed in accordance with CIO Standard 2106-S-01 will typically have a valid

approval for a period not to exceed five years (as defined in the Standard), though they can be 

revised at any time to reflect mission or scope change, reorganization, the emergence of new 

tools, or any other pertinent changes to the QMS.

QMPs prepared by external parties in accordance with CIO Standard 2106-S-02 may have an

EPA approval valid for no more than five years for State, local, and Tribal governments or the 

length of the extramural agreement for all other extramural agreement holders. The period for 

which a Quality Management Plan is valid is defined in the Quality Management Plan of the 

EPA organization sponsoring the work. They also can be revised at any time to reflect changes in 

scope or other issues affecting the implementation of the QMS.

1.5 SUPERSESSION

This document, EPA Handbook for Quality Management Plans (CIO Handbook 2106-H-02 

QMP) is the original issue of QMP handbook that conforms to the current editions of CIO Policy

2106 and CIO Procedure 2106-P-01 as well as CIO Standard 2106-S-01 and CIO Standard 2106-

S-02.
Handbook QMP 5 December 2012 Agency Review Draft

CHAPTER 2

QUALITY MANAGEMENT PLAN PREPARATION, 

SUBMISSION, REVIEW, AND APPROVAL

2.1 QUALITY MANAGEMENT PLAN PREPARATION

This section relates to Standard Clause 7.2 Quality Management Plan, and Annex A QMP

Preparation Responsibility (Clause A2.1).

In EPA, the organization’s senior management ensures the preparation, approval, and 

implementation of a QMP that covers all environmental programs for which the management is 

accountable. In practice, these activities are often delegated to other managers and staff.

For non-EPA organizations, the organization’s senior management ensures the preparation of a 

QMP (or equivalent acceptable document content) that covers all work specified by the external 

agreement and for which the organization’s management is accountable. (See Sections 2.4 and 

3.3 of this handbook for examples of equivalent acceptable QMP content documents.)

In both cases, senior management includes the senior executive responsible and accountable for 

mission accomplishment and overall operations. Senior management ensures that the QMS 

documented in the QMP conforms to the requirements of the applicable Standard.

The same provisions apply to the preparation of “Programmatic QMPs.” Management authority

and quality-related responsibilities for the Program are clearly established before the QMP is 

prepared.

Managers and staff involved in preparing QMPs can use this document in conjunction with the 

applicable Standard.

2.2 QUALITY MANAGEMENT PLAN SUBMISSIONAND MANAGEMENT 

APPROVAL

This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A 

QMP Preparation Responsibility, QMP Submission and Approval, and Agency Review and 

Approval of the QMP (Clauses A2.1, A2.2, A2.3).

For EPA organizations, the QMP is approved internally and signed by the organization’s senior

manager as well as its Senior Information Official (SIO). The senior manager and SIO indicate 

their approval of the QMP on the signature page of the document. The senior managers are listed 

in the Standards. In most instances, QMPs below the Regional or Program Office level can be 

signed by designated representatives for the SIO. These delegations are normally documented at 

least in the QMP but may also be done through the official delegations system.

For a “Programmatic QMP,” a responsible senior manager is designated as the signing official 

for the QMP.

The QMP is also approved and signed by the QA representative to the senior manager of the 

organization or Program. This is typically the Director of Quality Assurance (DQA) or Quality

Assurance Manager (QA Manager) for the organization.
Handbook QMP 6 December 2012 Agency Review Draft

The senior manager may also require concurrence on the QMP by appropriate subordinate line 

managers to document the acceptance of implementation responsibilities in the QMS.

2.3 QUALITY MANAGEMENT PLAN REVIEW AND APPROVAL FOR USE

Following internal approval, an EPA QMP is submitted to the Office of Environmental 

Information (OEI) Quality Staff for Agency-level review and approval in accordance with CIO 

Standard 2106-S-01. Each QMP is reviewed to determine compliance with Agency-wide Quality

Program requirements, including conformance to the Internal Standard. The review of the QMP 

focuses on the substance of the group’s QA management process and not on format or technical 

details. QMPs that adequately address all program elements described in the Standard are

typically approved for a period not to exceed five years. The next section describes situations 

that may cause a QMP to be revised.

For non-EPA organizations, the QMP is reviewed and approved by the organization’s senior

management as well as the designated QA representative of the senior management prior to its 

being submitted to EPA. The senior manager may require concurrence on the QMP by

appropriate subordinate line managers to encourage the acceptance and implementation of the 

QMS.

Following organization approval, the QMP is submitted to the EPA organization responsible for 

the external agreement for review and approval in accordance with CIO Standard 2106-S-02.

Each EPA organization defines its process for reviewing QMPs submitted under external 

agreements. This process includes procedures for verifying that the QMP is consistent with the 

EPA organization’s goals and for notifying the organization of deficiencies to be addressed in the 

QMP prior to its acceptance by EPA. QMPs prepared by external parties may have an EPA 

approval valid for no more than five years for State, local, and Tribal governments or the length 

of the extramural agreement but not to exceed five years for all other extramural agreement 

holders. The period for which a Quality Management Plan is valid is defined in the Quality 

Management Plan of the EPA organization sponsoring the work.

2.4 QUALITY MANAGEMENT PLAN REVISIONS

This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A

QMP Revisions (Clause A2.4).

For EPA organizations, although the QMP approval by the Agency may be valid for up to five 

years, all Agency QMSs are reviewed at least annually by their organizations (as required by 

CIO Procedure 2106-P-01). This review includes an evaluation of the effectiveness of the QMP

and its implementation. The process of developing and annually updating the QMP provides an 

opportunity for management and staff to review and clarify roles and responsibilities, to address 

problem areas, and to acknowledge successes. Having an accurate and relevant QMP at all times 

is an essential part of every QMS and keeps technical staff informed about the quality program.

Changes in quality policy and procedures are documented in a timely fashion by QMP revisions.

In general, a copy of any EPA QMP revisions made during the year are encouraged to be 

submitted to the OEI Quality Staff as an attachment to, or as part of, the organization’s QA 

Annual Report and Work Plan which is required by CIO Procedure 2106-P-01.
Handbook QMP 7 December 2012 Agency Review Draft

A revised QMP may be submitted to the Quality Staff for review at any time. Revisions occur

under any of the following conditions:

 expiration of the five-year approval duration of the QMP;

 major structural change to the organization (e.g., addition or deletion of a division or 

branch) that requires Agency review;

 significant change in the organization’s mission, such as may occur in the reauthorization 

or revision of enabling environmental legislation, or reduction in resources (e.g., budget 

allocation, reduction in personnel) that may adversely impact the QMS;

 results of audits or assessments and corresponding corrective actions impacting the QMS; 

or

 any other significant change to the organization’s QMS (such as the adoption of new 

quality tools or processes).

In general, the decision to revise a QMP is the responsibility of the organization. In some cases, a 

formal revision of the QMP may not be necessary, but it is helpful if the QMP describes how 

changes will be retained and documented until a formal revision is needed and how the changes 

will be disseminated to users.

As required by the Standards, all appropriate personnel performing work for the organization 

through external agreements (e.g., active contractors, assistance agreement holders) are advised 

of any changes that may affect their work for EPA so that they may be aware of and apply any 

new requirements. To enhance performance, the QMP can describe the process for such 

notifications and who is responsible.

For non-EPA organizations, a QMS review at least annually by the organization includes an 

evaluation of the effectiveness of the QMP. When necessary, the organization revises its QMP to 

incorporate changes to the QMS.

Similar to the conditions cited above requiring revision and resubmission for EPA QMPs, 

revisions to a non-EPA organization’s QMP may become necessary due to:

 a change in the scope (such as a modification to the contract statement of work) in the 

external agreement;

 significant changes in the non-EPA organization’s mission;

 an organizational change within the non-EPA organization or its subcontractors that 

requires notification to EPA and other affected parties; or

 any other significant change to the organization’s QMS (such as the adoption of new 

quality tools or processes).

A revised QMP may be submitted at any time and is subject to the same review and approval 

processes as the original QMP. All personnel (or affected parties) performing work for the 

organization or program can be notified of significant changes to the QMS and the QMP as 

required by CIO Standard 2106-S-02.
Handbook QMP 8 December 2012 Agency Review Draft

2.5 COMBINED QUALITY MANAGEMENT PLAN WITH QUALITY ASSURANCE 

PROJECT PLAN

This section relates to Standards Clause 7.5.2 Combined QMP-QAPP, and Annex B QAPP

Preparation Responsibilities and Approvals (Clause B2.1).

In some circumstances, EPA may recommend the preparation of a combined QMP and QAPP as 

a more appropriate means to document organizational level and project-level environmental data 

activities, particularly when there is no available QMP for the organization performing the work.

Examples may include (but are not limited to) research projects, inspection programs, and 

monitoring networks involving multiple participants, among others. In such cases, a combined 

QMP and QAPP can be a more appropriate means to document organizational-level and projectlevel environmental data activities, particularly when a full QMP is not needed. Combined QMPQAPPs are not used for external agreements containing multiple projects that may be determined 

over the life of the agreement (e.g., mission contracts, program grants). A combined QMP-QAPP

incorporates some elements of the QMP and some elements of the QAPP. In general, such 

documents are largely expanded QAPPs. As with any QMP, it can document the management 

controls that will be applied to the planning, implementation, and assessment of quality activities

for the program or project, and can describe the processes to be followed to carry out those 

activities.

It is EPA’s responsibility to ensure that all elements of a combined QMP-QAPP, as defined by 

the EPA organization, are addressed by the organization performing the work. The combined 

QMP-QAPP is subject to the same EPA review and approval as a QAPP.

HELP BOX – WHEN SHOULD A QMP BE REVISED?

Situations that might make it advisable to revise an existing QMP include but are not limited 

to:

 change in reporting relationships among Quality Managers or other quality

management personnel;

 a corrective action that prompts the need to revise the existing QMS;

 expansion of a laboratory facility to incorporate new categories of methods and 

analytes; and

 significant decreases in budget allocations to the QMS.

On the other hand, the following situations may not lead to a formal QMP revision, but could 

be addressed through using a letter of notification to the approving organization:

 personnel changes (e.g., a new Quality Manager);

 a corrective action that facilitates implementation of the existing QMS; and

 revisions to existing Standard Operating Procedures.
Handbook QMP 9 December 2012 Agency Review Draft

As with any quality document produced by or for an EPA organization, the combined QMPQAPP performs as part of the Quality Management System (QMS) of the EPA organization 

sponsoring the work and will be expected to align with that QMS.

Further information for a combined QMP-QAPP is provided in Handbook on Quality Assurance 

Project Plans (EPA CIO Handbook 2106-H-05 QAPP), Section 1.6.

HELP BOX – COMBINED QMP/QAPP

Some examples of situations in which a combined QMP/QAPP might be appropriate may

include (but are not limited to):

 routine inspection programs where inspections are conducted in the same manner for 

all visits and samples of opportunity can be expected (e.g., EPA lead and/or radon 

programs);

 small organizations that do infrequent work with EPA funds (e.g., competitively

awarded one-time grants, environmental education grants);

 one-time mobilization to a previously unvisited field location, such as for monitoring 

assistance to a Tribal location;

 necessary quick-turnaround data analysis that may involve new laboratory facilities, 

methods, or equipment;

 environmental measurements related to technology performance, where QMS 

planning for the operation of the technology is already addressed by another 

document;

 academic research activities of a one-time nature wherein a formal QMP is more detail 

than needed; or

 a monitoring program conducted by a volunteer organization where the size and 

structure of the organization is limited.
Handbook QMP 10 December 2012 Agency Review Draft

CHAPTER 3

HANDBOOK ON QUALITY MANAGEMENT PLAN CONTENT

This introduction does not correlate directly to a clause in the Standards.

Annex A of CIO Standard 2106-S-01 and Standard 2106-S-02 list the elements that must be 

addressed in a QMP. However, the Standard also indicates that all elements may not apply to an 

organization or program. If that is the case, then the QMP provides explanation and rationale to 

document why the QMP element does not apply. (It is likely that this will occur when 

“Programmatic QMPs” are being developed.)

The QMP elements listed in the Standards are as follows:

 general guidelines;

 conformance with policies, procedures, and standards;

 organizational and program field of application;

 organization, responsibility, and authority;

 QMS description;

 personnel competence;

 external agreements;

 documents and records management;

 use of information technology methods and sources;

 planning;

 implementation of work;

 assessment, oversight, and response;

 data review, validation and verification, and data usability reporting; and

 quality improvement.

In the sections to follow, suggestions will be provided for each of the QMP elements. Since the 

text in the Standards is very detailed, the discussions here will refer to the Standards often. It will 

be beneficial for developers of QMPs to use this document in conjunction with the applicable 

Standard.

3.1 GENERAL GUIDELINES

This section relates to Standard Clauses 1, 2, 7. 2, Purpose, Scope and Applicability, 

Requirements, Quality Management Plan, and Annex A General Requirements (Clause A3).
Handbook QMP 11 December 2012 Agency Review Draft

3.1.1 Overview of QMP Content

This section relates to Standard Clauses 7..2, Quality Management Plan, and Annex A General

Requirements (Clause A3.1).

A QMP addresses the requirements of CIO Standard 2106-S-01 (for EPA organizations) or CIO 

Standard 2106-S-02 (for non-EPA external organizations). It contains descriptions of the 

processes by which the organization plans, implements, documents, and determines the 

effectiveness of its quality management practices, including QA and QC activities, to help 

management to obtain results of its technical work that are of the type and quality needed for 

their intended use. The QMP also reflects the organization’s commitment to quality management 

principles and practices.

Including discussion of those activities, policies, and procedures that are common to all projects 

can reduce duplication between QMPs and QAPPs. Such discussions in the QMP provide an 

“umbrella” under which individual project activities may be performed, and there is then no need 

to repeat them in each of the organization’s QAPPs.

Documents with a different title, such as a Quality Manual or a Program Management Plan, if 

they fulfill the purpose of a QMP and contain the content as detailed in this chapter are 

acceptable.

For EPA organizations, the QMP can be prepared in accordance with the Agency’s Open 

Government Plan, reflecting the principles of transparency, participation, and collaboration.

3.1.2 Administrative Elements

This section relates to Standard Clauses 7.2, Quality Management Plan, and Annex A General 

Requirements, and Organization, Responsibility, and Authority (Clauses A3.1 and 3.4).

The format of a QMP may be tailored by the organization, but in every case the QMP includes

the following administrative elements:

 title page;

 table of contents;

 approvals page with appropriate signatures;

 revision history; and

 applicable references.

A document control format may be helpful to organize the QMP and facilitate future revisions.

The completed QMP can reflect that it has received the approvals described in Section 2.2.

Figure 3 is an excerpt of a hypothetical QMP approval page incorporating document control 

format.
Handbook QMP 12 December 2012 Agency Review Draft

Figure 3. Example of Document Control Format

3.1.3 Application of Graded Approach in QMPs

This section relates to Standard Clause 7.2 Quality Management Plan.

The QMP may incorporate, where applicable, the concept of the graded approach, which is the 

process of applying quality and management controls to an item or work activity according to the 

intended use and degree of confidence needed in the results. For instance, the QMP may state 

that more complex or significant projects carried out by the organization will have a more 

rigorous QAPP, whereas simpler or less significant projects may have a streamlined QAPP. It

may also embody the graded approach by specifying that required approvals and reviews can be 

delegated under defined circumstances. It is important to remember that the graded approach is 

not an opportunity to “cut corners” by bypassing critical management controls. All EPA QMPs 

can certainly include the graded approach, but non-EPA organizations may not need it.

The application of graded approach to specific work is more commonly found at the project 

level. A more detailed discussion of the concept and its application to projects can be found in 

Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Section 1.3) [EPA 

2013c]. For both the QMP and the QAPP, the level of detail can be commensurate with the 

stated goals of the organization or project and the intended use of the results of the 

environmental data collection or production.

DCN: QMP-01-00

Revision: 0

Date: July 2010

Page 12 of 49

APPROVAL PAGE

Deputy Assistant Administrator:

Printed Name Signature Date

Director of QA:

Printed Name Signature Date
Handbook QMP 13 December 2012 Agency Review Draft

3.2 QUALITY MANAGEMENT PLAN CONFORMANCE TO REQUIREMENTS

This section relates to Standard Clause 6 Related Regulations, Policies, and Procedures, and 

Annex A Conformance with Policies, Procedures, and Standards (Clause A3.2).

For EPA organizations, a QMP can cite the external and internal policies, procedures, standards, 

and guidelines to which the QMP conforms. This helps users to understand the scope and extent 

of the QMS more clearly. These may include the documents listed in Figure 1 previously. In 

addition, see Section 6 of CIO Standard 2106-S-01 for a list of other potentially applicable 

guidelines. In addition, Clause 1 of the same Standard states that Informative Annexes (or parts 

thereof) may become requirements if so decided by the organization. Documenting this 

commitment in the organization’s QMP makes it clear when and to whom the organization 

expects it to apply.

Non-EPA organizations may be required to demonstrate and document that the QMP conforms 

to CIO Standard 2106-S-02 as part of a procurement solicitation, assistance negotiation, or other 

pre-award condition, and for any relevant terms of an applicable external agreement. In addition, 

the QMP can cite any other requirements that are pertinent to the organization. Examples of 

other standards and policy documents include (but are not limited to):

 ISO 9001 (Current Edition), Quality Management Systems – Requirements [ISO 2008]

 Uniform Federal Policy for Implementing Environmental Quality Systems

(EPA-505-F-03-001) (Current Edition) [EPA 2005]

These sources could be used to produce quality management documents that are equivalent to an 

EPA QMP. For example, under ISO 9001 the equivalent quality management document is 

usually called the “quality manual” but other titles may be used.

HELP BOX – GRADED APPROACH EXAMPLES

The application of the graded approach concept to the operation of a QMS will vary from one 

organization to another. Here are a few examples of how the graded approach may be 

reflected in a QMP:

 a categorization scheme for ranking projects conducted by the organization by their 

significance and complexity, with higher-ranking projects receiving more thorough 

QA planning and review;

 establishment of a cost threshold for external agreements, with agreements above the 

threshold being subject to on-site audits;

 streamlined QAPP review and approval for short-duration projects;

 records retention schedules tiered to the long-term importance of a given project; and

 designation of criteria for determining which of an organization’s projects will be 

subject to formal peer review.
Handbook QMP 14 December 2012 Agency Review Draft

Conceptually, these documents will address the same general management controls as well as 

roles and responsibilities that are expected in an EPA QMP. In addition, the use of the Uniform 

Federal Policy (UFP) format for QAPPs has been accepted by several EPA organizations as 

equivalent to the requirements given in the CIO Standard 2106-S-01 and CIO Standard 2106-S02. The UFP reference given above describes the Uniform Federal Policy for the creation of a 

QMS and related quality management documents. It is analogous to CIO Standard 2106-S-02 

and the QMP requirements in the Standard. Users can confirm acceptability of this format with 

the accepting EPA organization before using it to prepare QMPs or QAPPs.

3.3 ORGANIZATIONAL AND PROGRAM FIELD OF APPLICATION

This section relates to Standard Clauses 2.1 and 2.2, Scope and Applicability, and Annex A

Organizational and Program Field of Application (Clause A3.3).

QMPs prepared using this handbook can focus on the programs and processes necessary to 

assure quality in environmental data collection, production, and use. EPA Regions may have 

Division- and Branch-level QMPs for decentralized Quality Management Systems. Such QMPs 

may be approved at the Regional level (i.e., not by the OEI Quality Staff). Likewise, states may

have Office or Division QMPs under their parent state QMP (or independent QMS) that are not 

reviewed or approved by EPA when they are outside the scope of the external agreement with 

EPA.

The QMP can clearly identify the organizations or programs and processes to which it applies.

 For EPA organizations, the QMP can address quality management (i.e., management 

controls, roles and responsibilities of managers and staff, quality processes to be applied, 

etc.) for a specific organizational unit, such as a Region, Headquarters Program Office, or 

Office of Research and Development Laboratory or Center, or a subordinate organization 

within those units. The same criteria can also apply to any “Programmatic QMPs” or suborganization QMPs developed by the organization.

HELP BOX – WHAT IS CONFORMANCE?

Conformance (also referred to as conformity) means fulfillment of specific requirements. In 

particular, a QMP conforms to a Standard or similar document if it addresses all applicable 

requirements provided in the Standard.

Conformance with a Standard is distinct from voluntary use of helpful suggestions found in a 

Standard. For instance, QMPs created per EPA CIO Standard 2106-S-01 will conform to 

ANSI/ASQ E4, but they may also use elements of ISO 9001 without the need to conform 

fully to the ISO standard.

Conformance is also distinct from compliance, which is a legal term for meeting the 

requirements of specific laws and regulations.
Handbook QMP 15 December 2012 Agency Review Draft

 For non-EPA organizations, the QMP can address quality management for the 

organization that has an external agreement with EPA. As indicated above, the same 

criteria can also apply to any “Programmatic QMPs” or sub-organization QMPs 

developed by the organization. Examples of sub-organizations are listed in Section 1.2.

3.4 ORGANIZATION, RESPONSIBILITY, AND AUTHORITY

This section relates to Standard Clause 7, Requirements, and Annex A Organization, 

Responsibility, and Authority (Clause A3.4).

3.4.1 Organizational Issues

This section relates to Standard Clause 7 Requirements, and Annex A Organization, 

Responsibility, and Authority (Clause A3.4).

The QMP can present the organization’s Quality Policy, which is a summary statement of the 

organization’s commitment to quality in its environmental data operations. This Policy can vary

in length and degree of specificity. Minimally, the Quality Policy can reflect:

 senior management’s commitment to and engagement with the QMS;

 accountability for quality at all management and staff levels; and

 a commitment to the allocation of adequate resources to the QMS.

Leadership on the part of senior management is crucial to the success of the QMS. The QMP can

describe the specific steps management takes to ensure that the QMS is understood and 

implemented throughout the organization’s environmental data functions. These steps could

include:

 serving as a vocal champion of the QMS;

 ensuring that quality management positions are filled by qualified staff who are supported 

by their management chain;

 actively participating in specific QMS functions such as training and assessments; and

 building quality-related commitments into the organization’s employee performance 

appraisal system.

It is useful to include in the QMP an organizational chart depicting the main components of the 

organization, and highlighting the locations and reporting relationships of all staff with specific 

QMS responsibilities. The hypothetical EPA organizational chart in Figure 2 presents an 

example of how QMS-related relationships can be displayed.
Handbook QMP 16 December 2012 Agency Review Draft

Figure 2. Hypothetical organizational chart depicting QA relationships

If sub-unit QMPs are tiered under an organization-wide QMP, they can be listed in the main 

organization’s QMP, and the relationships between the documents can be defined, along with a 

description of the review and approval procedures for sub-unit QMPs.

3.4.2 Roles and Responsibilities

This section relates to Standard Clause 8 Roles and Responsibilities, and Annex A Organization, 

Responsibility, and Authority (Clause A3.4).

The QMP can include discussion of all roles and responsibilities applicable to the QMS, such as:

 senior management;

 line management/supervisors;

 quality management personnel (Director of Quality, Quality Managers, etc.);

 project managers;

 technical staff; and

 administrative staff.

It is particularly important that the roles and responsibilities of specifically designated quality

management positions (such as Quality Manager, Quality Assurance Coordinator, or Quality 

Assurance Officer) can have detailed explanation in the QMP. For example, representatives 

authorized to sign QAPPs and/or review technical reports on behalf of the QA Manager can be 

defined in the QMP.
Handbook QMP 17 December 2012 Agency Review Draft

Critical to the success of QMS implementation is a full description of and commitment to how 

quality management personnel are permitted functional independence from line management, as 

well as direct access to senior management as necessary to address quality issues. This functional 

independence can be documented in the QMP and delineated to ensure control of organizational 

risks that can arise through conflict of interest.

3.4.3 Dispute Resolution

This section relates to Standard Clause 7.4 Quality System Assessments, and Annex A 

Organization, Responsibility, and Authority (Clause A3.4).

The QMP can describe the process for resolution of any quality-related disputes that are not 

resolved through normal channels and can document those responsible for its implementation.

This process can be designed, in general, to ensure that differences in opinion regarding quality

system implementation are:

 fully documented by the concerned parties;

 resolved at the lowest level of management, if possible; 

 efficiently brought to the attention of senior management if required to facilitate 

resolution; and

 subsequently tracked to ensure that the agreed-upon resolution is accomplished in 

practice and is documented appropriately.

Dispute resolution is an important process and it can be carefully designed. All managers and 

staff who could be affected by or participate in such a process can agree to conform to the 

process. An example of a quality-related dispute would be a disagreement between a line 

manager and a quality manager regarding corrective actions that are necessary to resolve an audit 

finding of non-conformance. The dispute resolution process can ensure that disputes can be 

addressed and resolved with due process and in a timely manner so that work schedule and 

performance are not impeded.

3.5 QUALITY MANAGEMENT SYSTEM DESCRIPTION

This section relates to Standard Clause7 Requirements, and Annex A QMS Description (Clause 

A3.5).

The principal purpose of the QMP is to describe the QMS that is being implemented by an 

organization or program and to establish the roles and responsibilities for managers and staff for 

its implementation.

Accordingly, the QMP can summarize the content of the QMS and describe how it is structured.

The QMS “big picture” can be briefly described up front, with the remainder of the QMP then 

providing details of the QMS.
Handbook QMP 18 December 2012 Agency Review Draft

The document Guidance for Developing a Quality System for Environmental Programs (EPA 

QA/G-1) [EPA 2002b] provides guidance on developing a QMS that conforms to the 

requirements of the Quality Standards. In addition to this document, users may find it helpful to 

examine the following documents for their discussions of QMS elements:

 ANSI/ASQ E4, Quality Systems for Environmental Data and Technology Programs –

Requirements with Guidance for Use. [ANSI 2004]

 ISO 9001, Quality management systems – Requirements. [ISO 2008]

In general, QMS in the EPA-wide Quality Program embody a Plan-Do-Check-Act quality model.

A QMS consists of the quality management controls and processes needed for planning, 

implementing, assessing, and improving the effectiveness of quality processes and practices 

applied to the collection, production, and use of environmental data. Management controls are a 

system of management functions to enable managers to (1) determine that the operations of a 

program or organization satisfy predetermined goals and objectives and that performance is in 

line with standards and specifications; and (2) implement any remedial actions needed to ensure 

that human and other resources are being used in the most effective and efficient way possible in 

achieving the organization’s mission.

HELP BOX – QUESTIONS FOR QMP DEVELOPERS

Developers of QMPs for environmental data collection, production, and use should be able to 

answer questions such as the following:

 Who are the principal stakeholders for our environmental data operations (they may

include other organizational units, other levels of government, members of the public, 

etc.)?

 What is our process for identifying, reconciling, and prioritizing these stakeholders’ 

requirements and expectations?

 What specific environmental data will be collected, produced, or used to meet these 

requirements and expectations?

 What processes and management controls will be carried out?

 What roles and responsibilities will be assigned for the execution of these processes 

and management controls?

 How will we evaluate the effectiveness of our processes and their outputs on an 

ongoing basis?

 How will we know the extent to which stakeholders’ requirements and expectations 

are being met?
Handbook QMP 19 December 2012 Agency Review Draft

3.6 PERSONNEL COMPETENCE

This section relates to Standard Clause .6 Personnel Competence, and Annex A Personnel

Competence (Clause A3.6).

This part of the QMP might begin with the organization’s policy statement on ensuring the 

competence of personnel to perform the technical aspects of project work as well as the QA and 

QC functions that are designed to meet the organization’s project and mission objectives. It 

would also be beneficial to describe management responsibility for support and/or commitment 

to ongoing training in technical fields both currently relevant as well as in advance of initiatives.

It is important that the personnel responsible for the implementation of the QMS elements 

possess sufficient knowledge, experience, and understanding of quality principles, processes, and 

practices.

Personnel competence is generally composed of the requisite technical knowledge, relevant 

education, work experience, and specialized training (as required). Personnel competence can be 

maintained and demonstrated through means such as the following or in combination thereof:

 Training: Technical and management training, which may include classroom, electronic, 

and on-the-job, can be provided by the organization as needed.

 Certification: Statutory, regulatory, or professional certifications may be required to 

perform certain operations (e.g., laboratory analyst competency).

 Observation: Formal supervisory observation may be required before a staff member is 

authorized for independent performance of an activity such as operating measurement 

and testing equipment.

The processes for carrying out these or other measures to ensure personnel competence can be 

documented in the QMP. In addition, the QMP can describe how objective evidence of personnel 

job proficiency will be documented and maintained for the duration of the project or activity

affected, or longer if required by statute or policy. Accurate and up-to-date records on personnel 

competence are valuable for numerous QMS-related purposes, such as QAPP and SOP 

development, audit preparation, and design of corrective actions. Such records provide 

management with documentation that staff members are competent to perform the work 

assigned. Workers without adequate competence pose a risk to the successful completion of the 

work and possibly to the organization’s liability in the case of an accident.

Finally, the QMP can describe how and when any need to implement new or revised training, 

implement certification of personnel, or define additional personnel qualifications will be 

reevaluated by the organization. This is advisable because the organization’s evolving mission 

may lead to new training requirements.
Handbook QMP 20 December 2012 Agency Review Draft

3.7 EXTERNAL AGREEMENTS

This section relates to Standard 01 Clause 7. 11 Management of Quality Requirements for 

External Agreements, Standard 02 Clause 7.11 Management of Quality Requirements for 

Subagreements, and Annex A External Agreements (Clause A3.7).

3.7.1 EPA Organizations

Extramural agreements provide EPA with a critical means of accomplishing work related to the 

Agency’s mission. Accordingly, it is very important for the organization to ensure that 

appropriate quality requirements are applied to its suppliers, subcontractors, and other external 

agreement holders. This is vital to the success of the QMS, because many projects involving 

environmental data collection, production, and use may utilize inputs from outside the primary

organization. These may be focused contributions, such as those of testing laboratories; or in 

other cases, a sub-recipient may be assigned overall responsibility for a major activity such as a

large, complex field sampling activity.

For EPA organizations, external agreements may include:

 acquisition of services and items through contracts, task orders, and other acquisition 

agreements;

 assistance agreements such as grants and cooperative agreements;

 interagency agreements negotiated with other Federal, State, Tribal, or local 

governments;

 international agreements negotiated with governmental, non-governmental organizations, 

or other entities;

 settlements and agreements (e.g., administrative orders, consent agreements) resulting 

from enforcement actions;

 sub-awards whose recipients are accountable to the prime recipient of an external 

agreement for environmental data activities provided under the agreement; and

 voluntary partner programs (e.g., voluntary monitoring).

The QMP can describe how the organization will:

 determine whether the scope of a proposed or implemented external agreement includes 

the collection, production, or use of environmental data; and

 ensure that appropriate quality practices and procedures are defined and applied to 

external agreements as appropriate.

An EPA organization can identify, develop, and document the processes and procedures 

(including management and staff responsibilities) for ensuring that all external agreement 

holders apply appropriate quality requirements. EPA organizations can identify and document 

how recipients of external agreements require their own suppliers (e.g., contractors, 

subcontractors, or financial assistance recipients) to maintain a quality system consistent with 

EPA requirements. This provision applies only to those suppliers providing services or items 

directly affecting the quality of results or products from environmental data activities.
Handbook QMP 21 December 2012 Agency Review Draft

EPA organizations have considerable flexibility in requiring documentation on the scope of 

quality practices in external agreements for environmental data. For example, under acquisitions, 

an EPA organization can provide in the QMP a summary of the organization’s process for 

implementing the EPA Contracts Management Manual [EPA 2009a] specification for using a 

QA Review Form (QARF). In addition, the QMP may describe how the QARF is integrated in 

the EPA Acquisition System (EAS).

Similarly, the QMP may include a description of the process for assistance agreements which is 

documented through the Integrated Grants Management System (IGMS). Moreover, the QMP 

may describe the processes for Inter-Agency Agreements (IA) and other assistance agreements.

For further information, EPA organizations may refer to EPA Order 5700 Change 6, Assistance 

Administration Manual [EPA 1984], and to EPA Order 1610, Interagency Agreement Policies, 

Procedures, and Guidance Manual [EPA 2008a].

Each EPA organization’s Quality Manager (or designee) can be authorized to review 

documentation for pending external actions (e.g., Statements of Work and contractor proposals) 

to ensure that appropriate quality-related requirements are being addressed and incorporated.

This authority can be clearly defined in the QMP. Moreover, for projects involving multiple EPA 

organizations with shared responsibilities, the QMP can describe how the lead responsibility for 

QA will be determined and assigned. In the case of IAs, when EPA provides the funding or a 

substantive majority of resources expended, the application of QA and oversight responsibility

remaining with EPA is negotiated into the agreement. When EPA does work for another agency

or department, the EPA organization’s quality management system and QA oversight is applied, 

using a graded approach, by default unless otherwise negotiated in the agreement.

3.7.2 Non-EPA Organizations

Non-EPA organizations that have external agreements with EPA can discuss how the quality

requirements in their QMS will flow down to their suppliers and sub-recipients associated with 

environmental data operations. However, any requirements on sub-recipients (e.g., 

subcontractors) are made through the prime recipient (e.g., contractor), as there is no direct 

contractual relationship between EPA and external agreement sub-recipients.

The QMP can list the functions that are likely to be implemented by suppliers or sub-recipients 

(e.g., field sampling services, laboratory testing, and information technology support). For each 

of those areas, the QMP would then describe how the QMS will be applied to sub-recipients. For 

example, non-EPA organizations may:

 require lower-tier sub-recipients to prepare their own QMPs or QAPPs;

 review procurement documents for sub-recipients;

 conduct audits or on-site inspections of supplier facilities and operations;

 conduct validation of data supplied by contract laboratories;

 maintain acceptance criteria for lower-tier sub-recipients; or

 issue stop-work orders or requests for corrective actions to sub-recipients.
Handbook QMP 22 December 2012 Agency Review Draft

3.8 DOCUMENTS AND RECORDS MANAGEMENT

This section relates to Standard Clause 7.10 Documents and Records Management, and Annex A

Documents and Records Management (Clause A3.8).

An organization’s process for management of quality-related documents and records is often 

crucial to the success of a QMS, which is ultimately not just about achieving quality

performance, but also about demonstrating (that is, documenting) that quality performance has 

been achieved. Accordingly, it is very important the documents and records affecting the quality

of the organization’s environmental data operations be identified and documented in the QMP.

In general, the QMP can:

 describe the maintenance and management of major quality planning documents, 

including the organization’s QMP and QAPPs;

 define other documents that will be considered quality-related and managed accordingly;

 describe characteristics (e.g. of function or content) to be used in identifying qualityrelated documents not anticipated by or named in the QMP;

HELP BOX – EXAMPLES OF QUALITY REQUIREMENTS FOR SUPPLIERS

Suppliers of services and indirectly through the suppliers, their sub-recipients involved in 

environmental data collection, production, and use may be required to:

 develop QMPs describing their quality system;

 develop QAPPs describing and documenting explicit project details;

 develop SOPs for certain processes and practices;

 commit to fulfilling specific QA and QC requirements established by the sponsoring 

organization;

 submit to audits to confirm their capability to satisfy quality criteria (e.g., performance 

evaluations, technical systems audits);

 obtain review and approval from the sponsoring organization for any mid-course 

changes;

 demonstrate that all QA and QC requirements have been carried out in practice;

 provide objective evidence of quality in products and services provided by their own 

suppliers and sub-recipients, including source selection, source inspections, supplier 

audits, and examination of deliverables; and

 review all deliverables against the acceptance criteria established by the sponsoring 

organization.
Handbook QMP 23 December 2012 Agency Review Draft

 document how all technical guidance documents are prepared, reviewed, approved, 

issued, used, and revised (this can apply to internally developed and externally acquired

documents);

 define what records will be generated and how they will be maintained;

 describe how the organization will ensure that documents and records accurately reflect 

completed work;

 discuss retention times, archiving, retrieval, and disposition of quality-related documents 

and records;

 describe processes for handling documents and records to assure their accessibility, 

protection from damage and deterioration, and means of retention, including discussion 

of the roles and responsibilities for management and staff;

 document chain of custody and confidentiality procedures for evidentiary records; and

 commit to compliance with all statutory, contractual, and assistance agreement 

requirements for records from environmental programs, including providing adequate 

preservation of key records necessary to support the mission of the organization.

EPA organizations are required by CIO Standard 2106-S-01 to comply with Agency policy on 

documents and records. The QMP developers can refer to the provisions of Section 6 of the EPA 

Records Management Policy [EPA 2009b] to define this process for their QMP. This section lists 

the minimum requirements for an EPA records management program, including creation, 

management, maintenance, protection, and disposal of records. In addition, QMP developers can

document any additional organization-specific processes for management of documents and 

records as described above.

For non-EPA organizations, the QMP can document or refer to the organization’s own records 

management policy and procedures and to the applicable terms and conditions of the external 

agreement. In general, Federal regulations governing external agreements will specify documents 

and records management requirements. Adherence to such requirements can be affirmed in the 

organization’s QMP. Any differences for specific projects can be documented in the QAPP; see

Handbook on Quality Assurance Project Plans (EPA CIO Handbook 2106-H-05 QAPP) for 

more information.

3.9 USE OF INFORMATION TECHNOLOGY METHODS AND SOURCES

This section relates to Standard Annex A Use of Information Technology Methods and Sources

(Clause A3.9).

Especially with recent advances, information technology (IT) plays an increasingly important 

role in the collection, production, and use of environmental data. As noted in CIO Standards 

2106-S-01 and 2106-S-02, IT encompasses a wide range of activities, functions, and sources for 

environmental data collection, production, and use, including general computer software 

programs, computer models, information systems, and web-based applications.
Handbook QMP 24 December 2012 Agency Review Draft

The QMP can present the organization’s general approach to applying QA and QC processes and 

practices to applications involving information systems management. Activities addressed by the 

QMP can include consideration of acquired results from IT methods and sources, such as 

services that collect and provide access to data and information.

Organizations can develop and document the general processes, procedures, and personnel 

responsibilities for determining when QA planning documents (e.g., QAPPs) are needed and for 

evaluating the suitability of environmental data obtained from information technology sources.

The QMP can describe and document the processes related to the preparation, review, approval, 

and use of QA planning documents for all IT-related applications. Developers of QMPs may find 

the discussions in the Standards and in their Annexes A and B to be useful in defining the 

organization’s QMP content for IT activities.

Environmental data products generated using IT applications and sources include (but may not 

be limited to) the following:

 Computer mathematical models and other electronic methods: Environmental data 

produced from computer programs and models, mathematical models, and other 

electronic methods and sources [e.g., Geographic Information Systems (GIS)] can be 

reviewed, evaluated, verified, and validated in order to confirm their suitability for use.

For more information, see the Web site of EPA’s Council for Regulatory Environmental 

Modeling (http://www.epa.gov/crem/), which is intended to promote consistency and 

consensus among environmental model developers and users.

 EPA-owned information systems: Quality principles can be applied to the development, 

modification, operation, and management of EPA information systems to ensure that the 

system is designed to satisfy the intended uses and user requirements and that data 

maintained in the system can be audited and verified from entry to retrieval. Examples of 

EPA information systems include CERCLIS (Superfund), TRI (toxics), and SDWIS

(drinking water); see http://www.epa.gov/enviro/index.html which is a centralized 

repository of these systems.

 Other information systems and the Internet: Environmental data collected from other 

information sources (including the Internet) can receive special attention to ensure that 

the data can be used for their intended purpose. The QMP can commit to the development 

and documentation of project-specific evaluation criteria in applicable QAPPs.

 Computer software and hardware: Computer software and hardware relevant to the 

mission and objectives of the organization can be controlled appropriately to satisfy the 

organization’s objectives. QMP developers can refer to applicable OEI procedures and 

guidance for information on software and hardware controls; these resources are 

compiled at http://www.epa.gov/irmpoli8/policies/index.html.

For EPA organizations, the QMP can discuss the organization’s commitment to complying with 

all applicable CIO policies, procedures, standards, and guidelines, as well as organizationspecific policies and procedures. It can describe the process for ensuring that data and 

information produced by or on behalf of the Agency meet applicable requirements and standards.
Handbook QMP 25 December 2012 Agency Review Draft

For non-EPA organizations, the QMP can describe organization-specific systems, policies, and 

approaches. If the applicable external agreement provides for compliance with specific EPA CIO 

polices, the QMP can also reflect that commitment.

In addition, a QMP for either an EPA or an external organization can discuss any particular 

information systems issues that could be applicable across the range of environmental data 

operations covered in the QMP. Examples may include (but are not limited to):

 field data logging processes;

 Laboratory Information Management Systems;

 modeling or other uses of data;

 electronic data bases;

 computer programs used to analyze environmental data; and

 systems for posting and sharing data (such as the Environmental Information Exchange 

Network).

Project-specific processes, procedures, and evaluation criteria when developed are documented 

in the corresponding QAPPs.

3.10 PLANNING

This section relates to Standard Clause: Systematic Planning (7.7), and Annex A Planning

(Clause A3.10).

3.10.1 The QMP as the Foundation of QMS Planning

This section relates to Standard Clause: Systematic Planning (7.7), and Annex A Planning

(Clause A3.10).

The QMP can describe how the organization satisfies the planning requirements of the QMS.

Planning is generally regarded as the most important phase of a quality system; in the absence of 

a systematic focus on planning along with an appropriate set of planning tools, a QMS can 

devolve into a mere inspection program for detecting flawed environmental data.

The foundation of the planning approach built into the QMS is provided by the QMP itself. The 

specific QMP elements described in this handbook beginning with Section 3.7 (ensuring 

personnel competence, promoting quality performance in external agreements, and so on) 

constitute the building blocks of the organization’s overall plan for ensuring effective delivery of 

environmental data that meet the needs of the organization’s stakeholders and customers.

3.10.2 Commitment to Project-Specific Planning

This section relates to Standard Clause, Systematic Planning (7.7), and Annex A Planning

(Clause A3.10).
Handbook QMP 26 December 2012 Agency Review Draft

In addition to this overall planning framework, the QMP can make a specific commitment to the 

use of project-level systematic planning for any activities involving the collection, production, 

and use of environmental data. Systematic planning for a project entails the following aspects at 

a minimum:

 defining the project, or creating an overall framework upon which the project will be 

developed;

 establishing the scope of the investigation, including its temporal, physical, and logistical 

boundaries;

 documenting the needs or demands of the project, which may include conceptual models 

or sampling and analytical methods and procedures; and

 defining protocols for tracking and reporting on project progress.

The particulars of the systematic planning approach depend on the focus of the project – whether

it involves sampling and analysis, data analysis, a modeling application, or another type of 

environmental data operation. But in all cases, the organization can establish and document in 

the QMP a systematic planning process that:

 identifies all relevant customers, stakeholders, and interested parties (as applicable), as 

well as their needs and expectations, for the results of the work to be performed;

 identifies any internal or external resource, equipment, or technical expertise necessary 

for successful performance;

 identifies the technical and quality goals that meet the needs and expectations of the 

customer;

 translates the technical and quality goals into specifications that will produce the desired 

result;

 considers any cost and schedule constraints within which project activities are required to 

be performed;

 specifies the QA and QC activities required for tracking project performance; and

 identifies measurable acceptance criteria or performance measures by which the results 

will be evaluated and the suitability of the environmental data for their intended use will 

be determined.

The general planning processes (including applicable design steps) can be described. All 

planning documentation can be reviewed and approved for implementation by authorized 

personnel before the affected planned work commences. Such documentation includes, but is not 

limited to, work plans, schedules, standard operating procedures (SOPs), and QAPPs.

The QMP can describe in specific terms the organization’s process for developing, reviewing, 

approving, implementing, and revising QAPPs. Detailed information is available in Handbook

on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Section 1.7) [EPA 2013c].
Handbook QMP 27 December 2012 Agency Review Draft

The QMP can also document the process for ensuring that all versions of planning documents, 

procedures, and related documents will be retained in accordance with EPA policy and 

procedures for documents and records, and in accordance with extramural agreement regulations.

HELP BOX -- OVERVIEW OF SYSTEMATIC PLANNING ELEMENTS

The particulars of the systematic planning approach depend on the focus of a given project, 

but the QMP should describe the organization’s overall approach to addressing and 

documenting the following elements:

Organization: Identification and involvement of the project manager, sponsoring 

organization and responsible official, project personnel, stakeholders, scientific experts, 

customers, suppliers, and Quality Manager.

Schedule: Identification of project schedule, resources (including budget), milestones,

regulatory or contractual requirements, and circumstances that could adversely affect the 

progress of the project.

Project Goal: Description of the project goals, objectives, study questions and conceptual 

model that describe the environmental issue or concern, contamination pathways, and general 

impact on human health.

Quality Assurance: Specification of needed quality assurance and quality control activities 

to ensure performance criteria (for new data) or acceptance criteria (for existing data) are 

satisfied.

Data Needs: Identification of the type of data needed, how the data will be used to support 

the project’s objectives, and identification of potential limitations in data interpretation.

Criteria: Determination of the quantity of data needed and specification of performance 

criteria for measuring quality for new data, or acceptance criteria for the use of existing data.

Data Collection: Description of how and where the data will be obtained (including existing 

data or data from other sources where the quality may be unknown or undocumented) and 

identification of any constraints on data collection.

Analysis: Description of how data will be analyzed, evaluated, and assessed against its 

intended use with respect to the established performance or acceptance criteria.
Handbook QMP 28 December 2012 Agency Review Draft

3.11 IMPLEMENTATION OF WORK

This section relates to Annex A, Implementation of Work (Clause A3.11).

3.11.1 General Considerations

The QMP can describe the general processes designed to ensure that all work involving 

environmental data collection, production, or use is performed according to approved planning 

and technical documents (e.g., QAPPs, SOPs) and in the prescribed sequence described in them.

The benefits of systematic planning can be lost without careful project implementation aimed at 

producing the desired results.

Although the details of project implementations are to be found in QAPPs, the QMP can address 

the organization’s overall approach to documentation of procedures, sample and data integrity, 

performance monitoring, and approval of deviations from established processes.

3.11.2 Documentation of Implementing Procedures

SOPs (or similar tools such as Reference Methods) routinely play a central role in promoting 

effective and consistent implementation of repetitive data acquisition tasks. An SOP is a

document that details the method or procedure for implementing an operation, analysis, or action 

with thoroughly prescribed techniques and steps. SOPs are typically approved as the “official” 

method for performing certain routine or repetitive tasks. The process for the development and 

implementation of SOPs along with the roles and responsibilities of managers and staff can be

documented in the QMP. SOPs can be written in a format that can be readily understood by the 

user (i.e., using plain English to the extent possible), with sufficient detail and clarity to ensure 

that effective results can be expected and achieved consistently. The process for the 

documentation of SOPs in the QMP may include (but may not be limited to):

 identification of operations needing procedures;

 preparation of procedures, including form, content, and applicability;

 testing and evaluation of procedures (as necessary) to confirm their acceptable 

performance;

 review and approval of documented procedures; and

 assurance that up-to-date SOPs are available and those out-of-date SOPs are revised or 

withdrawn as needed.

 SOPs no longer in use are maintained by the organization to support documentation of 

the projects using that version according to program and records management for that 

organization.

Procedures that specify technical requirements can be reviewed for adequacy and approved 

before use by qualified personnel who are independent of the entity developing the procedure.

For more information on the development of SOPs, developers of a QMP may find it helpful to 

examine EPA QA/G-6, Guidance for Preparing Standard Operating Procedures (EPA 2007).
Handbook QMP 29 December 2012 Agency Review Draft

3.11.3 Sample and Data Integrity

This section relates to Standard Annex A, Implementation of Work (Clause A3.11).

The general processes and roles and responsibilities of personnel relative to sample and data 

handling and custody can be described in the QMP. For instance, if the organization will 

maintain a chain of custody within or across projects, its approach can be explained in the QMP.

Specific processes and roles and responsibilities of personnel pertaining to sample and data 

integrity for individual projects are presented in the applicable QAPPs.

3.11.4 Performance Monitoring

This section relates to Standard Annex A, Implementation of Work (Clause A3.11).

Level of management oversight and verification for the implementation of work is normally

determined based on the importance of the particular project and the intended use of the project 

results.

The QMP can describe how the implementation of work processes will be monitored by the 

organization, including the routine measurement of performance against established technical 

and quality specifications to ensure continued satisfactory performance. It is important that the

independence of personnel monitoring the work performance be ensured and that this 

independence be commensurate with the nature and importance of the activity. The QMP may

need to address how to handle situations wherein complete independence of personnel cannot be 

achieved, such as within a small organization. Developers of the QMP may consider possible 

safeguards that could be utilized to provide for a practicable level of independence.

3.11.5 Deviations from Approved Processes

This section relates to Standard Annex A Implementation of Work (Clause A3.11).

The QMP can describe the process for how deviations from approved processes and procedures 

will be resolved (if possible), documented, and reported to management. The process and roles 

and responsibilities of managers and staff for addressing these deviations and any waivers from 

established policies, procedures, and standards can be described in the QMP. If the deviation 

necessitates a revised version of a planning document such as a QAPP or SOP, it is important to 

the reproducibility, defensibility, and utility of results that the current and subsequent versions of 

the document are indicated using a document control format as described in Section 3.2.2.
Handbook QMP 30 December 2012 Agency Review Draft

3.12 ASSESSMENT, OVERSIGHT, AND RESPONSE

This section relates to Standard Clause: Assessment Oversight, and Response (7.8), and Annex A 

Assessment, Oversight, and Response (Clause A.3.12).

3.12.1 Types of Assessments

CIO Standards 2106-S-01 and 2106-S-02 require organizations to develop and implement 

processes for assessment and oversight, and for responses to them. Assessments typically have 

specific criteria or measures of success associated with them. In general, an assessment is an 

evaluation process used to measure the performance or effectiveness of a system and its 

elements. As used here, assessment is an all-inclusive term used to denote activities such as the 

following:

 Data Quality Audit (DQA) – a scientific and statistical evaluation of validated data to

determine if the data are of the right type, quality, and quantity to support their intended 

use.

 Performance Evaluation (PE) – a type of audit in which the quantitative data generated 

in a measurement system are obtained independently and compared with routinely

obtained data to evaluate the proficiency of an analyst or laboratory.

 Quality System Audit (QSA) – a documented activity performed to verify, by

examination and evaluation of objective evidence, that applicable elements of the quality

management system are suitable and have been developed, documented, and effectively 

implemented in accordance with specified requirements.

HELP BOX – MANAGEMENT-ORIENTED SOP TOPICS

Many Standard Operating Procedures relate to the details of data collection, production, and 

use, and are typically addressed in a QAPP. Examples of management-oriented SOPs that 

may be provided or referenced in a QMP include the following:

 Standard Operating Procedure Development

 Quality Assurance Project Plan Development

 Quality Training System

 External Agreement Review Process

 Management of Quality Records

 Quality System Assessment Process

 Corrective Action Process

 Dispute Resolution Process
Handbook QMP 31 December 2012 Agency Review Draft

 Readiness Review – a systematic, documented review of the readiness of the start-up or 

continued use of a facility, process, or activity. Readiness reviews are typically conducted 

before proceeding beyond project milestones and prior to initiation of a major phase of 

work.

 Surveillance – continual or frequent monitoring and verification of the status of an entity 

and the analysis of records to ensure that specified requirements are being fulfilled.

 Technical Systems Audit (TSA) – a thorough, systematic, on-site, qualitative audit of a 

project or operation that may include facilities, equipment, personnel, training, 

procedures, recordkeeping, data validation, data management, and reporting aspects of a 

technical system.

In the general context of assessments, the related concept of oversight pertains to monitoring 

activities or operations to confirm that they conform to approved plans. That is, are those 

implementing the work doing what they said they would do? In the list above, “surveillance” 

encompasses aspects of both assessment and oversight.

Users of this handbook may find that the term “audit” is used more commonly than 

“assessment”.

3.12.2 Assessment Planning

Assessments offer the assurance that projects and programs are being implemented in accordance 

with established plans. In addition, assessment findings provide the content that leads from 

“check” to “act” – that is, they identify opportunities for preventive actions, corrective actions,

and continuous improvement. Accordingly, when assessments of QMS are planned and 

scheduled by the organization regularly.

Their evaluated results can determine the suitability and effectiveness of the implemented QMS 

and of the quality performance of the environmental programs to which it applies.

The QMP can describe the process used by management to determine during the planning stage 

the type(s) of assessment activities appropriate for a particular project, and the assessment tool to 

be used. The types and frequency of assessments can be selected based on the particular needs of 

the organization and on the nature of its environmental data operations. Some projects might 

benefit from periodic technical systems audits in the field to ensure that samples are collected in 

accordance with plans, while other projects might be best served by rigorous data quality audits.

A similar approach may be applied to determining the depth and intensity of an assessment (i.e.,

how many issues will be addressed, how many assessors will be assigned, and how many days 

the assessment will last). The optimum scope and balance of assessments is a program-specific 

determination.

For non-EPA organizations, specific requirements for assessments may be included in the 

external agreement. In some cases, assessments may be explicitly required in the scope of the 

external agreement such as in field monitoring activities or laboratory confirmation of sampling 

results.
Handbook QMP 32 December 2012 Agency Review Draft

Documenting the scope and objectives of every assessment in an assessment plan aids in the 

planning process, permits consistency of approach over time, and provides the basis against 

which to document and evaluate results.

A QMP can description of how such plans are generally reviewed and approved, and by whom

reinforces the use of this valuable preparation tool and assists the assessment team in engaging 

with the appropriate parties. In this manner, assessments (or audits) can be performed according 

to approved written procedures (e.g., audit plan), based on careful planning of the scope of the 

assessment, the information needed and expected from the assessment, and any limits or 

constraints encountered (e.g., seasonal considerations when collecting water or soil samples).

3.12.3 Assessment Conduct and Follow-Up

Description in the QMP of the organization’s approach for conducting and using the results of 

assessments and oversight activities, including the roles and responsibilities of managers and 

staff in this process increases the probability of these activities being accomplished.

The general processes and roles and responsibilities of personnel relative to conduct of

assessments, implementation of response actions, and verification of responses can be also 

described in the QMP. This includes procedures for ensuring the objectivity, independence, and 

competence of assessment personnel. Specific needs can be defined in individual QAPPs for the 

activity.

The organization can have a system for selecting assessors who have the appropriate level of 

competence (i.e., capability, experience, and training) to meet the needs of the organization’s 

assessment program. Again, the specific competence needed for a particular assessment can be 

addressed in the QAPP, but the QMP can address the more general, overarching expectations. 

Assessors (or auditors) can also have demonstrated independence from the work being assessed 

and can have no real or perceived conflict of interest.

Again, specific processes and roles and responsibilities of personnel pertaining to assessment of 

individual projects can be described in the applicable QAPPs, as discussed in CIO Handbook

2106-H-05 QAPP, Sections 2.4, 3.4, and 4.4.

Assessment results can be documented, reported to, and reviewed by management. This process 

can be described in the QMP along with the roles and responsibilities of involved managers and 

staff. The QMP can describe the process for identifying, conducting, and tracking corrective 

actions in response to assessments. The organization can ensure that corrective actions are made 

on a timely basis and that the effectiveness of their implementation is confirmed and 

documented.

The OEI Quality Staff typically conducts an assessment (i.e., Quality System Audit) of an EPA 

organization’s Quality Management System at least every five years, but may do so more 

frequently if needed. Assessments of a non-EPA organization’s Quality Management System are 

normally conducted as provided for in the external agreement, or at least every five years if the 

agreement is for an extended period. It is recommended that external organizations conduct selfassessments in accordance with a schedule defined in their QMP and agreed to by EPA.
Handbook QMP 33 December 2012 Agency Review Draft

For more information on planning, conducting, and reporting audits, EPA QA/G-3, Guidance on 

Assessing Quality Systems [EPA 2003] and ISO 19011, Guidelines for Quality and/or 

Environmental Management Systems Auditing [ISO 2002] may be helpful.

Other guidance on the planning, implementation, and reporting of audits and assessments that a 

user may find useful in developing a QMP is available in the following documents:

 Guidance on Technical Audits and Related Assessments for Environmental Data 

Operations (EPA QA/G-7) [EPA 2000]

 Data Quality Assessment: A Reviewer’s Guide (EPA QA/G-9R) [EPA 2006a]

 Data Quality Assessment: Statistical Tools for Practitioners (EPA QA/G-9S) [EPA

2006b]

HELP BOX – CORRECTIVE ACTION PROCESS

The QMP should describe the organization’s overall approach for designing, implementing, 

and tracking corrective actions taken in response to audit findings. The corrective action 

process should be the responsibility of the organizational unit to which the finding applies, 

and should be sufficiently formal and transparent that management can confirm that the 

finding has been resolved. The following outline describes one formal approach that may be 

used to plan and conduct a corrective action process.

Step 1: Brainstorm to identify the possible causes of the finding.

Step 2: Agree on likely root causes. (A number of standard quality tools are available to help 

identify root causes of a problem.)

Step 3: Develop a list of potential solutions.

Step 4: Decide what solution to evaluate more fully.

Step 5: Develop a plan for implementing the solution and tracking its effectiveness. Develop 

a plan that breaks down the corrective action into discrete steps with an associated schedule.

Step 6: Implement and monitor the corrective action based on the corrective action plan.

Step 7: Develop a process for evaluating the effectiveness of the corrective action once it has 

been implemented.

Step 8: Determine the effectiveness of the implemented actions. Consider need to improve 

the solution or rethink root causes and start again.
Handbook QMP 34 December 2012 Agency Review Draft

3.13 QUALITY IMPROVEMENT

To facilitate continuous improvement within the organizations Quality System describe in the 

QMP:

 the organization's process for ensuring continual quality improvement, including the 

management process for determining, planning, implementing, and evaluating the 

effectiveness of quality improvement activities including:

o who (organizationally) is responsible for quality improvement; and

o the corrective action program to ensure that conditions adverse to quality are

 identified promptly, and

 corrected as soon as practical.

 how the organization shall detect and prevent quality problems, and

 how staff at all levels are encouraged to identify and establish communications among 

customers and suppliers, identify process improvement opportunities, identify problems, 

and offer solutions to those problems.

In order to enhance the effectiveness of corrective actions to be effective, it is strongly 

recommended that they include:

 identification of root causes of problems,

 determination of whether the problem is unique or has more generic implications, and a 

recommendation of procedures to prevent recurrence.

3.14 DATA REVIEW, VALIDATION AND VERIFICATION, AND DATA USABILITY 

REPORTING

This section relates to Standard Clause: 7.9 Data Review, Validation and Verification, and Data 

Usability Reporting, and Annex A Assessment, Oversight, and Response (Clause A.3.12) and

Quality Improvement (Clause A.3.13).

3.14.1 Planning for Data Review

The use of any environmental data for decisions follows the review of such data to determine the 

extent to which the technical and quality objectives for the data were met. Both Quality

Standards require that organizations develop and implement appropriate review processes to 

confirm the suitability and adequacy of data for their intended use. The primary purpose of a 

QMS for environmental data collection, production, and use is to ensure that the resulting data 

have the necessary integrity, are defensible, and have adequate documentation to allow for other 

possible uses.

The criteria for the adequacy of such data will largely be specific to the project and can be 

addressed by the QAPP; however, it is appropriate for the QMP to document the general 

approaches and array of evaluation tools and to describe the general roles and responsibilities of 

managers and staff in their use.

The QMP can describe the possible or routine processes by which results obtained from 

environmental data collection or production are reviewed, verified, validated, and qualified prior 

to their intended use.
Handbook QMP 35 December 2012 Agency Review Draft

The QMP should also commit to documenting any necessary deviation from routine processes 

and/or the development or use of alternative processes. Where possible, this discussion includes

identification of the roles and responsibilities of management and staff relative to these 

functions. Specific processes and the roles and responsibilities of personnel pertaining to the 

evaluation of individual projects can be described in the applicable QAPPs.

The Quality Standards require that any project reports that contain data, or report the results of 

environmental data operations (e.g., data summaries) are to be reviewed independently (i.e., by 

others than those who produced the data or the reports) to confirm that the data or results are 

presented correctly. The QMP can describe the general process for the review and approval of 

these reports and can document the roles of managers and staff needing to participate. The 

Standards require that any limitations or restrictions on the use of the data shall be expressed 

quantitatively to the extent practicable and shall be documented in any reporting of the data.

When adequate documentation of the data accompanies its reporting it will reduce, but not 

completely eliminate, the risk of the data being misinterpreted or misused for decisions.

3.14.2 Special Considerations for Data Review

If environmental data are collected from sources that did not use a QMS or whose quality is 

unknown or undocumented, the organization can review and evaluate these data for their 

suitability for use. Examples of such sources may include:

 academic research and journal articles;

 previous studies of contamination at a particular waste site; and

 reports from volunteer groups such as a watershed association.

In addition, when data collected for purposes other than that of the current project are to be used, 

the QMP can describe the general process to be utilized for determining the suitability of the data 

for the new use. Details pertaining to a specific project can be described in the applicable QAPP.

Key elements of a review of environmental data for potential use include:

 the need to satisfy the major requirements of the Information Quality Guidelines (IQG), 

e.g., objectivity, utility, and integrity – see EPA/260R-02-008, Guidelines for Ensuring 

and Maximizing the Quality, Objectivity, Utility, and Integrity of Information 

Disseminated by the Environmental Protection Agency [EPA 2002a];

 the principal indicators of environmental quality, e.g., precision, bias, representativeness, 

comparability, completeness, and sensitivity – see Appendix B of EPA CIO Handbook

2106-H-05 QAPP; and

 the general usability factors, e.g., relevance, temporal compatibility, spatial inclusivity, 

credibility, aggregation level, compatibility, reproducibility, and degree of augmentation

– see Sections 2.5, 3.5, and 4.5 of EPA CIO Handbook 2106-H-05 QAPP.

The QMP may note that some uses of environmental data may not require a QAPP, but the data 

can still be appropriately vetted for their use. For example, the use of data for public information 

products may not need the rigor of a QAPP, but would at least need some review to confirm their 

adequacy for use.
Handbook QMP 36 December 2012 Agency Review Draft

In contrast, a technical decision or report from a large field sampling activity would be expected 

to have complete and comprehensive quality documentation to support the results. If possible, 

the QMP can specify the general circumstances under which a QAPP for such products will or 

will not be needed. (See CIO Handbook 2106-H-05 QAPP for details.)

3.14.3 Related Types of Reviews

Contentious or significant activities may be subjected to review during the project proposal 

and/or planning and design stage. In some instances, it is also appropriate to conduct various 

reviews over the lifecycle of a project.

If the organization applies a peer review process to information products, the QMP can describe 

the general process to be used, including selecting peer reviewers, interacting with them, and 

using their review comments. For EPA organizations, the current edition of the EPA Peer 

Review Handbook [EPA 2006c] provides further information on this topic.

Non-EPA organizations may have their own peer review process which can be documented in 

the QMP as applicable. To maximize its value, peer review can begin as early in the planning

phase as possible.

For EPA organizations, disseminated information products, including those that contain 

environmental data or conclusions derived from environmental data, may be subject to the EPA 

IQG [EPA 2002a] requirements. As part of compliance with the EPA IQG, organizations can

incorporate the principles of objectivity, utility and integrity into their existing review 

procedures, as appropriate.

For additional information on project-level data review and evaluation of usability, users can

refer to Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Sections 

2.5, 3.5, and 4.5) [EPA 2013c].
Handbook QMP 37 December 2012 Agency Review Draft

REFERENCES

ANSI 2004 American National Standards Institute and American Society for Quality,

Quality Systems for Environmental Data and Technology Programs – Requirements with

Guidance for Use (ANSI/ASQ E4-2004.

http://asq.org/quality-press/display

EPA 1984 U.S. Environmental Protection Agency, EPA Order 5700 Change 6, Assistance

Administration Manual.

http://www.epa.gov/oarm

EPA 2000 U.S. Environmental Protection Agency, Guidance on Technical Audits and Related

Assessments for Environmental Data Operations (EPA QA/G-7).

http://www.epa.gov/quality/qs-docs/g7-final.pdf

EPA 2002a U.S. Environmental Protection Agency, Guidelines for Ensuring and Maximizing

the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental

Protection Agency.

http://www.epa.gov/quality/informationguidelines/documents/EPA_InfoQualityGuidelines.pdf

EPA 2002b U.S. Environmental Protection Agency, Guidance for Developing Quality

Systems for Environmental Programs (EPA QA/G-1)

http://www.epa.gov/quality

EPA 2003 U.S. Environmental Protection Agency, Guidance on Assessing Quality Systems

(EPA QA/G-3).

http://www.epa.gov/quality/qs-docs/g3-final.pdf

EPA 2005 U.S. Environmental Protection Agency, Uniform Federal Policy for Implementing 

Environmental Quality Systems (EPA-505-F-03-001).

http://www.epa.gov/fedfac/pdf/ufp_v2_final.pdf

EPA 2006a U.S. Environmental Protection Agency, Data Quality Assessment: A Reviewer’s

Guide (EPA QA/G-9R).

http://www.epa.gov/quality/qs-docs/g9r-final.pdf

EPA 2006b U.S. Environmental Protection Agency, Data Quality Assessment: Statistical Tools

for Practitioners (EPA QA/G-9S). 

http://www.epa.gov/quality/qs-docs/g9s-final.pdf

EPA 2006c U.S. Environmental Protection Agency, Peer Review Handbook 3

rd Edition.

http://www.epa.gov/peerreview/pdfs/Peer%20Review%20HandbookMay06.pdf

EPA 2007 U.S. Environmental Protection Agency, EPA QA/G-6, Guidance for Preparing

Standard Operating Procedures.

http://www.epa.gov/quality/qs-docs/g6-final.pdf
Handbook QMP 38 December 2012 Agency Review Draft

EPA 2008a U.S. Environmental Protection Agency, EPA Order 1610, Interagency Agreement

Policies, Procedures, and Guidance Manual.

http://www.epa.gov/oarm

EPA 2008b U.S. Environmental Protection Agency, Procedure for Quality Policy (CIO 

Procedure 2106-P-01).

http://www.epa.gov/irmpoli8/policies/2106p01.pdf

EPA 2008c U.S. Environmental Protection Agency, Quality Policy, (CIO Policy 2106).

http://www.epa.gov/irmpoli8/policies/21060.pdf

EPA 2009a U.S. Environmental Protection Agency, Contracts Management Manual.

http://www.epa.gov/oarm

EPA 2009b U.S. Environmental Protection Agency, EPA Records Management Policy.

(CIO Policy 2155.1).

http://www.epa.gov/records/policy/2155/rm_policy_cio_2155_1_2.pdf

EPA 2013a U.S. Environmental Protection Agency, Quality Standard for Environmental Data

Collection, Production, and Use by EPA Organizations. (CIO Standard 2106-S-01).

http://www.epa.gov/quality

EPA 2013b U.S. Environmental Protection Agency, Quality Standard for Environmental Data

Collection, Production, and Use by Non-EPA (External) Organizations.

(CIO Standard 2106-S-02).

http://www.epa.gov/quality

EPA 2013c U.S. Environmental Protection Agency, Handbook on Quality Assurance Project

Plans. (CIO Handbook 2106-H-05 QAPP).

http://www.epa.gov/quality

ISO 2002 International Organization for Standardization 19011, Guidelines for Quality and/or

Environmental Management Systems Auditing.

http://www.iso.org

ISO 2005 International Organization for Standardization 17025, General Requirements for the

Competence of Testing and Calibration Laboratories.

http://www.iso.org

ISO 2008 International Organization for Standardization 9001, Quality Management Systems –

Requirements.

http://www.iso.org
Handbook QMP 39 December 2012 Agency Review Draft

APPENDIX A

TERMS AND DEFINITIONS

NOTE: The following terms and definitions are taken from Quality Standard for Environmental 

Data Collection, Production, and Use by EPA Organizations (CIO Standard 2106-S-01.0) as 

well as Quality Standard for Environmental Data Collection, Production, and Use by Non-EPA 

(External) Organizations (CIO Standard 2106-S-02).

Assessment: The evaluation process used to measure the performance or effectiveness of a 

system and its elements. As used here, assessment is an all-inclusive term used to denote any of 

the following: audit, performance evaluation, management review, peer review, inspection, or 

surveillance. 

Competence: The ability to apply knowledge, skills, and experience to an activity to achieve 

intended results.

Conformity: The fulfillment of a requirement.

Corrective Action: An action to eliminate the cause of a nonconformity or undesirable situation 

and to prevent recurrence.

Data: A collection of facts and estimates from which conclusions may be drawn. 

Data Usability: The process of determining and ensuring that the quality of the data produced 

meets the intended use of the data.

Dissemination: Process of distributing information to the public that represents an official EPA 

endorsed opinion or decision. [Examples of information not considered a dissemination are 

information intended only for government employees; EPA responses to requests for Agency

records under the Freedom of Information Act (FOIA), the Privacy Act, The Federal Advisory

Committee Act (FACA) or other similar laws; correspondence directed to individuals or persons; 

ephemeral information; and distribution of information in documents filed in or prepared 

specifically for a judicial case or an administrative adjudication.] (Source: Section 5.3 & 5.4, 

EPA Information Quality Guidelines)

Document: Recorded information regardless of physical form or characteristics including 

individual records or items of non-record materials.

Environmental Data: Any data or information pertaining to the environment that describe

measured outputs from processes, environmental conditions in a specific location; ecological 

effects and consequences; health effects and consequences; biological, chemical, and 

radiological conditions; or the performance of environmental technology. For EPA, 

environmental data include information collected directly from measurements, produced from 

models, and compiled from other sources such as databases, information systems, literature, or 

the Internet.
Handbook QMP 40 December 2012 Agency Review Draft

Environmental Data Collection: Process of acquiring or gathering environmental data through 

various means including (but not limited to) sampling and analysis activities, retrieval from 

information systems and the literature, and receipt from EPA partners and the regulated 

community.

Environmental Data Operations: Work performed to collect, produce, use, or report 

environmental data.

Environmental Data Production: Process of generating environmental data through various 

means including (but not limited to) the use of measurement instrumentation, information 

technology, computer models, and data analysis tools (e.g., statistics, risk assessment methods).

Environmental Programs: Activities involving the environment, including but not limited to: 

characterization of environmental processes and conditions; environmental monitoring; 

environmental research and development; the design, construction, and operation of 

environmental technologies; and laboratory operations on environmental samples.

Environmental Technology: An all-inclusive term used to describe pollution monitoring, 

measurement and control devices and systems, waste treatment processes and storage facilities, 

and site remediation technologies and their components that may be utilized to remove pollutants 

or contaminants from or prevent them from entering the environment. 

Extramural Agreement: Legal agreement between EPA and a non-EPA organization for the 

acquisition of items or services by EPA or financial assistance to a non-EPA organization. Such 

agreements include acquisition agreements (e.g., contracts, work assignments, delivery orders, 

task orders), assistance agreements (e.g., cooperative agreements, research grants, state and local 

grants), and EPA-funded Interagency Agreements (IAs) with other governmental entities.

Graded Approach: The process of basing the level of application of managerial controls 

applied to an item or work according to the intended use of the results and the degree of 

confidence needed in the quality of the results.

Guidance: A non-mandatory compilation of advice, examples, best practices, or past experience.

Guidance may supplement procedures.

Handbook: A non-mandatory compilation of advice, examples, best practices, or past 

experiences, may be revised according to the issuing Office’s Peer Review Policy.

Information: For purposes of this Standard, information means any communication or 

representation of knowledge such as facts or data, in any medium or form, including, but not 

limited to, textual, numerical, graphic, cartographic, narrative, or audiovisual forms (OMB 

Information Quality Guidelines).

Information Product: Any book, paper, map, machine-readable material, audiovisual 

production, or other documentary material, regardless of physical form or characteristic (OMB 

Circular A-130).
Handbook QMP 41 December 2012 Agency Review Draft

Information Quality Guidelines (IQG): An Agency document that defines a basic standard of 

quality (including objectivity, utility, and integrity) for information products disseminated by

EPA. For influential information products, the basic standard of quality also includes 

reproducibility and transparency.

Information System: An organized collection, storage, and presentation system of data for 

decision making, progress reporting, and for planning and evaluation of programs. It can be 

either manual or computerized, or a combination of both.

Information Technology: The study, design, development, implementation, support, or 

management of computer-based information systems, particularly software applications and 

computer hardware.

Informative Annex: A part of a Standard that gives additional information as guidance which is 

intended to assist the understanding or use of the Standard. An informative annex is not 

considered to be part of the Standard when setting audit criteria.

Integrity (information): Assurance that the information is protected from unauthorized access 

or change and is not compromised through corruption or falsification.

Management Controls: A system of management functions to enable managers to (1) determine 

that the operations of a program or organization satisfy predetermined goals and objectives, and 

that performance is in line with standards and specifications; and (2) implement any remedial 

actions needed to ensure that human and other resources are being used in the most effective and 

efficient way possible in achieving the organization’s mission.

Management System: A system to establish policy and objectives and to achieve those 

objectives (ISO 9001). A management system may describe the policies, objectives, principles, 

authority, responsibilities, accountability, and implementation plan of an organization for 

conducting work and producing environmental data. Management systems include ISO 9001 on 

quality management, ISO 14001 on environmental management, and OHSAS 18000 on 

occupational health and safety.

Normative Annex: A part of a Standard that gives provisions (requirements) additional to those 

requirements in the body of the Standard. A normative annex is considered to a part of the 

Standard when setting audit criteria.

Organization: Company, corporation, firm, enterprise, or institution, or part thereof, whether 

incorporated or not, public or private, that has its own functions and administration. In the 

context of this Standard, an EPA organization may be an Office, Region, National Research 

Center or Laboratory, or a sub-unit such as a division, branch, section, or team.

Policy: A high-level statement about an Agency requirement designed to influence and 

determine decisions, actions, and other matters. It is usually driven by statute, executive order, 

the mandate of an oversight agency or Congress, or the head of the organization.

Prime Recipient: Organization that directly receives funding, support, or authorization to 

implement a project or program.
Handbook QMP 42 December 2012 Agency Review Draft

Procedure (CIO Procedure): The required steps, course of action, or processes needed to 

accomplish or satisfy a policy.

Process: A set of interrelated resources and activities which transforms inputs into outputs. 

Examples of processes include analysis, design, data collection, operation, fabrication, and 

calculation.

Product: The intended result or final output of an activity or process that is disseminated or 

distributed among EPA organizations or outside of EPA.

Programmatic QMP: a management plan that defines the structure and framework for 

planning, implementing, and evaluating quality for a specific policy, or program.

Quality: The totality of features and characteristics of a product or service that bear on its ability

to meet the stated or implied needs and expectations of the user.

Quality Assurance (QA): A management or oversight function that deals with setting policy

and running an administrative system of management controls that cover planning, 

implementation, review and maintenance to ensure environmental data are meeting their 

intended use.

Quality Assurance Manager (QA Manager): The individual designated as the principal 

manager within the organization having management oversight and responsibilities for planning, 

documenting, coordinating, and assessing the effectiveness of the QMS for the organization.

NOTE: Other personnel having QA or QC duties may be referred to as QA Officer and QA 

Coordinator.

Quality Assurance Project Plan (QAPP): A document describing in comprehensive detail the 

necessary QA, QC and other technical activities that must be implemented to ensure that the 

results of the work performed will satisfy the stated performance objectives and criteria.

Quality Control (QC): The overall system of technical activities that measure the attributes and 

performance of a process, item, or service against defined standards to verify that they meet the 

stated requirements established by the customer; operational techniques and activities that are 

used to fulfill requirements for quality.

Quality Management: That aspect of the overall quality management system of the 

organization that determines and implements the quality policy. Quality management typically 

includes strategic planning, allocation of resources, and other systematic activities (e.g., 

planning, implementation, and assessment) pertaining to the application of quality practices to 

the organization’s programs.

Quality Management Plan (QMP): A formal document or manual that describes the QMS in 

terms of the organizational structure, functional responsibilities of management and staff, lines of 

authority, and required interfaces for those planning, implementing and assessing all activities

conducted.
Handbook QMP 43 December 2012 Agency Review Draft

Quality Management System (QMS): A structured and documented management system 

describing the policies, objectives, principles, organizational authority, responsibilities, 

accountability, and implementation plan of an organization for ensuring quality in its work 

processes, products (items), and services. The quality management system provides the 

framework for planning, implementing, documenting, and assessing work performed by the 

organization and for carrying out required QA and QC activities.

Quality Program: The totality of management controls, processes, and documentation in EPA’s 

planning, implementation, and assessment of applying quality to the creation of Agency 

environmental data.

Record: A document stating results retrieved or providing evidence of activities (ISO 

9000:2005). NOTE: A federal record is an information resource in any format that is needed to 

describe Agency activities (44 U.S.C. § 3301).

Requirement: An expression of the content of a Standard that conveys criteria to be fulfilled is 

compliance is to be claimed and from which no deviation is permitted.

Service: A discrete function that performs one or more operations and returns a set of results to 

an external requester.

Standard: An accepted, consensus-based specification which defines systems, processes, 

methodologies, or practices. It provides a basis for assuring consistent and acceptable minimum 

levels quality, performance, safety, and reliability. Standards usually are included in or 

accompany procedures.

Standard Operating Procedure (SOP): A written document that details the method for an 

operation, analysis, or action with thoroughly prescribed techniques and steps, and that is 

officially approved as the method for performing certain routine or repetitive tasks.

Transparency (Information): Assurance that information is supported by sufficient metadata 

when disseminated.

Usability Assessment: Evaluation of data based upon the results of data validation and 

verification for the decision(s) being made. Reviewers assess whether the process execution and 

resulting data meet quality objectives based on the criteria given in the QAPP.

User: An organization, group, or individual that utilizes the results or products from 

environmental programs or the customer for whom the results or products were collected or 

created.

Utility (Information): Assurance that information is useful for its intended purpose.

Validation (Information): Confirmation by examination and provision of objective evidence 

that the particular requirement for which the information is intended are fulfilled; the process of 

determining whether the specifications were appropriate and that the verified results will meet 

the data user’s needs.
Handbook QMP 44 December 2012 Agency Review Draft

Verification (Information): Confirmation by examination and provision of objective evidence 

that validated information fulfills specified requirements; the process of checking whether the 

information met the project’s specifications.