Document ID: EPA-HQ-OEI-2012-0774-0004
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2012-12-26T05:00Z

Handbook for Preparing
Quality Management Plans

                                       

                              Agency Review Draft
                                December, 2012
                                       
                                       
                                       
                                       
This document is a review draft. This information is distributed solely for the purpose of pre-dissemination review under applicable information quality guidelines. It has not been formally disseminated by EPA. It does not represent and should not be construed to represent any Agency determination or policy. It is being circulated for review of its technical accuracy.
FOREWORDThe U.S. Environmental Protection Agency (EPA) uses Quality Management Plans (QMPs) to document how an organization will plan, implement, and assess the effectiveness of its Quality Management System (QMS) applied to environmental data collection, production, and use. The requirements and specifications are listed in Annex A of Quality Standard For Environmental Data Collection, Production, and Use By EPA Organizations (EPA CIO 2106-S-01) and Quality Standard For Environmental Data Collection, Production, and Use By Non-EPA (External) Organizations (EPA CIO 2106-S-02.0) (current versions). 
This document:
   * provides technical and policy advice; 
   * provides implementation information;
   * does not substitute for statutes EPA administers or their implementing regulations;
   * is not a regulation;
   * does not impose legally-binding requirements on EPA, states, or the regulated community;
   * is not mandatory.
EPA may: 
   * apply this handbook to any particular program or project to the extent appropriate in light of site-specific facts;
   * not apply this handbook to a particular situation based upon the specific circumstances; and
   * update, improve, revise, replace or withdraw this handbook at any time.

This document is one of several guidance and handbook documents from the U.S. Environmental Protection Agency, Office of Environmental Information (OEI) Quality Staff. EPA Quality Program documents may be downloaded from the Quality Home Page:
http://www.epa.gov/quality/qa_docs.html
In many instances, regional or program quality assurance personnel and project management personnel will be engaged for specific implementation information. These personnel are listed here:
http://www.epa.gov/quality/contacts.html
Questions regarding this document or other documents can be directed to the OEI Quality Staff:
U.S. EPA
OEI/Quality Staff (2811R) 
Washington, DC 20460
Phone: (202) 564-6830
FAX: (202) 565-2441
e-mail: quality@epa.govTABLE OF CONTENTS
CHAPTER 1  INTRODUCTION	1
1.1	Background and Purpose	1
1.2	Quality Management Plans in the EPA Quality Management System	1
1.3	Intended Audience	4
1.4	Period of Applicability	4
1.5	Supersession	4
CHAPTER 2  QUALITY MANAGEMENT PLAN PREPARATION,  SUBMISSION, REVIEW, AND APPROVAL	5
2.1	Quality Management Plan Preparation	5
2.2	Quality Management Plan Submissionand management Approval	5
2.3	Quality Management Plan Review and approval for use	6
2.4	Quality Management Plan Revisions	6
2.5	Combined Quality Management Plan with Quality Assurance Project Plan	8
CHAPTER 3  HANDBOOK ON QUALITY MANAGEMENT PLAN CONTENT	10
3.1	General Guidelines	10
3.1.1	Overview of QMP Content	11
3.1.2	Administrative Elements	11
3.1.3	Application of Graded Approach in QMPs	12
3.2	Quality Management Plan Conformance to Requirements	13
3.3	Organizational and Program Field of Application	14
3.4	Organization, Responsibility, and Authority	15
3.4.1	Organizational Issues	15
3.4.2	Roles and Responsibilities	16
3.4.3	Dispute Resolution	17
3.5	Quality Management System Description	17
3.6	Personnel Competence	19
3.7	External Agreements	20
3.7.1	EPA Organizations	20
3.7.2	Non-EPA Organizations	21
3.8	Documents and Records Management	22
3.9	Use of Information Technology Methods and Sources	23
3.10	Planning	25
3.10.1	The QMP as the Foundation of QMS Planning	25
3.10.2	Commitment to Project-Specific Planning	25
3.11	Implementation of Work	28
3.11.1	General Considerations	28
3.11.2	Documentation of Implementing Procedures	28
3.11.3	Sample and Data Integrity	29
3.11.4	Performance Monitoring	29
3.11.5	Deviations from Approved Processes	29
3.12	Assessment, Oversight, and Response	30
3.12.1	Types of Assessments	30
3.12.2	Assessment Planning	31
3.12.3	Assessment Conduct and Follow-Up	32
3.13 Quality Improvement	34
3.14	Data Review, Validation and Verification, and Data Usability Reporting	34
3.14.1	Planning for Data Review	34
3.14.2	Special Considerations for Data Review	35
3.14.3	Related Types of Reviews	36
REFERENCES	37
APPENDIX A  TERMS AND DEFINITIONS	39
LIST OF ACRONYMS

ANSI	American National Standards Institute
ASQ	American Society for Quality
CIO	Chief Information Officer
DQA	Data Quality Audit
DQA	Director of Quality Assurance
EAS	EPA Acquisition System
GIS	Geographic Information System
IA	Inter-agency Agreement
IGMS	Integrated Grants Management System
IQG	Information Quality Guidelines
ISO	International Organization for Standardization (not an acronym)
IT	Information Technology
OEI	Office of Environmental Information
PDR	Pre-Dissemination Review
PE	Performance Evaluation
QA	Quality Assurance
QA Manager	Quality Assurance Manager
QAPP	Quality Assurance Project Plan
QARF	Quality Assurance Review Form
QC	Quality Control
QMP	Quality Management Plan
QMS	Quality Management System
QSA	Quality System Audit
SIO	Senior Information Official
SOP	Standard Operating Procedure
TSA	Technical Systems Audit
UFP	Uniform Federal Policy
                                       
CHAPTER 1

INTRODUCTION
1.1	Background and Purpose
This section relates to Standard Clauses 1 Purpose, 2 Scope and Applicability, and Annex A, Introduction (Clause A1).
This handbook assists users with creation of Quality Management Plans (QMPs) or equivalent documents to satisfy the requirements of the EPA Quality Program as defined by the following two EPA Standards: 
         *       Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations, (CIO 2106-S-01), "Internal Standard" (EPA 2013a); and
         *       Quality Standard for Environmental Data Collection, Production, and Use by Non-EPA (External) Organizations, (CIO 2106-S-02), "External Standard" (EPA 2013b).
This handbook is not a standalone document, but intended to be used in conjunction with the Standards for the preparation of QMPs that satisfy EPA requirements. It is offered as a helpful resource for those who need to prepare QMPs that comply with these EPA Standards.
It can be used by both EPA organizations and by organizations conducting activities through external agreements with EPA. This handbook is intended to help its users prepare QMPs that summarize the overall management structure in which individual environmental data operations will take place and describes in general terms the quality tools that the organization will apply to individual projects and who is responsible for implementing them.
By definition, handbook is, not mandatory, and application is at the discretion of the user. This handbook offers guidelines, advice, and examples that will help users to develop QMPs that satisfy the requirements of the Standards.
The QA/R-2 document, EPA QA/R-2, Requirements for Quality Management Plans, has been withdrawn except for limited use under special circumstances. The QA/R-2 document is no longer used for developing QMPs except where permitted by the Standards.
1.2	Quality Management Plans in the EPA Quality Management System
This section relates to Standard Clause 7.2 Quality Management Plan, 7.5.2 Combined QMP-QAPP and Annex A Introduction (Clause A1).
The Standards identified in the previous section require EPA organizations (CIO Standard 2106-S-01) as well as non-EPA organizations operating under an external agreement with EPA (CIO Standard 2106-S-02) to develop, document, and implement a Quality Management System (QMS) that conforms to applicable EPA policies and procedures. A QMS is a structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for ensuring quality in activities involving environmental data.
The QMS provides the framework for planning, implementing, documenting, and assessing work performed by the organization and for carrying out quality assurance (QA) and quality control (QC) activities for environmental data operations and environmental technology performance activities.
A key step in the development and documentation of an organization's QMS is the creation of a Quality Management Plan (QMP). The QMP is unique to an organization and describes the management controls and technical processes and practices necessary to plan, implement, document, and assess the effectiveness of the organization's QMS. In general, a QMP provides a road map for carrying out the QMS, including its scope, the details and justification for any exclusion, a description of the QMS processes, and roles and responsibilities for QMS implementation. The QMP when sufficiently clear and thorough can serve as the basis for a Quality System Audit. That is, the QMP documents what quality activities are performed, what management controls are applied to the planning, implementation, and assessment of such activities, and who is responsible for the implementation and its management oversight. The QMP also documents the relationships among all parts of the organization for the application of quality to the organization's internal and external work. Such documentation enables the user to understand who does what and when.
Organizations (or a group of organizations) may also choose to develop and implement QMPs for large, complex programs involving environmental data collection, production, and use. These QMPs are typically referred to as "Programmatic QMPs" and the same criteria as for organization QMPs applies to them. The preparation and approval of "Programmatic QMPs" is discussed in Chapter 2.
The Standards state that QMPs may be prepared by the following organizations:
All EPA organizations involved with the production, collection, or use of environmental data, including Headquarters Offices, Program Offices, Regions, and National Research Laboratories and Centers. Their sub-organizations may have QMPs as well to document the details of the QMS applied to that sub-organization and its implementation.
All non-EPA organizations performing authorized activities that produce, collect, or use environmental data on behalf of EPA. These may include (but not be limited to):
States,
Tribes,
local governments,
regulated parties,
entities party to EPA settlement or consent agreements resulting from enforcement actions,
volunteer organizations,
contractors,
cooperative agreement holders,
grantees,
other federal governmental departments and agencies,
non-governmental organizations,
international governments/organizations, and
educational institutions.
Some organizations, including some EPA Program Offices and Regions, may elect to require their sub-organizations (e.g., subordinate offices, divisions within a Region) to develop individual QMPs. In some cases, an organization may develop additional QMPs at a sub-organizational unit (e.g., Divisions within a Region, Laboratories within an Office) in order to provide more implementation details. The lower-level QMPs are tiered to the organization-wide QMP and do not need to repeat the details contained in the latter document. Sub-organizational QMPs can be beneficial because they provide the flexibility to document any unique circumstances within the sub-organization that may have bearing on the QMS (e.g., the functions of a Laboratory Information Management System).
The Standards also require that these organizations prepare Quality Assurance Project Plans (QAPPs) or equivalent documents as appropriate, in order to describe and document explicit project details. QAPPs are an output of project planning and provide a road map to project implementation. They are distinct from QMPs in that they focus on particular data-related projects rather than on the organizational level; they are a project-specific view of QMS implementation. QAPPs describe in comprehensive detail the QA, QC and other technical activities that can be implemented to ensure that the results of the work performed will satisfy the stated performance objectives and criteria. They describe the activities of a project in the acquisition of environmental data or information from direct measurement activities, existing data, or generated by models.
The QMP can define and describe the overall structure and framework for planning, implementing, and evaluating environmental data operations, which in turn are described and documented by QAPPs for individual projects or tasks.
Figure 1 depicts the relationship between QMPs and QAPPs. More information on the preparation of QAPPs is available in Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP) [EPA 2013c]. As discussed in more detail in the QAPP guidance, QAPPs include those for individual projects or tasks, general or generic QAPPs (which may also be called QA Program Plans) that may be applied to multiple activities, and may include combined QMP-QAPPs for some specific activities that address elements of both the QMP and the QAPP.

Figure 1.	Functional Relationship of QMP and QAPPs to the QMS for an Organization
1.3	Intended Audience
This section relates to Standard Clause 2.1, Scope, and Annex A, Quality Management Plan Requirements (Clause A3).
This document is intended primarily for those responsible for writing, reviewing, approving, or using QMPs in accordance with EPA Standards. It is also useful for anyone who wants a detailed understanding of the role of QMPs in EPA's QMS, or more generally for those seeking an example of how a Quality Management System can conform to national or international consensus standards.
1.4	Period of Applicability
This section relates to Standard Clause 7.2 Quality Management plan, and Annex A Agency Review and Approval of the QMP (Clause A2.3).
This handbook is valid for five years from publication. Then it will be subject to review, resulting in one of the following:
revision,
reaffirmation without change, or
withdrawal.
If circumstances warrant, this handbook may also be revised during the course of the five-year period.
EPA QMPs developed in accordance with CIO Standard 2106-S-01 will typically have a valid approval for a period not to exceed five years (as defined in the Standard), though they can be revised at any time to reflect mission or scope change, reorganization, the emergence of new tools, or any other pertinent changes to the QMS.
QMPs prepared by external parties in accordance with CIO Standard 2106-S-02 may have an EPA approval valid for no more than five years for State, local, and Tribal governments or the length of the extramural agreement for all other extramural agreement holders. The period for which a Quality Management Plan is valid is defined in the Quality Management Plan of the EPA organization sponsoring the work. They also can be revised at any time to reflect changes in scope or other issues affecting the implementation of the QMS.
1.5	Supersession
This document, EPA Handbook for Quality Management Plans (CIO Handbook 2106-H-02 QMP) is the original issue of QMP handbook that conforms to the current editions of CIO Policy 2106 and CIO Procedure 2106-P-01 as well as CIO Standard 2106-S-01 and CIO Standard 2106-S-02.
CHAPTER 2

QUALITY MANAGEMENT PLAN PREPARATION, 
SUBMISSION, REVIEW, AND APPROVAL
2.1	Quality Management Plan Preparation
This section relates to Standard Clause 7.2 Quality Management Plan, and Annex A QMP Preparation Responsibility (Clause A2.1).
In EPA, the organization's senior management ensures the preparation, approval, and implementation of a QMP that covers all environmental programs for which the management is accountable. In practice, these activities are often delegated to other managers and staff.
For non-EPA organizations, the organization's senior management ensures the preparation of a QMP (or equivalent acceptable document content) that covers all work specified by the external agreement and for which the organization's management is accountable. (See Sections 2.4 and 3.3 of this handbook for examples of equivalent acceptable QMP content documents.)
In both cases, senior management includes the senior executive responsible and accountable for mission accomplishment and overall operations. Senior management ensures that the QMS documented in the QMP conforms to the requirements of the applicable Standard.
The same provisions apply to the preparation of "Programmatic QMPs." Management authority and quality-related responsibilities for the Program are clearly established before the QMP is prepared.
Managers and staff involved in preparing QMPs can use this document in conjunction with the applicable Standard.
2.2	Quality Management Plan Submissionand management Approval
This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A QMP Preparation Responsibility, QMP Submission and Approval, and Agency Review and Approval of the QMP (Clauses A2.1, A2.2, A2.3).
For EPA organizations, the QMP is approved internally and signed by the organization's senior manager as well as its Senior Information Official (SIO). The senior manager and SIO indicate their approval of the QMP on the signature page of the document. The senior managers are listed in the Standards. In most instances, QMPs below the Regional or Program Office level can be signed by designated representatives for the SIO. These delegations are normally documented at least in the QMP but may also be done through the official delegations system.
For a "Programmatic QMP," a responsible senior manager is designated as the signing official for the QMP.
The QMP is also approved and signed by the QA representative to the senior manager of the organization or Program. This is typically the Director of Quality Assurance (DQA) or Quality Assurance Manager (QA Manager) for the organization.
The senior manager may also require concurrence on the QMP by appropriate subordinate line managers to document the acceptance of implementation responsibilities in the QMS.
2.3	Quality Management Plan Review and approval for use
Following internal approval, an EPA QMP is submitted to the Office of Environmental Information (OEI) Quality Staff for Agency-level review and approval in accordance with CIO Standard 2106-S-01. Each QMP is reviewed to determine compliance with Agency-wide Quality Program requirements, including conformance to the Internal Standard. The review of the QMP focuses on the substance of the group's QA management process and not on format or technical details. QMPs that adequately address all program elements described in the Standard are typically approved for a period not to exceed five years. The next section describes situations that may cause a QMP to be revised.
For non-EPA organizations, the QMP is reviewed and approved by the organization's senior management as well as the designated QA representative of the senior management prior to its being submitted to EPA. The senior manager may require concurrence on the QMP by appropriate subordinate line managers to encourage the acceptance and implementation of the QMS.
Following organization approval, the QMP is submitted to the EPA organization responsible for the external agreement for review and approval in accordance with CIO Standard 2106-S-02. Each EPA organization defines its process for reviewing QMPs submitted under external agreements. This process includes procedures for verifying that the QMP is consistent with the EPA organization's goals and for notifying the organization of deficiencies to be addressed in the QMP prior to its acceptance by EPA. QMPs prepared by external parties may have an EPA approval valid for no more than five years for State, local, and Tribal governments or the length of the extramural agreement but not to exceed five years for all other extramural agreement holders. The period for which a Quality Management Plan is valid is defined in the Quality Management Plan of the EPA organization sponsoring the work.
2.4	Quality Management Plan Revisions
This section offers handbook on Standard Clause 7. 2 Quality Management Plan, and Annex A QMP Revisions (Clause A2.4).
For EPA organizations, although the QMP approval by the Agency may be valid for up to five years, all Agency QMSs are reviewed at least annually by their organizations (as required by CIO Procedure 2106-P-01). This review includes an evaluation of the effectiveness of the QMP and its implementation. The process of developing and annually updating the QMP provides an opportunity for management and staff to review and clarify roles and responsibilities, to address problem areas, and to acknowledge successes. Having an accurate and relevant QMP at all times is an essential part of every QMS and keeps technical staff informed about the quality program. Changes in quality policy and procedures are documented in a timely fashion by QMP revisions. In general, a copy of any EPA QMP revisions made during the year are encouraged to be submitted to the OEI Quality Staff as an attachment to, or as part of, the organization's QA Annual Report and Work Plan which is required by CIO Procedure 2106-P-01.
A revised QMP may be submitted to the Quality Staff for review at any time. Revisions occur under any of the following conditions:
expiration of the five-year approval duration of the QMP;
major structural change to the organization (e.g., addition or deletion of a division or branch) that requires Agency review;
significant change in the organization's mission, such as may occur in the reauthorization or revision of enabling environmental legislation, or reduction in resources (e.g., budget allocation, reduction in personnel) that may adversely impact the QMS;
results of audits or assessments and corresponding corrective actions impacting the QMS; or
any other significant change to the organization's QMS (such as the adoption of new quality tools or processes).
 
In general, the decision to revise a QMP is the responsibility of the organization. In some cases, a formal revision of the QMP may not be necessary, but it is helpful if the QMP describes how changes will be retained and documented until a formal revision is needed and how the changes will be disseminated to users.
As required by the Standards, all appropriate personnel performing work for the organization through external agreements (e.g., active contractors, assistance agreement holders) are advised of any changes that may affect their work for EPA so that they may be aware of and apply any new requirements. To enhance performance, the QMP can describe the process for such notifications and who is responsible.
For non-EPA organizations, a QMS review at least annually by the organization includes an evaluation of the effectiveness of the QMP. When necessary, the organization revises its QMP to incorporate changes to the QMS.
Similar to the conditions cited above requiring revision and resubmission for EPA QMPs, revisions to a non-EPA organization's QMP may become necessary due to:
a change in the scope (such as a modification to the contract statement of work) in the external agreement;
significant changes in the non-EPA organization's mission;
an organizational change within the non-EPA organization or its subcontractors that requires notification to EPA and other affected parties; or
any other significant change to the organization's QMS (such as the adoption of new quality tools or processes).
 
A revised QMP may be submitted at any time and is subject to the same review and approval processes as the original QMP. All personnel (or affected parties) performing work for the organization or program can be notified of significant changes to the QMS and the QMP as required by CIO Standard 2106-S-02.
HELP BOX  -  WHEN SHOULD A QMP BE REVISED?
Situations that might make it advisable to revise an existing QMP include but are not limited to:
change in reporting relationships among Quality Managers or other quality management personnel;
a corrective action that prompts the need to revise the existing QMS;
expansion of a laboratory facility to incorporate new categories of methods and analytes; and
significant decreases in budget allocations to the QMS.
On the other hand, the following situations may not lead to a formal QMP revision, but could be addressed through using a letter of notification to the approving organization:
personnel changes (e.g., a new Quality Manager);
a corrective action that facilitates implementation of the existing QMS; and
revisions to existing Standard Operating Procedures.
2.5	Combined Quality Management Plan with Quality Assurance Project Plan
This section relates to Standards Clause 7.5.2 Combined QMP-QAPP, and Annex B QAPP Preparation Responsibilities and Approvals (Clause B2.1).
In some circumstances, EPA may recommend the preparation of a combined QMP and QAPP as a more appropriate means to document organizational level and project-level environmental data activities, particularly when there is no available QMP for the organization performing the work. Examples may include (but are not limited to) research projects, inspection programs, and monitoring networks involving multiple participants, among others. In such cases, a combined QMP and QAPP can be a more appropriate means to document organizational-level and project-level environmental data activities, particularly when a full QMP is not needed. Combined QMP-QAPPs are not used for external agreements containing multiple projects that may be determined over the life of the agreement (e.g., mission contracts, program grants). A combined QMP-QAPP incorporates some elements of the QMP and some elements of the QAPP. In general, such documents are largely expanded QAPPs. As with any QMP, it can document the management controls that will be applied to the planning, implementation, and assessment of quality activities for the program or project, and can describe the processes to be followed to carry out those activities.
It is EPA's responsibility to ensure that all elements of a combined QMP-QAPP, as defined by the EPA organization, are addressed by the organization performing the work. The combined QMP-QAPP is subject to the same EPA review and approval as a QAPP.
As with any quality document produced by or for an EPA organization, the combined QMP-QAPP performs as part of the Quality Management System (QMS) of the EPA organization sponsoring the work and will be expected to align with that QMS.
Further information for a combined QMP-QAPP is provided in Handbook on Quality Assurance Project Plans (EPA CIO Handbook 2106-H-05 QAPP), Section 1.6.
HELP BOX  -  COMBINED QMP/QAPP
Some examples of situations in which a combined QMP/QAPP might be appropriate may include (but are not limited to):
   * routine inspection programs where inspections are conducted in the same manner for all visits and samples of opportunity can be expected (e.g., EPA lead and/or radon programs);
small organizations that do infrequent work with EPA funds (e.g., competitively awarded one-time grants, environmental education grants);
one-time mobilization to a previously unvisited field location, such as for monitoring assistance to a Tribal location;
necessary quick-turnaround data analysis that may involve new laboratory facilities, methods, or equipment;
environmental measurements related to technology performance, where QMS planning for the operation of the technology is already addressed by another document;
academic research activities of a one-time nature wherein a formal QMP is more detail than needed; or
a monitoring program conducted by a volunteer organization where the size and structure of the organization is limited.
CHAPTER 3

HANDBOOK ON QUALITY MANAGEMENT PLAN CONTENT
This introduction does not correlate directly to a clause in the Standards.
Annex A of CIO Standard 2106-S-01 and Standard 2106-S-02 list the elements that must be addressed in a QMP. However, the Standard also indicates that all elements may not apply to an organization or program. If that is the case, then the QMP provides explanation and rationale to document why the QMP element does not apply. (It is likely that this will occur when "Programmatic QMPs" are being developed.)
The QMP elements listed in the Standards are as follows:
general guidelines;
conformance with policies, procedures, and standards;
organizational and program field of application;
organization, responsibility, and authority;
QMS description;
personnel competence;
external agreements;
documents and records management;
use of information technology methods and sources;
planning;
implementation of work;
assessment, oversight, and response;
data review, validation and verification, and data usability reporting; and
quality improvement.
 
In the sections to follow, suggestions will be provided for each of the QMP elements. Since the text in the Standards is very detailed, the discussions here will refer to the Standards often. It will be beneficial for developers of QMPs to use this document in conjunction with the applicable Standard.
3.1	General Guidelines
This section relates to Standard Clauses 1, 2, 7. 2, Purpose, Scope and Applicability, Requirements, Quality Management Plan, and Annex A General Requirements (Clause A3).

3.1.1	Overview of QMP Content
This section relates to Standard Clauses 7..2, Quality Management Plan, and Annex A General Requirements (Clause A3.1).
A QMP addresses the requirements of CIO Standard 2106-S-01 (for EPA organizations) or CIO Standard 2106-S-02 (for non-EPA external organizations). It contains descriptions of the processes by which the organization plans, implements, documents, and determines the effectiveness of its quality management practices, including QA and QC activities, to help management to obtain results of its technical work that are of the type and quality needed for their intended use. The QMP also reflects the organization's commitment to quality management principles and practices.
Including discussion of those activities, policies, and procedures that are common to all projects can reduce duplication between QMPs and QAPPs. Such discussions in the QMP provide an "umbrella" under which individual project activities may be performed, and there is then no need to repeat them in each of the organization's QAPPs.
Documents with a different title, such as a Quality Manual or a Program Management Plan, if they fulfill the purpose of a QMP and contain the content as detailed in this chapter are acceptable.
For EPA organizations, the QMP can be prepared in accordance with the Agency's Open Government Plan, reflecting the principles of transparency, participation, and collaboration.
3.1.2	Administrative Elements
This section relates to Standard Clauses 7.2, Quality Management Plan, and Annex A General Requirements, and Organization, Responsibility, and Authority (Clauses A3.1 and 3.4).
The format of a QMP may be tailored by the organization, but in every case the QMP includes the following administrative elements:
title page;
table of contents;
approvals page with appropriate signatures;
revision history; and
applicable references.
A document control format may be helpful to organize the QMP and facilitate future revisions. The completed QMP can reflect that it has received the approvals described in Section 2.2. Figure 3 is an excerpt of a hypothetical QMP approval page incorporating document control format.
                                                                               
                                                                 DCN: QMP-01-00
                                                                    Revision: 0
                                                                Date: July 2010
                                                                    Page 2 of 2

                                 APPROVAL PAGE
Deputy Assistant Administrator:
                                       
                                       
                                       
                                 Printed Name
                                   Signature
                                     Date

Director of QA:
                                       
                                       
                                       
                                 Printed Name
                                   Signature
                                     Date

Figure 3.	Example of Document Control Format

3.1.3	Application of Graded Approach in QMPs
This section relates to Standard Clause 7.2 Quality Management Plan.
The QMP may incorporate, where applicable, the concept of the graded approach, which is the process of applying quality and management controls to an item or work activity according to the intended use and degree of confidence needed in the results. For instance, the QMP may state that more complex or significant projects carried out by the organization will have a more rigorous QAPP, whereas simpler or less significant projects may have a streamlined QAPP. It may also embody the graded approach by specifying that required approvals and reviews can be delegated under defined circumstances. It is important to remember that the graded approach is not an opportunity to "cut corners" by bypassing critical management controls. All EPA QMPs can certainly include the graded approach, but non-EPA organizations may not need it.
The application of graded approach to specific work is more commonly found at the project level. A more detailed discussion of the concept and its application to projects can be found in Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Section 1.3) [EPA 2013c]. For both the QMP and the QAPP, the level of detail can be commensurate with the stated goals of the organization or project and the intended use of the results of the environmental data collection or production.
HELP BOX  -  GRADED APPROACH EXAMPLES
The application of the graded approach concept to the operation of a QMS will vary from one organization to another. Here are a few examples of how the graded approach may be reflected in a QMP:
   * a categorization scheme for ranking projects conducted by the organization by their significance and complexity, with higher-ranking projects receiving more thorough QA planning and review;
   * establishment of a cost threshold for external agreements, with agreements above the threshold being subject to on-site audits;
   * streamlined QAPP review and approval for short-duration projects;
   * records retention schedules tiered to the long-term importance of a given project; and
   * designation of criteria for determining which of an organization's projects will be subject to formal peer review.

3.2	Quality Management Plan Conformance to Requirements
This section relates to Standard Clause 6 Related Regulations, Policies, and Procedures, and Annex A Conformance with Policies, Procedures, and Standards (Clause A3.2).
For EPA organizations, a QMP can cite the external and internal policies, procedures, standards, and guidelines to which the QMP conforms. This helps users to understand the scope and extent of the QMS more clearly. These may include the documents listed in Figure 1 previously. In addition, see Section 6 of CIO Standard 2106-S-01 for a list of other potentially applicable guidelines. In addition, Clause 1 of the same Standard states that Informative Annexes (or parts thereof) may become requirements if so decided by the organization. Documenting this commitment in the organization's QMP makes it clear when and to whom the organization expects it to apply.
Non-EPA organizations may be required to demonstrate and document that the QMP conforms to CIO Standard 2106-S-02 as part of a procurement solicitation, assistance negotiation, or other pre-award condition, and for any relevant terms of an applicable external agreement. In addition, the QMP can cite any other requirements that are pertinent to the organization. Examples of other standards and policy documents include (but are not limited to):
ISO 9001 (Current Edition), Quality Management Systems  -  Requirements [ISO 2008]
Uniform Federal Policy for Implementing Environmental Quality Systems (EPA-505-F-03-001) (Current Edition) [EPA 2005]
These sources could be used to produce quality management documents that are equivalent to an EPA QMP. For example, under ISO 9001 the equivalent quality management document is usually called the "quality manual" but other titles may be used.
Conceptually, these documents will address the same general management controls as well as roles and responsibilities that are expected in an EPA QMP. In addition, the use of the Uniform Federal Policy (UFP) format for QAPPs has been accepted by several EPA organizations as equivalent to the requirements given in the CIO Standard 2106-S-01 and CIO Standard 2106-S-02. The UFP reference given above describes the Uniform Federal Policy for the creation of a QMS and related quality management documents. It is analogous to CIO Standard 2106-S-02 and the QMP requirements in the Standard. Users can confirm acceptability of this format with the accepting EPA organization before using it to prepare QMPs or QAPPs.State that a QMP addresses specific organization(s and business line(s). "umbrella" document.

HELP BOX  -  WHAT IS CONFORMANCE?
Conformance (also referred to as conformity) means fulfillment of specific requirements. In particular, a QMP conforms to a Standard or similar document if it addresses all applicable requirements provided in the Standard.
Conformance with a Standard is distinct from voluntary use of helpful suggestions found in a Standard. For instance, QMPs created per EPA CIO Standard 2106-S-01 will conform to ANSI/ASQ E4, but they may also use elements of ISO 9001 without the need to conform fully to the ISO standard.
Conformance is also distinct from compliance, which is a legal term for meeting the requirements of specific laws and regulations.
3.3	Organizational and Program Field of Application
This section relates to Standard Clauses 2.1 and 2.2, Scope and Applicability, and Annex A Organizational and Program Field of Application (Clause A3.3).
QMPs prepared using this handbook can focus on the programs and processes necessary to assure quality in environmental data collection, production, and use. EPA Regions may have Division- and Branch-level QMPs for decentralized Quality Management Systems. Such QMPs may be approved at the Regional level (i.e., not by the OEI Quality Staff). Likewise, states may have Office or Division QMPs under their parent state QMP (or independent QMS) that are not reviewed or approved by EPA when they are outside the scope of the external agreement with EPA.
The QMP can clearly identify the organizations or programs and processes to which it applies.
For EPA organizations, the QMP can address quality management (i.e., management controls, roles and responsibilities of managers and staff, quality processes to be applied, etc.) for a specific organizational unit, such as a Region, Headquarters Program Office, or Office of Research and Development Laboratory or Center, or a subordinate organization within those units. The same criteria can also apply to any "Programmatic QMPs" or sub-organization QMPs developed by the organization.
    
    
For non-EPA organizations, the QMP can address quality management for the organization that has an external agreement with EPA. As indicated above, the same criteria can also apply to any "Programmatic QMPs" or sub-organization QMPs developed by the organization. Examples of sub-organizations are listed in Section 1.2.
3.4	Organization, Responsibility, and Authority
This section relates to Standard Clause 7, Requirements, and Annex A Organization, Responsibility, and Authority (Clause A3.4).
3.4.1	Organizational Issues
This section relates to Standard Clause 7 Requirements, and Annex A Organization, Responsibility, and Authority (Clause A3.4).
The QMP can present the organization's Quality Policy, which is a summary statement of the organization's commitment to quality in its environmental data operations. This Policy can vary in length and degree of specificity. Minimally, the Quality Policy can reflect:
senior management's commitment to and engagement with the QMS;
accountability for quality at all management and staff levels; and
a commitment to the allocation of adequate resources to the QMS.
 
Leadership on the part of senior management is crucial to the success of the QMS. The QMP can describe the specific steps management takes to ensure that the QMS is understood and implemented throughout the organization's environmental data functions. These steps could include:
serving as a vocal champion of the QMS;
ensuring that quality management positions are filled by qualified staff who are supported by their management chain;
actively participating in specific QMS functions such as training and assessments; and
building quality-related commitments into the organization's employee performance appraisal system.
 
It is useful to include in the QMP an organizational chart depicting the main components of the organization, and highlighting the locations and reporting relationships of all staff with specific QMS responsibilities. The hypothetical EPA organizational chart in Figure 2 presents an example of how QMS-related relationships can be displayed.

Figure 2.	Hypothetical organizational chart depicting QA relationships
If sub-unit QMPs are tiered under an organization-wide QMP, they can be listed in the main organization's QMP, and the relationships between the documents can be defined, along with a description of the review and approval procedures for sub-unit QMPs.
3.4.2	Roles and Responsibilities
This section relates to Standard Clause 8 Roles and Responsibilities, and Annex A Organization, Responsibility, and Authority (Clause A3.4).
The QMP can include discussion of all roles and responsibilities applicable to the QMS, such as:
senior management;
line management/supervisors;
quality management personnel (Director of Quality, Quality Managers, etc.);
project managers;
technical staff; and
administrative staff.

It is particularly important that the roles and responsibilities of specifically designated quality management positions (such as Quality Manager, Quality Assurance Coordinator, or Quality Assurance Officer) can have detailed explanation in the QMP. For example, representatives authorized to sign QAPPs and/or review technical reports on behalf of the QA Manager can be defined in the QMP.

Critical to the success of QMS implementation is a full description of and commitment to how quality management personnel are permitted functional independence from line management, as well as direct access to senior management as necessary to address quality issues. This functional independence can be documented in the QMP and delineated to ensure control of organizational risks that can arise through conflict of interest.
3.4.3	Dispute Resolution
This section relates to Standard Clause 7.4 Quality System Assessments, and Annex A Organization, Responsibility, and Authority (Clause A3.4).
The QMP can describe the process for resolution of any quality-related disputes that are not resolved through normal channels and can document those responsible for its implementation. This process can be designed, in general, to ensure that differences in opinion regarding quality system implementation are:
fully documented by the concerned parties;
resolved at the lowest level of management, if possible; 
efficiently brought to the attention of senior management if required to facilitate resolution; and
subsequently tracked to ensure that the agreed-upon resolution is accomplished in practice and is documented appropriately.

Dispute resolution is an important process and it can be carefully designed. All managers and staff who could be affected by or participate in such a process can agree to conform to the process. An example of a quality-related dispute would be a disagreement between a line manager and a quality manager regarding corrective actions that are necessary to resolve an audit finding of non-conformance. The dispute resolution process can ensure that disputes can be addressed and resolved with due process and in a timely manner so that work schedule and performance are not impeded.
State that an effective QMS flows from the top  -  clear policies and organizational authorities are vital. 
3.5	Quality Management System DescriptionThis section relates to Standard Clause7 Requirements, and Annex A QMS Description (Clause A3.5).
The principal purpose of the QMP is to describe the QMS that is being implemented by an organization or program and to establish the roles and responsibilities for managers and staff for its implementation.
Accordingly, the QMP can summarize the content of the QMS and describe how it is structured. The QMS "big picture" can be briefly described up front, with the remainder of the QMP then providing details of the QMS.
The document Guidance for Developing a Quality System for Environmental Programs (EPA QA/G-1) [EPA 2002b] provides guidance on developing a QMS that conforms to the requirements of the Quality Standards. In addition to this document, users may find it helpful to examine the following documents for their discussions of QMS elements:
ANSI/ASQ E4, Quality Systems for Environmental Data and Technology Programs  -  Requirements with Guidance for Use. [ANSI 2004]
ISO 9001, Quality management systems  -  Requirements. [ISO 2008]
In general, QMS in the EPA-wide Quality Program embody a Plan-Do-Check-Act quality model. A QMS consists of the quality management controls and processes needed for planning, implementing, assessing, and improving the effectiveness of quality processes and practices applied to the collection, production, and use of environmental data. Management controls are a system of management functions to enable managers to (1) determine that the operations of a program or organization satisfy predetermined goals and objectives and that performance is in line with standards and specifications; and (2) implement any remedial actions needed to ensure that human and other resources are being used in the most effective and efficient way possible in achieving the organization's mission.
HELP BOX  -  QUESTIONS FOR QMP DEVELOPERS
Developers of QMPs for environmental data collection, production, and use should be able to answer questions such as the following:
Who are the principal stakeholders for our environmental data operations (they may include other organizational units, other levels of government, members of the public, etc.)?
What is our process for identifying, reconciling, and prioritizing these stakeholders' requirements and expectations?
What specific environmental data will be collected, produced, or used to meet these requirements and expectations?
What processes and management controls will be carried out?
What roles and responsibilities will be assigned for the execution of these processes and management controls?
How will we evaluate the effectiveness of our processes and their outputs on an ongoing basis?
How will we know the extent to which stakeholders' requirements and expectations are being met?
3.6	Personnel Competence
This section relates to Standard Clause .6 Personnel Competence, and Annex A Personnel Competence (Clause A3.6).
This part of the QMP might begin with the organization's policy statement on ensuring the competence of personnel to perform the technical aspects of project work as well as the QA and QC functions that are designed to meet the organization's project and mission objectives. It would also be beneficial to describe management responsibility for support and/or commitment to ongoing training in technical fields both currently relevant as well as in advance of initiatives. It is important that the personnel responsible for the implementation of the QMS elements possess sufficient knowledge, experience, and understanding of quality principles, processes, and practices.
Personnel competence is generally composed of the requisite technical knowledge, relevant education, work experience, and specialized training (as required). Personnel competence can be maintained and demonstrated through means such as the following or in combination thereof:
Training: Technical and management training, which may include classroom, electronic, and on-the-job, can be provided by the organization as needed.
Certification: Statutory, regulatory, or professional certifications may be required to perform certain operations (e.g., laboratory analyst competency).
Observation: Formal supervisory observation may be required before a staff member is authorized for independent performance of an activity such as operating measurement and testing equipment.
 
The processes for carrying out these or other measures to ensure personnel competence can be documented in the QMP. In addition, the QMP can describe how objective evidence of personnel job proficiency will be documented and maintained for the duration of the project or activity affected, or longer if required by statute or policy. Accurate and up-to-date records on personnel competence are valuable for numerous QMS-related purposes, such as QAPP and SOP development, audit preparation, and design of corrective actions. Such records provide management with documentation that staff members are competent to perform the work assigned. Workers without adequate competence pose a risk to the successful completion of the work and possibly to the organization's liability in the case of an accident.
Finally, the QMP can describe how and when any need to implement new or revised training, implement certification of personnel, or define additional personnel qualifications will be reevaluated by the organization. This is advisable because the organization's evolving mission may lead to new training requirements.
3.7	External Agreements
This section relates to Standard 01 Clause 7. 11 Management of Quality Requirements for External Agreements, Standard 02 Clause 7.11 Management of Quality Requirements for Subagreements, and Annex A External Agreements (Clause A3.7).
3.7.1	EPA Organizations
Extramural agreements provide EPA with a critical means of accomplishing work related to the Agency's mission. Accordingly, it is very important for the organization to ensure that appropriate quality requirements are applied to its suppliers, subcontractors, and other external agreement holders. This is vital to the success of the QMS, because many projects involving environmental data collection, production, and use may utilize inputs from outside the primary organization. These may be focused contributions, such as those of testing laboratories; or in other cases, a sub-recipient may be assigned overall responsibility for a major activity such as a large, complex field sampling activity.
For EPA organizations, external agreements may include:
acquisition of services and items through contracts, task orders, and other acquisition agreements;
assistance agreements such as grants and cooperative agreements;
interagency agreements negotiated with other Federal, State, Tribal, or local governments;
international agreements negotiated with governmental, non-governmental organizations, or other entities;
settlements and agreements (e.g., administrative orders, consent agreements) resulting from enforcement actions;
sub-awards whose recipients are accountable to the prime recipient of an external agreement for environmental data activities provided under the agreement; and
voluntary partner programs (e.g., voluntary monitoring).
 
The QMP can describe how the organization will:
determine whether the scope of a proposed or implemented external agreement includes the collection, production, or use of environmental data; and
ensure that appropriate quality practices and procedures are defined and applied to external agreements as appropriate.

An EPA organization can identify, develop, and document the processes and procedures (including management and staff responsibilities) for ensuring that all external agreement holders apply appropriate quality requirements. EPA organizations can identify and document how recipients of external agreements require their own suppliers (e.g., contractors, subcontractors, or financial assistance recipients) to maintain a quality system consistent with EPA requirements. This provision applies only to those suppliers providing services or items directly affecting the quality of results or products from environmental data activities.
EPA organizations have considerable flexibility in requiring documentation on the scope of quality practices in external agreements for environmental data. For example, under acquisitions, an EPA organization can provide in the QMP a summary of the organization's process for implementing the EPA Contracts Management Manual [EPA 2009a] specification for using a QA Review Form (QARF). In addition, the QMP may describe how the QARF is integrated in the EPA Acquisition System (EAS).
Similarly, the QMP may include a description of the process for assistance agreements which is documented through the Integrated Grants Management System (IGMS). Moreover, the QMP may describe the processes for Inter-Agency Agreements (IA) and other assistance agreements. For further information, EPA organizations may refer to EPA Order 5700 Change 6, Assistance Administration Manual [EPA 1984], and to EPA Order 1610, Interagency Agreement Policies, Procedures, and Guidance Manual [EPA 2008a].
Each EPA organization's Quality Manager (or designee) can be authorized to review documentation for pending external actions (e.g., Statements of Work and contractor proposals) to ensure that appropriate quality-related requirements are being addressed and incorporated. This authority can be clearly defined in the QMP. Moreover, for projects involving multiple EPA organizations with shared responsibilities, the QMP can describe how the lead responsibility for QA will be determined and assigned. In the case of IAs, when EPA provides the funding or a substantive majority of resources expended, the application of QA and oversight responsibility remaining with EPA is negotiated into the agreement. When EPA does work for another agency or department, the EPA organization's quality management system and QA oversight is applied, using a graded approach, by default unless otherwise negotiated in the agreement.
3.7.2	Non-EPA Organizations
Non-EPA organizations that have external agreements with EPA can discuss how the quality requirements in their QMS will flow down to their suppliers and sub-recipients associated with environmental data operations. However, any requirements on sub-recipients (e.g., subcontractors) are made through the prime recipient (e.g., contractor), as there is no direct contractual relationship between EPA and external agreement sub-recipients.
The QMP can list the functions that are likely to be implemented by suppliers or sub-recipients (e.g., field sampling services, laboratory testing, and information technology support). For each of those areas, the QMP would then describe how the QMS will be applied to sub-recipients. For example, non-EPA organizations may:
require lower-tier sub-recipients to prepare their own QMPs or QAPPs;
review procurement documents for sub-recipients;
conduct audits or on-site inspections of supplier facilities and operations;
conduct validation of data supplied by contract laboratories;
maintain acceptance criteria for lower-tier sub-recipients; or
issue stop-work orders or requests for corrective actions to sub-recipients.
    
HELP BOX  -  EXAMPLES OF QUALITY REQUIREMENTS FOR SUPPLIERS
Suppliers of services and indirectly through the suppliers, their sub-recipients involved in environmental data collection, production, and use may be required to:
develop QMPs describing their quality system;
develop QAPPs describing and documenting explicit project details;
develop SOPs for certain processes and practices;
commit to fulfilling specific QA and QC requirements established by the sponsoring organization;
submit to audits to confirm their capability to satisfy quality criteria (e.g., performance evaluations, technical systems audits);
obtain review and approval from the sponsoring organization for any mid-course changes;
demonstrate that all QA and QC requirements have been carried out in practice;
provide objective evidence of quality in products and services provided by their own suppliers and sub-recipients, including source selection, source inspections, supplier audits, and examination of deliverables; and
review all deliverables against the acceptance criteria established by the sponsoring organization.
3.8	Documents and Records Management
This section relates to Standard Clause 7.10 Documents and Records Management, and Annex A Documents and Records Management (Clause A3.8).
An organization's process for management of quality-related documents and records is often crucial to the success of a QMS, which is ultimately not just about achieving quality performance, but also about demonstrating (that is, documenting) that quality performance has been achieved. Accordingly, it is very important the documents and records affecting the quality of the organization's environmental data operations be identified and documented in the QMP.
In general, the QMP can:
describe the maintenance and management of major quality planning documents, including the organization's QMP and QAPPs;
define other documents that will be considered quality-related and managed accordingly;
describe characteristics (e.g. of function or content) to be used in identifying quality-related documents not anticipated by or named in the QMP;
document how all technical guidance documents are prepared, reviewed, approved, issued, used, and revised (this can apply to internally developed and externally acquired documents);
define what records will be generated and how they will be maintained;
describe how the organization will ensure that documents and records accurately reflect completed work;
discuss retention times, archiving, retrieval, and disposition of quality-related documents and records;
describe processes for handling documents and records to assure their accessibility, protection from damage and deterioration, and means of retention, including discussion of the roles and responsibilities for management and staff;
document chain of custody and confidentiality procedures for evidentiary records; and
commit to compliance with all statutory, contractual, and assistance agreement requirements for records from environmental programs, including providing adequate preservation of key records necessary to support the mission of the organization.
EPA organizations are required by CIO Standard 2106-S-01 to comply with Agency policy on documents and records. The QMP developers can refer to the provisions of Section 6 of the EPA Records Management Policy [EPA 2009b] to define this process for their QMP. This section lists the minimum requirements for an EPA records management program, including creation, management, maintenance, protection, and disposal of records. In addition, QMP developers can document any additional organization-specific processes for management of documents and records as described above.
For non-EPA organizations, the QMP can document or refer to the organization's own records management policy and procedures and to the applicable terms and conditions of the external agreement. In general, Federal regulations governing external agreements will specify documents and records management requirements. Adherence to such requirements can be affirmed in the organization's QMP. Any differences for specific projects can be documented in the QAPP; see Handbook on Quality Assurance Project Plans (EPA CIO Handbook 2106-H-05 QAPP) for more information.
3.9	Use of Information Technology Methods and Sources
This section relates to Standard Annex A Use of Information Technology Methods and Sources (Clause A3.9).
Especially with recent advances, information technology (IT) plays an increasingly important role in the collection, production, and use of environmental data. As noted in CIO Standards 2106-S-01 and 2106-S-02, IT encompasses a wide range of activities, functions, and sources for environmental data collection, production, and use, including general computer software programs, computer models, information systems, and web-based applications.

The QMP can present the organization's general approach to applying QA and QC processes and practices to applications involving information systems management. Activities addressed by the QMP can include consideration of acquired results from IT methods and sources, such as services that collect and provide access to data and information.
Organizations can develop and document the general processes, procedures, and personnel responsibilities for determining when QA planning documents (e.g., QAPPs) are needed and for evaluating the suitability of environmental data obtained from information technology sources.
The QMP can describe and document the processes related to the preparation, review, approval, and use of QA planning documents for all IT-related applications. Developers of QMPs may find the discussions in the Standards and in their Annexes A and B to be useful in defining the organization's QMP content for IT activities.
Environmental data products generated using IT applications and sources include (but may not be limited to) the following:
Computer mathematical models and other electronic methods: Environmental data produced from computer programs and models, mathematical models, and other electronic methods and sources [e.g., Geographic Information Systems (GIS)] can be reviewed, evaluated, verified, and validated in order to confirm their suitability for use. For more information, see the Web site of EPA's Council for Regulatory Environmental Modeling (http://www.epa.gov/crem/), which is intended to promote consistency and consensus among environmental model developers and users.
EPA-owned information systems: Quality principles can be applied to the development, modification, operation, and management of EPA information systems to ensure that the system is designed to satisfy the intended uses and user requirements and that data maintained in the system can be audited and verified from entry to retrieval. Examples of EPA information systems include CERCLIS (Superfund), TRI (toxics), and SDWIS (drinking water); see http://www.epa.gov/enviro/index.html which is a centralized repository of these systems.
Other information systems and the Internet: Environmental data collected from other information sources (including the Internet) can receive special attention to ensure that the data can be used for their intended purpose. The QMP can commit to the development and documentation of project-specific evaluation criteria in applicable QAPPs.
Computer software and hardware: Computer software and hardware relevant to the mission and objectives of the organization can be controlled appropriately to satisfy the organization's objectives. QMP developers can refer to applicable OEI procedures and guidance for information on software and hardware controls; these resources are compiled at http://www.epa.gov/irmpoli8/policies/index.html.
Emphasize the importance of information technology to the collection, production, and use of environmental data. 
For EPA organizations, the QMP can discuss the organization's commitment to complying with all applicable CIO policies, procedures, standards, and guidelines, as well as organization-specific policies and procedures. It can describe the process for ensuring that data and information produced by or on behalf of the Agency meet applicable requirements and standards.For non-EPA organizations, the QMP can describe organization-specific systems, policies, and approaches. If the applicable external agreement provides for compliance with specific EPA CIO polices, the QMP can also reflect that commitment.
In addition, a QMP for either an EPA or an external organization can discuss any particular information systems issues that could be applicable across the range of environmental data operations covered in the QMP. Examples may include (but are not limited to):
field data logging processes;
Laboratory Information Management Systems;
modeling or other uses of data;
electronic data bases;
computer programs used to analyze environmental data; and
systems for posting and sharing data (such as the Environmental Information Exchange Network).
Project-specific processes, procedures, and evaluation criteria when developed are documented in the corresponding QAPPs.
3.10	Planning
This section relates to Standard Clause: Systematic Planning (7.7), and Annex A Planning (Clause A3.10).
3.10.1	The QMP as the Foundation of QMS Planning
This section relates to Standard Clause: Systematic Planning (7.7), and Annex A Planning (Clause A3.10).
The QMP can describe how the organization satisfies the planning requirements of the QMS. Planning is generally regarded as the most important phase of a quality system; in the absence of a systematic focus on planning along with an appropriate set of planning tools, a QMS can devolve into a mere inspection program for detecting flawed environmental data.
The foundation of the planning approach built into the QMS is provided by the QMP itself. The specific QMP elements described in this handbook beginning with Section 3.7 (ensuring personnel competence, promoting quality performance in external agreements, and so on) constitute the building blocks of the organization's overall plan for ensuring effective delivery of environmental data that meet the needs of the organization's stakeholders and customers.
3.10.2	Commitment to Project-Specific Planning
This section relates to Standard Clause, Systematic Planning (7.7), and Annex A Planning (Clause A3.10).

In addition to this overall planning framework, the QMP can make a specific commitment to the use of project-level systematic planning for any activities involving the collection, production, and use of environmental data. Systematic planning for a project entails the following aspects at a minimum:
defining the project, or creating an overall framework upon which the project will be developed;
establishing the scope of the investigation, including its temporal, physical, and logistical boundaries;
documenting the needs or demands of the project, which may include conceptual models or sampling and analytical methods and procedures; and
defining protocols for tracking and reporting on project progress.
 
The particulars of the systematic planning approach depend on the focus of the project  -  whether it involves sampling and analysis, data analysis, a modeling application, or another type of environmental data operation. But in all cases, the organization can establish and document in the QMP a systematic planning process that:
identifies all relevant customers, stakeholders, and interested parties (as applicable), as well as their needs and expectations, for the results of the work to be performed;
identifies any internal or external resource, equipment, or technical expertise necessary for successful performance;
identifies the technical and quality goals that meet the needs and expectations of the customer;
translates the technical and quality goals into specifications that will produce the desired result;
considers any cost and schedule constraints within which project activities are required to be performed;
specifies the QA and QC activities required for tracking project performance; and
identifies measurable acceptance criteria or performance measures by which the results will be evaluated and the suitability of the environmental data for their intended use will be determined.
The general planning processes (including applicable design steps) can be described. All planning documentation can be reviewed and approved for implementation by authorized personnel before the affected planned work commences. Such documentation includes, but is not limited to, work plans, schedules, standard operating procedures (SOPs), and QAPPs.
The QMP can describe in specific terms the organization's process for developing, reviewing, approving, implementing, and revising QAPPs. Detailed information is available in Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Section 1.7) [EPA 2013c].
The QMP can also document the process for ensuring that all versions of planning documents, procedures, and related documents will be retained in accordance with EPA policy and procedures for documents and records, and in accordance with extramural agreement regulations.
HELP BOX -- OVERVIEW OF SYSTEMATIC PLANNING ELEMENTS
The particulars of the systematic planning approach depend on the focus of a given project, but the QMP should describe the organization's overall approach to addressing and documenting the following elements:
Organization: Identification and involvement of the project manager, sponsoring organization and responsible official, project personnel, stakeholders, scientific experts, customers, suppliers, and Quality Manager.
Schedule: Identification of project schedule, resources (including budget), milestones, regulatory or contractual requirements, and circumstances that could adversely affect the progress of the project.
Project Goal: Description of the project goals, objectives, study questions and conceptual model that describe the environmental issue or concern, contamination pathways, and general impact on human health.
Quality Assurance: Specification of needed quality assurance and quality control activities to ensure performance criteria (for new data) or acceptance criteria (for existing data) are satisfied. 
Data Needs: Identification of the type of data needed, how the data will be used to support the project's objectives, and identification of potential limitations in data interpretation.
Criteria: Determination of the quantity of data needed and specification of performance criteria for measuring quality for new data, or acceptance criteria for the use of existing data.
Data Collection: Description of how and where the data will be obtained (including existing data or data from other sources where the quality may be unknown or undocumented) and identification of any constraints on data collection.
Analysis: Description of how data will be analyzed, evaluated, and assessed against its intended use with respect to the established performance or acceptance criteria.

3.11	Implementation of Work
This section relates to Annex A, Implementation of Work (Clause A3.11).
3.11.1	General Considerations
The QMP can describe the general processes designed to ensure that all work involving environmental data collection, production, or use is performed according to approved planning and technical documents (e.g., QAPPs, SOPs) and in the prescribed sequence described in them. The benefits of systematic planning can be lost without careful project implementation aimed at producing the desired results.
Although the details of project implementations are to be found in QAPPs, the QMP can address the organization's overall approach to documentation of procedures, sample and data integrity, performance monitoring, and approval of deviations from established processes.
3.11.2	Documentation of Implementing Procedures
SOPs (or similar tools such as Reference Methods) routinely play a central role in promoting effective and consistent implementation of repetitive data acquisition tasks. An SOP is a document that details the method or procedure for implementing an operation, analysis, or action with thoroughly prescribed techniques and steps. SOPs are typically approved as the "official" method for performing certain routine or repetitive tasks. The process for the development and implementation of SOPs along with the roles and responsibilities of managers and staff can be documented in the QMP. SOPs can be written in a format that can be readily understood by the user (i.e., using plain English to the extent possible), with sufficient detail and clarity to ensure that effective results can be expected and achieved consistently. The process for the documentation of SOPs in the QMP may include (but may not be limited to):
identification of operations needing procedures;
preparation of procedures, including form, content, and applicability;
testing and evaluation of procedures (as necessary) to confirm their acceptable performance;
review and approval of documented procedures; and
assurance that up-to-date SOPs are available and those out-of-date SOPs are revised or withdrawn as needed.
SOPs no longer in use are maintained by the organization to support documentation of the projects using that version according to program and records management for that organization.
Procedures that specify technical requirements can be reviewed for adequacy and approved before use by qualified personnel who are independent of the entity developing the procedure. For more information on the development of SOPs, developers of a QMP may find it helpful to examine EPA QA/G-6, Guidance for Preparing Standard Operating Procedures (EPA 2007).
3.11.3	Sample and Data Integrity
This section relates to Standard Annex A, Implementation of Work (Clause A3.11).
The general processes and roles and responsibilities of personnel relative to sample and data handling and custody can be described in the QMP. For instance, if the organization will maintain a chain of custody within or across projects, its approach can be explained in the QMP.
Specific processes and roles and responsibilities of personnel pertaining to sample and data integrity for individual projects are presented in the applicable QAPPs.
3.11.4	Performance Monitoring
This section relates to Standard Annex A, Implementation of Work (Clause A3.11).
Level of management oversight and verification for the implementation of work is normally determined based on the importance of the particular project and the intended use of the project results.
The QMP can describe how the implementation of work processes will be monitored by the organization, including the routine measurement of performance against established technical and quality specifications to ensure continued satisfactory performance. It is important that the independence of personnel monitoring the work performance be ensured and that this independence be commensurate with the nature and importance of the activity. The QMP may need to address how to handle situations wherein complete independence of personnel cannot be achieved, such as within a small organization. Developers of the QMP may consider possible safeguards that could be utilized to provide for a practicable level of independence.
3.11.5	Deviations from Approved Processes
This section relates to Standard Annex A Implementation of Work (Clause A3.11).
The QMP can describe the process for how deviations from approved processes and procedures will be resolved (if possible), documented, and reported to management. The process and roles and responsibilities of managers and staff for addressing these deviations and any waivers from established policies, procedures, and standards can be described in the QMP. If the deviation necessitates a revised version of a planning document such as a QAPP or SOP, it is important to the reproducibility, defensibility, and utility of results that the current and subsequent versions of the document are indicated using a document control format as described in Section 3.2.2.
HELP BOX  -  MANAGEMENT-ORIENTED SOP TOPICS
Many Standard Operating Procedures relate to the details of data collection, production, and use, and are typically addressed in a QAPP. Examples of management-oriented SOPs that may be provided or referenced in a QMP include the following:
Standard Operating Procedure Development
Quality Assurance Project Plan Development
Quality Training System
External Agreement Review Process
Management of Quality Records
Quality System Assessment Process
Corrective Action Process
Dispute Resolution Process

3.12	Assessment, Oversight, and Response
This section relates to Standard Clause: Assessment Oversight, and Response (7.8), and Annex A Assessment, Oversight, and Response (Clause A.3.12).
3.12.1	Types of Assessments
CIO Standards 2106-S-01 and 2106-S-02 require organizations to develop and implement processes for assessment and oversight, and for responses to them. Assessments typically have specific criteria or measures of success associated with them. In general, an assessment is an evaluation process used to measure the performance or effectiveness of a system and its elements. As used here, assessment is an all-inclusive term used to denote activities such as the following:
Data Quality Audit (DQA)  -  a scientific and statistical evaluation of validated data to determine if the data are of the right type, quality, and quantity to support their intended use.
Performance Evaluation (PE)  -  a type of audit in which the quantitative data generated in a measurement system are obtained independently and compared with routinely obtained data to evaluate the proficiency of an analyst or laboratory.
Quality System Audit (QSA)  -  a documented activity performed to verify, by examination and evaluation of objective evidence, that applicable elements of the quality management system are suitable and have been developed, documented, and effectively implemented in accordance with specified requirements.
Readiness Review  -  a systematic, documented review of the readiness of the start-up or continued use of a facility, process, or activity. Readiness reviews are typically conducted before proceeding beyond project milestones and prior to initiation of a major phase of work.
Surveillance  -  continual or frequent monitoring and verification of the status of an entity and the analysis of records to ensure that specified requirements are being fulfilled.
Technical Systems Audit (TSA)  -  a thorough, systematic, on-site, qualitative audit of a project or operation that may include facilities, equipment, personnel, training, procedures, recordkeeping, data validation, data management, and reporting aspects of a technical system.
In the general context of assessments, the related concept of oversight pertains to monitoring activities or operations to confirm that they conform to approved plans. That is, are those implementing the work doing what they said they would do? In the list above, "surveillance" encompasses aspects of both assessment and oversight.
Users of this handbook may find that the term "audit" is used more commonly than "assessment".
3.12.2	Assessment Planning
Assessments offer the assurance that projects and programs are being implemented in accordance with established plans. In addition, assessment findings provide the content that leads from "check" to "act"  -  that is, they identify opportunities for preventive actions, corrective actions, and continuous improvement. Accordingly, when assessments of QMS are planned and scheduled by the organization regularly.
Their evaluated results can determine the suitability and effectiveness of the implemented QMS and of the quality performance of the environmental programs to which it applies.
The QMP can describe the process used by management to determine during the planning stage the type(s) of assessment activities appropriate for a particular project, and the assessment tool to be used. The types and frequency of assessments can be selected based on the particular needs of the organization and on the nature of its environmental data operations. Some projects might benefit from periodic technical systems audits in the field to ensure that samples are collected in accordance with plans, while other projects might be best served by rigorous data quality audits. A similar approach may be applied to determining the depth and intensity of an assessment (i.e., how many issues will be addressed, how many assessors will be assigned, and how many days the assessment will last). The optimum scope and balance of assessments is a program-specific determination.
For non-EPA organizations, specific requirements for assessments may be included in the external agreement. In some cases, assessments may be explicitly required in the scope of the external agreement such as in field monitoring activities or laboratory confirmation of sampling results.
Documenting the scope and objectives of every assessment in an assessment plan aids in the planning process, permits consistency of approach over time, and provides the basis against which to document and evaluate results.
A QMP can description of how such plans are generally reviewed and approved, and by whom reinforces the use of this valuable preparation tool and assists the assessment team in engaging with the appropriate parties. In this manner, assessments (or audits) can be performed according to approved written procedures (e.g., audit plan), based on careful planning of the scope of the assessment, the information needed and expected from the assessment, and any limits or constraints encountered (e.g., seasonal considerations when collecting water or soil samples).
3.12.3	Assessment Conduct and Follow-Up
Description in the QMP of the organization's approach for conducting and using the results of assessments and oversight activities, including the roles and responsibilities of managers and staff in this process increases the probability of these activities being accomplished.
The general processes and roles and responsibilities of personnel relative to conduct of assessments, implementation of response actions, and verification of responses can be also described in the QMP. This includes procedures for ensuring the objectivity, independence, and competence of assessment personnel. Specific needs can be defined in individual QAPPs for the activity.
The organization can have a system for selecting assessors who have the appropriate level of competence (i.e., capability, experience, and training) to meet the needs of the organization's assessment program. Again, the specific competence needed for a particular assessment can be addressed in the QAPP, but the QMP can address the more general, overarching expectations. Assessors (or auditors) can also have demonstrated independence from the work being assessed and can have no real or perceived conflict of interest.
Again, specific processes and roles and responsibilities of personnel pertaining to assessment of individual projects can be described in the applicable QAPPs, as discussed in CIO Handbook 2106-H-05 QAPP, Sections 2.4, 3.4, and 4.4.
Assessment results can be documented, reported to, and reviewed by management. This process can be described in the QMP along with the roles and responsibilities of involved managers and staff. The QMP can describe the process for identifying, conducting, and tracking corrective actions in response to assessments. The organization can ensure that corrective actions are made on a timely basis and that the effectiveness of their implementation is confirmed and documented.
The OEI Quality Staff typically conducts an assessment (i.e., Quality System Audit) of an EPA organization's Quality Management System at least every five years, but may do so more frequently if needed. Assessments of a non-EPA organization's Quality Management System are normally conducted as provided for in the external agreement, or at least every five years if the agreement is for an extended period. It is recommended that external organizations conduct self-assessments in accordance with a schedule defined in their QMP and agreed to by EPA.
For more information on planning, conducting, and reporting audits, EPA QA/G-3, Guidance on Assessing Quality Systems [EPA 2003] and ISO 19011, Guidelines for Quality and/or Environmental Management Systems Auditing [ISO 2002] may be helpful.
Other guidance on the planning, implementation, and reporting of audits and assessments that a user may find useful in developing a QMP is available in the following documents:
Guidance on Technical Audits and Related Assessments for Environmental Data Operations (EPA QA/G-7) [EPA 2000]
Data Quality Assessment: A Reviewer's Guide (EPA QA/G-9R) [EPA 2006a]
Data Quality Assessment: Statistical Tools for Practitioners (EPA QA/G-9S) [EPA 2006b]
HELP BOX  -  CORRECTIVE ACTION PROCESS
The QMP should describe the organization's overall approach for designing, implementing, and tracking corrective actions taken in response to audit findings. The corrective action process should be the responsibility of the organizational unit to which the finding applies, and should be sufficiently formal and transparent that management can confirm that the finding has been resolved. The following outline describes one formal approach that may be used to plan and conduct a corrective action process.
Step 1: Brainstorm to identify the possible causes of the finding.
Step 2: Agree on likely root causes. (A number of standard quality tools are available to help identify root causes of a problem.)
Step 3: Develop a list of potential solutions.
Step 4: Decide what solution to evaluate more fully.
Step 5: Develop a plan for implementing the solution and tracking its effectiveness. Develop a plan that breaks down the corrective action into discrete steps with an associated schedule.
Step 6: Implement and monitor the corrective action based on the corrective action plan.
Step 7: Develop a process for evaluating the effectiveness of the corrective action once it has been implemented.
Step 8: Determine the effectiveness of the implemented actions. Consider need to improve the solution or rethink root causes and start again.

3.13 Quality Improvement
 To facilitate continuous improvement within the organizations Quality System describe in the QMP:
   * the organization's process for ensuring continual quality improvement, including the management process for determining, planning, implementing, and evaluating the effectiveness of quality improvement activities including:
         o  who (organizationally) is responsible for quality improvement; and
         o the corrective action program to ensure that conditions adverse to quality are
               #  identified promptly, and
               # corrected as soon as practical.
   * how the organization shall detect and prevent quality problems, and
   * how staff at all levels are encouraged to identify and establish communications among customers and suppliers, identify process improvement opportunities, identify problems, and offer solutions to those problems.
 In order to enhance the effectiveness of corrective actions to be effective, it is strongly recommended that they include:
   * identification of root causes of problems,
   * determination of whether the problem is unique or has more generic implications, and a recommendation of procedures to prevent recurrence.

3.14	Data Review, Validation and Verification, and Data Usability Reporting
This section relates to Standard Clause: 7.9 Data Review, Validation and Verification, and Data Usability Reporting, and Annex A Assessment, Oversight, and Response (Clause A.3.12) and Quality Improvement (Clause A.3.13).
3.14.1	Planning for Data Review
The use of any environmental data for decisions follows the review of such data to determine the extent to which the technical and quality objectives for the data were met. Both Quality Standards require that organizations develop and implement appropriate review processes to confirm the suitability and adequacy of data for their intended use. The primary purpose of a QMS for environmental data collection, production, and use is to ensure that the resulting data have the necessary integrity, are defensible, and have adequate documentation to allow for other possible uses.
The criteria for the adequacy of such data will largely be specific to the project and can be addressed by the QAPP; however, it is appropriate for the QMP to document the general approaches and array of evaluation tools and to describe the general roles and responsibilities of managers and staff in their use.
The QMP can describe the possible or routine processes by which results obtained from environmental data collection or production are reviewed, verified, validated, and qualified prior to their intended use. 
The QMP should also commit to documenting any necessary deviation from routine processes and/or the development or use of alternative processes. Where possible, this discussion includes identification of the roles and responsibilities of management and staff relative to these functions. Specific processes and the roles and responsibilities of personnel pertaining to the evaluation of individual projects can be described in the applicable QAPPs.
The Quality Standards require that any project reports that contain data, or report the results of environmental data operations (e.g., data summaries) are to be reviewed independently (i.e., by others than those who produced the data or the reports) to confirm that the data or results are presented correctly. The QMP can describe the general process for the review and approval of these reports and can document the roles of managers and staff needing to participate. The Standards require that any limitations or restrictions on the use of the data shall be expressed quantitatively to the extent practicable and shall be documented in any reporting of the data. When adequate documentation of the data accompanies its reporting it will reduce, but not completely eliminate, the risk of the data being misinterpreted or misused for decisions.
3.14.2	Special Considerations for Data Review
If environmental data are collected from sources that did not use a QMS or whose quality is unknown or undocumented, the organization can review and evaluate these data for their suitability for use. Examples of such sources may include:
academic research and journal articles;
previous studies of contamination at a particular waste site; and
reports from volunteer groups such as a watershed association.
In addition, when data collected for purposes other than that of the current project are to be used, the QMP can describe the general process to be utilized for determining the suitability of the data for the new use. Details pertaining to a specific project can be described in the applicable QAPP. Key elements of a review of environmental data for potential use include:
the need to satisfy the major requirements of the Information Quality Guidelines (IQG), e.g., objectivity, utility, and integrity  -  see EPA/260R-02-008, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental Protection Agency [EPA 2002a];
the principal indicators of environmental quality, e.g., precision, bias, representativeness, comparability, completeness, and sensitivity  -  see Appendix B of EPA CIO Handbook 2106-H-05 QAPP; and
the general usability factors, e.g., relevance, temporal compatibility, spatial inclusivity, credibility, aggregation level, compatibility, reproducibility, and degree of augmentation  -  see Sections 2.5, 3.5, and 4.5 of EPA CIO Handbook 2106-H-05 QAPP.
The QMP may note that some uses of environmental data may not require a QAPP, but the data can still be appropriately vetted for their use. For example, the use of data for public information products may not need the rigor of a QAPP, but would at least need some review to confirm their adequacy for use.
In contrast, a technical decision or report from a large field sampling activity would be expected to have complete and comprehensive quality documentation to support the results. If possible, the QMP can specify the general circumstances under which a QAPP for such products will or will not be needed. (See CIO Handbook 2106-H-05 QAPP for details.)
3.14.3	Related Types of Reviews
Contentious or significant activities may be subjected to review during the project proposal and/or planning and design stage. In some instances, it is also appropriate to conduct various reviews over the lifecycle of a project.
If the organization applies a peer review process to information products, the QMP can describe the general process to be used, including selecting peer reviewers, interacting with them, and using their review comments. For EPA organizations, the current edition of the EPA Peer Review Handbook [EPA 2006c] provides further information on this topic.
Non-EPA organizations may have their own peer review process which can be documented in the QMP as applicable. To maximize its value, peer review can begin as early in the planning phase as possible.
For EPA organizations, disseminated information products, including those that contain environmental data or conclusions derived from environmental data, may be subject to the EPA IQG [EPA 2002a] requirements. As part of compliance with the EPA IQG, organizations can incorporate the principles of objectivity, utility and integrity into their existing review procedures, as appropriate.
For additional information on project-level data review and evaluation of usability, users can refer to Handbook on Quality Assurance Project Plans (EPA Handbook H-05 QAPP, Sections 2.5, 3.5, and 4.5) [EPA 2013c].
REFERENCES
ANSI 2004 American National Standards Institute and American Society for Quality, 
Quality Systems for Environmental Data and Technology Programs  -  Requirements with
 Guidance for Use (ANSI/ASQ E4-2004.
http://asq.org/quality-press/display

EPA 1984 U.S. Environmental Protection Agency, EPA Order 5700 Change 6, Assistance
 Administration Manual.
http://www.epa.gov/oarm
 
EPA 2000 U.S. Environmental Protection Agency, Guidance on Technical Audits and Related
 Assessments for Environmental Data Operations (EPA QA/G-7).
http://www.epa.gov/quality/qs-docs/g7-final.pdf

EPA 2002a U.S. Environmental Protection Agency, Guidelines for Ensuring and Maximizing
the Quality, Objectivity, Utility, and Integrity of Information Disseminated by the Environmental
 Protection Agency.
http://www.epa.gov/quality/informationguidelines/documents/EPA_InfoQualityGuidelines.pdf

EPA 2002b U.S. Environmental Protection Agency, Guidance for Developing Quality
Systems for Environmental Programs (EPA QA/G-1)
http://www.epa.gov/quality

EPA 2003 U.S. Environmental Protection Agency, Guidance on Assessing Quality Systems
(EPA QA/G-3).
http://www.epa.gov/quality/qs-docs/g3-final.pdf

EPA 2005 U.S. Environmental Protection Agency, Uniform Federal Policy for Implementing 
Environmental Quality Systems (EPA-505-F-03-001).
 http://www.epa.gov/fedfac/pdf/ufp_v2_final.pdf 

EPA 2006a U.S. Environmental Protection Agency, Data Quality Assessment: A Reviewer's
Guide (EPA QA/G-9R).
http://www.epa.gov/quality/qs-docs/g9r-final.pdf

EPA 2006b U.S. Environmental Protection Agency, Data Quality Assessment: Statistical Tools
for Practitioners (EPA QA/G-9S). 
 http://www.epa.gov/quality/qs-docs/g9s-final.pdf

EPA 2006c U.S. Environmental Protection Agency, Peer Review Handbook 3[rd] Edition.
http://www.epa.gov/peerreview/pdfs/Peer%20Review%20HandbookMay06.pdf

EPA 2007 U.S. Environmental Protection Agency, EPA QA/G-6, Guidance for Preparing
Standard Operating Procedures.
http://www.epa.gov/quality/qs-docs/g6-final.pdf

EPA 2008a U.S. Environmental Protection Agency, EPA Order 1610, Interagency Agreement
Policies, Procedures, and Guidance Manual.
http://www.epa.gov/oarm

EPA 2008b U.S. Environmental Protection Agency, Procedure for Quality Policy (CIO 
Procedure 2106-P-01).
http://www.epa.gov/irmpoli8/policies/2106p01.pdf

EPA 2008c U.S. Environmental Protection Agency, Quality Policy, (CIO Policy 2106).
http://www.epa.gov/irmpoli8/policies/21060.pdf

EPA 2009a U.S. Environmental Protection Agency, Contracts Management Manual.
http://www.epa.gov/oarm

EPA 2009b U.S. Environmental Protection Agency, EPA Records Management Policy.
(CIO Policy 2155.1).
 http://www.epa.gov/records/policy/2155/rm_policy_cio_2155_1_2.pdf 

EPA 2013a U.S. Environmental Protection Agency, Quality Standard for Environmental Data
 Collection, Production, and Use by EPA Organizations. (CIO Standard 2106-S-01).
http://www.epa.gov/quality
 
EPA 2013b U.S. Environmental Protection Agency, Quality Standard for Environmental Data
Collection, Production, and Use by Non-EPA (External) Organizations.
 (CIO Standard 2106-S-02).
http://www.epa.gov/quality

EPA 2013c U.S. Environmental Protection Agency, Handbook on Quality Assurance Project
Plans. (CIO Handbook 2106-H-05 QAPP).
http://www.epa.gov/quality

ISO 2002 International Organization for Standardization 19011, Guidelines for Quality and/or
Environmental Management Systems Auditing.
http://www.iso.org

ISO 2005 International Organization for Standardization 17025, General Requirements for the
Competence of Testing and Calibration Laboratories.
http://www.iso.org

ISO 2008 International Organization for Standardization 9001, Quality Management Systems  - 
Requirements.
http://www.iso.org
APPENDIX A

TERMS AND DEFINITIONS

NOTE: The following terms and definitions are taken from Quality Standard for Environmental Data Collection, Production, and Use by EPA Organizations (CIO Standard 2106-S-01.0) as well as Quality Standard for Environmental Data Collection, Production, and Use by Non-EPA (External) Organizations (CIO Standard 2106-S-02).
Assessment: The evaluation process used to measure the performance or effectiveness of a system and its elements. As used here, assessment is an all-inclusive term used to denote any of the following: audit, performance evaluation, management review, peer review, inspection, or surveillance. 
Competence: The ability to apply knowledge, skills, and experience to an activity to achieve intended results.
Conformity: The fulfillment of a requirement.
Corrective Action: An action to eliminate the cause of a nonconformity or undesirable situation and to prevent recurrence.
Data: A collection of facts and estimates from which conclusions may be drawn. 
Data Usability: The process of determining and ensuring that the quality of the data produced meets the intended use of the data.
Dissemination: Process of distributing information to the public that represents an official EPA endorsed opinion or decision. [Examples of information not considered a dissemination are information intended only for government employees; EPA responses to requests for Agency records under the Freedom of Information Act (FOIA), the Privacy Act, The Federal Advisory Committee Act (FACA) or other similar laws; correspondence directed to individuals or persons; ephemeral information; and distribution of information in documents filed in or prepared specifically for a judicial case or an administrative adjudication.] (Source: Section 5.3 & 5.4, EPA Information Quality Guidelines)
Document: Recorded information regardless of physical form or characteristics including individual records or items of non-record materials.
Environmental Data: Any data or information pertaining to the environment that describe measured outputs from processes, environmental conditions in a specific location; ecological effects and consequences; health effects and consequences; biological, chemical, and radiological conditions; or the performance of environmental technology. For EPA, environmental data include information collected directly from measurements, produced from models, and compiled from other sources such as databases, information systems, literature, or the Internet.
Environmental Data Collection: Process of acquiring or gathering environmental data through various means including (but not limited to) sampling and analysis activities, retrieval from information systems and the literature, and receipt from EPA partners and the regulated community.
Environmental Data Operations: Work performed to collect, produce, use, or report environmental data.
Environmental Data Production: Process of generating environmental data through various means including (but not limited to) the use of measurement instrumentation, information technology, computer models, and data analysis tools (e.g., statistics, risk assessment methods).
Environmental Programs: Activities involving the environment, including but not limited to: characterization of environmental processes and conditions; environmental monitoring; environmental research and development; the design, construction, and operation of environmental technologies; and laboratory operations on environmental samples.
Environmental Technology: An all-inclusive term used to describe pollution monitoring, measurement and control devices and systems, waste treatment processes and storage facilities, and site remediation technologies and their components that may be utilized to remove pollutants or contaminants from or prevent them from entering the environment. 
Extramural Agreement: Legal agreement between EPA and a non-EPA organization for the acquisition of items or services by EPA or financial assistance to a non-EPA organization. Such agreements include acquisition agreements (e.g., contracts, work assignments, delivery orders, task orders), assistance agreements (e.g., cooperative agreements, research grants, state and local grants), and EPA-funded Interagency Agreements (IAs) with other governmental entities.
Graded Approach: The process of basing the level of application of managerial controls applied to an item or work according to the intended use of the results and the degree of confidence needed in the quality of the results.
Guidance: A non-mandatory compilation of advice, examples, best practices, or past experience. Guidance may supplement procedures.
Handbook: A non-mandatory compilation of advice, examples, best practices, or past experiences, may be revised according to the issuing Office's Peer Review Policy.
Information: For purposes of this Standard, information means any communication or representation of knowledge such as facts or data, in any medium or form, including, but not limited to, textual, numerical, graphic, cartographic, narrative, or audiovisual forms (OMB Information Quality Guidelines).
Information Product: Any book, paper, map, machine-readable material, audiovisual production, or other documentary material, regardless of physical form or characteristic (OMB Circular A-130).

Information Quality Guidelines (IQG): An Agency document that defines a basic standard of quality (including objectivity, utility, and integrity) for information products disseminated by EPA. For influential information products, the basic standard of quality also includes reproducibility and transparency.
Information System: An organized collection, storage, and presentation system of data for decision making, progress reporting, and for planning and evaluation of programs. It can be either manual or computerized, or a combination of both.
Information Technology: The study, design, development, implementation, support, or management of computer-based information systems, particularly software applications and computer hardware.
Informative Annex: A part of a Standard that gives additional information as guidance which is intended to assist the understanding or use of the Standard. An informative annex is not considered to be part of the Standard when setting audit criteria.
Integrity (information): Assurance that the information is protected from unauthorized access or change and is not compromised through corruption or falsification.
Management Controls: A system of management functions to enable managers to (1) determine that the operations of a program or organization satisfy predetermined goals and objectives, and that performance is in line with standards and specifications; and (2) implement any remedial actions needed to ensure that human and other resources are being used in the most effective and efficient way possible in achieving the organization's mission.
Management System: A system to establish policy and objectives and to achieve those objectives (ISO 9001). A management system may describe the policies, objectives, principles, authority, responsibilities, accountability, and implementation plan of an organization for conducting work and producing environmental data. Management systems include ISO 9001 on quality management, ISO 14001 on environmental management, and OHSAS 18000 on occupational health and safety.
Normative Annex: A part of a Standard that gives provisions (requirements) additional to those requirements in the body of the Standard. A normative annex is considered to a part of the Standard when setting audit criteria.
Organization: Company, corporation, firm, enterprise, or institution, or part thereof, whether incorporated or not, public or private, that has its own functions and administration. In the context of this Standard, an EPA organization may be an Office, Region, National Research Center or Laboratory, or a sub-unit such as a division, branch, section, or team.
Policy: A high-level statement about an Agency requirement designed to influence and determine decisions, actions, and other matters. It is usually driven by statute, executive order, the mandate of an oversight agency or Congress, or the head of the organization.
Prime Recipient: Organization that directly receives funding, support, or authorization to implement a project or program.
Procedure (CIO Procedure): The required steps, course of action, or processes needed to accomplish or satisfy a policy.
Process: A set of interrelated resources and activities which transforms inputs into outputs. Examples of processes include analysis, design, data collection, operation, fabrication, and calculation.
Product: The intended result or final output of an activity or process that is disseminated or distributed among EPA organizations or outside of EPA.
Programmatic QMP: a management plan that defines the structure and framework for planning, implementing, and evaluating quality for a specific policy, or program.
Quality: The totality of features and characteristics of a product or service that bear on its ability to meet the stated or implied needs and expectations of the user.
Quality Assurance (QA): A management or oversight function that deals with setting policy and running an administrative system of management controls that cover planning, implementation, review and maintenance to ensure environmental data are meeting their intended use.
Quality Assurance Manager (QA Manager): The individual designated as the principal manager within the organization having management oversight and responsibilities for planning, documenting, coordinating, and assessing the effectiveness of the QMS for the organization. NOTE: Other personnel having QA or QC duties may be referred to as QA Officer and QA Coordinator.
Quality Assurance Project Plan (QAPP): A document describing in comprehensive detail the necessary QA, QC and other technical activities that must be implemented to ensure that the results of the work performed will satisfy the stated performance objectives and criteria.
Quality Control (QC): The overall system of technical activities that measure the attributes and performance of a process, item, or service against defined standards to verify that they meet the stated requirements established by the customer; operational techniques and activities that are used to fulfill requirements for quality.
Quality Management: That aspect of the overall quality management system of the organization that determines and implements the quality policy. Quality management typically includes strategic planning, allocation of resources, and other systematic activities (e.g., planning, implementation, and assessment) pertaining to the application of quality practices to the organization's programs.
Quality Management Plan (QMP): A formal document or manual that describes the QMS in terms of the organizational structure, functional responsibilities of management and staff, lines of authority, and required interfaces for those planning, implementing and assessing all activities conducted.
Quality Management System (QMS): A structured and documented management system describing the policies, objectives, principles, organizational authority, responsibilities, accountability, and implementation plan of an organization for ensuring quality in its work processes, products (items), and services. The quality management system provides the framework for planning, implementing, documenting, and assessing work performed by the organization and for carrying out required QA and QC activities.
Quality Program: The totality of management controls, processes, and documentation in EPA's planning, implementation, and assessment of applying quality to the creation of Agency environmental data.
Record: A document stating results retrieved or providing evidence of activities (ISO 9000:2005). NOTE: A federal record is an information resource in any format that is needed to describe Agency activities (44 U.S.C. § 3301).
Requirement: An expression of the content of a Standard that conveys criteria to be fulfilled is compliance is to be claimed and from which no deviation is permitted.
Service: A discrete function that performs one or more operations and returns a set of results to an external requester.
Standard: An accepted, consensus-based specification which defines systems, processes, methodologies, or practices. It provides a basis for assuring consistent and acceptable minimum levels quality, performance, safety, and reliability. Standards usually are included in or accompany procedures.
Standard Operating Procedure (SOP): A written document that details the method for an operation, analysis, or action with thoroughly prescribed techniques and steps, and that is officially approved as the method for performing certain routine or repetitive tasks.
Transparency (Information): Assurance that information is supported by sufficient metadata when disseminated.
Usability Assessment: Evaluation of data based upon the results of data validation and verification for the decision(s) being made. Reviewers assess whether the process execution and resulting data meet quality objectives based on the criteria given in the QAPP.
User: An organization, group, or individual that utilizes the results or products from environmental programs or the customer for whom the results or products were collected or created.
Utility (Information): Assurance that information is useful for its intended purpose.
Validation (Information): Confirmation by examination and provision of objective evidence that the particular requirement for which the information is intended are fulfilled; the process of determining whether the specifications were appropriate and that the verified results will meet the data user's needs.
Verification (Information): Confirmation by examination and provision of objective evidence that validated information fulfills specified requirements; the process of checking whether the information met the project's specifications.