United States Office of Research and EPAI600/R·94!038a 

Environmental Protection Development· April 1994 

Agency Washington DC 20460 

"'~'oEPA Quality Assurance 

Handbook for 

Air Pollution 

Measurement 

Systems 

Volume I: A Field 

Guide to .Environmental 

Quality Assurance 

Quality Assu'rance Handbook 

for 

EPA-600/R-94/038a 

A~.r Pollution M_easurement Systems 

Volume I - A Field Guide to 

Environmental Quality Assurance 

1993 

by 

Monica Nees 

U.S. Environmental Prot-ection Agency 

Office of Research and Development 

Atmospheric Research and Exposure Assessment Laboratory 

Research Triangle Park, NC 27711 

@ Printed on Recycled Paper 
OVERVIEW OF THE INTERIM EDITION OF VOLUME I 

The Quality Assurance (QA) Handbook is comprised of five 

volumes: volume I (Principles), Volume II (Ambient Air Methocs , 

Volume III (Stationary Source Methods), Volume IV (Meteo~ologi~~l 

Measurements) ,and Volume V (Precipitation Measurement Systems)_ 

Much of the material in Volumes II, III and V are out-of-date a:.d 

some portions of these volumes have long been out-of-print. 

EPA is now preparing an updated version of,the QA Handboc-:: 

series which will be available in September 1995: To meet the 

needs of the user community until the updated version is 

available, EPA has published Interim Editions of Volumes I, II, 

III, IV and V. Each volume of the Interim Editions, is being 

issued as a complete unit with out-of-date sections either 

deleted or modified using addendum sheets and handwritten 

notations in the text. 

This volume and the other four volumes of the Interim 

Edition of the QA Handbook are available at no charge from: 

USEPA/ORD 

Center for Environmental Research Information 

26 West Martin Luther King Drive 

Cincinnati, Ohio' 45268 

Since this volume was updated in 1993, only minor changes 

will be done to it in the updating process. The updated versic~ 

will be available in septe~er 1995. 

The user of the QA Handbook is cautioned to bear in mind 

that the information provided in the handbook is for guidance 

purposes only. EPA regulations are published in the Code of 

Federal Regulations (CFR). When information in the CFR confli~-:s 

with information in the QA Handbook, the CFR shall be considere~ 

the authoritative and legally bonding document. 

William J. Mitchell 

Chief 

Quality Assurance Support Branct 
,......--.. ACKNOWLEDGEMENTS 

This completely new version ~f Volume 1 of the Ouality Assurance Handbook 

for Air Pol/ution Measurement Systems was prepared by Dr. Monica Nees under 

three affiliations. First, as a chemist enrollee of the Senior Environmental 

Employment program of NCB A and tbe U.S. Environmental Protection Agency (EPA), . 

she developed the initial drafts under the direction of Dr. BilrMitchell, Chie.f, Quality 

Assurance Support Branch, Atmospheric Research and Exposure Assessmentlaboratory, U.S. EPA, Research Triangle Park, North Carolina. Then, as a senior 

scientist at ManTech Environmental Technology, Inc., Research Triangle Park, North 

Carolina, she completed work for publication under the direction of Kenneth J. 

Caviston, Supervisor, Quality Assurance and Other Support, under EPA contract 68-

00-0106 . 

DISCLAIMER 

This document has been reviewed in accordance with the U.S. Environmental 

Protection Agency's peer review policy and has been approved for publication. 

Mention of trade names or commercial products does not constitute EPA 

endorsement or recommendation for use. 

iii 
FOREWORD 

Throughout the world, air quality is a critical concern. In the United States and 

Canada, air monitoring is not the responsibility of just the federal governments. 

States and provinces, local governments, private industries, and environmental 

organizations are also participating. Elsewhere, especially in those countries in 

which air quality is beginning to be addressed,' national 'governments are th'e 

principal monitors. 

The purpose of these monitoring efforts is not to collect data, because data are 

only the beginning, not the end, of environmental investigations. Data should not 

be stored and forgotten, but should be used to make informed decisions affecting 

the health and well-being of planet Earth. Application of the principles of quality 

assurance ailows decision makers to know the quality of the data on which their . . . . 

actions are based. 

William Zinsser in his book On Writing Well calls the instructional manual "one 

of the most forbidding swamps in the English language." I hope that this field guide 

is not. It focuses on the fundamentals that transcend national borders, academic 

disciplines, and even specific envi'ronmenial media. Like a field guide used in 

birdwatching, it does not tell everything, but only the most important things. It is 

designed to be used in the field or laboratory, not stored on a shelf. And, although 

the examples are chosen from air monitoring, the principles can readily be applied to 

any type of environmental monitoring. 

This field guide does not give detailed instructions for preparing a quality 

assurance plan. Instead, it emphasizes the thought processes and rationales fordesigning any good data collection program with quality assurance as an integral 

part. Once this occurs, preparing a quality assurance plan using the format specified 

by any sponsoring organization wi.ll be straightforward. 

iv 

Monica Nees 

1993 
HOW TO USE THIS FIELD GUIDE 

This field guide replaces Volume I, Principles, of the Ouality Assurance 

Handbook for Air Pollution Measurement tv" ~thods, first published in the late 1970s . 

and updated in 1984. Using a common-sense approach, it explains the unifying 

concepts underlying all environmental quality assurance, in about one- tenth the 

-number of pages of its predecessors. 

Such a massive reduction was possible by the elimination of duplication of 

numerous definitions, examples, appendices, and details also found in Volumes II 

through V. of the handbook. Then the basic principles could be revealed and 

studied. Once the user understands the principles, he or she can consult the other 

volumes for necessary details. Volume 1/, Ambient Air Specific Methods, for instance, 

includes b.oth a lengthy intr«:>ductory chapter on quality assurance for ambient air 

methods and detailed guidance on nearly a dozen individual test methods. 

By design, the field guide covers only the "Big Picture_" Written for a broad 

audience, it is intended for use both byfield and laboratory personnel and by their 

managers in planning all aspects of environmental data collection. Its sections cover 

all phases of the life cycle of any such project, from planning through final report 

writing. Throughout, the importance of planning is stressed again and again. Each 

section is self- contained, for ease in future reference. The best way to use the field 

guide, however, is first to read it completely to get an overview and then to consult 

individual sections as needed. 

By applying the principles described in ·the field guide to his or her own 

projects, the user will make certain that all data collected will meet project n-eeds. 

Because that data will be of known and documented quality, others will be able to 

use it with confidence too_ And that is what quality assurance is all about. 

For additional information, contact: 

Chief, Quality Assurance Support Branch 

Quality Assurance and Technical Support Division 

Atmospheric Research and Exposure Assessment Laboratory 

U.S. Environmental Protection Agency 

Research Triangle Park, North Carolina 27711 

USA 

v 

CONTENTS 

SECTION PAGE 

Acknowledgements ••••••.••••••••••• :......................................... iii 

Disclaimer .•..•..••••..••••••••••••••.••••••••••.••••••••.•••••.••.••••.••••..• iii 

Foreword ••••••••••••••••••••••••••••••••• ' • • • • • • • • • • • • • . . • • • • • • • • • • • • • • • • • . • . • iv 

How to Use This Field Guide ................................................ , • . • • v 

PLANNINGANDORGANIZATION .... ;............................................. 1-1 

1.1 PROJ ECT DESCRIPTION • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • . • • . . . . • • • • • . . . . . . . • 1-1 

1.2 ORGANIZATIONAL CHART •.•••••••.•••••••••.•••.•••••.•.••••• •.•• • •• • • . • • . 1-1 

1.3 JOB DESCRIPTIONS •••••..••••.••••••••.•••••.•••.•••..•••••••••.••.••••..• 1-2 

1.3 .. 1 Project Manager •..•.....•••.•••.••••............ ~ • . • • • • . . • • • • • • • • • • 1-3 

1.3.2 Quality Assurance Manager •••.•••••••••••••••••••••••.•••.•••••••.•• 1-3 

2 PLANS AND REPORTS ••••.••••••••••••••••••••••••••••••••••••••••••.•.•••...•. 2-1 

2.1 BEFORE DATA COLLECTION ••••.••••.••••••••••••.••••..••••••••.••••••••.. 2-1 

2.2 DURING DATA COLLECTION ••••.••• ;....................................... 2-2 

2.3 AFTER DATA COLLECTION ••••••••••.•• '.................................... 2-2 

3 STANDARD OPERATING PROCEDURES ........................................... 3-' 

3.1 PURPOSE................................................................. 3-1 

3.2 CONTENTS ••••••••.••••• ;................................................ 3-2 

3.3 HOUSEKEEPING DETAILS ••.•••••••..••.....•...................•.......... 3-3 

4 PREVENTIVE MAINTENANCE.................................................... 4-1 

4.1 EXAMPLES •••••••••••••..•..•.•..•.•.•••..••••••••••••••••••••••••••••... 4-1 

4.2 REQUIREMENTS ••••.•••••..••••••..•...••••••.•••••••.•••••••••..•••....• 4-2 

5 SAMPLE COLLECTION, HANDLING, AND ANALYSIS ................................. 5-1 

5.1 SELECT SAMPLING SITES BASED ON DATA QUALITY NEEDS ...•....••..•....... 5-1 

S.2 UNDERSTAND THE REASONS BEHIND THE PROCEDURES ••..•••...••..••..•.•. 5-1 

S.3 USE THE SAME CONDITIONS FOR STANDARDS AND SAMPLES ••••••••••••••... 5-2 

5.4 USE QUALITY CONTROL CHECKS AND STANDARDS •••••••.•.•.•••.••.••••• 5-2 

5.5 KNOWWHERETHESAMPLESAREANDBEABLETOPROVEIT ••••••••••••••••• 5-2 

6 DATACOLLECTIONANDHANDLING .... .'......................................... 6-1 

6.1 KNOWWHYTHEDATAMUSTBECOLLECTED :............................... 6-1 

6.2 DOCUMENT EVERYTHING THOROUGHLY .; ..•••••• ~......................... 6-1 

6.3 CALIBRATE INSTRUMENTSANDTESTSOFTWARE ••••••• ••••••• ••••• •••••••••• 6-2 

6.4 PRESERVETHEORIGINALDATA ............................................ 6-2 

(continu@d) 

vii 
CONTENTS 

SECTION 

6 DATA COLLECTION AND HANDLING (Continued) ................................ . 

6.5 USE ONLY VALIDATED DATA .............................................. . 

6.6 USE TABLES OR GRAPHS TO PRESENT SUMMARY STATISTICS •....•••••..•••... 

6.7 LEAVE SOPHISTICATED DATA HANDLING TECHNIQUES TO THE STATISTICIANS " 

6.8 BEWARE OF USING DATA COLLECTED FORANOTHER PURPOSE .••••..•..••...• 

7 STATISTICAL TERMS AND DATA QUALITY INDICATORS ••.••..• ' •••••. ~ .•••••...•••• 

7.1 STATISTICAL TERMS •...••.••••..•••••••.••••••••••••••••..••••••••••••••.• 

7.1.1 Arithmetic Mean .•••••••••. ~ •••••••••••••••..••••...•..••••.•...•••. 

7.1.2 Standard Deviation and Variance .................................... . 

7.1.3 Geometric Mean •••••••••••••••••••••••.•••.•.....•••...•..••....... 

7.1.4 Geometric Standard Deviation •.••••••••.•••••.••....••.••...•••....• 

7.2 DATA QUALITY INDICATORS •.•..•••..•••.••••..••.••••.•••••••...••••....• 

7.2.1 Precision .•••..•••••••...•••..••.•••••..••••••••.•.•.•••••...•••••.. 

7.2.2 Accuracy •••••..•••.• , •••.••••••••••••• ',' ••.••••.•..•••••••.•••••••. 

7.2.3 Completeness .•...•••••••••••••••.•••••••..•••••••..•••••••...•••.. 

7.2.4 Method Detection Li'mit •••••.••••••••••••••••.•••••••••.•.•••••..••. 

7.2.5 Representativeness .•..•.••••....•••••...••••••..•.•• , •••...•••.••... 

7.2.6 Comparability .•••••••...••••••••..•••••.•..•••••••..••••••••.•••••• 

8 AUDITS •..•...•••••..••••••••.••.•••...•••.•••.•••••.•....••..••..••••••••..•• 

8.1 DOCUMENTATION ..•••..•••..•••.•••..••• ' ••...••••••..••.••••...•••••••.. 

8.2 AUDIT TYPES •••••..••••.•••.•••••••. ' ..•••••••...••••••......•••....••••.. 

8.2.' Technical Systems Audit ........................................... .. 

8.2.2 Performance Evaluation Audit •••..••••••.•...••.•.....••..•....••... 

8.2.3 Audit of Data Quality •...•••••••..•••.••.....•.•••......••••..•••.•• 

8.2.4 Management Systems Audit .••••••.•.•••••...•..••....•.••••••.•.•.• 

8.3 AUDIT PROCEDURES ••.•.•••.•••.••••..•••..••••.••...••.•••.....••••...•• 

8.3.' preaudit Activities .••...••..•..•••••.•••.•••••..•..•••••.•••.••••... 

8.3.2 Conducting the Audit •...•••••.•..•••••••..••••••...•.•••••....••• ~ • 

8.3.3 Preparation of the Audit Report •••••••••••••• , •.•••.....•••••......•• 

8.3.4 Postaudit Report Activities ••••••...•••••••..•.•..•.......•••.......•• 

9 CORRECTIVE ACTION •••••••..••••.••..•••..•••••.•..•••••...•••••..••..•••••••. 

9.1 ROUTINE MEASUREMENTS ••..•..••••.•..•••••••.••.••••••..•..•••••••..••• 

9.2 MAJOR PROBLEMS •••••••••..•••••••••••••••••••••••••••••..• ' ••••••..••••• 

10 BIBLIOGRAPHY 

viii 

~"--

PAGE 

6-1 

6-2 

6·2 

6-2 

6-3· 

7-1 

7-' 

7-1 

7·' 

7·2 

7·2 

7-3 

7-3 

7-3 

7-4 

7-4 

7-5 

7-5 

8-1 

8-1 

8-2 

8·2 

8-2 

8-2 

8-3 

8·3 

8-3 

8·4 

8·5 

8·5 

9-' 

9·1 

9·1 

10·' 
FIGURES 

FIGURE PAGE 

1-1 Example of Organizational Chart ••••••••••••••••••••••..•••..•••.••••.•.••....•. 1-2 

9-1 Corrective Action Form 9-3 

TABLES 

TABLE PAGE 

3-1 Suggested Format for a Field or Laboratory 

Standard Operating Procedure .•••• ;............................................ 3-2 

3-2 Tracking System for Standard Operating Procedures " ..• . ... •. ..•. . .••• . . . . . . .. • . 3-3 

3-3 Document Control Format .....••...•.•....••.••....•...••...•....••.......•.•.. 3-4 

5-1 Principles6fSampleColiection,Handling,andAnalysis •••.••••.•••••••••.••••..••• 5-1 

6-1 Principles of Data Collection and Handling 6-1 

ix 

SECTION 1 

PLANNING AND ORGANIZATION 

Projects brilliantly conceived will not be brilliantly executed without good 

planning and organization. Project success depends on the leadership and 

organizational skills of the proJect manager. The mah~g!!r notonly must knowwhat 

needs to be done, but also must share that -knowledge so that all staff membe·rs 

understand precisely how they fit into the "Big Picture." 

1.1 PROJECT DESCRIPTION 

A detailed project description forms the basis for all other planning and 

organizational activities. The critical personnel and resource needs should arise from 

the project description - and not the other way around. 

The project manager and other key personnel jointly develop the project 

description, which must contain the following six components. 

• What is going to be done 

• Why it is necessary to qo it 

• Who will do it 

• How it will be accomplished 

• Where it will be done 

• When it will be carried out 

Unless all six are addressed in test and' quality assurance (QA) plans; the project 

description is incomplete and subject to misinterpretation. Section 2 describes these 

components in more detail, in the context of reports required before, during, and 

after data collection. 

1.2 ORGANIZATIONAL CHART 

A clearly presented organizational chart is one of the most important products 

of the planning process because it names all key individuals in charge of every major 

activity of the project. Figure 1 p 1 shows a simple organizational chart. If possible, 

the names of all team members sh~uld be included; th~se of all supervisors must be. 

1p1 
All subcontractors ·must be listed too, with clear lines of reporting, to prevent the alltoo-common "floating subcontractor syndrome." 

QA 

Manager 

(Name) 

Field 

Sampling 

Supervisor 

(Name) 

I 

I 

I 

I 

I 

I 

I 

I 

I _______________ J 

Project 

Manager 

(Name) 

Laboratory 

, Analysis 

Supervisor 

(Name) 

Figure 1·1. Examp~e of Organizational Chart 

Data 

Analysis 

Supervisor 

(Name) 

Because the QA manager must be able to give completely unbiased advice on 

data quality to the project manager, he or she should be organizationally 

independent of the project manager and all other data collection and handling 

personnel. This special relationship is shown by a dotted line on the organizational 

chart. 

1.3 JOB DESCRIPTIONS 

Rather than list job responsibilities for each and every conceivable position, this 

section examines the responsibilities of on.ly two, the project manager and the QA 

manager, in some detail. Not every project will have a laboratory supervisor, for 

1-2 
instance, because it may entail only analysis of data already collected. But every 

project will have a manager and a QA manager, even if they are one and the same 

person on small projects. 

, .3.1 Project Manager 

Like the captain of a sllip, the project manager is ultimately responsible for 

-everything that happens in his or her project, induding the QA necessary to achieve 

the data quality required by the project's sponscir: The . manager's primary 

responsibilities are liaison with the sponsor, planning, budgeting, staffing, and 

overall coordination and review. Just as no one would expect a ship's captain to 

perform every operation on board, no one expects the project manager to do 

everything single-handedly. That is why a staff is hired. Frequently, the project 

manager appoints a QA manager for assistance in developing and implementing the 

QAlquality control (QC) needed to achieve the required data quality. The ultimate 

responsibility for QAlQC, however, as for any other project function, still resides with 

the project manager. 

1.3.2 Quality Assurance Manager 

Two definitions will help in understanding, the duties of the QA manager: 

QUALITY CONTROL is everything YOU do to make certain that 

your project is performing" up to specs." 

QUALITY ASSURANCE is everything you have SOMEONE ELSE do 

to assure you that your QC is being done "according to specs." 

Thus, if the same" individual who, performs the work also does the checking for 

quality, that checking is quality control. Running duplicate samples in the laboratory 

is a common QC procedure. If a different individual does the checking, that is an 

example of quality assurance. A project review by auditors from another company is 

a typical QA activity. 

A review, however, need not be performed by a different company; more 

commonly, it is done by the QA manager: within the same organization but 

completely independent of the data- collecting staff. The QA manager protects the 

project manager from poor quality data that do not fulfill project needs. Thus, 

anything that affects data quality comes under the purview of the QA manager. 

Most of the activities of a QA manager involve the review of project activities 

and the preparation or review of reports. The mix depends on the wishes of the 

1-3 
project manager. Frequently. the OA manager is assigned to prepare the QA plan, 

review all other documents generated during the project. and carry out other tasks 

specified by the project manager. 

The following sections describe various project functions in detail: reports; 

standard operating procedures (SOPs); preventive maintenance; sample collection, 

handling, and analysis; data collection and handling; audits; and corrective action. 

Because all impact on data quality, all must be addressed by the QA ma'nager. ' . Not addressed in these sections, however, is one other important function 

often assigned to the QA manager, that of training coordinator. Everyone must be 

trained well enough to produce the highest quality of data needed by the project. 

A common mistake is to provide training only for field and laboratory 

personnel, while neglecting the clerical staff and managers. Anything that affects 

data quality is a suitable topic for training. Thus, the clerical staff must be trained 

continuously to take full advantage of the ever-changing enhancements in word 

processing systems and managers need training on topics ranging from financial 

information systems to handling personnel problems. 

Hiring staff with appropriate formal education is only the first step in building 

a competent team. Next comes' on-the-job training under the guidance of a 

knowledgeable mentor who teaches the skills and nuances specific to the particular 

task and organization. Short courses, both on-site and off-site. develop well-defined 

sets of skills in a specific area. Formal courses at a college or university give a more 

in-depth mastery of a subject. 

A combination of training activities will be needed for most projects. Some 

form oftraining will be needed for someone throughout the life ofthe project. 

1-4 
SECTION 2 

PLANS AND REPORTS 

Anyone -aliergic R to writing anything on paper will not thrive in 

environmental data collection. Sponsors and supervisors require a steady stream of 

reports, from before a project beg'insuntil after it is completed. Writing a' good 

report is not that much different from writing a good newspaper article. 'Both 

processes concentrate on the six key principles of Who, What, Where. When. Why. 

and How, but the relative emphasis given to each depends on where the report fits 

into the life cycle ofthe project. 

Rare, indeed, is the project that spawns only one report; instead, many 

different types are usually produced. The beginning of data collection in the field or 

laboratory is the benchmark. Planning docume,nts are written before data collection 

begins, progress reports while it is under way, and final reports after it is completed. 

Nobody wants to read a report that is t09 long and incoherent. Applying the six 

principles can prevent such a report from ever being written. The best "mix of the 

six" depends on whether the report comes before, during, or after data collection. 

2.1 BEFORE DATA COLLECTION 

The most important project reports are those written before the first piece of 

data is collected. These planning documents include all six principles, but the most 

important are Who, What, and How. They specify, by name, Who is in charge of 

What part of the project and How, in detail, the work will be accomplished. Each 

and every part must be included because success of the project depends on how well 

all of the parts fit together. A simple organizational chart is mandatory. If the 

relationships are difficult to draw, they will be even more difficult to execute. 

Examples oftypical planning documents include the following. 

• Data quality objectives reports 

• Work or test plans 

• Quality assurance plans 

• Site selection, sampling, and analytical procedures (if not included in 

the work plan) , . 

• Standard operating procedures 

2-1 
• Data handling protocols 

• Corrective action plans 

• Others, as necessary 

Although Who, What, and How predominate, Where, When, and Why cannot 

be neglected. Geographical location (the Where) can be a critical vari"able in field . . 

work. The When includes' not only the spe-cific ~ours, days, months, 'or years of. 

project duration but als'o such important topics as seasonal and diurnal variation$, 

Although the Why is more subtle than the other principles, knowing Why the 

data have to be collected is critical to the success of any project. The reason is quite 

simple: data m'ust be collected for a purpose, Different purposes require different 

data collection plans. Proj~ct planners can devise the best one only if they know the 

end uses of the data, Planning documents must clearly state the purposes behind 

data col'lection, so that both current and future users understand the limitations on 

using the data for decision making. They also establish the competence of the 

project team to do the job right the first time, on time, and within budget. They 

describe what is anticipated and thus serve as yardsticks by which to measure 

progress. 

The QA manager reviews and approves the QA plans, which include key 

- sections from many other planning documents, but the project manager must sign 

off on all documents. Although each staff member is responsible for the quality of 

his or her part of the project, the project manager is responsible for the quality of 

the entire undertaking. 

2.2 DURING DATA COLLECTION 

Progress reports, the most commonly written reports during the data collection 

phase of a project, continuously answer the question, "How are we doing?" The 

standards used are the ones previously stipulated in the planning documents. Audit 

reports and corrective action reports are also prepared in this phase. Audits, 

whether performed internally or by outside organilations, assess What is being done 

and How well. Whenever corrective action is taken, the report describes What the 

problem was and How it was solved. 

2.3 AFTER DATA COLLECTION 

If planning and progress reports are well prepared, writing the final report 

should not be an overwhelming burden. Its purpose is to summarize and analyze -
to say What happened and Why, but not to meditate on every single data point. 

Appendices and references to earlier reports can take care of that. 

The final report, which frequently mirrors the test or work plan in sequence 

and approach. covers all six principles. It is also a self-audit, assessing How well the 

standards spelled out in the planning documents were met, and clearly explaining 

any limitations on data use for both pr~sent and future users. This careful analysis in 

a final report for on~ project may also serve as a springboard toa new one in which 

currently unresolved problems may be solved. 
--------------------------------------------
SECTION 3 

STANDARD OPERATING PROCEDURES 

When should a manager decide that an SOP needs to be written? The answer is 

deceptively simple: as soon as the procedure becomes standardized - and not 

before. The argument that no procedure, is ever standardized, however, is used all 

too frequently only to avoid putting anything on paper. 

Although the time and effort spent in preparing an SOP can be significant, 

there are important long-term benefits. No longer will the same procedure have to 

be described again and again in test plans, QA project plans, audits, and other 

reports. Instead, it can be incorporated by reference, with a copy ·attached to the 

report. But saving data, not merely saving time, is the main reason for preparing an 

. SOP. Data collected using fully documented procedures have much higher credibility 

and defensibility. Because well-written SOPs focus on routine operations, their users 

can concentrate primarily on nonroutine problem solving. 

3.1 PURPOSE 

An SOP is written so thai the procedure will be performed consistently by 

everyone, every time. Deciding whether a particular procedure is a candidate for an 

SOP is helped by answering two questions: 

• Does the procedure significantly affect data quality? 

• Is the procedure repetitive or routine? 

Preparing an SOP is indicated ifthe answer to both questions is YES. 

Targeting the proper audience can be the most difficult task. Obviously, the 

SOP should be written at a level of detail appropriate to the end users. If 

backgrounds of the users are unknown, target the SOP for a nnew hire, n a technician 

with at least two years of college and _ one year of experience in the appropriate 

field. This approach usually ensures that the SOP has enough detail without 

becoming overwhelming. 

Few routine laboratory or field projects can be described completely in just one 

SOP. Several will be needed, and deciding how best to divide the topics will take 

careful planning. In general, an SOP for each of several smaller segments is much 

better and easier to write than one large SOP for an entire operation. 

3-1 
3.2 CONTENTS 

Table 3-1 shows a suggested format for an SOP, including numerous examples 

of items that could be included in each section-. The examples shown are onlya few 

of the many that could be covered, depending on the, particular procedure. 

Occasionally, deciding whether an item belongs in one section or another can be a 

problem. The important thing is to put it somewhere, rather than leave it out. 

TABLE 3-1. SUGGESTED FORMAT FOR A FIELD OR LABORATORY STANDARD 

OPERATING PROCEDURE 

Section, 

1. Scope and Application 

2. Summary of Method 

3. Definitions 

4. Interferences 

A. TECHNICAL SECTIONS 

Typical Examples 

Overview outlining purpose, range, sensitivity, 

acceptance criteria 

Overview describing sampling criteria and 

analytical methods, method and instrumentation 

detection limits, reasons for deviations from 

Federal Register methods 

All acronyms, abbreviations, specialized terms 

Sources of contamination 

5. Personnel Requirements Educational level and training of intended SOP 

users, number of operators required 

6. Facilities Requirements 

7. Safety Precautions 

8. Apparatus 

9. Reagents/Materials 

Mobile analytical laboratory, air conditioning, 

types of electricity, fume hood -

Types of respirators, carbon monoxide monitors, 

special handling procedures; hazard warnings, 

placed immediately BEFORE relevant part of text 

larger items such as a meteorological tower, audit 

device, pH meter, gas chromatograph 

All chemicals used, including distilled or deionized 

water; grades of reagents; materials include 

smaller items such as filter paper, boiling chips, 

tub'ing, electrical wiring 

(continued) 

3-2 
----- TABLE 3-1. SUGGESTED FORMAT FOR A FIELD OR LABORATORY STANDARD 

OPERATING PROCE~URE (Continued) 

Section 

10. SampleslSampling 

Proced u res -

11. Calibrationl 

Standa rd ization 

12. Analysis Procedures 

13. Calculations 

14. Data Reporting 

, 5. Corrective Action 

16. Method Precision and 

Accuracy 

A. TECHNICAL SECTIONS 

Typical Examples 

Sample preparation, collection, storage, transport, 

and data sheets 

Preparation of standards and standard curves, 

frequency and schedule of calibrations 

Standard and custom-tailored methods for all 

analytes in all matrices 

Data reduction, validation, and statistical 

treatment, including confidence levels and -outliers 

Selection criteria, format, equations, units 

Criteria for initiation; individuals responsible 

Tabular or narrative summary 

B. QUALITY CONTROL SECTIONS 

Section 

1. QC Checks 

2. QC Controls 

Typical Examples 

Precision, accuracy, repeatability, reproducibility, 

blanks, spikes, replicates, selection criteria, and 

frequency summarized in tables 

Audits, notebook checks, blind samples; control 

charts and graphs; actions to be taken when QC 

data approaches or exceeds QC limits 

C. REFERENCE SECTION 

Standard reference methods, reports, SOPs, 

journal articles; avoid citing unpublished 

documents -
3.3 HOUSEKEEPING DETAILS 

Once an organization commits to SOPs, many new SOPS will be prepared in the 

same length. of time it took to do the first one. And, as refinements become 

available, older SOPs will need to be updated, preferably without having to rekey 

the entire text. A tracking system is a must in handling this ever-increasing 

workload. 

. Initially, a simple system shown in Table 3-2 will be sufficient.· The title should 

be as specific as possible; generic titles such as "Atmospheric·Monitoring" usually are 

too broad to be truly descri ptive of the SOP. 

TABLE 3-2. TRACKING SYSTEM FOR STANDARD OPERATING PROCEDURES 

General Information 

SOP Number 

Title 

Date 

Specific Example 

Site Selection Criteria for Meteorological 

Monitoring at Heavily Forested Areas 

July 1, 1992 

To accommodate later revisions, however, a more detailed "document control 

format" is frequently used for tracking documents from the very beginning of the 

SOP program. The information shown in Table 3-3 is placed on the upper right-hand 

corner of each page. 

TABLE 3·3. DOCUMENT CONTROL FORMAT 

General Information 

SOP Number 

Section Number 

Revision Number 

Date of Issue 

Page of 

Specific Example 

SOP-25 

Section 3 

Revision NO.1 

July 29, 1992 

Page 5 of 12 

3-4 
The original version is always labeled as Revision O. If, for example, page 5 of 

Section 3 needs to be updated, the changes are made and issued as Revision 1, 

together .with instructions to replace page 5 of Revision 0 with the new page 5 of 

Revision 1. Thus, the value of a ring-binder format becomes obvious. 

A complete set of SOPs is stored for reference in one place, usually the office of 

the QA manager. The most important copies, however, are the dog-eared, coffeestained ones in the field and laboratory; SOPs are meant to be used, riotj"ust tilea. 

3-5 

SECTION 4 

PREVENTIVE MAINTENANCE 

Because breakdowns and repairs use up the time needed for preventive 

maintenance, buying reliabl.e equipment is the best way to guarantee enough time 

. for planned maintenance. Reliable equipment, which does the job right (almost) 

every time, has fewer breakdowns and requires less time for troubleshooting. 

Several steps are involved in getting reliable equipment. 

• Procurement: Ordering the Rright stuffR 

• Inspection: Checking that everything came in 

• Control: Knowing its whereabouts at all times 

• Testing: Proving it does what it should do 

• Training: Teaching the operators how to use it 

Once these steps are carried out, the equipment and the project should run 

smoothly, with little downtime for repairs·. 

, . 

Merely setting up a detailed schedule of preventive maintenance is not 

enough; actually following it is the critical step. Auditors pay particular attention to 

whether planned maintenance activities were indeed performed. Because 

individual air pollution and meteorological monitoring methods include detailed 

descriptions of r~quired preventive maintenance, this section focuses only on 

features common to all methods.· 

4.1 EXAMPLES 

Many types of preventive maintenance are needed to achieve good data 

quality. The following are only a few examples. 

• Clean the sample manifold 

• Replace vacuum pump filters 

• Lubricate pump box blower motors 

• Change data tape 

4-1 
Each activity by itself may seem insignificant, but, when coupled with dozens of 

others, the net result is a program with more reliable data, less downtime, and much 

less cost in dollars, time, and grief. 

4.2 REQUIREMENTS 

A good preventive maintenance program must include the following items. 

• Short description of each procedure _ 

• Schedule and frequency for performing each procedure 

• Supply of critical spare parts on hand, not merely on a list 

• List of maintenance' contracts for instruments used in critical 

measurements 

• Documentation showing that maintenance has been performed as 

required by the maintenance. contract, QA project plan, or test plan 

For convenience, summarize as much of this information as possible in tables. 

4-2 
SECTION 5 

SAMPLE COLLECTION, HANDLING, AND ANALYSIS 

At first glance, covering sample collection, handling, and analysis in one section 

sounds like a tall order. But because sampling and analysis share so many 

characteristics - calibration, contaminatioD, and sample custody, to mention-only a 

few - considering them as a unit is logical. Because oth~r sections of the QA 

Handbook describe individual methods in greater detail, this one can examine the -

underlying principles common to all. These principles are first summarized in 

Table 5-1, then discussed briefly in the following sections. 

TABLE 5·1. PRINCIPLES OF SAMPLE COLLECTION, 

HANDLING, AND ANALYSIS 

1. Select Sampling Sites Based on Data Quality Needs 

2. Understand the Reasons Behind the Procedures 

3. Use the Same Conditions for Standards and Samples 

4. Use Quality Control Checks and Standards 

5. Know Where the Samples Are and Be Able to Prove It 

5.1 SELECT SAMPLING SITES BASED ON DATA QUALITY NEEDS 

Although convenience and previous use are attractive features of any sampling 

site, the driving force behind site selection must be the data quality needs of the 

project. If a site cannot provide suitable samples, it is useless for the project. Once 

project needs are specified, a statistician should be consulted for help In site 

selection; sampling strategy; and the type, frequency, and number of samples 

required to attain the desired level of confidence in the results. 

5.2 UNDERSTAND THE REASONS BEHIND THE PROCEDURES 

All procedures should explain why certain steps are used, not just how to 

perform them. For example, here are only a few of many precautions taken to 

prevent contamination during the cI~aning and handling of air monitoring 

equipment and samples: glass fiber, quartz, or Teflon filters are handled with 

5·' 
· tweezers, not bare hands; clean cotton gloves, not surgical rubber gloves with 

potentially contaminating powder, are also used to handle the filters; dedicated or 

disposable glassware is used for standards; and glassware for anion analysis is not 

cleaned with soap, which could leave a residue containing anionic contaminants, but 

with multiple rinsings of deionized water. Similar explanations should be a part of 

all procedures, especially SOPs. The more reasons that are given, the more likely the 

procedure will be understood, appreciated, and followed. 

5.3 USE THE SAME CONDITIONS FOR STANDARDS AND SAMPLES 

Simple as this admonition sounds, it goes unheeded all too frequently in both 

field and laboratory. For example, suppose the expected concentration of an 

analyte is around 200 ppm. Even a careful calibration in the 0 to 20 ppm range is 

meaningless at the 10-fold higher concentration. Calibrations must be made over 

the full span of expected concentrations. Gas cylinders and regulators need to 

equilibrate for at least 24 hours· to adjust for changes in temperature and altitude 

before being calibrated and used. Leak checks must be made under the same 

pressure to be used during data collection. Only when standards are subjected to 

the same treatment as the samples can meaningful data be obtained. 

5.4 USE QUALITY CONTROL CHECKS AND STANDARDS 

Quality control checks and standards show when the system is out-of-control 

and corrective action is needed. High-quality precision and accuracy data are 

derived from blanks, replicates, spikes, standards, and other QC checks. Calibration 

standards, which should be verified regularly, are also used throughout sampling 

and analysis. To avoid the possibility of being precise but not accurate, QC check 

samples should nofbe the same ones used for calibration standards. 

5.5 KNOW WHERE THE SAMPLES ARE AND B'E ABLE TO PROVE IT 

Proof is especially important for high visibility projects where litigation is a 

distinct possibility. Strict sample custody procedures protect against losses, mixups, 

accidental contamination, and tampering. Although good sample labels, custody 

seals, and tracking sheets are essential for maintaining sample integrity, dedicated 

sample custodians are the most important factors. Chain-of-custodyforms must be 

used for all sample transfers, not only between field and laboratory, but also from 

one field (or laboratory) group to another. Projects of lesser visibility also benefit 

from similar, though less stringent, procedures. 

5-2 

- .---------------------------..:.-
SECTION 6 

DATA COLLECTION AND HANDLING 

Entire books have been written on data collection, validation, reduction, 

analysis, storage, and retrieval, yet this chapter covers the same topics in only a f~w 

pages. How? By focusing on the fundamental, princi,ples comm'on to many of these" 

steps in the data-gathering process. These principles are ffrst summarized in Table 6-

1, then discussed briefly in the following sections. 

TABLE 6·1. PRINCIPLES OF DATA COLLECTION.AND HANDLING 

1. Know Why the Data Must Be Collected 

2. Document Everything Thoroughly 

3. Calibrate Instruments and Test Software 

4. Preserve the Original Data 

5. Use Only Validated Data 

6. Use Tables or Graphs to Present Summary Statistics 

7. Leave Sophisticated Data Handling Techniques to the Statisticians 

8. Beware of Using Data Collected for Another Purpose 

6.1 KNOWWHYTHE DATA MUST BE COLLECTED 

How data will be used dictates how they must be collected. Consider, for 

example, just a few of the many questions to be answered before beginning air 

'monitoring studies: How many sites? Are "all, sites equally important, or are some 

more important than others? Will sampling be continuous or episodic? Over what 

time period? How many samples are neede~? Statistical expertise is required to 

answer questions like these and to design a cost-effective data collection program 

that will yield data good enough for confident decision making. 

6-1 
6.2 DOCUMENT EVERYTHING THOROUGHLY 

From data collection through data use, the motto is "Write it down!" Nothing 

enhances the credibility of a data collection program more than thoroug'hly detailed 

documentation. Data usability, for future as well as present applications, depends 

on how well all ofthe details are recorded. 

6.3 CALIBRATE INSTRUMENTS AND TEST SOFTWARE 

Improperly calibrated instruments frequently cause, poor results. ,All 

calibrations must be directly traceable to a standard of recognized accuracy, such as 

those from the National Institute of Standards and Technology. All calibrations must 

also include a .zero-span check covering the full range of concentrations expected 

during data collection. Linearity of instrumental response must be demonstrated,-

not assumed. Software, too, must be tested thoroughly, to verify that it is 

performing as planned. If not, data collection, validation, reduction, and analYSis 

can be jeopardized. 

6.4 PRESERVE THE ORIGINAL DATA 

Whatever is done in data processing, especially in data reduction, the original 

data must be preserved and all derivative data must be directly traceable to them. 

All data transformations must also be preserved. Back-up files, whether computer or 

manual, are mandatory. Only protected data allow a second chance for analysis if 

critical problems arise on the first attempt. 

6.5 USE ONLY VALIDATED DATA 

To catch data errors and biases at the earliest_possible stage, data validation is 

,used to compare each data point against prespecified criteria. Whether performed 

by humans or computers, during or after data collection, it asks the question "Is this 

specific piece of data reasonable?" Only validated data can proceed to the next 

step. Abnormally high or low values cannot be discarded automatically. Instead, 

they must be examined statistically to determine if they truly fall outside the 

expected range. They may be real values on the .tails of a distribution curve or they 

may be invalid as shown by standard tests. ,Or, as sometimes happens, their 

occurrence is simply unexplainable. Decisions to use or discard suspect data can be 

made only after these validity checks. 

6-2 
6.6 USE TABLES OR GRAPHS TO PRESENT SUMMARY STATISTICS 

Air monitoring studies, particularly those with mUltiple stations and automatic 

recording devices, produce vast quantities of data impossible to comprehend in the 

raw state. Trends become apparent only after data are reduced and tables or graphs 

are used to present summary statistics. Graphs are frequently more informative than 

tables for presenting numerical data bec.ause patterns and magnitudes are easier to 

comprehend. Statistics used most often are the number of observations, means, and 

standard deviations, with others included as needed. Presenting numerical data in 

narrative form throughout a report is a poor alternative because the 

interrelationships among scattered data are easily lost. 

6.7 LEAVE SOPHISTICATED DATA HANDLING TECHNIQUES TO THE STATISTICIANS 

Amateur statistics can be nearly as dangerous as amateur surgery. Powerful 

software packa.ges are widely available for data validation and analysis, but using 

them without a thorough understanding of their limitations and underlying 

statistical assumptions almost guarantees severe over- or under-interpretation of the 

data. Key topics such as graphical display of data, identification of outliers, 

regression analysis, analysis of variance, and how to handle zero or nondetected 

data require advanced statisti~al techniques. To extract the maximum information 

from a data set, statisticians must participate in the design phase too, rather than 

just the data analysis. 

6.8 BEWARE OF USING DATA COLLECTED FOR ANOTHER PURPOSE 

The temptation to use existing data rather than collect new data is especially 

strong when budgets are tight. Succumbing to that temptation can be disastrous, 

unless all of the restrictions applicable to the previous data are known and 

documented. 

6-3 

SECTION 7 

STATISTICAL TERMS'AND DATA QUALITY INDICATORS 

Previous sections have discussed data qualitatively. This section summarizes 

how data are described quantitatively by statistical terms and data quality 

indicators. Definitions and equations are ~ccompanied.by brief descriptions of th~ 

conditions when the specific terms should or should ·not be used. For ease in 

reference, the equations are numbered at the right of the page. 

7.1 STATISTICAL TERMS 

In Volume 1, Principles, of the first edition of the Quality Assurance Handbook 

for Air Pollution Measurement Systems, there were almost 200 pages dealing with 

statistics. Here they have been condensed to less than 6, which no doubt will cause 

consternation to some. But this is a field guide, and a field guide covers only the 

most important things. 

7.1.1 Arithmetic Mean 

Whenever data plots show a roughly symmetrical (bell-shaped or normal) 

distribution, the average value 'is called the arithmetic mean. ·It is simply the sum of 

the individual values divided by the number of values in the data set: 

where 

X = arithmetic mean 

n = number of values 

Xi = individual data values 

- 1 x= - !:X_ (1) 

n l 

Calculating the arithmetic mean without first plotting the data to verify a 

symmetrical distribution can lead to faulty data interpretation. See Section 7.1.3 for 

a discussion of when the arithmetic mean is particularly inappropriate. 

7-1 
7.1.2 Standard Deviation and Variance 

The standard deviation, used to measure the dispersion or spread of data, is 

defined as fqllows: 

8= 

where 

s = standard deviation 

Xi = individual data values 

n = number of values 

2 2 EX. - (tX,) In , , (2) 

n-l 

The square of the standard devi'ation, called the variance, is another frequently used 

measure of data dispersion. 

Programmable calculators require only that the raw data be entered in a 

specified manner. All computations are then performed automatically. Thus, in 

actual practice, it is no longer necessary to manually compute the tedious squarings 

required by Equation 2. 

7.1.3 Geometric Mean 

Plots of air monitoring data frequently show a skewed, nonsymmetrical 

distribution. For these cases, the g'eometric mean rather than the arithmetic mean is 

a better measure ofthe average value. The geometric mean is defined as the antilog 

ofthe average of the logarithms of the data values: 

- 1 X = antilogb (- E logbX,) 

g n I 

(3) 

where 

Xg = geometric mean 

n = number of values 

logtXi = logarithms of individual data values 

Either common, logarithms (log10) or natural logarithms (loge) can be .used to 

calculate the geometric mean. The necessary tables of logs and antilogs are found in 

mathematics and statistics textbooks and in standard reference books such as the 

Handbook of Chemistry and PhYSics. Software pr~grams are also available. 

7-2 
7.1.4 Geometric Standard Deviation 

Th@ geometric standard deviation, used when data are distributed lognormally 

rather than' normally, is ~efin@d as follows: 

. where 

s = antilOg 8 

Sg = geometric standard deviation 

logXi = logarithm of indiv,idual data values 

n = numberofvalues 

7.2 DATA QUALITY INDICATORS 

(4) 

"How good are the data?" Because project success depends on the answer, 

data quality is used as an indicator of project performance. Six terms frequently 

used to describe data quality are precision, accuracy, completeness, method 

detection limit, representativeness, and comparability. Each is defined in the 

following sections, but, as shown there, the definitions are not always quantitative 

or universally accepted. Nevertheless, the definitions do provide a common ground 

for discussions on data quality. ' 

7.2.1 Precision 

Precision is a measure of agreement among two or more determinations of the 

same parameter under similar conditions. Two terms used to describe precision are 

relative percent difference (RPD) and relative standard deviation (RSD) (also called 

the coeff.icient of variation), depending on whether two or mOJe than two replicates 

are used. ' 

If precision is calculated from duplicate measurements, use 

where 

RPD 

XI 

X2 

100 (Xl - X.) 

RPD=----

(Xl + X.}/2 . 

= relative percent difference 

= larger of the two values 

= smaller of the two values 

7-3 

(5) 
If precision is calculated from three or more replicates, use RSD rather than 

RPD: 

RSD = 100 (siX> (6) 

where 

RSD = relative standard deviation 

s = standard deviation (see Equation 2) 

X = mean of replicate analyses 

For two replicates, RSD = RPD/V2 

7.2.2 Accuracy 

Accuracy is the degree of agreement between a measured value and the true, 

expected, or accepted value. It is frequently expressed in terms of percent recovery 

(% R) whether Standard Reference Materials (SRMs) or spiked samples (known 

concentrations of test materials added to samples) are used. 

If SRMs are used, accuracy is expressed as follows: 

where 

%R 

CM 

CSRM 

= percent recovery 

= measured concentration of SRM . . 

= actua I concentration of SRM 

When spikes are added to samples, %R is calculated as follows: 

%R = 100 (C - C )lC , U BO 

wher~ 

%R :;; percent recovery, 

Cs = measured concentration in spiked aliquot 

Cu = measured concentration in unspiked aliquot 

Csa = actual concentration of spik.e 

7-4 

(7) 

(8) 
When measurement systems for ambient air monitoring are audited, accuracy 

is expressed as follows: 

RPD = 100 (C - c)le m II II 

where 

RPl) = relative percent difference 

Cm . - measured value of audit standard 

Co. = actual value of audit standard 

7.2.3 Completeness 

(9) 

Completeness is a measure of the amount of valid data obtained compared 

with that expected to be obtained under normal operating conditions. It is defined 

as follows for all measurements: 

%C = 100 (n In) .11 (10) 

where 

%C = percent completeness 

nv = number of valid measurements 

n = total number of planned measurements 

The above equation is a simplified definition. In actuality, %C must be tied to the 

specific statistical level of confidence needed'for decision making. Obviously, a 

decision needing, saY,a 99% confidence level needs more valid data than one 

requiring only an 80% level. A statistician should be consulted for guidance on this 

topic. 

7.2.4 Method Detection Limit 

. The method detection limit (MOL), the lowest concentration of an analytethat 

can be measured by a given procedure, is as much a statistical as an analytical 

concept, and there are numerous definitions. One definition favored by statisticians 

is as follows: 

where 

MDL 

s 

t(n-I, I-a = 0,99) 

MDL = st (Il-l. I-a"" 0.99) (11) 

L 

= method detection liniit 

= standard deviation' of the replicates at the lowest 

concentration 

= Student's t-value appropriate to a 99% confidence level 

and a standard deViation estimate with n-1 degrees of 

freedom 

7-5 
Field and laboratory personnel frequently use a much simpler version: 

MDL = (N)(s) (12) 

where 

MDL = method detection limit 

N = a multiplier between 3' and 10 

s :: standard deviation 

7.2.5 Representativeness 

Representativeness expresses how closely a sample reflects the characteristics 

of the substance for which it is a surrogate. Ideally, the representativeness of the 

sample would be 100%; practically, however, the quantitative value is rarely known. 

Every effort is made to ensure that the sample is truly representative, by using such 

techniques as thorough mixing to obtain homogeneity, duplicate analyses, and such. 

Problems with uniformity are not so great with air samples as with liquids or solids 

because of the nature of the air media. 

7.2.6 Comparability 

Comparability refers to how confidently one data set can be compared with 

another. Ideally, all data would be completely comparable, so comparability would 

be 100%. Practically, because t~e data were collected under different conditions 

and for different purposes, comparing data sets must be done very cautiously. See 

Section 6_8 for more details. 

7-6 
SECTIONS 

AUDITS 

Managers need to know how well things are going on their projects. Is a 

particular project performing according to specifications? An audit, a management 

tool used to answer that question, is a formal, detailed .study of one or more aspects 

of a project by independent auditors. The ·project is not audited at random, but 

against specific criteria previously determined by the manager to be critical to 

project success. Many audits are held shortly after the project has become 

operational, to detect and correct problems before they affect data quality 

adversely. 

A cooperative effort of auditors and auditees (to gather the needed 

information efficiently and completely) gives the best results. There is no room for 

"Gotcha!" in any audit. 

The audit report describes any problems found and may suggest appropriate 

corrective actions. Equally important, it also covers those aspects that were 

operating as specified. Thus, t~e manager learns what is going well, not just what 

needs attention. 

An audit focuses on one or more of the following components of a project. 

• People 

• Proced u res 

• Equipment 

• Data 

• Documentation 

The success of any project depends on how well the people follow procedures, 

operate equipment, collect and interpret data, and carefully document their 

activities. 

8.1 DOCUMENTATION 

A poor paper trail can lead to even poorer audit results. During their on-site 

visit, auditors can observe only the cUrrer.1t operations first hand; for previous ones, 

they must depend on written documentation. Verbal assurances from the auditees 

8-1 
are not enough to convince the auditors that proper procedures had, indeed, been 

-followed. Only clear, complete, written documentation can do that. 

8.2 AUDIT TYPES 

The QA project plan is the basis for all four audit types described in the 

following sections. Although the audit is used to determine w~ether criteria 

stipulated 1n the plan are being met, any. additional findings are also included in the' 

report. 

8.2.1 Technical Systems Audit 

The technical systems audit, a qualitative on-site evaluation of an entire 

measurement'system, is used frequently in an air monitoring program, It looks at 

everything - all facilities, equipment, systems, record keeping, data validation, 

operations, maintenance, calibration procedures, reporting requirements, and QC 

procedures. ,Findings from this,global revie~ ca~ then be used to focus efforts on 

specific parts of the measurement system that need attention to obtain the desired 

data quality. Systems audits are normally done immediately before, or shortly after, 

measurement systems are operational, and should also be performed on a regularly 

scheduled basis throughout the lifetime of the project. 

8.2.2 Performance-Evaluation Audit 

The performance evaluation audit, also used frequently in air monitoring 

studies, is a quantitative evaluation of a part or parts of a measurement system, 

including all associated data acquisition and reduction procedures. It involves the 

analysis of a reference material of known value or composition and critical to the 

success of the project. -The reference material is usually disguised as a typical proj~ct 

sample so that the operator or analyst will riot give it any undue special attention, 

Long-term projects require regularly scheduled performance audits. Although a 

performance audit may show that a system is out-of-control, a systems audit may be 

needed to pinpoint the cause and target the corrective action. 

8.2.3 Audit of Data Quality 

An audit of data quality exhaustively evaluates the methods used to collect, 

interpret, and report data quality. The following criteria are evaluated against the 

QA project plan and other pertinent guidelines: 

8-2 
• Recording and transfer of raw data 

• Calculations, including equations used for presentation of data 

• Documentation of data-handling procedures 

• Selection and discussion of data-quality indicators, including precision, 

accuracy, representativeness, comparability, and completeness 

8.2.4 Management Systems Audit , 

A management systems audit (or review) examines the structures and processes 

used by management to achieve the desired data quality. Broad in scope, it 

frequently covers multiple projects within a larger program. laboratory and field 

personnel rarely participate directly in this type of audit. 

8.3 AUDIT PROCEDURES 

Detailed planning is the essence of any good audit. Without it, the resulting 

chaos causes short tempers and sloppy work; with it, the ensuing cooperation fosters 

harmony and success. In addition to auditor and auditee, a third party, the sponsor, 

plays a key role. As commonly occurs in government and industry, a sponsor funds 

the project and requests the audit. The following sections describe critical 

interactions among these three,parties. If only auditor and auditee are involved, the 

audit procedure is simpler because the auditor assumes the functions of the sponsor. 

8.3.1 Preaudit Activities 

Decisions made by the sponsor in the preaudit planning phase determine the 

course of the audit. As shown in the following summary, all, three parties 

communicate extensively to ensure that there will be no hidden agendas and no 

surprises. 

A. RESPONSIBILITIES OF THE SPONSOR 

The sponsor's project manager and QA manager decide on the following 

audit details. 

(1) Intent, scope, cost, and frequency o-f auditing activities 

(2) Parts of project to be audited 

(3) Audit schedule 

(4) Qualifications needed for auditors 

8-3 
(5) Action to be taken by auditors if they discover out-of-control 

situations 

(6) Potential for organizational conflict of interest between auditors, 

and auditees 

(7) Selection of proposed auditors 

Of these items, (5) is the most critical. Out-at-control ,situations can arise, ' 

in the tield, the laboratory, or in data handling operations, What shoutd 

the auditors do? Correct the problem immediately and cite it in the 

report? Take no corrective action and cite the problem in the report? 

Use some other approach? Whatever the answer, it must be spelled out 

and agreed upon by all parties before the audit can begin. The sponsor's 

project manager then notifies the auditee of the purpose and scope of 

the audit and requests comments on the following items. 

(8) Acceptability of preceding points (1) through (7) 

(9) Actual or perceived, current or potential, conflicts of interest 

(10) Necessity for a preaudit, face-to-face meeting of auditor, auditee, 

and sponsor 

(11) Location, date, and time of meeting, if requested in item (10) 

B. RESPONSIBILITIES OF THE AUDITEE 

The auditee or the sponsor's project manager then sends the following 

information to the auditor. 

(1) Details of project operation (SOPs, site locations, QA project plan, 

operator proficiency and training, sampling schedule, etc.) 

(2) Name of person to contact for additional information 

C. RESPONSIBILITIES OF THE AUDITOR 

The auditor responds by sending the following information to the auditee. 

(1) Standard operating procedures to be, used in the audit 

(2) Parts of the project to be audited, and by whom 

(3) Qualifications of the auditors 

8-4 
(4) Name of person to contact for additional information 

(5) Authority and responsibility of the auditors to take action if a 

problem is found . . 

Note: All parties must address all of the above points and come 

to an agreement on them before the audit begins. 

8.3.2 Conducting the Audit 

The audit should proceed smoothly because of the preaudit agreements. Steps 

in the actual audit are as follows. 

A. The audit is conducted according to the preaudit agreements. If any 

party feels that changes are needed, it must then notify all other 

parties and gain approval before deviating from the agreements. 

B. Auditor informs auditee (on site or by phone/faX/E-Mail, as 

appropriate) of preliminary audit findings and recommendations for 

corrective action. . 

C. Auditor tries to resolve any disagreements before feaving the site. 

D. If disagreements between auditee and auditor cannot be resolved, 

auditor contacts spon.sor's project manager, QA manager, or the 

auditee's project manager, depending on the preaudit agreements. 

E. In the audit report, the auditor includes the outcome of this 

postaudit discussion and identifies still unresolved disagreements. 

8.3.3 Preparation of the Audit Report 

An audit report is the last step in the auditing process. As shown in the 

sequence below, the auditee has significant input. 

A. Auditor briefs sponsor's project manager and QA manager on the 

audit findings. 

B. Auditor prepares draft audit report and submits it, and all supporting 

data, to the QA manager. 

C. The QA manager determines if th~ report meets the sponsor's 

guidelines for clarity, accuracy, completeness, etc. (If not, the report 

is returned for revision.)' . 

D. Once the draft report is accepted by the QA manager, it is sent to 

both the sponsor's project manager and to the auditee. 

8-5 
E. The sponsor's project manager and the auditee send their written 

comments to the QA manager, not to the auditor. 

F. After reviewing the comments, the QA manager discusses them with 

the auditor, and, 'if necessary, arranges a meeting of all appropriate 

parties. If disagreements remain, the QA manager will recommend 

to the sponsor a course of action such as 

(1) Repeat the part of the audit in question; 

(2) Issue the audit report, but include a statemtmtthat the auditee 

has questioned a particular audit finding; or ' 

(3) Delete the item(s) under question from the report. 

If disagreements still remain, the sponsor's project manager receives the final, 

report only after the sponsor has approved the proposed course of action. If 

there are no disagreements, the QA manager releases the final report to the 

sponsor's project manager, with 'a copy to the sponsor and the auditee. 

8.3;4 Postaudit Report Activities 

The audit report is not the end of the audit. If major problems were 

discovered, the auditee must institute corrective action (see Section 9). If the 

problems were critically compromising to data quality, a special follow-up audit 

might be necessary to verify that the corrective action was adequate to allow data 

collection to resume. Corrective actions for minor problems are checked at the next 

regularly scheduled audit. 

8-6 
SECTION 9 

CORRECTIVE ACTION 

Few projects run perfectly; fewer still automatically correct the many problems, 

large and small, that inevitably arise. For that, competent, responsible people are 

required. Both assigning and accepting responsibility are critical to the succe5s of 

. any corrective action plan .. 

9.1 ROUTINE MEASUREMENTS 

Many corrective action plans are already embedded in the QC checks used for 

all routine measurements. Acceptance criteria or tolerance limits are contingency 

plans that state that "If this happens, then WE will do the following:". The nWEn 

cannot be left unspecified in the corrective action plan; a person or persons 

(chemical analyst, stack sampling operator, etc.) must be designated by title or 

function, and, if possible, by name. A statement such as nlf this measurement 

activity is out of control, all sampling will be stoppedn is unacceptable because it 

does not indicate who is responsible for making that decision. 

Field and laboratory personnel will be able to make most of the corrective 

actions needed. They must then document these actions in the appropriate 

notebooks or logbooks so that a record exists of the problems encountered and the 

solutions discovered. 

9.2 MAJOR PROBLEMS 

Sometimes, however, problems occur that field and labt?ratory staff members 

are unab~e to solve, despite their best efforts. ,These problems can aris~ during 

routine operations or as a result of performance evaluation and technical systems 

audits. Staff members must immediately bring these major problems to the 

attention of their supervisor or other individuals designated in their test or QA 

project plans to handle the problem. Because many individuals could become 

involved in the corrective action, the notification is best done by a standard 

corrective action form, a copy of which is shown in Figure 9-1. 
CORRECTIVE ACTION FORM 

Project Title _________ -'--__ _ Project No. -----

I. REQUEST FOR ASSISTANCE 

To: A Date: _ .. ---------- From: _--=B:,...-_ Signature: ________ _ 

Problem: (1) Nature 

(2) Suspected Cause --------------------

II.. PROPOSED CORRECTIVE ACTION 

To: B Date: ---------- From: _---.A..:....-_ Signature: _______ _ 

Suggestion: _________ ---'-______________ _ 

III. RESULTS OF PROPOSED CORRECTIVE ACTION 

To: A Date: --------- From: _---=B;......._ Signature: ~-------

Results: 

Figure 9·1. Corrective Action Form. 

9-2 
The form has three parts: 

• Request for Assistance 

• Proposed Corrective Action 

• Results of Proposed Corrective Action 

A three-part, no-carbon:-required, corrective action form is highly recommended, 

especially for field use, where photocopiers are rarely available. Space is provided 

for signatures and a brief outline of the problem, the proposed solution, and the 

results. Each person signing the form should feel free to attach any other needed 

material, but must also keep a copy of the complete packet in his or her own filesfor 

ready access should a similar problem arise. 

9-3 

SECTION 10 

BIBLIOGRAPHY 

1. American National Standard: Generic Guidelines for Auditing of Quality 

Systems. ANSI/ASQC Standard Q1-1986. American Society for Quality Control, 

Milwaukee, Wisconsin. 1986. 13 pp. 

Detailed description and flow charts of the entire auditing process. 

2. Dixon, W,J. and F,J. Massey, Jr. 1969 Introduction to Statistical Analysis, ThirdEdition. New York, NY: McGraw-Hili, Inc. -

-A clearly written basic textbook, still a standard for statistical analysis. 

3. Interim Guidelines and Specifications for Preparing Quality Assurance Project 

Plans. QAMS-005/80. U.S. Environmental Protection Agency, Washington, DC. 

1980. 

The ancestor of all U.S. EPA guidance' documents for preparing QA 

project plans. Still the standard, but will be replaced in 1993 by a 

completely new document consistent with recommendations of the 

American Society for Quality Control and the American National 

Standards Institute. 

4. Porter, L.F. Guideline. for Design, Installation, Operation, and Quality 

Assurance for Dry Deposition Monitoring Networks. EPA/600/3-88/047. U.S. 

Environmental Protection Agency, Research Triangle Park, NC. 1988. 500 + pp_ 

Practical guidance on the design of monitoring projects and 

associated QA/QC. Lengthy and detailed, but a readable format. 

Covers all phases from design to installation to implementation of 

monitoring networks. 

5. Simes, G.F. Preparing Perfect Project Plans: A Pocket Guide for the 

Preparation of Quality Assurance Project Plans. EPA/600/9-89/087. U. S. 

Environmental Protection Agency, Cincinnati, OH. 1989.62 pp. 

A companion document to those listed in item (6) below. Uses fourcategory approach to project classification and associated QA. For 

each category, provides hints and checklists for preparing QA project 

plans. , 

6. Simes, G.F. Preparation Aids for the Development of Category I (II, III, IV) 

Quality Assurance Project Plans. ,EPAl600/8-911003, EPA/600/8-91/004, 

EPAl600/S-91/005, EPAl600/S-91/006. U. S. Environmental Protection Agency, 

Cincinnati,OH. 1991.65,63,57,35 pp., respectively. 

Companion documents to that listed in item (5) above. Uses a fourcategory approach to project c1assification and associated QA. For 

10-1 
each category, gives more detail than the Pocket Guide. Includes 

many examples of summary tables for presenting QA requirements. 

7. Quality A$surance Handbook for Air Pollution Measurement Systems. U.S. 

Environmental Protection Agency, Research Triangle Park, NC. Dates variable. 

Volume I Now: A Field Guide to Environmental Quality Assurance. 

1993. Replacement for: Principles. 

EPA-600/9-76-00S,1976. 

Volume II Ambient Air Spedfic Methods. 

EPA-600/4-77-027a,1977. 

Volume III Stationery Source Specific Methods. 

EPAl600/4-77-027b,1977. 

Volume IV Meteorological Methods. 

E PA/600/4-90/00 3, 1989. 

Volume V Manual for Precipitation Measurement Systems. 

EPA-600/4-82-042a & b, 1983. 

A large 5-volume compendium of all things related to ambient air 

measurements, including, but not restricted to, test methods, 

sampling, analysis, and QA. Individual methods revised occasionally 

since 1976, but many are both out of date and out of print. Use the 

EPA document numbers shown above to obtain the most recent 

updates if available. Major revisions are planned for the entire 

Handbook. Those for Volume I are completed with publication of 

this field guide. Work on Volumes II and III has begun. Volumes IV 

and V, published more recently, will be addressed later. Pending 

funding, the target for completing all revisions is the mid-1990s. 

8. Quality Assurance Project Plan for the Atmospheric Research and Exposure 

Assessment Laboratory. U.S. Environmental Protection Agency, Research 

Triangle Park, NC. 1990. 

Example of a 'broad gUidance document for an entire laboratory 

rather than a single project. Emphasizes job descriptions, QA 

oversight requirements, and protocols for the preparation and 

review of a wide variety of documents. Other U.S. EPA laboratories 

have similar plans tailored to their special needs. 

*U.S. GOVERNMENT PRINTING OFFICE: 1994 • 550-00IIII03U 

10-2