System Audit of the Ambient Monitoring Program:

San Diego County Air Pollution Control District

September 28 – September 30, 2010

Conducted by:

Air Quality Analysis Office Quality Assurance Office 

Air Division Management and Technical 

Services Division

US EPA Region 9

75 Hawthorne Street

San Francisco, California 94105

Final Report: April 2012
ii

INDEX

PAGE

GLOSSARY OF ACRONYMS........................................................................iii

EXECUTIVE SUMMARY...............................................................................1

TSA ACTIVITIES.............................................................................................3

OVERVIEW OF AIR MONITORING PROGRAM .....................................7

FINDINGS.................................................................................................14

APPENDIX A, Summary of Findings.............................................................52

APPENDIX B, Good Practice Recommendations.........................................55

APPENDIX C, Organizational Chart........................................................56
iii

GLOSSARY OF ACRONYMS

APCD.....................................Air Pollution Control District

AQNS.....................................Air Quality Network Section

AQS....................................... Air Quality System

BAM...................................... Beta Attenuation Mass Monitor (continuous PM2.5)

CARB.....................................California Air Resources Board

CARE.....................................Community Air Risk Evaluation

CO..........................................Carbon Monoxide

CFR........................................Code of Federal Regulations

DMS.......................................Data Management System

DQO.......................................Data Quality Objective

EC.......................................... Elemental Carbon

EPA....................................... Environmental Protection Agency

ESC....................................... Environmental Systems Corporation

ETS....................................... Electronic Technicians Section

FEM....................................... Federal Equivalent Method

FRM....................................... Federal Reference Method

GC-MS...................................Gas Chromatography-Mass Spectrometry

MDL.......................................Method Detection Limit

MET/PE................................. Meteorology and Performance Evaluation

MTSD.................................... Monitoring and Technical Services Section

NAAQS..................................National Ambient Air Quality Standard

NAMS....................................National Air Monitoring Station

NIST.......................................National Institute for Standards and Technology

NOx/NOy...............................Nitrogen Oxides

O3........................................... Ozone

OC..........................................Organic Carbon

ORD.......................................Office of Research and Development

PAMS.....................................Photochemical Assessment Monitoring Stations

PE...........................................Performance Evaluation

PM..........................................Particulate matter

PM2.5...................................... Particulate matter 2.5 microns or less in aerodynamic 

diameter

PM10....................................... Particulate matter 10 microns or less in aerodynamic 

diameter

PQAO.....................................Primary Quality Assurance Organization
iv

QA..........................................Quality Assurance

QAPP..................................... Quality Assurance Project Plan

QC..........................................Quality Control

SPM....................................... Special Purpose Monitor

SOP........................................ Standard Operating Procedure

SLAMS.................................. State or Local Air Monitoring Station

SO2......................................... Sulfur Dioxide

SQL........................................Sample Quantitation Limit

SRP........................................ Standard Reference Photometer

STN........................................Speciation Trends Network 

TAD....................................... Technical Assistance Document

TSA........................................Technical System Audit

USEPA or EPA......................United States Environmental Protection Agency

VOC.......................................Volatile Organic Compound
1

EXECUTIVE SUMMARY

This document is a report on the findings made by the U.S. EPA Region 9 (EPA) while 

conducting a Technical Systems Audit (TSA) on the air monitoring program of the San Diego 

County Air Pollution Control District (hereafter referred to as “the District”). A TSA is an onsite review and inspection of a state or local ambient air monitoring program to assess its 

compliance with established regulations governing the collection, analysis, validation, and 

reporting of ambient air quality data. This TSA meets the requirements for EPA audits of the 

District’s monitoring organization required by 40 CFR Part 58, Appendix A, Section 2.5.

The TSA was conducted by EPA staff from September 28 through September 30, 2010.

The District Division that is the subject of this TSA is the Monitoring and Technical Services 

Division (MTSD). The audit team interviewed management and staff on specific aspects of the 

air monitoring program including network design, field operations, laboratory operations, data 

handling, quality assurance and quality control procedures. The audit team also inspected some 

of the monitoring sites operated by the District. The site inspections consisted of an interview 

with the site operator (when possible), review of station and instrument logbooks, and evaluation

of station siting with respect to EPA requirements for probe siting1

. The laboratory inspection 

included a review of the particulate matter program for mass determinations, laboratory analysis 

for volatile organic compounds (VOC), OC/EC (organic carbon/elemental carbon) and carbonyl 

sample analysis. 

The TSA is one of the ways that EPA provides oversight to ensure that data collected by 

state and local agencies meets certain minimum data quality objectives. Other assessments such 

as network reviews and performance evaluations are also used to collect information on the 

overall quality of ambient air monitoring data. These assessments also enable agencies to 

identify and correct those program elements which may be adversely affecting the quality of 

ambient air data. The results of the TSA are summarized here and fully described in this report, 

along with recommended actions to address the findings. The specific actions to be taken by the 

District will be determined through negotiations between EPA and the District and will be 

documented in a corrective action plan prepared by the District.

EPA would like to thank all the staff and management of the District for their support and 

cooperation during the audit.

A. Program Strengths:

The District has a robust ambient air monitoring program that is generally well-run. Some 

specific examples of the strength of the District’s ambient air monitoring program include:

• Sites are well set up with good access, and are clean, organized, and have spare parts onsite.

 

1 40 CFR, Part 58, Appendix E, “Probe and Monitoring Path Siting Criteria for Ambient 

Air Quality Monitoring”
2

• Site operators perform quality control (QC) checks every morning, ensuring quick 

identification of instrument problems and minimizing instrument down time and data 

loss.

• There is a good system in place to track preventive maintenance.

• There is good back up for field sites since field technicians rotate sites and build expertise 

for numerous sites in the network.

• There is effective, routine communication between sections within MTSD (e.g., morning 

coordination meetings).

• The current data management system for continuous data is very robust.

• In most cases, there is good training for technicians.

• For the TO-15 method, the laboratory has established a retention time of 30 days from 

canister cleaning and certification, after which canisters are to be re-cleaned prior to 

shipment and use in the field.

• OC/EC measurements are very robust: Standard Operating Procedures (SOPs) are written 

and adhered to and the analyses are performed using state-of-the-art instrumentation.

• Staff and management throughout the MTSD were found to be knowledgeable, diligent, 

and dedicated.

B. Program Major Findings:

During the TSA, EPA identified areas where the District’s monitoring program should be 

strengthened. The major findings are: 

1. There is not a central, independent, dedicated Quality Assurance (QA) Manager

responsible for communicating and ensuring that quality assurance activities are carried 

out in field operations, information management and laboratory operations. (Finding 1)

2. The District’s Quality Assurance program documentation, including a Quality 

Management Plan (QMP), Quality Assurance Project Plans (QAPPs) and SOPs, was not 

complete and was not approved internally or by EPA at the time of the audit. (Finding 2)

3. The District lacks a QA Officer position whose sole responsibility is to oversee quality 

management of the laboratory. (Finding 5)

4. Data management issues for particulate matter (PM) resulted in incorrect invalidation of 

PM data. (Finding 19)

The individual findings are reported in the topic sections of this document and are also 

summarized in Appendix A. In addition to those areas which should be strengthened, EPA also 

identified areas which could be strengthened and those recommendations and suggested best 

practices are included in Appendix B for the information and use of the District’s management.
3

TSA ACTIVITIES

From September 28 through September 30, 2010, the U.S. EPA Region 9 Air Quality 

Analysis Office and the Region 9 Quality Assurance Office conducted a Technical System Audit 

(TSA) of the District’s ambient air monitoring program. This TSA reviews one aspect of the 

District’s program responsibility, collecting and analyzing ambient air quality data, which is the 

responsibility of the Monitoring and Technical Services Division of the District.

The TSA covered the following program areas:

• General / Quality Management

- Program organization

- Facilities

- Independent quality assurance and quality control

- Planning documents (including QMP, QAPPs, & SOPs)

- General documentation policies

- Training

- Corrective action

- Quality improvement

- External performance audits

• Network Management / Field Operations

- Network design

- Changes to the network since the last audit

- Proposed changes to the network

- Field support

• Laboratory Operations: toxics and particulate matter

- Routine operations

- Quality control

- Laboratory preventive maintenance

- Laboratory record keeping

- Laboratory data acquisition and handling

- Specific pollutants: PM10, PM2.5, toxics, PAMS, and OC/EC

• Data and Data Management

- Data handling

- Software documentation

- Data validation and correction

- Data processing

- Internal reporting

- External reporting

EPA also tracked supporting documentation for data points/sets from calendar years 2008 and 

2009.

The EPA staff who conducted the TSA were Meredith Kurpius, Katherine Hoag, Michael 

Flagg, and Elfego Felix of the Regional Air Division’s Air Quality Analysis Office and Steve 

Remaley of the Regional Quality Assurance Office. Participating managers and supervisors of 

the Monitoring and Technical Services Division and positions at the time of the TSA were:
4

Mahmood Hossain, Director of Monitoring and Technical Services (MTSD)

David Shina, Supervisor of the Air Quality Network Section (AQNDS)

David Craig, Supervisor of the Electronic Technicians Section (ETS)

This report is divided the following sections:

• Executive Summary – describes the purpose of the TSA and a summary of the major 

findings; 

• TSA Activities – outlines the timing of this TSA and the programs that were covered;

• Overview of Air Monitoring Program – describes the District’s Air Monitoring Program;

• Findings – collection of positive, major, and minor findings that includes details 

associated with each finding;

• Appendix A – summarized list of findings with priority ranking;

• Appendix B – Recommendations that are not required by Federal regulation but would 

strengthen the District’s monitoring program; and,

• Appendix C – District organizational chart 

The Findings section includes positive findings along with findings that require corrective action.

The findings are grouped by program area. Recommendations to resolve findings are provided 

for each finding that requires corrective action to give some indication of the EPA expectations 

as to how findings can be addressed. If the District has other approaches or alternatives to 

address the concerns identified, EPA will consider them, provided the corrective action 

adequately addresses the finding. Appendix A is a summarized list of findings and each finding, 

excluding positive findings, is grouped according to priority, as either highest priority, moderate 

priority, or all other significant findings. Higher priority findings are those that: (1) are overarching issues, (2) have a greater impact on regulatory decisions, and/or (3) have a greater effect 

on data quality. Regardless of priority, all findings, excluding positive findings, need to be 

addressed in a corrective action report. EPA also identified areas that could be strengthened but 

corrective action is not required: these are listed as Good Practice Recommendation in Appendix 

B.

Network Management

EPA interviewed David Shina and reviewed the District’s Annual Network plan as part 

of this TSA. The most recent network plan, submitted in 2010, includes all of the required 

information about the monitoring stations. The air monitoring network within San Diego County 

meets or exceeds the requirements for the minimum number of monitoring sites designated as 

SLAMS for all of the criteria pollutants.

Field Operations

As part of this TSA, three Electrical Instrument Technicians in the ETS were 

interviewed, in addition to Mr. Craig. The Technicians interviewed were Arthur Agas, Noel 

Seefeldt, and Garnet West. We also interviewed Mr. Shina and observed a quarterly gaseous 

audit and a PM2.5 flow check at Escondido with David Roque and Larry Owusu. All four 

technicians and the auditor demonstrated a thorough knowledge of the monitoring equipment. In 

lieu of formal QAPPs and SOPs, the technicians had their own copies of District operational 

manuals, including information in three-ring binders that provided all critical information on 

instrument performance and acceptance criteria. 
5

EPA visited six of the District’s monitoring stations during the course of the TSA 

(Alpine, Donovan, Otay Mesa, Escondido, Camp Pendleton, and El Cajon). All sites met the 

siting criteria of 40 CFR 58, Appendix E. A more thorough evaluation was performed at all sites 

except Otay Mesa. The evaluation included inspection of the inlet manifolds, examination of 

station and instrument log books, and a check of whether appropriate QC checks and QA audits 

were being performed.

All five sites evaluated were well-run. All sites had clean inlet manifolds. Station 

logbooks and instrument logbooks were up to date and contained relevant information on 

operations and maintenance activities. The stations were clean and neat.

Toxics Laboratory Operations

The District’s laboratory staff is one of the most important assets of the laboratory. Some 

of the analysts/chemists are new to their positions, but were found to be properly credentialed, 

knowledgeable, skilled and dedicated. It was found that the District’s laboratory is in compliance 

with respect to performing analyses according to EPA methods in most areas of the laboratory, 

but does not have the benefit of a dedicated laboratory Quality Assurance Officer to oversee the 

review and documentation of quality assurance results and implementation of corrective actions 

on a consistent basis. This is believed to have contributed to the findings. Analysts working on 

the following programs were interviewed as part of this TSA.

Quality Assurance: All the laboratory staff and managers were asked about quality 

assurance in the laboratory.

OC-EC Carbon Analysis:

The analyst working on OC-EC has several years of experience with the analysis and 

recently returned from training at Desert Research Institute (DRI), where the carbon 

analyzer currently in use was designed and made. The laboratory adheres to the SOP 

written by DRI and updated its SOP to conform to the recently updated DRI SOP. From

the interview and observation, it was concluded that the analyst is skilled and 

knowledgeable and is working with state-of-the-art instrumentation and current SOPs.

The department is performing well and there are no findings requiring corrective action.

PAMS:

The analysts for the PAMS program and the PAMS Canister Cleaning Department were 

interviewed and found to be knowledgeable and generally doing an effective job with the 

resources available. Some deviations from guidance protocols have been made that were 

well thought out. However, EPA has some additional suggestions in some areas that are

presented in the findings.

Method TO-15 Analysis:

EPA interviewed staff conducting analytical work and cleaning canisters for the TO-15 

method. The TO-15 analyst is tenured, knowledgeable and skilled and generally adheres 

to guidance and good laboratory practice. The Department monitors the certification and 

ongoing stability of calibration standards, a current focus of the region. It is noted the 

laboratory is using a quadratic curve fit for the multi-point calibration in place of the 

linear multi point fit and use of average response factor specified in Method TO-15.
6

Method TO-15 is a guidance document. EPA evaluated the quadratic calibration process 

used by the laboratory and has determined it to be an acceptable deviation from the 

procedure described in method TO-15. An EPA statistician evaluated the calibration used 

by the laboratory and determined it to be acceptable. 

Carbonyls, Method TO-11 Analysis:

The Carbonyls Department is staffed by a new analyst who is knowledgeable and appears 

to have good organizational skills. The Department appears to be in a transitional period 

where procedures and documentation are being reviewed against method requirements.

The method is generally being followed.

Particulate Matter Laboratory

EPA visited the particulate matter (gravimetric) laboratory and interviewed Berdine 

Heckman, the lead PM2.5/PM10 weigher and lead PM10 data reviewer. The particulate matter 

laboratory was well-maintained, neat and well-organized. The PM10 and PM2.5 laboratory 

measurements demonstrate technical expertise.

Data Management

This section covers data management for criteria pollutants (O3, CO, NO2, SO2, PM2.5, 

and PM10); non-criteria pollutant data management is addressed in the laboratory section. The 

following staff were interviewed based on their roles in data management:

Adam Canter (System Administrator)

David Shina (Supervisor of AQNS)

Berdeen Heckman (PM laboratory staff)

Bill Brick and Lora Kear-Padilla also have responsibilities for data validation but neither were 

available during the TSA. Bill Brick provided a document entitled “Level II Data Validation 

Procedures for Hourly Air Quality and Meteorological Data” that describes the validation 

procedures.

Quality Assurance/Quality Control

EPA interviewed the MTSD Director, AQNS Supervisor, and ETS Supervisor during the 

TSA and reviewed quality management documents. The District’s quality management system 

meets the basic EPA requirements. The District has submitted a QMP and a QAPP that have 

been reviewed by EPA. The District’s performance conforms to the data quality objectives 

and/or the systematic planning developed by EPA for criteria pollutants. EPA and CARB 

perform national performance and technical evaluations of the monitoring network. 
7

OVERVIEW OF AIR MONITORING PROGRAM

General Program

The San Diego Air Pollution Control District (District) is the government agency that 

regulates sources of air pollution within San Diego County. The San Diego County Board of 

Supervisors serves as the Air Pollution Control Board. The District funds its activities through a 

combination of State and Federal grants, permit fees charged to businesses and industries, and 

vehicle registration fees. The District is organized into five main areas, each of which plays a 

role in protecting the public from the adverse impacts of air pollution. The District Division that 

is the subject of this TSA is the Monitoring and Technical Services Division (MTSD).

The MTSD is managed by the Director, Mahmood Hossain, who reports directly to the 

Air Pollution Control Officer of the District, Bob Kard. The MTSD is organized into five 

sections, four of which are responsible for producing and submitting ambient air quality data to 

EPA: Division Chief, Air Quality Network, Electronic Technicians and Meteorology. The 

operations of the fifth section, Source Testing, were not evaluated during this TSA. The Air 

Quality Network Section (AQNS), managed by David Shina, oversees the laboratory, most 

QA/QC activities, and network planning. The Electronic Technicians Section (ETS), managed by 

David Craig, oversees the operation of stations and instruments.

The organization of the District’s MTSD divides QA/QC responsibility between various 

sections and various personnel. For example, calibrations and audits are performed by the Air 

Quality Network Section; routine QC is done by field operators in the Electronic Technicians 

Section; laboratory personnel are responsible for validation of data generated by analytical 

instruments (Air Quality Network Section); and data validation for continuous pollutants 

(outside the laboratory) is performed by the Air Quality Network and Meteorology Sections. All 

QA functions are embedded within the sections. No one functions in a formal QA role to oversee 

all ambient air monitoring operations at the management level.

Network Management

The ambient air monitoring network in San Diego consists of ten ambient air monitoring 

sites that utilize a variety of air pollutant instruments and four meteorology sites. Numerous sites 

have been operating for 20 to 30 years. These long term sites allow the District to examine trends 

in air quality and assess the progress being made in reducing ambient concentrations of criteria 

pollutants. The following table summarizes the air monitoring sites in San Diego County and the 

pollutants monitored.

Table 1: Air Monitoring Sites Operated by the District.
8

Source: San Diego Air Pollution Control District 2009 Ambient Air Monitoring Network Plan, 

Submitted June 29, 2010.

The District network of Photochemical Assessment Monitoring Stations (PAMS) consists 

of four sites: Camp Pendleton is a Type 1 (upwind/background/transport) PAMS site, Kearny 

Mesa and El Cajon are Type 2 (maximum precursor emission/central business district) sites, and 

Alpine is a Type 3 (downwind/maximum ozone) site. The District's PAMS network, operating 

schedule and forecasting scheme have been approved by EPA. The District has an approved 

NCore site (National Core multi-pollutant monitoring station), El Cajon, which began collecting 

NCore data beginning January, 2011. Other air monitoring activities include PM2.5 speciation 

monitoring and air toxics as indicated in Table 1.
9

Field Operations

Field operation duties are split between two sections within the MTSD, the AQNS and 

the ETS. The day-to-day operations at the monitoring stations are performed by staff in the ETS.

The ETS has ten full time employees. The AQNS staff performs the calibrations, standard 

certifications, and quality control and quality assurance activities at the monitoring sites, in 

addition to running the District laboratory.

Toxics Laboratory Operations

Analytical laboratories provide support for measurement methods that are either too 

complex or sensitive to perform in the field environment. In order to provide these services, 

laboratories employ complex instrumentation for which staff need highly specialized training.

For ambient air samples to provide useful information or evidence, laboratory analyses must 

meet the following four basic requirements: 

1. Equipment must be frequently and properly calibrated and maintained.

2. Personnel must be qualified to perform the analysis.

3. Analytical procedures must be in accordance with accepted practice.

4. Complete and accurate records must be kept.

Laboratory operations are managed by David Shina, Supervisor of the AQNS. The 

District’s laboratory facility is located in the headquarters office in San Diego, California.

Analyses are performed in-house by laboratory staff with the exception of metal samples which, 

at the time of the TSA, were being collected and stored, but not analyzed, due to staff turnover. It 

was anticipated that a recently hired chemist would process the backlog of metals samples. One 

of the primary responsibilities of laboratory is to process the PM filters, which includes 

preparation, weighing, tracking, and storing PM2.5 and PM10 filters. In addition to PM 

responsibilities, the laboratory is also responsible for air toxics (including NATTS) analysis and 

PAMS monitoring. The compounds that the District routinely analyzes include:

• PAMS hydrocarbons 

• VOCs (Method TO-15 target compounds) 

• Carbonyls, (Method TO-11 acetone, acetaldehyde, and formaldehyde) 

• Carbon by OC/EC 

There are a number of additional activities the laboratory undertakes to support the collection 

and analysis of air pollutants. These include canister cleaning and preparation, data validation, 

and sample storage.

Particulate Matter Laboratory (Gravimetric Laboratory)

The District’s gravimetric laboratory is housed in a small building (large environmental 

chamber) located in the headquarters’ courtyard. PM2.5 and PM10 have a lead staff for weighing 

with a back-up and a lead staff for data review with a back-up. Filters are weighed manually, but 

data are automatically entered into the GLIMS database. 

Data Management
10

Data management includes data collection, the data validation process, and a data 

management system. Data management at the District follows two separate tracks: one for 

continuous (i.e., non-laboratory) data and one for laboratory data.

Continuous Data

Data Collection/Data Management System 

Air quality data produced by continuous analyzers in the field are stored in dataloggers 

and station computers. Data are transmitted hourly directly to the central computer via telemetry.

Data flows directly into an SQL database at the District office. Once in the SQL database,

EDES/DTX (formerly ESC) can then be used to view the data. At this point in the data stream,

data are sent to AirNow, the district website, the district backup, and CARB.

Initial data review occurs at the end of each week and second and third levels of review 

occur at the end of each month (see description below). Changes to data may occur after each of 

these reviews and are implemented using EDES/DTX: the original SQL database is overwritten 

when changes are implemented. The original data are saved in hard copy, on the backup website, 

and as text files. The System Administrator is the only person who has access to edit the 

database, with an administrative version of EDES/DTX that is password protected. Once a 

month’s data are finalized, EDES/DTX is used to generate an AQS file for that month and 

uploaded through CDX. The District submits all required data to the EPA AQS database,

including concentration data for all criteria pollutants, supporting precision and accuracy data 

and data from the Photochemical Assessment Monitoring Station (PAMS) network. Data 

submitted to AQS is certified annually by the District.

The District’s central computer is backed up on tape and disk storage every night.

Weekly archives are stored offsite at a data storage facility. Loss of data in the event of 

significant computer problems is expected to be minimal given the redundant backup systems in 

place at the air monitoring stations and at the offsite data storage facility. 

Data Validation

The data are first reviewed by the station technicians. At the end of each week, the station 

technicians send an email to the AQNS Supervisor and the System Administrator identifying any 

data issues. Station technicians suggest data treatment (e.g., invalidation) and a flag. The 

Supervisor and System Administrator confer and document their decision on a printed copy of 

the email that is signed and stored with a hard copy of the hourly data. In some cases, flags are 

generated automatically by the datalogger, e.g., the maintenance flag is generated whenever

instruments are in maintenance mode. Flags that are generated by station dataloggers and any

other information describing the data being processed are included in the data stream at this point 

as well. The System Administrator has to change automatic flags that are inappropriately applied 

(e.g., maintenance flag applied to points during a calibration) manually. All flags identified at the 

weekly review are implemented by the System Administrator.

At the end of each month the System Administrator generates a preliminary AQS file and 

provides this to the Senior Meteorologist for the next level of review (referred to as Level II data 

validation). This process is new, having been implemented within the past year. The Senior 

Meteorologist conducts the following activities, as described in “Level II Data Validation 

Procedures for Hourly Air Quality and Meteorological Data”:
11

• Computerized data screening for consistency and accuracy

• Manual inspection of the data 

• Visual inspection of graphical representations of summary statistics

• Detailed data investigation where problems are found or suspected

• Data resolution where problems are found 

• Comparison of summary statistics with data from other sites within the monitoring 

network.

The System Administrator implements any additional changes and the data are finalized and

submitted to the AQS database.

Laboratory Data (PM)

Data Collection/Data Management System:

PM data are automatically logged in the GLIMS data system as the filters are weighed.

After data are validated, they are converted to AQS format using an Excel conversion program.

The lead PM Specialist emails data to the System Administrator for storage until all monthly 

data have been reviewed and are ready for upload. When data are ready for submission, the

System Administrator uploads all laboratory and non-laboratory data.

Data Validation for PM2.5:

The first level of review of PM data is performed by the field technician or assistant, who 

notes any issues on the chain-of-custody form. The person in charge of weighing PM2.5 filters, 

reviews the chain-of-custody form for any unusual issues. These are noted in the GLIMS system,

and in a separate logbook for the lead PM2.5 data review. At the next level, the issues identified 

are evaluated as to whether the data should be invalidated. That decision is noted in the same 

logbook. In addition, the lead reviewer looks at all the PM2.5 data, not only those that have been 

specifically noted.

Data Validation for PM10:

The first level of review of PM data is performed by the field technician or assistant, who 

notes any issues on the chain-of-custody form. The person in charge of weighing PM2.5 filters

reviews the chain-of-custody form for any unusual issues. These are noted in the GLIMS system.

At the next level, the decision is made whether or not to invalidate data. As the data review team

personnel are the same as the weighing team, each person tries to review data from filters that 

s/he did not weigh; however, this is not always possible. After the second level of validation,

data are invalidated in the GLIMS system, as needed.

Quality Control and Quality Assurance

Quality management is the system put in place that oversees quality assurance, quality 

control, and quality improvement activities. EPA requires that ambient air monitoring agencies 

have a quality management system that conforms to 40 CFR Part 50 Appendix A and the EPA 

quality policy (EPA Order CIO 2105.0). Additionally, EPA grant regulations require each 

grantee to provide for QA activities (40 CFR Part 31.45).
12

Specifically, 40 CFR Part 50 Appendix A Section 2 requires that each ambient air 

monitoring PQAO conforms to certain quality management practices. These include:

• Having a documented quality system that meets EPA requirements for QMPs and 

QAPPs.

• Having a quality management function that is independent of air monitoring operations.

• Developing or adopting DQOs, or equivalent systematic planning procedures, for all 

monitoring programs.

• Participating in National Performance Evaluation Programs, which consist of 

performance audits used to independently determine program adequacy, national 

monitoring network performance, and national consistency.

• Submitting to Technical Systems Audits by EPA at a frequency of every three years or 

less.

• Using certified reference materials to standardize monitoring equipment.

EPA views the application of these quality management system components as integral to a

credible monitoring program. Insufficient quality management and control has been cited as 

rationale to support legal challenges to NAAQS designation decisions. 

Quality control and quality assurance are the two components of quality management 

within a monitoring program that support the assumption that the data they collect represent the 

true air quality of the area. They are the means by which an organization manages its quality 

activities in a systematic, organized manner and they provide a framework for planning, 

implementing, and assessing work performed by an organization. A properly developed QA/QC 

program encompasses a variety of technical and administrative elements, including policies and 

objectives, organizational authority, responsibilities, accountability, and procedures and 

practices. Quality assurance is a management or oversight function that includes setting policy 

and running an administrative system of management controls that cover planning, 

implementation, the review of data collection activities, and the use of data in decision making. 

Quality control is a technical function that includes all the scientific precautions, such as 

calibrations and verifications, needed to acquire data of known and appropriate quality. 

MTSD divides QA responsibility among various sections and personnel. For example, 

calibrations and audits are performed under the Air Quality Network Section, while routine QC 

is done by field operators under the Electronic Technicians Section. All QA functions are 

embedded within the sections rather than having an assigned QA function that would oversee all 

ambient air monitoring operations at the management level. Most quality assurance functions 

reside in the Air Quality Network Section, managed by David Shina. In general, different people 

perform calibrations rather than performing audits of a specific instrument type.
13

FINDINGS

Program Area Finding 

Numbers

EPA Contact

General 1-2 Meredith Kurpius

Network Management 3 Meredith Kurpius

Field Operations 4 Meredith Kurpius

Laboratory – QA 5-6 Steve Remaley

Laboratory – PAMS 7-12 Steve Remaley

Laboratory – TO-15 13-14 Steve Remaley

Laboratory – TO-11 15-16 Steve Remaley

Particulate Matter (Gravimetric) Laboratory 17 Michael Flagg

Data Management 18-20 Meredith Kurpius

Quality Assurance/Quality Control 21-22 Kate Hoag
14

Finding # 1

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: General

Finding:

There is no independent, dedicated QA Manager responsible for communicating and ensuring 

that quality assurance activities are carried out in field operations, information management, and 

laboratory operations. Most QA functions were being carried out but some specific oversights 

were identified such as:

• There is no secondary, independent QA review for logbook review, data review or peer

review of benchtop laboratory data .

• Biweekly precision QC checks are not always performed according to schedule.

• MDLs for TO-15 and PAMS missed or not performed.

• There is a lack of training files and Initial Demonstration of Analyst Proficiency.

• There is no over-arching corrective action process.

• Internal audits (i.e., data tracking) are not being performed.

• Some PM2.5 data points were incorrectly invalidated.

Discussion:

40 CFR, Part 58, Appendix A, Section 2.2, requires that each ambient air monitoring 

organization have an independent quality assurance function. Quality assurance functions are 

currently spread throughout the District’s air monitoring sections. While the organizational 

structure may remain the same, a QA Manager should be designated to provide leadership to the 

various sections. The designated individual should have overall QA authority to conduct 

independent oversight and to require corrective action. In order to function effectively, the QA 

Manager should oversee, rather than perform, staff level duties. 

The QA Manager would be responsible for:

• Periodic audits of the quality management system (management system reviews); routine 

procedures’ and data quality to identify areas of improvement and to ensure that monitoring 

programs continue to follow sound and documented procedures consistently.

• Review of performance for certain QA/QC activities, such as MDLs; span/precision checks; 

annual certification of the O3 transfer and flow standards; the certification of calibration gas 

cylinders for NOx, CO and SO2 instrument calibration; and the conduct of flow audits of PM

instruments samplers.

• Development of a formal corrective action process.

• Routine review and tracking of precision and accuracy data. 

In general, EPA QA/R-5, EPA Requirement for Quality Assurance Project Plans, discusses 

assessments that should be elements of a quality management program. These include 

“surveillance, management systems reviews, readiness reviews, technical systems audits, 

performance evaluations, audits of data quality, and data quality assessments.”

References:

40 CFR, Part 58, Appendix A, Section 2.2

EPA QA/R-5, EPA Requirement for Quality Assurance Project Plans
15

Recommendation to Address Finding:

Establish a centralized, independent and dedicated QA Manager position. This position should be 

responsible for establishing, communicating, and overseeing (approving) the QA/QC activities 

conducted in the field, information management, and laboratory operations. Additionally, 

because laboratory oversight involves knowledge of complex chemical methods different from 

the technical expertise employed to run field stations, there should be separate laboratory and 

field operations quality assurance responsibilities assigned to support the QA Manager (see 

Finding 5). 
16

Finding # 2

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: General

Finding:

The District’s Quality Assurance program documentation, including a Quality Management Plan 

(QMP), Quality Assurance Project Plans (QAPPs) and Standard Operating Procedures (SOPs), 

was not complete and was not approved internally or by EPA at the time of the audit. 

Discussion:

EPA requires that organizations develop a QAPP for each type of ambient pollutant being 

measured. The QAPP integrates all technical and quality aspects of a project, including planning, 

implementation, and assessment. The purpose of the QAPP is to document planning activities for 

collecting environmental data operations and to provide a project-specific “blueprint” for 

obtaining the type and quality of environmental data needed for a specific decision or use.

The QAPP documents the quality assurance and quality control that are applied to an 

environmental data operation to assure the results obtained are of the type and quality needed 

and expected. Further guidance on developing QAPPs can be found in the guidance documents 

“EPA Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, March 2001, and 

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002. 

The District has many SOPs that are used internally, but these have not been submitted as part of 

a QAPP for EPA approval. The District is currently developing SOPs for gaseous pollutant 

audits and NCore monitoring activities.

References:

EPA/240/B-01/003, March 2001

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002.

Recommendation to Address Finding:

The District should work with EPA to develop approved QA planning documents. The District 

should develop and implement a schedule to update these documents on a routine schedule.
17

Finding # 3

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Network Management

Finding:

The scale of representativeness for some sites does not reflect contributing sources.

Discussion:

PM10 at the Otay Mesa – Border Crossing is heavily influenced by emissions from the busy 

border crossing that is adjacent to the monitor. Currently Otay Mesa PM10 is listed as 

neighborhood scale, which is intended to characterize “conditions throughout some reasonably 

homogenous urban sub-region with dimensions of a few kilometers” (40 CFR 58, Appendix D, 

Section 4.6(b)(3)). The scale designation does not reflect the very localized impacts of the truck 

crossing, which significantly impact the PM10 levels at the site. NO2 would be similarly affected 

by local emissions from the border crossing. While some O3 may be scavenged by high levels of 

NO, EPA does not expect this to be the dominant influence and therefore O3 should remain listed 

as neighborhood scale.

NO2 at El Cajon is listed as an urban scale site with a monitoring objective of source impact.

Urban scale is intended to represent areas of several kilometers up to 50 kilometers and to 

characterize area-wide concentrations (40 CFR 58, Appendix D). As such, urban scale is not an 

appropriate scale for a source impact measurement.

References:

40 CFR 58, Appendix D

Recommendation to Address Finding:

Review PM10 and NO2 at the Otay Mesa – Border Crossing site and change scale of 

representativeness as appropriate. Review the monitoring objective and scale of 

representativeness of NO2 at El Cajon and choose the appropriate combination.
18

Finding # 4

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Field Operations

Positive Finding:

Overall, the field station operations are performed very well. The positive findings include:

• Competent and dedicated field station operators and managers.

• Comprehensive training support for staff.

• Field station operators have primary responsibility for no more than three sites.

• Instruments are kept in good working order.

• Very well organized, clean and safe field stations.

• Good backup system of rotating field operators.

• Knowledgeable audit staff.

• Preventive maintenance is tracked effectively.

• Operators perform QC checks daily, ensuring quick identification of instrument problems 

and minimizing instrument down time and data loss.

Discussion:

N/A

References:

N/A

Recommendation to Address Finding:

N/A
19

Finding # 5

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – QA

Finding:

The laboratory does not have an independent QA Officer who can ensure effective 

implementation of a QA program. Instead, each analyst is responsible for reviewing the quality 

of the data s/he has collected. As a result, some routine QA functions are not being performed 

and others are not being performed on schedule (see Finding 1).

Discussion:

The laboratory would benefit significantly from centralizing the QA function and assigning an 

independent QA Officer. The QA Officer function is best executed completely independently 

from the data generation process and influence of pressures to meet deadlines. The observation 

of lack of secondary peer review and missed MDL studies indicates that the analysts are pressed 

for time and would benefit from a QA Officer who would be able to bring resolution to quality 

assurance issues.

References:

40 CFR, Part 58, Appendix A, Section 2.2

EPA QA/R-5, EPA Requirement for Quality Assurance Project Plans

Recommendation to Address Finding:

It is recommended that a full time QA Officer be assigned to oversee implementation of the 

laboratory’s QA Program. QA records should be centralized in order to facilitate independent 

monitoring of routine activities and the corrective action process to ensure non-conformances 

and out of control events are resolved.
20

Finding # 6

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – QA

Finding:

The District’s SOP for training does not reflect current practice regarding laboratory training.

Discussion:

The District’s SOP states that new employees will receive training from instrument vendors, 

however the audit revealed that funding was not available for recent hires to receive vendor 

training. Instead, recent analyst hires receive on the job mentoring from an experienced analyst.

Additionally, a formalized procedure for assessing and documenting analyst proficiency is not 

documented (i.e., Initial Demonstration of Proficiency).

SOPs serve to document official policy for critical functions and can play an important role in 

demonstrating data defensibility and evidentiary role. It is important therefore that SOPs to 

reflect actual practice. Alternate procedures, such as mentoring, should be documented in the 

SOP. The procedure for assessing new analyst proficiency could be improved upon by requiring 

analysts to demonstrate proficiency in running an audit sample and/or passing a written test.

Results of Initial Demonstration of Proficiency and ongoing assessments and training should be 

documented in a dedicated training file.

References:

Recommendation to Address Finding:

The training SOP should be revised to reflect actual practice. Additionally, the SOP should 

describe how Initial Demonstration of Proficiency of new analysts will be assessed and any 

planned ongoing assessments. Records should be documented in a dedicated training file.
21

Finding # 7

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory - PAMS

Finding:

The PAMS GC-FID is an old instrument and may not be performing reliably.

Discussion:

Staff stated that some components of the GC-FID, such as the concentrator, are no longer 

supported by the equipment vendor. The transfer lines were recently replaced to eliminate active 

sites, which caused unacceptable recoveries for some compounds reflected in the Regional PE 

sample study and laboratory standard recoveries. Although recoveries improved by 20% with the 

replacement lines, staff stated recoveries are still low by 20%. Additionally, the current 

equipment configuration does not have the apparatus that facilitates the dilution of standards 

used for multipoint calibration and method detection limit studies (MDLs). Most importantly, 

responses from staff suggest the instrument detector response does not always give reproducible 

results on successively analysis for daily calibration and the efforts to identify and correct the 

problem have not been successful.

References:

Recommendation to Address Finding:

The PAMS program is currently undergoing a reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention will affect the District’s PAMS program. It is recommended that the 

laboratory research the purchase of replacement instrumentation. Replacing the instrument with 

either a new GC-FID or an upgrade to a GC equipped with a dual FID-mass spectrometer 

detector configuration would represent a significant upgrade for the detection process, reducing 

uncertainty as to compound identification. The dual GC mass spectrometer – FID configuration 

has the advantage of allowing (simple) calibration of the FID detector with propane for 

quantitation. The mass spectrometer detector would provide superior target and non target 

compound identification and reduce co-elution and thus the effort needed to interpret very 

complex PAMS chromatograms.
22

Finding # 8

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – PAMS

Finding:

Multi-point calibrations are not being performed according to guidance and good laboratory 

practice. A single point calibration is being performed instead.

Discussion:

Laboratory staff stated that multi-level calibrations are not being performed due to the difficulty

of diluting calibration gas standards without the benefit of dynamic dilution equipment.

According to guidance (PAMS TAD, Sec. 2.3.4.2, Table 2-17), a multi-point calibration 

consisting of a minimum of three points bracketing the expected sample concentration range is to 

be performed prior to analysis at the start of the season and when system maintenance is 

performed. Not performing a multi-point calibration may produce less accurate data.

References:

Recommendation to Address Finding:

The PAMS program is currently undergoing reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention will affect the District’s PAMS program. Calibration, including 

multi-point calibration, should be performed according to guidance to ensure the most accurate 

sample quantitation. Based on analyst responses, it is believed that the use of upgraded PAMS 

instrumentation with dynamic dilution equipment should address this issue.
23

Finding # 9

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory - PAMS

Finding:

PAMS method detection limit (MDL) studies are not being performed as required by guidance.

According to staff, MDL studies have not been performed because the instrument lacks the 

dilution equipment accessory necessary to perform the studies at low concentration levels.

Discussion:

The PAMS TAD (Table 2-17) specifies that MDL studies must be performed according to 

procedures specified in 40 CFR 136 Part B. MDL studies should be performed prior to the start 

of the PAMS season. Acceptance criteria of 2 ppbC or better for sensitivity must be met in order 

to achieve method data quality objectives. MDL studies are needed so that accurate reporting 

limits may be documented and submitted whenever non-detect data is reported.

References:

PAMS TAD (Table 2-17)

40 CFR 136 Part B

Recommendation to Address Finding:

The PAMS program is currently undergoing a reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention will affect the District’s PAMS program. MDL studies should be 

performed according to guidance. The purchase of new testing equipment with the dilution 

accessory would facilitate the process and address this issue.
24

Finding # 10

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory - PAMS

Finding:

Internal standards with calculated relative retention times and reference peaks are not being used 

as part of the target peak compound identification process as prescribed in the PAMS guidance 

document. In addition, quality control measures supportive of proper peak identification and 

quantitation used to be, but are no longer being, performed as recommended in guidance.

Examples of these quality control measures include periodic GC/MS sample confirmation of a 

percentage of analysis, or periodically re-analyzing samples on another, well-characterized GC 

system to identify co-eluting compounds. Staff stated that co-elution is not evaluated.

Discussion:

The PAMS TAD provides guidance on peak identification (Sec. 2.3.7.2). Although gas 

chromatographic systems are generally stable, variations in conditions do occur that can shift 

absolute retention times. This can result in improper or missed compound identification. The use 

of internal standards with calculated relative retention times would address this concern and 

generally facilitate the target compound identification process. This seems especially important 

with the complex chromatography of PAMS analysis. Although the PAMS GC FID method is an 

acceptable method that can provide accurate results, it is believed that upgrading the system 

configuration to a dual column - dual detector FID and MSD or similar configuration would 

yield significant benefits. Quantitation could still be performed by calibrating the FID detector 

with propane, while the mass spectrometer would be used for (more precise) compound 

identification. Such a system configuration may represent a natural evolution for PAMS analysis.

It is already being used in some commercial laboratories. Incorporation of a MS detector into the 

system configuration would greatly facilitate the target and any non-target contaminant peak 

identification process, while significantly reducing the human error factor. The process would 

add a higher level of confidence to contaminant identification inherently afforded by mass 

spectral identification. The mass spectrometer detector would eliminate co-elution issues and the 

need to manually edit and integrate co-eluting peaks.

References:

PAMS TAD, Sec. 2.3.7.2

Recommendation to Address Finding:

The PAMS program is currently undergoing a reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention affects the District’s PAMS program. At a minimum, it is 

recommended that the laboratory use internal standards as prescribed in the PAMS method.

GC/MS confirmation or cross duplicate analyses on a second GC system should also be

performed on a percentage of samples according to guidance. GC/MS should be used to resolve 

issues with samples containing co-eluting compounds in place of manual integration of 

unresolved gas chromatographic peaks. The laboratory is encouraged to investigate and consider 

upgrading its system configuration to one that includes a mass selective detector for compound 

identification.
25

Finding # 11

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory - PAMS

Finding:

According to analyst staff, replicate samples are being analyzed, but precision and corrective 

action criteria have not been established.

Discussion:

Guidance prescribes replicate analysis at a frequency of 10% of samples (PAMS TAD, Table 2-

17). Guidance further requires sample analysis to be repeated whenever precision for target 

concentrations greater than 5 times the MDL exceed ±25% RPD. Precision results that fall 

outside prescribed limits are an indication of analytical problems that require attention.

References:

PAMS TAD, Table 2-17

Recommendation to Address Finding:

The PAMS program is currently undergoing a reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention will affect the District’s PAMS program. It is recommended that the 

laboratory determine MDLs and then establish precision and corrective action criteria.
26

Finding # 12

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – PAMS 

Finding:

Staff stated that during the PAMS season when the canister selected for blanking fails to meet 

criteria, the cans are used for collection and the data are flagged. However, guidance that states 

that the entire bank of canisters being cleaned should be rejected. Additionally, laboratory staff 

stated that although the “bad canister” that failed blanking is re-cleaned, it is not necessarily retested per the TAD and the District’s SOP. The SOP states, “If the batch of canisters requires recleaning, the same canister should be selected for a QC sample and analyzed again.”

Discussion:

Cleaning canisters to meet predetermined criteria ensures they are free of surface contamination 

and that samples collected are representative. Not following this process results in flagged data 

which are less useful because there is uncertainty as to the true source of target compound 

detects. Both the PAMS guidance document and the District’s SOP state that a blanked canister 

that failed the certification process should be re-tested following re-cleaning to confirm the 

canister meets certification criteria.

References:

TAD Sec. 2.5.2.3, pg. 105

TAD, Sec 2.5.2.3, pg. 105; SOP pg. 4

Recommendation to Address Finding:

The PAMS program is currently undergoing a reinvention at the national level. Prior to 

implementation of any corrective actions, the District should confer with EPA to determine if the 

PAMS program reinvention will affect the District’s PAMS program. It is recommended that the 

laboratory adhere to guidance and reject the bank of cleaned cans when a certification blank fails 

to meet criteria. If this is not possible due to resource/time constraints then it is recommended 

that all canisters in the batch are flagged for the failed compounds.
27

Finding # 13

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – TO-15

Positive Finding:

The laboratory has established a retention time for TO-15 canisters of 30 days from canister 

cleaning and certification, after which canisters are to be re-cleaned prior to shipment and use in 

the field.

Discussion:

N/A

References:

N/A

Recommendation to Address Finding:

N/A
28

Finding # 14

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – TO-15

Finding:

The laboratory performed MDL studies in 2007 and 2009, but did not perform an MDL study in 

2008.

Discussion:

MDL studies should be performed annually according to procedures described in 40 CFR Part 

136 or any time significant maintenance is performed on, or changes are made to, the instrument 

(TO-15 TAD, Sec 11.2). MDL studies are necessary when non-detect results for samples are 

reported and to ensure the method objective sensitivity of 0.2 ppbv is met for target compounds.

References:

TO-15 TAD, Sec 11.2

40 CFR Part 136

Recommendation to Address Finding:

MDL studies should be performed at a minimum annually or any time significant changes are 

made to the instrument. The target method sensitivity for Method TO-15 is 0.2 ppbv.
29

Finding # 15

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – TO-11

Positive Finding:

Duplicate and co-located samples are being analyzed as a QA measure.

Discussion:

N/A

References:

N/A

Recommendation to Address Finding:

N/A
30

Finding # 16

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Laboratory – TO-11

Finding:

Analyst staff stated that field blank results are not being submitted and that the field sample 

results are being submitted corrected for field blanks.

Discussion:

Carbonyl sample data should not be field blank corrected. Field blank results and sample results 

should be reported separately. Consistency at each site is of utmost importance because the 

purpose of NATTS is long term trending. If data for one year are corrected, all other data must 

be corrected.

References:

Recommendation to Address Finding:

The laboratory should report uncorrected carbonyl sample data. Results for field blanks should 

also be reported.
31

Finding # 17

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: PM Laboratory

Positive Finding:

Many of the previous findings from EPA’s 2006 TSA have been addressed and the laboratory 

has made many improvements in the past years.

Discussion:

N/A

References:

N/A

Recommendation to Address Finding:

N/A
32

Finding # 18

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Data Management

Positive Finding:

Overall, the current data management system for continuous data is very robust. Specific 

examples of this effective system include:

• Communication between the System Administrator and the AQNS Supervisor was 

effective and well-documented.

• The System Administrator was able to access specific data points for continuous data 

quickly as well as the documentation for decisions made regarding specific data points.

• The recent improvements in data validation that include analyses and visual tools such as 

graphs and regressions provide for a very high quality level of review.

Discussion:

N/A

References:

N/A

Recommendation to Address Finding:

N/A
33

Finding # 19

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Data Management

Finding:

Data management issues for PM resulted in incorrect invalidation of PM data.

Discussion:

We found numerous instances of lost data due to improper data management in the PM 

laboratory. In some cases valid PM2.5 data were invalidated in AQS with no record of the reason 

and in other cases data were simply missing with no valid reason or documentation. Some data 

were subsequently found and uploaded to AQS. Specific examples include:

• PM2.5 data at Escondido on January 1, 2009 were invalidated in AQS due to a laboratory

error. The original file for the Escondido data point was found to have a valid entry of 64.9 

μg/m3

, which was overwritten with the AR null data code in a later file with no associated 

documentation. The invalidated Escondido data point was the highest value of the year.

• PM2.5 data at El Cajon on March 2, 2009 were invalidated and entered into AQS as due to a 

machine malfunction. The El Cajon data point was missing in the GLIMS database, although 

it was determined in the laboratory books to have been a valid sample. The District believes 

that this data point was overwritten with a data point from 1999 due to a data file issue that 

the District encountered when the barcode reader was reprogrammed.

• Calexico data for December 17, 2007 was originally missing from AQS. Following an 

inquiry by EPA, these data were located and uploaded to AQS. The District indicated that 

data for this data were not originally in the data file that was sent to the System 

Administrator.

While most PM data are stored and uploaded reliably, the incidents above indicate that the data 

management and validation of PM data could be made more robust.

References:

Recommendation to Address Finding:

There should be one more level of data validation, similar to that of the gaseous pollutants, and 

data tracking performed by the QA Manager.
34

Finding # 20

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: Data Management / PM Laboratory

Finding:

Full secondary review of PM10 filter data is not being performed by an independent reviewer.

Discussion:

Currently, PM10 filters are weighed and the initial data review is performed by the 

laboratory technicians. Unlike the PM2.5 independent, secondary data review process, the 

secondary data review for PM10 is often performed by the same staff who weighs the filters and 

performs the primary data review.

References:

Recommendation to Address Finding:

Secondary review of PM10 filter data should be performed by a staff member who is independent 

of the weighing and data collection process.
35

Finding # 21

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: QA/QC

Finding:

There is no formal tracking of failed gaseous audits.

Discussion:

EPA quality management standards (EPA QA/R-2, Quality Improvement Section) require that 

management and staff “ensure that conditions adverse to quality are” prevented, identified 

promptly, fully defined, corrected, prevented from recurring, and documented as corrective 

actions which are tracked to closure. There should be a formal, documented mechanism for 

elevating potentially significant corrective actions originating from the audit staff concerning 

audit findings.

References:

EPA QA/R-2, Quality Improvement Section

Recommendation to Address Finding:

Develop a formal corrective action process for the gaseous audit program.
36

Finding # 22

Agency: San Diego County APCD

Date of Audit: September 28-30, 2010

Program Area: QA/QC

Finding:

Some documentation could be improved to ensure complete records retention at the site and 

transmission of accurate site operations information to data validators.

Discussion:

EPA observed that the District has some excellent forms and procedures to document activities 

and that the written documentation by the field technicians was generally very thorough.

However, some information is stored in binders at the site and at the office, and the written 

comments did not always match what was recorded on the QC sheets. For example, some of the 

calibration sheets were missing in the instrument logs. Also, there is not a clear record of PM 

calibrations and outcome at each site, as there is for O3 calibrations. EPA noted that audits 

performed by CARB should also be logged with the other calibration information. The lack of 

clear documentation for a recent CARB audit led to it being flagged as routine 

maintenance/repair.

General information should be recorded in logbooks that are bound, tracked, and have page 

numbers in order to ensure legal defensibility. Routine checks may be recorded on prepared 

forms (including particulate monitor flow checks) that include page numbers or another method 

of ensuring information is not missing.

References:

Quality Assurance Handbook for Air Pollution Measurement Systems, Volume II, Section 5

Recommendation to Address Finding:

Ensure that all field/audit/calibration staff are aware of documentation procedures. Add summary 

sections to the logbooks for CARB audits and also PM2.5 audits and calibrations. Consistency 

and completeness of the logbook information may also be improved by implementing a more 

formal review by the ETS manager, David Craig.
37

APPENDIX A: SUMMARY OF FINDINGS

Highest Priority

Finding 1: There is no independent, dedicated QA Manager responsible for communicating and 

ensuring that quality assurance activities are carried out in field operations, information 

management, and laboratory operations. Most QA functions were being carried out but some 

specific oversights were identified such as:

• There is no secondary, independent QA review for logbook review, data review or peer

review of benchtop laboratory data.

• Biweekly zero/span QC checks are not always performed according to schedule.

• MDLs for TO-15 and PAMS missed or not performed.

• There is a lack of training files and Initial Demonstration of Analyst Proficiency.

• There is no over-arching corrective action process.

• Internal audits (i.e., data tracking) are not being performed.

• Some PM2.5 data points were incorrectly invalidated.

Finding 2: The District’s Quality Assurance program documentation, including a Quality 

Management Plan (QMP), Quality Assurance Project Plans (QAPPs) and Standard Operating 

Procedures (SOPs), was not complete and was not approved internally or by EPA at the time of 

the audit.

Finding 5: The laboratory does not have an independent Quality Assurance (QA) Officer who 

can ensure effective implementation of a QA program. Instead, each analyst is responsible for

reviewing the quality of the data s/he has collected. As a result, some routine QA functions are 

not being performed and others are not being performed on schedule (see Finding 1).

Finding 19: Data management issues for PM resulted in incorrect invalidation of PM data.

Moderate Priority

Finding 3: The scale of representativeness for some sites does not reflect contributing sources.

Finding 7: The PAMS GC-FID is an old instrument and may not be performing reliably.

Finding 20: Full secondary review of PM10 filter data is not being performed by an independent 

reviewer.

Finding 21: There is no formal tracking of failed gaseous audits.

Finding 22: Some documentation could be improved to ensure complete records retention at the 

site and transmission of accurate site operations information to data validators.

All Other Significant Findings

Finding 6: The District’s SOP for lab training does not reflect current practice.

Finding 8: PAMS multi-point calibrations are not being performed according to guidance and 

good laboratory practice. A single point calibration is being performed instead.
38

Finding 9: PAMS method detection limit (MDL) studies are not being performed as required by 

guidance. According to staff, MDL studies have not been performed because the instrument

lacks the dilution equipment accessory necessary to perform the studies at low concentration 

levels.

Finding 10: Internal standards with calculated relative retention times and reference peaks are 

not being used as part of the target peak compound identification process as prescribed in the 

PAMS guidance document. In addition, quality control measures supportive of proper peak 

identification and quantitation used to be, but are no longer being, performed as recommended in 

guidance. Examples of these quality control measures include periodic GC/MS sample 

confirmation of a percentage of analysis, or periodically re-analyzing samples on another, wellcharacterized GC system to identify co-eluting compounds. Staff stated that co-elution is not 

evaluated.

Finding 11: According to analyst staff, replicate samples are being analyzed for PAMS, but 

precision and corrective action criteria have not been established. Additionally, staff stated they 

have discussed control charting precision results, but this has not been implemented to date.

Finding 12: Staff stated that during the PAMS season when the canister selected for blanking 

fails to meet criteria, the cans are used for collection and the data are flagged. However,

guidance that states that the entire bank of canisters being cleaned should be rejected.

Additionally, laboratory staff stated that although the “bad canister” that failed blanking is recleaned, it is not necessarily re-tested per the TAD and the District’s SOP. The SOP states, “If 

the batch of canisters requires re-cleaning, the same canister should be selected for a QC sample 

and analyzed again.”

Finding 14: The laboratory performed TO-15 MDL studies in 2007 and 2009, but did not 

perform an MDL study in 2008.

Finding 16: Analyst staff stated that field blank results are not being submitted and that the field 

sample results are being submitted corrected for field blanks for TO-11.

Positive Findings

Finding 4: Overall, the field station operations are performed very well. The positive findings 

include:

• Competent and dedicated field station operators and managers.

• Comprehensive training support for staff.

• Field station operators have primary responsibility for no more than three sites.

• Instruments are kept in good working order.

• Very well organized, clean and safe field stations.

• Good backup system of rotating field operators.

• Knowledgeable audit staff.

• Preventive maintenance is tracked effectively.

• Operators perform QC checks daily, ensuring quick identification of instrument problems 

and minimizing instrument down time and data loss.
39

Finding 13 (Positive): The laboratory has established a retention time for TO-15 canisters of 30 

days from canister cleaning and certification, after which canisters are to be re-cleaned prior to 

shipment and use in the field.

Finding 15 (Positive): Duplicate and co-located samples are being analyzed as a QA measure

for TO-15.

Finding 17 (Positive): Many of the previous findings from EPA’s 2006 TSA have been 

addressed for the PM laboratory and the laboratory has made many improvements in the past 

years.

Finding 18 (Positive): The current data management system for continuous data is very robust.
40

APPENDIX B: GOOD PRACTICE RECOMMENDATIONS

NETWORK DESIGN RECOMMENDATION 1:

Frequent land use changes on large dirt lots by the border and construction activities near both 

Otay Mesa-Donovan and Alpine sites indicate the possibility of local PM sources interfering 

with the PM measurements. SDAPCD should track the activity of these local fugitive dust 

sources and evaluate whether they may be affecting the representativeness of the siting scale 

intended for the PM monitors at the Otay Mesa-Donovan and Alpine sites.

LABORATORY RECOMMENDATION 1:

Laboratory staff responses indicated that the VOC laboratory does not have a separate air 

handling system. It was unknown whether there was positive pressure. Keeping VOC areas free 

of solvents and maintaining a separate air handling system be important in contamination 

management. This may not present a problem and is pointed out to the laboratory as a 

suggestion. The laboratory may wish to evaluate background levels of contamination in the VOC 

laboratory. A logical time to do this may be at the time of purchase and installation of a new 

PAMS instrument because the PAMS and TO-15 instrumentation are housed in the same area.

LABORATORY RECOMMENDATION 2:

Duplicate and co-located samples are being analyzed as a QA measure, which is a positive 

finding. Making control charts of the results would be a further enhancement. As with all quality 

control results, it is of paramount importance that QC results are given attention by analyst and 

quality assurance staff when corrective action is needed, and that outliers are taken into account 

when data are evaluated against their intended use.

LABORATORY RECOMMENDATION 3:

Based on the analyst interview, holding times for DNPH cartridges are occasionally longer than 

the recommended two weeks. It is recommended that the laboratory strive to meet the 

recommended two week hold time.

LABORATORY RECOMMENDATION 4:

The laboratory does not assign expiration dates to new sampling cartridges and allows cartridges 

to be used beyond the 90 days prescribed by the method. Where it discusses the preparation of 

DPNH cartridges, Method TO-11 states, “that cartridges will maintain their integrity for up to 90 

days stored in refrigerated, capped shipping tubes” (TO-11 note, sec. 9.5.2.16). Initial blank lot 

concentrations are provided with commercially purchased cartridges. The auditor views the shelf 

life as stable, but given the significant concern expressed throughout Method TO-11 over 

potential laboratory contamination, it is prudent to be concerned over potential contamination 

during storage. The level of contamination will depend on how the cartridges are stored and if 

they are exposed to contaminants. Therefore, it is likely best practice to recertify for QA 

documentation purposes after 90 days, as suggested by guidance. The District indicated that the 

manufacturer of the cartridges, Waters, allows for a hold time of up to six months. It is 

recommended that, in addition to storing unused cartridges under refrigeration, the laboratory 

verify that cartridges meet blank certification requirements by analyzing a blank before using the

lot past the 90-day PAMS season. Alternatively, the District can follow up with Waters to 

determine the basis for the six month hold time, given studies that show contamination after 90 
41

days.

PM LABORATORY RECOMMENDATION 1:

The laboratory does not have a formalized system for archiving PM2.5 filters. PM2.5 filters are 

currently stored in a refrigerator located in the toxics laboratory. Currently, all filters are stored 

randomly, without any system in place to track the archived filters. While the laboratory

technician was able to locate specific filters on request, a formalized process would provide 

structure and create any easy and traceable way of storing filters. Establish procedures to 

formalize the PM2.5 filter archiving system.

QUALITY ASSURANCE RECOMMENDATION 1:

Conducting data tracking audits on a routine basis may help identify issues throughout the data 

collection, verification and reporting process, and therefore, improve data quality.
42

APPENDIX C: ORGANIZATIONAL CHART