System Audit of the Ambient Monitoring Program:

California Air Resources Board

June - September, 2011

Conducted by:

Air Quality Analysis Office Quality Assurance Office 

Air Division Management and Technical 

Services Division

US EPA Region 9

75 Hawthorne Street

San Francisco, California 94105

Final Report: October 2012
i

PAGE

GLOSSARY OF ACRONYMS.........................................................................ii

EXECUTIVE SUMMARY................................................................................1

TSA ACTIVITIES..............................................................................................5

OVERVIEW OF STATE AIR MONITORING PROGRAM ......................10

OVERVIEW OF LOCAL DISTRICT AIR MONITORING PROGRAMS

IMPERIAL COUNTY APCD..............................................................26

MENDOCINO COUNTY AQMD.......................................................29

SAN JOAQUIN VALLEY APCD........................................................32

FINDINGS.................................................................................................35

APPENDIX A, Summary of Findings...........................................................145

APPENDIX B, Organizational Chart......................................................152

APPENDIX C, CARB Data Validation Documents....................................155
ii

GLOSSARY OF ACRONYMS

ADAM................................... Aerometric Data and Management

AmAPCD...............................Amador County Air Pollution Control District

AMNS....................................Air Monitoring Northern Section

AMSS.....................................Air Monitoring Southern Section

AnAQMD.............................. Antelope Valley Air Quality Management District

APCD.....................................Air Pollution Control District

AQAS.....................................Air Quality Analysis Section

AQDA....................................Air Quality Data Action

AQDAS..................................Air Quality Data Acquisition System

AQDB.................................... Air Quality Data Branch

AQMD................................... Air Quality Management District

AQS....................................... Air Quality System

AQSB.....................................Air Quality Surveillance Branch

ATP........................................Acceptance Test Procedure

BAM...................................... Beta Attenuation Mass Monitor (continuous PM2.5)

BAAQMD..............................Bay Area Air Quality Management District

BCAQMD..............................Butte County Air Quality Management District

CAA.......................................Clean Air Act

CaCAPCD..............................Calaveras County Air Pollution Control District

CARB.....................................California Air Resources Board

CBSA.....................................Core-based Statistical Area

CO..........................................Carbon Monoxide

CoC........................................ Chain of Custody

CoCAPCD............................. Colusa County Air Pollution Control District

CFR........................................Code of Federal Regulations

DQO.......................................Data Quality Objective

EC.......................................... Elemental Carbon

EDCAQMD...........................El Dorado County Air Quality Management District

EKAPCD............................... Eastern Kern Air Pollution Control District

EPA....................................... Environmental Protection Agency

FEM....................................... Federal Equivalent Method

FRAQMD.............................. Feather River Air Quality Management District

FRM....................................... Federal Reference Method

GBUAPCD............................ Great Basin Unified Air Pollution Control District

GCAPCD............................... Glenn County Air Quality Management District
iv

GPS........................................ Global Positioning System

ICAPCD.................................Imperial County APCD

IZS......................................... Internal Zero/Span

LakeCAQMD.........................Lake County Air Quality Management District

LassenCAPCD.......................Lassen County Air Pollution Control District

LC.......................................... Local Conditions

LIMS......................................Laboratory Information Management System

MaCAQMD...........................Mariposa County Air Quality Management District

MBUAPCD............................Monterey Bay Unified Air Pollution Control District

MeCAQMD...........................Mendocino County AQMD

MDAQMD.............................Mojave Desert Air Quality Management District

MFE....................................... Mass Flow Element

MLD.......................................Monitoring and Laboratory Division

MoCAPCD.............................Modoc County Air Pollution Control District

MSA.......................................Metropolitan Statistical Area

NAAQS..................................National Ambient Air Quality Standard

NCore.....................................National Core multi-pollutant monitoring stations 

NCUAQMD...........................North Coast Unified Air Quality Management District

NELAC.................................. National Environmental Laboratory Accreditation 

Conference

NIST.......................................National Institute for Standards and Technology

NLB....................................... Northern Laboratory Branch

NPAP..................................... National Performance Audit Program

NSAQMD.............................. Northern Sierra Air Quality Management District

NSCAQMD........................... Northern Sonoma County Air Pollution Control District

O3........................................... Ozone

OC..........................................Organic Carbon

OMB...................................... Office of Management and Budget

OPAS..................................... Operations, Planning, and Assessments Section

OSS........................................ Operations Support Section

PAMS.....................................Photochemical Assessment Monitoring Stations

Pb........................................... Lead

PCAPCD................................Placer County Air Pollution Control District

PEP.........................................Performance Evaluation Program

PM..........................................Particulate matter

PM2.5...................................... Particulate matter 2.5 microns or less in aerodynamic 

diameter

PM10....................................... Particulate matter 10 microns or less in aerodynamic 

diameter

POC........................................Parameter Occurrence Code

PQAO.....................................Primary Quality Assurance Organization

PTSD......................................Planning and Technical Support Division

QA..........................................Quality Assurance

QAPP..................................... Quality Assurance Project Plan
v

QAS....................................... Quality Assurance Section

QC..........................................Quality Control

QMB...................................... Quality Management Branch

QMP.......................................Quality Management Plan

RH..........................................Relative Humidity

SBCAPCD............................. Santa Barbara County Air Pollution Control District

SCAQMD.............................. South Coast Air Quality Management District

SDCAPCD............................ San Diego County Air Pollution Control District

ShCAQMD............................ Shasta County Air Quality Management District

SiCAPCD...............................Siskiyou County Air Pollution Control District

SJVAPCD.............................. San Joaquin Valley APCD

SLAMS.................................. State or Local Air Monitoring Station

SLOCAPCD...........................San Luis Obispo County Air Pollution Control District

SMAQMD............................. Sacramento Metro Air Quality Management District

SOP........................................ Standard Operating Procedure

SPM....................................... Special Purpose Monitor

SO2......................................... Sulfur Dioxide

STP.........................................Standard Temperature and Pressure

TAD....................................... Technical Assistance Document

TeCAPCD............................. Tehama County Air Pollution Control District

TSA........................................Technical System Audit

TSP.........................................Total Suspended Particulate

TuAPCD................................ Tuolumne County Air Pollution Control District

VCAPCD............................... Ventura County Air Pollution District

VOC....................................... Volatile Organic Compound

Y-SAQMD.............................Yolo-Solano Air Quality Management District
1

EXECUTIVE SUMMARY

This document is a report of the findings made by EPA while conducting a Technical 

Systems Audit (TSA) on the air monitoring program of the California Air Resources Board 

(CARB). A TSA is an on-site review and inspection of a state or local ambient air monitoring 

program to assess its compliance with established regulations governing the collection, analysis, 

validation, and reporting of ambient air quality data. This TSA meets the requirements for EPA 

audits of CARB’s monitoring organization as described in 40 CFR Part 58, Appendix A, Section 

2.5.

Key Findings from TSA:

Finding Potential Impact Recommended Corrective 

Action

Need to formalize PQAO 

(Primary Quality Assurance 

Organization) structure of 

CARB PQAO

Lack of coordination and 

oversight resulting in 

compromised data quality

CARB and local districts 

within CARB PQAO need to 

identify a mechanism to define 

and formally implement the 

partnership

Lack of approved/adopted 

quality system documents

Data defensibility 

questioned

Inconsistent operation of 

ambient air monitoring 

network within CARB and 

within the CARB PQAO

CARB is finalizing its 

updated quality system 

documents

Local districts can adopt 

CARB’s or prepare their 

own

CARB needs effective 

mechanism to share 

updates/changes/additions 

to quality system 

documents such as SOPs

Network management, which 

includes network plans, 

network assessments, site 

closures, and data 

certification, have been 

inconsistently managed across 

air agencies in California

Inability to approve network 

plans

Lack of understanding of 

network requirements

Regulatory decisions 

hindered by loss of required 

sites and lack of data 

certification

More coordination and 

oversight by CARB on 

network plans and data 

certification

Improve analyses and 

coordination for the 5-year 

Network Assessment

Better coordination with 

EPA on site closures
2

Data validation: lack of 

coordination and training has 

resulted in inadequate and 

inconsistent data validation

Erroneous data in the Federal 

regulatory air data system

Training on data validation

Define roles/responsibilities

for data validation

Data audits by CARB

Inconsistent field operations Loss of data

Erroneous data

Defensibility of data 

compromised

Comprehensive training

Audits by EPA and CARB

Establish a field operator 

network

Coordination between CARB 

and local districts needs to be 

improved

All the issues above PQAO listserv

CAPCOA Air Monitoring 

Managers Committee

Standard conference 

calls/meetings for CARB 

PQAO

CARB, an organization under the umbrella of the California Environmental Protection 

Agency, is the governmental agency delegated under State law with the authority and 

responsibility for collecting ambient air quality data as directed by the Clean Air Act (CAA) of 

1977 and CAA Amendments of 1990. CARB and local air pollution control districts (hereafter 

referred to as “local districts”) operate ambient monitoring stations throughout the State. CARB 

is designated as the Primary Quality Assurance Organization (PQAO) for the entire State with 

the exception of the ambient air monitoring programs of the Bay Area Air Quality Management 

District (BAAQMD), the South Coast Air Quality Management District (SCAQMD), and the 

San Diego County Air Pollution Control District (SDAQMD). Many of the smallest local 

Districts do not have active air monitoring programs and rely solely on CARB for the operation 

of monitoring stations within their jurisdictions.

The TSA was conducted by EPA Region 9 staff from June to September, 2011. The audit 

evaluates all air monitoring activities since the previous EPA TSA, which was conducted during 

the summer of 2007. The audit team interviewed management and staff on specific aspects of the 

ambient air monitoring program including network design, field operations, laboratory 

operations, data handling, quality assurance and quality control procedures. The audit team also 

inspected some of the CARB-operated monitoring sites. The site inspections consisted of an 

interview with the site operator when possible, review of station and instrument logbooks, and 

evaluation of the station siting with respect to EPA requirements for probe siting (40 CFR 58, 

Appendix E). The laboratory inspection included a review of the particulate matter program for 

mass determinations, laboratory analysis for volatile organic compounds (VOC), Organic 

Carbon/Elemental Carbon (OC/EC), hexavalent chromium, and carbonyl sample analysis. 

Since CARB oversees the quality of data collected by local districts within the CARB 

PQAO, EPA also reviewed field operations, data management and quality assurance activities at 

local districts. For this TSA, it was not possible to evaluate all of the 21 local districts within the 

CARB PQAO that collect ambient air quality data; the audit team reviewed operations at three 

local districts: San Joaquin Valley Air Pollution Control District (SJVAPCD), Imperial County 
3

Air Pollution Control District (ICAPCD), and Mendocino County Air Quality Management 

District (MeCAQMD). The local districts included in the CARB PQAO have their own 

organizational structures and these vary depending on the size of the local district program. 

SJVAPCD was chosen for review because it is the largest local district in the CARB PQAO and 

has the most significant air quality issues. ICAPCD was chosen as an example of a medium size 

organization and also because of the unique air quality problems that exist in that air basin. 

Finally, MeCAQMD was chosen to be representative of the smaller districts. 

The TSA is part of the oversight system by which EPA ensures that data collected by 

state, local, and tribal agencies meet certain minimum data quality objectives. Other assessments,

such as network reviews and performance evaluations, are also used to collect information on the 

quality of ambient air monitoring data. These assessments enable agencies to identify and correct

those program elements which may be adversely affecting the quality of ambient air data. The 

results of the TSA are summarized here and fully described in this report, along with 

recommended actions to address the findings. The specific actions to be taken by CARB will be 

determined through negotiations between EPA and CARB and will be documented in a 

corrective action plan prepared by CARB. 

EPA would like to thank all the staff and management of CARB and the local districts for 

their support and cooperation during the audit.

A. Program Strengths:

CARB has extensive experience and expertise in ambient air monitoring.

CARB operates a robust audit program, which benefits the entire State of California.

Local districts within the CARB PQAO are committed to addressing air quality concerns 

in their areas and see monitoring as the means by which to assess air quality.

CARB and the local districts that participated in this TSA are dedicated to collecting 

credible and defensible air quality data.

CARB has developed good infrastructure for conducting ambient air monitoring.

B. Program Major Findings:

CARB needs to complete the process of putting a formal PQAO into place. [Finding G1]

The QA Management Branch does not have the structure and sufficient staff to manage 

QA oversight of the PQAO districts. [Finding G2]

While progress has been made on updating the CARB QA Manual with a QMP and 

QAPPs or equivalent documents, the process is behind schedule. [Finding G3]

Local districts within the CARB PQAO do not have updated quality system 

documentation for all activities. [Finding G4]

Coordination between CARB, the local districts and EPA needs to be improved. [Finding 

G6]

Not all agencies within the CARB PQAO have an approved network plan. The current 

approach to network plans does not provide for a determination of network adequacy on a 

statewide basis. [Finding NM1] 

Field sites are operated inconsistently at both CARB and non-CARB sites throughout the 

PQAO. [Findings FO1-8, IMP5-8, MEN4-10, SJV5-8)
4

Data within the CARB PQAO are not validated using consistent procedures. (Findings 

DM2, SJV9, IMP10, and MEN11)

There are numerous deficiencies in the data certification process for the CARB PQAO, 

including: 

o Not all NAAQS-compliant data within the CARB PQAO are routinely certified. 

o Data certified by CARB for local districts are not typically reviewed or validated. 

o Data are routinely certified by local agencies, but responsibility has not been 

formally delegated to any local agencies within the State of California. [Finding 

DM6]

Data uploaded for local districts by CARB’s Air Quality Analysis Section are not 

consistently validated. Erroneous data have been entered into AQS. [Findings DM5, 

IMP10, MEN11]

The individual findings are reported in the topic sections of this document and are also 

summarized in Appendix A. 
5

TSA ACTIVITIES

In the summer of 2011, EPA Region 9 conducted a Technical System Audit (TSA) of the 

ambient monitoring program operated by and overseen by CARB. EPA staff interviewed 

management and staff in three branches of CARB Monitoring and Laboratory Division (MLD) 

and one branch of the Planning and Technical Support Division (PTSD). The TSA covered the 

areas of Air Monitoring Network Management, Field Operations, Laboratory Operations, Data 

and Data Management, and Quality Assurance. In addition, the EPA staff reviewed these same 

areas as implemented by three local districts: SJVAPCD, ICAPCD, and MeCAQMD.

CARB managers and staff were very accommodating to the EPA audit team, making 

themselves and their staff available for many interviews, procedural reviews and monitoring site 

visits. Branch Chiefs interviewed were:

Ken Stroud – Chief, Air Quality Surveillance Branch (AQSB), MLD

Michael Miguel – Chief, Quality Management Branch (QMB), MLD

Cindy Castronovo – Chief, Northern Laboratory Branch, MLD

Karen Magliano – Chief, Air Quality Data Branch (AQDB), PTSD

Many other individual section managers and staff were interviewed in Sacramento and in the 

field. We appreciate the fact that CARB gave the EPA audit team access to all key personnel 

involved in the collection and quality assurance of ambient air quality data.

The EPA regional staff members conducting the TSA were Elfego Felix, Michael Flagg, 

Katherine Hoag, Meredith Kurpius, and Gwen Yoshimura of the EPA Region 9’s Air Quality 

Analysis Office, and Mathew Plate and Steve Remaley of the EPA Region 9 Quality Assurance 

Office. In addition to the EPA Audit Team, Matthew Lakin and Eugenia McNaughton, Managers 

of EPA Region 9's Air Quality Analysis Office and Quality Assurance Office, respectively, 

attended the opening and closing meeting representing EPA management.

The TSA began with a general meeting with CARB managers and staff on June 7, 2011 

at the Monitoring and Laboratory Division office in Sacramento, CA and continued during the 

months of June, July, August, and September, 2011. The TSA covered the following program 

areas:

General / Quality Management.

- Program organization.

- Facilities.

- Independent quality assurance and quality control.

- Planning documents (including QMP, QAPPs, & SOPs).

- General documentation policies.

- Training.

- Corrective action.

- Quality improvement.

- External performance audits.

Network Management / Field Operations.

- Network design.

- Changes to the network since the last audit.

- Proposed changes to the network.
6

- Field support.

Laboratory Operations: toxics and particulate matter.

- Routine operations.

- Quality control.

- Laboratory preventive maintenance.

- Laboratory record keeping.

- Laboratory data acquisition and handling.

- Specific pollutants: PM10, PM2.5, and toxics.

Data and Data Management.

- Data handling.

- Software documentation.

- Data validation and correction.

- Data processing.

- Internal reporting.

- External reporting .

As part of the TSA, EPA tracked supporting documentation for data points/sets from calendar 

year 2010.

This report is divided the following sections: 

Executive Summary – describes the purpose of the TSA and summarizes the major 

findings.

TSA Activities – outlines the timing of this TSA and the programs that were covered.

Overview of Air Monitoring Program – describes the District’s Air Monitoring Program.

Findings – collection of findings and recommendations that includes details associated 

with findings.

Appendix A – list of findings.

Appendix B – CARB organizational charts.

Appendix C – CARB data validation procedures.

The findings and recommendations in this report are grouped by program area. 

Recommended actions to address findings are provided to give some indication of EPA’s 

expectations. If CARB or local districts have other approaches or alternatives to address the 

concerns identified, EPA will consider them, provided the corrective action adequately addresses 

the finding. 

Network Management

EPA interviewed Karen Magliano, Gayle Sweigert and Pheng Lee, and reviewed 

CARB’s Annual Network Plan and Annual Network Assessment as part of this TSA. The most 

recent Annual Network Plan was submitted in July, 2011. CARB submitted a five-year Annual 

Network Assessment as required by 40 CFR 58.10. Both documents address a portion of the 

monitoring network of California; they present information from the smaller agencies that do not 

produce their own reports as well as CARB sites. While both documents include the information 

that is required per CFR, and were approved in 2010 and 2011 as meeting all of the requirements 

for annual network plans, the current structure does not provide for a determination of network 

adequacy or robustness of state-level network planning. To review network adequacy, EPA 

reviewed all the network plans available for the CARB PQAO, in addition to site lists that EPA 
7

has compiled. EPA reviewed the SLAMS monitoring network for the CARB PQAO and 

determined that the network is adequate for all areas within the CARB PQAO.

Field Operations

The CARB site technicians interviewed were Bob Land and Rick Rigsby. Both 

demonstrated a thorough knowledge of the monitoring equipment for which they were 

responsible.

EPA visited five of CARB’s monitoring stations (Colusa, Sutter Buttes, Tuscan Buttes, 

Willows, and Yuba City). A more thorough evaluation was performed at Colusa, Willows, and 

Yuba City. The evaluation at these sites included inspection of the inlet manifolds, examination 

of station and instrument log books, and an evaluation as to whether appropriate QC checks and 

QA audits were being performed. All visible inlet manifolds appeared to be clean. Station 

logbooks and instrument logbooks were not consistently used, and entries often did not follow a 

formal protocol. Some QC checks were not being consistently recorded, and problems were not 

systematically documented. Generally we found that the station operators were very 

knowledgeable, but recordkeeping, corrective action, training, and oversight could be improved.

On June 27, 2011 EPA conducted a review of CARB’s instrument testing, certification, 

and repair procedures. During the review, EPA had the opportunity to interview CARB’s 

Operation Support Section Manager, Reggie Smith, as well as visit the instrument laboratory and 

stockroom for spare parts, both of which are located at the CARB MLD main laboratory. In 

general, considering the extent of CARB’s network, EPA found that the agency maintains an 

excellent instrument testing, certification, and repairs program. Some improvements could be 

made to track malfunctioning equipment so that they may be repaired and reinstalled in order to 

reduce offline time.

Data Management

This section covers data management for criteria pollutants (O3, CO, NO2, SO2, PM2.5,

and PM10); non-criteria pollutant data are addressed in the laboratory section. The following 

managers/staff who have data management responsibilities were interviewed: 

Norma Montez -Air Pollution Specialist, data validator for continuous data for CARB 

sites

Gayle Sweigert - AQS data entry for non-CARB sites, data certification

Pheng Lee and Dwight Oda -AQS data entry for non-CARB sites, data certification

Michael Werst - PM filter data

Quality Assurance/Quality Control

EPA interviewed the MLD Quality Management Branch Chief, MLD Air Quality 

Surveillance Branch Chief, Quality Assurance Section (QAS) Manager, Operations Planning and 

Assessment Section Manager, Air Monitoring North Section Manager, and staff in the Quality 

Assurance Section. EPA evaluated a QAS performance audit and site evaluation at the Yuba City 

monitoring site. Members of the audit team interviewed staff of and reviewed procedures for the 

CARB Standards Laboratory.
8

CARB’s quality management system meets the basic EPA requirements. CARB has a 

QA Manual that has been approved and is currently being updated. CARB’s core program 

conforms to or exceeds the method quality objectives systematically developed by EPA for 

criteria pollutants. EPA and CARB perform national performance and technical evaluations of 

the monitoring network.

Managers and staff interviewed included:

Mike Miguel, QMB Chief

Ken Stroud, AQSB Chief

Merrin Wright, QAS Manager

Joe Guerrero, Air Monitoring North Section (AMNS) Manager

Jeff Wright, Operations Planning and Assessments Section (OPAS) Manager

Chris Deidrick, QAS Staff

Hien Tran, QAS Staff

Patrick Rainey, QAS Staff

Harnek Nijjar, QAS Staff

Particulate Matter Laboratory

EPA visited two gravimetric particulate matter laboratories and interviewed the following 

staff as part of the audit: 

Michael Werst - Inorganic Laboratory Section Manager, 

Ranjit Ahuja - Air Pollution Specialist, lead PM10 laboratory technician, 

Brenda Saldana - Air Pollution Specialist, lead PM2.5 laboratory technician, and 

Michelle Fristoe - Air Pollution Specialist, backup PM2.5 laboratory technician. 

While the laboratory facilities are primarily used to process and weigh PM10 and PM2.5

filters, back-up PM10 and PM2.5 balances are maintained at both facilities so that filters may be 

weighed in either laboratory if issues arise at the primary laboratory location. Both of the 

particulate matter laboratories were well-maintained, neat, and well-organized. Generally, the 

PM10 and PM2.5 laboratory measurements are performed with very good level of technical 

expertise.

Toxics Laboratory Operations

The analysts/chemists were found to be knowledgeable, skilled, and dedicated. CARB is 

in compliance with respect to performing analyses according to EPA methods in most areas of 

the laboratory. Analysts in the following programs were interviewed:

Quality Assurance

All the laboratory staff and managers were asked about quality assurance in the 

laboratory. The laboratory does not have a dedicated QA officer, but most method quality 

assurance activities are being routinely performed throughout the individual departments. 

Some corrective actions from the previous TSA were observed, and there were some 

repeat findings. Staff appeared very receptive to QA findings and suggestions. 

Canister Cleaning - MLD 020
9

Sample Custodian Judy Hodgkins (Air Pollution Specialist) was interviewed. Canister 

cleaning is generally being performed adequately in accordance with accepted protocols. 

Findings in this department mostly focused on outdated SOPs that do not reflect current 

procedure. Procedures to improve quality assurance were discussed. 

Carbonyls Department - MLD 022

John Medina is an analyst with many years of analytical experience but is new to 

carbonyls analysis. Some deviations from method and CARB protocols were identified. 

Hexavalent Chromium - MLD 039

Howard Bakes is responsible for the hexavalent chromium analysis. These analyses are 

generally being performed according to established protocols. The peer review process 

described was exemplary. 

Aromatic and Halogenated Compounds - MLD 066 & Oxygenated Hydrocarbons and 

Nitriles (MLS 058)

Steve Madden and John Bricarello were interviewed. These methods were developed by 

CARB and are generally performed according to protocol. Some opportunities to improve 

documentation were identified.

Review of Agencies within the CARB PQAO

Since CARB oversees the quality assurance of data collected by local districts1within the 

CARB PQAO, EPA also reviewed field operations, data management and quality assurance 

activities at local districts. As it was not possible to evaluate all of the 20 local districts within 

the CARB PQAO that collect ambient air quality data, the audit team reviewed operations at 

three local districts: SJVAPCD, ICAPCD, and MeCAQMD.

Each of the local districts included in the CARB PQAO has its unique organizational 

structure. SJAPCD was chosen for review because it is the largest local district in the CARB 

PQAO and has the most significant air quality issues. ICAPCD was chosen as an example of a 

medium size organization and also because of the unique air quality problems that exist in that 

air basin. Finally, MeCAQMD was chosen to be representative of the small districts. 

1 According to the California State and Local Air Monitoring Network Plan (2011), prepared by the CARB 

Planning and Technical Support Division, Air Quality Data Branch, 21 local Districts operate air monitoring 

stations in the CARB PQAO.
10

OVERVIEW OF THE STATE AIR MONITORING PROGRAM

State and Local Monitoring Agencies within the State of California

CARB, an organization under the umbrella of the California Environmental Protection 

Agency, is the governmental agency delegated under State law with the authority and 

responsibility for collecting ambient air quality data as directed by the CAA of 1977 and CAA

Amendments of 1990. Specifically, CAA Section 110(a)(2)(B)(i) directs the State to “provide for 

establishment and operation of appropriate devices, methods, systems, and procedures necessary 

to...(i) monitor, compile, and analyze data on ambient air quality...” 

There are 35 local air pollution control districts in the state of California (Table 1). Three 

of these local air districts, BAAQMD, SCAQMD, and SDCAPCD are PQAOs. Twenty-one of 

the remaining air districts and CARB, comprising the CARB PQAO, collect ambient air 

monitoring data.

A PQAO is a monitoring organization or a coordinated aggregation of such organizations 

that is responsible for a set of stations that monitors the same pollutant and for which data quality 

assessments can logically be pooled because they have similar quality systems in place. 

Specifically, 40 CFR Part 50 Appendix A Section 3 requires that each ambient air monitoring 

PQAO conform to certain quality management practices. These include:

Having a documented quality system that meets EPA requirements for QMPs and 

QAPPs.

Having a quality management function that is independent of air monitoring operations.

Developing or adopting DQOs, or equivalent systematic planning procedures, for all 

monitoring programs.

Participating in National Performance Evaluation Programs, which consist of 

performance audits used to independently determine program adequacy, national 

monitoring network performance, and national consistency.

Undergoing Technical Systems Audits by EPA at a frequency of every three years or less 

as needed.

Using certified reference materials to standardize monitoring equipment.

EPA views these quality management system components as indispensible to maintain a credible 

monitoring program. Insufficient quality management and control has been cited as rationale to 

support legal challenges to NAAQS designation decisions. 

CARB oversees the quality assurance of data collected by local districts within the 

CARB PQAO. Although both CARB and local air pollution control districts operate ambient 

monitoring stations throughout the state, responsibility for ambient air monitoring ultimately 

rests with CARB. Further, it is the responsibility of CARB to provide QA oversight to ensure 

that data quality within the CARB PQAO meets CFR requirements and conforms to quality 

standards approved in the QAPP.
11

Table 1. List of Air Pollution Control Agencies in California

Air Pollution Control District Ambient Air Monitoring by 

Agency for NAAQS Compliance*

PQAO

Amador County (Am)APCD N CARB

Antelope Valley (An)AQMD Y CARB

Bay Area (BA)AQMD Y BAAQMD

Butte County (BC)AQMD N CARB

Calaveras County (CaC)APCD N CARB

CARB Y CARB

Colusa County (CoC)APCD N CARB

El Dorado County (EDC)AQMD N CARB

Feather River (FR)AQMD N CARB

Glenn County (GC)APCD N CARB

Great Basin Unified (GBU)APCD Y CARB

Imperial County (IC)APCD Y CARB

Eastern Kern (EK)APCD Y CARB

Lake County (LakeC)AQMD Y CARB

Lassen County (LassenC)APCD N CARB

Mariposa County (MaC)AQMD N CARB

Mendocino County (MeC)AQMD Y CARB

Modoc County (MoC)APCD N CARB

Mojave Desert (MD)AQMD Y CARB

Monterey Bay Unified (MBU)APCD Y CARB

North Coast Unified (NCU)AQMD Y CARB

Northern Sierra (NS)AQMD Y CARB

Northern Sonoma County 

(NSC)APCD

Y CARB

Placer County (PC)APCD Y CARB

Sacramento Metro (SM)AQMD Y CARB

San Diego County (SDC)APCD Y SDCAPCD

San Joaquin Valley (SJV)APCD Y CARB

San Luis Obispo County 

(SLOC)APCD

Y CARB

Santa Barbara County (SBC)APCD Y CARB

Shasta County (ShC)AQMD Y CARB

Siskiyou County (SiC)APCD Y CARB

South Coast (SC)AQMD Y SCAQMD

Tehama County (TeC)APCD Y CARB

Tuolumne County (Tu)APCD N CARB

Ventura County (VC)APCD Y CARB

Yolo-Solano (Y-S)AQMD Y CARB

*In some cases CARB and local agencies share ambient air monitoring responsibilities for areas and sites within a 

local agency’s jurisdiction.
12

The ambient air monitoring program in the State of California encompasses many air 

quality assessment activities, including collecting and analyzing data for federal criteria 

pollutants and many other air pollutants of concern; collecting data from special studies as 

directed by the Board; assessing monitoring methods that are used by the State and local districts 

in compliance with federal and state regulations; conducting annual performance audits of all 

monitoring equipment within the PQAO; implementing a calibration and certification of 

measurement standards program; and conducting training in the operation of ambient air 

monitoring instruments. 

Organization of CARB’s Ambient Air Monitoring Program

Responsibility for overseeing the ambient air monitoring program for CARB resides 

within the following branches:

Branch Division

Air Quality Surveillance Branch (AQSB) Monitoring and Laboratory Division (MLD)

Northern Laboratory Branch (NLB) Monitoring and Laboratory Division (MLD)

Quality Management Branch (QMB) Monitoring and Laboratory Division (MLD)

Air Quality Data Branch (AQDB) Planning and Technical Support (PTSD)

QA responsibility for CARB is covered primarily by the Quality Management Branch 

(QMB). Michael Miguel is the QMB Branch Chief. The QMB oversees the development of 

quality management documents with contributions from other branches, conducts on-site audits, 

reviews precision and accuracy data, and initiates corrective action requests (i.e., AQDAs), 

among other duties. The Air Quality Surveillance Branch (AQSB) and Air Quality Data Branch 

(AQDB) contribute to QA activities primarily through data validation activities. QC 

responsibility is handled primarily by the AQSB, which includes field operations, calibrations, 

and repair, precision and accuracy data review/submittal, instrument acceptance testing, 

determination of CARB Federal/State ambient monitoring methods, development of SOPs, 

interpretation of CFRs pertaining to monitoring criteria/methods, and a variety of other

monitoring functions.

The management of QA/QC for local districts within the CARB PQAO is specific to each 

local district. The CARB PQAO does have common QA oversight for instrument audits, since 

CARB conducts audits of all sites within California, but other aspects of QA/QC are specific to 

the relationship that CARB has with each local district. In most cases the local district is 

responsible for QC aspects of the air monitoring program (e.g., zero/precision/span checks, 

calibration, and regular on-site review and maintenance). CARB repairs/replaces instruments for 

some districts, while others do this locally. Except for instrument audits, the approach to QA is 

variable. Many local districts have their own quality management documents; others reference 

the CARB quality management documents. CARB validates the data that it generates (i.e., data 

from their own sites and any filters they weigh), but expects that data generated by local districts 

has been validated by the local district, even in the cases where CARB enters the data into AQS. 

For data certification, CARB certifies its data (i.e., continuous data collected by CARB that are 

limited to CARB sites, and filters weighed by CARB, which includes CARB and non-CARB 
13

sites) and also data for air districts for which it enters data into AQS. CARB expects that all 

other data is certified by the local district.

Network Management

Responsibility for network management lies in the Air Quality Data Branch (AQDB). 

Karen Magliano is the AQDB Branch Chief. Network management responsibilities include 

network assessment and network plan preparation, analyses and decision-making for CARB 

system modifications, and review of requests by local agencies within the CARB PQAO for 

system modifications. Network management for non-CARB sites has traditionally been managed 

by the local agency.

The state network consists of monitoring stations operated by CARB and the local 

districts. The network covers 15 air basins. The four PQAOs in the State of California operate 

monitoring networks that provide data from all the air basins. The three local district PQAOs 

operate their own monitoring networks that provide data for the following air basins: San 

Francisco Bay, San Diego County, South Coast, and Salton Sea2. The CARB PQAO operates 

multiple monitoring networks that cover the following air basins: Great Basin, Lake County, 

Lake Tahoe, Mojave Desert, Mountain Counties, North Central Coast, North Coast, Northeast 

Plateau, Sacramento Valley, Salton Sea, San Joaquin Valley, and South Central Coast. In some 

instances, several local districts operate the monitoring networks in a given air basin. 

Furthermore, the boundaries of metropolitan and micropolitan statistical areas (MSAs), which 

are established by the US Census Bureau, may also overlap air basins and local monitoring 

districts. EPA uses the population statistics of MSAs to determine the minimum SLAMS 

monitoring requirements for criteria pollutants. 

Table 2 summarizes the number of criteria pollutant monitoring sites operated in the 

CARB PQAO.

Table 2: Summary of Criteria Pollutant Monitors in the CARB PQAO

Operating Agency Ozone CO NO2 SO2 PM2.5

3

PM10 TSP

Lead

CARB 3445 1252 38 24 2

AnAPCD 1 1 1 0 1 1 0

BCAPCD 0 0 0 0 1 0 0

GBUAPCD 0 0 0 0 1 17 0

ICAPCD 3 1 1 0 2 6 0

EKAPCD 0 0 0 0 1 1 0

LakeCAQMD 1 0 0 0 1 0 0

MeCAQMD 1 0 0 0 2 1 0

MDAQMD 5 2 3 2 1 5 0

MBUAPCD 5 1 1 0 6 2 0

2

South Coast AQMD collects ambient air monitoring data in Coachella Valley which is part of the Salton Sea Air 

Basin.

3

Includes all FRM/FEM instruments and also non-FEM BAM instruments.

4

Includes Arvin – Di Giorgio (060295002) and Shafter (060296001).

5

Includes Shafter (06029601)
14

NCUAQMD 2 2 2 2 3 5 0

NSAQMD 1 0 1 0 9 0 0

NSCAPCD 1 0 0 0 0 3 0

PCAPCD 2 0 0 0 3 3 0

SMAQMD 764 5 1 5 5 1

SJVAPCD 13 4 9 0 15 7 0

SLOCAPCD 5 0 3 1 3 4 0

SBCAPCD 1075

8

1196

10 2 511 0

ShCAQMD 2 0 0 0 4 5 0

SiCAPCD 1 0 0 0 1 1 0

TeCAPCD 1 0 0 0 1 1 0

VCAPCD 5 0 2 0 9 3 0

Y-SAQMD 2 0 0 0 1 3 0

TOTALS 102 25 51 14 110 102 3

Source: CARB/PTSD/AQDB

EPA reviewed the monitoring network within the CARB PQAO and found that the 

number of sites meets minimum monitoring requirements per 40 CFR 58, Appendix D.

In 2006, EPA added a requirement for Annual Monitoring Network Plans and Five-year 

Network Assessments (40 CFR 58.10). At the time when the first network plan was due (July, 

2007), numerous local agencies within the CARB PQAO expressed an interest in submitting 

their own plan. On this basis, an agreement was reached between EPA, CARB and local districts 

wherein agencies within the CARB PQAO that wanted to submit their own plan could follow the 

process and submit a network plan directly to EPA. CARB submitted a network plan to cover 

those agencies that did not choose to submit a plan (Table 3). The five-year network assessment 

that was due in 2010 followed a similar process. The network plan per 40 CFR 58.14 (a) must 

include recommendations from the network assessment.

6

Includes newly established Lincoln site (AQS ID 06-061-2001).

7

Includes 6 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors 

(AQS IDs: 06-083-1021, 06-083-1013, 06-083-1025, 06-083-1018, 06-083-1014, 06-083-4003).

8

Includes 3 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors 

(AQS IDs: 06-083-1025, 06-083-1008, and 06-083-4003).

9

Includes 8 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors 

(AQS IDs: 06-083-1025, 06-083-1021, 06-083-1018, 06-083-1013, 06-083-2004, 06-083-1014, 06-083-0011, 

06-083-1008, and 06-083-4003).

10 Includes 4 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors 

(AQS IDs: 06-083-1025, 06-083-1013, 06-083-4003, and 06-083-1020).

11 Includes 2 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors 

(AQS IDs: 06-083-1025 and 06-083-4003).
15

Table 3. List of Agencies Drafting Annual Network Plans in California.

Source: CARB 2012 Annual Monitoring Network Plan for Small Districts.

Requirements for developing an adequate network are not based on agency jurisdiction, 

but rather on Core-Based Statistical Areas12 (CBSAs) or PQAOs. A collaborative approach 

between CARB, EPA, and local districts is needed to evaluate whether the requirements for an 

adequate network continue to be met.

Requests for changes to the network may occur outside the network plan process by the 

submission of a letter to EPA. In recent years, CARB and EPA have discussed proposed site 

changes via conference calls. Once a decision is made, CARB sends a formal request to EPA for 

12 The definition of a Core-Based Statistical Area can be found at http://www.census.gov: Metropolitan and 

micropolitan statistical areas (metro and micro areas) are geographic entities defined by the Office of 

Management and Budget (OMB) for use by Federal statistical agencies in collecting, tabulating, and publishing 

Federal statistics. The term "Core Based Statistical Area" (CBSA) is a collective term for both metro and micro 

areas. A metro area contains a core urban area of 50,000 or more population, and a micro area contains an urban 

core of at least 10,000 (but less than 50,000) population. Each metro or micro area consists of one or more 

counties and includes the counties containing the core urban area, as well as any adjacent counties that have a 

high degree of social and economic integration (as measured by commuting to work) with the urban core.
16

approval. When a local district wishes to modify its network, e.g., shutting down or relocating an 

existing site or establishing a new one, it will often consult with CARB informally before 

submitting a request to EPA, but the process for local districts has been generally less consistent.

Field Operations

Network operations at CARB are primarily performed by the MLD AQSB. Ken Stroud is 

the Branch Chief of AQSB. AQSB duties include the operation of CARB monitoring sites, 

monitoring support for CARB special studies, and general air monitoring support, such as repair 

and calibration facilities. AQSB also assists local districts with instrument trouble-shooting and 

repair as resources allow. This section of the TSA report addresses AQSB's general operations, 

the calibration program, and field operations of the AQSB at CARB-operated criteria pollutant 

monitoring sites. 

Providing training and performing instrument certifications are the responsibilities of the 

Operations Support Section (OSS) within AQSB. The OSS also provides independent review 

and approval of field SOPs. Other responsibilities include instrument repair and technical 

support. While support (e.g., training, field procedures, and other technical support) is available 

to the all local districts in California, the AQSB does not actively manage the local districts’ field 

monitoring quality systems nor does it have the resources to do so. 

Laboratory Operations

Analytical laboratories provide support for measurement methods that are either too complex or 

sensitive to perform in the field environment. In order to provide these services, laboratories 

have highly trained staff in charge of the complex instrumentation. If analyses are to be used for 

regulatory purposes, they must meet the following criteria: 

Equipment must be frequently and properly calibrated and maintained. 

Personnel must be qualified to make the analysis. 

Analytical procedures must be in accordance with accepted practice. 

Complete and accurate records must be kept. 

The CARB MLD Northern Laboratory Branch (NLB)13 is divided into three sections: the 

Inorganic Laboratory Section, Organic Laboratory Section, and the Special Analysis Section. 

Cindy Castronovo is the Branch Chief of NLB. The laboratory facility is adequate for NLB’s 

needs. The laboratory provides analytical support for the criteria pollutants PM10 and PM2.5.

Additionally, the laboratory supports the EPA PM Speciation Trends Network (STN), the 

California Air Toxics Monitoring Network, and Special Study Monitoring. CARB’s NLB 

laboratory facility is located in Sacramento, California. Analyses are performed in-house by 

laboratory staff. 

Particulate Matter Laboratory (Gravimetric Laboratory)

The gravimetric laboratory operations are managed by Michael Werst, Inorganic 

Laboratory Section Manager. Analyses are performed in-house by laboratory staff. The 

13 CARB also has a Southern Laboratory Branch but this laboratory does not handle ambient air monitoring 

sample and analyses but rather source testing and other non-ambient samples.
17

particulate matter laboratory is responsible for the handling of PM filters, which includes 

preparation, weighing, tracking, and storing PM2.5 and PM10 filters. 

Toxics Laboratory

In addition to PM responsibilities, the laboratory is also responsible for air toxics 

monitoring analysis. This TSA focused on the following compounds:

Carbonyls (Method MLD022 MEK, acetaldehyde).

Hexavalent chromium.

Aromatic and halogenated compounds.

Oxygenated hydrocarbons and nitriles.

There are a number of additional activities that the laboratory undertakes to support the 

collection and analysis of air pollutants. These include canister cleaning and preparation, data 

validation, and sample storage. 

Data Management

Data management generally involves data collection and validation supported by a data 

management system. A primary goal of the EPA’s Quality System is “to ensure that 

environmental programs and decisions are supported by data of the type and quality needed for 

their intended use...” (EPA Quality Manual for Environmental Programs, EPA Order 5360A1 

[EPA, 2000a]). Achievement of this goal involves planning, implementation and assessment of 

the data collection process. Data verification and validation are key steps in the assessment of 

environmental measurements. EPA defines data verification as the process of evaluating 

completeness, correctness and compliance of a data set against the method requirements. Data 

validation extends the verification process to determine the analytical quality of a data set. As a 

part of this TSA, EPA evaluated CARB’s process of data handling, verification, validation, 

storage and upload to AQS of ambient monitoring measurements.

On-going data collected from ambient air monitoring stations can either be generated by an 

analyzer in situ (continuous data) or by subsequent laboratory analyses of a sample (laboratory

data). Choosing an appropriate data management process depends on whether the data are 

generated continuously on site or in the laboratory. Within the CARB PQAO, the different 

agencies collect and generate air quality data. In addition to the on-going collection of air quality 

data, periodic QC checks generate data that must also be managed. Table 4 lists which types of 

data are uploaded by which agency:
18

Table 4. Summary of data upload and certification responsibilities for the State of 

California.

Agency 

Operating Site

Agency 

Uploading 

Continuous Data

Certified By? Agency 

Uploading PM 

Filter-based 

Laboratory Data

Certified By?

AnAPCD MDAQMD Not certified* MDAQMD Not certified

CARB CARB (MLDAQSBCARB (AQAS) CARB (NLB) / 

VCAPCD - PM2.5 

(2 CARB 

sites)/SDCAPCD 

(1 CARB site)

CARB (AQAS)/

not certified/not 

certified

GBUAPCD GBUAPCD GBUAPCD GBUAPCD GBUAPCD

ICAPCD CARB (AQAS) CARB (AQAS) CARB (NLB) -

PM10 / SDCAPCD 

-PM2.5

CARB 

(AQAS)/PM2.5-

not certified

LakeCAQMD CARB (AQAS) CARB(AQAS) CARB (AQAS) CARB (AQAS)

MeCAQMD CARB (AQAS) CARB (AQAS) N/A N/A

MDAPCD MDAPCD Not certified* MDAPCD Not certified

NCUAQMD CARB (AQAS) CARB (AQAS) CARB (AQAS)-

PM10

+

/

(Bay Area)-PM2.5

CARB(AQAS)/

PM2.5-not certified

NSAQMD NSAQMD NSAQMD CARB (NLB) CARB (AQAS)

MBUAPCD MBUAPCD MBUAPCD BAAQMD Not certified

NSCountyAPCD CARB (AQAS) CARB (AQAS) CARB (AQAS)-

PM10

CARB (AQAS)

PCAPCD CARB (AQAS) CARB(AQAS) N/A N/A

SMAPCD CARB (AQAS) CARB (AQAS) CARB (NLB) –

PM2.5 and PM10

coarse / CARB 

(AQAS) – PM10

+

CARB (AQAS) 

PM2.5, PM10 coarse 

and PM10

+

SJVAPCD SJVAPCD SJVAPCD VCAPCD Not certified

SLOCAPCD SLOCAPCD SLOCAPCD N/A N/A

SBCAPCD SBCAPCD SBCAPCD SBCAPCDxSBCAPCD

ShCAQMD ShCAQMD ShCAQMD CARB (NLB) CARB (AQAS)

SiCAPCD CARB (AQAS) CARB (AQAS) CARB (NLB) CARB (AQAS)

TeCAPCD CARB (AQAS) CARB (AQAS) CARB (NLB) CARB (AQAS)

VCAPCD VCAPCD VCAPCD VCAPCD VCAPCD

Y-SAQMD CARB (AQAS) CARB (AQAS) CARB (NLB) CARB (AQAS)

* O3 data for 2009-2011 was certified by the local district in 2011 only.

+ These data are weighed by local district but uploaded by CARB (AQAS)

x Applies only to filter-based PM10 measurements. Continuous sampling will begin in 2012.
19

California has five organizational units in two different Divisions of CARB, and 26 

separate Air Pollution Control Districts through which ambient monitoring data enters EPA’s 

AQS database. Responsibility for managing the state’s CAA-required ambient monitoring data is 

divided between the following groups (descriptions of each group’s role follows the list):

1. Continuous data from CARB-operated field monitoring stations – MLD/AQSB, Ken 

Stroud, AQSB Chief.

2. CARB laboratory analytical data for both CARB and non-CARB Sites – MLD/NLB,

Cindy Castronovo, NLB Chief .

3. Quality assurance performance audit program data – MLD/QAS, Merrin Wright, QAS 

Manager .

4. Special purpose monitoring projects and Standards Laboratory– QMB/OPAS, Jeff 

Wright, OPAS Manager; and MLD/Special Purpose Monitoring Section (SPMS), Mac

McDougall, SPMS Manager.

5. Local district site data: operated by local district but AQS-uploaded by CARB –

PTSD/AQAS, Gayle Sweigert, AQAS Manager .

6. Local district site data: operated and AQS-uploaded by local district – various local air 

pollution control agencies.

7. Local District Site Data: Operated by Local District with Laboratory Analyses by a 

Different Laboratory that is not CARB – responsibility is variable and unclear.

1. Continuous Data from CARB-operated Field Monitoring Stations

Continuous data collected at CARB-operated field monitoring stations includes data from 

all continuous air quality analyzers (O3, NO2, CO, SO2, and non-filter based PM) and 

meteorology data. The CARB-operated ambient monitoring stations are managed by Ken Stroud, 

AQSB Chief. There are two regional Supervisors, Joe Guerrero (AMNS), and Fernando Amador 

(Air Monitoring Southern Section [AMSS]). Air quality data measured by the continuous 

analyzers at the field stations operated and maintained by CARB are stored in data loggers and 

station computers. Each station is polled hourly by modem and the data are transmitted directly 

to CARB’s central computer system in Sacramento. The computer system consists of a server 

located within a leased facility and a backup server located in a separate leased facility. The data 

are collected in the AQDAS, which was developed by EMC, Inc. The AQDAS (now AQDAS-II) 

is CARB’s primary management tool for data collection, validation, and reporting of data 

obtained at CARB-operated stations. Data are retained in AQDAS-II for 180 days within which 

time they are uploaded to AQS. Once submitted to AQS, the data are downloaded to the CARB 

database Air Data Management System (ADAM). ADAM is CARB’s official state database for 

ambient air quality data. Chart recorders and data loggers located at each station provide a 

supplemental record for the data validation process; the printouts are stored for four years as 

primary data records. 

At the time of the audit, the updated SOP for data validation had not been finalized but 

the Air Pollution Specialist, who conducts the second-level review, provided some overview 

sheets that describe the data review process (see Appendix C). The first review (first-level 

validation) of the data is performed by the CARB station operators. Each field operator has 

password-protected access to data from his/her own field sites. If data require correction, the 

station operator makes a notation on the data logger or chart recorder at the station and edits the 
20

data set. The need for data correction may originate from the QAS in the form of an Air Quality 

Data Action (AQDA). In these cases, the field operator reviews the data and determines and 

justifies the appropriate action.

The second-level review is done by an Air Pollution Specialist and/or an Air Resources 

Engineer; this includes a review of data flags, completeness, QC charts, audit results, monthly 

max/mins, and maintenance check sheets. The overview sheets for the second-level review 

(Appendix C) indicate that the second-level review should include data comparisons, such as 

tracking of pollutants, NO2 x, PM2.5<PM10, etc. No formal process of data comparison 

(e.g., charts, figures, calculations) was observed. Instead, the second-level reviewer scans 

printouts of data and spot checks strip charts. Any data corrections identified in the first-level 

review are reviewed and confirmed by the Air Pollution Specialist. In general, the Air Pollution 

Specialist looks at daily quality control checks, required QA checks (e.g., audits), monthly 

maintenance checks, and outliers. We did not observe any reviews of concentration patterns 

(e.g., seasonal or diurnal) or levels, nor any review of instrument drift. The data stream then 

proceeds to the next level of review (third-level validation) by the appropriate Section Manager 

(Northern or Southern Section) who reviews data for completeness and considers any significant 

issues that have been identified by previous-level review. 

At this point, a final data validation summary report, the monthly data report, is 

produced in the form of a memo to the AQSB Branch Chief identifying any significant issues for 

each site and reporting on completeness for all parameters. Upon approval by the AQSB Branch 

Chief, the data are stored in the state archive system and submitted to the EPA AQS database. 

AQS is submitted by the data validator, Norma Montez, through a password-protected system on 

her computer. In summary, data validation for continuous pollutants goes through the following 

steps:

1. Review by station operator.

2. Review by Air Pollution Specialist and/or Air Resources Engineer.

3. Review by Section Manager.

4. Review by AQSB Branch Chief.

5. Upload to AQS by Norma Montez.

Overall, CARB submits all required data to the EPA AQS database, including 

concentrations for all criteria pollutants, and supporting precision and accuracy information. 

CARB certifies these data annually as required by regulation (40 CFR 58.15).

2. CARB Laboratory Analytical Methods for both CARB and Non-CARB Sites

Overview:

Data flow in the laboratory begins with the chemist, who runs the analytical method and 

generates the data (gravimetric or chemical analysis). Once collected, all laboratory data are 

stored in CARB’s Laboratory Information Management System (LIMS). The original LIMS was 

a product purchased from Perkin-Elmer, but the system has had many modifications to customize 

it for use by CARB over the years. The LIMS database, housed in the Monitoring and 

Laboratory Division, is backed up to tape once per week. It is accessible to all chemists and 

managers. The system makes use of limited access and password-protection for security. The 
21

raw data in the system are stored for five years. LIMS assigns QC flags as defined by CARB 

SOPs. All data are subjected to peer-review for level-two data validation, which is followed by 

reviewing and ‘locking’ of the data by laboratory managers. Data peer-review groups are 

organized around the analytical methods: PM10, PM2.5, PM2.5 speciation, and TSP-lead. The QC 

criteria as written in the laboratory and analytical methods are used for data validation.

Gravimetric (PM) Laboratory:

PM10 and PM2.5 filters arrive at the NLB with a Chain of Custody (CoC) form. The 

gravimetric laboratory handles filters from both CARB and local districts. Samples are linked 

with a barcode, which is read with a barcode reader. Mass data that are linked to its barcode are 

entered automatically from the balance. The chemist enters field information from the CoC form 

manually, which does not include the mass data. Flags can be identified by anyone on the CoC. 

The PM data management process includes many useful features including:

Automatic checks on parameters such as flow and pressure and outlier values are 

highlighted .

Hold times for filters are tracked and priority of use indicated based on hold times .

A scheduling report identifying missing samples.

The field operators review instrument operation and note any need to flag data on the 

CoC. The first level of data validation is done by the station operator. For sample validation, the 

chemist who weighs the filters does the second level of data validation. The chemist verifies 

sample receipts, information on CoC, correct logging of data into LIMS, and QC data. A 

monthly data package is generated and provided to a peer chemist (i.e., someone not involved in 

the data generation) for the third-level data validation. After the third-level data validation is 

done, the monthly data package, along with any notes on the data, is provided to the Inorganic 

Laboratory Section Manager for fourth-level review. The NLB Chief approves the data. The 

cover sheet on the monthly data package includes a summary of the results of each level of 

review. Once data are approved by management, they are locked and can only be changed with 

management approval. Data are uploaded to AQS after approval by NLB Chief.

All PM data weighed and entered by the CARB laboratory is certified by CARB annually 

as required by regulation (40 CFR 58.15).

Toxics Laboratory:

The toxics laboratory follows the same general protocols for data entry and validation as 

the rest of the laboratory. 

3. CARB Quality Assurance Data 

Quality assurance performance data include state-wide annual performance audits 

conducted by CARB and a comparison between CARB and EPA’s audit systems findings. The 

Quality Assurance Section in the MLD conducts performance evaluation audits and technical 

system audits at ambient air monitoring stations throughout the state. Performance audits of each 

local air pollution control district are conducted annually for gaseous criteria pollutant 

monitoring and particulate matter monitoring flow. The results of the audits are maintained 

online on the CARB website and are uploaded by CARB to AQS in most cases. In some 
22

instances, CARB has not received update rights to some local district’s screening files in AQS;

in these cases the local district or EPA uploads the data. EPA conducts an annual comparison 

with the CARB audit vehicles to ensure comparability with EPA’s National Performance Audit 

Program (NPAP) and Performance Evaluation Program (PEP); these data are uploaded to AQS

by CARB.

In addition to collecting and managing data from audits, the QAS reviews quality 

assurance data for the entire state. At least once a quarter, QAS staff retrieve AMP 255 reports 

from AQS for all California sites and all pollutants. QAS staff review the report for 

inconsistencies and work with each agency to address any issues. QAS also verifies and validates 

the AMP 255 reports for the data certification process.

4. CARB Special Purpose Monitoring

Special Purpose Monitoring is conducted on an as-needed basis by the following two 

sections within MLD: the Operations Planning and Assessment Section, in the Quality 

Management Branch, and the Special Purpose Monitoring Section, in AQSB. They are 

responsible for emerging air monitoring issues. In most cases, the data are uploaded to AQS. 

EPA did not interview staff in this section about data management practices.

5. Collected by Local District / uploaded to AQS by CARB

The Air Quality Analysis Section (AQAS) in the Planning and Technical Support 

Division is responsible for uploading continuous air quality data (O3, NO2, CO, SO2, and nonfilter based PM), PM data weighed by the local district, and meteorology data from those local 

districts without direct access to AQS. These districts include: ICAPCD, LakCAQMD, 

MeCAQMD, NCUAQMD, NSCAPCD, PCAPCD, SMAQMD, SiCAPCD, TeCAPCD, and YSAQMD (Table 4). The AQAS is located at the Cal EPA Headquarters building in Sacramento, 

CA and is managed by Gayle Sweigert. AQAS uploads continuous PM data from local districts, 

as well as PM filters data that are weighed by LakeCAQMD, NCAQMD, NSCAPCD and 

SMAPCD. PM filters that are collected by local districts and then weighed by CARB are

managed by the CARB MLD Northern Laboratory Branch.

Data are received electronically by email or as hard copy through the mail from the ten 

local districts listed in the previous paragraph. Local districts typically send their data monthly in 

the form of an electronic file. When data arrive at AQAS, they are logged in and uploaded to 

AQS, which runs them through AQS review protocols. Staff sends an email to the district staff 

confirming the upload, which has attached the AQS raw data report and the raw data inventory 

report. If the AQS review protocols identify a potential issue, AQAS staff contacts the district to 

resolve it. According to the CARB SOP, AQAS staff may not alter data without consent from the 

district. AQAS staff do not validate the data. CARB assumes that the data to be uploaded to 

AQAS for local districts have been fully validated, and carry the appropriate flags.

Data uploaded by AQAS for the ten districts listed above are certified by CARB 

annually, as required by regulation (40 CFR 58.15).
23

6. Collected and Uploaded to AQS by Local District

The following districts have access and authority to upload their data to AQS: 

GBUAPCD, MDAQMD (includes AnAPCD), NSAQMD, MBUAPCD, ShCAQMD, SJVAPCD, 

SLOCAPCD, SBCAPCD, and VCAPCD (see Table 4). The reporting of data into AQS by these 

local districts was agreed upon and commemorated in district-specific Memorandums of 

Understanding signed by the districts, CARB, and EPA starting in 2002. All levels of data 

management are handled by the local districts; CARB is neither involved nor familiar with data 

management protocols of districts that submit their own data. CARB expects the data that it does 

not upload to be certified per regulation (40 CFR 58.15) by either the local district collecting the 

data or by the agency uploading the data.

7. Collected by Local District / Laboratory Analyses Performed by a Laboratory Other Than 

CARB

All continuous data collection is managed by each local district; CARB uploads data for 

ten of the districts (see Table 4). Data generation and management for PM filter-based data are 

not only district-specific, but can even be site-specific. Some districts weigh their filters and 

upload and certify their PM data. Other districts collect the filters and send them to CARB for 

weighing, validation, upload, and certification. Other districts collect the filters and send them to 

another local district that has a weigh laboratory; responsibility for data validation, upload, and 

certification in these cases is not always clearly defined. Finally, some districts may send filters 

from some sites to CARB and filters from other sites to a different local district that may be more 

convenient to the site. In the case where all filters are sent to a different local district, 

responsibility for data validation, upload, and certification of the filters is not always clearly 

defined. For example, until 2011, filters from a CARB site in the San Joaquin Valley were being 

sent to and weighed by VCAPCD.

Quality Assurance/ Quality Control

An organization’s quality management system includes quality assurance, quality control, 

and quality improvement activities. EPA requires that ambient air monitoring agencies have a 

quality management system that conforms to 40 CFR Part 58 Appendix A and the EPA quality 

policy (EPA Order CIO 2106.0). Additionally, EPA grant regulations specifically require each 

grantee to provide for QA activities (40 CFR Part 31.45). Specifically, 40 CFR Part 58 Appendix 

A Section 2 requires that each ambient air monitoring PQAO conforms to certain quality 

management practices. These include:

A documented quality system that meets EPA requirements for QMPs and QAPPs.

A quality management function that is independent of air monitoring operations.

Stated data quality objectives or equivalent systematic planning procedures for all 

monitoring programs.

Participation in National Performance Evaluation Programs, which consist of 

performance audits to independently determine program adequacy, national monitoring 

network performance, and national consistency.

EPA-led Technical Systems Audits every three years or less.

Use of certified reference materials to standardize monitoring equipment.
24

EPA views these quality management system components as being integral to maintaining a 

credible monitoring program. Insufficient quality management and control has been cited in 

support of legal challenges to NAAQS designation decisions. 

Quality assurance and quality control are the two components of a quality management 

system for a monitoring program that serve to document the assertion that the data collected 

represent the true air quality of the area. They are the means by which an organization manages 

its quality aspects in a systematic, organized manner to provide a framework for planning, 

implementing, and assessing work performed by an organization. A properly developed QA/QC 

program encompasses a variety of technical and administrative elements, including policies and 

objectives, organizational authority, responsibilities, accountability, and procedures and 

practices. Quality assurance is a management or oversight function setting policy and overseeing 

an administrative system of management controls that cover planning, implementation, the 

review of data collection activities, and the use of data in decision making. Quality control is a 

technical function that includes all the scientific precautions, such as calibrations and 

duplications needed to acquire data of known and adequate quality. 

The CARB Quality Management Branch (QMB) is composed of two sections: the 

Quality Assurance Section (QAS) and the Operations Planning and Assessment Section (OPAS), 

which includes the Standards Laboratory. The QAS’s primary responsibilities include:

Conducting performance audits of MLD and district monitoring instruments.

Assisting with system audits of California air districts.

Updating standard operating procedures (SOPs) specific to the QAS’s activities.

Validating MLD’s field generated monitoring data (accuracy assessments).

Preparing annual reports on the status of QA activities occurring in MLD.

Preparing data quality summary reports for Reporting Organizations and districts in 

California. 

The OPA section is responsible for providing recommendations to MLD laboratories to ensure 

the defensibility of the laboratory data. The Standards Laboratory resides in OPA and performs 

standards certifications for all MLD gaseous, O3, flow, and meteorological transfer standards. 

Most districts within the CARB PQAO choose to employ these services as well. 

QC-related functions are performed by the Air Quality Surveillance Branch (AQSB). The 

AQSB performs several quality management functions. These include: 

Developing and administering the training program for instrument operators.

Performing instrument acceptance testing.

Validating MLD field generated monitoring data.

Developing, preparing and reviewing SOPs for CARB’s air monitoring program.

Determining monitoring methods used in CARB’s ambient air monitoring network.

QA-related functions in the Northern Laboratory Branch include:

Developing laboratory and ambient air collection test procedures.

Conducting analyses of ambient air samples and consumer products.
25

Performing self-assessments quarterly and producing a quality control summary report 

for the Division Chief.

QA-related functions are divided among CARB’s air monitoring operations. As a result, 

it can be difficult for the QMB to coordinate QA activities. Moreover, the scope and organization 

of the various QA activities are not fully understood by the QMB. With the exception of AQDA 

forms that are issued primarily out of the QAS and the Standards Laboratory in OPA, and the 

technical bulletins from the AQSB, corrective action is limited. The process would benefit from 

expansion in scope and documentation. CARB has all the necessary components for an effective 

and robust QA system. Each division involved in the collection and reporting of ambient air data 

understands and performs the relevant QA functions. Expanding the oversight authority of the 

QMB, and developing and implementing an expanded corrective action process would enhance

CARB’s QA system.

The QMB staff are not aware of the extent to which QA activities are performed at the 

local districts. The districts in the CARB PQAO are expected to follow the MLD Quality 

Assurance Program Plan (QAPP). 
26

OVERVIEW OF IMPERIAL COUNTY AIR POLLUTION CONTROL DISTRICT AIR 

MONITORING PROGRAM

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of 

ICAPCD. ICAPCD is currently part of CARB PQAO, but this audit included an agency-specific 

assessment of network design, field operations, data handling, quality assurance and quality 

control procedures. EPA staff interviewed ICAPCD management and staff and visited all the 

monitoring sites located in Imperial County: Calexico Ethel, Niland, Brawley, Westmorland, and 

El Centro.

ICAPCD managers and staff were very accommodating, making themselves available for 

many interviews, procedural reviews and monitoring site visits. Management and staff 

interviewed were:

Brad Poiriez – Air Pollution Control Officer

Reyes Romero – Assistant Air Pollution Control Officer

Jesus Rameriz – Air Pollution Control Division Manager

Monica Soucier – Air Pollution Control Division Manager

Michael Green – Air Pollution Control Technician

Jon Barroga – Air Pollution Control Technician

Emmanual Sanchez – Air Pollution Control Environmental Coordinator 

Overall, the monitoring staff is very dedicated, knowledgeable, and operates the 

monitoring network to the best of their ability. As described in the attached findings, the major 

deficiency in the monitoring program is the lack of a quality system for ambient air monitoring. 

A quality system is the means by which an organization manages the quality of the monitoring 

information it produces in a systematic, organized manner; it provides a framework for planning 

implementing, assessing and reporting work performed by an organization and for carrying out 

required quality assurance and quality control activities.

Some of the findings in this TSA pertain to CARB’s role as a PQAO and its relationship

and oversight responsibilities to local districts. Increased communication and coordination 

between ICAPCD and CARB is needed to effectively maintain the ambient air monitoring 

network in Imperial County.

Network Management

There are five monitoring sites in Imperial County. Four sites are operated by ICAPCD 

(Table 2).

ICAPCD submits its Annual Monitoring Network Plan directly to EPA. If a local district 

within the CARB PQAO, such as ICAPCD, seeks to make changes to its network outside the 

Annual Monitoring Network Plan process, the suggested process is for the district to first work 

with CARB to resolve potential issues and then submit its request to EPA per 40 CFR 58.14 with 

a copy to CARB.
27

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for PM10,

PM2.5 and O3 are being met for Imperial County.

Field Operations

Some quality control checks and maintenance are performed in accordance with EPA 

regulations. Field technicians are responsible for day-to-day operations, as well as minor 

instrument repair and preliminary data validation. The monitoring stations operated by the 

district are set up to perform automated nightly internal zero/spans (IZS). One-point precision 

checks for O3 are performed manually, but PM10 and PM2.5 flow verifications are not performed 

by ICAPCD operators. 

ICAPCD does not have its own SOPs; staff stated that they use CARB SOPs. Hardcopies 

of the SOPs are kept at the sites and online via the CARB website. Operators keep track of 

unusual events or anomalies for continuous instruments in the station log, though records at the 

sites are generally not sufficiently detailed or organized. Any special events or anomalies for 

PM10 and PM2.5 filters are recorded on the CoC sheet and sent to CARB and SDCAPCD with the 

filter, respectively (see Laboratory Operations). Although standard logbooks are not in place, 

alternative documentation methods are utilized such as notes on the station calendar and monthly 

maintenance sheets.

Corrective actions and repair/maintenance are dealt with on a case-by-case basis. 

Generally, minor equipment repairs are performed by ICAPCD, while major repairs are 

performed by the CARB Southern California office in El Monte.

Laboratory Operations

ICAPCD does not operate a PM laboratory; it sends PM10 filters to CARB and PM2.5

filters to SDCAPCD. ICAPCD uses CARB laboratories for the following analyses: O3 primary 

standard verification, protocol gas certification, flow, and meteorological transfer standard 

calibration, instrument calibration, and major instrument repair. 

Data Management

Currently, neither ICAPCD nor CARB are assessing whether the required data quality 

objectives and measurement quality objectives have been achieved. 

ICAPCD station operators perform a preliminary assessment of the gaseous and 

continuous PM10 raw data, though this process is generally not documented or performed 

following an SOP or other procedures outlined in a relevant QAPP. After initial editing is 

performed, raw text files are sent to CARB for submission to AQS. Neither ICAPCD nor CARB 

perform further (i.e. Level II) validation of the data. 

Filter based PM10 and PM2.5 data are processed, validated, and submitted to AQS by 

CARB Northern Laboratory Branch and SDCAPCD, respectively, and follow procedures

outlined in agency-specific QAPPs and SOPs.
28

QA/QC

ICAPCD conducts some QA/QC activities and relies on CARB to support others. QA/QC 

activities conducted by ICAPCD include one-point QC checks for gaseous monitors. ICAPCD 

relies on a CARB site operator to perform the required flow verifications for PM10 and PM2.5.

ICAPCD should be conducting these activities, but does not have the equipment required to 

perform the required checks. CARB also conducts the following QA/QC activities: gaseous 

annual performance evaluations and semi-annual flow rate audits for PM10 and PM2.5, and flow, 

meteorological ozone, and gaseous standard verifications and certifications. EPA contractors visit 

the sites annually, making calibration checks on the PM monitors, as well as conducting

comparison monitoring using EPA field equipment.

Although ICAPCD staff stated that they are using CARB Quality Management 

documents, this audit indicated that ICAPCD is not currently following approved CARB QAPPs, 

SOPs, or approved equivalents. ICAPCD does not have a QA manager, a formal corrective 

action process, or an independent audit program.
29

OVERVIEW OF MENDOCINO COUNTY AIR POLLUTION CONTROL DISTRICT 

AIR MONITORING PROGRAM

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of 

MeCAQMD. MeCAQMD is currently part of the CARB PQAO. MeCAQMD funds the bulk of 

its monitoring program with district funds. The district does not receive any direct support 

funding for its monitoring program from CARB. MeCAPCD’s sources of funding for air 

monitoring activities are EPA’s CAA 103 grant support for PM2.5 ($5000 per site), permit fees, 

and vehicle fees. Since 2000, there has been a substantial drop in permit revenue, which has 

resulted in a 40% reduction in staffing, including the elimination of one full time air quality 

monitoring technician.

This audit included an agency-specific assessment of network design, field operations, 

data handling, quality assurance and quality control procedures. In July 2011, EPA staff 

interviewed MeCAQMD management and staff and visited all four of the monitoring sites 

located in Mendocino County: the Ukiah AQMD (06-045-0008), Ukiah Library (06-045-0006), 

Willits (06-045-2002), and Fort Bragg (06-045-0002).

The MCAQMD manager and staff were very accommodating, making themselves and their 

staff available for interviews, procedural reviews and monitoring site visits. Management and 

staff interviewed were:

Christopher D. Brown – Air Pollution Control Officer

Robert Scaglione – Senior Air Quality Specialist

Overall, the staff and manager were dedicated and professional, and very knowledgeable 

about the county and potential pollution sources. As described in the attached findings, the 

major deficiency in the monitoring program is the lack of a structured and formalized framework 

that is inherent to a functioning quality system required for ambient air monitoring. Some of the 

findings in this TSA pertain to CARB’s role as a PQAO and its relationship and oversight 

responsibilities to local districts. Increased communication and coordination between 

MeCAQMD and CARB is needed to effectively maintain the ambient air monitoring network in 

Mendocino County.

Network Management

There are four monitoring sites in Mendocino County, as identified in Table 5. All four 

sites are operated by MeCAQMD.
30

Table 5. Ambient Air Monitoring Network in Mendocino County, California

AQS ID Station O3 PM10 continuous PM2.5 continuous

06-045-0002 Fort Bragg X (POC 2)

06-045-0006

Ukiah 

Library X (POC 3)

06-045-0008

Ukiah 

AQMD 

(Gobbi) X (POC 3)

06-045-2002 Willits X (POC 1)

The MeCAQMD network is included in CARB’s Annual Monitoring Network Plan. If a

local district within the CARB PQAO, such as MeCAQMD, seeks to make changes to its 

network outside the Annual Monitoring Network Plan process, the suggested process is for the 

district to first work with CARB to resolve potential issues and then submit its request to EPA 

per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements, as outlined in 40 CFR 58, Appendix D for PM10,

PM2.5 and O3 are being met in Mendocino County.

Field Operations

MCAQMD operates a network of O3 and PM monitoring instruments. The manager and 

staff responsible for field operations are Christopher D. Brown (Air Pollution Control Officer) 

and Robert Scaglione (Senior Air Quality Specialist).

The field technician exhibited a thorough knowledge of equipment operations. Some 

quality control checks and maintenance are performed in accordance with EPA regulations. The 

field technician is responsible for day-to-day operations as well as instrument repair and 

maintenance at the assigned stations. CARB performs calibrations of all instruments twice a 

year and performs audits twice a year. 

The monitoring stations operated by the district are set up to perform automated QC 

checks. Zero/span checks for O3 are programmed to occur weekly at 3 a.m., and one-point 

precision checks are performed manually about once a week. Leak checks of PM instruments 

are performed once per month. MCAQMD does not consistently complete and document 

monthly flow checks on PM instruments. 

MeCAQMD uses CARB SOPs, however, hard copies of the SOPs were not available and 

the staff did not know where electronic versions were kept. The site operator has the instrument 

manuals but not the SOPs. Several types of documents are used to track performance and 

maintenance at the four sites, including station logs, a monthly maintenance and service log for 

the PM sites, a weekly QA/QC checklist and separate maintenance log for the O3 site. Logs and 

checklists are archived into binders at the Ukiah MeCAQMD office. If anything unusual is 

noted, this information is passed along to CARB data validators with the email that conveys the 

data.
31

Corrective actions and repair/maintenance are dealt with on a case-by-case basis. 

Generally, minor equipment repairs are performed by MeCAQMD, while major repairs are 

performed by CARB.

Laboratory Operations

MeCAQMD relies on CARB for laboratory, calibration, and audit support. 

Data Management

Currently, neither MeCAQMD nor CARB are assessing whether the required data quality 

objectives and measurement quality objectives have been achieved. 

MeCAQMD station operators perform a preliminary assessment of the gaseous and 

continuous PM10 and PM2.5 raw data, though this process is generally not documented or 

performed with a predefined set of SOPs or other procedures outlined in a relevant QAPP. Raw 

text files are sent to CARB for submission to AQS, along with any relevant notes. After AQAS 

staff has uploaded the data to AQS, a confirmation email is sent to district staff, which has the 

AQS raw data report and raw data inventory report attached. MCAQMD does not have access to 

AQS and therefore does not subsequently check what is uploaded into AQS. 

QA/QC

MeCAQMD conducts some QA/QC activities, while relying on CARB to support others.

QA/QC activities conducted by MeCAQMD include one-point QC checks for its O3 monitor. 

MeCAQMD does not always perform monthly flow verifications for PM10 and PM2.5. CARB 

conducts the following QA/QC activities: semi-annual calibrations, semi-annual audits for O3,

PM10 and PM2.5, and O3, flow, and gaseous certifications and calibrations lab services. EPA 

contractors also conduct annual site visits and calibration checks on the PM monitors, as well as 

comparison monitoring with EPA field equipment.
32

OVERVIEW OF SAN JOAQUIN VALLEY AIR POLLUTION CONTROL 

DISTRICTAIR MONITORING PROGRAM

As part of the CARB TSA, EPA also reviewed the ambient air monitoring activities of 

SJVAPCD. SJVAPCD is currently part of the CARB PQAO but this audit included an agencyspecific assessment of network design, field operations, data handling, quality assurance and 

quality control procedures. EPA staff interviewed SJVAPCD management and staff and 

performed site evaluations at Fresno-Drummond, Clovis, Tranquility, and Hanford, and made 

site visits to Huron and Corcoran.

The SJVAPCD managers and staff were very accommodating, making themselves and 

their staff available for many interviews, procedural reviews and monitoring site visits. 

Management and staff interviewed were:

Michael Carrera – Central Region Compliance Manager

Nathan Trevino – Supervising Air Quality Instrument Technician

Ashley Ross – Air Quality Instrument Technician

Olan Bailey – Air Quality Instrument Technician

Errol Villegas – Air Quality Planning Manager

Steve Shaw – Supervising Air Quality Specialist, Air Quality Analysis Group

Peter Biscay – Air Quality Specialist, Air Quality Analysis Group 

Jennifer Ridgway – Air Quality Specialist, Air Quality Analysis Group

Overall, the monitoring and data analysis staff are very dedicated and knowledgeable, 

operating the monitoring network and producing data of known quality to the best of their 

ability. As described in the findings, the major deficiency in the monitoring program is the lack 

of a structured and formalized framework that is inherent to a functioning quality system 

required for ambient air monitoring. Some of the findings in this TSA pertain to CARB’s role as 

the PQAO and its relationship and oversight responsibilities to local districts. Increased 

communication and coordination between SJVAPCD and CARB is needed to effectively 

maintain the ambient air monitoring network in the San Joaquin Valley.

Network Management

At the time of this TSA, there were 31 monitoring sites in the San Joaquin Valley.

Twenty-one sites are operated by SJVAPD, 8 sites are operated by CARB and 2 sites are 

operated jointly by CARB and SJVAPCD (Table 2). In addition, there are two sites operated by 

the National Park Service and one site operated by the Tachi Yokut Tribe on the Santa Rosa 

Rancheria.

SJVAPCD submits its own Annual Monitoring Network Plan directly to EPA that 

addresses all the sites within its jurisdiction. In general, CARB and SJVAPCD have worked 

informally to address some network modifications; finding SJV4 addresses the deficiencies in 

this process. If a local district within the CARB PQAO, such as SJVAPCD, seeks to make 

changes to its network outside the Annual Monitoring Network Plan process, the suggested 
33

process is for the district to first work with CARB to resolve potential issues and then submit its 

request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for O3, 

NO2, CO, PM2.5, PM10, PAMS and NCore are being met in the San Joaquin Valley. However, 

there are outstanding network modifications that have not been approved for the following sites: 

Corcoran, Bakersfield-Golden State Highway, and Arvin-Bear Mountain Road (CARB site).

Field Operations

SJVAPCD operates a network of O3, NO2, CO, PM2.5, PM10, and PAMS monitoring 

instruments. The following manager(s) and staff are responsible for Field Operations:

Michael Carrera – Central Region Compliance Manager

Nathan Trevino – Supervising Air Quality Instrument Technician

SJVAPCD field technicians interviewed exhibited a thorough knowledge of equipment 

operations and an interest in producing high quality data that meet all the regulatory 

requirements. Most quality control checks and maintenance are performed in accordance with 

EPA regulations. Field technicians are responsible for day-to-day operations as well as 

instrument preventive maintenance and minor repairs at their assigned stations. If the repairs are 

major and cannot be completed by the staff at the headquarters office, the instruments are sent to 

the manufacturer. The senior air quality instrument technician performs calibrations of the 

monitors at the required frequency. The monitoring group schedules routine maintenance and 

calibrations together to reduce instrument downtime between modifications to an instrument and 

the required follow-up calibration. SJVAPCD relies primarily on CARB’s Standards Laboratory 

for O3, flow, and gaseous certifications and calibrations and instrument manufacturers for major 

instrument repair.

The monitoring stations operated by the district are set up to perform daily automated QC 

checks on gaseous instruments using certified standards, which exceeds EPA requirements. 

Flow verifications are performed quarterly for manual PM10 samplers, semiannually for manual 

PM2.5 samplers, and biweekly for continuous PM2.5 and PM10 samplers. 

All stations maintain log books to document site visits, preventive maintenance, 

resolution of operational problems, and corrective actions taken. Logbooks were generally 

detailed, but what is recorded could be more standardized. Operators archive station logbooks at 

the central monitoring office. Other station records include QC checklists and maintenance 

sheets that are also archived at the monitoring station and at the central office. All necessary 

calibration information is available to the field operators.

Corrective actions and repair/maintenance are dealt with on a case-by-case basis in a 

timely fashion; however, there is no formalized corrective action process to document decisions 

or solutions and help communicate them to all field personnel.

The SJVAPCD does not operate a laboratory; it relies on VCAPCD for PM weighing 

laboratory support and on a contract laboratory for PAMS analyses. 
34

Data Management

SJVAPCD manages all of the ambient monitoring data generated and uploaded to AQS 

by the district. Data quality objectives and measurement quality objectives have been defined 

for SJVAPCD’s program. Station operators perform data collection and sample handling 

according to specific SOPs for most pollutants and the first level data validation for their 

stations. Senior staff and supervisor observe activities on an ongoing basis to ensure full 

implementation of current and recently changed procedures.

The supervising air quality instrument technician verifies and validates data through level 

two and the supervising air quality specialist and staff in the Air Quality Analysis group perform 

level three validation. Based on a Memorandum of Understanding with the district, CARB and 

EPA, SJVAPCD submits gaseous and continuous PM data to AQS. Prior to submitting the data 

to AQS, Air Quality analysis staff complete a two-page checklist to document the review 

elements. Filter based PM10 and PM2.5 data are processed, validated, and submitted to AQS by 

VCAPCD that follows procedures outlined in its agency specific QAPPs and SOPs.

QA/QC

SJVAPCD conducts some QA/QC activities while relying on CARB to support others. 

QA/QC activities conducted by SJVAPCD include: zero, precision and span checks, routine 

maintenance and calibrations. CARB conducts the following QA/QC activities: annual 

performance audits and NPAP audits for gaseous instruments and one of the semiannual flow 

audits of PM instruments. EPA’s contractor performs the remaining semiannual flow audit and 

the PM2.5-PEP audits. Starting in January 2012, CARB began to conduct both of the semiannual 

flow audits. The SJVAPCD follows its own SOPs, but does not have SOPs for some activities, 

nor up-to-date QMP or QAPPs. The SJVAPCD does not have an independent QA manager 

responsible for overseeing the agency’s Quality System. However, the district does provide 

some independence in the activities that would typically fall to a QA manager by having 

operations and calibrations performed by different people. In addition, the third level data review 

and validation are separated from field operations and data collection by two levels of 

management. CARB’s audits, which include siting evaluations, and TSAs to be performed by 

CARB also provide independent oversight of the district’s operations.
35

FINDINGS

Program Area Finding 

Numbers

EPA Contact

General G1-G6 Audit Team, Meredith 

Kurpius (lead)

Network Management NM1-NM3 Meredith Kurpius

Field Operations FO1-FO17 Gwen Yoshimura

Data Management DM1-DM9 Meredith Kurpius

QA Management QA1-QA7 Mathew Plate

PM Laboratory PM1-PM4 Michael Flagg

Toxics Laboratory TL1-TL21 Steve Remaley

ICAPCD IMP1-IMP10 Michael Flagg

MeCAQMD MEN1-MEN12 Gwen Yoshimura

SJVAPCD SJV1-SJV12 Kate Hoag
36

Finding # G1

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding:

[Previous Finding M1] CARB needs to complete the process of putting a formal PQAO into 

place. 

Description:

CARB has taken steps to strengthen the CARB PQAO by:

Appointing a PQAO contact.

Improving the field audit and technical audit program of PQAO districts.

Beginning to provide QA training.

Reviewing PQAO districts’ quality control data prior to routine data certifications.

Beginning to review PQAO districts’ SOPs.

Starting a process to put in place agreements with PQAO districts.

Evaluating and controlling the standards used by the PQAO through the standards 

laboratory and during technical audits.

The CARB PQAO is able to produce data of known quality that can withstand legal and 

technical challenges to state and Federal regulatory decisions.

In order to complete the process of integrating CARB’s PQAO districts into a formal PQAO, the 

organization should be defined in greater detail. It should be noted that a PQAO can only be 

created and maintained if the organization conforms to the five criteria defined by EPA 

regulation (see 40 CFR 58, Appendix A, 3.1).

CARB has begun to define the organization of the PQAO by identifying contacts and performing 

outreach to the PQAO districts. In order to fully define the PQAO CARB must:

Formally identify which districts, monitoring sites, and pollutants are included.

Complete the process of having formal agreements in place between the districts and 

CARB.

Develop and implement an organized and comprehensive training program to support the 

CARB PQAO.

Complete the CARB Quality Management Plan that defines PQAO organization, roles, 

and activities.

In order to strengthen the PQAO so that it produces data of known and consistent quality, CARB 

should continue working to meet the five criteria. Below is a summary of the work to which 

CARB has committed to achieve this goal. 

(1) Although all field operators are not CARB staff, CARB can continue to take steps to ensure 

that all PQAO field operators have the benefit of access to background information and support 
37

by:

Implementing routine training programs that are available to all personnel in the PQAO.

Increasing the level of technical support that is available to PQAO districts.

Enhancing communication between CARB and the PQAO districts.

(2) The CARB PQAO has a “universal” QAPP and SOPs. In order to ensure that the procedures 

described are consistently followed throughout the PQAO, CARB should:

Continue to update these documents and inform and train PQAO staff on changes.

Continue to review and approve SOPs from PQAO Districts and make these SOPs 

available to the entire PQAO.

Continue to evaluate adherence of PQAO districts to the QAPP and SOPs.

(3) The CARB PQAO has a standards laboratory that is available to all districts, but some PQAO 

standards are certified by outside sources. To ensure comparable standards throughout the 

PQAO, CARB should:

Continue to inventory all the standards and their traceability used by the PQAO.

Continue to evaluate the performance of standards sent to the standards laboratory and 

issue corrective actions as necessary.

Determine the need to consolidate some of the standards/standard certifications used by 

the PQAO in order to promote consistency and save resources.

(4) The CARB PQAO has a common QA/QC evaluation group. However, most of the PQAO 

districts do not have QA support staff except those available from CARB. In order to meet the 

criteria for a common QA, CARB needs to:

Create a line of QA communication between PQAO staff and QA staff that is separate 

from the audit process.

Continue to work on corrective action processes that PQAO district staff can use to 

elevate QA issues to CARB’s QA program.

Ensure that consistent data validation procedures are in place.

(5) The CARB PQAO does not have support of common management, headquarters, or 

laboratory facilities, with the exception of some analytical laboratory analyses performed by the 

MLD laboratory for some districts. CARB should promote common management practices by:

Creating standards for oversight of monitoring stations and operations.

Providing training to monitoring managers.

References:

40 CFR Part 58, Appendix A, Section 3.1:

Primary Quality Assurance Organization. A primary quality assurance organization is defined as 

a monitoring organization or a coordinated aggregation of such organizations that is responsible 

for a set of stations that monitors the same pollutant and for which data quality assessments can 

logically be pooled. Each criteria pollutant sampler/monitor at a monitoring station in the 

SLAMS network must be associated with one, and only one, primary quality assurance 

organization.

40 CFR Part 58, Appendix A, Section 3.1.1:
38

Each primary quality assurance organization shall be defined such that measurement uncertainty 

among all stations in the organization can be expected to be reasonably homogeneous, as a result

of common factors. Common factors that should be considered by monitoring organizations in 

defining primary quality assurance organizations include:

(a) Operation by a common team of field operators according to a common set of procedures;

(b) Use of a common QAPP or standard operating procedures;

(c) Common calibration facilities and standards;

(d) Oversight by a common quality assurance organization; and

(e) Support by a common management, laboratory or headquarters.

QA Handbook Volume II, Section 4.2:

Appropriate training should be available to employees supporting the Ambient Air Quality 

Monitoring Program, commensurate with their duties. Such training may consist of classroom 

lectures, workshops, web-based courses, teleconferences, vendor provided, and on-the-job 

training.

Recommendation to Address Finding:

CARB should formalize the PQAO and develop regular communications with the local districts 

to ensure that the five PQAO criteria are being met.
39

Finding # G2

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding: 

The QMB does not have the structure and staff to manage QA oversight of the PQAO districts.

Description:

CARB should provide QA oversight of local district air monitoring programs. The designation of 

the QMB Chief as the primary QA contact for the PQAO districts would clearly indicate that the 

authority lies with CARB. Formal agreements between the districts and CARB are needed to 

support this authority, as noted in Finding G1. In order to meet these needs, the QMB will need 

to develop a corresponding organizational structure and staff expertise.

References:

40 CFR Part 58, Appendix A:

2.2 Independence of Quality Assurance. The monitoring organization must provide for a quality 

assurance management function- that aspect of the overall management system of the 

organization that determines and implements the quality policy defined in a monitoring 

organization's QMP. Quality management includes strategic planning, allocation of resources 

and other systematic planning activities (e.g., planning, implementation, assessing and reporting) 

pertaining to the quality system. The quality assurance management function must have 

sufficient technical expertise and management authority to conduct independent oversight and 

assure the implementation of the organization's quality system relative to the ambient air quality 

monitoring program and should be organizationally independent of environmental data 

generation activities.

Recommendation to Address Finding:

The QMB should develop the expertise, organization, and tools to oversee the CARB PQAO 

effectively. Corrective action to address this finding could include, but is not limited to, hiring a 

PQAO coordinator, assigning CARB staff responsibilities of overseeing activities within the 

CARB PQAO, conducting TSAs of the local districts within the CARB PQAO, and developing 

network planning and data validation tools for use by CARB and local districts within the CARB 

PQAO.
40

Finding # G3

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding:

[Previous Finding M6] While progress has been made on updating the CARB QA Manual with a 

QMP and QAPPs or equivalent documents, the process is behind schedule.

Description:

The CARB QA Manual was regularly updated until 2007. Based on EPA’s TSA finding in 

2007, CARB agreed to update or replace the QA Manual with a document that conformed to the 

requirements of the EPA QA system. In order for CARB’s system to be up-to-date, complete, 

and useful, current QA planning documents are needed. In addition, QAPPs/SOPs should be 

revised when standards or instruments change.

References:

In accordance with 40 CFR Parts 31 and 35, grant recipients are required to document their 

quality systems. Specific ambient air monitoring requirements are found in 40 CFR, Part 58, 

Appendix A, Section 2.1, “EPA Requirements for Quality Management Plans (QA/R-2)”, 

EPA/240/B-01/002, March 2001 and “EPA Requirements for QA Project Plans (QA/R-5)”, 

EPA/240/B-01/003, March 2001.

Recommendation to Address Finding:

CARB should complete updating QA planning documents as soon as possible. SOPs should be 

updated and submitted with updated QAPPs, following a clear timeline. The QMP and QAPPs 

should be reviewed and resubmitted to EPA for approval every 5 years.
41

Finding # G4

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding:

Local districts within the CARB PQAO do not always have updated quality system 

documentation for all activities.

Description:

Quality system documents include Quality Management Plans (QMPs), Quality Assurance 

Project Plans (QAPPs), and Standard Operating Procedures (SOPs). Local districts within the 

CARB PQAO can either adopt CARB’s quality system documents or prepare their own. Not all 

local districts within the CARB PQAO have their own approved quality system documents or 

use CARB’s (see Findings MEN1, IMP1, and SJV2)

References:

40 CFR 58 App. A 2.0, Quality System Requirements

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Further guidance on developing QAPPs can be found in the guidance documents “EPA 

Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, March 2001, and 

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002

Recommendation to Address Finding:

Each local district within the CARB PQAO should have its own quality system documents 

approved by CARB or formally adopt the CARB quality system documents. CARB should 

oversee adoption and approval of quality system documents.
42

Finding # G5

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding:

[Previous Findings QM1 and M3] QA Authority and interactions between QMB and the other 

branches should be expanded and formalized. The corrective action system should be developed 

to include actions taken, in addition to reports issued by the QA auditors and the calibration 

laboratory.

Description:

Based on feedback and observations made during the audit, CARB MLD is relying on the QMB 

to provide independent QA leadership. In order for the QMB to fulfill this role, the other MLD 

branches should acknowledge the QMB’s QA authority and staff people should be able to raise 

QA issues to the QMB. The QMB should be able to exercise QA authority and oversight in a 

judicious and cooperative manner.

The QMB should be involved in:

Planning air monitoring activities and programs.

Overseeing the implementation of monitoring.

Evaluating monitoring data and programs.

In addition to QA/QC support, the specific tasks that must be conducted by the QA 

independently are:

Implementation of the QMP.

Review and approval of QAPPs and other monitoring plans.

Review and approval of QA components of SOPs.

Approval of formal corrective actions.

QA system training.

Documentation of required training.

Performing periodic internal audits (performance, technical, and data).

Review of data quality summaries and/or control charts (including AMP255 reports).

Evaluation of data validation process/reports.

Evaluation of final data used to make regulatory decisions.

Several specific issues were noted that should be addressed and may be indicative of the broader 

issue of the QMB’s role in providing independent QA.

The QMB Chief was not fully exercising the full extent of his authority and oversight 

over the AQSB.

The AQSB was hesitant to characterize the QMB’s role in special projects as oversight.

The QMB does not have approval authority for SOPs produced by the other MLD 

branches.

Updates to the new QAPP sections requested by QMB from the other branches have not 
43

been completed.

New monitoring projects were initiated without QMB involvement in the planning 

process.

During field audits, the auditors perform instrumental tasks that are the responsibility of 

the station operators. 

The CARB QMB has expanded the corrective action (AQDA) process to include calibration 

laboratory and siting. However, the CARB PQAO has not established a corrective action 

process that is comprehensive and can be initiated by CARB or district staff. When a significant 

quality problem or area for improvement is identified, there should be a formal process to ensure 

that the problem is addressed throughout the PQAO. The process should be “blind” to the 

initiator; it should allow for bottom-up, non-punitive initiation of formal corrective actions.

Several issues identified by staff should have been elevated as formal corrective actions requiring 

systematic changes (see specific findings). 

References:

40 CFR Part 58, Appendix A

See Finding 2, References

EPA QA/R-2

3.11 QUALITY IMPROVEMENT

Purpose – To document how the organization will improve the organization’s quality system.

Specifications – Identify who (organizationally) is responsible for identifying, planning,

implementing, and evaluating the effectiveness of quality improvement activities and describe 

the process to ensure continuous quality improvement, including the roles and responsibilities of

management and staff, for ensuring that conditions adverse to quality are prevented and 

identified promptly including a determination of the nature and extent of the problem.

Recommendation to Address Finding:

The QMB role/authority, including its independence, should be formalized and detailed in the 

CARB QMP. Additionally, it is recommended that the QMB approach QA tasks in a manner 

that balances independence and cooperation. 
44

Finding # G6

Agency: CARB

Date of Audit: Summer 2011

Program Area: General

Finding:

Coordination between CARB and districts and EPA should be improved.

Description:

Several findings identified during this TSA relate to insufficient coordination and 

communication between CARB and the local districts within the CARB PQAO, including:

Previously unreported, but valid PM2.5 samples for ICAPCD were found at SDCAMD 

that impacted a regulatory decision.

Issues with CARB data validation were identified for ICAPCD and MeCAQMD.

District staff sometimes lacked knowledge of QMP, QAPPs, SOPs.

CARB and the local agencies must take ownership of the data quality and work together to 

develop processes to avoid the recurrence of problems. 

References:

Recommendation to Address Finding:

CARB should develop a process to routinely share information with districts (e.g., a PQAO 

listserve). CARB and the local districts should create a mechanism for resolving issues between 

the agencies in a well-documented and transparent manner and articulate clear expectations of 

the roles and responsibility of all the agencies in the PQAO through an MOU, as suggested by 

EPA Region 9’s PQAO strategy, to provide a framework for developing such processes.
45

Finding # NM1

Agency: CARB

Date of Audit: Summer 2011

Program Area: Network Management

Finding:

Not all agencies within the CARB PQAO have approved network plans since this became a 

requirement in 2006. The current approach to network plans does not provide for a determination 

of network adequacy on a statewide basis. 

Description:

There are 35 local air pollution control districts in the state of California (see Table 1) in addition 

to CARB. Three of these local air districts, BAAQMD, SCAQMD, and SDCAPCD are their own 

PQAO and submit annual monitoring network plans. The remaining 32 districts are within the 

CARB PQAO. Twenty-one air districts plus CARB collect ambient air monitoring data under 

the CARB PQAO. In 2012, nine of the districts within the CARB PQAO prepared and 

submitted their own annual monitoring network plan. CARB prepared and submitted an annual 

monitoring network plan for the remaining local districts in California and for its network. All 

districts in California except for MDAQMD/AVAPCD were covered in a network plan in 2011. 

In the past, not all local districts within the CARB PQAO that had assumed responsibility for 

submitting an annual monitoring network plan have fulfilled the obligation (e.g., NSAQMD, 

MDAQMD, and AVAPCD). As a result, regulatory monitors in these areas have been operated 

for some period without an approved annual monitoring network plan. However, data quality did 

not appear to be compromised for these periods.

Although the network plans for California have been approved by Region 9, the current system 

of multiple network plans produces information that cannot be easily combined. Since 

monitoring network requirements often span multiple districts, plans that contain inconsistent 

information do not provide for a determination of network adequacy on a statewide basis, which 

is required as part of the annual monitoring network plan process. 

References:

40 CFR 58.10(a) addresses network plan requirements.

Recommendation to Address Finding:

CARB should compile an overlay of information from the annual monitoring network plans that 

are submitted by local districts in the State of California. To accomplish this, it is suggested that:

All local districts that submit annual monitoring network plans for the State of California 

provide a copy to CARB no later than the time of the annual monitoring network plan 

submittal.

CARB compile information from the plans addressing key requirements that apply across 

multiple districts (e.g., minimum monitoring requirements and co-location requirements).

o The summary should be created between the deadline for Annual Network Plan 

submittal (July 1) and 30 days prior to annual monitoring network plan approval 

(Nov. 1).

o After receiving network plans, CARB should work with districts to reduce 

missing/deficient information.
46

o If the summary contains information that is different from that submitted in 

annual monitoring network plans, CARB should provide an opportunity for public 

comment.

EPA review the submitted plans and the CARB summary and document the basis for 

approval/disapproval decisions.

If a district with responsibility for submitting a plan does not fulfill its obligation, CARB 

provide the required information for that year.
47

Finding # NM2

Agency: CARB

Date of Audit: Summer 2011

Program Area: Network Management

Finding:

The network assessment does not meet all CFR requirements.

Description:

Several districts submit separate network assessments for the State of California. Since 

requirements for the ambient air monitoring network extend beyond the boundaries of local 

districts, the assessment must be done at a multi-jurisdictional level. At the time the report was 

drafted, CARB’s network assessment for small agencies found that the minimum monitoring 

requirements were met, the monitoring objectives defined in appendix D were met, all operating 

sites were critical for the implementation of State and federal air quality standards, and none 

were proposed to be discontinued. The CARB network assessment for small agencies did not 

address whether new sites were needed, whether existing sites were no longer needed and could 

be terminated, or whether new technologies were appropriate for incorporation into the ambient 

air monitoring network, as required by CFR.

References:

40 CFR 58.10(d) addresses network assessments and states:

“The State, or where applicable local, agency shall perform and submit to the EPA Regional 

Administrator an assessment of the air quality surveillance system every 5 years to determine, at 

a minimum, if the network meets the monitoring objectives defined in appendix D to this part, 

whether new sites are needed, whether existing sites are no longer needed and can be terminated, 

and whether new technologies are appropriate for incorporation into the ambient air monitoring 

network.”

Recommendation to Address Finding:

CARB should conduct its own state-wide assessment. Alternatively, CARB could develop a 

process to compile and synthesize information from network plans from local districts into a 

comprehensive network plan that addresses the CFR requirements.
48

Finding # NM3

Agency: CARB

Date of Audit: Summer 2011

Program Area: Network Management

Finding:

There are PM10 monitors listed in local conditions (LC; parameter code 85101), but not Standard 

Temperature and Pressure (STP; parameter code 81102 in AQS), as required for FRM/FEM 

instruments. 

Description:

All PM10 measurements collected with FRM/FEM instruments are required to be entered into 

AQS as STP (parameter code 81102). It is acceptable to report data under both LC and STP 

parameter codes. The following CARB monitors were identified as entered under only the LC 

code:

South Lake Tahoe (060170011), POC 2.

Mojave – Poole (060290011), POC 3.

Bakersfield – California (060290014), POC 5.

Paso Robles (060792004), POC 2.

San Luis Obispo (060794002), POC 3.

Santa Barbara (060830011), POC 1.

Santa Maria (060831008), POC 2.

The following non-CARB sites that are within the CARB PQAO were identified as being entered 

under only the LC code:

Brawley (060250007), POC 3, Imperial County APCD.

Niland (060254004), POC 3, Imperial County APCD.

Corcoran (060310004), POC 7, San Joaquin Valley APCD.

Madera (060392010), POC 3, San Joaquin Valley APCD.

Lakeport (060333001), POC 2, Lake County AQMD.

Anderson Springs (060333010), POC 1, Lake County AQMD.

Glenbrook (060333011), POC 1, Lake County AQMD.

Nipomo (060794002), POC 2, San Luis Obispo County APCD.

References:

40 CFR 50, Appendix J, Section 2.2 (Note that there are no CFR regulations that specify 

operation and data treatment of PM10 FEM instruments. In the absence of regulations specific to 

PM10 FEM instruments, they must adhere to requirements for FRM instruments, unless stated 

otherwise.)

Recommendation to Address Finding:

CARB should change or add the parameter code for the CARB sites listed above to STP (i.e.,

parameter code 81102). CARB should work with non-CARB districts within its PQAO to have 

the non-CARB sites listed above changed to STP.
49

Finding # FO1

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field

Finding: 

[Related Previous Findings GB3, SJV3, & NS2] Documentation at the CARB field sites is 

inadequate and not reviewed by management.

Description:

The level and consistency of documentation at the AQSB managed field stations was inadequate 

to reconstruct the monitoring that was conducted and to resolve definitively the data quality 

issues identified.

AQSB site operators use a variety of different documents to record information pertaining to site 

operations. These include station logbooks, station maintenance sheets, paper strip charts, and 

data report sheets. Field staff also do not consistently document when they notice something out 

of the ordinary about the site that could impact readings (construction, weather).

Several specific issues regarding documentation were noted:

There is no clear direction as to where information regarding instrument issues that could 

impact the validity of data is recorded and how such information is transmitted to the data 

validators.

It is not clear that information recorded by the site operators on monthly data reports is 

retained as an official record.

There is not recent evidence 

that there has been

management review of the 

documentation produced by 

the station operators.

Initials do not routinely 

accompany entries.

Use of pencil and erasing of 

records was observed.

Use of white-out on Chain of 

Custody forms was observed.

Entries in logbooks are 

incomplete, without sufficient information as to who was present at the site, serial 

numbers of problematic instruments, descriptions of actions taken, and how much data 

could be impacted.

There are no field maintenance logbooks for instruments. Logs are kept at the repair 

shop.

MLD should develop a consistent approach to site documentation and review. This may involve 
50

a short-term solution to improve documentation consistency and completeness and a long term 

solution to convert all site documentation to electronic records that can be more efficiently 

produced, reviewed, and incorporated into the data validation process.

References:

QA Handbook Volume II, Section 5 

2003 NELAC Standard (Quality Systems Section), which suggests documentation required for 

all aspects of ambient air monitoring operations, provides the following guidance:

5.5.5.5 The laboratory shall maintain records of each major item of equipment and its software 

significant to the environment tests performed. The records shall include at least the following:

a) The identity of the item of equipment and its software. 

b) The manufacturer’s name, type identification, and serial number or other unique 

identification.

c) Checks that equipment complies with the specification (see 5.5.5.2).

d) The current location.

e) The manufacturer’s instructions, if available, or reference to their location.

f) Dates, results and copies of reports and certificates of all calibrations, adjustments, 

acceptance criteria, and the due date of next calibration.

g) The maintenance plan, where appropriate, and maintenance carried out to date;

documentation on all routine and non-routine maintenance activities and reference 

material verifications.

h) Any damage, malfunction, modification or repair to the equipment.

i) If available, condition when received (e.g., new, used, reconditioned).

Recommendation to Address Finding:

Field documentation should be improved and a process developed and implemented to provide 

defensible electronic documentation. See Finding SJV7 Description section for recommendations 

as to the type of standard information that could be included in logbooks.
51

Finding # FO2

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

Management oversight of site operators needs strengthening.

Description:

As stated in Finding FO1, the site operators are not consistently following EPA guidance for 

regulatory ambient air quality data collection. Due in part to the geographic extent of the 

network, management oversight of the site operations is especially challenging. Nonetheless, 

procedures for management controls are needed to ensure that site operations produce robust data 

for regulatory decisions.

References:

Recommendation to Address Finding:

EPA recommends that managers develop checklists and conduct regular site visits. It may be 

helpful for the managers to participate in routine training for field operators so the knowledge 

base is similar. EPA also recommends using the logbooks and other records (e.g., maintenance 

logs and calibration sheets) as oversight aids. To this end, electronic records may be useful.
52

Finding # FO3

Agency: CARB 

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

CARB field operators have not been trained on new SOPs.

Description:

CARB field operators were generally proficient with the procedures they use to conduct their 

monitoring activities; however, field operators did not always understand why it was important 

to follow specific protocols and were found to be lax in following requirements in some 

instances. Further, it was noted that training and/or demonstration of proficiency was not 

adequately documented.

References:

Recommendation to Address Finding:

CARB should develop a formal system to ensure and document that all staff are familiar with the 

quality management system and are trained and proficient at the monitoring tasks that they are 

performing. Such a system could include trainings for field staff when SOPs are developed or 

revised; periodic refresher courses; monthly site operator meetings; regular manager visits to 

sites; standard logbooks that get checked and signed off on regularly. 
53

Finding # FO4

Agency: CARB 

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

Residence time calculations were not available at any CARB sites visited.

Description:

Residence time is defined as the amount of time it takes for a sample of air to travel from the

opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 states 

that for the reactive gases (O3, NO2, and SO2) residence times must be less than 20 seconds. 

Additionally, it is recommended that the residence time within the manifold and sample lines to 

the instruments is less than 10 seconds. The station technicians should calculate the residence 

time, document it in the station logbook, and periodically verify the data. 

There were no clear records of residence time of the sampling lines at each site. The site 

operators did not know how recently the residence time had been recalculated. At a minimum, 

the residence time should be calculated for each instrument after any change is made to the 

sampling train, such as the removal or addition of other instruments, and posted at each site. 

The station technicians should calculate the residence time, document it in the station logbook or 

other form available at the site, and periodically verify the measurement. 

References:

40 CFR Part 58, Appendix E Section 9

Recommendation to Address Finding:

CARB should calculate residence times for all gaseous monitors, modify sites with residence 

times in excess of 20 seconds, with a goal of 10 seconds, and evaluate any impact on compliance 

data due to excessive residence times. CARB should also have residence time calculated and 

posted or accessible on-site.
54

Finding # FO5

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

Delay in sending PM2.5 samples has resulted in loss of data.

Description:

PM2.5 samples are subject to a maximum of a 30-day hold from the time the samples are taken to 

when they are conditioned and weighed. If samples are not maintained at temperatures below the 

average ambient temperature during sampling, the hold time is limited to 10 days. Samples at 

one site, Yuba City, were held too long at the station post-collection. In some cases the delay has 

resulted in the need for immediate conditioning/weighing in laboratory (e.g., 12/4/10) and in 

other cases has resulted in invalidation (e.g., 9/22/10 and 6/17/10 through 6/20/10).

References:

40 CFR 50, Appendix L, Section 8.3.6

Recommendation to Address Finding:

PM2.5 samples should be sent to the laboratory within 15 days of sample collection. Field 

operation managers should ensure that the protocol is followed.
55

Finding # FO6

Agency: CARB 

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

PM make-up samples are not being taken in accordance with EPA guidance.

Description:

According to EPA’s April 1999 Guideline on Data Handling Conventions for the PM NAAQS, 

PM10 make up samples may count toward completeness when collected no more than 7 days 

after a scheduled sample or if they are collected between the missed sample day and the next 

scheduled sampling date. For example, a missed sample for a 1-in-6 day schedule could be made 

up before the next scheduled sample day or the day following the next scheduled sample day.

References:

http://epa.gov/ttncaaa1/t1/memoranda/pmfinal.pdf, pg 32-33

Recommendation to Address Finding:

CARB should ensure that the PM10 QAPP/SOP describes this situation, and that the field 

operators are aware of this provision.
56

Finding # FO7

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

PM10 QC checks are not consistently recorded. There is no document in which field operators are 

directed to record this information.

Description:

PM10 QC checks are being carried out by CARB field operators, but the checks are not 

consistently documented. The monthly check sheet does not have a monthly flow rate 

verification entry.

References:

Recommendation to Address Finding:

CARB should include PM10 QC checks on the monthly check sheet or in some other document

and ensure that field operators consistently record the information.
57

Finding # FO8

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

CARB field staff do not check data after sending information to the CARB offices. 

Description:

Station operators were unable to account for some data in AQS. EPA found an instance where 

the station operator made an incorrect note, which resulted in a data point being entered into 

AQS that should have been invalidated. 

Station operators make notes in the station log, on monthly check sheets, on strip charts, and on 

the monthly data report. They make notes on all flags contained in the monthly data report, edit 

the data, and then send everything to the data validator, who reviews the information and calls 

with any questions. 

The station operators do not review the data after the data validator makes changes, and do not 

look at the data entered into AQS. They often are not aware that there has been a problem, do

not know why certain flags have been entered or why data were invalidated. 

References:

Recommendation to Address Finding:

CARB should review the data validation SOP and determine if there should be an additional step 

where station operators review the data validator’s changes or the AQS data entry.
58

Finding # FO9

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding: 

The Yuba City site has several significant siting issues that need to be resolved.

Description:

The Yuba City site monitors for the 

following pollutants for comparison 

to the NAAQS: O3, NO2, PM10

(high vol. filter-based), PM2.5 (filterbased)

The site also has a PM2.5 BAM that 

is used for non-NAAQS purposes.

The monitors are on the roof of a 

small commercial building in a 

generally residential neighborhood. 

The gaseous probe is on the 

northeastern portion of the roof. 

The particulate monitors are on the 

southern portion of the roof and the 

BAM inlet is on the northwestern 

portion of the roof.

The gaseous probe is within 3 meters of trees and 4 meters from the roadway. This probe must 

be at least 10 meters from the roadway and the drip line of adjacent trees. This could be resolved 

by moving the probe to the south and trimming the adjacent trees.

The particulate monitors are within 6 meters of a tree(s) to the east and 10 meters of a tree to the 

southwest. The instruments must be at least 10 meters from adjacent trees (a distance of 20 

meters is preferable). This could be resolved by trimming trees.
59

The meteorological tower is too short for the surrounding trees and buildings. It is recommended 

that this tower be elevated to 10 meters above the roof height, if possible. If the tower cannot be 

adjusted, the data should be used with caution.

References:

40 CFR Part 58 Appendix E

Recommendation to Address Finding:

CARB should resolve siting issues by moving probes/monitors and/or trimming trees.
60

Finding # FO10

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding:

Records indicate that calibrations of gaseous pollutant instruments are not consistently done 

according to a schedule.

Description:

Staff and management indicated that calibrations are performed every six months. Calibrations 

were typically done within the six-month timeframe, but there were instances when instruments 

were not calibrated for 9-16 months.

References:

Quality Assurance Handbook for Air Pollution Measurement Systems, Volume 2

Section 12.3

Appendix D, O3 Validation Template

Recommendation to Address Finding:

EPA recommends calibrating instruments every six months if zero/span checks are done 

biweekly and annually if zero/span checks are done daily.
61

Finding # FO11

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding: 

[Previous Finding AQSB7] The number of NO2 titration points taken during calibration does not 

meet regulatory requirements.

Description:

EPA regulation requires that NO2 calibrations be verified with a minimum of 3 points; 5 points 

are recommended. The AQSB calibration group only takes 2 NO2 titration points. 

References:

40 CFR Part 50, Appendix F describes the requirements for NO2 calibration.

Section 1.5.9.4 states: “Maintaining the same FNO, FO, and FD as in section 1.5.9.1, adjust the 

O3 generator to obtain several other concentrations of NO2 over the NO2 range (at least five 

evenly spaced points across the remaining scale are suggested).”

Recommendation to Address Finding:

CARB should add at least one more titration point to NO2 calibrations.
62

Finding # FO12

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding: 

Multi-point calibrations of PM2.5 instruments are not done routinely.

Description:

The AQSB calibration group performs single point calibrations of PM2.5 instruments every six 

months. There is no provision for these instruments to be checked with a multi-point calibration 

on a regular basis, as required by 40 CFR Part 50, Appendix L. It is recommended that multipoint checks be performed annually for sampler flow. Multi-point checks of the PM2.5 sampler 

temperature and pressure sensors should also be performed if physically possible.

References:

40 CFR Part 50, Appendix L

9.1.1 Multipoint calibration and single-point verification of the sampler's flow rate measurement 

device must be performed periodically to establish and maintain traceability of subsequent flow 

measurements to a flow rate standard.

Recommendation to Address Finding:

CARB should implement routine multipoint calibrations of PM2.5 instruments.
63

Finding # FO13

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations

Finding: 

[Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air being 

used in the program.

Description:

Zero air scrubbers are used in place of certified zero air for instrument calibrations. This is a 

common practice and acceptable. Because zero air is used to generate the zero point and the 

calibration mixes, it must be treated as a standard. As such, zero air scrubber maintenance and 

verification must be documented.

References:

QA Handbook Volume II

40 CFR Part 50, Appendix F

1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the 

NO/NOX/NO2analyzer or which might react with either NO, O3, or NO2in the gas phase 

titration.

Recommendation to Address Finding:

CARB should document the quality of zero air when maintenance is performed on the zero air 

scrubbers and on a periodic basis.
64

Finding # FO14

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field

Finding: 

Span and precision gases used in the field are not being calibrated routinely.

Description:

In order to reduce the number of gaseous standards that are recertified, AQSB has not had the 

field standards used for span and precision checks of CO, NO2, and SO2 certified routinely. EPA 

regulations require that standards used to perform the required QC checks every two weeks must 

be certified. AQSB continued to use certified gases for routine instrument calibrations. 

References:

QA Handbook Volume II

40 CFR Part 50, Appendix F

1.3.1 NO concentration standard: Gas cylinder standard containing 50 to 100 ppm NO in 

N2 with less than 1 ppm NO2. This standard must be traceable to a National Bureau of 

Standards (NBS) NO in N2Standard Reference Material (SRM 1683 or SRM 1684), an 

NBS NO2Standard Reference Material (SRM 1629), or an NBS/EPA-approved 

commercially available Certified Reference Material (CRM).

40 CFR Part 58, Appendix A

3.2.1 One-Point Quality Control Check for SO2, NO2, O3, and CO. . . . The standards 

from which check concentrations are obtained must meet the specifications of section 2.6 

of this appendix.

2.6.1 Gaseous pollutant concentration standards (permeation devices or cylinders of 

compressed gas) used to obtain test concentrations for CO, SO2, NO, and NO2 must be 

traceable to either a National Institute of Standards and Technology (NIST) Traceable 

Reference Material (NTRM) or a NIST-certified Gas Manufacturer's Internal Standard 

(GMIS), certified in accordance with one of the procedures given in reference 4 of this 

appendix. Vendors advertising certification with the procedures provided in reference 4 

of this appendix and distributing gasses as “EPA Protocol Gas” must participate in the 

EPA Protocol Gas Verification Program or not use “EPA” in any form of advertising.

Recommendation to Address Finding:

CARB should certify all field gases used to perform QC checks.
65

Finding # FO15

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations – Instrument Testing, Certification, and Repair

Finding:

Instruments removed from the field are not always efficiently tracked and returned to the repair 

laboratory facility for diagnosis, repair, and reuse. Loss of data has occurred due to unavailability 

of spare instruments.

Description:

The instrument tracking/information system has at least three different components: 1) an 

electronic database intended to keep basic tracking information for all of the agency’s supplies 

and equipment; 2) a hard copy Parts and Supplies binder kept in the MLD Stockroom that at the 

time of the audit displayed a last revision date of August 2009; and 3) an instrument filing 

cabinet kept in the MLD instrument laboratory, with the intention that each instrument have its 

own individual folder with detailed information about acceptance tests, repairs, and other 

relevant history. It is likely that all of these sources together contain most of the useful and 

necessary information needed to accompany an instrument. It may become difficult to find and 

correlate information from the three different systems. A better approach would be to combine 

all three into one centralized system specific to monitoring equipment.

The operations support manager stated that replaced instruments are sometimes left at sites and 

may go unnoticed until there is a shortfall in the laboratory. In such cases, the approach used to 

find these missing instruments can be rather tedious, involving calling multiple sites before

finding the orphaned instrument. The instrument tracking system maintained by the agency is not 

efficient and has the potential to impact data completeness. In one instance, data completeness 

was impacted at the Sutter Buttes site during the summer of 2011 when a malfunctioning O3

instrument was not promptly replaced due to the lack of a spare. CARB should develop a system 

that tracks instruments so that they are diagnosed and repaired promptly to be available for reuse.

References:

Quality Assurance Handbook for Air Pollution Measurement Systems Volume II: Ambient Air 

Quality Monitoring Program (December 2008), Sections 11 & 13.

Recommendation to Address Finding:

CARB should establish and implement a robust and centralized monitoring equipment tracking

system that allows for prompt and accurate tracking containing all relevant information (i.e.

repairs, calibrations, etc.) of instruments. Given long distances between locations in the state and 

associated shipping costs and time, CARB may choose to explore the establishment of a second 

instrument laboratory facility to serve the Southern California region.
66

Finding # FO16

Agency: CARB

Date of Audit: 2011

Program Area: Field Operations

Positive Finding: 

CARB is working to improve communication with field staff.

Description:

CARB’s monitoring field operations manager is instituting quarterly meetings with all field staff 

in order to improve communications. EPA supports this as a way to improve consistency and 

coordination between the field staff across California.

References:

Recommendation to Address Finding:

N/A
67

Finding # FO17

Agency: CARB

Date of Audit: Summer 2011

Program Area: Field Operations – Instrument Testing, Certification, and Repair

Positive Finding:

CARB maintains a well-equipped stockroom of spare parts, maintains a large equipment 

purchase order history, and develops thorough equipment testing procedures that are regularly 

updated.

Description:

CARB maintains a stockroom within the MLD facilities that is well equipped with all types of 

replacement parts and spares. This usually allows for timely preventive care and operation of the 

monitoring network.

During the audit, CARB was able to provide most of the documentation for large equipment 

purchases including biding specifications based on EPA regulations, purchase receipts and 

condition reports, contract manufacturer service agreements, and procedures/conditions for the 

release of funds to the manufacturer.

CARB develops extensive procedures to test all newly acquired instruments. Several of these 

Acceptance Test Procedures (ATP) were examined during the audit. CARB develops these forms 

specific to each instrument model. The ATP forms contain detailed procedures and specifications 

that shop technicians should check when instruments are first received. The ATP forms allow for 

review and approval by the Operations Support Manager, as well as the Air Quality Surveillance 

Branch Chief.

References:

Quality Assurance Handbook for Air Pollution Measurement Systems Volume II: Ambient Air 

Quality Monitoring Program (December 2008), Sections 11.

Recommendation to Address Finding:

N/A
68

Finding # DM1

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

The data validation and review/verification procedures performed by AQSB, NLB, and AQAS 

are not formally published in a control-copied SOP.

Description:

SOPs detail the work procedures that are to be conducted or followed within an organization. 

SOPs document the way activities are to be performed to ensure consistent conformance to 

technical and quality system requirements and to support data quality. SOPs are intended to be 

specific to the organization or facility whose activities are described and assist that organization 

to maintain their quality control and quality assurance processes and ensure compliance with 

governmental regulations. Well-written SOPs can also serve as training materials and as 

references for staff, particularly if they are updated regularly (the recommendation is every three 

years). SOPs should be distributed in a manner that ensures that only the most recent versions are 

used and that historical SOP revisions are retained (these are sometimes called “controlledcopies”). SOPs should also be developed to enable individuals to transition into new positions. 

Deviations and changes from SOPs should be dated, documented, and kept in a bound or 

electronic document routinely accessed by and accessible to all staff.

References:

40CFR Part 58 Appendix A Section 3.1.1 states that “Each primary quality assurance 

organization shall be defined such that measurement uncertainty among all stations in the 

organization can be expected to be reasonably homogeneous, as a result of common factors. 

Common factors...include use of a common QAPP or standard operating procedures”.

Guidance for Preparing Standard Operating Procedures", EPA/240/B-01/004, March 2001

Recommendation to Address Finding:

CARB should finalize control-copied SOPs for the data validation and review/verification 

procedures in the AQSB.
69

Finding # DM2

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

[Previous Finding M7] Data submitted by local districts within the CARB PQAO are not 

validated using consistent procedures. (See Findings SJV9, IMP10, and MEN11)

Description:

In order to maintain a consistent data set, a PQAO should have a standard for routine data 

validation. However, the CARB QA Manual does not require a specific validation scheme for 

each of the criteria pollutants. This results in data validation that is inconsistent and has the 

appearance of being arbitrary, which is of special concern when data are used for NAAQS 

determination.

It is unclear to agencies within the CARB PQAO what the roles and responsibilities are for data 

validation and submittal. For example, CARB/AQAS uploads continuous data for two of the 

local districts we audited as part of this TSA. These districts expected that CARB would validate 

their data as part of this process. In fact, CARB /AQAS does not validate data for any agency. 

This misunderstanding has resulted in unvalidated and sometimes erroneous data being entered 

into AQS.

References:

Recommendation to Address Finding:

CARB and local districts should establish SOPs for data validation. They should establish formal 

documentation that outlines roles and responsibilities for data review and submittal. All CARB 

PQAO agencies should receive data validation training. 
70

Finding # DM3

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

[Previous Finding DM5] AQAS does not ensure that local district data are validated prior to 

upload to AQS.

Description:

CARB/AQAS uploads continuous data for ten local districts. CARB has an SOP for its staff who 

upload district data into AQS, but there is no formal documentation that guides roles and 

responsibilities for ensuring that appropriate data validation and submittal procedures are 

followed by the local districts. Several local districts are not validating data prior to submittal to 

CARB/AQAS for upload. Those local districts that do validate their data are not following any 

consistent approach.

References:

CARB/PTSD SOP for Ambient Air Quality Data Management, 2009.

Recommendation to Address Finding:

CARB should ensure that all local districts having the responsibility for submitting data directly 

to AQS following consistent procedures for reviewing and validating data before they are 

submitted to AQS. Formal documentation should be developed that define roles and 

responsibilities for data review and submittal between CARB and each local district within the 

CARB PQAO.
71

Finding # DM4

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

A few instances of erroneous continuous data were identified in AQS for CARB sites.

Description:

CARB’s AQSB validates continuous data for CARB sites, which involves reviewing >50,000 

data points per month. Data review performed during the audit identified missing data that 

should not have been invalidated and incorrect data that were not identified and corrected. The 

erroneous data were not identified by any level of review.

References:

Recommendation to Address Finding:

CARB should organize data validation training and finalize SOPs to establish appropriate 

procedures for data validation (see previous findings on data validation). Additional tools and/or 

resources should be assigned to data validation. For example, developing data visualization tools 

to assist in reviewing large sets of data may make the review of CARB continuous data more 

efficient and effective. Data audits by an independent section of CARB (e.g., Quality 

Management Branch) would help identify systematic deficiencies with data validation as well as 

specific data issues. CARB should develop data tools (e.g., flags, figures, tables) to conduct 

effective and efficient data audits.
72

Finding # DM5

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

Erroneous continuous data were identified in AQS for non-CARB sites within the CARB PQAO.

Description:

Each district within the CARB PQAO is expected to validate its own data; however, this is not 

done consistently (see Findings IMP10, MEN11, and SJV10). EPA identified incorrect data 

being collected by local districts and submitted to AQS.

References:

Recommendation to Address Finding:

CARB should organize data validation training and finalize SOPs to establish appropriate 

procedures for data validation (see Finding DM2). Each local district should either adopt CARB 

SOPs or develop their own. CARB and each local district should formally agree on consistent 

data validation procedures. Audits by an independent section of CARB (e.g., Quality 

Management Branch) would help identify systematic deficiencies with data validation as well as 

specific data issues. CARB should develop tools (e.g., flags, figures, tables) to conduct effective 

and efficient data audits.
73

Finding # DM6

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

[Previous Finding DM6] There are numerous deficiencies in the data certification process for the 

CARB PQAO, including:

Not all NAAQS-compliant data within the CARB PQAO are routinely certified. 

Data certified by CARB for local districts are not typically reviewed or validated. 

Data are routinely certified by agencies, but responsibility has not been formally 

delegated to any local agencies within the State of California.

Description:

Numerous agencies collect, analyze, and submit regulatory ambient air monitoring data. Often 

the same agency does not perform all of these activities and so it is not clear which agency 

should certify data. Ultimate authority for certifying data rests with the State, but can be 

delegated to local agencies. Historically, the responsibility for certifying data has not had formal 

delegation. The audit revealed cases where regulatory data submitted to AQS had not been 

certified by any agency (see Table 3). The lack of a formal structure for data certification within 

the State has resulted in incomplete and inappropriate data certification with the potential to 

jeopardize regulatory decisions. Additionally, CARB submits data for ten districts within the 

CARB PQAO and certifies these data without reviewing or verifying that the district validated 

the data. As a result, some unvalidated, erroneous data have been certified by CARB and 

submitted to AQS.

References:

40 CFR 58.15

EPA guidance on data certification states:

“2. What types of monitoring organizations must certify their data? 

State and local government monitoring organizations must certify their data. A state official 

should certify all data submitted for affected monitors in that state, except where 

responsibility for compliance with 40 CFR Part 58 requirements has been delegated to a local 

monitoring agency. Note that even if multiple monitoring organizations are considered to be 

with a single Primary Quality Assurance Organization, the certification must come from the state 

level, or from each local agency which has delegated responsibilities for compliance with 40 

CFR Part 58.”

Recommendation to Address Finding:

CARB should establish a formal structure for data certification. This includes identification of all 

data to be certified and the parties responsible for certification and formal delegation to those 

parties. In addition, CARB and local districts should establish formal roles and responsibilities so 

that no unvalidated data are certified and entered into AQS.
74

Finding # DM7

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

Data, including those for design value sites, have been changed after they are certified and 

subsequently not recertified.

Description:

The AQDB occasionally requests changes to data, based on continued higher level analyses, after 

certification. The data are not recertified.

40 CFR Part 58.15 requires data be certified by May 1 of each year. Since the data are considered 

certified, they are official, and not subject to change after submittal of the certification letter. 

Changing data after certification is a significant concern, as the expectation is that the data will 

not change and may be used for attainment and decision making purposes. Data verification 

should take place before upload to AQS, not after, when they may impact numerous decisions 

already made by several organizations. Any changes to data that occur subsequent to data 

certification must be recertified. Uncertified data cannot be used for regulatory decisions.

References:

40 CFR Part 58.15

Recommendation to Address Finding:

All data changes and certification should take place consistent with deadlines established in Part 

58.15. If data need to be changed after they are certified, they should be recertified.
75

Finding # DM8

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

Some local districts within the CARB PQAO are listed as their own PQAO in AQS.

Description:

The following agencies, which are within the CARB PQAO, are listed as their own PQAOs in 

AQS:

GBUAPCD

MeCAPCD

SJVAPCD

SBCAPCD

SiCAPCD

TCAPCD

In some cases some parameters/sites for the local districts are under the CARB PQAO and other 

parameters/sites are under their own PQAO.

References:

40 CFR 58, Appendix A, Section 3.1

Recommendation to Address Finding:

CARB should identify the cases where data in AQS from agencies within the CARB PQAO are 

listed as a different PQAO and work with the districts to have data listed under the CARB 

PQAO. EPA can change the PQAO designation with approval from the local district and CARB.
76

Finding # DM9

Agency: CARB

Date of Audit: Summer 2011

Program Area: Data Management

Finding:

There were several instances of CARB altering data collected by local districts without 

communicating with the district.

Description:

CARB/AQAS enters continuous data for ten local districts. Local districts are expected to 

validate their data and submit them for direct upload to AQS. AQAS runs a routine AQS report 

that detects outliers. If any outliers are identified, AQAS staff must request that the district 

review the outliers, and revise the data outliers if necessary. It is AQAS policy, as specified in 

the Data SOP, not to revise local district data without the district’s consent. 

AQAS uploads continuous data for two of the districts that EPA visited during this TSA. Within 

the past year, for both districts there were instances where data had been altered without AQAS 

communicating with the local district.

References:

Recommendation to Address Finding:

CARB and local districts should establish formal documentation to establish roles and 

responsibilities for data review and submittal. CARB/AQAS should follow its policy to have 

each local agency revise its own data.
77

Finding # QA1

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

The QA Audit group has made an effort to improve its documentation process; however, several 

inconsistencies were noted.

Description:

During the certification process for CARB’s National Performance Audit Program, it was noted 

that the QMB performance audit group should make several improvements to its audit 

documentation process. These recommendations have been partially implemented through 

improvements to field documentation and logbooks.

Several discrepancies were noted:

The audit trailer logbook entries are incomplete and written in pencil.

There is no indication that the trailer logbook was recently reviewed by management.

The equipment maintenance records were not current.

Field sheets are filled out in pencil and transferred to electronic documents. As these 

sheets may be maintained as official records for the data validator, they should be 

completed in indelible ink.

In order to ensure the data produced by the ambient air monitoring network can withstand legal 

challenge, documentation must be complete, definitive and sufficient to be used as evidence for 

CARB/EPA designation decisions.

References:

QA Handbook, Volume II, Chapter 5

EPA’s APTI Course 444, Air Pollution Enforcement Student Manual, Chapter 8 

Evidence and documentation are not necessarily the same thing. Evidence is used to 

establish the truth for an issue being contested in court or a formal hearing. Good 

documentation may become evidence or support evidence. Bad documentation will only 

raise more questions and often causes the truth to be lost.

Recommendation to Address Finding:

The performance audit group should continue improving its documentation procedures.
78

Finding # QA2

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

The audit trailer evaluated was using one expired gas cylinder along with others that had not 

been certified annually as required for the EPA National Performance Audit Program (NPAP). 

Description:

Of the gas cylinders being used in the audit van, only the high concentration multi-blend gas had 

been certified within the last year. The low concentration CO cylinder had not been certified in 

over three years and was presumably past its certification period. 

The NPAP program states that gases should be certified annually. Because the CARB audit 

program performs NPAP audits, this criterion must be met as part of the CARB NPAP program.

References:

NPAP Field SOP 4/7/11

3.1.7.4 Annual Gas Cylinder Certification 

NPAP mobile audit laboratory compressed gas standards should be certified annually using the 

EPA Calibration Gas Traceability Protocol. This could be done through CARB’s Standards 

Laboratory, any of the gas vendors, EPA Region 2, 7 or OAQPS support contractor.

Recommendation to Address Finding:

CARB should certify gases used for audits at least annually.
79

Finding # QA3

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

The QA Section is not tracking monitors to ensure that 25% of monitors are being audited per 

calendar quarter.

Description:

EPA regulation requires that at least 25% of SLAMS monitors are audited each calendar quarter 

and that every monitor is evaluated at least once per year. The QAS has been meeting this 

schedule by distributing site audits throughout the year. It is recommended that the evaluations 

be tracked by dividing the number of monitors audited each quarter by the total monitors in the 

network. While there is no requirement to schedule evaluations tracking each of the four 

gaseous criteria pollutants, it is recommended that this be a secondary goal of the program.

References:

40 CFR Part 58, Appendix A

“3.2.2 Annual performance evaluation for SO2, NO2, O3, or CO. Each calendar quarter (during

which analyzers are operated), evaluate at least 25 percent of the SLAMS analyzers that monitor 

for SO2, NO2, O3, or CO such that each analyzer is evaluated at least once per year. If there are 

fewer than four analyzers for a pollutant within a primary quality assurance organization, it is 

suggested to randomly evaluate one or more analyzers so that at least one analyzer for that 

pollutant is evaluated each calendar quarter. The evaluation should be conducted by a trained, 

experienced technician other than the routine site operator.”

Recommendation to Address Finding:

Evaluate the current systems with regard to their compliance with the requirement to audit 25% 

of monitors per quarter.
80

Finding # QA4

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

The connection to the inlet on the audit trailer could pull in outdoor air. 

Description:

The probe connection being used by QAS might be creating a 

Venturi effect, bringing in outdoor air. When the diameter or 

size of a tube or pipe is increased there is a resulting pressure 

drop that can overcome the inherent pressure differential and 

cause a Venturi effect that may overcome excess system 

pressure.

By reconfiguring the design of the inlet attachment, the 

possibility of bringing in outdoor air can be significantly 

reduced. Note that excess flow from the vent should always 

be verified.

References:

Recommendation to Address Finding:

CARB should reconfigure the inlet hardware to reduce the possibility of Venturi pumping and 

verify excess flow before and after each audit, or verify that ambient air is not being pulled in.
81

Finding # QA5

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

Auditors do not review all applicable siting information in AQS prior to an audit.

Description:

It was found that the GPS coordinates for the site where the audit program was reviewed were 

incorrect in AQS. The QAS was unaware of this discrepancy as staff had not evaluated the 

accuracy of the AQS siting information. Because AQS is the repository of official information 

on each monitoring site and the information is used by EPA to make regulatory decisions and in 

research studies, it should be periodically verified.

References:

40 CFR Part 58

§ 58.16 Data submittal and archiving requirements.

(a) The State, or where appropriate, local agency, shall report to the Administrator, via

AQS all ambient air quality data and associated quality assurance data for SO2, CO, O3, NO2,

NO, NOY, NOX, Pb-TSP mass concentration, Pb-PM10 mass concentration, PM10 mass 

concentration, and PM2.5 mass concentration; for filter-based PM2.5 FRM/FEM the field blank 

mass, sampler-generated average daily temperature, and sampler-generated average daily 

pressure; chemically speciated PM2.5 mass concentration data; PM10–2.5 mass concentration; 

chemically speciated PM10–2.5 mass concentration data; meteorological data from NCore and 

PAMS sites; average daily temperature and average daily pressure for Pb sites if not already

reported from sampler generated records; and metadata records and information specified by 

the AQS Data Coding Manual (http://www.epa.gov/ttn/airs/airsaqs/manuals/manuals.htm ).

Recommendation to Address Finding:

CARB auditors should verify that site information in AQS is correct.
82

Finding # QA6

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

[Previous Findings M4 & OPA2] Quality assurance for special projects is not developed in a 

process consistent with EPA quality system requirements. 

Description:

When EPA grant funds are used by CARB to collect environmental data, or when data are used 

to support an EPA regulatory decision, data collection and use must be covered by a quality 

system that meets EPA requirements.

MLD does not have oversight authority for monitoring projects that are conducted entirely or 

initiated by other CARB Divisions or California Air Districts. The quality assurance planning 

and implementation for these projects is generally not transparent to MLD or EPA. 

AQSB does implement quality assurance planning for special projects where MLD plays a 

significant role. These projects may or may not include planning and implementation review by 

the QMB. 

References:

40 CFR Parts 31 and 35

Grant recipients are required to document their quality systems.

CIO 2105.0 (formerly 5360.1 A2) 

Approval Date: May 5, 2000

5. SCOPE AND FIELD OF APPLICATION.

a. Scope. This Order defines the minimum requirements for quality systems supporting EPA 

environmental programs that encompass: 

(1) the collection, evaluation, and use of environmental data by or for EPA, and 

(2) the design, construction, and operation of environmental technology by EPA. 

b. Applicability to Environmental Programs. This Order applies to (but is not limited to) the 

following environmental programs: 

(1) the characterization of environmental or ecological systems and the health of human 

populations; 
83

(2) the direct measurement of environmental conditions or releases, including sample collection, 

analysis, evaluation, and reporting of environmental data; 

(3) the use of environmental data collected for other purposes or from other sources (also termed 

“secondary data”), including literature, industry surveys, compilations from computerized data 

bases and information systems, results from computerized or mathematical models of 

environmental processes and conditions; and 

(4) the collection and use of environmental data pertaining to the occupational health and safety 

of personnel in EPA facilities (e.g., indoor air quality measurements) and in the field (e.g.,

chemical dosimetry, radiation dosimetry). 

Recommendation to Address Finding:

CARB should develop a process in the QMP to ensure that special projects collect monitoring 

data suitable for their intended use and include independent quality assurance planning, 

implementation and assessment support. QMB should work with other CARB divisions to 

promote good quality assurance practices for all data collection activities.
84

Finding # QA7

Agency: CARB

Date of Audit: Summer 2011

Program Area: QA Management

Finding: 

Mass flow elements (MFEs) are used to establish calibration points outside of their calibrated 

range. 

Description:

The factory calibration range for the MFE for the BGI Tetracal devices goes down to 1.8 

standard liters per minute (slm). However, the lowest calibration point used in this calibration is 

0.2 slm. While this is significantly below the calibrated range, the MFE’s linear range should 

extend well below this flow rate. The MFE should be calibrated below 0.2 slm so that stability 

of the standard is objectively measured across its linear range.

References:

NIST Handbook 150-2G, National Voluntary Laboratory Accreditation Program, Calibration 

Laboratories, Technical Guide for Mechanical Measurements, Section 2.6

2.6.5.3 A laboratory that certifies artifacts to tolerances should demonstrate a 

measurement uncertainty which does not exceed 50% of the tolerance. Exceptions to this 

ratio will be accepted for measurement systems which are documented to be state-of-theart.

NELAC Standard 2003 (Quality Systems Section):

5.5.5.2.2.1 Initial Instrument Calibration

The following items are essential elements of initial instrument calibration:

f) The lowest calibration standard shall be the lowest concentration for which quantitative 

data are to be reported (see Appendix C). Any data reported below the lower limit of 

quantitation should be considered to have an increased quantitative uncertainty and shall 

be reported using defined qualifiers or flags or explained in the case narrative.

g) The highest calibration standard shall be the highest concentration for which quantitative 

data are to be reported (see Appendix C). Any data reported above this highest standard 

should be considered to have an increased quantitative uncertainty and shall be reported 

using defined qualifiers or flags or explained in the case narrative.

Recommendation to Address Finding:

The flow calibration laboratory should adjust the MFEs calibration range to be below the lowest 

flow ranges expected.
85

Finding # PM1

Agency: CARB

Date of Audit: Summer 2011

Program Area: PM Laboratory

Finding: 

Communication of post-weigh information and transmission of documentation to local districts 

should be improved.

Description:

The PM Laboratory supports filter weighing operations for a number of districts throughout 

California. Some agencies have indicated that post-weigh PM data have not always been 

transmitted in a timely fashion. Communication of PM data to local districts should be 

considered time critical, especially when there are exceedances of the standard.

References:

Recommendation to Address Finding:

CARB should formalize communication procedures in order to maintain timely transmission of 

post-weigh PM data that is consistent with the expectations and needs of the local districts.
86

Finding # PM2

Agency: CARB

Date of Audit: Summer 2011

Program Area: PM Laboratory

Finding: 

The PM Laboratory does not have a formal corrective action process for addressing issues with 

PM filter collection.

Description:

Currently, the existing corrective action process for CARB is limited to the QMB performance 

audit program and Standards Laboratory calibration services. A similar process should be applied

to the PM Laboratory. The PM Laboratory supports filter weighing operations for a number of 

districts throughout California and often receives filters that have been damaged or deemed 

invalid due to other operational issues (i.e. filters received after required weighing period). 

Currently, these issues are communicated informally via email or phone call conversations. Due 

to the recurring nature of these issues, which result in data loss, the PM Laboratory should 

develop a mechanism to minimize these losses through a corrective action process.

References:

EPA quality management standards (EPA QA/R-2, Quality Improvement Section) require that 

management and staff “ensure that conditions adverse to quality are” prevented, identified 

promptly, fully defined, corrected, prevented from recurring, and documented as corrective 

actions which are tracked to closure.”

Recommendation to Address Finding:

CARB should establish a formal corrective action process for the PM Laboratory.
87

Finding # PM3

Agency: CARB

Date of Audit: Summer 2011

Program Area: PM Laboratory

Finding:

Documentation of activities in the PM10 and PM2.5 laboratories should be improved.

Description:

Although the majority of activities in the PM Laboratory are adequately tracked and 

documented, there are some areas where improvements should be made. Specific examples 

include:

The PM10 Laboratory does not maintain a general laboratory logbook.

Expiration and replacement of electrostatic strips are not documented.

Post-weigh conditioning times are noted on post-it notes and not formally documented.

Honeywell charts are primary records that are accompanied by a digital Dickson logger, 

but RH/Temperature are not transferred or tracked in LIMS.

Post-it notes are placed on archived Honeywell charts when RH/Temperature goes out of 

specification and is not formally documented.

Temperature of the refrigerator used for cold storage of filters is not documented.

Removal of filters from cold storage is not documented.

References:

Recommendation to Address Finding:

CARB should implement or improve documentation of identified activities.
88

Finding # PM4

Agency: CARB

Date of Audit: Summer 2011

Program Area: PM Laboratory

Finding: 

PM10 trip blanks are not being used to assess potential bias from filter transport and handling.

Description:

Trip blanks controls are useful in assessing potential contamination of filters from transport and 

laboratory handling.

References:

Table 10.1 in Section 10.2: Internal vs. External Quality Control of the QA Handbook for Air 

Pollution Measurement Systems, Volume II, EPA-454/b-08-002, December 2008 identifies trip 

blanks as a method for assessing bias due to contamination or operator error.

Recommendation to Address Finding:

CARB should collect PM10 trip blanks periodically.
89

Finding # TL1

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

The canister cleaning SOP does not reflect the current cleaning procedure.

Description:

The number of cleaning cycles for a newly acquired cleaning system has been reduced from nine, 

as stated in the SOP, to five. Staff stated the SOP is being re-written. 

SOPs document an agency's official policies and procedures to which staff should adhere to 

obtain consistent and reliable data. They are required as part of an agency's approved QAPP. 

SOPs are used in training staff in agency accepted analytical methodology and help demonstrate 

data defensibility.

References:

40 CFR Part 58, Appendix A

Recommendation to Address Finding:

CARB should update SOPs to reflect current practice to be compliant with 40 CFR Part 58, 

Appendix A. It is recommended that SOPs be reviewed annually at a minimum and updated as 

needed. It is further suggested that analysts initial the SOPs to indicate that they have read and 

understand the SOP and have had an opportunity to discuss them with their supervisor.
90

Finding # TL2

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

An SOP is not documented for the batch certification of cleaned canisters. The canister cleaning 

SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method. 

Description:

CARB staff stated that current criteria are documented in the Quality Assurance Manual (QAM), 

but not in the SOP. It is unclear why there are different cleaning criteria for the two methods. 

Refer to previous finding. 

References:

40 CFR 58, Appendix A

Recommendation to Address Finding:

Consistency across methods would facilitate the use of the QAM as a reference. Analysts’ initials 

on SOPs would document that they understand approved current procedures. 
91

Finding # TL3

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

The batch certification of cleaned canisters described by staff for methods MLD 058 and MLD 

066 differs from existing VOC guidance.

Description:

PAMS guidance recommends that one cleaned canister out of eight be certified and Method TO15, on which these methods are based, recommends certifying every canister. CARB currently 

tests one cleaned canister of twelve for residual contamination as part of the certification process.

Batch certification can identify excessively dirty canisters or a malfunction of the cleaning 

system during the cleaning cycle, but may be inadequate to certify that every canister in a batch 

is actually clean. Analyzing one out of twelve instead of eight canisters introduces even greater 

uncertainty. The uncertainty increases if the dirtiest canister in a batch is not selected as the 

certification canister. 

References:

PAMS TAD, Method TO-15.

Recommendation to Address Finding:

CARB should either certify more of the canisters in each cleaning batch in accordance with 

current VOC method guidance for these modified methods or conduct studies to demonstrate the 

batch certification process is effective for the specific contaminants in these methods by 

analyzing every canister in a batch for a period of time. Confidence that the current practice is 

adequate would be increased if the number of canisters that failed certification is maintained in a 

log. Developing a trend chart of the level of contamination detected during canister certification 

may also serve to alert staff that cleaning equipment maintenance is needed. 
92

Finding # TL4

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

[Related to Previous Finding OL19]. Pre-cleaning concentrations are not recorded in a logbook 

to allow for the selection of the most highly contaminated canister for batch certification. 

Description:

Canisters are randomly selected for certification. As a result of a finding from the previous TSA, 

CARB has initiated a system of marking canisters that have been selected for testing as part of 

the batch certification to ensure that eventually all canisters are tested.

Certifying the canister with the most highly contaminated sample concentrations during batch 

certification in accordance with guidance would provide a higher level of confidence that the 

entire batch of canisters has been effectively cleaned. 

References:

PAMS TAD, Sec. 2.5.2.3

Recommendation to Address Finding:

CARB should select the most highly contaminated canisters cleaned in a batch for analysis to 

certify that the batch is free of contamination. 
93

Finding # TL5

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

Canisters are not routinely leak tested as prescribed in guidance. Instead, canisters are vacuum 

leak tested only when gross leaks are suspected. 

Description:

Canisters can become contaminated over time from leaks and micro leaks, which are not obvious 

from monitoring canister gauge readings. Method TO-15, on which methods MLD 066 and MLD 

058 are based, describes the process for leak testing canisters in Sec. 8.4.1.1 and establishes a 

criterion of +2 psig, beyond which the pressure should not vary. Canisters may also become 

contaminated over time through micro leaks. The PAMS TAD, Section 2.5.3.6, states that 

“...obvious leaks may be checked by submerging canisters in water, but to check for micro leaks, 

the canister should be evacuated and its pressure observed for several days with a sensitive 

absolute pressure gauge connected."

References:

Method TO-15, Sec. 8.4.1.1; PAMS TAD, Sec. 2.5.3.6

Recommendation to Address Finding:

CARB should establish a program of monitoring for leaks and micro leaks as prescribed in 

Method TO-15 and the PAMS TAD.
94

Finding # TL6

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

[Previous Finding OL21]. A retention time policy for re-cleaning and blanking canisters once 

they have been certified clean has not been established. 

Description: 

The Canister Custodian confirmed that she observes reappearance of contamination in cleaned 

canisters over time. Canisters may become contaminated over time through small leaks or micro 

leaks that may not be obvious from monitoring canister gauge readings (see Description, Finding 

5). Additionally, PAMS TAD, Section 2.5.3.2 states that "... canisters may appear 

uncontaminated immediately after cleaning, but will out-gas contaminants upon storage for 

several weeks. All canisters in use should be blanked checked frequently and particularly after 

extended periods of storage, to ensure that significant contamination does not appear." EPA 

observed probable out-gassed contaminants from canister surfaces in a recent PAMS PE Study of 

Air District laboratories conducted by the Region 9 QA Office.

References: 

PAMS TAD, Section 2.5.3.6

Recommendation to Address Finding:

CARB should establish a retention time for cleaned and certified canisters after which they must 

be re-cleaned and certified. A retention time of 30 days has been adopted by the EPA Region 9 

Laboratory and other laboratories. 
95

Finding # TL7

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Canister Cleaning

Finding:

The CARB SOP states that old canisters are reconditioned, but this is inconsistently practiced.

Description: 

Staff stated that the reconditioning procedure was determined to be ineffective and has been 

discontinued. Similar information was presented at the 2011 Air Conference in Dallas, Texas. 

A procedure for reconditioning based on best available information does not currently exist.

References: 

40 CFR Part 58, Appendix A

Recommendation to Address Finding: 

The CARB SOP relating to reconditioning of canisters should be revised or deleted, since the 

SOP does not reflect current practice or recommended procedure. CARB may wish to implement 

a procedure in its place for evaluating old canisters for replacement, as it has been observed that 

coatings on older canisters are more subject to retaining and out-gassing residual contaminants 

over time.
96

Finding # TL8

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding:

CARB has not established a holding time for cartridges once samples have been collected for 

extraction or analysis.

Description: 

Cartridges are kept for some weeks in the field before shipping them to the laboratory. Staff 

stated that cartridges are generally analyzed within the four weeks recommended by the cartridge 

vendor (Waters), but not within 14 days as specified in Method TO-11 or 30 days following 

extraction specified in the method. 

Exceeding method prescribed holding times can result in data being qualified due to potential 

loss of sample or a risk of contamination from extraneous sources, even under refrigeration. 

Exceeding prescribed method holding times can result in data that are more vulnerable to 

challenge. 

References: 

Method TO-11

Recommendation to Address Finding: 

CARB should establish a policy for holding time based on the TO-11 method holding time 

(preferred) or, if necessary, conduct and document internal research demonstrating that a 

variance from the published holding time is justified. 
97

Finding # TL9

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding:

The laboratory does not assign expiration dates to new sampling cartridges and allows cartridges 

to be used beyond the 90 days prescribed by the method. 

Description: 

Method TO-11 states in its discussion of the preparation of DNPH cartridges “that cartridges will 

maintain their integrity for up to 90 days stored in refrigerated, capped shipping tubes” (TO-11

note, sec. 9.5.2.16). Initial blank lot concentrations are provided with commercially purchased 

cartridges. Given the significant concern expressed throughout Method TO-11 over potential 

laboratory contamination, it is prudent to be alert to potential contamination during storage. One 

of the air districts assigns a six month expiration date to cartridges. A commercial laboratory 

(AAC Laboratory, Ventura, CA) confirmed that it routinely monitors and observes that blank 

concentrations increase over time, although not past criteria levels. The level of contamination 

will depend on how the cartridges are stored and if they become exposed to contaminants. 

Therefore, unused cartridge lots are probably best recertified for QA documentation purposes 

after 90 days, as suggested by guidance. 

References: 

Method TO-11, Note in sec. 9.5.2.16

Recommendation to Address Finding: 

As recommended in Method TO-11 and confirmed by laboratory experience, it should be 

verified that cartridges meet blank certification requirements by analyzing a blank cartridge 

before using them past the 90-day PAMS season.
98

Finding # TL10

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding: 

CARB's procedure for analyzing DNPH lot blanks differs from the SOP.

Description: 

The SOP states that 5% of new DNPH cartridges will be analyzed as lot blanks. Staff stated the 

practice has been changed to one cartridge per lot rather than one per box. 

The SOP should be updated to reflect current practice. SOPs document an agency’s official 

policies and procedures that staff are to adhere to obtain consistent and reliable data and are 

required as part of an agency’s approved QAPP as required by 40 CFR 58 Appendix A.

References: 

40 CFR Part 58, Appendix A

Recommendation to Address Finding: 

CARB should update the SOP to reflect current practice. 
99

Finding # TL11

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding: 

No criterion is provided in the CARB SOP for passing DNPH lot cartridge blanks. 

Description: 

Carbonyl Method TO-11 prescribes acceptance criteria for lot blanks of less than 0.15 

μg/cartridge (formaldehyde) and less than 0.10 μg/cartridge (acetaldehyde). The SOP should be 

consistent with practice. Furthermore, the criterion used by CARB of 2X RL is not appropriate. 

The RL must be higher than the blank contamination.

References: 

Method TO-11, sec. 5.8

Recommendation to Address Finding: 

CARB should establish appropriate acceptance criteria for cartridge blanks consistent with the 

method prescribed criteria and the SOP should be updated to reflect current practice. 
100

Finding # TL12

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding: 

Gloves are not worn as a contamination protection measure when handling cartridges. A 

nitrogen-purged glove bag is not used for extractions. 

Description: 

Method TO-11 cautions against the unintentional contamination of eluted samples due to 

aldehyde and ketone contamination in laboratory air, inks, adhesives, packaging, and vials with 

plastic caps. The use of gloves is prescribed when handling the cartridges. Extracting the 

cartridges in a nitrogen-purged glove further reduces the risk of contamination. Food and drink 

residue on hands can also present a contamination problem, in addition to safety issues related to

working with acetonitrile without proper protection. The use of a glove box will vary with the 

laboratory air environment; working in a very clean hood may be sufficient. 

References: 

Method TO-11, Sec. 10.7, 11.2

Recommendation to Address Finding: 

CARB should evaluate its current procedures to ensure that current contamination control 

measures are adequate.
101

Finding # TL13

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding: 

[Previous Finding OL3] Staff stated that field blanks are not being analyzed at a frequency of 

10% of field samples, as specified in Method TO-11, nor is there an SOP describing the 

procedure for the submission of field blanks. 

Description: 

During the previous TSA, staff stated that CARB was correcting sample results based on an 

average of field blank results from a study performed 15 years prior. The study was outdated and 

sample results should not be corrected. During the current TSA, staff stated that sample results 

are no longer being subtracted, but that field blanks are not being collected as prescribed in the 

method. Field blanks increase the level of confidence that sample contamination detected is not 

from extraneous sources. 

References: 

Method TO-11, Sec. 13.3.1 Table 5-3

Recommendation to Address Finding:

CARB should use field blanks at the method prescribed frequency of 10% of field samples 

collected; a minimum of one field blank per sample collection batch is recommended. Data for 

field blanks and sample results should be reported separately. 
102

Finding # TL14

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory – Carbonyls

Finding: 

CARB does not analyze trip blanks when needed.

Description: 

Method TO-11 states that it is desirable to collect trip blanks at a frequency of 10% of field 

samples. In addition to field blanks and laboratory blanks, if field blank analysis show 

contamination, trip blanks should be collected and analyzed to distinguish between sources of 

contamination. 

References: 

Method TO-11, Sec. 13.3.1 & Table 5-3

Recommendation to Address Finding:

CARB should review its policy for blanks and consider the value of collecting and analyzing trip 

blanks in addition to laboratory and field blanks if analytical results indicate blank 

contamination. 
103

Finding # TL15

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory – Carbonyls

Finding: 

Method TO-11 states that samples should be re-analyzed when results are 10% above the 

criterion, but the analyst was not aware of this criterion.

Description: 

Up-to-date SOPs help train analysts new to accepted laboratory procedures. Having analysts 

initial SOPs annually to indicate that they have read the SOPs and have had an opportunity to 

discuss them with their supervisor is also valuable. Some laboratories administer a written test to 

qualify an analyst to perform a new method. Keeping charts of the duplicate results with control 

lines indicating the criterion can ensure that laboratory QC criteria are given adequate attention at 

the time of analysis.

References: 

Method TO-11, Sec. 13.4.1 & Table 5-3, MLD 022 Sec. 9.6

Recommendation to Address Finding;

CARB should review the system of qualifying analysts to perform methods new to them and 

should implement procedures to ensure they are knowledgeable concerning all required QC. 
104

Finding # TL16

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Carbonyls

Finding: 

Working standards are tracked and used for six months, while the CARB SOP states that 

standards should be retained for four months under refrigeration.

Description: 

Periodic reviews by the supervisor of logbooks and internal audits or reviews by a Quality 

Assurance Officer would help ensure that replacement schedules are kept. Using expired 

standards can result in inaccurate data and legal challenges. 

References: 

CARB SOP; Method TO-11; GLP

Recommendation to Address Finding:

CARB should implement a policy to ensure that standards used for instrument calibration are 

replaced before they exceed holding times. 
105

Finding # TL17

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Hexavalent Chromium

Finding: 

Site name and sampling dates are recorded on a piece of tape loosely stuck to sample cartridges;

the tape occasionally falls off, making it difficult to identify samples. 

Description: 

A better system for labeling samples is needed to increase confidence that a data point is 

appropriately identified with a particular sample. 

References: 

Recommendation to Address Finding: 

CARB should devise a permanent system for labeling cartridges.
106

Finding # TL18

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - Hexavalent Chromium

Positive Finding: 

Hexavalent chromium data undergo peer review, supervisory review, review by the Branch 

Chief, and a final review before going to AQS.

Description: 

Data go through a multi-tiered review process. The data review procedure described represents a 

best practice.

References: 

Recommendation to Address Finding: 

N/A
107

Finding # TL19

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - MLD058, Aromatic & Halogenated Compounds, VOCs 

(Modified TO-15)

Finding: 

[Previous Finding OL5] There is no secondary review of logbooks.

Description: 

Secondary review of logbooks by supervisory or QA staff can help ensure that proper protocol is 

being followed.

References: 

Recommendation to Address Finding: 

Instrument run logbooks should routinely be reviewed and signed by supervisory or QA staff.
108

Finding # TL20

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - MLD058, Aromatic & Halogenated Compounds, VOCs 

(Modified TO-15)

Finding: 

CARB does not analyze audit samples or through-the-probe audit samples as suggested in Sec. 

9.7 of the CARB SOP. 

Description:

Audit samples are an important quality assurance tool to ensure the accuracy of analytical data. 

Through-the-probe audits help document that the sample and analysis system are within 

acceptable control limits. Staff stated that the SOP to analyze audit samples had been followed in 

the past, and have requested that this be reinstituted.

References: 

CARB SOP, Section 9.7

Recommendation to Address Finding: 

CARB is encouraged to use audit samples and through the probe audit samples when possible, as 

suggested in the CARB SOP, or CARB should revise its SOP to reflect actual practices. 
109

Finding # TL21

Agency: CARB

Date of Audit: Summer 2011

Program Area: Toxics Laboratory - MLD 066 Oxygenated Hydrocarbons & Nitriles 

(Modified TO-15)

Finding: 

Appendix V in the CARB SOP lists the standards that were used in 2003 and has not been 

updated to reflect the standards currently being used. 

Description: 

Staff stated current standards are found in the QC report. Outdated information in an SOP can 

lead to misunderstanding in practice and would represent a vulnerability if data are challenged. 

SOPs should be updated to reflect practice.

References: 

40 CFR 58, Appendix A (SOPs)

Recommendation to Address Finding: 

SOPs should represent current practices and be updated on a regular schedule or as needed in 

accordance with 40 CFR 58, Appendix A.
110

Finding # IMP1

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial – General

Finding: 

The ICAPCD ambient air monitoring program is not operating under an approved QAPP.

Description:

EPA requires that organizations develop a QAPP for each type of ambient pollutant being 

measured. The QAPP integrates all technical and quality aspects of a project, including planning, 

implementation, and assessment. The purpose of the QAPP is to document planning results for 

environmental data operations and to provide a project-specific “blueprint” for obtaining the type 

and quality of environmental data needed for a specific decision or use. The QAPP documents 

the quality assurance and quality control that are applied to an environmental data operation to 

assure the results obtained are of the type and quality needed and expected. 

References:

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans,

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Further guidance on developing QAPPs can be found in the guidance documents “EPA 

Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, March 2001, and 

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002

Recommendation to Address Finding:

ICAPCD is currently part of the CARB PQAO, which is responsible for maintaining consistency 

in the collection and assessment of ambient air quality data throughout the State of California so 

that the data may be combined to give meaningful information. 

ICAPCD should develop agency specific QAPPs that are consistent with existing CARB QAPPs 

or formally adopt the applicable existing CARB QAPPs. 
111

Finding # IMP2

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - General

Finding:

ICAPCD has not established an appropriate quality system for ambient air monitoring.

Description:

A quality system is the means by which an organization manages the quality of the monitoring 

information it produces in a systematic, organized manner. It provides a framework for planning 

implementing, assessing and reporting work performed by an organization and for carrying out 

required quality assurance and quality control activities. While the monitoring staff at ICAPCD 

is very knowledgeable and operates the ambient air monitoring network diligently, the lack of a 

structured quality system is needed to effectively and appropriately implement ambient air 

monitoring requirements. Major components of a quality system include:

Independence of Quality Assurance.

QMPs and QAPPs.

Data Quality Performance Requirements (Data Quality Objectives, DQOs).

QA/QC Activities.

References:

40 CFR 58 App. A 2.0, Quality System Requirements

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Recommendation to Address Finding:

ICAPCD should implement a quality system consistent with EPA requirements and applicable 

guidance.
112

Finding # IMP3

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Network Management

Finding: 

Assessment of PM10 or PM2.5 sampling frequency throughout the ICAPCD network has not been 

performed as required.

Description:

The minimum required monitoring schedules for PM10 in the area of expected maximum 

concentration should be based on the relative level of that monitoring site concentration with 

respect to the 24-hour standard, as illustrated in Figure 1 of 40 CFR 58.12. The most recent year 

of data must be considered to estimate the air quality status at the site near the area of maximum 

concentration no less frequently than as part of each 5-year network assessment. 

For PM2.5, required sites that meet the following criteria are required to sample at a 1-in-3 day 

sampling frequency:

Design value sites that are within + 10% of the NAAQS.

Sites where one or more 24-hour values have exceeded the NAAQS each year for a 

consecutive period of at least 3 years. 

In addition, required design value sites that are within 5% of the NAAQ must maintain an 

everyday sample schedule.

EPA may not be able to make attainment determinations from site where appropriate sampling 

frequency is not achieved.

References:

40 CFR 58.12 (e)

40 CFR 58.12 (d)(ii) and (iii)

Recommendation to Address Finding:

CARB and ICAPCD should perform the required analysis to ensure PM10 and PM2.5 monitoring 

in Imperial County operate at the appropriate sampling frequency. Necessary changes to 

sampling schedules should be made as expeditiously as possible. 
113

Finding # IMP4

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Network Management

Finding: 

Neighborhood scale may be inappropriate for PM10 at the Westmorland site. 

Description:

The area surrounding the Westmorland monitoring site is mostly residential surrounded by active 

agricultural fields, but is located on the Westmorland Wastewater Treatment property and may 

be influenced by local activity and not representative of a neighborhood spatial scale for PM10.

The area directly adjacent to the monitoring site is mainly comprised of unpaved areas that are 

disturbed by vehicle traffic and heavy equipment. Due to similar surface conditions throughout 

the area, the PM10 monitor is appropriately sited, but may be more appropriately characterized as 

a having a middle scale of representation.

Neighborhood scale defines concentrations within some extended area of the city that has 

relatively uniform land use with dimensions in the 0.5 to 4.0 km range. The neighborhood and 

urban scales listed below have the potential to overlap in applications that concern secondarily 

formed or homogeneously distributed air pollutants, while middle scale defines the 

concentrations typical of areas up to several city blocks in size with dimensions ranging from 

about 100 m to 0.5 km.

References:

40 CFR App. D 1.2 (b)

40 CFR App. D 4.6 (b)

Recommendation to Address Finding:

ICAPCD should evaluate the spatial scales associated with PM10 monitoring at Westmorland and 

make changes to AQS and the next annual network plan, if appropriate.
114

Finding # IMP5

Agency: CARB – ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Field Operations

Finding: 

One-point flow rate verifications for PM10 and PM2.5 are not performed by ICAPCD as required 

and are not well documented.

Description:

ICAPCD has not purchased flow rate transfer standards, and therefore the monitoring staff does 

not perform one-point flow rate verifications as required. Currently, a nearby CARB site 

operator, responsible for the Calexico Ethel monitoring site, performs all flow rate verifications 

on an “as needed” basis. Based on the available documentation at the monitoring sites, these 

checks have been missed in the past and have not been well documented. Many records were 

outdated or incomplete. Also, flow rate transfer standard certification records are not maintained 

by ICAPCD. Due to a lack of consistent documentation, it is unclear when flow rate verifications 

have been performed and whether the flow rate transfer standard used to perform the checks has 

been certified relative to an authoritative standard as required. 

A one-point flow rate verification check must be performed at least once every month on each 

automated analyzer used to measure PM10 and PM2.5. For ICAPCD, these should be performed 

monthly on the PM10 BAM1020’s operating at Niland and Brawley and filter based PM2.5 

monitors at El Centro. The same issues are present for high-volume PM10 samplers, which are 

required to have one-point flow rate verifications performed on a quarterly basis. 

References:

40 CFR 58 App. A 2.6 

40 CFR 58 App. A 3.2.3

40 CFR 58 App. A 3.3.2

Recommendation to Address Finding:

ICAPCD should obtain the appropriate flow rate transfer standards for automated PM10

analyzers, filter based PM2.5 samplers, and high-volume PM10 samplers and perform flow rate 

verifications as required by regulation. Also, these activities should be consistently documented 

by ICAPCD. 
115

Finding # IMP6

Agency: CARB – ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Field Operations

Finding: 

Residence time for gaseous monitors operated by ICAPCD is not established.

Description:

The residence time is defined as the amount of time that it takes for a sample of air to travel from 

the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 

states that for the reactive gases (O3, NO2, and SO2) residence times must be less than 20 

seconds. Additionally, it is recommended that the residence time within the manifold and 

sample lines to the instruments should be less than 10 seconds. The station technicians should 

calculate the residence time, document it in the station logbook, and periodically verify the data. 

There was not a clear record of residence time of the sampling lines at each site. Also, the site 

operators did not know how recently the residence time had been recalculated. At a minimum, 

the residence time should be calculated for each instrument after any change is made to the 

sampling train, such as the removal or addition of other instruments, and posted at each site. 

The station technicians should calculate the residence time, document it in the station logbook or 

other form available at the site, and periodically verify the data. 

References:

40 CFR 58 App. E 9 (c)

Recommendation to Address Finding:

In order limit the potential for significant losses of O3 through the sampling line, residence times 

should be calculated, documented, and tracked. If residence times are higher than those required 

by regulation, ICAPCD should make the necessary changes to the sampling train to reduce the 

residence time. 
116

Finding # IMP7

Agency: CARB – ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Field Operations

Finding: 

ICAPCD is internally post weighing high-volume PM10 filters without proper PM lab or quality 

control measures.

Description:

Traditionally, all high volume PM10 filters are processed and weighed by CARB in appropriately 

controlled environments and necessary quality control and quality assurance techniques. 

ICAPCD stated that often the post-weigh information is not transmitted back to ICAPCD from 

CARB in a timely manner. As a result, ICAPCD has implemented a post-weigh procedure for 

PM10 high-volume filters in order to get a preliminary assessment of whether the samplers are 

measuring exceedances of the standard, so that the appropriate planning actions and preparation

can occur immediately after the sample has been collected. 

These preliminary post-weighing procedures are not performed in a controlled environment nor 

do they follow the required quality control procedures. Furthermore, the weighing and 

subsequent handling of these filters prior to the official CARB post-weigh may introduce bias in 

the sample. 

References:

Method 2.12 Sec. 7

Recommendation to Address Finding:

ICAPCD should discontinue the internal post-weighing practices and work with CARB to 

establish an appropriate procedure for the timely transmittal of CARB post-weigh information to 

ICAPCD.
117

Finding # IMP8

Agency: CARB – ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial – Field Operations

Finding: 

Documentation of ICAPCD air monitoring activities is not complete.

Description:

Accurate and complete documentation is essential to the collection of air quality data used for 

regulatory purposes. Appropriate documentation includes, but is not limited to, standard 

operating procedures for all aspects of an organization's program, data quality assessments, 

logbooks tracking actual day-to-day operations, and records of quality control, quality assurance, 

and maintenance checks. Oversight of personnel and activities involved in the collection, 

processing and submittal of data is facilitated by procedures that are standardized and responsible 

personnel record their compliance with these procedures.

Currently, ICAPCD does not have a formal or consistent process for documenting air quality 

monitoring activities. For example, many records are maintained on loose-leaf paper or post-it

notes (instrument maintenance records, PM10 make-up sample dates, PM10 motor repair, and 

notes on changes made to the data in the database). Many records or entries in logbooks are 

made in pencil, not initialed, and were limited in information or specificity. 

In response to a data tracking request, documentation of flow rate verifications and calibrations 

of PM10 analyzers could not be located, and documentation supporting data invalidation was not 

present. 

References:

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Recommendation to Address Finding:

ICAPCD should develop and implement procedures for maintaining adequate documentation of 

ambient air monitoring activities.
118

Finding # IMP9

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Network Management

Finding: 

There are potential siting issues at the Calexico Ethel site.

Description:

The Calexico Ethel monitoring site is located in the parking lot of a high school in a mostly 

residential area. The primary concern is the distance of the monitoring site to nearby trees. Trees 

can act as obstructions in cases where they are located between the air pollutant sources or source 

areas and the monitoring site, and where the trees are of a sufficient height and leaf canopy 

density to interfere with the normal airflow around the probe, inlet, or monitoring path. The 

scavenging effect of trees is greater for O3 than for other criteria pollutants and monitoring 

agencies must take steps to consider the impact of trees on O3 monitoring sites. To reduce the 

potential interference/obstruction, the probe or inlet must be at least 10 m or further from the drip 

line of trees. 

Other potential issues include monitor spacing on the roof and the distance of the collocated 

PM2.5 monitors to the trailer. Generally, the distance from the obstacle to the probe, inlet, or 

monitoring path must be at least twice the height that the obstacle protrudes above the probe or 

inlet.

References:

40 CFR 58 App. D 4 (a)

40 CFR 58 App. D 5

40 CFR 48 App. A 3.2.6.3

Recommendation to Address Finding:

CARB staff have been working with EPA and ICAPCD to relocate the site. EPA recommends 

continuing this effort in a timely manner.
119

Finding # IMP10

Agency: CARB - ICAPCD

Date of Audit: Summer 2011

Program Area: Imperial - Data Management

Finding:

ICAPCD is not adequately reviewing and editing data. 

Description:

(See Finding DM2)

The current database does not allow staff to adequately review and edit data. ICAPCD uses a 

WinCollect data management system developed by Ecotech for data acquisition, storage, and 

processing. Monitoring staff has indicated that the current system is hard to work with, and that 

data review and editing are cumbersome and time consuming tasks. For example, the data system 

can only automatically assign one flag (AY: “QC Control Points”) to the raw data. As a result, 

monitoring staff must manually edit hourly text files to make any adjustments. This process 

introduces the potential for errors in the data and reduces monitoring staff’s ability to effectively 

review and edit data appropriately.

ICAPCD staff has contacted Ecotech about these issues, but have a difficult time implementing 

changes to the system, as Ecotech has been largely unresponsive.

ICAPCD would benefit from data validation training.

References:

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Recommendation to Address Finding:

ICAPCD should work with CARB to implement a more efficient and comprehensive data 

management system and to obtain data validation training.
120

Finding # MEN1

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – QA Management

Finding:

MeCAQMD staff was not familiar with the CARB QMP or instrument SOPs. 

Description:

Staff appeared to be trained and proficient with the procedures that are used to conduct his 

monitoring activities. However, it was noted that this training and/or demonstration of 

proficiency was not adequately documented.

MeCAQMD staff stated that the district operates under CARB’s QMP and SOPs. The staff was 

not aware where electronic or hard copies of QMP or SOPs could be found. Although the staff 

was not able to find these documents when EPA was on site, they were later located in 

MeCAQMD files and accessible online. MeCAQMD noted that CARB’s SOPs are not entirely 

relevant to the MeCAQMD sites since they refer to different data acquisition systems that are not 

used by MeCAQMD.

References:

Recommendation to Address Finding:

MeCAQMD should review CARB QMP and relevant SOPs (see 

http://www.arb.ca.gov/aaqm/qa/pqao/pqao.htm). MeCAPCD should develop a formal system to 

ensure and document that all staff are familiar with the quality management system and are 

trained and proficient at the monitoring tasks that they are performing. MeCAPCD should also 

ensure that field operators have access to information in relevant SOPs at the field station, either 

in hard copy or electronically. Finally, MeCAQMD should develop SOPs for activities that are 

not covered by CARB SOPs.
121

Finding # MEN2

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Network Management

Finding:

MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2006 but is 

erroneously listed as its own PQAO in AQS.

Description:

40 CFR 58.1 defines a PQAO as “a monitoring organization or other organization that is 

responsible for a set of stations that monitor the same pollutant and for which data quality 

assessments can be pooled. Each criteria pollutant sampler/monitor at a monitoring station in the 

SLAMS and SPM networks must be associated with one, and only one, primary quality 

assurance organization.” Many requirements specified in 40 CFR 58, such as those for 

collocation, QAPPs, QMPs, and audits, are determined on a PQAO basis.

MeCAQMD is part of CARB’s PQAO, not its own PQAO.

References:

40 CFR 58.1

Recommendation to Address Finding:

MeCAQMD should be replaced by CARB as the official PQAO listing in AQS for all 

MeCAQMD data in order to reflect the current PQAO structure in California. Although EPA 

does not encourage MeCAQMD to do so, if it wishes to become its own PQAO or join another 

PQAO, MeCAQMD should follow the procedure outlined in EPA Region 9’s PQAO Strategy to 

ensure that all regulatory requirements continue to be met.
122

Finding # MEN3

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Field Operations

Positive Finding:

MeCAQMD stations were well-maintained. Staff and manager were professional and helpful, 

and very knowledgeable about the county and potential pollution sources. The station operator 

was proactive about troubleshooting instrument issues. 

Description:

References:

Recommendation to Address Finding:

N/A
123

Finding # MEN4

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino - Field Operations

Finding: 

One-point QC checks (flow verifications) for PM10 and PM2.5 are not consistently performed by 

the MeCAQMD site operator.

Description:

Mendocino site operators maintain “Maintenance and Service Log” sheets at each PM10 and 

PM2.5 site. These include a line for monthly flow rate verifications. These are not regularly 

notated as having occurred (see photo below). MeCAQMD indicated that resource constraints 

prevent this requirement from being met consistently. 

References:

40 CFR part 58 Appendix A, section 3.2.3. “Flow Rate Verification for Particulate Matter.”

Recommendation to Address Finding:

MeCAQMD should conduct and log monthly flow rate verifications at each PM site every 

month.
124

Finding # MEN5

Agency: California Air Resources Board – Mendocino County AQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Field Operations

Finding:

MeCAQMD logbook entries are not consistently made and are not consistently in the most 

defensible form. Handwritten notes are occasionally illegible due to water (rain) marks.

Description:

Logbooks should be in the form of bound log books with numbered pages and all entries initialed 

and made in indelible ink. Corrections should be made by drawing a single line through the 

information, initialing and dating. Information such as instrument down times should be 

included. 

References:

Section 5 of the QA Handbook states that records supporting the operation of air monitoring 

measurement systems should be retained for at least three years or, if the records are part of any 

litigation, claim, negotiation, etc., the records should be kept until the issue has been resolved. 

Recommendation to Address Finding:

MeCAQMD should create and follow a logbook entry convention to ensure thorough and 

defensible record-keeping. The records maintained by individuals should be periodically 

evaluated to ensure they are consistent, secure, regularly maintained, and (for electronic records) 

backed-up. MeCAQMD should take steps to prevent water damage to entries or loss of 

information due to misplaced single log sheets.
125

Finding # MEN6

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino - Field Operations

Finding:

Residence time calculations were not available at the Ukiah Gobbi site.

Description:

The residence time is defined as the amount of time that it takes for a sample of air to travel from 

the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 

states that for the reactive gases (O3, NO2, and SO2) residence times must be less than 20 

seconds. Additionally, it is recommended that the residence time within the manifold and 

sample lines to the instruments should be less than 10 seconds. The station technician should 

calculate the residence time, document it in the station logbook, and periodically verify the data. 

There was not a clear record of residence time of the sampling lines at the Gobbi site. The site 

operator did not know how recently the residence time had been recalculated. At a minimum, the 

residence time should be calculated for the instrument after any change is made to the sampling 

train, such as the removal or addition of other instruments, and posted at each site. 

The station technician should calculate the residence time, document it in the station logbook or 

other form available at the site, and periodically verify the data (e.g., annually). 

References:

40 CFR Part 58, Appendix E Section 9

Recommendation to Address Finding:

MeCAQMD should calculate residence times for all gaseous monitors. MeCAQMD should 

modify sites with residence times in excess of 20 seconds with a goal of 10 seconds, and evaluate 

any impact on compliance data due to excessive residence times. Residence times should be 

posted or accessible on-site.
126

Finding # MEN7

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino - Field Operations

Finding:

Trees at the Ukiah Gobbi and Ukiah Library sites should be evaluated against siting 

requirements.

Description:

40 CFR Part 58, Appendix E, Section 5 states “trees can provide surfaces for SO2, O3, NO2

adsorption or reactions, and surfaces for particle deposition . . . to reduce this possible 

interference/obstruction, the probe, inlet, or at least 90 percent of the monitoring path must be at 

least 10 meters from the drip line of trees.” 

The tree drip line at Gobbi is coming close to the 10 m distance. The trees at the Library site 

appeared to meet siting requirements at the time of the TSA, but should be monitored over time.

References:

40 CFR Part 58, Appendix E, Section 5

Recommendation to Address Finding:

MeCAPCD should verify that trees are meeting siting requirements, and check regularly.
127

Finding # MEN8

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino - Field Operations

Finding:

The internal shelter thermostat is not operating correctly at the Ukiah Gobbi site and has not been 

addressed to provide defensible data.

Description:

The Ukiah Gobbi site operator determined that the internal shelter temperature is off by 4°, and is 

manually correcting the data. The issue and correction have not been formally documented.

References:

Recommendation to Address Finding:

The issue should be addressed using a corrective action form. The resolution should be 

defensible and well-documented. Data that could have been affected by erroneous temperature 

readings should be checked and flagged or invalidated if appropriate.
128

Finding # MEN9

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Field Operations

Finding:

MeCAQMD has no system for tracking and controlling station and instrument logbooks.

Description:

Field procedures require that logbooks be kept. However, these logbooks were not tracked, 

identified, and archived in a manner to ensure that the critical documentation they contain will be 

accessible and defensible.

References:

Recommendation to Address Finding:

MeCAQMD should develop a standardized procedure for creating, labeling, and archiving 

logbooks.
129

Finding # MEN10

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Field Operations

Finding:

MeCAQMD should have formalized training requirements for all air monitoring staff.

Description:

MeCAQMD makes an effort to participate in trainings when opportunities arise. However, there

is no formal program to ensure that staff are trained on procedures and demonstrate proficiency 

on tasks directly related to their job functions. 

The QA Handbook, Section 4 discusses the need for a formalized training program. EPA 

recognizes that funding is limited and it is often difficult to send people to trainings. Developing 

a formalized training program can help agencies identify what trainings are needed, what the 

highest priority issues are, and what resources are available. If it is not possible to fulfill the 

training need immediately, the training plan allows agencies to look for future funding or other 

opportunities. 

EPA also encourages agencies to formalize and document on-the-job trainings. Trainings could 

be given by staff to provide common understanding and competency and minimize future 

problems and questions. In-house trainings could include information on the Envista program; 

training for site operators, data users, and data validators on AQS flags and why they are 

important; new QAPP/SOP training; data validation and analysis; instrument operation and 

maintenance training.

MeCAQMD should coordinate with CARB on trainings.

References:

The QA Handbook, Section 4

Recommendation to Address Finding:

MeCAQMD should develop a formal training program and tracking system to ensure that all 

staff and management are familiar with the relevant QAPPs and SOPs related to producing data 

in the field, data management and tracking, quality assurance, and all data systems. 
130

Finding # MEN11

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino – Data Management

Finding:

MeCAQMD does not provide CARB AQAS with datasets that have been fully quality assured 

and ready for upload to AQS.

Description:

MeCAQMD was unable to account for some data in AQS. For example, instances were 

observed where AQS was missing a data point, or had a value when the County showed a span 

check with no associated value. MeCAQMD gives CARB an AQS-formatted file with e-mailed 

notes. The local agency leaves it to CARB’s discretion whether the e-mailed notes should result 

in flagged data. There is no SOP for conducting data review and validation, and data are not 

checked after they are sent to CARB for entry into AQS.

References:

Recommendation to Address Finding:

CARB should ensure that all local agencies under the PQAO have a data validation SOP, and 

MeCAQMD and CARB should work together to ensure that an SOP is in place for MeCAQMDcollected data. The SOP should include a procedure whereby the local agency checks any 

changes, including flagging, that CARB makes to the data, either before or after posting, as well 

as a procedure to discuss any issues the agency may have with the changes. The local agency 

should keep correspondence and data they send to CARB where it can be easily retrieved and 

reviewed. This SOP should also include procedures to ensure that data are provided to CARB in 

a form and condition that is ready for direct upload. See Finding DM2.
131

Finding # MEN12

Agency: CARB - MeCAQMD

Date of Audit: Summer 2011

Program Area: Mendocino - QA Management

Finding:

MeCAQMD does not use a formal corrective action system.

Description:

MeCAQMD staff does a considerable amount of troubleshooting. But information as to what the 

initial problem was, when the issue was first noted, what steps were taken to resolve the issue, 

and when it was resolved is not consistently recorded and is kept in different locations. 

EPA quality management standards (EPA QA/R-2, Quality Improvement Section) require that 

management and staff “ensure that conditions adverse to quality are” prevented, identified 

promptly, fully defined, corrected, prevented from recurring, and documented as corrective 

actions which are tracked to closure. There is a corrective action process for the performance 

audit program. There should also be a formal, documented mechanism for elevating potentially 

significant corrective actions originating from the laboratory staff or field operators.

References:

Recommendation to Address Finding:

MeCAQMD should institute a formal corrective action process for problems originating in the 

field or laboratory.
132

Finding # SJV1

Agency: CARB – SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - General

Positive Finding:

In general, the SJVAPCD monitoring program is robust and the agency staff and managers 

involved in the program are committed to the objective of producing defensible data of known 

quality. 

Description:

Specific examples of good practices observed during this audit include:

1. Management and staff at all levels are actively engaged in improving the program, including 

monitoring systems and other data collection processes.

2. Staff and managers that participated were professional and helpful during the audit.

3. Field operators are skilled and knowledgeable.

4. Site operators receive good hands-on training with frequent information sharing and issues 

communication via meetings held every other Monday.

5. A new system is being developed for verifying zero air generators.

6. There are three levels of independent data review and the process is well documented.

References:

Recommendation to Address Finding:

N/A
133

Finding # SJV2

Agency: CARB – SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - QA Management

Finding:

SJVAPCD does not have updated quality system documentation for all activities.

Description:

A quality system is the means by which an organization manages the quality of the monitoring 

information it produces in a systematic, organized manner. It provides a framework for planning 

implementing, assessing and reporting work performed by an organization and for carrying out 

required quality assurance and quality control activities. While the monitoring staff at SJVAPCD 

is very knowledgeable and operates the ambient air monitoring network diligently, the lack of a 

structured quality system reduces its ability to implement ambient air monitoring requirements

effectively and appropriately. Major components of a quality system include:

Independence of Quality Assurance.

QMP, QAPPs and SOPs.

Data Quality Performance Requirements (Data Quality Objectives, DQOs).

QA/QC activities.

References:

40 CFR 58 App. A 2.0, Quality System Requirements

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002

Further guidance on developing QAPPs can be found in the guidance documents “EPA 

Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, March 2001, and 

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002

Recommendation to Address Finding:

SJVAPCD management may adopt CARB’s Quality System documentation, develop its own, or 

adopt CARB’s with changes to match its program. Any deviation from the CARB QMP or 

QAPPs documentation must be formally approved by CARB as the lead agency in the PQAO.
134

Finding # SJV3

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Network Management

Finding:

SJVAPCD has experienced significant data losses at required monitoring sites, including sites 

critical for demonstrating compliance with the NAAQS.

Description:

There have been several recent examples of significant data loss due to downtime for temporary 

site closures for repairs and site relocations, including the Corcoran and Bakersfield-Golden 

State Highway sites. The upgrades were necessary for safety and long-term longevity of a 

station, and the site relocations in question were largely driven by circumstances beyond 

SJVAPCD’s control. However, these modifications could be implemented in a manner that 

would minimize the amount of data loss, including better communication or the construction of 

temporary sites to cover data collection during site closure.

References:

Recommendation to Address Finding:

SJVAPCD should develop a process to ensure that routine site maintenance or unexpected site 

relocations do not compromise data completeness.
135

Finding # SJV4

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Network Management

Finding:

SJVAPCD has initiated network modifications for several required sites without seeking EPA 

approval as required by 40 CFR 58.14.

Description:

Monitoring agencies are required per 40 CFR 58.14 to seek EPA’s approval for network 

modifications, including site closure or relocation. SJVAPCD has often informally 

communicated network changes but has not always followed the formal process as required by 

40 CFR 58.14. The request submitted to EPA must address how the criteria in 40 CFR 58.14 are 

met. Early communication between agencies is particularly crucial for high concentration or 

design value sites in order to develop acceptable plans for concurrent monitoring at the old and 

new sites in order to meet future data needs.

References:

40 CFR 58.14

Recommendation to Address Finding:

SJVAPCD should work with CARB and EPA to develop a plan for site closure or relocation that 

meets agencies’ needs and federal requirements.
136

Finding # SJV5

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Field Operations

Finding:

The residence time of flow between the inlet and each instrument was not posted at each 

SJVAPCD site.

Description:

The residence time is defined as the amount of time that it takes for a sample of air to travel from 

the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 

states that for the reactive gases (O3, NO2, and SO2) residence times must be less than 20 

seconds. Additionally, it is recommended that the residence time within the manifold and 

sample lines to the instruments should be less than 10 seconds. The station technicians should 

calculate the residence time, document it in the station logbook, and periodically verify the data. 

There was not a clear record of residence time of the sampling lines at each site. Also, the site 

operators did not know how recently the residence time had been recalculated. At a minimum, 

the residence time should be calculated for each instrument after any change is made to the 

sampling train, such as the removal or addition of other instruments, and posted at each site. 

The station technicians should calculate the residence time, document it in the station logbook or 

other form available at the site, and periodically verify the data. 

References:

40 CFR Part 58, Appendix E Section 9

Recommendation to Address Finding:

SJVAPCD should calculate residence times for all gaseous monitors. SJVAPCD should modify 

sites with residence times in excess of 20 seconds, with a goal of 10 seconds, and evaluate any 

impact on compliance data due to excessive residence times. Residence times should be posted 

or accessible on-site. Each time a modification is made to the sampling train, the residence time 

should be recalculated and posted.
137

Finding # SJV6

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV – Field Operations

Finding:

Some SJVAPCD site logbooks lacked specific information about the date or type of maintenance 

performed on an instrument. 

Description:

In general, documentation should show data are of adequate quality, as well as any related 

unusual circumstances. Documentation of the activities occurring at monitoring stations should 

be consistent throughout the network and should, at a minimum, include all repairs, calibrations, 

audits, or other maintenance performed. Maintaining complete logbooks will help to develop a 

comprehensive history of the station. This will aid field technicians pinpoint and assess problems 

that may arise with the station and provide important information for data validation. 

Overall documentation at sites was generally thorough; however, more specifics should be 

included in logbooks at the site. For example, an entry noting that maintenance was performed 

on a certain date should identify the instrument and either what specific activities were 

performed or where that information can be found. Currently, SJVAPCD does not have a 

standard system in place to ensure consistency of documentation. 

Standardizing logbook entries to include the following may be helpful. This might include: 

Date, time and initials of the person(s) who have arrived at the site.

Visitors.

Brief description of the weather (e.g., clear, breezy, sunny, raining).

Brief description of exterior of the site. Any changes that might affect the data should be 

recorded – for instance, if someone is parking a truck or tractor near the site, this may 

explain high NOX values.

Any unusual noises, vibrations, or anything out of the ordinary.

Records of any station maintenance or routine operations performed.

Description of the work accomplished at the site (e.g., calibrated instruments, repaired 

analyzer).

Dates that instrumentation were repaired or changed and serial numbers of replacement 

instruments.

Detailed information about the instruments that may be needed for repairs or 

troubleshooting.

Other pertinent information recorded in other logbooks.

References:

QA Handbook Volume II, Section 11.2.3, Instrument and Site Logs
138

Recommendation to Address Finding:

SJVAPCD should have a more consistent policy as to what station operators enter into the station 

logbook, including routine maintenance, instrument repair, audits and calibrations, and logbook 

closeout comments. Logbooks should be routinely reviewed to ensure that pertinent information 

is being recorded.
139

Finding # SJV7

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Field Operations

Finding:

There is no documentation of management review of station logbooks and other site activities for 

SJVAPCD operated sites.

Description:

The monitoring manager plays a very active oversight role, including in-person site visits and 

checks of log books and maintenance sheets. This practice is very useful and should be 

documented by initialing the site logbook or maintenance sheet, to indicate what was reviewed. 

If the vacant position of senior technician were filled, that person could assume some of the 

responsibilities currently performed by the manager, including this oversight role.

References:

Recommendation to Address Finding:

The SJVAPCD monitoring manager or senior technician should initial logbook or maintenance 

sheet and indicate what dates were reviewed.
140

Finding # SJV8

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Field Operations

Finding:

SJVAPCD site operators do not have a quick visual way to identify changes in instrument 

performance or QC checks that would indicate instrument issues, nor do they have the ability to 

remotely check on data or site operations.

Description:

The SJV monitoring network covers a large geographic area, so identifying efficiencies for site 

operators in their routine site maintenance is critical for resource management. Operators spend 

much time verifying data. Providing remote access capability to real-time site data or instrument 

meta-data would help identify priority issues and make the operator’s trips to the sites more 

efficient. Visual tools to track instrument performance or QC checks would also reduce the 

amount of time needed for level 1 data validation review. 

References:

The use of control charts to monitor quality control parameters is recommended in the QA

Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring 

Program, December 2008. 

Recommendation to Address Finding:

The district is currently developing a data management system, and should consider the potential 

benefit of having a system capable of remote access and different types of data visualization. As 

SJVAPCD develops new tools for data access and review, the use of control charts to track longterm performance of the instruments graphically should be considered.
141

Finding # SJV9

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV – Field Operations

Finding:

SJVAPCD experiences data loss due to instrument malfunction.

Description:

Even though routine maintenance and calibrations are scheduled to minimize it, significant 

downtime occurs, possibly the result of running instruments beyond the expected life cycle, and

past the time when support from the manufacturer is available. 

References:

Recommendation to Address Finding:

SJVAPCD should budget for equipment replacement according to the life expectancy of the 

monitor and have backup instrumentation ready for field deployment in the case of a prolonged 

instrument breakdown. 

SJVAPCD should also compile a list of equipment replacement needs and share it with EPA in 

the event that equipment replacement funds become available.
142

Finding # SJV10

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV – Data Management

Finding:

It is unclear whether SJVAPCD is using appropriate criteria to invalidate or flag PM10 data. 

Description:

SJVAPCD has a thorough, multi-level data validation process, but it is unclear that appropriate 

criteria are being used to invalidate or flag data, specifically in the case of continuous PM data. 

Standard data review and validation procedures should be documented in detail, including the 

criteria used to flag and invalidate data.

References:

Guidance for Preparing Standard Operating Procedures (QA/G-6) EPA600/B-07/001/April 2007.

Recommendation to Address Finding:

SJVAPCD should develop step-by-step instructions for data review and validation in SOPs or 

QAPPs, including specific criteria for appropriate flagging of data. 
143

Finding # SJV11

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - Data Management

Finding:

SJVAPCD experiences significant resource inefficiencies for staff and management as the 

current data management system relies solely on manual inputs.

Description:

While SJVAPCD’s current three-level data review process is very thorough, performing this 

process entirely by hand is inefficient and very time-consuming. Implementing a new data 

management system should decrease the amount of time needed for this task and free up muchneeded resources. In developing a new system, all staff involved in the current review process

should participate, as well as talking to other monitoring agencies that have recently developed 

these systems, to ensure that any system under consideration encompasses all necessary features.

References:

Recommendation to Address Finding:

SJVAPCD should continue to work on upgrading the data management system, identifying the 

components that are necessary for accomplishing SJVAPCD’s monitoring program goals.
144

Finding # SJV12

Agency: CARB - SJVAPCD

Date of Audit: Summer 2011

Program Area: SJV - QA Management

Finding:

SVJAPCD does not have a formal corrective action process in place.

Description:

EPA quality management standards (EPA QA/R-2, Quality Improvement Section) require that 

management and staff “ensure that conditions adverse to quality are” prevented, identified 

promptly, fully defined, corrected, prevented from recurring, and documented as corrective 

actions that are tracked to closure. 

While corrective action seems to be occurring in a timely fashion for most issues, a formal 

corrective action process would serve as documentation for the issues being resolved, capture the 

process and keep it consistent through staff or management turnover, and share the results of the 

corrective action with staff.

References:

EPA QA/R-2, Quality Improvement Section

Recommendation to Address Finding:

SVJAPCD should institute a corrective action process to provide a formal, documented 

mechanism for elevating potentially significant corrective actions originating from field or data 

review operations.
145

APPENDIX A: SUMMARY OF FINDINGS

G1: [Previous Finding M1] CARB needs to complete the process of putting a formal PQAO into 

place.

G2: The QMB does not have the structure and staff to manage QA oversight of the PQAO 

districts.

G3: [Previous Finding M6] While progress has been made on updating the CARB QA Manual 

with a QMP and QAPPs or equivalent documents, the process is behind schedule.

G4: Local districts within the CARB PQAO do not always have updated quality system 

documentation for all activities.

G5: [Previous Findings QM1 and M3] QA Authority and interactions between QMB and the 

other branches should be expanded and formalized. The corrective action system should be 

developed to include actions taken, in addition to reports issued by the QA auditors and the 

calibration laboratory.

G6: Coordination between CARB and districts and EPA should be improved.

NM1: Not all agencies within the CARB PQAO have approved network plans since this became 

a requirement in 2006. The current approach to network plans does not provide for a 

determination of network adequacy on a statewide basis.

NM2: The network assessment does not meet all CFR requirements.

NM3: There are PM10 monitors listed in local conditions (LC; parameter code 85101), but not 

Standard Temperature and Pressure (STP; parameter code 81102 in AQS), as required for 

FRM/FEM instruments. 

FO1: [Related Previous Findings GB3, SJV3, & NS2] Documentation at the CARB field sites is 

inadequate and not reviewed by management.

FO2: Management oversight of site operators needs strengthening.

FO3: CARB field operators have not been trained on new SOPs.

FO4: Residence time calculations were not available at any CARB site visited.

FO5: Delay in sending PM2.5 samples has resulted in loss of data.

FO6: PM make-up samples are not being taken in accordance with EPA guidance.
146

FO7: PM10 QC checks are not consistently recorded. There is no document in which field 

operators are directed to record this information.

FO8: CARB field staff do not check data after sending information to CARB offices.

FO9: The Yuba City site has several significant siting issues that need to be resolved.

FO10: Records indicate that calibrations of gaseous pollutant instruments are not consistently 

done according to a schedule.

FO11: [Previous Finding AQSB7] The number of NO2 titration points taken during calibration 

does not meet regulatory requirements.

FO12: Multi-point calibrations of PM2.5 instruments are not done routinely.

FO13: [Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air 

being used in the program.

FO14: Span and precision gases used in the field are not being calibrated routinely.

FO15: Instruments removed from the field are not always efficiently tracked and returned to the 

repair laboratory facility for diagnosis, repair, and reuse. Loss of data has occurred due to 

unavailability of spare instruments.

FO16 (Positive): CARB is working to improve communication with field staff.

FO17 (Positive): CARB maintains a well equipped stockroom of spare parts, maintains a large 

equipment purchase order history, and develops thorough equipment testing procedures that are 

regularly updated.

DM1: The data validation and review/verification performed by AQSB, NLB, and AQAS, are 

not formally published in a control-copied SOP.

DM2: [Previous Finding M7] Data submitted by local districts within the CARB PQAO are not 

validated using consistent procedures. (See Findings SJV9, IMP10, and MEN11)

DM3: [Previous Finding DM5] AQAS does not ensure that local district data are validated prior 

to upload to AQS.

DM4: A few instances of erroneous continuous data were identified in AQS for CARB sites.

DM5: Erroneous continuous data were identified in AQS for non-CARB sites within the CARB 

PQAO.

DM6: [Previous Finding DM6] There are numerous deficiencies in the data certification process 

for the CARB PQAO, including:
147

Not all NAAQS-compliant data within the CARB PQAO are routinely certified. 

Data certified by CARB for local districts are not typically reviewed or validated. 

Data are routinely certified by agencies within the State of California, but responsibility 

has not been formally delegated to any local agencies within the State of California.

DM7: Data, including those used for design value sites, have been changed after they are 

certified and subsequently not recertified.

DM8: Some local districts within the CARB PQAO are listed as PQAOs in AQS.

DM9: There were several instances of CARB altering data collected by local districts without 

communicating with the district.

QA1: The QA Audit group has made an effort to improve its documentation process; however, 

several inconsistencies were noted.

QA2: The audit trailer evaluated was using one expired gas cylinder along with others that had 

not been certified annually as required for the EPA National Performance Audit Program 

(NPAP). 

QA3: The QA Section is not tracking monitors to ensure that 25% are being audited per calendar 

quarter.

QA4: The connection to the inlet on the audit trailer could pull in outdoor air.

QA5: Auditors do not review all applicable siting information in AQS prior to an audit.

QA6: [Previous Findings M4 & OPA2] Quality assurance for special projects is not developed in 

a process consistent with EPA quality system requirements.

QA7: Mass flow elements (MFEs) are used to establish calibration points outside of their 

calibrated range.

PM1: Communication of post-weigh information and transmission of documentation to local 

districts should be improved.

PM2: The PM laboratory does not have a formal corrective action process for addressing issues 

with PM filter collection.

PM3: Documentation of activities in the PM10 and PM2.5 laboratories should be improved.

PM4: PM10 trip blanks are not being used to assess potential bias from filter transport and 

handling.

TL1: The canister cleaning SOP does not reflect the current cleaning procedure.
148

TL2: An SOP is not documented for the batch certification of cleaned canisters. The canister 

cleaning SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method. 

TL3: The batch certification of cleaned canisters described by staff for methods MLD 058 and 

MLD 066 differs from existing VOC guidance. 

TL4: [Previous Finding 19]. Pre-cleaning concentrations are not recorded in a logbook to allow 

for the selection of the most highly contaminated canister for batch certification. 

TL5: Canisters are not routinely leak tested as prescribed in guidance. Instead, canisters are 

vacuum leak tested only when gross leaks are suspected. 

TL6: [Previous Finding 21]. A retention time policy for re-cleaning and blanking canisters once 

they have been certified clean has not been established. 

TL7: The CARB SOP states that old canisters are reconditioned, but this is inconsistently 

practiced. 

TL8: CARB has not established a holding time for cartridges once samples have been collected 

for extraction or analysis.

TL9: The laboratory does not assign expiration dates to new sampling cartridges and allows 

cartridges to be used beyond the 90 days prescribed by the method. 

TL10: CARB's procedure for analyzing DNPH lot blanks differs from the SOP. 

TL11: No criterion is provided in the CARB SOP for passing DNPH lot cartridge blanks. 

TL12: Gloves are not worn as a contamination protection measure when handling cartridges. A

nitrogen-purged glove bag is not used for extractions. 

TL13: [Previous Finding OL3] Staff stated that field blanks are not being analyzed at a 

frequency of 10% of field samples, as specified in Method TO-11, nor is there an SOP describing 

the procedure for the submission of field blanks. 

TL14: CARB does not analyze trip blanks when needed. 

TL15: Method TO-11 states that samples should be re-analyzed when results are 10% above the 

criterion, but the analyst was not aware of this criterion.

TL16: Working standards are tracked and used for six months, while the CARB SOP states that 

standards should be retained for four months under refrigeration.

TL17: Site name and sampling dates are recorded on a piece of tape loosely stuck to sample 

cartridges; the tape occasionally falls off, making it difficult to identify samples.
149

TL18 (Positive): Hexavalent chromium data undergo peer review, supervisory review, review by 

the Branch Chief, and a final review before going to AQS. 

TL19: [Previous Finding OL5] There is no secondary review of logbooks.

TL20: CARB does not analyze audit samples or through-the-probe audit samples as suggested in 

Sec. 9.7 of the CARB SOP. 

TL21: Appendix V in the CARB SOP lists the standards that were used in 2003 and has not been 

updated to reflect the standards currently being used. 

IMP1: The ICAPCD ambient air monitoring program is not operating under an approved QAPP.

IMP2: ICAPCD has not established an appropriate quality system for ambient air monitoring.

IMP3: Assessment of PM10 or PM2.5 sampling frequency throughout the ICAPCD network has 

not been performed as required.

IMP4: Neighborhood scale may be inappropriate for PM10 at the Westmorland site.

IMP5: One-point flow rate verifications for PM10 and PM2.5 are not performed by ICAPCD as 

required and are not well documented.

IMP6: Residence time for gaseous monitors operated by ICAPCD is not established.

IMP7: ICAPCD is internally post weighing high-volume PM10 filters without proper PM lab or 

quality control measures.

IMP8: Documentation of ICAPCD air monitoring activities is not complete.

IMP9: There are potential siting issues at the Calexico Ethel site.

IMP10: ICAPCD is not adequately reviewing and editing data.

MEN1: MeCAPCD staff was not familiar with the CARB QMP or instrument SOPs.

MEN2: MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2006 but 

is erroneously listed as its own PQAO in AQS.

MEN3 (Positive): MeCAQMD stations were well-maintained. Staff and the manager were 

professional and helpful, and very knowledgeable about the county and potential pollution 

sources. The station operator was proactive about troubleshooting instrument issues. 

MEN4: One-point QC checks (flow verifications) for PM10 and PM2.5 are not consistently 

performed by MeCAPCD site operators.
150

MEN5: MeCAPCD logbook entries are not consistently made and are not always in the most 

defensible form. Handwritten notes are occasionally illegible due to water (rain) marks.

MEN6: Residence time calculations were not available at the Ukiah Gobbi site.

MEN7: Trees at the Ukiah Gobbi and Library sites should be evaluated against siting 

requirements.

MEN8: The internal shelter thermostat is not operating correctly at the Ukiah Gobbi site and the 

issue has not been addressed to provide defensible data.

MEN9: MeCAPCD has no system for tracking and controlling station and instrument logbooks.

MEN10: MeCAQMD should have formalized training requirements for all air monitoring staff.

MEN11: MeCAQMD does not provide CARB AQAS with datasets that have been fully quality 

assured and ready for upload to AQS.

MEN12: MeCAQMD does not use a formal corrective action system.

SJV1 (Positive): In general, the SJVAPCD monitoring program is robust and the agency staff 

and managers involved in the program are committed to producing defensible data of known 

quality.

SJV2: SJVAPCD does not have updated quality system documentation for all activities.

SJV3: SJVAPCD has experienced significant data losses at required monitoring sites, including 

sites critical for demonstrating compliance with the NAAQS.

SJV4: SJVAPCD has initiated network modifications for several required sites without seeking 

EPA approval required by 40 CFR 58.14.

SJV5: The residence time of flow between the inlet and each instrument was not posted at every 

SJVAPCD site.

SJV6: Some SJVAPCD site logbooks lacked specific information about the date or type of 

maintenance performed on an instrument.

SJV7: There is no documentation of management review of station logbooks and other site 

activities for SJVAPCD operated sites.

SJV8: SJVAPCD site operators do not have a quick visual way to identify changes in instrument 

performance or QC checks that would indicate instrument issues, nor do they have the ability to 

remotely check on data or site operations.

SJV9: SJVAPCD experiences data loss due to instrument malfunction.
151

SJV10: It is unclear whether SJVAPCD is using appropriate criteria to invalidate or flag PM10

data.

SJV11: SJVAPCD experiences significant resource inefficiencies for staff and management as 

the current data management system relies solely on manual inputs.

SJV12: SVJAPCD does not have a formal corrective action process in place.
152

APPENDIX B: CARB ORGANIZATIONAL CHARTS

Organization of CARB’s Divisions and Offices
153

Organizational Chart for CARB’s Monitoring and Laboratory Division (MLD)
154

Organizational Chart for CARB’s Planning and Technical Support Division
155

APPENDIX C: CARB DATA VALIDATION DOCUMENTS PROVIDED BY DATA 

VALIDATION STAFF14

14 CARB’s QAPP is the formal document that outlines data validation procedures for CARB.
156
157
158
159
160