S)~41~

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

REGION IX

~ PRO~ 75 Hawthorne Street

San Francisco, CA 94105.3901

AUG 312016

Mr. Richard Corey

Executive Officer

California Air Resources Board

Transportation and Toxic Division

Post Office Box 2815

Sacramento, California 95812

Dear Mr. Corey:

I am writing to transmit the findings from the U.S. Environmental Protection Agency’s (EPA’s)

2015 Technical Systems Audit (TSA) of the California Air Resources Board’s (CARB’s)

ambient air monitoring program. This audit was performed in accordance with 40 CFR Part 58

Appendix A, Section 2.5, which requires EPA to conduct TSAs of primary quality assurance

organizations (PQAO5) every three years.

After EPA’s previous TSA in 2011, CARB made substantial improvements to strengthen data

quality within CARB and the CARB PQAO. The addition of new staff, restructuring to enhance

independent oversight, and other changes within CARB have improved the quality and

defensibility of air quality data collected by the entire CARB PQAO. EPA commends you and

your staff for these efforts.

EPA reviewed CARB’s ambient air monitoring program, including network management, field

operations, laboratory operations, quality assurance, and data management procedures. In

addition to visiting laboratories, offices, and air monitoring stations run by CARB staff, EPA

also reviewed the air monitoring programs at four local districts within CARB’s PQAO:

Antelope Valley Air Quality Management District (AVAQMD), Mojave Desert Air Quality

Management District (MDAQMD), North Coast Unified Air Quality Management District

(NCUAQMD), and San Luis Obispo County Air Pollution Control District (SLOCAPCD).

Generally, we found that the CARB PQAO’s air monitoring program is robust and meets EPA

requirements. As with all TSAs, we also identified areas that can be strengthened and have

accordingly listed these findings and suggested actions to address them in the report. Your

agency has been very responsive to the audit findings and has already begun working on

corrective actions. The formal corrective action response is due to EPA within 60 days of receipt

of the final TSA report (enclosed).

I want to thank you and your staff for your full cooperation and participation during the audit.

The results of this audit significantly contribute to EPA’s overall understanding of ambient

monitoring in the State of California.

Pnrned on Reycled Paper
Please contact me if you have questions, or your staff may contact Anita Lee at (415) 972-3958

or Gwen Yoshimura at (415) 947-4134.

Sincerely,

~

Elizab~J.

Actin: • irector, Air Division

Enclosure

cc: Brad Poiriez, AVAQMD and MDAQMD

Brian Wilson, NCUAQMD

Larry Allen, SLOCAPCD

cc (via email): Michael Benjamin, CARB

Mike Miguel, CARB

Patrick Rainey, CARB

Michael Werst, CARB

Kenneth Stroud, CARB

Ranjit Bhullar, CARB

Ravi Ramalingam, CARB

Sylvia Vanderspek, CARB

Chris Collins, MDAQMD

Wendy Caruso, NCUAQMD

Karl Tupper, SLOCAPCD

Jaime Contreras, SLOCAPCD
 

System Audit of the Ambient Monitoring Program:

California Air Resources Board

April - August, 2015

Conducted by:

Air Quality Analysis Office Quality Assurance Office 

Air Division Environmental Management 

Services Division

US EPA Region 9

75 Hawthorne Street

San Francisco, California 94105

Final Report: August 2016
Final Report – August 2016

Contents

GLOSSARY OF ACRONYMS........................................................................................... i

EXECUTIVE SUMMARY ................................................................................................ 1

TSA ACTIVITIES.............................................................................................................. 5

OVERVIEW OF THE STATE AIR MONITORING PROGRAM ................................. 10

OVERVIEW OF THE LOCAL DISTRICT AIR MONITORING PROGRAMS:

ANTELOPE VALLEY & MOJAVE DESERT AQMD .............................................. 29

NORTH COAST UNIFIED AQMD ............................................................................ 34

SAN LUIS OBISPO COUTY APCD........................................................................... 37

FINDINGS........................................................................................................................ 41

APPENDIX A: SUMMARY OF FINDINGS ................................................................ 187

APPENDIX B: CARB ORGANIZATIONAL CHARTS .............................................. 195

APPENDIX C: MOJAVE DESERT AQMD ORGANIZATIONAL CHARTS & 

MONITORING STATIONS MAP................................................................................. 200

APPENDIX D: NORTH COAST UNIFIED AQMD ORGANIZATIONAL CHART . 203

APPENDIX E: SAN LUIS OBISPO COUNTY APCD ORGANIZATIONAL CHART

......................................................................................................................................... 204

APPENDIX F: MATRIX OF 2011 TSA FINDINGS .................................................... 205
Final Report - August 2016

i

GLOSSARY OF ACRONYMS

ACAPCD ...............................Amador County Air Pollution Control District

ADAM ...................................Aerometric Data and Management

AMNS....................................Air Monitoring Northern Section

AMSS.....................................Air Monitoring Southern Section

APCD.....................................Air Pollution Control District

AQAS.....................................Air Quality Analysis Section

AQDA....................................Air Quality Data Action

AQDAS..................................Air Quality Data Acquisition System

AQDB ....................................Air Quality Data Branch

AQMD ...................................Air Quality Management District

AQMIS...................................Air Quality and Meteorological Information System

AQPSD ..................................Air Quality Planning and Science Division

AQS........................................Air Quality System

AQSB.....................................Air Quality Surveillance Branch

AVAQMD..............................Antelope Valley Air Quality Management District

AV/MD AQMD.....................Antelope Valley and Mojave Desert Air Quality Management Districts

BAAQMD..............................Bay Area Air Quality Management District

BAM...................................... Beta Attenuation Mass Monitor (continuous PM2.5)

BCAQMD..............................Butte County Air Quality Management District

CAA.......................................Clean Air Act

CaCAPCD..............................Calaveras County Air Pollution Control District

CAMNAT ..............................California Monitoring Network Assessment Tool

CAN.......................................Corrective Action Notification

CARB.....................................California Air Resources Board

CBSA.....................................Core-based Statistical Area

CFR........................................Code of Federal Regulations

CO..........................................Carbon Monoxide

CoC........................................ Chain of Custody

CoCAPCD..............................Colusa County Air Pollution Control District

CPAQAB ...............................Consumer Products and Air Quality Assessment Branch

CSA........................................Combined Statistical Area

DMS.......................................Data Management System

DQO.......................................Data Quality Objective

EDCAQMD........ ...................El Dorado County Air Quality Management District

EKAPCD................................Eastern Kern Air Pollution Control District

EPA........................................Environmental Protection Agency

FEM....................................... Federal Equivalent Method

FRAQMD.............................. Feather River Air Quality Management District

FRM....................................... Federal Reference Method

FTP.........................................File Transfer Protocol
Final Report - August 2016

ii

GBUAPCD............................ Great Basin Unified Air Pollution Control District

GCAPCD............................... Glenn County Air Quality Management District

ICAPCD.................................Imperial County APCD

ILS..........................................Inorganic Laboratory Section

ISO .........................................International Organization for Standardization

IZS..........................................Internal Zero/Span

LC.......................................... Local Conditions

LCAPCD................................Lassen County Air Pollution Control District

LCAQMD ..............................Lake County Air Quality Management District

LIMS......................................Laboratory Information Management System

LSS.........................................Laboratory Support Section

MaCAQMD ...........................Mariposa County Air Quality Management District

MBUAPCD............................Monterey Bay Unified Air Pollution Control District

MCAPCD...............................Modoc County Air Pollution Control District

MDAQMD.............................Mojave Desert Air Quality Management District

MeCAQMD........................... Mendocino County AQMD

MFE....................................... Mass Flow Element

MLD.......................................Monitoring and Laboratory Division

MSA.......................................Metropolitan Statistical Area

NAAQS..................................National Ambient Air Quality Standard

NCore.....................................National Core multi-pollutant monitoring stations 

NCUAQMD...........................North Coast Unified Air Quality Management District

NELAC.................................. National Environmental Laboratory Accreditation Conference

NIST.......................................National Institute for Standards and Technology

NLB........................................Northern Laboratory Branch

NO..........................................Nitric Oxide

NO2 ........................................Nitrogen Dioxide

NOx ........................................Nitrogen Oxides

NPAP..................................... National Performance Audit Program

NPEP......................................National Performance Evaluation Program

NSAQMD.............................. Northern Sierra Air Quality Management District

NSCAQMD............................Northern Sonoma County Air Pollution Control District

O3........................................... Ozone

OIS .........................................Office of Information Services

OLS........................................Organics Laboratory Section

OMB...................................... Office of Management and Budget

OSS........................................ Operations and Data Support Section

PAMS.....................................Photochemical Assessment Monitoring Stations

Pb........................................... Lead

PCAPCD................................Placer County Air Pollution Control District

PM..........................................Particulate matter

PM2.5...................................... Particulate matter 2.5 microns or less in aerodynamic diameter

PM10....................................... Particulate matter 10 microns or less in aerodynamic diameter
Final Report - August 2016

iii

PMOD....................................Planning, Monitoring and Outreach Division

POC........................................Parameter Occurrence Code

PQAO.....................................Primary Quality Assurance Organization

PSD ........................................Prevention of Significant Deterioration

QA..........................................Quality Assurance

QAPP..................................... Quality Assurance Project Plan

QAS........................................Quality Assurance Section

QC..........................................Quality Control

QCM ......................................Quality Control Manual

QMB...................................... Quality Management Branch

QMP.......................................Quality Management Plan

QMS.......................................Quality Management Section

R&R.......................................Roles and Responsibilities

RH..........................................Relative Humidity

SAS ........................................Special Analysis Section

SASS......................................Spiral Aerosol Speciation Sampler

SBCAPCD............................. Santa Barbara County Air Pollution Control District

SCAQMD.............................. South Coast Air Quality Management District

SDCAPCD............................ San Diego County Air Pollution Control District

ShCAQMD............................ Shasta County Air Quality Management District

SiCAPCD...............................Siskiyou County Air Pollution Control District

SJVAPCD.............................. San Joaquin Valley APCD

SLAMS.................................. State or Local Air Monitoring Station

SLOCAPCD...........................San Luis Obispo County Air Pollution Control District

SMAQMD..............................Sacramento Metro Air Quality Management District

SO2......................................... Sulfur Dioxide

SOP........................................ Standard Operating Procedure

SPM........................................Special Purpose Monitor

SQL........................................Structured Query Language

STP.........................................Standard Temperature and Pressure

TeCAPCD..............................Tehama County Air Pollution Control District

TSA........................................Technical System Audit

TSP.........................................Total Suspended Particulate

TTP ........................................Through The Probe

TuCAPCD..............................Tuolumne County Air Pollution Control District

VCAPCD............................... Ventura County Air Pollution District

VOC.......................................Volatile Organic Compound

VSCC.....................................Very Sharp Cut Cyclones

WINS .....................................Well Impactor Ninety-Six

Y-SAQMD.............................Yolo-Solano Air Quality Management District

ZPS.........................................Zero/Precision/Span
Final Report - August 2016

1

EXECUTIVE SUMMARY

This document is a report of the findings made by the U.S. Environmental Protection Agency, Region 9 

(EPA) while conducting a Technical Systems Audit (TSA) on the air monitoring program of the 

California Air Resources Board (CARB). A TSA is an on-site review and inspection of a state or local 

ambient air monitoring program to assess its compliance with established regulations governing the 

collection, analysis, validation, and reporting of ambient air quality data. This TSA meets the 

requirements for EPA audits of CARB’s monitoring organization as described in 40 CFR Part 58, 

Appendix A, Section 2.5.

EPA notes that CARB has made substantial changes since the previous TSA to strengthen its quality 

system. These include:

 An entirely new Quality Management Section was established within the Quality Management 

Branch (QMB) that includes positions for Primary Quality Assurance Organization (PQAO) 

liaisons. Each district within the CARB PQAO is assigned to a specific liaison to improve 

coordination and dialogue between the districts and CARB. CARB has been allocating budget, 

time and personnel to allow the liaisons to visit their districts in person at least once a year. 

 CARB held three PQAO trainings, offering them in northern and southern California, with 

participation from the CARB PQAO districts as well as the other California PQAOs. 

 Roles and Responsibilities (R&R) documents have been created between CARB and each of the 

twenty-one CARB PQAO districts that conduct regulatory monitoring. These documents 

describe districts’ and CARB’s responsibilities in order for the PQAO to function successfully. 

 A new data management system (DMS) has been installed, allowing for improved data flow 

from the field, easier data visualization and editing. 

 A finalized data validation standard operating procedure (SOP) outlines the steps involved in 

data validation, including what the different validation levels are, who does them, and what they 

entail. 

 A Quality Assurance (QA) webpage was created that houses information about the CARB QMB, 

the PQAO structure and contacts, quality management documents, such as a quality management 

document repository, QA activities with links to Corrective Action Notifications (CANs) and Air 

Quality Data Actions (AQDAs), and services and resources, including information on the CARB 

standards laboratory.

 CARB has been working with the districts and EPA to create a California Monitoring Network 

Assessment Tool (CAMNAT), to assist in development of annual monitoring network plans and 

network assessments.

CARB has instituted these changes, many of which are noted as positive findings in this report, while 

managing site relocations, particulate matter (PM) filter rejection challenges, changes in instrumentation 

(including increased use of continuous PM10 and PM2.5 instruments), as well as carrying on the daily air 

monitoring system operations. While some of these changes have been fully implemented and are 

operating smoothly, the majority are relatively new. As a result, a significant portion of the 2015 TSA 

findings emphasize implementation and checks to ensure that new systems are operating as desired. 

EPA also notes that CARB implemented certain changes between the time the findings contained in this 

report were written and the issuance of this final report. These changes include: restructuring the 

Standards Laboratory; sending regulatory Pb-TSP filters to an external lab; creating a quality 

assurance/quality control (QA/QC) oversight position for the Northern Laboratory; and issuing a 
Final Report - August 2016

2

technical bulletin addressing minimum values reporting. Other actions, such as development of a user 

guide/SOP for each DMS and CARBLogger, and offering a CAN session during the next PQAO 

training, are under development. As the TSA report is created to document what was observed during 

the audit itself, findings that call for improvements related to these items are still reflected in this report. 

We look forward to final resolutions of these findings, and commend CARB for the work already 

underway.

CARB, an organization under the umbrella of the California Environmental Protection Agency, is the 

governmental agency delegated under state law with the authority and responsibility for collecting 

ambient air quality data as directed by the Clean Air Act (CAA) of 1977 and CAA Amendments of 

1990. CARB and local air pollution control districts (hereafter referred to as “local districts” or 

“districts”) operate ambient monitoring stations throughout the State of California. CARB is designated 

as the PQAO for the entire state with the exception of the ambient air monitoring programs of the Bay 

Area Air Quality Management District (BAAQMD), the South Coast Air Quality Management District 

(SCAQMD), and the San Diego County Air Pollution Control District (SDCAQMD). Many of the 

smallest local districts do not have active air monitoring programs and rely solely on CARB for the 

operation of monitoring stations within their jurisdictions.

The TSA was conducted by EPA staff from April to September, 2015. The audit evaluates all air 

monitoring activities since the previous EPA TSA, which was conducted during the summer of 2011. 

The audit team interviewed management and staff on specific aspects of the ambient air monitoring 

program including network design, field operations, laboratory operations, data handling, quality 

assurance and quality control procedures. The audit team also inspected some of the CARB-operated 

monitoring sites. The site inspections consisted of an interview with the site operator when possible, 

review of station and instrument logbooks, and evaluation of the station siting with respect to EPA 

requirements for probe siting (40 CFR 58, Appendix E). The laboratory inspection included a review of 

the standards and calibrations labs, the PM program for mass determinations, and laboratory analysis for 

Pb. 

Since CARB oversees the quality of data collected by local districts within its PQAO, EPA also 

reviewed field operations, data management and quality assurance activities at local districts. For this 

TSA, it was not possible to evaluate all of the 21 local districts within the PQAO that collect ambient air 

quality data; the audit team reviewed operations at four local districts: Antelope Valley Air Quality 

Management District (AVAQMD), Mojave Desert Air Quality Management District (MDAQMD), 

North Coast Unified Air Quality Management District (NCUAQMD), and San Luis Obispo County Air 

Pollution Control District (SLOCAPCD). The local districts included in the PQAO have their own 

organizational structures and these vary depending on the size of the local district program. The four 

agencies visited are geographically diverse, located in northern, central, and southern California. 

The TSA is part of the oversight system by which EPA ensures that data collected by state, local, and 

tribal agencies meet certain minimum data quality objectives. Other assessments, such as network 

reviews and performance evaluations, are also used to collect information on the quality of ambient air 

monitoring data. These assessments enable agencies to identify and correct those program elements that

may be adversely affecting the quality of ambient air data. The results of the TSA are summarized here 

and fully described in this report, along with recommended actions to address the findings. The specific 

actions to be taken by CARB will be determined through negotiations between EPA and CARB and will 

be documented in a corrective action plan prepared by CARB. 
Final Report - August 2016

3

EPA would like to thank all the staff and management of CARB and the local districts for their support 

and cooperation during the audit.

A. Program Strengths:

 CARB has put into place multiple systems and infrastructure to promote better communication 

between CARB and the districts within the PQAO. The most notable of these is the staffing of a 

Quality Management Section, which has been critical in implementation of the PQAO.

 CARB’s increased interactions with the districts, impressive quality assurance website, Quality 

Management Plan (QMP), PQAO trainings, and Roles and Responsibilities documents are 

resulting in more consistency across the CARB PQAO and improved understanding of 

requirements and processes.

 CARB has extensive experience and expertise in ambient air monitoring.

 Local districts within the PQAO are committed to addressing air quality concerns in their areas 

and see monitoring as the means by which to assess air quality. Some local districts have created 

a strong culture of continually working to improve their current practices and tools, and take the 

initiative to identify issues and improvements early.

 CARB and the local districts that participated in this TSA are dedicated to collecting credible 

and defensible air quality data.

B. Major Program Findings:

 Districts within the PQAO need additional training and technical support. [Findings G1, 

AV/MD5, NCU3]

 Template forms used in the field and in the laboratories are not being properly validated or 

reviewed. During EPA’s brief inspection of forms as part of the TSA, EPA found errors in forms 

used in the field and by the standards laboratory [Findings FO1, SL4, PM7]

 CARB and district QAPP/SOPs are out of date or not being followed. [Findings FO5, PM5, 

AV/MD4, NCU2, SLO2, SLO3]

 The quality of zero air is not being documented by CARB or the PQAO districts. [Findings FO4,

SL7, AVMD11, NCU13]

 Air Quality System (AQS) coding is not used consistently. [Findings DM1, DM2, DM3]

 The process of implementation and follow-up for resolving AQDAs and CANs and internal 

corrective actions is incomplete. [Findings QA1, QA2, AV/MD6, NCU14, SLO1]

 The CARB Standards Laboratory QA System has a number of deficiencies. [Findings SL1, SL3]

 The Standards Laboratory corrective action process is not consistent with International 

Organization for Standardization (ISO) or National Institute of Standards and Technology 

(NIST) requirements. [Finding SL2] 

 Calibration laboratory staff have not had sufficient training. [Finding SL5]

 The CARB PM Laboratory is not tracking required temperature and humidity statistical 

information in accordance with EPA’s specified methodology. [Finding PM1]

 One district laboratory did not track temperature and humidity. [Finding AV/MD19]

 Several districts do not have complete and/or independent data review and validation procedures. 

[Findings AV/MD1, NCU15, SLO5]

 The CARB Pb Laboratory was not meeting EPA requirements. [Finding Pb1]

 The number of staff and managers and the amount of their time allocated for air monitoring were 

inadequate for the current network and activities at one district. [Finding AV/MD1]

 One district was using less stringent data quality performance levels for gaseous pollutants than 

those in EPA regulation and guidance. [Finding NCU12]
Final Report - August 2016

4

The individual findings are reported in the topic sections of this document and are summarized in 

Appendix A. The major findings are those that are over-arching, impact regulatory decisions, or data 

quality. It has been EPA Region 9’s experience that all monitoring programs have areas that could be 

improved. Unless explicitly stated, EPA’s findings are not cause for data invalidation. Overall, the 

CARB PQAO has a robust air monitoring program that produces reliable, defensible, valid data for 

regulatory use. 
Final Report - August 2016

5

TSA ACTIVITIES

In the spring and summer of 2015, EPA Region 9 conducted a TSA of the ambient monitoring program 

operated by and overseen by CARB. This report reflects the status of the program at the time of the 

audit. EPA staff interviewed management and staff in three branches of CARB Monitoring and 

Laboratory Division (MLD) and one branch of the Air Quality Planning and Science Division 

(AQPSD). The TSA covered the areas of Air Monitoring Network Management, Field Operations, 

Laboratory Operations, Data and Data Management, and Quality Assurance. In addition, the EPA staff 

reviewed these same areas as implemented by four local districts: Antelope Valley & Mojave Desert Air 

Quality Management District (AQMD), North Coast Unified AQMD, and San Luis Obispo County Air 

Pollution Control District (APCD). CARB staff shadowed EPA during portions of this TSA.

CARB managers and staff were very accommodating to the EPA audit team, making themselves and 

their staff available for many interviews, procedural reviews and monitoring site visits. Branch Chiefs 

interviewed were:

Ken Stroud – Chief, Air Quality Surveillance Branch (AQSB), MLD

Michael Miguel – Chief, QMB, MLD

Michael Werst – Chief, Northern Laboratory Branch (NLB), MLD

Ravi Ramalingam – Chief, Consumer Products and Air Quality Assessment Branch (CPAQAB), 

AQPSD

Many other individual section managers and staff were interviewed in Sacramento and in the field. We 

appreciate the fact that CARB gave the EPA audit team access to all key personnel involved in the 

collection and quality assurance of ambient air quality data.

The EPA regional staff members conducting the TSA were Gwen Yoshimura, Dena Vallano, and Elfego 

Felix of EPA Region 9’s Air Quality Analysis Office, and Mathew Plate and Joseph Eidelberg of the 

EPA Region 9 Quality Assurance Office. In addition to the EPA Audit Team, Meredith Kurpius and 

Eugenia McNaughton, managers of EPA Region 9's Air Quality Analysis Office and Quality Assurance 

Office, respectively, attended the opening and closing meeting representing EPA management.

The TSA began with a general meeting with CARB managers and staff on February 24, 2015 at the 

MLD office in Sacramento, CA, with field activities and interviews commencing in April and 

continuing through August, 2015. The TSA covered the following program areas:

 General / Quality Management.

- Program organization.

- Facilities.

- Independent QA and quality control (QC).

- Planning documents (including QMP, Quality Assurance Project Plans (QAPPs), & 

SOPs).

- General documentation policies.

- Training.

- Corrective action.

- Quality improvement.

- External performance audits.

 Network Management / Field Operations.

- Network design.

- Changes to the network since the last audit.
Final Report - August 2016

6

- Proposed changes to the network.

- Field support.

 Laboratory Operations: Standards and Calibrations, PM, and Pb

- Routine operations.

- Quality control.

- Laboratory preventive maintenance.

- Laboratory record keeping.

- Laboratory data acquisition and handling.

- Specific pollutants: PM10, PM2.5, and Pb.

 Data and Data Management.

- Data handling.

- Software documentation.

- Data validation and correction.

- Data processing.

- Internal reporting.

- External reporting

As part of the TSA, EPA tracked supporting documentation for data points/sets from calendar years 

2013 and 2014.

This report is divided the following sections: 

 Executive Summary – describes the purpose of the TSA and summarizes the major findings.

 TSA Activities – outlines the timing of this TSA and the programs that were covered.

 Overview of Air Monitoring Program – describes the Air Monitoring Programs of CARB and the 

local districts that were part of this TSA.

 Findings – collection of findings and suggested actions to address findings that include details 

associated with findings.

 Appendix A – list of findings.

 Appendix B – CARB organizational charts.

 Appendix C, D, and E – District organizational charts.

 Appendix F – matrix of findings from the 2011 TSA.

The Findings section includes positive findings and best practice recommendations along with those that 

require corrective action. The findings are grouped by program area. Suggested actions to address

findings are provided for each one that requires corrective action to give some indication of EPA’s

expectations as to how they may be addressed. If the air agency has other approaches or alternatives to 

address the concerns identified, EPA will consider them, provided the corrective action adequately 

addresses the finding. Appendix A is a summarized list of findings. Within each program area, findings, 

are ordered according to priority. Higher priority findings are those that: (1) are over-arching issues, (2) 

have a greater impact on regulatory decisions, and/or (3) have a greater effect on data quality. EPA has 

also included best practice recommendations for some program areas. Regardless of priority, all 

findings, excluding positive findings and recommendations, should be addressed in a corrective action 

report.

Appendix F contains a matrix summarizing the findings from the previous TSA and their status. The 

improvements noted in this TSA directly reflect changes made as a part of the corrective action process. 

EPA commends CARB on the work done to address these numerous, wide-reaching findings. We will 

continue to work with CARB to close out the few remaining open findings from the 2011 TSA, while 

also moving forward to address the new findings from this report.
Final Report - August 2016

7

Network Management

EPA interviewed Gayle Sweigert and Craig Anderson (manager and air pollution specialist within the 

Air Quality Analysis Section, respectively), as well as Ravi Ramalingam and Reginald Smith (manager, 

Operation and Data Support Section), and reviewed the CARB Annual Monitoring Network Plan and 

Five-year Network Assessment as part of this TSA. CARB submitted a 2015 Five-Year Network 

Assessment and Network Plan as required by 40 CFR 58.10. Both documents address a portion of the 

monitoring network of California; they present information from the smaller agencies that do not 

produce their own reports as well as CARB sites. As part of the Annual Network Plan process, EPA 

reviewed all Annual Network Plans associated with the CARB PQAO and provided detailed comments 

as part of that process. 

Field Operations

The CARB site technicians interviewed were Tony Simoni, Glen Jennings, Carl Camp, John Latham, 

Phil Wagner, Adolfo Garcia, Steven Theantanoo, David Cardiel, Debbie Henson, Ralph Robles, and 

Kathy Edwards. In general, the site technicians demonstrated a thorough knowledge of the monitoring 

equipment for which they were responsible.

EPA visited the following nine CARB-operated monitoring stations:

 Yuba City in Sutter County (Air Quality System (AQS) ID 06-101-0003),

 Jackson-Clinton Road in Amador County (AQS ID 06-005-0002),

 San Andreas-Gold Strike in Calaveras County (AQS ID 06-009-0001),

 Sonora in Tuolumne County (AQS ID 06-109-0005),

 South Higuera in San Luis Obispo County (AQS ID 06-079-2006),

 Paso Robles in San Luis Obispo County (AQS ID 06-079-0005),

 Bakersfield-California in Kern County (AQS ID 06-029-0014),

 Oildale in Kern County (AQS ID 06-029-0232), and

 Mojave-Poole in Los Angeles County (AQS ID 06-037-9033).

Evaluation at these sites included inspection of the inlet manifolds, examination of station and 

instrument log books, and an evaluation as to whether appropriate QC checks and QA audits were being 

performed. 

Data Management

This section covers data management for criteria pollutants (O3, CO, NO2, SO2, Pb, PM2.5, and PM10). 

The following managers/staff who have data management responsibilities were interviewed:

 Kenneth Stroud, Reginald Smith - continuous data for CARB-operated sites. (Reginald Smith’s 

staff upload the data generated by Kenneth Stroud’s branch.)

 Gayle Sweigert - CARB AQS data entry for district-operated sites, data certification. (Dwight 

Oda, a staff member within Gayle Sweigert’s group, uploads the data.)

 Michael Werst, Brenda Saldana, Samantha Scola – PM and Pb filter data for CARB- and districtoperated sites using the CARB laboratory. (Samantha Scola uploads the filter data generated by 

the laboratory.)
Final Report - August 2016

8

Quality Assurance/Quality Control

EPA interviewed the QMB Chief, Quality Management Section Manager, Quality Assurance Section 

(QAS) Manager, AQSB Chief, Air Monitoring North Section Manager, Air Monitoring South Section 

Manager, Operations and Data Support Section Manager, and staff within each section. 

Managers and staff interviewed included, but was not limited, to:

 Mike Miguel, QMB Chief

 Patrick Rainey, Quality Management Section Manager

 Ranjit Bhullar, QAS Manager

 Kenneth Stroud, AQSB Chief

 Joe Guerrero, Air Monitoring North Section Manager

 Fernando Amador, Air Monitoring South Section Manager

 Reginald Smith, Operation and Data Support Section Manager

CARB’s quality management system met the basic EPA requirements. CARB had a QMP that was 

approved in July 2013. CARB’s core program conformed to or exceeded the method quality objectives 

systematically developed by EPA for criteria pollutants. EPA and CARB perform national performance 

and technical evaluations of the monitoring network.

Laboratory Operations

Standards and Calibrations

EPA interviewed staff (Ranjit Bhullar, Jerry Freeman, and Robert Russell) and reviewed procedures for 

the CARB Standards Laboratory. EPA focused on review of the flow calibration program and data 

tracking. 

Particulate Matter Laboratory

EPA visited two gravimetric PM laboratories and interviewed the following staff as part of the audit:

Michael Werst – Chief, Northern Laboratory Branch

Brenda Saldana – Inorganic Laboratory Section Manager

Ranjit Ahuja – PM10 Analyst

Nial Maloney – PM2.5 Analyst

Alicia Adams – PM2.5 Analyst

Jeff Wright – Laboratory Support Section Manager

Samantha Scola – Air Pollution Specialist

Dan Tackett – Staff Air Pollution Specialist

While the laboratory facilities are primarily used to process and weigh PM10 and PM2.5 filters, back-up 

PM10 and PM2.5 balances are maintained at both facilities so that filters may be weighed in either 

laboratory if issues arise at the primary laboratory location. Both of the PM laboratories were wellmaintained, neat, and well-organized. Generally, the PM10 and PM2.5 laboratory measurements were

performed with a high level of technical expertise.

Pb Laboratory

On May 4 and 5, 2015, EPA interviewed Betsy Ronsse, the designated analyst who performed sample 

preparation and analysis of lead-TSP. Ms. Ronsse was responsible for performing acid digestion of TSP 
Final Report - August 2016

9

filters using a heated ultrasonic water bath with subsequent analysis of digestates using a graphite 

furnace atomic absorption instrument. Ms. Ronsse was responsible for ensuring compliance with quality 

control requirements and preparing reports.

EPA reviewed the practices for conformance to the Laboratory SOP and all EPA approved TSP-Pb 

methods. EPA reviewed each step of the method to determine if it was substantively the same practice 

described by an already approved EPA method. The CARB method was not considered substantively 

the same as any one approved EPA method and some steps in the CARB method were not analogous to 

any step in approved EPA methods. 

As part of its 2015 Annual Monitoring Network Plan, submitted July 2015, CARB requested EPA 

approval to close the one Pb-TSP monitor in its network. EPA approved this closure on November 13, 

2015. As CARB no longer analyzes Pb-TSP for regulatory purposes, this laboratory operation is no 

longer active.

Review of Agencies within the CARB PQAO

Since CARB oversees the QA of data collected by local districts1 within the PQAO, EPA also reviewed 

field operations, data management and quality assurance activities at local districts. As it was not 

possible to evaluate all of the 21 local districts within the PQAO that collect ambient air quality data, the 

audit team reviewed operations at four local districts: AVAQMD, MDAQMD, NCUAQMD, and 

SLOCAPCD.

Each of the local districts included in the CARB PQAO has its unique organizational structure. All four 

districts were small in terms of staff but the geographic area and number of monitors varied for each 

district: NCUAQMD operated only a few sites over a large geographic area; SLOCAPCD operated more 

sites over a smaller geographic area; and AVAQMD and MDAQMD, which shared staff and facilities, 

operated sites and ran their own PM weigh lab. The four agencies are geographically diverse, located in 

northern, central, and southern California.

 

1 According to the California State and Local Air Monitoring Network Plan (2015), prepared by CARB, 21 local Districts 

operate air monitoring stations in the CARB PQAO.
Final Report - August 2016

10

OVERVIEW OF THE STATE AIR MONITORING PROGRAM

State and Local Monitoring Agencies within the State of California

CARB, an organization under the umbrella of the California Environmental Protection Agency, is the 

governmental agency delegated under state law with the authority and responsibility for collecting 

ambient air quality data as directed by the CAA of 1977 and CAA Amendments of 1990. Specifically, 

CAA Section 110(a)(2)(B)(i) directs the State to “provide for establishment and operation of appropriate 

devices, methods, systems, and procedures necessary to...(i) monitor, compile, and analyze data on 

ambient air quality...” 

At the time of the audit, there were 35 local air pollution control districts in the state of California (Table 

1). Three of these local air districts, BAAQMD, SCAQMD, and SDCAPCD, are PQAOs. Twenty-one of 

the remaining air districts and CARB, comprising the CARB PQAO, collect ambient air monitoring data

for National Ambient Air Quality Standard (NAAQS) compliance. 

A PQAO is a monitoring organization or a coordinated aggregation of such organizations that is 

responsible for a set of stations that monitors the same pollutant and for which data quality assessments 

can logically be pooled because they have similar quality systems in place. Specifically, 40 CFR Part 58

Appendix A Section 3 requires that each ambient air monitoring PQAO conform to certain quality 

management practices. These include:

 Having a documented quality system that meets EPA requirements for QMPs and QAPPs.

 Having a quality management function that is independent of air monitoring operations.

 Developing or adopting data quality objectives (DQOs), or equivalent systematic planning 

procedures, for all monitoring programs.

 Participating in National Performance Evaluation Programs, which consist of performance audits 

used to independently determine program adequacy, national monitoring network performance, 

and national consistency.

 Undergoing TSAs by EPA at a frequency of every three years or less as needed.

 Using certified reference materials to standardize monitoring equipment.

EPA views these quality management system components as indispensable to maintain a credible 

monitoring program. Insufficient quality management and control has been cited as rationale to support 

legal challenges to NAAQS designation decisions. 

CARB oversees the quality assurance of data collected by local districts within the PQAO. Although 

both CARB and local districts operate ambient monitoring stations throughout the state, responsibility 

for ambient air monitoring ultimately rests with CARB. Further, it is the responsibility of CARB to 

provide QA oversight to ensure that data quality within the PQAO meets code of federal regulations 

(CFR) requirements and conforms to quality standards approved in the QAPP.
Final Report - August 2016

11

Table 1. List of Air Pollution Control Agencies in California

Air Pollution Control District Ambient Air Monitoring by 

District for NAAQS Compliance*

PQAO

Amador County (AC)APCD N CARB

Antelope Valley (AV)AQMD Y CARB

Bay Area (BA)AQMD Y BAAQMD

Butte County (BC)AQMD N CARB

Calaveras County (CaC)APCD N CARB

CARB Y CARB

Colusa County (CoC)APCD N CARB

El Dorado County (EDC)AQMD N CARB

Feather River (FR)AQMD N CARB

Glenn County (GC)APCD N CARB

Great Basin Unified (GBU)APCD Y CARB

Imperial County (IC)APCD Y CARB

Eastern Kern (EK)APCD Y CARB

Lake County (LC)AQMD Y CARB

Lassen County (LC)APCD N CARB

Mariposa County (MaC)AQMD N CARB

Mendocino County (MeC)AQMD Y CARB

Modoc County (MC)APCD N CARB

Mojave Desert (MD)AQMD Y CARB

Monterey Bay Unified (MBU)APCD Y CARB

North Coast Unified (NCU)AQMD Y CARB

Northern Sierra (NS)AQMD Y CARB

Northern Sonoma County 

(NSC)APCD

Y CARB

Placer County (PC)APCD Y CARB

Sacramento Metro (SM)AQMD Y CARB

San Diego County (SDC)APCD Y SDCAPCD

San Joaquin Valley (SJV)APCD Y CARB

San Luis Obispo County 

(SLOC)APCD

Y CARB

Santa Barbara County (SBC)APCD Y CARB

Shasta County (ShC)AQMD Y CARB

Siskiyou County (SiC)APCD Y CARB

South Coast (SC)AQMD Y SCAQMD

Tehama County (TeC)APCD Y CARB

Tuolumne County (TuC)APCD N CARB

Ventura County (VC)APCD Y CARB

Yolo-Solano (Y-S)AQMD Y CARB

*In some cases CARB and local districts share ambient air monitoring responsibilities for areas and sites within a local 

district’s jurisdiction.

The ambient air monitoring program in the State of California encompasses many air quality assessment 

activities, including collecting and analyzing data for federal criteria pollutants and many other air 

pollutants of concern; collecting data from special studies as directed by the CARB Board; assessing 
Final Report - August 2016

12

monitoring methods that are used by the state and local districts in compliance with federal and state 

regulations; conducting annual performance audits of all monitoring equipment within the PQAO; 

implementing a calibration and certification of measurement standards program; and conducting training 

in the operation of ambient air monitoring instruments. 

Organization of the CARB Ambient Air Monitoring Program

At the time of the audit, responsibility for overseeing the ambient air monitoring program for CARB 

resided within the following branches:

Branch Division

Air Quality Surveillance Branch (AQSB) Monitoring and Laboratory Division (MLD)

Northern Laboratory Branch (NLB) Monitoring and Laboratory Division (MLD)

Quality Management Branch (QMB) Monitoring and Laboratory Division (MLD)

Consumer Products & Air Quality 

Assessment Branch (CPAQAB)

Air Quality Planning and Science Division 

(AQPSD) 

QA responsibility for CARB is covered primarily by the QMB. Michael Miguel is the QMB Chief. The 

QMB oversees the development of quality management documents with contributions from other 

branches, conducts on-site audits, reviews precision and accuracy data, and initiates corrective action 

requests (i.e., AQDAs), among other duties. The AQSB, NLB, and CPAQAB contribute to QA activities 

primarily through data validation activities. QC responsibility for field activities is handled primarily by 

the AQSB, which includes field operations, calibrations, and repair, precision and accuracy data 

review/submittal, instrument acceptance testing, determination of CARB federal/state ambient 

monitoring methods, development of SOPs, interpretation of CFRs pertaining to monitoring 

criteria/methods, and a variety of other monitoring functions. QC responsibilities for laboratory 

activities is handled primarily by NLB, which includes laboratory operations, calibrations, laboratory 

QC, data review/submittal, development of SOPs, interpretation of CFRs pertaining to laboratory 

criteria/methods, and a variety of other laboratory functions. 

The management of QA/QC for local districts within the CARB PQAO is consistent with the EPA 

requirements for the five common factors of a PQAO, as applicable, and defined in the R&R document 

developed between CARB and each district. However, details and responsibilities are specific to each 

local district based on their current structure and resources. 

The CARB PQAO has a number of mechanisms to achieve common QA oversight. CARB conducts 

audits of all sites within California thus ensuring common QA for instruments. CARB conducts or 

participates in TSAs for all districts within the PQAO. CARB maintains an on-line repository of the 

approved quality management documents (QMP, QAPP, SOPs) adopted and used by districts 

throughout the PQAO for the operation of their ambient air monitoring network. CARB validates all 

continuous and laboratory data generated by CARB sites and laboratory operations (collected from 

CARB and non-CARB sites), and uploads and certifies data generated by districts for which CARB has 

AQS data authority. Local districts for which CARB has AQS entry and certification responsibilities are 

expected to provide a letter to CARB stating that data have been validated according to approved 

procedures, and concurrence with certification of the data. 

In most cases the local district is responsible for daily QC aspects of the air monitoring program (e.g., 

zero/precision/span checks, calibration, and regular on-site review and maintenance), validation and 
Final Report - August 2016

13

certification of data according to CARB and/or EPA approved procedures (except as noted above), and 

the review and updating of quality management documents to ensure they reflect current practices and 

requirements. 

Network Management

At the time of the TSA, responsibility for network management lay within the CPAQAB. Ravi 

Ramalingam is the CPAQAB Chief. Network management responsibilities include network assessment 

and network plan preparation, analyses and decision-making for CARB system modifications, and 

review of requests by local districts within the PQAO for system modifications. Network management 

for non-CARB sites has traditionally been managed by the local district.

The state network consists of monitoring stations operated by CARB and the local districts. The network 

covers 15 air basins. The four PQAOs in the state operate monitoring networks that provide data from 

all the air basins. Three local district PQAOs operate their own monitoring networks that provide data 

for the following air basins: San Francisco Bay, San Diego County, South Coast, and Salton Sea2. The 

CARB PQAO operates multiple monitoring networks that cover the following air basins: Great Basin, 

Lake County, Lake Tahoe, Mojave Desert, Mountain Counties, North Central Coast, North Coast, 

Northeast Plateau, Sacramento Valley, Salton Sea, San Joaquin Valley, and South Central Coast. In 

some instances, several local districts operate the monitoring networks within an air basin. Furthermore, 

the boundaries of metropolitan and micropolitan statistical areas (MSAs), and core-based statistical 

areas (CBSAs) 3and combined statistical areas (CSAs), which are established by the US Census Bureau, 

may also overlap air basins and local monitoring districts. EPA uses the population statistics of 

MSAs/CBSAs to determine the number of the minimum state/local air monitoring station (SLAMS)

monitoring requirements for some criteria pollutants. Other minimum SLAMS monitoring requirements 

are based on source emissions, or a combination of emissions/sources and population.

Table 2 summarizes the number of criteria pollutant monitoring sites operated in the CARB PQAO at 

the time of the TSA.

 

2 South Coast AQMD collects ambient air monitoring data in Coachella Valley, which is part of the Salton Sea Air Basin.

3 The definition of a Core-Based Statistical Area can be found at http://www.census.gov: Metropolitan and micropolitan 

statistical areas (metro and micro areas) are geographic entities defined by the Office of Management and Budget (OMB)

for use by Federal statistical agencies in collecting, tabulating, and publishing Federal statistics. The term "Core Based 

Statistical Area" (CBSA) is a collective term for both metro and micro areas. A metro area contains a core urban area of 

50,000 or more population, and a micro area contains an urban core of at least 10,000 (but less than 50,000) population. 

Each metro or micro area consists of one or more counties and includes the counties containing the core urban area, as 

well as any adjacent counties that have a high degree of social and economic integration (as measured by commuting to 

work) with the urban core.
Final Report - August 2016

14

Table 2: Summary of Criteria Pollutant Monitors in the CARB PQAO4

Operating Agency O3 CO NO2 SO2 PM2.55 PM10 Pb6

CARB 3477 1482 21 22 1

AVAPCD 1 1 1 0 1 1 0

BCAPCD 0 0 0 0 1 0 0

GBUAPCD 2 0 0 0 2 12 0

ICAPCD 2 1 1 0 2 4 0

EKAPCD 0 0 0 0 1 1 0

LCAQMD 1 0 0 0 1 3

9

0

MeCAQMD 1 0 0 0 2 1 0

MDAQMD 6 2 3 2 1 5 0

MBUAPCD 6 1 1 0 6 2 0

NCUAQMD 2 2 2 2 2 1 0

NSAQMD 1 0 0 0 5 0 0

NSCAPCD 1 0 0 0 0 3 0

PCAPCD 4

10 0 0 0 4 2 0

SMAQMD 6 4 5 0 5 5 1

SJVAPCD 13 5 10 0 16 12 0

SLOCAPCD 6 0 3 1 3 4 0

SBCAPCD 1011 4

12 913 714 2 515 0

ShCAQMD 4 0 0 0 2 3 0

SiCAPCD 1 0 0 0 1 1 0

TeCAPCD 1 0 0 0 1 1 0

VCAPCD 5 0 2 0 5 2 0

Y-SAQMD 2 0 0 0 2 3 0

TOTALS 109 27 51 14 86 91 2

Source: CARB/AQPSD/AQDB

 

4 Primary source for monitoring sites was the QA Monitoring Site database. Sites reported as closed on the QA database were not 

counted in the totals. Several gaseous sites listed in CAMNAT but not in the QA database were also included.

5

Includes all federal reference method (FRM) and federal equivalent method (FEM) instruments and also non-FEM beta attenuation 

method (BAM) instruments.

6 Pb is measured by total suspended particulate (TSP) at Fresno NCore, and Pb-PM10 at Sacramento Del Paso Manor NCore.

7

Includes Arvin – Di Giorgio (060295002) and Shafter (060296001).

8

Includes Shafter (06029601)

9 The PM10 monitors in Lake County report in local conditions (LC).

10 Includes newly established Lincoln site (AQS ID 06-061-2001).

11 Includes 6 long-term prevention of significant deterioration (PSD) monitors that are overseen by Santa Barbara County APCD but 

operated by contractors (AQS IDs: 06-083-1021, 06-083-1013, 06-083-1025, 06-083-1018, 06-083-1014, 06-083-4003).

12 Includes 3 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06-

083-1025, 06-083-1008, and 06-083-4003).

13 Includes 8 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06-

083-1025, 06-083-1021, 06-083-1018, 06-083-1013, 06-083-2004, 06-083-1014, 06-083-0011, 06-083-1008, and 06-083-4003).

14 Includes 4 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06-

083-1025, 06-083-1013, 06-083-4003, and 06-083-1020).

15 Includes 2 long-term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06-

083-1025 and 06-083-4003).
Final Report - August 2016

15

In 2006, EPA added a requirement for Annual Monitoring Network Plans and Five-year Network 

Assessments (40 CFR 58.10). At the time when the first network plan was due (July 2007), 

numerous local agencies within the CARB PQAO expressed an interest in submitting their own 

plan. On this basis, an agreement was reached between EPA, CARB, and local districts wherein 

agencies within the CARB PQAO that wanted to submit their own plan could follow the process 

and submit a network plan directly to EPA. CARB submitted a network plan to cover those 

agencies that did not choose to submit a plan (Table 3). The Five-year Network Assessment that 

was due in 2010 and 2015 followed a similar process. The network plan per 40 CFR 58.14 (a) 

must include recommendations from the network assessment.

Table 3. List of Agencies in the ARB PQAO Drafting Annual Network Plans in California.

Air districts included in ARB plan

Air districts in ARB PQAO drafting their 

own plan

 Amador County APCD Great Basin Unified APCD *

 Antelope Valley AQMD Monterey Bay Unified APCD

 Butte County AQMD North Coast Unified AQMD

 Calaveras County APCD Sacramento Metropolitan AQMD

 Colusa County APCD San Joaquin Valley APCD

 Eastern Kern APCD San Luis Obispo County APCD

 El Dorado County AQMD Santa Barbara County APCD

 Feather River AQMD

 Glenn County APCD

 Imperial County APCD

 Lake County AQMD

 Lassen County APCD

 Mariposa County APCD

 Mendocino County AQMD

 Modoc County APCD

 Mojave Desert AQMD

 Northern Sierra AQMD

 Northern Sonoma County APCD

 Placer County APCD

 Shasta County AQMD

 Siskiyou County APCD

 Tehama County APCD

 Tuolumne County APCD

 Ventura County APCD

 Yolo-Solano AQMD*

Source: 2015 Annual Network Plan for 25 Air Districts in California. *Yolo-Solano is included in the 

Bay Area network assessment and Great Basin is included in the CARB network assessment. Otherwise, 

all other districts in the CARB Annual Monitoring Network Plan are included in CARB’s 2015 Five-year 

Network Assessment.
Final Report - August 2016

16

Requirements for developing an adequate network are not based on agency jurisdiction, but 

rather on CBSAs or PQAOs. A collaborative approach between CARB, EPA, and local districts 

is needed to evaluate whether the requirements for an adequate network continue to be met. To 

assist in the evaluation of network adequacy, EPA funded development of the CAMNAT 

database that provides monitoring information for the entire state. This database was launched in 

the first quarter of 2015 and detailed site reports were incorporated in the 2015 CARB Annual 

Monitoring Network Plan. 

Requests for changes to the network may occur outside the network plan process by the 

submission of a letter to EPA. In recent years, CARB and EPA have discussed proposed site 

changes via conference calls. Once a decision is made, CARB or the District sends a formal 

request to EPA for approval. When a local district wishes to modify its network, e.g., shutting 

down or relocating an existing site or establishing a new one, it consults with CARB. CARB 

works with the district and often arranges a conference call with EPA. Based on the discussion 

with CARB and EPA, the local district will draft a letter requesting (1) site termination, or (2) 

site relocation. 

Field Operations

Network operations at CARB were primarily performed by the MLD AQSB. Ken Stroud is the 

Chief of AQSB. AQSB duties include the operation of CARB monitoring sites, monitoring 

support for CARB special studies, and general air monitoring support, such as repair and 

calibration facilities. AQSB also assists local districts with instrument trouble-shooting and 

repair as resources allow. This section of the TSA report addresses AQSB's general operations,

the calibration program, and field operations of the AQSB at CARB-operated criteria pollutant 

monitoring sites. 

Providing training and performing instrument certifications are the responsibilities of the 

Operations and Data Support Section (OSS) within AQSB. The OSS also provides independent 

review and approval of ARB field SOPs. Other responsibilities include instrument repair and 

technical support. While support (e.g., training, field procedures, and other technical support) is 

available to the all local districts in California, the AQSB does not actively manage the local 

districts’ field monitoring quality systems. 

Laboratory Operations

Analytical services at CARB were provided by MLD’s NLB. Michael Werst is Chief of NLB. 

NLB is divided into four sections: Inorganic Laboratory Section (ILS), Laboratory Support 

Section (LSS), Organics Laboratory Section (OLS), and Special Analysis Section (SAS). LSS 

provides laboratory support to the other three laboratories that includes the Laboratory 

Information Management System (LIMS) Administrator, Laboratory QA/QC Officer, Laboratory 

Health and Safety Officer, hazardous waste management, and sample control. ILS and OLS 

analyze samples to support federal and state ambient air quality monitoring and toxic air 

contaminant programs, including pesticide monitoring. The ambient air quality programs consist 

of criteria pollutants such as PM, including PM2.5, PM10, PMcoarse, and Pb-TSP. ILS and OLS 

also provide analytical services for EPA’s PM2.5 Speciation Trends Network (ions, metals, 
Final Report - August 2016

17

elemental/organic carbon, and wood smoke tracers) and California’s PM10 Ions Programs. The 

toxic air contaminant programs include volatile organic compounds, carbonyl compounds, 

metals, hexavalent chromium, and various pesticides by request from California’s Department of 

Pesticide Regulation. SAS provides analytical services for California’s consumer products and 

composition wood product programs.

Data Management

Air quality data measured by continuous analyzers in the CARB ambient air quality network are 

captured by a data acquisition system. CARB’s data acquisition system is composed of two 

major components: 1) a PC based system utilizing a custom based Linux software package 

referred to as CARBLogger; and 2) DMS which is an SQL database developed by Sonoma 

Technology Inc.

CARBLogger continuously queries station analyzers via the instruments communication ports, 

and processes one-minute concentration and meta-data files (or hourly data for instruments that 

only provide hourly records). CARBLogger transmits air quality data to DMS every hour via a

file transfer protocol (FTP) server operated by the CARB Office of Information Services (OIS). 

DMS then ingests one-minute air quality data, aggregates hourly averages, performs automated 

QC checks and transmits hourly ambient data back to the FTP server, where data are available 

for real-time data applications such as the CARB Air Quality and Meteorological Information 

System (AQMIS) and the national AIRNow System, established by EPA, National Oceanic and 

Atmospheric Administration, National Park Service, tribal, state, and local agencies to provide 

the public easy access to national air quality information.

DMS is the primary management tool for data collection, validation, and reporting of data 

obtained at CARB-operated stations. All raw data, QC records and chain of custody (CoC) 

documentation are retained in DMS for at least three years. Once data are reviewed by the data 

review process, they are uploaded to the EPA’s AQS. Once submitted to AQS, the data are 

downloaded to the CARB database Air Data Management System (ADAM). ADAM is the

official state database for ambient air quality data. Hard copy files (QC forms, calibration and 

audit reports, control charts, data reporting memos etc.) are stored locally at CARB headquarters

and provide a supplemental record for the data validation process.

At the time of the TSA, air monitoring responsibilities were divided among five organizational 

units in two different Divisions of CARB. The State of California has 25 separate air pollution 

control agencies through which ambient monitoring data enters the AQS database. Responsibility 

for managing the state’s ambient monitoring data required by the Clean Air Act (CAA) was

divided among the following groups (descriptions of each group’s role follows the list):

1. Continuous data from CARB-operated field monitoring stations – MLD/AQSB, Ken 

Stroud, AQSB Chief.

2. CARB laboratory analytical data for both CARB and non-CARB Sites – MLD/NLB, 

Michael Werst, NLB Chief.

3. QA performance audit program data and Standards Laboratory – MLD/QAS, Ranjit 

Bhullar, QMB/QAS Manager.
Final Report - August 2016

18

4. Local district site data: operated by local district but AQS-uploaded by CARB – AQPSD,

Gayle Sweigert.

5. Local district site data: operated and AQS-uploaded by local district – various local 

districts.

6. Local district site data: Operated by local district with laboratory analyses by a different 

laboratory that is not CARB – data upload is the responsibility of the District that 

performs the analyses. This is defined for each District in the Roles and Responsibilities 

documents and QA Bulletins. 

1. Continuous Data from CARB-operated Field Monitoring Stations

Continuous data collected at CARB-operated field monitoring stations includes data from all 

continuous air quality analyzers (O3, NO2, CO, SO2, CO2, CH4, N2O, and non-filter-based PM) 

and meteorology data. The CARB-operated ambient monitoring stations are managed by Ken 

Stroud, AQSB Chief. There are two regional Supervisors, Joe Guerrero (Air Monitoring North

Section [AMNS]), and Fernando Amador (Air Monitoring South Section [AMSS]). Air quality 

data measured by the continuous analyzers at the field stations operated and maintained by 

CARB are stored in data loggers and station computers. Each station is polled through broadband 

internet connections and the data are transmitted directly to the central computer system in 

Sacramento. The computer system consists of a server located within the State of California 

Federated Data Center and a backup server located in a separate leased facility. The data are 

collected in the Data Management System (DMS), which was developed by Sonoma 

Technology, Inc. The DMS is the primary management tool for data collection, validation, and 

reporting at CARB-operated stations. Data are retained in DMS for multiple years, however, data 

submittals to AQS occur on a monthly basis. Once submitted to AQS, the data are replicated to 

the ADAM database, the official state database for ambient air quality data. 

An updated SOP for data validation was finalized in March 2016. The first review (first-level 

validation) of the data is performed by the station operators. Station operators review minute data 

and take corrective actions as necessary. This review includes: max/min; outliers; repeating 

values; automatically flagged values; diurnal and seasonal trends; QC data and instrument 

maintenance sheets; review/edit/flag/comment data; and document notes on the monthly 

maintenance data sheet/station logbook/DMS editor notes. Station operators also review QC data 

daily if possible, perform comparisons of “buddy sites” and collocated data, review data for 

irregularities or issues, and inform the second level reviewers if follow up actions, such as 

calibrations, are needed. The station operator compiles a monthly data submittal package for 

each site consisting of copies of each relevant calibration control chart, instrument “monthly 

quality maintenance check sheets,” that month’s station logbook pages, the data capture report, 

the station operator’s name, and what data have been reviewed in DMS. The package is reviewed 

by the station operator for completeness, and then submitted to the assigned second-level data 

reviewer.

The second-level review is done by an Air Pollution Specialist and/or an Air Resources 

Engineer. The review consists of reviewing the min/max and exceedances, data flags, monthly 

maintenance check sheets, QC charts, and perform parameter comparisons and “buddy site”

comparisons. If QC issues may affect the data, the second-level reviewer edits in DMS and any 
Final Report - August 2016

19

corrective action is documented in the station logbook and monthly maintenance check sheet. 

The second-level reviewers are also responsible for reviewing the monthly calibration control 

chart webpage to confirm that the data edits were incorporated into DMS. Second-level data 

reviewers must initial and date each page in the monthly data submittal package to signify 

review, and sign and lock hourly data in DMS. Significant issues should be noted on the cover 

page of the data package submitted to the third-level review.

The third-level review is done by the appropriate Section Manager (Northern or Southern 

Section) who ensures that data are complete, the instruments are meeting operational criteria, and 

the stations have been maintained properly. The Section Managers also consider any significant 

issues that have been identified by previous-level review, perform “buddy site” comparisons and 

compare collocated data.

At this point, a final data validation summary report is sent to the AQSB Chief with an initialed 

cover memo. The report includes all control charts, percent data capture, monthly data 

summaries, station logbook copies, and the cover memo, which identifies any significant issues 

for each site. The AQSB Chief reviews the summary report and memo and performs 

comparisons of “buddy sites” and collocated data. He then initials the cover memos, gives the 

cover memo to the designated staff in OSS for submittal to EPA’s AQS database, and provides 

copies of the cover memo to station operators and second level-data reviewers. 

In summary, data validation for continuous pollutants goes through the following steps:

1. Review by station operator.

2. Review by Air Pollution Specialist and/or Air Resources Engineer.

3. Review by Section Manager.

4. Review by AQSB Chief.

5. Upload to AQS by OSS designated staff.

Overall, CARB submits all required data to the EPA AQS database, including concentrations for 

all criteria pollutants, and supporting precision and accuracy information. CARB certifies these 

data annually as required by regulation (40 CFR 58.15).

2. CARB Laboratory Analytical Methods for both CARB and Non-CARB Sites

Overview of Laboratory Data Management

Data management describes the basic flow of analytical data from generation, review 

(verification and validation), and reporting. Laboratory staff and management have responsibility 

for review of the data management system. The laboratory utilizes a LIMS database to carry out 

data management activities. 

LIMS

LIMS facilitates the recording, verification and validation, transmittal, reduction, analysis, 

management, storage, retrieval, and reporting of analytical data generated by the laboratory. 

LIMS is maintained by the LIMS administrator. The administrator creates and/or modifies 
Final Report - August 2016

20

approved laboratory staff access to LIMS; creates and modifies methods, data templates and 

transfers, and data reports; and has access to modify locked data in LIMS. All sample and 

analysis information must be entered into LIMS or recorded in bound or electronic notebooks. 

Changes to any data in LIMS must be made by authorized individuals and approved by 

management. 

 

All users must be authorized by management and receive a password to access LIMS. Different 

privileges are given to authorized users depending on need. LIMS users are the laboratory staff 

who have approved access to the programs and data; they may log in, transfer, modify data, 

and/or run reports in order to validate data generated. Access may include read-only, data entry 

and preliminary data modification, data reporting, and client distribution.

LIMS can be accessed to generate laboratory analytical work lists, summaries, and data reports, 

where applicable. Work lists are used by the analyst to schedule and prioritize sample analyses 

(e.g. sample hold times, inventory, etc.). Summary reports give laboratory staff an overall picture 

of samples logged into LIMS, and enable them to review raw and quality control data that have 

been transferred into LIMS.

Initial Data Assessment

Samples are analyzed and the instrument QC results are reviewed by the analyst to decide if 

sample analysis is valid prior to transfer into LIMS. When QC criteria are not met, corrective 

action should be taken as needed, such as re-analysis, dilution, re-integration, etc. Any sample 

result that has been invalidated must be reported as "invalid” or “not analyzed,” and the reason 

documented. All results reported as "not detected" must be associated with a reference value, 

such as level of quantitation, estimated quantitation limit, or minimum reporting value.

Laboratory staff will contact site operators, or other appropriate staff, directly when issues arise 

that require clarification to validate a sample at log-in, when a sample is invalidated, or when a 

make-up sample is recommended. This notification is performed as soon as possible, and the 

issue is documented on the CoC or sample report form, according to established laboratory 

procedures. If invalidated samples occur repeatedly and are deemed by management to be 

indicative of a systemic issue, management will utilize the CAN process to initiate a formal 

corrective action process in order to inform all responsible and impacted parties; document the 

issue and resolution; and prevent potential future data loss. If a CAN is determined to be

unnecessary, management will document how the issue was resolved and which parties were 

notified of the issue.

Data Transfer to LIMS

Data from the analytical system is uploaded to LIMS manually or electronically by the analyst. 

Assistance from the LIMS administrator may be needed in certain circumstances. Instrument to 

LIMS transfers should be verified by the analyst. In management-approved special situations, 

where LIMS transfer and storage is unnecessary, the data must be electronically stored in an 

appropriate file. All raw data should be archived appropriately.
Final Report - August 2016

21

Data Analysis Records

All raw data, calculations, observations, validation information, and results generated by the 

analyst must be placed in an appropriate computer file, bound or electronic laboratory notebook 

or other approved format. For bound notebooks, all entries must be initialed and dated by the 

analyst. All analysis hardcopies must be stored in an appropriate filing system until archived.

Any raw analytical data stored on a computer hard drive should be routinely backed up. A 

backup copy of all instrument software, including NLB developed parameters, should be made 

after the initial development. All backup files are labeled.

An instrument maintenance logbook must be assigned to each instrument. All calls for service, 

repair records, reconfigurations, or changes to the instrument operating parameters must be 

recorded, dated, and signed by the analyst or instrument service representative. The logbook 

must be kept with the instrument and be available for inspection at any time.

Analytical Data Reports

Analytical data reports are generated by the analyst and submitted for review/approval after 

initial data assessment and transfer to LIMS in order to verify and validate the data. At a 

minimum, the following must be included in the data package: cover page with signature and 

date blocks, title of method, program, or project name, timeframe of analyses, and pertinent 

documentation related to the analytical run; analytical run sequence; data with comments and 

flags; quality control data; control charts; and calibrations.

LIMS Verification and Validation

LIMS has been programmed to automatically verify and validate data entered into the database. 

Any data outside QC criteria are highlighted for analyst, peer, and management review and 

comment. QC parameters programmed into LIMS come from federal regulations, NLB’s Quality 

Control Manual (QCM), and SOPs. All programming has been tested and verified by the LIMS 

administrator.

Data Review and Approval

The data review and approval process consists of a series of checks to ensure the analytical data 

generated by the laboratory and transferred to LIMS meet all the method specific QC criteria. 

The multistep process includes, at a minimum, analyst and peer review followed by management 

review and approval prior to submittal to clients. All levels of review and approval are initialed 

and dated on the cover page of the data package.

Analyst Review

The following items are verified by the analyst who performed the extractions and analyses, as 

appropriate for the method, as required by the method SOP:

 Extraction solvents and volumes.

 Instrument conditions. 
Final Report - August 2016

22

 Analytical run conducted per SOP.

 Expiration dates of standards.

 Retention times. 

 Integrations. 

 Peak identifications. 

 Analytical sequences. 

 Environmental conditions. 

 QC (such as duplicates, standards, blanks, controls). 

 Data reduction and calculations.

 CoC data login to LIMS.

 Raw data concentrations transferred to LIMS. 

 Holding times.

 Calibrations. 

 Parameters of SOP and QCM are met. 

 Anomalies and corrective actions are documented and management notified, as 

necessary.

Peer Review

The following items are verified by a second analyst:

 Data package completeness. 

 Spot-check calculations. 

 Check for documentation of unusual events. 

 Corrective action review.

 Check for outliers. 

 Analytical run sequence.

 Dilutions performed as necessary.

 QC (such as duplicates, standards, blanks, controls). 

 Expiration dates of standards.

 Reasons for invalid samples. 

 Flags and comments.

 Parameters. 

 SOP and QCM are met. 

If necessary, a data package is returned to the analyst for edits or clarification. After corrections 

are made, the data package is returned to the peer reviewer for confirmation. Once peer review 

is complete, the peer reviewer signs and/or initials, and dates the analytical data package. 

Management Review and Approval

The following is reviewed by management prior to data release:

 Data package completeness. 

 Spot-check calculations. 

 Check for documentation of unusual events. 
Final Report - August 2016

23

 Corrective action review.

 Check for outliers. 

 Analytical run sequence.

 QC (such as duplicates, standards, blanks, controls). 

 Expiration dates of standards.

 Reasons for invalid samples. 

 Flags and comments. 

 Check for analyst and peer review. 

 Parameters of SOP 

 QCM are met. 

If necessary, a data package is returned to the analyst for edits or clarification. After corrections 

are made, the data package will be returned to management for confirmation. Once review is 

complete, management(s) signs and/or initials and dates the analytical data package. 

Data Release and Reporting

Data generated by the laboratory goes through the defined data review and approval process 

prior to release and reporting to clients. During the review and approval process, the data files in 

LIMS are locked to ensure that data cannot be changed without proper management 

authorization. Data in LIMS can still be viewed (read only) by management and laboratory staff. 

If changes to finalized data are made, the full review and approval process shall be conducted to 

amend the data report. The approved data reports are provided to management, who sends the 

hardcopy and/or electronic data report to the client or the client's representative. 

Amendment to Data

Finalized and approved data may be amended in LIMS, AQS, and client reports per management 

approval. After the request is approved, laboratory staff and management follow the data review 

and approval process. If changes to finalized data are made, the client is notified and sent a 

revised report. Data may be amended for reasons such as CANs, AQDAs, requests by clients (i.e. 

requests to exclude codes), etc.

Data Archive

All final hardcopy reports with the analyst review, peer review, and management approval 

signatures are filed in a secure manner. Access to hardcopy and LIMS files is limited to 

authorized individuals only. Laboratory retention of hardcopy and electronic LIMS data files 

shall be a minimum of five years. Final archiving and/or destruction of all data reports are

approved by management.

3. CARB QA Data

QA data are generated from state-wide performance evaluation audits conducted by CARB. 

These audits are performed independently from the ambient air monitoring operations. QAS is 

responsible for conducting performance audits of criteria and non-criteria pollutant analyzers, 
Final Report - August 2016

24

particulate samplers, meteorological equipment, and laboratory analyses utilized for generating 

ambient level measurements. QAS also reports quality assessment and quality control results. 

Performance audits within the entire CARB PQAO are conducted annually for gaseous pollutant 

monitoring, mass analysis laboratories and meteorological instruments, and semi-annually for 

particulate matter samplers. The results of the audits are maintained online on the CARB website 

and are uploaded by CARB to AQS. EPA conducts an annual comparison with the CARB audit 

vehicles to ensure comparability with EPA National Performance Audit Program (NPAP); these 

data are uploaded to AQS by CARB.

In addition to collecting and managing data from audits, QAS reviews quality assurance data for 

the entire state. At least once a quarter, QAS staff retrieves the “QA Data Quality Indicator 

Report” (AQS report code AMP256) from AQS for all California sites and all pollutants. QAS 

staff review the reports for inconsistencies and completeness and works with each agency to 

address any issues. QAS also verifies and validates the “Certification Evaluation and 

Concurrence Report” (AQS report code AMP600) during the annual data certification process.

4. CARB Special Purpose Monitoring

Special Purpose Monitoring is conducted on an as-needed basis by the Special Purpose 

Monitoring Section, in AQSB. Depending on the nature of the project, data may be or may not 

be uploaded to AQS. 

5. Collected by Local District / Uploaded to AQS by CARB

The Air Quality Analysis Section (AQAS) in AQPSD is responsible for uploading continuous air 

quality data (O3, NO2, CO, SO2, and non-filter based PM), PM data that are weighed by the local 

district, and meteorology data from those local districts without direct access to AQS. These 

districts include: ICAPCD, LCAQMD, MeCAQMD, NCUAQMD, NSCAPCD, PCAPCD, 

SMAQMD, SiCAPCD, TeCAPCD, and Y-SAQMD (Table 4). The AQAS is located at the Cal 

EPA Headquarters building in Sacramento, CA and is managed by Gayle Sweigert. AQAS 

uploads continuous PM data from local districts, as well as PM filters data that are weighed by 

LCAQMD, NCAQMD, and SMAPCD. PM filters that are collected by local districts and then 

weighed by CARB are managed by the CARB MLD NLB. 

Data are received electronically by email from the ten local districts listed in the previous 

paragraph. Local districts typically send their data monthly or quarterly in the form of an 

electronic file. When data arrive at AQAS, they are logged in and uploaded to AQS, which runs 

them through AQS review protocols. Staff sends an email to the district staff confirming the 

upload, which has attached the AQS raw data report and the raw data inventory report. If the 

AQS review protocols identify a potential issue, AQAS staff contacts the district to resolve it. 

The districts are expected to submit only fully validated and reviewed data that are accompanied 

with a data validation letter consistent with the protocols identified in the roles and 

responsibilities documents. If a district sends data without a data validation letter, CARB 

contacts the district to let them know the data will not be uploaded until a data validation letter is 
Final Report - August 2016

25

received. CARB assumes that the data to be uploaded to AQS for local districts have been fully 

validated, and carry the appropriate flags. 

Data uploaded by AQAS for the ten districts listed above are certified by CARB annually, as 

required by regulation (40 CFR 58.15).

Table 4. Summary of data upload and certification responsibilities for the State of 

California.

Operation of 

Monitoring 

Site

Upload of 

Continuous Data

Certification Upload of PM Filterbased Laboratory 

Data

Certification

AVAQMD MDAQMD MDAQMD MDAQMD MDAQMD

CARB CARB CARB CARB/*North Carolina CARB

EKAPCD N/A N/A CARB/ SDCAPCD (PM2.5) CARB

GBUAPCD GBUAPCD GBUAPCD GBUAPCD GBUAPCD

ICAPCD CARB CARB CARB - PM10/

SDCAPCD - PM2.5

CARB – PM10/

ICAPCD – PM2.5

LCAQMD CARB CARB CARB+ CARB+

MeCAQMD CARB CARB N/A N/A

MDAQMD MDAQMD MDAQMD MDAQMD MDAQMD

MBUAPCD MBUAPCD MBUAPCD BAAQMD - PM2.5 MBUAPCD

NCUAQMD CARB CARB BAAQMD - PM2.5

NCUAQMD -

PM2.5

NSAQMD NSAQMD NSAQMD CARB CARB

NSCAQMD CARB CARB N/A N/A

PCAPCD CARB CARB Pending Termination Pending 

Termination

SMAQMD CARB CARB CARB - PM2.5

CARB - PM10

+

CARB - PM2.5,

CARB - PM10

+

SJVAPCD SJVAPCD SJVAPCD CARB CARB

SLOCAPCD SLOCAPCD SLOCAPCD N/A N/A

SBCAPCD SBCAPCD SBCAPCD N/A N/A

ShCAQMD ShCAQMD ShCAQMD CARB CARB

SiCAPCD CARB CARB CARB CARB

TeCAPCD CARB CARB CARB CARB

VCAPCD VCAPCD VCAPCD N/A N/A

Y-SAQMD CARB CARB CARB CARB

Note: Operating agencies are responsible for data management from generation through validation, except as noted above. 

Agencies not performing upload and certification of their own data must provide a letter to the agency performing data upload 

and certification verifying that data have been validated according to a CARB and/or EPA approved procedure. 
Final Report - August 2016

26

* Spiral Aerosol Speciation Sampler (SASS) filters sent to EPA in North Carolina for the Bakersfield California Avenue site

only.

+ These data are weighed by local air district but uploaded by CARB. 

6. Collected and Uploaded to AQS by Local District 

The following districts have access and authority to upload their data to AQS: GBUAPCD, 

MDAQMD (includes AVAPCD), NSAQMD, MBUAPCD, ShCAQMD, SJVAPCD, 

SLOCAPCD, SBCAPCD, and VCAPCD (see Table 4). The reporting of data into AQS by these 

local districts was agreed upon and commemorated in district-specific Memorandums of

Understanding signed by the districts, CARB, and EPA starting in 2002. All levels of data 

management are handled by the local districts and these responsibilities are formalized in the 

R&R documents established between the local districts and CARB. Districts are required to 

adopt and follow CARB procedures for validation and certification of data, or submit their own 

procedures to CARB for review and approval. CARB and district procedures are posted on the 

on-line document repository and verified by CARB during the TSA process. Additionally, 

CARB provides training and guidance on the data validation and certification processes via inperson (PQAO training), webinar, and on-line resources. CARB expects the data that it does not 

upload to be certified per regulation (40 CFR 58.15) by either the local district collecting it or by 

the agency uploading it.

7. Collected by Local District / Laboratory Analyses Performed by a Laboratory Other 

Than CARB

All continuous data collection is managed by each local district; CARB uploads data for ten of 

the districts (see Table 4). Data generation and management for PM filter-based data are not only 

district-specific, but can even be site-specific. Some districts weigh their filters and upload and 

certify their PM data. Some collect the filters and send them to CARB for weighing, validation, 

upload, and certification. Other districts collect the filters and send them to a local district that 

has a weigh laboratory; responsibility for data validation, upload, and certification in these cases 

is defined in the R&R document. Finally, some districts may send filters from some sites to 

CARB and filters from other sites to a different local district that may be closer to the site. In the 

case where all filters are sent to a different local district, responsibility for data validation, 

upload, and certification of the filters is defined in the R&R document.

Quality Assurance/ Quality Control

An organization’s quality management system includes QA, QC, and quality improvement 

activities. EPA requires that ambient air monitoring agencies have a quality management system 

that conforms to 40 CFR Part 58 Appendix A and the EPA quality policy (EPA Order CIO 

2106.0). Additionally, EPA grant regulations specifically require each grantee to provide for QA 

activities (40 CFR Part 31.45). Specifically, 40 CFR Part 58 Appendix A Section 3 requires that 

each ambient air monitoring PQAO conforms to certain quality management practices. These 

include:

 A documented quality system that meets EPA requirements for QMPs and QAPPs.

 A quality management function that is independent of air monitoring operations.
Final Report - August 2016

27

 Stated data quality objectives or equivalent systematic planning procedures for all 

monitoring programs.

 Participation in National Performance Evaluation Programs, which consist of 

performance audits to independently determine program adequacy, national monitoring 

network performance, and national consistency.

 EPA-led TSA every three years or less.

 Use of certified reference materials to standardize monitoring equipment.

EPA views these quality management system components as being integral to maintaining a 

credible monitoring program. Insufficient quality management and control has been cited in 

support of legal challenges to NAAQS designation decisions. 

QA and QC are the two components of a quality management system for a monitoring program 

that serve to document the assertion that the data collected represent the true air quality of the 

area. They are the means by which an organization manages its quality aspects in a systematic, 

organized manner to provide a framework for planning, implementing, and assessing work 

performed by an organization. A properly developed QA/QC program encompasses a variety of 

technical and administrative elements, including policies and objectives, organizational 

authority, responsibilities, accountability, and procedures and practices. QA is a management or 

oversight function setting policy and overseeing an administrative system of management 

controls that cover planning, implementation, the review of data collection activities, and the use 

of data in decision making. QC is a technical function that includes all the scientific precautions, 

such as calibrations and duplications needed to acquire data of known and adequate quality. 

The CARB QMB is composed of three sections: QAS, the Quality Management Section (QMS), 

and the Office of Emergency Response. EPA did not include the Office of Emergency Response 

with this TSA. Primary responsibilities of QAS include:

 Conducting performance audits of MLD and district monitoring instruments. 

 Conducting TSAs of California air districts. 

 Updating SOPs specific to QAS activities. 

 Validating MLD’s field generated monitoring data (accuracy assessments).

 Providing verification/certification services within the Standards Laboratory.

QMS performs quality management oversight activities for CARB internally as well as for the 

entire PQAO. Primary responsibilities of QMS include:

 Assisting with system audits of California air districts

 Preparing data quality summary reports for the CARB PQAO and other PQAOs in 

California. 

 Providing review and approval of quality management documents for CARB and districts 

within the PQAO.

 Coordinating PQAO training on the fundamentals of ambient air monitoring. 

 Conducting liaison activities to enhance communication and QA/QC activities across the 

PQAO. 

QC-related functions are performed by AQSB. AQSB performs several quality management 

functions, including: 
Final Report - August 2016

28

 Developing and administering the training program for instrument operators.

 Performing instrument acceptance testing.

 Validating MLD field generated monitoring data.

 Developing, preparing and reviewing SOPs for the air monitoring program.

 Determining monitoring methods used in the ambient air monitoring network. 

QA-related functions in NLB include:

 Developing laboratory and ambient air collection test procedures.

 Conducting analyses of ambient air samples and consumer products.

 Performing self-assessments and producing an annual quality control summary report for 

the Division Chief.
Final Report - August 2016

29

OVERVIEW OF THE MOJAVE DESERT AIR QUALITY MANAGEMENT DISTRICT 

AIR MONITORING PROGRAM

General Program

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of the Mojave 

Desert Air Quality Management District (MDAQMD). All information contained within this 

report section reflects the status of the MDAQMD monitoring program at the time of audit (April 

17, 2015). MDAQMD is currently part of CARB PQAO; this audit included an agency-specific 

assessment of network design, field operations, data handling, quality assurance and quality 

control procedures. As Mojave staff operate the Antelope Valley Air Quality Management 

District (AVAQMD) regulatory ambient air monitoring network, findings noted in this report 

may apply to both agencies.

Agency Description

MDAQMD is the local agency responsible for controlling emissions from stationary sources of 

air pollution within the desert portion of San Bernardino County and the Palo Verde Valley area 

of Riverside County. Covering over 20,000 square miles of primarily desert terrain, MDAQMD

implements air quality programs required by state and federal mandates, enforces rules and 

regulations based on air pollution laws, and educates business and residents about their role in 

protecting air quality. The Governing Board consists of appointed representatives from each 

incorporated city within its jurisdiction, the County Supervisors whose districts are contained in 

whole or in part within its jurisdiction, and one public member appointed by the Governing 

Board. The Governing Board appoints an Air Pollution Control Officer who serves as the 

Executive Officer of the district and controls day to day operations. S/he is also the enforcement 

officer for the applicable laws, rules, and regulations. MDAQMD funds its activities through a 

variety of fees, including but not limited to permit fees charged to local business and industry 

subject to its rules and regulations. MDAQMD is organized into a variety of divisions each with 

its own specific duties and responsibilities. The divisions primarily subject to this TSA are 

Ambient Air Monitoring and Information Systems Departments (Appendix C). 

EPA interviewed Christopher Collins, monitoring program manager; Aleksey Astakhov, station 

operator and repair technician; and Orlando Salinas, station operator and particulate laboratory 

technician. EPA visited four monitoring sites located in Lancaster, Victorville, Lucerne Valley, 

and Phelan. CARB staff shadowed EPA during this TSA.

MDAQMD staff were very accommodating and worked with EPA to explain procedures and 

obtain the documentation requested. 

Network Management

At the time of the TSA, the ambient air monitoring network operated and maintained by

MDAQMD staff consisted of six ambient air monitoring sites within MDAQMD and one site 
Final Report - August 2016

30

within AVAQMD that utilized a variety of air pollutant instruments. Table 5 provides 

information on the seven sites operated by MDAQMD.

Table 5. Sites, Pollutants Monitored, and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Barstow 06-071-0001 CO

NO/NO2/NOx

PM10 – Continuous

O3

MDAQMD

Hesperia 06-071-4001 PM10 – Continuous

O3

MDAQMD

Lucerne Valley 06-071-0013 PM10 – Continuous MDAQMD

Phelan 06-071-0012 O3 MDAQMD

Trona 06-071-1234 H2S

NO/NO2/NOx

PM10 – Continuous

O3

SO2

MDAQMD

Victorville 06-071-0306 CO

NO/NO2/NOx

PM10 – Continuous

PM2.5 – Filter (1in6)

SO2

O3

MDAQMD

Lancaster106-037-9033 CO

NO/NO2/NOx

PM10 – Continuous

PM2.5 – Filter (1in6)

O3

MDAQMD1

1 The Lancaster site is operated by MDAQMD, but is located within AVAQMD. 

A full description of the network is provided in the CARB Annual Monitoring Network Plan. At 

the time of the audit, CARB ran an additional site in MDAQMD (Blythe), and the National Park 

Service maintained a site in MDAQMD as well (Joshua Tree). See Appendix C for a map 

depicting the monitoring network within the Mojave Desert Air Basin.

MDAQMD submits an annual review of the ambient air monitoring stations located in its 

jurisdiction to CARB. The stations house monitoring instruments that measure ambient levels of 

gaseous and particulate (solid and liquid aerosol) air pollutants. Most of the stations also collect 

meteorological data. The annual monitoring network plan is submitted to AQPSD, which 

compiles information and sends an annual network plan on behalf of multiple air agencies within 

the PQAO to EPA. When MDAQMD seeks to make changes to its network outside the annual 

monitoring network plan process, it does so in agreement with CARB in order to resolve 

potential issues. After changes are approved, CARB submits its request to EPA per 40 CFR 

58.14. The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for PM10, 
Final Report - August 2016

31

PM2.5 and O3 were being met for MDAQMD; additional details can be found in the CARB 

annual monitoring network plan and the EPA response letter.

Field Operations

The Lead Air Quality Instrument Technician (hereafter referred to as “Lead Tech”) and the Air 

Quality Instrument Technician (hereafter referred to as “Instrument Tech”) handled all the dayto-day operations for their assigned sites, including instrument installation, repair and 

maintenance, sample collection, first-level data review, some quality control activities, and 

general field site maintenance. 

Technicians were responsible for three to four sites and visited each site at least two times per 

month. The monitoring stations were set up to perform automated nightly internal zero/spans 

(IZS). PM10 and PM2.5 flow-rate verifications were performed by both the Lead Tech and the 

Instrument Tech for their assigned sites. Corrective actions and repair/maintenance were dealt 

with on a case-by-case basis. 

In order to establish independence or peer cross verification, the Lead Tech and Instrument Tech 

reviewed each other’s work.

Laboratory Operations

At the time of the audit, MDAQMD operated a PM laboratory in which PM2.5 filters were

analyzed for comparison to the NAAQS. The results of the analyses were uploaded to AQS. The 

procedures for the lab were delineated in the SOP created by CARB. The file can be located 

here: http://arb.ca.gov/airwebmanual/aqsbdocs1/v2apxaj.pdf

Data Management

Data management generally involves data collection, the data validation process, and a data 

management system. According to MDAQMD, their data management system followed the 

process described below at the time of the audit. 

Data management followed two separate tracks, one for lab data and one for continuous (i.e., 

non-lab) data. 

The filter cycle included the following:

1. Laboratory conditioning.

2. Schedules, assignments and sample tracking.

3. Laboratory data and QC imported into ACCESS database.

4. ACCESS stored, managed data, and performed calculations.

5. Data were prepared to be loaded into AQS.

6. AQS report generated to support data certification.

The continuous gas and PM data cycle included the following:

1) Concentrations were measured by regulatory ambient air analyzers.
Final Report - August 2016

32

2) Each analyzer transmitted its data to a data logger, where the data were date- and timestamped, collated, organized and recorded.

3) The data were collected from the data loggers via computer software into the server, and 

stored in a central database.

 AirVision software pulled data from data loggers on an hourly basis.

 AirVision stored raw data on servers, and one-hour readings are available for QA/QC.

4) The data were reviewed using peer-review process.

 Level I, II and III checks were performed on a saved copy of raw data.

 Level I checks were made by assigned technician.

 Level II checks were made by non-assigned technician or supervisor.

 Level III checks were made by supervisor and non-assigned site technician.

5) Data went through a QA/QC process.

 Performance evaluations were based on audits performed by MDAQMD staff and 

external auditors.

 Internal checks and audits performed by Lead Tech and Instrument Tech were

reviewed by manager.

 External audits were performed by CARB.

6) Data were archived and made available to the public via the MDAQMD website.

 Hourly files were stored.

 QA/QC data files were available to the public upon request.

 Certified and historical data were available through AQS.

QA/QC

40 CFR Part 58 Appendix A Section 2 requires that each ambient air monitoring PQAO 

conforms to certain quality management practices. These include:

1) Having a documented quality system that meets EPA requirements for QMPs and 

QAPPs.

 As part of a PQAO, MDAQMD follows the CARB QMPs and QAPPs.

2) Having a quality management function that is independent of air monitoring operations.

 As part of a PQAO, CARB performs independent QA checks on the MDAQMD 

system.

3) Developing or adopting DQOs, or equivalent systematic planning procedures, for all 

monitoring programs.

 MDAQMD follows the CARB DQOs for its monitoring program.

4) Participating in National Performance Evaluation Programs (NPEP), which consist of 

performance audits used to independently determine program adequacy, national 

monitoring network performance, and national consistency.

 As part of a national network MDAQMD participates in NPAP for gaseous 

instruments and the NPEP for particulate samplers. The NPAP and NPEP audits are 

performed by EPA Region IX contractors.

 EPA performed the last NPAP audit on 11/21/2014 on the Trona site (06-071-1234). 

The O3, SO2, NO/NOx instruments were audited.

5) Participating in TSAs by EPA at a frequency of every three years or less.
Final Report - August 2016

33

 As part of a national network CARB is subject to TSAs performed by EPA; EPA may 

audit MDAQMD as part of the TSA.

6) Using certified reference materials to standardize monitoring equipment.

 Equipment is calibrated to Standards certified by CARB.

 All standards are certified every year.

 All equipment is calibrated at least once a year using the certified standards.

How QA/QC was structured for MDAQMD

MDAQMD conducted most QC activities and relied on CARB for support for QA audits. QC 

activities conducted by MDAQMD included:

 One-point QC checks for gaseous monitors that occur nightly (nightly calibrations).

 Monthly required monthly flow verifications for PM10 and PM2.5. 

CARB conducted the following:

 QA activities: gaseous annual performance evaluations.

 Semi-annual flow rate audits for PM10 and PM2.5.

 Meteorological audits.

 Gaseous and flow standard verifications and certifications. 

EPA contractors visited one site on 11/21/2014 to perform a NPAP audit on the following 

instruments:

 O3

 SO2

 NO/NOx

EPA contractors also performed NPEP audits on the PM monitors.
Final Report - August 2016

34

OVERVIEW OF NORTH COAST UNIFIED AIR QUALITY MANAGEMENT 

DISTRICT AIR MONITORING PROGRAM

General Program

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of North Coast 

Unified Air Quality Management District (NCUAQMD). All information contained within this 

report section reflects the status of the NCUAQMD monitoring program at the time of audit 

(May 26-28, 2015). 

NCUAQMD is currently part of CARB PQAO; this audit included an agency-specific 

assessment of network design, field operations, data handling, quality assurance and quality 

control procedures.

The NCUAQMD is located in the far northwestern portion of California. It is comprised of three 

counties: Del Norte, Humboldt, and Trinity, which together cover 7,753 square miles. It is 

bordered on the west by the Pacific Ocean and extends from the Oregon border south 

approximately 140 miles to the Mendocino County line in California. Eureka, the county seat of 

Humboldt County, is 284 miles north of San Francisco, 466 miles south of Portland, Oregon and 

on the coast of the Pacific Ocean. The area is made up of varied terrain, from coastal wetlands to 

rugged mountains.

Supervisors from each of the three counties serve on the NCUAQMD board along with a city 

representative from one of the cities within NCUAQMD’s jurisdiction. NCUAQMD primarily 

funds its activities through a combination of state and federal grants, permit fees charged to 

businesses/industries, and vehicle registration fees. 

NCUAQMD is responsible for ensuring federal and state ambient air quality standards are 

achieved and maintained within the county. At the time of the TSA, NCUAQMD was organized 

into four main areas, each of which plays a role in protecting the public from the adverse impacts 

of air pollution. The NCUAQMD Division that was the subject of this TSA was the Air 

Monitoring Division. The Air Monitoring Division included two staff members, both of whom 

were Air Pollution Specialists. The Air Monitoring Division was managed by the District Air 

Pollution Control Officer.

EPA staff interviewed NCUAQMD management and staff and visited two monitoring sites: 

 Eureka-Jacobs in Humboldt County (AQS ID 06-023-1004), and 

 Eureka-Humboldt Hill in Humboldt County (AQS ID 06-023-1005).

NCAQMD management and staff were accommodating, making themselves available for several 

interviews, procedural reviews and monitoring site visits. Management and staff interviewed 

were:

 Brian Wilson, Air Pollution Control Officer

 Wendy Caruso, Air Pollution Specialist

 Randy Dickau, Air Pollution Specialist 
Final Report - August 2016

35

Overall, the monitoring staff, despite its small size, demonstrated good knowledge and showed 

dedication to improving air monitoring operations. In the attached findings, EPA highlights 

several areas for suggested improvement. 

Network Management

At the time of the TSA, the ambient air monitoring network in NCUAQMD consisted of four 

ambient air monitoring sites that utilized a variety of air pollutant instruments. A full description 

of the network is provided in NCUAQMD’s Annual Monitoring Network Plan. Table 6 lists

sites, pollutants monitored, and operating agency.

Table 6. Site, Pollutants Monitored and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Eureka-Jacobs 06-023-1004 O3, NO2, SO2, CO, PM10, PM2.5 NCUAQMD

Eureka-Humboldt 

Hill 06-02301005 O3, NO2, SO2, CO, PM2.5 NCUAQMD

Crescent City 06-015-0006 PM2.5 NCUAQMD

Weaverville 06-105-0002 Non-FEM PM2.5 NCUAQMD

NCUAQMD submits its annual monitoring network plan directly to EPA. If a local district 

within the CARB PQAO seeks to make changes to its network outside the annual monitoring 

network plan process, the district should first work with CARB to resolve potential issues and 

then submit its request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for SLAMS were

being met by NCUAQMD; additional details can be found in NCUAQMD’s annual monitoring 

network plan and EPA’s response letter. 

Field Operations

Field technicians handle all day-to-day operations including: instrument installation, repair and 

maintenance; sample collection; first-level data review; some quality control activities; and 

general field site maintenance. At the time of the audit, NCUAQMD had one primary field 

technician who was responsible for visiting all four stations. The two gaseous stations in Eureka 

were visited multiple times each week. The two particulate-only sites were visited roughly once 

per month. The monitoring stations operated by the district were set up to perform automated 

zero/one point precision checks every third day. Automated span checks were run once per week. 

PM10 and PM2.5 flow verifications were performed monthly by NCUAQMD field technicians. 

Corrective actions and repair/maintenance were dealt with on a case-by-case basis. Generally, 

equipment repairs were performed by NCUAQMD personnel. On rare occasions, instruments 

may be sent to the vendor or CARB for repair.
Final Report - August 2016

36

Laboratory Operations

NCUAQMD does not operate a PM laboratory; instead it was sending PM2.5 filters to BAAQMD 

for gravimetric analysis. NCUAQMD used the CARB laboratory for the following analyses: O3

primary standard verification and calibrator mass flow controller certifications.

Data Management

Data management generally involves data collection, the data validation process, and a data 

management system. Data management at NCUAQMD followed two separate tracks, one for 

continuous (i.e., non-lab) data and one for lab data.

Continuous data were entered into the DR DAS© management system. Data were collected at 

the monitoring site using a central computer, and polled from the main office air monitoring 

server hourly. The raw data were backed up nightly and never overwritten. The data were edited 

and validated following the procedures in NCUAQMD’s data management and validation SOP. 

First level data review was done by the primary field technician Randy Dickau, who took note of 

any maintenance or observable events from the field. Second level data review was conducted 

from the office by Air Pollution Specialist Wendy Caruso, who graphed the data, noted 

problems, determined causes, and validated the data as warranted. Electronic records were kept 

to document the data validation steps. Once validated by NCUAQMD, an AQS file was 

generated and sent to CARB for AQS uploading. CARB would then run an automated final 

check on the continuous data prior to AQS uploading, and contact NCUAQMD to address any 

potential problems that arose during the automated check.

BAAQMD assisted NCUAQMD with PM2.5 filter data analysis at the time of the audit. 

BAAQMD shipped conditioned filters with CoC forms to NCUAQMD approximately every two 

to three weeks. NCUAQMD deployed filters to field instruments for data collection and then 

shipped filters back to BAAQMD for gravimetric analysis. Any field notes for data validation 

consideration were documented on the CoC forms by NCUAQMD and BAAQMD. Once 

analysis was completed by BAAQMD, raw weights (in spreadsheet format) and filter conditions 

were emailed back to NCUAQMD where the information was converted into a format 

appropriate for AQS. An AQS file was generated and sent to CARB for AQS uploading. 

QA/QC

NCUAQMD conducted some QA/QC activities and relied on CARB to support others. QA/QC 

activities conducted by NCUAQMD included one-point QC checks for gaseous monitors, and 

required flow verifications for PM10 and PM2.5. CARB conducted the following QA/QC 

activities: gaseous annual performance evaluations, semi-annual flow rate audits for PM10 and 

PM2.5, meteorological equipment audits, multi-point instrument calibrations, and photometer 

certification. EPA contractors visit the sites on occasion, conducting comparison monitoring 

using EPA field equipment.
Final Report - August 2016

37

OVERVIEW OF SAN LUIS OBISPO COUTY AIR POLLUTION CONTROL DISTRICT 

AIR MONITORING PROGRAM

General Program

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of San Luis 

Obispo County Air Pollution Control District (SLOCAPCD). All information contained within 

this report section reflects the status of the SLOCAPCD monitoring program at the time of audit 

(August 26-27, 2015). 

SLOCAPCD is currently part of CARB PQAO; this audit included an agency-specific 

assessment of network design, field operations, data handling, quality assurance and quality 

control procedures. 

At the time of the audit, the Board of Directors of the SLOCAPCD had twelve members 

comprising all five County Board of Supervisors and a representative from each of the seven 

incorporated cities located within the county. SLOCAPCD is responsible for ensuring federal 

and state ambient air quality standards are achieved and maintained within the county. State law 

assigns to local districts the primary responsibility for control of air pollution from stationary 

sources. This is typically accomplished through the adoption and implementation of rules and 

regulations. SLOCAPCD is also responsible for the inspection of stationary sources, monitoring 

of ambient air quality, development and updating of attainment plans, and maintenance of the 

emission inventory. SLOCAPCD funds its activities through a combination of permit and 

authority to construct fees, vehicle registration fees, property taxes, and a small portion of State 

and Federal aid. 

When the TSA was conducted, SLOCAPCD was organized into three main areas, each of which 

plays a role in protecting the public from the adverse health impacts of air pollution: 1) Program 

Administration and Management, 2) Engineering & Compliance and 3) Planning, Monitoring & 

Outreach. The SLOCAPCD Division that was the subject of this TSA is the Planning, 

Monitoring and Outreach Division (PMOD).

At the time of the TSA, PMOD was comprised of one Division Manager (Aeron Arlin Genet), 

five Planning Air Quality Specialists and two Monitoring Air Quality Specialists (Jaime 

Contreras and Karl Tupper). The operations of the SLOCAPCD eight air monitoring stations 

were divided equally amongst the Monitoring Air Quality Specialists. This includes field 

operations, data management and QA/QC.

EPA staff interviewed SLOCAPCD management and staff and visited four monitoring sites 

located in San Luis Obispo County: 

 CDF (AQS ID 06-079-2007),

 Mesa2 (AQS ID 06-079-2004),

 Nipomo Regional Park (AQS ID 06-079-4002), and

 Red Hills (AQS ID 06-079-8005).
Final Report - August 2016

38

SLOCAPCD managers and staff were very accommodating, making themselves available for 

many interviews, procedural reviews and monitoring site visits. Management and staff 

interviewed were:

 Aeron Arlin Genet – Planning, Monitoring and Outreach Division Manager

 Karl Tupper – Monitoring Air Quality Specialist

 Jaime Contreras – Monitoring Air Quality Specialist

Overall, the monitoring staff was very dedicated and knowledgeable. Their internal tracking 

forms, logbooks, and training forms were exemplary. 

Network Management

At the time of the TSA, the ambient air monitoring network in SLOCAPCD consisted of ten

ambient air monitoring sites that utilized a variety of air pollutant instruments. Eight sites were

operated by SLOCAPCD and two by CARB. A full description of the network is provided in 

SLOCAPCD’s annual monitoring network plan. Table 7 lists sites, pollutants monitored, and 

operating agency.

Table 7. Sites, Pollutants Monitored and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Atascadero 06-079-8001 (old)

06-079-8002 (new)

O3, NO2, PM10, 

PM2.5

SLOCAPCD

Carrizo Plain 06-079-8006 O3 SLOCAPCD

CDF 06-079-2007 PM10, PM2.5 SLOCAPCD

Grover Beach 06-079-2001 NONE, met only SLOCAPCD

Morro Bay 06-079-3001 O3, NO2 SLOCAPCD

Nipomo Regional Park 06-079-4002 O3, NO2, PM10 SLOCAPCD

Mesa2 06-079-2004 SO2, PM10, PM2.5 SLOCAPCD

Red Hills 06-079-8005 O3 SLOCAPCD

Oso Flaco 06-079-9001 PM10 SLOCAPCD

San Luis Obispo 06-079-0005 O3, PM10, PM2.5 CARB

Paso Robles 06-079-2006 O3, PM10 CARB

SLOCAPCD submits its annual monitoring network plan directly to EPA. If a local district 

within the CARB PQAO, such as SLOCAPCD, seeks to make changes to its network outside the 

annual monitoring network plan process, the district first works with CARB to resolve potential 

issues and then submits the request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for PM10, PM2.5

and O3 were being met for San Luis Obispo County; additional details can be found in 

SLOCAPCD’s annual monitoring network plan and EPA’s response letter. 

Field Operations

Field technicians handle all day-to-day operations, including instrument installation, repair and 

maintenance, data collection, first-level data review, all quality control activities, and general 
Final Report - August 2016

39

field site maintenance. Field technicians were responsible for eight SLAMS sites and visited

each site according to a biweekly or monthly schedule and whenever operational issues arise. 

The monitoring stations operated by SLOCAPCD were set up to perform automated nightly 

zero/precision/spans (ZPS). PM10 and PM2.5 flow verifications were performed by field 

technicians in a biweekly basis. 

Corrective actions and repair/maintenance were dealt with on a case-by-case basis. Generally, 

equipment repairs were performed by SLOCAPCD; occasionally repairs were performed by the 

instrument manufacturer. Detailed documentation of repairs was logged in the station’s 

electronic logbooks and separately entered in the Errata electronic logbook.

Responsibilities for the operation of the eight monitoring stations were assigned to two Air 

Quality Specialists. Each specialist was responsible for the complete QA/QC suite of data 

production and collection; data verification of each parameter was a complementary process that 

involved both specialist and supervising manager. QA steps include data validation and reporting 

to AQS.

Laboratory Operations

SLOCAPCD does not operate a PM laboratory. At the time of the audit, SLOCAPCD operated

continuous BAM 1020 PM10 and PM2.5 samplers, thus laboratory filter weight analysis was not 

required. SLOCAPCD used EPA laboratories for the certification the O3 photometer primary 

standard verification. Scott-Marrin gas manufacturer certifies EPA protocol gas cylinder blends. 

Mesa-Labs is the manufacturer and certifier for NIST-traceable flow transfer standards. Vaisala 

is the manufacturer and certifier for NIST-traceable relative humidity transfer standard. Met One 

Inc. is the manufacturer and wind tunnel verifier/certifier for sonic anemometers; SLOCAPCD

no longer uses mechanical cup-and-vane systems. Until recently, ICL Laboratories was the 

certifier for the NIST-traceable glass mercury thermometer transfer standard. Starting in 2015 

and pursuant to the Roles and Responsibilities agreement with CARB, SLOCAPCD has begun 

utilizing the CARB Standards Laboratory for certification of transfer standards; to date, the O3

transfer standard and ambient temperature sensor have been CARB certified. For major 

instrument repairs, the SLOCAPCD contracts the services of the instrument’s manufacturers.

Data Management

Data management generally involves data collection, the data validation process, and a data 

management system. 

SLOCAPCD utilized the AirVision data management system from Agilaire Corp. This is a 

program that resided in a SLOCAPCD server and performed routine data polling from each 

station’s data loggers. The data polling protocol was automated; each hour (or as needed) 

AirVision remotely collected data via telephone or cell phone modem connection. Each data 

packet from each site consisted of all the averaging methods including one-minute, five-minute, 

hourly averages and/or calibration verification responses. The polling program was a series of 

batch protocols that verified the complete transmission of data from the data logger to the server. 

AirVision and the data loggers automatically applied initial QA flags to the data. This initial 
Final Report - August 2016

40

automated flagging protocol allowed for an immediate preliminary data validation and was the 

base for a consistent evaluation of subsequent data validation for Levels II and III. The complete 

data validation protocol was described in Section 5 of the SLOCAPCD SOP.

QA/QC

SLOCAPCD conducted some QA/QC activities and relied on CARB to support others. QA/QC 

activities conducted by the SLOCAPCD included daily calibration verification at zero, precision 

and span, and semi-annual multi-point QC checks for gaseous monitors. The station gas diluters 

(calibrators) underwent quarterly flow verifications with a NIST-traceable flow transfer standard. 

The calibrators fitted with an O3 photometer were verified semi-annually with a Level III O3

transfer standard following the EPA six-by-six Transfer Standards for Calibration of Air 

Monitoring Analyzers for Ozone (EPA 454/B10-001). Continuous PM samplers underwent biweekly flow verifications for PM10 and PM2.5 using a NIST-traceable certified transfer standard 

that included ambient temperature and barometric pressure, quarterly full calibration verification 

and annual complete service and 72 hours zero test. For wind sensors, SLOCAPCD audited 

alignment and submitted all sonic anemometers to the manufacturer for annual service and wind 

tunnel calibration and certification.

CARB conducted the following QA/QC activities: gaseous annual performance evaluations and 

semi-annual flow rate audits for continuous PM10 and PM2.5, and ambient temperature sensors; 

siting criteria evaluation; residence time verification and inspects QC documentation verification 

for O3; and gaseous standard verifications and certifications. EPA contractors visited the sites 

annually (minimum) making flow calibration checks on the continuous PM monitors, as well as 

conducting PM coarse comparison using EPA FRM samplers.
Final Report - August 2016

41

FINDINGS

Program Area Finding Numbers EPA Contact

General G-Positive1-2; G1-G2 Audit Team, 

Mathew Plate

Network Management NM-Positive1 Gwen Yoshimura

Field Operations FO-Positive1-5; FO1-FO10; 

FO-Recommendation1

Mathew Plate, 

Gwen Yoshimura,

Dena Vallano

Data Management DM-Positive1-7; DM1-DM4

Gwen Yoshimura, 

Dena Vallano

QA Management QA-Positive1-5; QA1-QA2; 

QA-Recommendation1-2

Dena Vallano,

Mathew Plate, 

Gwen Yoshimura

Standards Laboratory SL-Positive1; SL1-SL8; 

SL-Recommendation1 Mathew Plate

PM Laboratory PM-Positive1-3; PM1-PM8 Mathew Plate

Pb Laboratory Pb1 Joe Eidelberg

AVAQMD & MDAQMD AV/MD-Positive1; AV/MD1-AV/MD24 Mathew Plate

NCUAQMD NCU-Positive1; NCU1-NCU15 Gwen Yoshimura

SLOCAQCD SLO-Positive1-2; SLO1-SLO5; SLORecommendation1 Dena Vallano
Auditor: All 

Final Report - August 2016

42

Finding # G – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: General

Finding:

A number of important projects were underway to improve the PQAO.

Description:

There were a number of activities underway that will further improve the CARB PQAO. Those

included as individual positive findings in this report were specifically discussed during the TSA. 

EPA commends CARB for addressing these important issues and working to establish 

procedures, processes, and infrastructure to support improvements to strengthen the CARB 

PQAO quality system.

EPA notes that some of the findings in the report may be able to be addressed through full 

implementation of one or more of the activities noted in the positive findings. While it is still 

appropriate for the findings to be included in the report, the positive findings indicate that CARB 

is aware of areas for improvement, and is already taking steps to address many of these issues.

References:

Suggested Actions to Address Finding:

NA
Auditor: All 

Final Report - August 2016

43

Finding # G – Positive Finding 2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: General

Finding:

CARB has been providing increased support to the CARB PQAO districts.

Description:

The CARB PQAO faces an ongoing challenge in ensuring consistency across its membership. 

Since the 2011 TSA, CARB has made substantial changes to improve consistency.

Districts noted improved support from and communication with CARB. Highlights include:

 Assistance from the PQAO liaisons on a number of issues, including facilitating 

resolution of data reporting questions, network changes, and instrument issues;

 CARB providing spare equipment for district use;

 PQAO trainings.

CARB has developed R&R documents with each district, offered data certification trainings, 

created an online document repository for QA documents, solicited feedback on documents such 

as the Annual Data Quality report and draft QA bulletins, continues to play an integral role on 

California Air Pollution Control Officers Association (CAPCOA) monthly monitoring calls, and 

assigns each PQAO liaison the task of visiting her/his districts on a regular basis. These are just 

some of the activities CARB has undertaken to improve communication and consistency across 

the PQAO.

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate 

Final Report - August 2016

44

Finding # G1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: General

Finding:

Not all districts have been fully attending trainings or identifying and requesting a sufficient 

level of technical support.

Description:

The CARB PQAO staff have worked diligently and have improved CARB PQAO-specific 

training significantly. This has benefitted the districts within the CARB PQAO and created more 

consistency in operations across the PQAO. However, especially in cases where the districts 

have very few staff and are more remotely located, districts may need CARB’s assistance to 

identify what additional trainings would benefit their staff. Districts should prioritize training so

that staff and managers receive the training required for proficient execution of their job 

responsibilities and meet their PQAO responsibilities.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Create a recommended training plan for: a) site operators, b) laboratory staff, c) QA/QC staff and 

management, and d) data validators. Share this training plan with the districts. 

Facilitate access to additional trainings by providing training resources, such as trainings offered 

online, training vendors, and other training opportunities as they arise.

As needed, work with individual districts to create a district-specific training plan.
Auditor: All

Final Report - August 2016

45

Finding # G2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: General

Finding:

Communication within the PQAO was not consistent. 

Description:

More effective communication between CARB and some PQAO districts could prevent 

monitoring network and data quality challenges. EPA found several instances where AQPSD had

not been notified of changes within a district. For example, when Mojave changed from filter 

PM10 instruments to continuous PM10 instruments, poor communication resulted in challenges 

handling AQS data. (See finding AV/MD7.) Districts were not consistently consulting CARB 

with QA or AQS questions. 

Although it is generally the responsibility of the districts to inform CARB of network 

adjustments (i.e., per the R&R) and to raise data-related questions to CARB, in its oversight role, 

CARB should have mechanisms in place to keep up-to-date on district network changes and to 

provide guidance to the districts. 

References:

40 CFR Part 58 Appendix A, Section 3:

3.1 Primary Quality Assurance Organization. A primary quality assurance organization is 

defined as a monitoring organization or a coordinated aggregation of such organizations that is 

responsible for a set of stations that monitors the same pollutant and for which data quality 

assessments can logically be pooled. Each criteria pollutant sampler/monitor at a monitoring 

station in the SLAMS network must be associated with one, and only one, primary quality 

assurance organization.

3.1.1 Each primary quality assurance organization shall be defined such that measurement 

uncertainty among all stations in the organization can be expected to be reasonably 

homogeneous, as a result of common factors. Common factors that should be considered by 

monitoring organizations in defining primary quality assurance organizations include:

(a) Operation by a common team of field operators according to a common set of 

procedures;

(b) Use of a common QAPP or standard operating procedures;

(c) Common calibration facilities and standards;

(d) Oversight by a common quality assurance organization; and

(e) Support by a common management, laboratory or headquarters.
Auditor: All

Final Report - August 2016

46

Suggested Actions to Address Finding:

Institute a system for the districts to inform CARB of relevant changes or questions, either 

directly or at general meetings. For example, the monthly CAPCOA call could be used to query 

local districts on any network adjustments or general questions.

Track what districts do not participate on regular calls and work with EPA to encourage regular 

communication.
Auditor: Gwen Yoshimura

Final Report - August 2016

47

Finding # NM – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Network Management

Finding:

The CARB PQAO has been working with EPA to create the CAMNAT database.

Description:

The CARB PQAO has a monitoring network that is operated both by the districts and CARB. 

Different operating agencies overlap across various geographic regions, and different parts of the 

PQAO create their own ANPs and network assessments, making it challenging to assess the 

network as a whole. Development of CAMNAT will improve determination of network 

adequacy and state-level planning.

References:

Suggested Actions to Address Finding:

NA
Auditor: Dena Vallano

Final Report - August 2016

48

Finding # FO - Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Staff were professional and helpful, and in most instances very knowledgeable regarding field 

operations. 

Description:

Specific examples of good practices observed during this audit include:

 CARB monitoring staff was actively engaged in improving the program, including 

monitoring systems and other data collection processes.

 Field technicians caught errors and made improvements with existing forms and 

documentation. 

 Field technicians developed excellent record-keeping, internal documentation and forms.

 The field technician in San Luis Obispo noted and addressed instrument issues promptly and 

maintained excellent record-keeping and documentation at the monitoring sites. 

 Where data issues were identified during data tracking, staff were particularly helpful in 

identifying the root causes of these data anomalies.

References:

Suggested Actions to Address Finding:

NA
Auditor: Dena Vallano

Final Report - August 2016

49

Finding # FO – Positive Finding 2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

The AMSS has developed an on-the-job training program for field technicians. 

Description:

Phil Wagner and Fernando Amador created an on-the-job training program for their most recent 

hire. This program could be expanded and adopted MLD-wide as a program for all CARB field 

technicians.

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

50

Finding # FO – Positive Finding 3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

A training database for field staff is under development. 

Description:

A 2011 TSA finding highlighted the need for keeping a training database. CARB was working to 

develop that training database for field technicians. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

51

Finding # FO – Positive Finding 4

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

CARB has been working with several manufacturers as they develop new instruments. 

Description:

CARB has been proactively working with instrument manufacturers: with MetOne to develop a 

modified, regulatorily-compliant SASS, with TSI on particle counters, and with MetOne on a 

black carbon sampler. Dedicating time to working with manufacturers on the front end increases 

the likelihood that the products, which typically take years to develop, will meet CARB’s 

particular needs when put on the market.

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

52

Finding # FO – Positive Finding 5

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

All regulatory PM2.5 BAM instruments within the PQAO were being converted to VSCCs.

Description:

All regulatory PM2.5 instruments operating with well impactor ninety-six (WINS) impactors were

being replaced within the entire PQAO. Replacing all WINS with very sharp cut cyclones 

(VSCC) will promote standardization and minimize potential bias in the system. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

53

Finding # FO1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Some field forms contained errors that were not properly identified or corrected in the initial 

form validation or subsequent peer reviews.

Description:

An example of what was found is highlighted in the BAM calibration form.

2014 version of form:

• The criterion on the form was misinterpreted as 0.02 LPM for each multipoint level. A 

criterion of 2% is consistent with regulation and guidance.

• There was an incorrect response to failing criteria. The calibrator either used data labeled 

as “failed” or rechecked until system passed.

• Flow is only evaluated against the Target and not the Display flow from the instrument.
Auditor: Mathew Plate

Final Report - August 2016

54

2015 Form Revision

 Recognizing that 0.02 was not achievable, the form was changed to a different, but 

incorrect, criterion of 0.2 LPM.

It was also noted on these forms that the term “fail,” as used for an as-is calibration, only 

indicates that recalibration is needed. However, criteria for potential impacts to pre-calibration 

data should also be established.

References:

40 CFR Part 50 Appendix L:

9.1.2 An authoritative flow rate standard shall be used for calibrating or verifying the 

sampler's flow rate measurement device with an accuracy of ±2 percent.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013, Appendix D
Auditor: Mathew Plate

Final Report - August 2016

55

Suggested Actions to Address Finding:

Review all field forms including calculations and criteria used and make corrections as needed.

Institute version control and additional peer review evaluation to ensure that forms are not 

altered.
Auditor: Gwen Yoshimura

Final Report - August 2016

56

Finding # FO2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Staff were not required to demonstrate proficiency when assigned new tasks.

Description:

EPA observed several cases where newly assigned instrument operations or calibration tasks 

were improperly executed, resulting in data loss. Data loss affects data quality. For example, 

see issues stated in FO1.

As noted in FO – Positive Finding 2, Phil Wagner and Fernando Amador implemented an onthe-job training program with their most recent hire, which could be expanded and adopted 

MLD-wide.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Expand and formalize on-the-job training programs and demonstrations of proficiency.
Auditor: Mathew Plate

Final Report - August 2016

57

Finding # FO3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

QC check gas is not delivered externally to the instruments at the San Luis Obispo, Mojave 

Poole, and other O3-only sites, as the 400E/T400 IZS O3 monitor delivers quality control (QC) 

check gas to the monitor internally.

Description:

CARB was performing O3 precision checks at the San Luis Obispo, Mojave Poole, and other 

sites using an API 400E/T400 internal zero/span (IZS) monitor option which is wholly internal 

to, although independent of, the monitor’s analytical bench. Because the plumbing is internal, 

the precision check CARB was performing at these sites did not include any check of the 

external lines, which could be experiencing scavenging or other issues affecting O3 detection. 

In addition, the O3 generator was not calibrated with a standard that was independent from the 

instrument.

This practice is potentially inconsistent with the EPA regulatory requirements for precision

and is not how CARB performs precision checks at multi-pollutant sites. 

References:

40 CFR Part 58 Appendix A Section 3.2.2.4 states “...and as much of the ambient air inlet 

system as is practicable”. 

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Transfer Standards for Calibration of Air Monitoring Analyzers for Ozone, EPA-454/B-13-

004, October 2013.

EPA Traceability Protocol for Assay and Certification of Gaseous Calibration Standards, 

May 2012.

Suggested Actions to Address Finding:

If CARB continues to use IZS systems to perform precision checks, CARB should 

demonstrate equivalency of this methodology, functionally and statistically, according to 

EPA’s Traceability Protocol for Assay and Certification of Gaseous Calibration Standards

such that the method is shown to meet EPA’s requirements, and is shown to provide QA 

equivalent to that at non- O3-only CARB sites.

Perform quarterly calibrations of O3 generators, as recommended in EPA guidance.
Auditor: Mathew Plate

Final Report - August 2016

58

Finding # FO4 [Finding FO13 from 2011 TSA]

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

MLD staff were not formally documenting the quality of zero air being used in the program.

Description:

Zero air scrubbers are used in place of certified zero air for instrument calibrations. This is a 

common, acceptable practice. Because zero air is used to generate the zero point and the 

calibration mixes, it must be treated as a standard. Therefore, zero air scrubber maintenance 

and verification must be documented.

During the 2015 TSA, MDL North appeared to have different practices than MDL South with 

respect to documenting the quality of zero air.

Since this was observed during the 2015 TSA, it is included as a finding in this report.

Because this finding was also a finding in the 2011 TSA (Finding FO13), documentation 

submitted to address this 2015 TSA finding will also address the 2011 TSA finding.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Document the quality of zero air on a periodic basis for all of MLD, at least annually or after 

any maintenance is performed.
Auditor: Mathew Plate

Final Report - August 2016

59

Finding # FO5

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

The field SOPs did not reflect many changes made to the program over the past several years.

Description:

Based on interviews with the field technicians and calibrators, it was clear that the lack of SOP 

and/or updated SOPs was affecting operations and potentially data quality. In some cases, the 

field technicians had been trained to do tasks; however, there were no SOPs or only inadequate 

SOPs available to reference for some infrequently performed operations. EPA noted this was 

the case specifically for CARBLogger, DMS, and BAM operations.

CARB management did note that there are plans for developing SOPs/user guides for 

CARBLogger and DMS.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Establish a schedule for updating SOPs.

Provide field technicians with work aids until SOPs are finalized.
Auditor: Dena Vallano

Final Report - August 2016

60

Finding # FO6

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Data loss has occurred at some sites due to CARBLogger malfunctions and lack of site 

operator awareness regarding how data can be recovered.

Description:

CARBLogger malfunctions have led to data loss at the San Luis Obispo and Paso Robles sites. 

During the TSA, it was observed that there were missing data points due to CARBLogger 

downtime from malfunctions or program updates on 15 separate days during the 2013-2014 

data record. This cursory review identified several missing hourly values per affected day at 

these two sites. The missing hourly values were not associated with qualifier codes in AQS 

(See Finding DM3). Given the limited extent of the TSA and the large number of sites within 

the CARB network, EPA was unable to determine the extent of this issue beyond the 

aforementioned sites. While data completeness for the audited sites met requirements, the 

instrument downtime and associated data loss could lead to vulnerabilities in overall data 

integrity and defensibility. 

Based on the site operator interview, troubleshooting as a result of a malfunction often 

required that the operator manually reboot the CARBlogger. The operator was not aware of a

backup system to recover lost data when the CARBLogger went down, which resulted in 

unrecoverable missing data points. The operator also indicated during the interview that these 

malfunctions may be a result of running the CARBLogger instruments beyond the expected 

life cycle and past the time when support from the manufacturer is available.

References:

State of California Air Resources Board, Air Monitoring QA, Vol. 5, Corrective Action 

Notification SOP (http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf)

Good Automated Laboratory Practices, Principles and Guidance to Regulations For Ensuring 

Data Integrity In Automated Laboratory Operations, EPA, Section 2-1, August 1995. 

Suggested Actions to Address Finding:

Train operators on the relevant SOPs that outline the recommended protocols and ways to 

recover data during these events. Implement a formal tracking process for addressing suspect 

or missing data, instrument problems, or other problems observed in the field. This process 

should include formal documentation so that the event can be reconstructed and reviewed to 
Auditor: Dena Vallano

Final Report - August 2016

61

prevent recurrence. If this is the type of issue that could be addressed through a CAN, ensure 

that station operators are aware that they can issue CANs for these issues.

Establish and effectively implement a backup system ready for field deployment in case of 

instrument breakdown. Compile a list of equipment replacement needs in the event that funds 

become available (from EPA or other funding sources). Budget for equipment replacement 

according to the life expectancy of the dataloggers.
Auditor: Gwen Yoshimura

Final Report - August 2016

62

Finding # FO7

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Filter data loss has resulted in several of the CARB-operated sites missing the 75% data 

completeness requirement for certain quarters.

Description:

EPA reviewed data from 2013 and 2014 for some of the CARB-operated sites running filterbased monitors. The following sites had quarters with less than 75% data completeness:

Yuba City PM2.5 POC1 – 69% data completeness, 2014 Q3.

- R&P 2025 sequential sampler. Issues with sampler not running/shuttle error resulting 

in significant data loss. 

Yuba City PM10 POC1 – 73% data completeness, 2013 Q3. (Note: this monitor has an AQS 

close date of June 28, 2014.)

- GMW HiVol sampler. Issues with time being out, scheduled but not collected, flow 

rate out.

Oildale PM10 POC2 – 73% data completeness, 2013 Q2. (switching to PM10 BAM) 

- GMW HiVol sampler. Issues with timer malfunctions, filter contamination (dirt), 

Dickson recorder malfunctions, burned brushes.

Bakersfield-California Ave PM10 POC1 – 53% data completeness, 2013 Q1.

- GMW HiVol sampler. Issues with filter leaks, samples scheduled but not collected, 

timer malfunction/sample time out of limits, filter tear.

PM2.5 and PM10 data completeness requirements call for 75% data completeness per quarter. 

Collecting less than 75% data per quarter could result in several years of invalid design values.

EPA recognizes that CARB is taking steps to replace filter-based instruments with continuous 

instruments in many locations within their network. For the remaining filter instruments, 

meeting data completeness continues to be a challenge.

References:

40 CFR part 50 Appendix K

40 CFR part 50 Appendix N

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 
Auditor: Gwen Yoshimura

Final Report - August 2016

63

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Continue to retire unreliable filter-based instruments. For those filter-based instruments

remaining in the network, review recent year’s data and determine if there are additional 

actions that, if taken, could minimize data loss.
Auditor: Gwen Yoshimura

Final Report - August 2016

64

Finding # FO8

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

In one instance, a field technician was not aware of the correct schedule for taking make-up 

samples on filter-based instruments.

Description:

Implementation of SOPs and CARB technical bulletins is necessary to ensure data quality, 

data defensibility, and compliance with regulatory requirements.

EPA noticed that PM10 filter samples were collected on an irregular basis at a site, i.e., not on 

the scheduled sample days and not on days that would count as make-up samples. When 

asked, the field technician did not know the correct protocol for collecting make-up samples. 

The field technician’s manager pointed out that there was a printed copy of the technical 

bulletin on hand at the site that explains the protocol for collecting make-up samples. This was 

a PM10 site that had recorded exceedances, and was an important site for making air planning 

decisions.

All site operators interviewed during the TSA were generally aware of the technical bulletins 

and SOP updates. They said they receive emails when new procedures or requirements are 

released, and significant changes are discussed at staff meetings.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Section 4, 5)

Suggested Actions to Address Finding:

Create a method for verifying that site operators understand the information contained in 

technical bulletins and SOP updates, and determine whether additional training is appropriate.

Take steps to support high data capture at critical sites.
Auditor: Dena Vallano

Final Report - August 2016

65

Finding # FO9

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

Recertification of sensor probes was not based on the recommended instrument 

manufacturer’s date of recalibration. 

Description:

The pressure and temperature sensor probes at the Paso 

Robles and San Luis Obispo stations missed annual 

recertification. CARB maintained that the 

certification’s effective date is upon installation rather 

than the recommended instrument manufacturer’s date 

of recalibration. For certification and recertification of 

standards, particularly in regard to flow-rate standards,

it is EPA’s position that the certification date is the date 

the instrument is officially certified and not the date 

placed into service. This includes both new purchases 

as well as re-certifications. 

References:

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambient Air Quality 

Monitoring Program, EPA-454/B-13-003, May 2013 

QA Eye, Office of Air Quality Planning and Standards, June 2014 

Suggested Actions to Address Finding:

Use instrument manufacturer’s date entered on the certificate of calibration to plan for the next 

annual certification date.
Auditor: Dena Vallano

Final Report - August 2016

66

Finding # FO10

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

The San Luis Obispo monitoring station did not have an adequate guard rail to assure safety 

during routine operations.

Description:

The San Luis Obispo monitors are located at the top of a roof ~10m from the ground with no 

guardrail to assure safety for technicians servicing roof monitors, meteorological sensors and 

probes. This is also unsafe for any auditors who must access the roof.

Since the time of the audit, CARB has been working to address safety concerns at this site.

References:

40 CFR 58, Appendix E

Suggested Actions to Address Finding:

Make corrective action a priority for stations that need guardrails for health and safety of 

monitoring and auditing staff. 

Evaluate the safety of monitoring locations being used across the monitoring network and 

install or re-enforce needed infrastructure. 
Auditor: Gwen Yoshimura

Final Report - August 2016

67

Finding # FO – Recommendation1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Field Operations

Finding:

The freezer used to store filters at Bakersfield-California was not kept secure and did not have 

a temperature gauge.

Description:

PM2.5 filters must be kept below a specific temperature. The site operator had been 

periodically checking the temperature of the freezer with a temperature gauge, and the filters 

had temperature strips. However, there was no permanent temperature gauge for the site 

operator to reference.

Samples should be kept in a secured location to minimize opportunity for contamination or 

alteration of the sample.

References:

40 CFR 50 Appendix L, Section 10.13

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Section 8)

Quality Assurance Guidance Document 2.12. Monitoring PM2.5 in Ambient Air Using 

Designated Reference or Class I Equivalent Methods. U.S. EPA, National Exposure Research 

Laboratory. Research Triangle Park, NC, November 1988 or later edition. (Section 8)

Suggested Actions to Address Finding:

Secure the freezer with a lock and purchase and install a temperature gauge.
Auditor: Gwen Yoshimura

Final Report - August 2016

68

Finding # DM – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB installed a new data management system.

Description:

Since the 2011 TSA, CARB completed installation of a new data management system. The new 

DMS allows for improved data flow from the field, easier data visualization and editing. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

69

Finding # DM – Positive Finding 2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was creating a new user guide/SOP for DMS.

Description:

CARB was in the process of creating a new user guide/SOP for DMS, specifically for users 

within the PQAO. When CARB started using DMS, Sonoma Technology, Inc. provided a user 

guide, and four training modules. CARB was working to consolidate these materials into a more 

user-friendly guide. This is extremely important given the importance of DMS in the data 

management process.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

70

Finding # DM – Positive Finding 3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was creating an SOP for CARBLogger.

Description:

In order for data to be successfully collected and reported in the field, it is essential that 

CARBLogger function properly and be used correctly. To improve field technician’s knowledge 

of this system, CARB was creating an SOP.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

71

Finding # DM – Positive Finding 4

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was upgrading DMS so that codes used in DMS match those used in AQS.

Description:

The latest version of DMS did not match DMS codes to AQS codes. CARB was working to 

upgrade DMS to allow DMS codes to map to AQS codes. This would improve the transfer of 

codes from DMS to AQS.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

72

Finding # DM – Positive Finding 5

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was creating guidance on minimum value reporting for PM instruments.

Description:

In the past, districts within the PQAO had different policies concerning how to report minimum 

values for PM instruments. In order to address questions and reduce inconsistencies across the 

PQAO, CARB was creating a QA bulletin that addresses how to report minimum values from 

PM instruments.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

73

Finding # DM – Positive Finding 6

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was working with the districts to ensure that all regulatory PM10 instruments are reporting 

data in standard temperature and pressure (STP). 

Description:

During the 2011 TSA, EPA found that some regulatory PM10 instruments were not reporting in 

STP. CARB has addressed this for all CARB-operated PM10 sites. CARB began coordinating 

with all districts to ensure that their regulatory PM10 monitors report in STP to be comparable to 

the federal NAAQS. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

74

Finding # DM – Positive Finding 7

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

CARB was creating a data visualization tool to aid data review. 

Description:

CARB was creating a data visualization tool that will make it easy for reviewers/users to identify 

potential issues, such as gaps in the data record. The existing data review formats did not 

facilitate identification of certain errors, such as missing hours of data. To aid data review, 

CARB was creating a tool that will allow for a quick visualization of the data.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

75

Finding # DM1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

QA qualifier codes were not entered in instances where critical QC criteria were missed.

Description:

In the two examples given below, the QC checks, typically conducted nightly, were missed for 

periods extending more than two weeks. One-point QC checks and zero/span checks are both 

listed in Appendix H of the EPA QA Handbook as critical criteria, and are required once every 

two weeks. 

CARB stated that during the data validation process, these data were retained based on the 

nightly calibrations bordering the missing time periods which showed that the O3 instrument 

was within acceptable limits. 
Auditor: Gwen Yoshimura

Final Report - August 2016

76

1. Mojave-Poole (AQS ID 06-037-9033) ozone data, December 19, 2013 – January 13, 2014. 

No nightly checks.

a) December 2013:
Auditor: Gwen Yoshimura

Final Report - August 2016

77

b) January 2014:

2. Bakersfield-California Avenue (AQS ID 06-029-0014) ozone, July 23, 2014 – August 16, 

2014. No valid nightly checks.

a) July 2014:
Auditor: Gwen Yoshimura

Final Report - August 2016

78

b) August 2014:

References:

40 CFR 58 Appendix A, Section 3.2.1

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Institute measures to ensure that when critical criteria are missed but data are still determined 

to be valid, a QA Qualifier code (e.g. “1 – Deviation from a CFR/ Critical Criteria 

Requirement”) is entered along with the affected data points into AQS.
Auditor: Gwen Yoshimura

Final Report - August 2016

79

Finding # DM2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

Calibrations were not always coded as such in AQS.

Description:

40 CFR 58.16 requires reporting of “all ambient air quality data and associated quality 

assurance data” to AQS. AQS is the official repository for ambient air data. During the TSA, 

EPA asked CARB staff about instances where AQS data pulls showed times of extended null 

codes indicating instrument down times, followed by data points. No calibration codes were 

entered into AQS in between the instrument down time/malfunction null codes and the valid 

data points. Because calibration codes were not entered into AQS, EPA questioned why the 

data were considered valid for upload to AQS.

While there may be instances when a calibration is not necessary, station logs associated with 

the situations EPA looked into during the TSA showed that calibrations had been performed 

but were not reflected in AQS. 

References:

40 CFR 58.16

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Section 17)

AQS coding manual. Available at http://www.epa.gov/aqs

Suggested Actions to Address Finding:

Implement procedures to ensure that the appropriate calibration null codes are entered when a 

calibration is performed, and that use of the appropriate codes is periodically assessed.
Auditor: Gwen Yoshimura

Final Report - August 2016

80

Finding # DM3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

Field technicians were unable to apply codes in CARBLogger and DMS to missing data, 

resulting in missing non-coded data in AQS.

Description:

In reviewing AQS, EPA found multiple instances of missing data without qualifier codes. The 

station operators indicated that they were unable to apply qualifier codes using the 

CARBLogger / DMS systems when no data were collected by an instrument. These codes can 

be applied at the time of AQS upload; however, this process is not followed consistently. In 

fact, the person applying codes is physically and operationally removed from staff who are 

aware of the operational issues. The codes applied cannot be reviewed by the field office staff 

and manager, as they are only found in AQS, not DMS.

References:

40 CFR Part 58.16 Data submittal and archiving requirements.

(a) The state, or where appropriate, local agency, shall report to the Administrator, via AQS all 

ambient air quality data and associated quality assurance data for SO2; CO; O3; NO2; NO; 

NOy; NOX; Pb-TSP mass concentration; Pb-PM10mass concentration; PM10 mass 

concentration; PM2.5 mass concentration; for filter-based PM2.5 FRM/FEM the field blank mass, 

sampler-generated average daily temperature, and sampler-generated average daily pressure; 

chemically speciated PM2.5 mass concentration data; PM10-2.5 mass concentration; 

meteorological data from NCore and PAMS sites; average daily temperature and average daily 

pressure for Pb sites if not already reported from sampler generated records; and metadata 

records and information specified by the AQS Data Coding Manual 

(http://www.epa.gov/ttn/airs/airsaqs/manuals/manuals.htm). The state, or where appropriate, 

local agency, may report site specific meteorological measurements generated by onsite 

equipment (meteorological instruments, or sampler generated) or measurements from the 

nearest airport reporting ambient pressure and temperature. Such air quality data and 

information must be submitted directly to the AQS via electronic transmission on the specified 

quarterly schedule described in paragraph (b) of this section.

(b) The specific quarterly reporting periods are January 1-March 31, April 1-June 30, July 1-

September 30, and October 1-December 31. The data and information reported for each 

reporting period must contain all data and information gathered during the reporting period, 

and be received in the AQS within 90 days after the end of the quarterly reporting period. For 

example, the data for the reporting period January 1-March 31 are due on or before June 30 of 

that year.
Auditor: Gwen Yoshimura

Final Report - August 2016

81

(c) Air quality data submitted for each reporting period must be edited, validated, and entered 

into the AQS (within the time limits specified in paragraph (b) of this section) pursuant to 

appropriate AQS procedures. The procedures for editing and validating data are described in 

the AQS Data Coding Manual and in each monitoring agency's quality assurance project plan.

Suggested Actions to Address Finding:

Update DMS to accept codes for data that were not collected.
Auditor: Dena Vallano

Final Report - August 2016

82

Finding # DM4

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Data Management

Finding:

Negative values were inconsistently invalidated for PM2.5 monitors. 

Description:

There was some inconsistency among site operators regarding the appropriate limit to flag 

and/or invalidate negative data for PM2.5 monitors. It was CARB’s policy to report continuous 

particulate matter values to AQS to the absolute minimum sample value or the negative of the 

federal method detection limit as posted on the AQS parameters table, whichever is greater. 

Field technicians were following CARB policy to invalidate anything < -2 μg/m3, however 

this process did not appear to be implemented consistently among operators. 

CARB has been working to make sure that minimum values are handled consistently across 

the PQAO. A QA technical bulletin on AQS Minimum Value Reporting Limits is being 

developed to provide clarification for the PQAO regarding the minimum values that should be 

reported to AQS for data from continuous particulate matter monitors.

References:

40 CFR Part 58.16

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Develop and implement a consistent process throughout the PQAO to determine when 

negative values can be reported and/or a standardized process to evaluate and correct 

instruments when excessive negative values are reported. 
Auditor: Gwen Yoshimura

Final Report - August 2016

83

Finding # QA – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

Since the 2011 TSA, the Quality Management Section has been created and, in coordination with 

other areas of CARB, has implemented significant changes within the PQAO.

Description:

QMS, an entirely new section, was established within the QMB. This section has been working 

with other sections of CARB to improve quality and consistency across the PQAO. Many of the 

improvements noted within this report are associated with work by QMS.

Unique to QMS are the PQAO liaison positions. Each district within the PQAO is assigned to a 

specific liaison. CARB has been allocating budget, time and personnel to allow the liaisons to 

travel to their districts at least once a year. Creation of these positions has provided key 

improvements to coordination and dialogue between the districts and CARB. While there is 

room for growth, significant improvement has been made since the 2011 TSA.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

84

Finding # QA – Positive Finding 2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

R&R documents have been created between CARB and each of the twenty-one districts that 

conduct regulatory monitoring.

Description:

CARB has worked with each of the twenty-one districts that conduct regulatory monitoring to 

create individualized R&R documents. These documents support a successful, functional PQAO 

structure. CARB plans to revisit these documents annually to ensure they continue to reflect 

current practices and provide adequate clarity.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

85

Finding # QA – Positive Finding 3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

CARB has created a QA webpage that presents PQAO QA information in one, easy to navigate 

webpage.

Description:

A QA webpage was created that houses information about the CARB QMB, the PQAO structure 

and contacts, quality management documents including a quality management document 

repository, QA activities including links concerning CANs and AQDAs, and services and 

resources such as information on the standards laboratory. This webpage assembles a wealth of 

information in one location, so that districts, site operators, and others may easily find 

information they need.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

86

Finding # QA – Positive Finding 4

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

CARB held three successful PQAO training modules, and is in the process of creating a 

condensed version of the training to be offered on a repeated basis.

Description:

Since the 2011 TSA, CARB created and held three training sessions for the PQAO, held in 

northern and southern California. Sessions were developed by a diverse group of PQAO 

members, and attended by PQAO districts as well as the other California PQAOs. CARB 

received very positive feedback, as well as requests for additional, ongoing trainings.

CARB is in the process of converting the three PQAO training modules into one three-day 

training. They anticipate holding this training every 18 months. This training provides a forum 

for districts and CARB to meet all together, to receive consistent training, and discuss current 

challenges and successes. EPA congratulates CARB on the quality and success of the previous 

trainings, and supports their ongoing work on this vital element of PQAO implementation.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

87

Finding # QA – Positive Finding 5

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

CARB was creating guidance as to how districts should review data once they are entered into 

AQS.

Description:

CARB has created a data validation SOP and shared it with the PQAO. The R&R documents for 

each district also require quarterly and annual data reviews. To support data review and 

consistency across the PQAO, CARB was creating a QA bulletin on post-AQS data review. This 

bulletin will provide guidance as to what districts should check for when conducting the 

quarterly and annual reviews.

References:

Suggested Actions to Address Finding:

NA
Auditor: Dena Vallano

Final Report - August 2016

88

Finding # QA1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

Improvements to the AQDA and CAN processes are necessary to make them more efficient 

and effective, and ensure timely resolution of monitoring issues.

Description:

The CAN process documents issues that may affect data quality, completeness, storage, or 

reporting. The objective of the CAN process is to improve data quality and to ensure 

compliance with state, federal, and local requirements. An AQDA is a request for an 

investigation of the validity of ambient air quality data for a certain period of time. 

Issues EPA observed during the TSA:

 CANs and AQDAs were not addressed according to the timelines in the SOP and 

PQAO R&R documents.

 In some instances, the data quality issues could have been easily resolved without 

initiating a formal CAN process.

 The full implications of these corrective actions were not understood. For example, see 

AV/MD8.

 Corrective actions resulting from an issued CAN were not communicated to the 

individual who initiated the CAN. This was the case with a CAN initiated by a CARB 

station operator.

During EPA’s limited review of CANs and AQDAs, the following issues were observed:

 A CAN was initiated in June 26, 2014 with an expected completion date of July 10, 

2014. However, the CAN was not communicated to the local district until August 19, 

2014. The CAN did not include an appropriate ID number. 

 A CAN was issued August 4, 2014. As of the date of the audit (summer 2015), no 

corrective action had been communicated to the initiator, either officially or 

unofficially. 

 AQDA 8186 was issued on December 24, 2013 for the Mojave-Poole site. In January 

2015, the QA group determined that 8186 had not been closed. In closing the AQDA, 

they determined that certified data in AQS required modification.

 CAN 51 was issued November 20, 2013 for expired gas cylinders at Mojave Desert 

AQMD’s Trona-Athol/Telescope site. The CAN form shows that new cylinders were 

not ordered until May 15, 2014, and the expired cylinders were not replaced until July 

1, 2014. The letter closing out the CAN stated, “The District should track cylinder 

expiration dates to ensure all instrument calibrations and verifications are completed 

using current gas standards.” On February 25, 2015, a CAN was issued for an expired 

standard at Mojave Desert AQMD’s Lancaster site. No action had been taken to 
Auditor: Dena Vallano

Final Report - August 2016

89

address this CAN when EPA visited Mojave Desert AQMD during the TSA on April 

15, 2015.

According to the SOP and PQAO R&R documents, CANs are to be completed within 45 days 

from the date of issuance.

During the audit, EPA found instances where CANs and AQDAs were not addressed in a 

timely fashion, and the full implications of the associated corrective actions were not always 

understood or communicated

EPA recognizes that CARB is providing an annual summary of CANs. However, this process 

could be more transparent (e.g. sharing those CANs ahead of the summary, even if not 

resolved, that could affect multiple agencies). 

EPA acknowledges that CARB has only recently instituted the CAN process widely, and is 

still working through implementation issues.

References:

CARB, Air Monitoring Quality Assurance Volume 5, Corrective Action Notification SOP, 

Appendix AN, October 2014 (http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf)

PQAO Roles and Responsibilities for the CARB and San Luis Obispo County Air Pollution 

Control District, June 2014 

(http://arb.ca.gov/aaqm/qa/pqao/repository/slo_rolesandresponsibilities.pdf)

CARB Quality Management Plan for Ambient Air Monitoring, Volume 1, Section 9, pp 37-

38, July 2013 (http://arb.ca.gov/aaqm/qa/pqao/repository/qmp_final.pdf#page=42)

Suggested Actions to Address Finding:

Meet the time limits established in the SOP and R&R document for communicating CANs and 

AQDAs to the local districts. 

Formally track and communicate initiation and closure of CANs and AQDAs through the 

CARB QMS/district liaisons to the affected parties and CAN/AQDA initiators.

Enhance transparency and timeliness of the system used to inform all districts when significant

issues and resolutions have arisen in CANs/AQDAs.
Auditor: Mathew Plate

Final Report - August 2016

90

Finding # QA2

Agency: CARB 

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

An AQDA that indicated a need for AQS to be changed was not appropriately closed out, and 

inaccurate AQS codes were entered overriding previously entered, more accurate AQS codes.

Description:

In August 2013, AQDAs were issued for the continuous PM10 and PM2.5 instruments at the 

Mojave Poole site (AQS ID 06-037-9033). The station operations staff investigated the problem 

and invalidated the appropriate data and applied the AS flag in DMS. These data were uploaded 

to AQS and certified in 2014 with the AS code. In January 2015, the QAS evaluated outstanding 

PM2.5 AQDAs and found that the PM2.5 AQDA from August 2013 for Mojave Poole had not 

been closed out and requested that the BJ code be applied to the data in AQS (QAS was not 

aware that an AS code had already been loaded to AQS). AQS was updated to BJ for PM2.5, with 

no change to the information in DMS. The field technician was not consulted before this change 

was made.

CARB’s June 30, 2015 AQDA SOP does not adequately address how data changes are to be

addressed in AQS. Section 4.0 AQDA Data Follow Up In AQS states, “When data must be 

invalidated, flagged or corrected, QMB staff will generate an AMP 350 report 45 days after the 

end of the quarter of the data range affected by the AQDA. Where CARB is not the direct 

submitter to AQS, the AQS submitter for that local district is responsible for the impacted data. 

The AQDA data resolutions will be compared to the AMP350 report for accuracy. If a 

discrepancy is noted or no action was taken by the monitoring organization, QMB will follow up 

to ensure AQS is properly updated.”

This does not adequately describe who is responsible for comparing the AQDA data resolution to 

the AMP350, nor is it clear who is responsible for editing AQS in instances when CARB is the 

submitter.

AQS codes are typically entered by the field technician, who knows what is happening at the 

site. Codes recommended by the field technician often differ from those applied by those who 

are further removed from the daily operation. 

References:

California Air Quality Management District, AQDA SOP. June 30, 2015.
Auditor: Mathew Plate

Final Report - August 2016

91

Suggested Actions to Address Finding:

Revise the AQDA SOP to describe:

 Who is responsible for determining whether a change to AQS is appropriate following an 

AQDA;

 What that change should be;

 Who makes the change;

 How data changes will be reflected in DMS.

Ensure that the field technicians are aware when data are being changed, and that those in CARB 

responsible for data certification/recertification are aware when data are changed.
Auditor: Mathew Plate

Final Report - August 2016

92

Recommendation # QA - Recommendation1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

Annual performance audits were not being used to identify significant practices that needed

correction.

Description:

While recognizing that the CARB performance auditor’s role is not to perform a technical audit 

of district staff, they do visit the districts more frequently than TSA staff. In fact, CARB has 

upgraded the performance audit program in recent years to be more comprehensive. EPA 

supports these changes, while noting that CARB’s auditors did not comment on issues such as 

the significant MDAQMD laboratory / PM sampling issues, and districts that are not following 

SOPs. 

References:

Suggested Actions to Address Finding:

Review the CARB performance auditor’s responsibilities, and determine if there is a list of 

critical items they could additionally check during their audits.
Auditor: Gwen Yoshimura

Final Report - August 2016

93

Recommendation # QA – Recommendation2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Quality Assurance

Finding:

Data validation and reporting responsibilities were unclear in one instance when a district 

utilized a non-CARB laboratory for analysis. This may extend to other such situations as well.

Description:

During the TSA, EPA found a situation where a site operated by a district sent samples for 

analysis to a laboratory other than CARB’s. The responsibilities for data reporting, review, and 

validation were unclear. See finding NCU16. 

While the CARB R&R documents address data management to some extent, and CARB’s April 

2016 Post-AQS Data Confirmation QA Bulletin states that roles, responsibilities, and 

communication between the District and laboratory should be clearly outlined and documented 

in such cases, it was unclear whether the other districts in the same situation had been following 

these guidelines. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

Suggested Actions to Address Finding:

Verify that roles, responsibilities, and communication between districts and laboratories are 

outlined and documented in the few cases where district-run sites utilize a non-CARB lab for 

analysis.
Auditor: Gwen Yoshimura

Final Report - August 2016

94

Finding # SL – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: Standards Laboratory

Finding:

CARB was working to restructure roles and responsibilities within QMB to improve

independence, review, and succession planning within the Standards Laboratory.

Description:

CARB was re-evaluating roles and responsibilities within QMB to determine if a different 

structure could better support independence, review, and succession planning within the 

Standards Laboratory. The new structure should separate staff who verify and certify the

equipment used during CARB audits from the staff using the equipment, and identify someone to 

oversee QA activities for the Standards Laboratory. 

EPA notes that such changes could address several findings noted in this report.

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

95

Finding # SL1

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

The Standards Laboratory QA system did not have written policies and procedures, a QA manual 

or a designated QA Officer.

Description:

The role of the Standards Laboratory in the CARB PQAO is unique. The Standards Laboratory 

provides an essential service: to ensure consistency of standards used throughout the PQAO. 

However, as a metrology laboratory, the Standards Laboratory must function independently of 

the monitoring and oversight functions at CARB.

The primary role of the Standards Laboratory is to provide independent and unbiased 

certification of standards for the CARB PQAO and other clients. Systems should be in place to 

ensure that independent QA systems and practices are implemented.

A documented QA system that addresses the issues of independence, integrity, conflicts of 

interest, an independent QA Officer, customer feedback, and quality improvement specifically 

for the Standards Laboratory should be developed.

References:

ISO/IEC 17025: 2005; Sections 4.1 & 4.2

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and 

General Requirements; NIST; February 2006

Suggested Actions to Address Finding:

Develop QA system policies and procedures for the Standards Laboratory.

Identify an independent QA Officer to handle Standards Laboratory issues, reviews, and 

corrective actions.

Develop a quality manual for the Standards Laboratory.
Auditor: Mathew Plate

Final Report - August 2016

96

Finding # SL2

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Corrective action processes for the Standards Laboratory were not consistent with ISO or 

NIST requirements.

Description:

Standards certification laboratories provide an essential service by providing the baseline 

against which all measurements taken by their clients monitoring programs are compared. 

They must be both independent and responsive to their clients’ needs.

When nonconforming calibration work is identified internally, the issue should be immediately 

corrected and the laboratory QA Officer should determine if a corrective action is required.

To be consistent with ISO 17025 and NIST 150, all corrective actions identified internally or 

by customers must be corrected, evaluated as to the root cause, and monitored independently 

for effectiveness. Additional internal audits of calibration procedures should be initiated 

immediately. 

If an internal (i.e., CARB PQAO) client’s standards grossly fail as-is calibration, the 

information should be reported promptly to the client and the PQAO quality management 

group for potential action. However, the Standards Laboratory must retain independence from 

evaluation and tracking of any resulting corrective actions. 

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and 

General Requirements; NIST; February 2006

Suggested Actions to Address Finding:

Evaluate the corrective action processes used for the Standards Laboratory and ensure that 

these are consistent with ISO and NIST requirements for complete, prompt, and independent 

response.
Auditor: Mathew Plate

Final Report - August 2016

97

Finding # SL3

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Calibrations/Certifications conducted by the Standards Laboratory were not reviewed by 

peers/management/QA in sufficient detail to prevent errors.

Description:

The purpose of the Standard Laboratory peer review system is for technicians to review the

work of their colleagues; however, there were no specific criteria for these reviews. 

All calibrations should be independently verified and validated prior to being delivered to 

clients. This should include verification of all data that were manually entered into data 

systems, evaluation of all calibration results, spot checks of supporting documents and 

calibration traceability, and spot checks of calculations to verify that errors were not 

introduced into spreadsheets. 

The auditor reviewed documentation for a few calibrations and noted several peer review 

discrepancies. Two examples are presented here:

1) Peer Review not Documented and/or Performed
Auditor: Mathew Plate

Final Report - August 2016

98

2) Change in Slope as the Result of a Potential Typo Missed (two photos)
Auditor: Mathew Plate

Final Report - August 2016

99

References:

QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality 

Monitoring Program, EPA-454/B-13-003, May 2013

Suggested Actions to Address Finding:

Implement a more complete second level/peer review. 

Implement additional levels of review that include periodic technical evaluations. This could 

be done by a QA Officer and/or a manager. 
Auditor: Mathew Plate

Final Report - August 2016

100

Finding # SL4

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Forms to document standard certification were not completely validated or revalidated on a 

routine basis.

Description:

The electronic forms used to document standard certifications should:

 Be initially validated to ensure they are correct for a range of inputs (validations must 

be documented).

 Be re-validated and assigned a new form version if any changes were made

 Be periodically spot-checked for errors.

 Include documentation for all aspects of each certification.

 Be evaluated for linearity using established criteria.

 Clearly identify which “cells” contain manually entered data requiring verification.

If forms have not been fully validated, errors may be present in existing calibrations.

The auditor focused on the low volume and high volume flow calibration forms. The 

following errors were noted, which may be representative of errors throughout the program.
Auditor: Mathew Plate

Final Report - August 2016

101

High Volume Form Example

Low Volume Form Example

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and 

General Requirements; NIST; February 2006
Auditor: Mathew Plate

Final Report - August 2016

102

5.4.7.2 When computers or automated equipment are used for the acquisition, processing, 

recording, reporting, storage or retrieval of test or calibration data, the laboratory shall 

ensure that:

a) computer software developed by the user is documented in sufficient detail and is 

suitably validated as being adequate for use;

Suggested Actions to Address Finding:

All forms should be validated and a locked copy of the validated forms should be saved.

Forms should be routinely evaluated.

Any form errors should be investigated and corrective action should be taken where 

appropriate. 
Auditor: Mathew Plate

Final Report - August 2016

103

Finding # SL5

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Calibration laboratory staff had not attended routine trainings or conferences that are specific 

to their field of work.

Description:

Standards Laboratory staff conduct operations that are unique within CARB. With the 

exception of the O3 standard reference photometer program, the calibrations completed are 

performed without significant external support or collaboration with other standards 

laboratories.

Standards Laboratory operations, critical to PQAO data collection, are performed by a small 

staff that works separately from the rest of the organization. Training and interactions with 

other calibration laboratories are critical for staff to be able to implement the program 

effectively.

There are available trainings and conferences, including an annual measurement conference in 

California. There also may be opportunities to work more collaboratively with the EPA 

protocol testing laboratories and State of California Metrology Laboratory within the 

California Department of Food and Agriculture (CDFA).

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and 

General Requirements; NIST; February 2006

QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality 

Monitoring Program, EPA-454/B-13-003, May 2013

Suggested Actions to Address Finding:

Develop a training and collaboration plan for Standards Laboratory staff.
Auditor: Mathew Plate

Final Report - August 2016

104

Finding # SL6

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

The main Standards Laboratory database and several data systems had not been updated 

recently and did not meet expectations of performance and/or documentation and compromise 

data defensibility.

Description:

Based on a cursory review of the database, errors were found. It was also not possible to track 

calibrator information during the audit visit due to the lack of a functional search or query 

function in the main tracking database. A database that does not allow this type of information 

to be pulled, i.e. does not provide adequate documentation, compromises data defensibility.

The single database reviewed during the TSA had missing information. For example, see 

highlighted cells below: 
Auditor: Mathew Plate

Final Report - August 2016

105

In order to be defensible, databases and data systems should:

 Be validated and conform to standards of programing.

 Transfer data digitally whenever possible to maintain data integrity.

 Not be prone to malfunctions, including locking-up and/or crashing.

 Document all aspects of the calibration process available.

 Be secure with limited and tracked access.

 Track all changes made to data.

The systems in use should be improved to comply with these requirements. 

When using data systems developed by outside parties (including EPA/NIST), CARB should 

work with these parties to develop data system improvements that ensure data integrity.

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and 

General Requirements; NIST; February 2006

5.4.7.2 When computers or automated equipment are used for the acquisition, processing, 

recording, reporting, storage or retrieval of test or calibration data, the laboratory shall 

ensure that:

a) computer software developed by the user is documented in sufficient detail and is 

suitably validated as being adequate for use;

b) procedures are established and implemented for protecting the data; such 

procedures shall include, but not be limited to, integrity and confidentiality of data 

entry or collection, data storage, data transmission and data processing;

c) computers and automated equipment are maintained to ensure proper functioning 

and are provided with the environmental and operating conditions necessary to 

maintain the integrity of test and calibration data.

Suggested Actions to Address Finding:

Develop an inventory and assessment of each database and data system used by the Standards 

Laboratory.

Work with EPA to develop a schedule to update the Standards Laboratory data systems and 

databases.
Auditor: Mathew Plate

Final Report - August 2016

106

Finding # SL7

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Zero air was not being treated as a standard and the quality of zero air was not formally 

documented by the Standards Laboratory.

Description:

Zero air scrubbers were being used in place of certified zero air for standard certifications. 

Because zero air is used to generate the zero point and the calibration mixes, it must be treated 

as a standard. As such, zero air scrubber maintenance and verification must be documented.

References:

EPA Traceability Protocol for Assay and Certification of Gaseous Calibration Standards,

September 1997, (EPA-600/R-97/121), Section 2.1.9, Zero Gas

Suggested Actions to Address Finding:

Institute procedures for certifying and documenting the quality of zero air.
Auditor: Mathew Plate

Final Report - August 2016

107

Finding # SL8

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

The materials used for plumbing and mixing for SO2 certifications were stainless steel. These 

are not appropriate materials for SO2 certifications.

Description:

Sulfur dioxide exhibits a slow response when being delivered and mixed in stainless steel 

materials. This could potentially result in a small systematic bias and/or imprecision when 

certifying standards.

As stated in 40 CFR Part 58 Appendix E, Section 9(a):

(a) For the reactive gases, SO2, NO2, and O3, special probe material must be used for 

point analyzers. Studies20-24 have been conducted to determine the suitability of 

materials such as polypropylene, polyethylene, polyvinyl chloride, Tygon®, aluminum, 

brass, stainless steel, copper, Pyrex® glass and Teflon® for use as intake sampling 

lines. Of the above materials, only Pyrex® glass and Teflon® have been found to be 

acceptable for use as intake sampling lines for all the reactive gaseous pollutants. 

Furthermore, the EPA25 has specified borosilicate glass or FEP Teflon® as the only 

acceptable probe materials for delivering test atmospheres in the determination of 

reference or equivalent methods. Therefore, borosilicate glass, FEP Teflon® or their 

equivalent must be the only material in the sampling train (from inlet probe to the back 

of the analyzer) that can be in contact with the ambient air sample for existing and new 

SLAMs.

While the above CFR reference is specifically for probe material used in instruments 

measuring ambient air, the same principle applies for materials being used in certification 

systems. 

Stainless steel materials used in the certification system can be either coated with an inert 

material or replaced with Teflon to improve system performance.

References:

Lee Byers & John W. Davis (1970) Sulfur Dioxide Adsorption and Desorption on Various 

Filter Media, Journal of the Air Pollution Control Association, 20:4, 236-238

Avraham Teitz (EPA Region 2) and Thien Bui (EPA Region 7), March 25, 2010, Standard 

Operating Procedures for the Verification of NO/NOx, SO2, and CO Concentrations in EPA 
Auditor: Mathew Plate

Final Report - August 2016

108

Protocol Gas Mixtures, Section 11.0 Apparatus and Materials

40 CFR Part 58 Appendix E, Section 9(a)

Suggested Actions to Address Finding:

Eliminate the use of uncoated stainless steel surfaces in the SO2 standard certification system.
Auditor: Mathew Plate

Final Report - August 2016

109

Recommendation # SL – Recommendation1

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Standards Laboratory

Finding:

Contracting issues affected the ability of the Standards Laboratory to certify standards and 

delayed required primary standard certifications.

Description:

CARB’s contracting requirements have limited its ability to contract directly with their low flow 

certification standard supplier and have delayed recalibration of the associated devices. 

This and similar contracting issues have the potential to limit the Standards Laboratory’s ability 

to provide quality and continuous services in the future. Because the Standards Laboratory 

provides a critical role in the PQAO, CARB should work to avoid delays or service denials due 

to contracting.

As explained in subsequent discussions with CARB, the specific situation that prompted this 

finding was addressed around the time of the TSA. Due to changes in CARB’s contracting 

language (language over which MLD does not have control), the vendor who certifies the flow 

devices using proprietary software did not sign the new contract. While this did prevent timely 

certification of those particular flow devices, CARB recognized the issue and started discussing 

the possibility of borrowing devices from other PQAOs. Before implementing this back-up plan, 

CARB identified a secondary device that was certified, and used this device in lieu of the expired 

devices. No data were compromised as a result. 

References:

Suggested Actions to Address Finding:

CARB should maintain a level of attention and ability to quickly respond to contracting issues as 

they arise, and work to prevent them whenever possible. 
Auditor: Gwen Yoshimura

Final Report - August 2016

110

Finding # PM – Positive Finding 1

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: PM Laboratory

Finding:

CARB was creating a laboratory support section.

Description:

CARB was in the process of creating a laboratory support section. This section will consolidate 

laboratory QA, sample handling, and database management and improve independence and 

review.

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

111

Finding # PM – Positive Finding 2

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: PM Laboratory

Finding:

CARB’s PM laboratories were found to be well maintained and well run.

Description:

Both of the PM laboratories were well maintained, neat, and well organized. Generally, the PM10

and PM2.5 laboratory measurements were being performed with a high level of technical 

expertise.

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

112

Finding # PM – Positive Finding 3

Agency: California Air Resources Board

Date of Audit: Summer 2015

Program Area: PM Laboratory

Finding:

CARB was creating a system to ensure consistent documentation when a filter does not run. 

Description:

In some instances, CARB sends a filter out to a station but the filter never collects any useable 

data (e.g. the instrument’s filter mechanism jams and the filter does not run). The documentation 

process CARB had in place did not specify how this situation was to be documented. CARB was 

working to establish a system to create consistent documentation when a filter does not run. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate

Final Report - August 2016

113

Finding # PM1

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

Statistical information on temperature and humidity compliance with method requirements was

not consistently calculated or tracked in accordance with EPA’s specified methodology.

Description:

EPA regulation requires statistical calculations of temperature and relative humidity variability 

and must be formally documented.

The MLD PM laboratory was using a system to track and graph temperature and humidity with 

calibrated traceable standards for both weigh rooms. The laboratory analysts evaluated

temperature and humidity prior to each weighing session. However, compliance with the 

temperature and humidity control ranges were not formally calculated for all weighing sessions. 

Calculations should be completed in accordance with the following sections of the Quality 

Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using Designated 

Reference of Class I Equivalent Methods, January 2016 (available at 

https://www3.epa.gov/ttn/amtic/files/ambient/pm25/qa/m212.pdf ):

 Section 9.4 Logging Systems.

 Section 10.4.1.3 Determining Compliance with Filter Conditioning Specifications

References:

40 CFR Part 50 Appendix J

7.4 Filter Conditioning Environment.

7.4.1 Temperature range: 15 to 30 C.

7.4.2 Temperature control: ±3 C.

7.4.3 Humidity range: 20% to 45% RH.

7.4.4 Humidity control: ±5% RH.

40 CFR Part 50 Appendix L

8.2 Filter conditioning. All sample filters used shall be conditioned immediately before 

both the pre- and post-sampling weighings as specified below. See reference 2 in section 

13.0 of this appendix for additional guidance.
Auditor: Mathew Plate

Final Report - August 2016

114

8.2.1 Mean temperature. 20 - 23° C.

8.2.2 Temperature control. ±2° C over 24 hours.

8.2.3 Mean humidity. Generally, 30-40 percent relative humidity; however, where it can 

be shown that the mean ambient relative humidity during sampling is less than 30 

percent, conditioning is permissible at a mean relative humidity within ±5 relative 

humidity percent of the mean ambient relative humidity during sampling, but not less 

than 20 percent.

8.2.4 Humidity control. ±5 relative humidity percent over 24 hours.

8.2.5 Conditioning time. Not less than 24 hours.

8.3.3 Filters must be conditioned at the same conditions (humidity within ±5 relative 

humidity percent) before both the pre- and post-sampling weighings.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013, Appendix D

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

Update the weigh room tracking system to calculate compliance with the EPA regulatory 

requirements for temperature and humidity.

Evaluate the temperature and humidity data for all conditioning and weighing sessions in 2013-

2015 to verify compliance with regulatory requirements.

If data were reported from weighing periods where temperature and/or humidity were out of 

control (including the 5% RH difference between weigh sessions for PM2.5), consult with EPA as 

to whether data should be invalidated or flagged, and what flag should be applied. 
Auditor: Mathew Plate

Final Report - August 2016

115

Finding # PM2

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

Preservation and custody requirements were not being met and/or tracked for samples that had 

been hand delivered to the MLD laboratory.

Description:

District and MLD field technicians were hand delivering samples from sites geographically 

close to Sacramento to the MLD PM sample handling room. These samples were not being 

signed over to a MLD laboratory sample custodian and were not being verified for 

preservation requirements when they arrived. There were indications that some of the field 

operators may not have been chilling samples from sample removal and during transport to 

MLD.

Based on these procedures, it was unclear if holding times and custody are properly 

maintained. Allowing uncontrolled access to the MLD sample handling room has custody 

implications for other filters stored there.

At the time of our visit, MLD had recently formed a Laboratory Support Section which 

includes sample control. There were indications that procedures were in the process of being 

put in place to address these issues.

References:

40 CFR Part 50 Appendix L

8.3.6 The post-sampling conditioning and weighing shall be completed within 240 

hours (10 days) after the end of the sample period, unless the filter sample is 

maintained at temperatures below the average ambient temperature during sampling 

(or 4 °C or below for average sampling temperatures less than 4 °C) during the time 

between retrieval from the sampler and the start of the conditioning, in which case 

the period shall not exceed 30 days. Reference 2 in section 13.0 of this appendix has 

additional guidance on transport of cooled filters.

2003 NELAC Standard, EPA/600/R-04/003

Section 5.4.12

Suggested Actions to Address Finding:
Auditor: Mathew Plate

Final Report - August 2016

116

Implement a more formal custody process for hand delivered samples, including the 

verification of receipt temperature.

Implement procedures to better control access to sample handling areas.

Evaluate practices used for preservation of hand delivered PM2.5 samples by field operators:

 Recommend best practices to staff and agencies hand delivering samples.

 Evaluate past data to determine if holding time requirements were violated.
Auditor: Mathew Plate

Final Report - August 2016

117

Finding # PM3

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

Laboratory blanks for PM2.5 were not processed in batches consistent with regulatory 

requirements.

Description:

The regulatory language in Appendix L indicates that blanks filters shall be weighed with each 

set of field filters. This language also implies that these blanks should be reweighed with their 

corresponding field sample filters.

The MLD laboratory was not initiating new laboratory blanks with each set of filters as 

required by regulation. Instead, they were using multiple weighing for laboratory blanks to 

represent multiple sets of filters. This practice does provide sufficient laboratory blank control 

but does not link “new” blanks with new sets of filters. 

Because the MLD sends filters to many different clients that are received and post-weighed at 

different times, EPA understands that it may be impractical to post-weigh each filter with the 

corresponding laboratory blank. However, CARB may be able to develop a system to simulate 

this practice. For example, post-weigh each laboratory blank with a field batch approximately 

30 days after pre weigh or another time period that corresponds with the average pre- and postweigh separation time for MLD.

References:

40 CFR Appendix L

8.3.7.2 New laboratory blank filters shall be weighed along with the pre-sampling 

(tare) weighing of each set of PM2.5 filters. These laboratory blank filters should 

remain in the laboratory in protective containers during the field sampling and should 

be reweighed as a quality control check.

Suggested Actions to Address Finding:

Change procedures for implementing laboratory blanks to be more consistent with regulation.
Auditor: Mathew Plate

Final Report - August 2016

118

Finding # PM4

Agency: U.S. EPA

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

PM2.5 MTL filter lots and the associated lot controls could not be properly evaluated and were

not evaluated at a sufficient frequency.

Description:

The MTL filters provided by EPA do not have lot numbers or manufactured dates associated 

with each box of filters. Given the number of filters that CARB orders and uses each year, it is 

possible that these represent multiple manufacturing lots/days. CARB was considering each 

annual EPA shipment as a filter lot.

Additionally, the MTL filters received by MLD have had a very high rejection rate, some 

boxes as high as 50%. EPA should be taking action to improve the vendor’s product quality. 

Should CARB continue to use MTL into the future, or other filter suppliers that do not identify 

the filter lots, filter lots and associated lot controls should be evaluated in accordance with 

EPA regulations and guidance documents.

References:

40 CFR Part 50 Appendix L

6.0 Filter for PM2.5 Sample Collection.

6.9 Filter weight stability. Filter weight loss shall be less than 20 μg, as measured in 

each of the following two tests specified in sections 6.9.1 and 6.9.2 of this appendix. 

The following conditions apply to both of these tests: Filter weight loss shall be the 

average difference between the initial and the final filter weights of a random sample 

of test filters selected from each lot prior to sale. The number of filters tested shall be 

not less than 0.1 percent of the filters of each manufacturing lot, or 10 filters, 

whichever is greater. The filters shall be weighed under laboratory conditions and 

shall have had no air sample passed through them, i.e., filter blanks. Each test 

procedure must include initial conditioning and weighing, the test, and final 

conditioning and weighing. Conditioning and weighing shall be in accordance with 

sections 8.0 through 8.2 of this appendix and general guidance provided in reference 2 

of section 13.0 of this appendix.

8.0 Filter Weighing.

8.3.7.1 New field blank filters shall be weighed along with the pre-sampling (tare) 

weighing of each lot of PM2.5 filters. These blank filters shall be transported to the 
Auditor: Mathew Plate

Final Report - August 2016

119

sampling site, installed in the sampler, retrieved from the sampler without sampling, 

and reweighed as a quality control check.

8.3.7.2 New laboratory blank filters shall be weighed along with the pre-sampling 

(tare) weighing of each set of PM2.5 filters. These laboratory blank filters should 

remain in the laboratory in protective containers during the field sampling and should 

be reweighed as a quality control check.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013, Appendix D

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

EPA should continue to share CARB’s experience with MTL filters with the EPA office in 

charge of filter contracts in order to resolve MTL filter lot and filter quality issues.

MLD should consider a more frequent lot blank analysis procedure until information on lots is 

available from the manufacturer.
Auditor: Mathew Plate

Final Report - August 2016

120

Finding # PM5

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

The PM10 laboratory SOPs had not been updated in over five years.

Description:

The PM10 SOP had not been formally updated in over five years. The laboratory analyst did

have a version of this SOP with some “interim” changes included. While this is a good 

practice between revisions, the PM10 “interim” changes are confusing because some were 

superseded without ever being formally implemented.

Also note, that while the PM2.5 SOP had been recently updated, some of the practices in the 

SOP did not reflect what the laboratory was currently doing.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013

Suggested Actions to Address Finding:

Update PM10 laboratory SOP.

Implement a process for keeping SOPs up-to-date.
Auditor: Mathew Plate

Final Report - August 2016

121

Finding # PM6

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

The procedure for processing filters for PM10 speciation led to data being invalidated 

unnecessarily in one observed instance.

Description:

In order to meet holding times for filter speciation analysis, PM10 filters were processed (cut) 

shortly after weighing. This process was routinely completed prior to PM10 data review. As a 

consequence, if the data review called for re-analysis of the filter, CARB was unable to 

reanalyze the filter for PM10 mass and the data point was lost. EPA found that multiple PM 

data points from a critical site (all as part of the same observed instance) were lost as a result 

of this practice.

In general, the most critical use for PM10 filter data from routine SLAMS is comparison to the 

NAAQS. The quality of data used to compare to the NAAQS should not be compromised in 

order to perform an optional secondary speciation analysis until after filter weights are 

validated. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013, Appendix D.

Suggested Actions to Address Finding:

MLD should ensure that PM filter weights are valid prior to releasing filters for speciation. 

If speciation must be performed and the sample review schedule cannot accommodate timely 

analysis of NAAQS data, CARB should install dedicated speciation samplers at monitoring 

sites.
Auditor: Mathew Plate

Final Report - August 2016

122

Finding # PM7

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

Not all databases and electronic forms were tracked and validated, compromising data 

defensibility.

Description:

The CARB laboratory has done a good job of developing a flexible database with onsite 

support and has taken steps to improve system functionality. However, the laboratory was 

using some electronic forms that were not being tracked and validated. For example, excel 

files are used to track and calculate the differences between the primary and working weights. 

This procedure was not described in the relevant SOP. Another example, described in the 

PM10 weigh lab SOP, is the use of excel files for calibrations of the PM10 filter extractions. 

Data defensibility is at risk when adequate documentation is not available. CARB must

continue to develop their system to track additional quality control and statistical parameters 

and to continue to support electronic transfer of data into the system. 

Also, a formal, documented validation of the laboratory database should be conducted to 

identify and address areas that compromise data defensibility.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013.

2003 NELAC Standard, EPA/600/R-04/003

Section 5.5.4.7.2

Suggested Actions to Address Finding:

Evaluate data systems for consistency with NELAC Section 5.5.4.7.2, including the 

development of a formal, documented validation process.
Auditor: Mathew Plate

Final Report - August 2016

123

Finding # PM8

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: PM Laboratory

Finding:

MLD weigh room practices did not protect the integrity of samples.

Description:

Maintaining sample integrity is necessary to support data quality and defensibility. EPA found 

several practices that could compromise the integrity of the PM samples, including:

 Staff other than weigh room technicians had access to weigh rooms, including 

janitorial staff. 

 Materials that could generate particulates were stored in the weigh rooms. 

 PM2.5 samples were handled in a manner that could subject samples to particulate

contamination. For example, samples were handled on counter tops in large batches.

 As noted in PM3, laboratory blanks were not batched with filters, which makes 

tracking laboratory contamination to specific filters less certain.

References:

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

2003 NELAC Standard, EPA/600/R-04/003

Suggested Actions to Address Finding:

Review and revise weigh room access, storage, and sample handling procedures.
Auditor: Joe Eidelberg

Final Report - August 2016

124

Finding # Pb1

Agency: California Air Resources Board

Date of Audit: June 4, 2015

Program Area: Pb Laboratory

Finding:

CARB’s Pb laboratory procedures did not meet EPA requirements.

Description:

EPA found that CARB used a method that was not approved as comparable to EPA methods. 

References:

40 CRF 58, Appendix C, Section 2.8, Modification of Methods by Users

40 CFR 53 Ambient Air Monitoring Reference and Equivalent Methods

Suggested Actions to Address Finding:

EPA understands that CARB has ceased analyzing Pb-TSP filters for NAAQS comparison. If 

this is confirmed, no further action is necessary. Should CARB wish to restart analyzing PbTSP filters for NAAQS comparison, they will need to conform to all EPA requirements and 

regulations.
Auditor: Mathew Plate

Final Report - August 2016

125

Finding # AV/MD - Positive Finding 1

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

AV/MD AQMD’s monitoring staff were dedicated and maintain sites spread out across a 

significant geographic area.

Description:

Overall, the two technicians are very dedicated and work hard to maintain a fair number of 

monitoring sites, some of which are a signification distance from their office. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Mathew Plate 

Final Report - August 2016

126

Finding # AV/MD1 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

The data review and validation process was not complete and/or independent. Numerous errors 

were found with data that had been loaded into AQS.

Description:

The AV/MD AQMD (i.e., AVAQMD and MDAQMD) had two technicians devoted to air 

monitoring full time and a manager who devoted less than half of his time to air monitoring. At 

this staffing level, technicians should review and validate one another’s work and all data should 

undergo a manager review prior to upload to AQS. The data in AQS should also be evaluated for 

accuracy on a regular basis.

At the time of the audit, a limited review of the continuous AV/MD AQMD data by one staff 

person and the manager was being performed. This review focused on information available in

the AirVision data system. There was no third level of review for these data. Supporting 

information (that is not available in AirVision) was not routinely reviewed. The review of data in 

the AirVision system focused on data where flags have been applied by the data system and/or 

notes were inserted by the site operator. The accuracy of data uploaded to AQS was also not 

verified.

For filter-based data, the weigh laboratory technician reviewed his own data and uploaded them

into AQS. There was no second level independent review and validation. Elements of these data 

could occasionally have been checked by the manager.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

Link to Data Verification and Validation resources: http://www.epa.gov/quality1/vandv.html

Suggested Actions to Address Finding:

Devote the resources necessary to have a complete, consistent, robust, and independent data 

review and validation process. 

• Formalize the implementation of a data validation procedure that incorporates multiple 

levels of independent review on a frequent (i.e. monthly) basis. 

• Assign data validation to staff/management independent of those generating results. 
Auditor: Mathew Plate 

Final Report - August 2016

127

• Integrate independent QA oversight throughout the data validation process, when 

possible. 

• Conduct independent data audits to help identify systematic deficiencies with data 

validation as well as specific data issues. Develop a procedure to evaluate AQS 

periodically for erroneous data. Develop data tools (e.g., flags, figures, tables) to conduct 

effective and efficient data audits.

• Ensure all staff involved in validation are familiar with acceptance criteria, applicable 

flags to apply when criteria fail, and have access to the data used in generating the 

validation report.

• Revise and update data validation SOP and QAPPs to reflect the new multi-level data 

validation process, and to include a process to determine when negative values can be 

reported and/or a process to evaluate and correct instruments as soon as possible when 

excessive negative values are reported. (See AV/MD18)

• Review and correct site/pollutant metadata in AQS. (See AV/MD17)

• Replace the current laboratory data system. (See AV/MD22)

Evaluate the continuous data that were recently produced (2014/2015) and apply additional 

review as needed.

Perform second-level review and validation for all the filter-based PM10 and PM2.5 data produced 

in 2014 and 2015, and correct data errors in AQS. (See AV/MD15, AV/MD19)

• Evaluate laboratory quality control for PM2.5 and procedures for consistency with 

EPA regulation, guidance, and standard laboratory practices (such as holding times, 

and temperature and humidity regulatory requirements), and determine the impact of 

deficiencies on the 2014-2015 data. (See AV/MD20, AV/MD21)

• Where temperature or humidity do not meet regulatory requirements, data must be 

invalidated; all other data should be flagged at a minimum. (See AV/MD19)

• Evaluate the impacts of transport temperature on the PM2.5 data from the Lancaster 

Station and apply flags or invalidate data as appropriate. (See AV/MD12)
Auditor: Mathew Plate 

Final Report - August 2016

128

Finding # AV/MD2 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

The number of staff and managers and the amount of their time allocated to the air monitoring 

program were inadequate for the current network and associated activities.

Description:

MDAQMD is responsible for operation and maintenance of seven sites across a sizeable 

geographic region. It also runs a weigh laboratory and is responsible for data review and 

validation for all of its sites. Given the findings EPA notes in this report, current staffing levels 

and managerial time did not appear to be adequate to effectively support the existing monitoring 

program.

References:

Suggested Actions to Address Finding:

Evaluate the monitoring network and associated workload and resources. Determine if workload 

can be reduced, if additional staff are needed, and/or additional management time is necessary to 

facilitate a high quality monitoring program.
Auditor: Mathew Plate 

Final Report - August 2016

129

Finding # AV/MD3 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

Documentation, while generally present, was sometimes incomplete, not legally defensible, 

poorly organized, and not reviewed by a peer or a manager.

Description:

For continuous data, information was generally recorded using field forms and in the AirVision 

system. The use of field and instrument logbooks was limited.

For filter-based data, information was generally recorded in the laboratory database and on 

sample tracking forms. The use of laboratory and instrument logbooks was limited.

 It could not be verified that peer and/or manager review were conducted for the forms 

evaluated.

 The system for organizing and storing field records did not appear to be consistent and 

there were opportunities for records to be lost or misplaced.

 Many of the records were written in pencil and some of the corrections were not properly 

marked. These practices compromise the defensibility of the data.

 The records reviewed did not sufficiently document aspects of the program so as to fully 

track the data generation process. Some activities were not being recorded and 

information such as instrument serial numbers was not always recorded.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. (Sections 5 & 6)

2003 NELAC Standard, EPA/600/R-04/003, June 2003 (Quality Systems Section), that suggests 

documentation required for all aspects of ambient air monitoring operations, provides the 

following guidance:

5.5.5.5 The laboratory shall maintain records of each major item of equipment and its 

software significant to the environment tests performed. The records shall include at least the 

following information:

a) The identity of the item of equipment and its software. 

b) The manufacturer’s name, type identification, and serial number or other unique 

identification.

c) Checks that equipment complies with the specification (see 5.5.5.2).

d) The current location.
Auditor: Mathew Plate 

Final Report - August 2016

130

e) The manufacturer’s instructions, if available, or reference to their location.

f) Dates, results and copies of reports and certificates of all calibrations, adjustments, 

acceptance criteria, and the due date of next calibration.

g) The maintenance plan, where appropriate, and maintenance carried out to date; 

documentation on all routine and non-routine maintenance activities and reference 

material verifications.

h) Any damage, malfunction, modification or repair to the equipment.

i) Condition when received (e.g., new, used, reconditioned), if available.

Suggested Actions to Address Finding:

Develop a system to generate, store, and review records of field and laboratory operations. 

AV/MD AQMD may want to consider adopting an electronic record keeping system for these

records.
Auditor: Mathew Plate 

Final Report - August 2016

131

Finding # AV/MD4 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

The AV/MD AQMD staff, while part of the PQAO, were generally not following CARB SOPs 

and QA planning documents as stated in their R&R document.

Description:

The AV/MD AQMD has agreed that it will follow the CARB PQAO SOPs and QAPPs where 

applicable and will generate its own if needed. AV/MD AQMD has evaluated its procedures and 

was working to identify and create SOPs for activities not covered by CARB. At the time of the 

TSA they had not implemented CARB SOPs and QA procedures for other aspects of their 

program. Note that the CARB PQAO needs to develop QAPPs to replace their PM2.5 QAPP from 

1998 and QA Manual from 2007.

For field operations, instead of following specific SOPs, AV/MD AQMD relied heavily on its 

senior field technician to ensure that procedures were adequate and consistent. For the most part 

this had been successful. However, several inconsistencies with standard practices were noted. 

Additionally, not having specific procedures in SOPs compromises the defensibility of the data. 

Relying on one staff person to maintain institutional knowledge could also be problematic in the 

future.

For laboratory operations, there was a transition several years ago to a new technician. Because 

of limited training and the lack of SOPs and a QAPP, these operations were compromised. 

References:

40 CFR Parts 31 and 35 (grant recipients are required to document their quality systems) 

Roles and Responsibilities document, CARB and AV/MD AQMDs. January 24, 2014. 

http://arb.ca.gov/aaqm/qa/pqao/repository/rr_docs.htm

EPA Requirements for QA Project Plans (QA/R-5), EPA/240/B-01/003, March 2001.

40 CFR 58 Appendix A, Section 2.1, Quality Management Plans and Quality Assurance Project 

Plans

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 
Auditor: Mathew Plate 

Final Report - August 2016

132

Suggested Actions to Address Finding:

Adopt or develop SOPs and a QAPP for the PM laboratory as soon as is practical. These should:

• Evaluate laboratory quality control for PM2.5 and institute procedures that are consistent 

with EPA regulation, guidance, and standard laboratory practices. (See AV/MD20)

• Include a system to consistently and completely evaluate temperature and humidity. All 

temperature and humidity data should be evaluated by a qualified technician during 

second level data review. (See AV/MD19)

• Develop a system to ensure that filters are used and weighed within established holding 

times. (See AV/MD21)

• Improve the PM filter sample custody and handling procedures. (See AV/MD23)

• Implement appropriate improvements to the weigh room procedures and conditions. (See 

AV/MD24)

• Include replacement of the current laboratory data system. (See AV/MD22)

Adopt or develop field SOPs. If CARB SOPs are adopted, the primary technician should help to 

develop a list of AV/MD AQMD-specific procedural differences. These should include 

procedures to:

• Inspect sampling inlets during each routine site visit. (See AV/MD14)

• Transfer the electronic files for each run from the sampler and use as needed in the 

review and validation process; describe how electronic files should be saved. (See 

AV/MD13)

• Calculate and review residence time. (See AV/MD9)

• Eliminate non-Teflon material from inlets/manifolds of gaseous monitoring sites. (See 

AV/MD10)

• Develop a periodic process for verifying the quality of zero air. (See AV/MD11)

• Develop a maintenance schedule for zero air scrubbers. (See AV/MD11)

• Document maintenance and verification of zero air. (See AV/MD11)

• Develop a cylinder replacement / re-certification schedule to prevent the use of cylinders 

beyond their expiration data. (See AV/MD8)

• Transport PM2.5 filters at a controlled temperature. (See AV/MD12)

Once SOPs/QAPPS are adopted, ensure that staff are aware of the documents, and have been 

trained on their use.
Auditor: Mathew Plate 

Final Report - August 2016

133

Finding # AV/MD5 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

The AV/MD AQMD training program was not robust enough to ensure that new staff are

properly trained, current staff are cross trained, and current staff’s knowledge is refreshed and 

up-to-date.

 

Description:

AV/MD AQMD staff attended the CARB PQAO training and have begun to work with staff in 

other California districts to expand their understanding of the program. However, it is unclear 

how new AV/MD AQMD staff will be trained in the future. 

Additionally, there was inadequate cross-training, possibly due to inadequate staff levels. 

If one or more staff person were to leave AV/MD AQMD, procedures may not be consistent and 

another loss in data integrity may occur. This concern is compounded by a lack of documented 

procedures/SOPs. 

Currently, additional training on PM2.5 procedures, station operation, and EPA’s AQS is urgently 

needed. 

References:

EPA Requirements for QA Project Plans (QA/R-5), EPA/240/B-01/003, March 2001.

QA Handbook for Air Pollution Measurement Systems, Volume II, EPA-454/b-08-002, May 

2013. (Section 4.0)

Suggested Actions to Address Finding:

Develop a training program that includes cross-training, succession planning, and tracking.
Auditor: Mathew Plate 

Final Report - August 2016

134

Finding # AV/MD6 (General)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – General

Finding:

The AV/MD AQMD did not rely on the CARB PQAO for independent QA and support.

Description:

CARB has developed an R&R document with the AV/MD AQMD. AV/MD AQMD has begun 

to implement the roles and responsibilities outlined in the document. As noted in several 

findings, AV/MD AQMD’s implementation of the PQAO documents and processes was

incomplete.

It was also noted that AV/MD AQMD was reluctant to ask for support from or initiate corrective 

actions with the CARB PQAO’s independent QA programs. This was evident in discussion with 

AV/MD AQMD staff and from several significant QA issues that had not been elevated to the 

PQAO. AV/MD AQMD’s lack of an independent QA element necessitates reliance on CARB’s 

quality system to provide independent interpretation and evaluation. This requires that AV/MD 

AQMD elevate significant issues to the CARB QA Manager for consultation. This can be done 

by self-issuing a CAN or by formally contacting the CARB QA Manager or his representatives. 

References:

40 CFR Part 58, Appendix A, Section 3.1:

Primary Quality Assurance Organization. A primary quality assurance organization is defined 

as a monitoring organization or a coordinated aggregation of such organizations that is 

responsible for a set of stations that monitors the same pollutant and for which data quality 

assessments can logically be pooled. Each criteria pollutant sampler/monitor at a monitoring 

station in the SLAMS network must be associated with one, and only one, primary quality 

assurance organization.

40 CFR Part 58, Appendix A, Section 3.1.1:

Each primary quality assurance organization shall be defined such that measurement 

uncertainty among all stations in the organization can be expected to be reasonably 

homogeneous, as a result of common factors. Common factors that should be considered by 

monitoring organizations in defining primary quality assurance organizations include:

(a) Operation by a common team of field operators according to a common set of 

procedures;

(b) Use of a common QAPP or standard operating procedures;

(c) Common calibration facilities and standards;
Auditor: Mathew Plate 

Final Report - August 2016

135

(d) Oversight by a common quality assurance organization; and

(e) Support by a common management, laboratory or headquarters.

Suggested Actions to Address Finding:

Institute mechanisms by which AV/MD AQMD communicates regularly with CARB. Among 

other topics, communications should address:

• Formal notifications to EPA and CARB regarding air monitoring system modifications.

• AQS data corrections, so that metadata are correct and samplers are properly closed. (See 

AV/MD7)

Develop and implement an internal corrective action process and respond to CANs and AQDAs 

in a timely fashion. (See AV/MD8)
Auditor: Mathew Plate 

Final Report - August 2016

136

Finding # AV/MD7 (Network Management)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Network Management

Finding:

The transition from filter-based PM10 to continuous PM10 was not properly defined.

Description:

AV/MD AQMD did not notify EPA and CARB of the transition dates from filter-based to 

continuous PM10 for each site. Additionally, information in AQS did not clearly define the 

transition. Filter-based samplers were not closed out, data were not combined, and at sites where 

both types of instruments were operating simultaneously, it is unclear which data set is intended 

to be primary.

References:

40 CFR Part 58

Suggested Actions to Address Finding:

See AV/MD7 portion of AV/MD6.
Auditor: Mathew Plate 

Final Report - August 2016

137

Finding # AV/MD8 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

The span/precision standard at the Lancaster site expired on 8/6/2014.

Description:

The span/precision cylinder at the Lancaster site was expired. This was noted two months earlier 

by CARB on 2/26/15 (CAN # 159).

Because the standard was expired there is the possibility that the concentrations being measured 

are not accurate. This has significant implications for the defensibility of the data. 

References:

40 CFR Part 58 Appendix A, Section 2.6

Suggested Actions to Address Finding:

Replace the expired cylinder.

Re-certify the expired cylinder to demonstrate that the data collected after 8/6/2014 are accurate.

See AV/MD8 portion of AV/MD4 and AV/MD6.
Auditor: Mathew Plate 

Final Report - August 2016

138

Finding # AV/MD9 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

AV/MD AQMD staff were not tracking residence time.

Description:

AV/MD AQMD relied on residence time calculations conducted by CARB auditors. These 

residence times only represent station 

conditions during audits, which occur 

once per year. In addition, EPA noted that 

CARB did not include the volume of 

“water/debris” bottle in residence time 

calculations at the Lancaster Station.

Without the bottle, the Lancaster 

residence time is 16-17 seconds when all 

three instrument are operating. If one 

instrument is taken off line, the residence 

time for the site is over the maximum 

specified residence time of 20 seconds. 

References:

40 CFR Part 58, Appendix E Section 9

Suggested Actions to Address Finding:

See AV/MD9 portion of AV/MD4.

If sites with multiple instruments have residence times over 10 seconds, add an additional pump 

to the manifold to ensure that residence time can be met when instruments are temporarily out of 

service.
Auditor: Mathew Plate 

Final Report - August 2016

139

Finding # AV/MD10 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

AV/MD AQMD staff were not evaluating manifold material requirements.

Description:

EPA noted that many of the seals used on the glass manifold were not Teflon as required by 

regulation.

References:

40 CFR Part 58, Appendix E Section 9

Suggested Actions to Address Finding:

See AV/MD10 portion of AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

140

Finding # AV/MD11 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

Zero air sources were not considered standards and maintained as recommended by the 

manufacturer.

Description:

Zero air scrubbers were used in place of certified zero air for instrument calibrations. This is a 

common practice and is acceptable. Because zero air is used to generate the zero point and the 

calibration mixes, it must be treated as a standard. As such, zero air scrubber maintenance and 

verification must be documented.

The zero air source observed at the Lancaster site had not been serviced in several years. There 

was no documentation as to the quality of the zero air or exactly when it had last been serviced. 

The instrument manual indicates that the NOx scrubber should be changed out every six months. 

The data evaluation also indicated that negative values were commonly seen during some 

periods of time. Negative values could be a sign of poor zero air quality during calibration and/or 

zero adjustments.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

40 CFR Part 50, Appendix F

1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the 

NO/NOX/NO2analyzer or which might react with either NO, O3, or NO2in the gas phase titration.

Suggested Actions to Address Finding:

See AV/MD11 portion of AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

141

Finding # AV/MD12 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

PM2.5 filters from the Lancaster site were not being transported at a controlled temperature back 

to the office.

Description:

EPA regulation requires that samples be transported and stored at temperatures less than 25°C 

for a 240-hour filter holding time and less than 4°C or the average ambient sampling temperature 

(whichever is higher) for a 30-day holding time. PM2.5 samples from the Lancaster site were not 

transported at a controlled temperature.

If samples were transported at 25°C or less, a 10-day holding time can be applied to these 

samples.

References:

40 CFR Part 50, Appendix L, Section 8.3.6 & 10.13

8.3.6 The post-sampling conditioning and weighing shall be completed within 240 hours 

(10 days) after the end of the sample period, unless the filter sample is maintained at 

temperatures below the average ambient temperature during sampling (or 4 °C or below 

for average sampling temperatures less than 4 °C) during the time between retrieval from 

the sampler and the start of the conditioning, in which case the period shall not exceed 

30 days. Reference 2 in section 13.0 of this appendix has additional guidance on 

transport of cooled filters.

10.13 . . . During the period between filter retrieval from the sampler and the start of the 

conditioning, the filter shall be maintained as cool as practical and continuously 

protected from exposure to temperatures over 25 °C to protect the integrity of the sample 

and minimize loss of volatile components during transport and storage . . .

Suggested Actions to Address Finding:

See AV/MD12 portions of AV/MD1 and AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

142

Finding # AV/MD13 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

The PM2.5 instrument files were not being downloaded from the samplers.

Description:

Each PM2.5 instrument records operational parameters during each sampling run. This 

information is used to demonstrate that instrument runs are valid and can be used to diagnose 

instrument or sample problems. AV/MD AQMD was only recording summary information from 

the PM2.5 instruments on the sampling sheets.

References:

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

See AV/MD13 portion of AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

143

Finding # AV/MD14 (Field Operations)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Field Operations

Finding:

The plug was missing from the rain cap at the Phelan Station and the Teflon O3 inlet tube was 

resting on the plastic funnel.

Description:

The gaseous sampler inlet should be protected from rain.

Because the inlet line is resting on plastic, plastic is essentially in the O3 sampler flow path.

References:

40 CFR Part 58, Appendix E Section 9

Suggested Actions to Address Finding:

Repair and re-orient the rain cap at the Phelan Station. 

See AV/MD14 portion of AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

144

Finding # AV/MD15 (Data Management)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Data Management

Finding:

Based on a limited review, EPA found a significant number of data errors in AQS.

Description:

The following types of errors were found in AQS:

 Values were reported that were actually the result of instrument maintenance issues.

 Values were included that were in error by an order of magnitude.

 Improper flags.

 Data were mistakenly invalidated instead of being flagged as exceptional events.

 Data were missing without flags being applied, particularly for filter-based data.

 Repeating data from an improper AQS upload.

Most of these issues were found during a limited review of data in AQS for several monitoring 

stations. As AV/MD AQMD does not routinely evaluate AQS for potential errors, these would 

not necessarily have been found.

References:

40 CFR Part 58.16

Suggested Actions to Address Finding:

See AV/MD15 portion of AV/MD1.
Auditor: Mathew Plate 

Final Report - August 2016

145

Finding # AV/MD16 (Data Management)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Data Management

Finding:

AV/MD AQMD’s system did not reliably or consistently back up all data.

Description:

In order to preserve data integrity, all data systems should have a reliable backup.

AV/MD AQMD was backing up most of the data to a central server, which has onsite backups. 

However, EPA noted that data from the laboratory data system had not been backed up to the 

server recently (this was an oversight that was corrected immediately). Other records and forms 

on satellite computers and laptops may not have been routinely backed up to the AV/MD AQMD 

server.

While the server was routinely backed up, there was no offsite backup of this system. Because 

AV/MD AQMD is centralized into one building, an incident at the building could cause a 

temporary or permanent loss of primary and backup records. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

Review all records stored on remote systems and develop an inventory.

Develop a schedule for backups to the server and a process to verify that backups are occurring.

Consider having an offsite backup of the AV/MD AQMD server.
Auditor: Mathew Plate 

Final Report - August 2016

146

Finding # AV/MD17 (Data Management)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Data Management

Finding:

Some of the metadata in AQS were missing or incorrect.

Description:

EPA found that site scale, a parameter that should be included in AQS, was not included for 

several pollutants at several sites.

Probe height was often incorrect in AQS.

References:

40 CFR Part 58.16

Suggested Actions to Address Finding:

See AV/MD17 portion of AV/MD1.
Auditor: Mathew Plate 

Final Report - August 2016

147

Finding # AV/MD18 (Data Management)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – Data Management

Finding:

The data record was incomplete for several pollutants as negative values are routinely 

invalidated.

Description:

AV/MD AQMD policy was that all negative values reported by continuous instruments should 

be invalidated. On some occasions, instruments had consistently reported negative values over a 

period of time and/or during certain hours of the day. As a result, the data record was not 

complete.

References:

40 CFR Part 58.16

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

Suggested Actions to Address Finding:

See AV/MD18 portion of AV/MD1.
Auditor: Mathew Plate 

Final Report - August 2016

148

Finding # AV/MD19 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

Temperature and humidity issues in the PM weigh room compromised the quality of data 

collected in 2014 and possibly earlier. Temperature and humidity were not tracked, so it was not 

easy to identify out-of-control situations.

Description:

AV/MD AQMD indicated that the previous weigh room temperature and humidity control 

system was malfunctioning, causing multiple spike and dips in temperature and humidity each 

hour and significant drift each day. In late 2014 a new control system was installed and these 

problems appeared to be resolved.

Although AV/MD AQMD identified this problem, no data were flagged or invalidated and no 

system was instituted to determine what levels of drift and fluctuation were acceptable.

Based on EPA’s evaluation of some of the temperature and humidity data, most or all of the 

affected PM2.5 data and some of the PM10 data from 2014 may need to be invalidated because 

regulatory requirements were not met.

According to PM2.5 monitoring regulations, temperature must be between 20-23oC and controlled 

within 2o C every 24 hours. Humidity must be between 30 and 40 percent RH, control within 5% 

RH over 24 hours and not more than 5% RH different between pre- and post-weigh. Temperature 

variability, humidity variability, and % RH difference between pre- and post-weigh are not 

evaluated by AV/MD AQMD.

References:

40 CFR Part 50 Appendix L, Section 8.0

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

See AV/MD19 portion of AV/MD1 and AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

149

Finding # AV/MD20 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

The quality control activities utilized for PM2.5 filter weighing were insufficient and inconsistent 

with EPA regulation, guidance, and general laboratory principles.

Description:

New laboratory blanks were not initiated with each new batch of filters as required by 40 CFR 

Part 50 Appendix L, and there was an insufficient number of new blank filters to assign one for 

each post-weigh batch as recommended by EPA Method 2.12. However, AV/MD AQMD was

selecting blank filters periodically and reweighing them routinely.

Samples were not routinely reweighed as laboratory duplicates as is recommended by Method 

2.12 (one per weigh session) and standard laboratory practices (5-10% duplicates). AV/MD 

AQMD did record duplicate readings from 50% of field samples. 

Lot blanks were not evaluated as required by Method 2.12. Without the evaluation of lot blanks, 

it was possible that the equilibration time of 24 hours for pre-weigh was insufficient to yield a 

consistent filter mass. 

The balance was not evaluated each day with masses that bracket the filter mass. AV/MD 

AQMD only used one mass, 200mg, to verify the balance. Additionally the mass selected was

significantly less than the filter masses, which are close to 400 mg.

References:

40 CFR Part 50 Appendix L

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

2003 NELAC Standard, EPA/600/R-04/003, June 2003 

D.5.2 Analytical Variability/Reproducibility

Matrix Spike Duplicates (MSDs) or Laboratory Duplicates – Shall be analyzed at a minimum of 

1 in 20 samples per sample batch...

Suggested Actions to Address Finding:

See AV/MD20 portions of AV/MD1 and AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

150

Finding # AV/MD21 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

Holding times were not being met for all PM filters.

Description:

Based on data that EPA tracked, filters T1643235 to T1643249 did not appear to meet postsampling holding time (there may be a typo on the weigh date as an equilibration time of one 

minute rather than 24 hours is entered on the post-run data entry log). Filters T1643242 to 

T1643249 did not meet the pre-sampling holding time.

Many filters from the Lancaster site may not have met the 10-day holding time for filters not 

transported at a reduced temperature.

References:

40 CFR Part 50 Appendix L

8.3.5 The pre-sampling (tare) weighing shall be within 30 days of the sampling period.

8.3.6 The post-sampling conditioning and weighing shall be completed within 240 hours 

(10 days) after the end of the sample period, unless the filter sample is maintained at 

temperatures below the average ambient temperature during sampling (or 4 °C or below 

for average sampling temperatures less than 4 °C) during the time between retrieval from 

the sampler and the start of the conditioning, in which case the period shall not exceed 

30 days. Reference 2 in section 13.0 of this appendix has additional guidance on 

transport of cooled filters.

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

See AV/MD21 portions of AV/MD1 and AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

151

Finding # AV/MD22 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

The data system used for PM weighing was not documented, understood by current staff, 

flexible, or fully validated.

Description:

AV/MD AQMD was using a laboratory data system for PM weighing that was programmed by 

former staff. This data system was not flexible enough to accommodate needed changes to the 

weighing process. In order to save time and reduce the number of errors in the weighing process, 

the data system should be able to receive data directly from the balances, flag data outside of 

quality control limits, allow for flexibility in data sheet outputs and report generation, and 

recognize balance checks.

There was no documentation as to how the data system was programed and validated, and its 

programing was not easily understood. Generally all electronic systems should be documented 

and validated. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

2003 NELAC Standard, EPA/600/R-04/003, Section 5.5.4.7.2

Suggested Actions to Address Finding:

See AV/MD22 portions of AV/MD1 and AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

152

Finding # AV/MD23 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

Storage and custody of PM filters were not consistent with general sample handling practice, 

EPA guidance and regulations.

Description:

The following were noted by EPA:

 The sample storage freezer was not secure and is used for storage of ice and food.

 Temperature in the sample storage freezer was not monitored regularly with a calibrated 

thermometer.

 Field sampling/custody forms were not always kept with the filters prior to receipt by the 

laboratory technician.

 Access to the weigh room was not limited.

 The dates and times that filters were prepared for transfer to the field and received from 

the field are not adequately tracked and custody was not always formally transferred 

between technicians.

 Filters were routinely stored three to four years - not five years as specified by EPA 

regulation - prior to disposal.

References:

2003 NELAC Standard, EPA/600/R-04/003: Section 5.4.12.2, Technical Records; Section 5.5.8, 

Handling of Samples

5.5.8.4 The laboratory shall have procedures and appropriate facilities for avoiding 

deterioration, contamination, loss or damage to the sample during storage, handling, 

preparation and testing. Handling instructions provided with the sample shall be 

followed. When samples have to be stored or conditioned under specified environmental 

conditions, these conditions shall be maintained, monitored and recorded. . . .

a) Samples shall be stored according to the conditions specified by preservation 

protocols: . . .

2) Samples shall be stored away from all standards, reagents, food and 

other potentially contaminating sources. Samples shall be stored in such 
Auditor: Mathew Plate 

Final Report - August 2016

153

a manner to prevent cross contamination.

40 CFR Part 58.16 (f) Data Submittal and Archiving Requirements

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

See AV/MD23 portion of AV/MD4.
Auditor: Mathew Plate 

Final Report - August 2016

154

Finding # AV/MD24 (PM Laboratory)

Agency: California Air Resources Board – Antelope Valley and Mojave Desert 

AQMDs

Date of Audit: April 17, 2015

Program Area: Antelope Valley and Mojave Desert AQMDs – PM Laboratory

Finding:

Several aspects of general weigh laboratory procedures were not consistent with EPA guidance.

Description:

The following deficiencies were noted:

 The primary and working weights were not used as recommended in Method 2.12.

 Filters received from EPA were not tracked.

 Cassette cleaning was not sufficient.

 Some materials observed in the weigh room could lead to particulate or static issues.

 The filter inspection process is not sufficiently rigorous.

 The balances were not isolated from vibrations.

 Old cassettes in use may need to be replaced.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013.

Quality Assurance Guidance Document 2.12, Monitoring PM2.5 in Ambient Air Using 

Designated Reference of Class I Equivalent Methods, January 2016

Suggested Actions to Address Finding:

See AV/MD24 portion of AV/MD4.
Auditor: Gwen Yoshimura

Final Report - August 2016

155

Finding # NCU – Positive Finding 1

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – General

Finding:

NCUAQMD was implementing a number of good practices in support of site and instrument 

maintenance, and data management.

Description:

NCUAQMD was implementing a number of good practices. Among others, EPA observed:

 The inside of monitoring shelters appeared clean and generally well kept.

 An inventory list of spare monitoring equipment was kept at monitoring sites.

 Copies of SOPs were found at the monitoring sites.

 Continuous instrument data was backed up nightly to NCU servers to help prevent data 

loss.

 Original raw ambient data records are kept even if invalidated. 

References:

Suggested Actions to Address Finding:

NA
Auditor: Gwen Yoshimura

Final Report - August 2016

156

Finding # NCU1 (General)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – General

Finding:

The current purpose/objectives of NCUAQMD’s NAAQS monitoring network were unclear.

Description:

During the audit, NCUAQMD staff provided some reasons behind the initiation of monitoring 

within NCUAQMD (permits, previous APCO directive, etc). Given the turnover in management 

and age of the permit, it was unclear why some monitors were continuing to be run and what 

were the current purpose/monitoring objectives for each one.

Staff did not know the monitor type of the non-FEM BAM PM2.5 monitor that has been operating 

at Weaverville since January 2014. NCUAQMD was under impression that after 24 months, all 

SPM NAAQS parameters must be converted to SLAMS.

NCUAQMD’s 2014 annual monitoring network plan labeled most monitors as special purpose,

while the 2015 TSA questionnaire forms labeled most monitors SLAMS.

A monitor’s purpose and monitor type affect what regulations apply. Resources are also required 

to maintain a monitoring network. It is therefore important for both planning and monitoring 

staff to know what is in their monitoring network, and assess annually whether the monitoring 

being conducted is appropriate.

References:

40 CFR 58, Appendix D

Suggested Actions to Address Finding:

Work with CARB to define the objectives of the monitoring network and types of monitors 

within the network. Align the network to fulfill those objectives, taking resources into account. 

Include updated information, including any SLAM site start-up or shut-down requests, to EPA 

for approval as part of the next Annual Monitoring Network Plan.
Auditor: Gwen Yoshimura

Final Report - August 2016

157

Finding # NCU2 (General/Quality Assurance)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – General/Quality Assurance

Finding:

QA document gaps were identified.

Description:

SOPs that NCUAQMD was using during the time of the audit had not been reviewed/approved 

by CARB, and/or QAPPs were not being followed. 

 NCUAQMD was using its own data management SOP for reviewing and validating 

NAAQS-comparable data. This SOP had not been reviewed or approved by CARB. 

 TECO gas and PM2.5 GRIMM 180 SOPs show as “in-progress” on CARB’s QA 

document online repository. All SOPs should be completed and approved by CARB.

 Some BAAQMD SOPs were present at sites, but there was no indication that 

NCUAQMD was using these according to CARB’s list of NCUAQMD SOPs.

 A records management plan for air monitoring activities was not being followed.

References:

40 CFR Part 58, Appendix A, Section 2 (SOPs)

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambient Air Quality 

Monitoring Program, EPA-454/B-13-003, May 2013. (Sections 5 and 17)

Suggested Actions to Address Finding:

Update QAPP/SOPs to represent current practices and to be consistent with EPA regulation and 

guidance on a regular schedule or as needed in accordance with 40 CFR 58, Appendix A. Submit 

QAPP/SOPs to CARB for approval, giving priority to NAAQS-related documents. This includes 

but is not limited to the following:

• Adjust action levels to be in conformance with EPA guidance and in consultation with 

CARB. (See NCU12)

• Follow EPA guidance, and NCUAQMD quality document requirements as outlined in the 

R&R document, regarding appropriate documentation. (See NCU5)

• Use the monthly maintenance sheets as described by CARB’s Quality Assurance Manual. 

The check sheets may be found online at 

http://www.arb.ca.gov/aaqm/qa/pqao/repository/qm_docs.htm . (See NCU6)

• Replace sampling lines on a schedule that is in conformance with the replacement 

schedule followed by the CARB PQAO. (See NCU7)
Auditor: Gwen Yoshimura

Final Report - August 2016

158

• Conduct and document regular cleaning of rooftop equipment. (See NCU8)

• Lock the PM filter refrigerator and limit access to personnel handling filters. (See NCU9)

• Data collected with FEM/FRM should be submitted into AQS promptly as required. (See 

NCU10)

• Adopt a procedure for verification of zero air. (See NCU13)

• Adopt a formalized and consistent corrective action process that can be used to resolve 

internal matters; involve CARB when appropriate. (See NCU14)

• Develop a document or process for documenting what QC checks and data validation 

processes are conducted for filter data by NCUAQMD, CARB, and NCUAQMD’s 

contract lab. (See NCU16)

• Ensure that NCUAQMD data management software is in conformance with section 14 of 

the QA Handbook. (See NCU11)

Provide operators with work aids until SOPs are finalized.

Develop a tracking system to indicate that operators have read and understand the QAPP/SOPs

and have had an opportunity to discuss them with their supervisor. 

Adopt and implement a records management plan.
Auditor: Gwen Yoshimura

Final Report - August 2016

159

Finding # NCU3 (General)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – General

Finding:

Training specifications/requirements for air monitoring personnel were not well defined and 

completion was not tracked.

Description:

NCUAQMD did not provide a training plan for air monitoring personnel. While the roles and 

responsibilities document between NCUAQMD and CARB discusses training, it is not an 

actual training plan. That is, it does not specify for each job position what courses must be 

taken, what training must be accomplished, and what proficiencies must be demonstrated by 

what date or with what frequency.

During the TSA, the primary field technician noted not being familiar with 40 CFR 58 

Appendix A requirements, EPA’s QA Manual, nor zero/precision/span QC checks conducted 

on gas instruments. The primary field technician did not attend CARB’s PQAO training 

Modules I & II and was not familiar with CARB’s technical bulletins. NCUAQMD did plan to 

send the primary site technician to the CARB PQAO Module III training.

Trainings should include review and sign-off of approved QA documents (e.g. QMP, QAPPs, 

SOPs) with which staff are expected to be familiar, attendance at all PQAO module trainings 

or review of videos that are posted to the web after the trainings, instrument manufacturer 

trainings, other CARB or EPA regularly offered trainings, and on-the-job training.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Section 4)

Suggested Actions to Address Finding:

Specify the training requirements and keep a training file for staff. Align training requirements 

with CARB PQAO requirements. This includes but is not limited to reviewing requirements 

for filling out CoC forms with staff, provide training if necessary and checking on 

implementation. (See NCU15)
Auditor: Gwen Yoshimura

Final Report - August 2016

160

Finding # NCU4 (General)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – General

Finding:

Major monitoring purchases have been problematic for NCUAQMD and CARB.

Description:

With the revision to the monitoring regulations in 2006, NCUAQMD formally became part of 

the CARB PQAO. Since that time, CARB has had a QA oversight role which necessitates 

communication and collaboration between districts within the CARB PQAO and CARB.

Instruments collecting data for comparison to the NAAQS are subject to 40 CFR 58 

Appendices A, C, and E. As one aspect of fulfilling its QA oversight role, CARB conducts 

audits of all sites within California to assist in meeting Appendix A requirements. In order to 

conduct audits, CARB must have the appropriate equipment and therefore should be consulted 

with prior to new instrument purchases. CARB also provides other services such as acceptance 

testing, calibrations, support for QA collocation, and troubleshooting.

NCUAQMD did not consult with CARB on purchases of major equipment, which has led to 

concerns about meeting PQAO requirements, challenges for CARB to provide support (such 

as audits and troubleshooting when instrument-specific issues arise), concerns with instrument 

comparability to the NAAQS, and overall difficulty meeting the commitments laid out in the 

R&R document.

References:

Suggested Actions to Address Finding:

Consult and come to agreement with CARB on major monitoring equipment purchases so that 

CARB can provide support (e.g. acceptance testing, calibrations, QA collocation, etc.), as 

outlined in the R&R document.
Auditor: Gwen Yoshimura

Final Report - August 2016

161

Finding # NCU5 (Field Operations)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Field Operations

Finding:

Field documentation gaps and defensibility issues were identified.

Description:

• A station log was kept via a Notepad text file in a computer inside the monitoring 

shelter. NCUAQMD staff indicated that notes from instruments within the monitoring 

shelter were recorded on this log.

o The Notepad text file was live every time it is opened and entries previously 

made could be altered without leaving a trace in the system.

o NCUAQMD staff indicated that logbook entry copies were manually 

downloaded to maintain some integrity and historical record, however these 

back-ups may have been occurring anywhere from three days to a month after 

original entry.

• NCUAQMD staff were unable to trace last manifold cleaning conducted at Jacobs. 

Staff recollection was that the cleaning had been conducted around the last time CARB 

auditors visited the site.

Entries made in electronic logbooks should not be easily alterable. It should be clear who 

made entries on what date, such that any alterations to entries can be seen in a data audit.

Logbooks are otherwise not defensible. What is recorded in the hardcopy logbook versus the 

electronic station logbook should be standardized and outlined in an SOP.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. 

Technical Note- Use of Electronic Logbooks for Ambient Air Monitoring, EPA, April 20, 

2016. Available at: 

https://www3.epa.gov/ttnamti1/files/policy/Electronic__Logbook_Final_%204_20_16.pdf

Suggested Actions to Address Finding:

See NCU5 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

162

Finding # NCU6 (Field Operations)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Field Operations

Finding:

Monthly maintenance check sheets, as prescribed by CARB’s SOPs, have not been used.

Description:

According to the February 2015 NCUAQMD/CARB R&R document, NCUAQMD has agreed to

follow CARB’s quality documents. Any deviations are to be specified in an addendum and 

submitted to CARB for review and approval. At the time of the audit, no deviations had been 

approved. CARB’s Quality Assurance Manual, Volume II Standard Operating Procedures for 

Air Quality Monitoring, section 2.0.1 General Operating Procedures, requires the use of 

“Monthly Quality Control Maintenance Check sheet[s]” to record specific QA/QC checks. 

Monthly quality control maintenance check sheets are included in CARB’s SOPs for each 

instrument. These check sheets were not found at the sites.

Following the onsite audit, NCUAQMD emailed examples of documents used to record weekly 

checks. One example for gaseous instruments (Humboldt Hill, Sep. 10, 2014 – Nov 4, 2014) 

follows:
Auditor: Gwen Yoshimura

Final Report - August 2016

163

The excel spreadsheet used to record PM2.5 FRM checks at Humboldt Hill:
Auditor: Gwen Yoshimura

Final Report - August 2016

164

These check sheets do not consistently contain the information required to be recorded in 

CARB’s SOP monthly maintenance check sheets. If NCUAQMD wishes to continue to use their 

own check sheets instead of the monthly maintenance check sheets in CARB’s SOPs, CARB

must approve their usage as part of NCUAQMD’s SOPs or an approved SOP addendum.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

Suggested Actions to Address Finding:

See NCU6 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

165

Finding # NCU7 (Field Operations)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Field Operations

Finding:

Sampling lines/tubes were not regularly replaced.

Description:

NCUAQMD staff noted that sampling lines are generally cleaned but not replaced. Sampling 

lines should be replaced once per year as they can deteriorate in ambient conditions (e.g. UV 

exposure) and more frequently as indicated by pollutant load or contamination.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

40 CFR part 58, Appendix E

Suggested Actions to Address Finding:

See NCU7 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

166

Finding # NCU8 (Field Operations)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Field Operations

Finding:

Some rooftop equipment was not clean.

Description:

Regular cleaning should be conducted and documented to avoid the build-up of debris that may

confound ambient data.

a. Gas inlet at Jacobs.
Auditor: Gwen Yoshimura

Final Report - August 2016

167

b. FRM PM2.5 sampler temperature probe at Jacobs.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. (Section 7, 11)

Suggested Actions to Address Finding:

See NCU8 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

168

Finding # NCU9 (Field Operations)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Field Operations

Finding:

The refrigerator containing PM filters that have been collected from the field was not locked.

Description:

The NCUAQMD building is generally restricted to staff entry. However, the refrigerator 

containing the PM samples has been kept in a common space kitchen accessible to all 

NCUAQMD staff (including non-monitoring staff).

The PM filter refrigerator contained a visible lock. NCUAQMD staff indicated that the key for 

this was not available.

Samples should be kept in a secured location to minimize opportunity for contamination or 

alteration of the sample.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. (Section 8)

“...Samples must be properly handled to ensure that there is no contamination and that 

the sample analyzed is actually the sample taken under the conditions reported. For this 

reason, whenever samples are not under the direct control of the sample custodian, they 

should be kept in a secured location. This may be a locked vehicle, locked refrigerator, or 

locked laboratory with limited access. It is highly recommended that all samples be 

secured until discarded. These security measures should be documented by a written 

record signed by the handlers of the sample on the COC form or in a laboratory 

notebook, indicating the storage location and conditions...”

Suggested Actions to Address Finding:

See NCU9 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

169

Finding # NCU10 (Data Management)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Data Management

Finding:

Data collected with FEM/FRM instruments were not uploaded to AQS as required by CFR.

Description:

As of the date of the TSA, GRIMM data from the Humboldt Hill, Weaverville, and Crescent 

City stations had not been entered into AQS, as discussed in EPA’s letter to NCUAQMD dated 

December 10, 2014.

At the time of the audit, NCUAQMD noted that Weaverville GRIMM data approved by EPA for 

NAAQS exclusion were pending submittal into AQS. NCUAQMD noted that other GRIMM 

data are being held back pending EPA’s review of an anticipated FEM waiver request for these

data.

EPA notes that on December 10, 2014, EPA sent NCUAQMD a detailed letter describing how 

data from the GRIMM, at various monitoring sites, should be considered. While the letter stated 

that EPA approved Weaverville for NAAQS exclusion, this approval was based on 40 CFR 

58.20(d) (SPMs operating under 24 months may receive approval to use the NAAQS exclusion 

designation in AQS). The letter consistently states that data collected, at Weaverville or 

elsewhere, must be entered into AQS, as required by 40 CFR 58.16.

CFR and grant requirements apply to these data, and require reporting to AQS.

References:

40 CFR 58.16

December 10, 2014 letter from Meredith Kurpius, U.S. EPA, to Richard L. Martin, NCUAQMD 

concerning GRIMM data.

Suggested Actions to Address Finding:

See NCU10 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

170

Finding # NCU11 (Data Management)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Data Management

Finding:

NCUAQMD had not verified that their data processing software meets performance guidelines 

consistent with the QA Handbook, Volume II, Section 14.

Description:

During the audit, NCUAQMD staff indicated that they did not know whether their data 

management software was consistent with the QA Handbook, Volume II, Section 14.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. (Section 14)

Suggested Actions to Address Finding:

See NCU11 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

171

Finding # NCU12 (Quality Assurance)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Quality Assurance

Finding:

The level of data quality performance requirements for gaseous pollutants were less stringent 

than those in EPA regulation and guidance.

Description:

40 CFR part 58 Appendix A, Section 2.3 sets forth data quality performance requirements. 

 O3 action level is 7%; NCUAQMD is using 10%.

 CO and SO2 action level is 10%; NCUAQMD is using 15%.

References:

40 CFR part 58, Appendix A, Section 2.3

Suggested Actions to Address Finding:

See NCU12 portion of NCU2.

In consultation with CARB and EPA, determine how to address historical data collected using 

the higher thresholds should be reported to AQS.

Work collaboratively with CARB to identify more effective means of communication such that 

NCUAQMD is fully aware of requirements.
Auditor: Gwen Yoshimura

Final Report - August 2016

172

Finding # NCU13 (Quality Assurance)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Quality Assurance

Finding:

NCUAQMD did not have a procedure for checking/verifying zero air.

Description:

Zero air scrubbers are used in place of certified zero air for instrument calibrations. Because zero 

air is used to generate the zero point and the calibration mixes, it must be treated as a standard. 

As such, zero air scrubber maintenance and verification must be documented. 

NCUAQMD indicated that it would like to participate in CARB’s efforts to implement a zero air 

verification procedure. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

40 CFR part 50, Appendix F

1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the 

NO/NOx/NO2 analyzer or which might react with either NO, O3, or NO2 in the gas phase 

titration.

Suggested Actions to Address Finding:

See NCU13 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

173

Finding # NCU14 (Quality Assurance)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Quality Assurance

Finding:

NCUAQMD had not established procedures to initiate corrective actions for resolution of 

internal matters. 

Description:

The NCUAQMD and CARB R&R document states that NCUAQMD will “Utilize [C]ARB’s 

CAN process to report instrument malfunctions, operational problems, data quality issues...” 

It is unclear whether NCUAQMD intends to utilize CARB’s CAN process to resolve internal 

matters as well.

During the audit, the staff relayed several ongoing instrument issues. Such systemic problems 

are the type that both NCUAQMD and CARB indicated would be appropriate to raise in a 

corrective action.

While CARB-issued CANs for the NCUAQMD network have been used and closed,

NCUAQMD staff indicated that the CARB CAN SOP had not yet been reviewed and 

NCUAQMD had not initiated any CANs. 

References:

EPA Requirements for Quality Assurance Project Plans, EPA/240/B-01/003, March 2001, 

§3.4.1

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Sections 1.2.4, 10, & 15)

State of California Air Resources Board, Air Monitoring QA, Vol. 5, SOP Corrective Action 

Notification (http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf)

Suggested Actions to Address Finding:

See NCU14 portion of NCU2.
Auditor: Gwen Yoshimura

Final Report - August 2016

174

Finding # NCU15 (Quality Assurance)

Agency: California Air Resources Board – North Coast Unified AQMD

Date of Audit: May 26-28, 2015

Program Area: North Coast Unified – Quality Assurance

Finding:

A formal agreement describing the roles of NCUAQMD and the PM gravimetric contract lab 

(BAAQMD) in the generation of filter PM data was not in place.

Description:

NCUAQMD staff indicated that a portion of the 103 funding is directed to the BAAQMD to 

conduct the gravimetric analysis of PM filters. Reviewing the documents shared with EPA 

during the audit, there did not appear to be any information about the QC checks performed by 

the lab or what documentation will be produced by whom. 

Filters are sampled and collected by NCUAQMD, sent to the contract lab for analysis, and the 

data are uploaded by CARB. EPA was unable to find documents with details about the roles 

each agency plays in filter PM data handling and validation.

 NCUAQMD shared a one-page document received from BAAQMD noting the QA/QC 

measures that the contract lab follows and affirming that the data can be used for 

NAAQS comparison. EPA would expect this type of information, as well as the filter 

dates/range and the contact at the contract lab responsible for NCUAQMD gravimetric 

analysis, to be included in each batch of results.

 The CARB PQAO liaison noted that similar district-contract lab agreements may be

available for NCUAQMD to use as a guide.

 NCUAQMD should understand what data validation, if any, is conducted by CARB.

EPA notes that the 103 grant document does not contain the level of detail necessary to 

address this finding. For example, the agreement should address information such as how PM 

filter CoC forms are filled out. During the audit it was noted that:

 None of the sample CoC forms documented the filter temperature at the various filter 

handling steps. A column to record filter temperature was included on the CoC forms, 

however, these were left blank.

 The “Valid Time” field was blank in all forms.

 Two examples of voided CoC forms were provided that did not identify who wrote the 

lab comments nor were they dated. These forms were for sample dates 11/12/13 and 

11/18/13.

 Inconsistent date formats are recorded on the CoC forms. NCUAQMD uses one format 

while the lab uses another. The template does not specify a date reporting format.

 Mass units are not specified on the CoC template form.
Auditor: Gwen Yoshimura

Final Report - August 2016

175

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. 

Suggested Actions to Address Finding:

See NCU15 portion of NCU2.
Auditor: Dena Vallano

Final Report - August 2016

176

Finding # SLO - Positive Finding 1

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - General

Finding:

In general, the SLOCAPCD monitoring program was extremely robust and the district staff 

and managers involved in the program were committed to producing defensible data of known 

quality.

Description:

Specific examples of good practices observed during this audit include:

 Management and staff at all levels were proactively engaged in improving the 

program, including monitoring systems and other data collection processes.

 Staff and managers who participated were helpful during the audit.

 Field technicians were skilled and knowledgeable.

 Staff and management have created excellent internal tools and tracking forms for data 

validation, field operations, and training. 

 A new system for independent data review was being implemented and the process is 

well documented.

References:

Suggested Actions to Address Finding:

NA
Auditor: Dena Vallano

Final Report - August 2016

177

Finding # SLO - Positive Finding 2

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - General

Finding:

SLOCAPCD field technicians were knowledgeable and produce excellent documentation.

Description:

SLOCAPCD field technicians were dedicated and knowledgeable. Their internal tracking 

forms, logbooks, and training forms were exemplary.

References:

Suggested Actions to Address Finding:

NA
Auditor: Dena Vallano

Final Report - August 2016

178

Finding # SLO1 (General)

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - General 

Finding:

SLOCAPCD had not fully adopted a centralized corrective action process to resolve internal 

issues.

Description:

EPA quality management standards (EPA QA/R-2, Quality Improvement Section) require that 

management and staff “ensure that conditions adverse to quality” are prevented, identified 

promptly, fully defined, corrected, prevented from recurring, and documented as corrective 

actions which are tracked to closure. A formalized corrective action process allows for clear 

tracking of problems, faster identification of systemic issues, and ensures that all those 

affected by the issue are notified and that the corrective action is followed through to the 

appropriate conclusion. 

At the time of the TSA, SLOCAPCD had formally adopted the CARB Corrective Action 

Notification (CAN) process, and had been involved in CARB-initiated CANs. SLOCAPCD 

had not yet initiated corrective actions for internal issues or consistently communicated such 

issues to managers. Internally, site troubleshooting was noted in logbooks, and in an “Errata” 

log, which documented periods of data invalidation along with the rationale. Systemic issues 

were noted in their SOPs. However, corrective action was sometimes occurring informally via 

email, phone call, or in-person communication without centralized documentation.

While instrument/data problems and their solutions were often documented, documentation 

varied across log books, errata logs, and data annotations. Management was typically not 

informed of corrective actions. A centralized documentation and communication process 

should be in place for addressing suspect or missing data, instrument problems, or other 

problems observed in the field. Discussions should be documented so that the event can be 

reconstructed and reviewed to prevent recurrence. Relevant email records should be preserved 

and attached to the corrective action. Management should be sent copies of the corrective 

actions in a timely fashion, based on the severity of the problem, indicating whether the 

problem is ongoing or resolved. When complete, a final close-out of the corrective action 

should be signed by the responsible party and documented in a logbook. 

References:

EPA Requirements for Quality Assurance Project Plans, EPA/240/B-01/003, March 2001, 

§3.4.1
Auditor: Dena Vallano

Final Report - August 2016

179

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. (Sections 1.2.4, 10, & 15)

State of California Air Resources Board, Air Monitoring QA, Vol. 5, SOP Corrective Action 

Notification (http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf)

Suggested Actions to Address Finding:

Adopt a centralized corrective action process that can be used to consistently resolve internal 

matters and includes formal communication to managers; involve CARB when appropriate 

(i.e. systemic issues or significant data loss).
Auditor: Dena Vallano

Final Report - August 2016

180

Finding # SLO2 (Quality Assurance)

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - Quality Assurance

Finding:

QAPPs had not been completed or updated for O3, SO2, NO2, CO, PM2.5, and PM10 monitoring 

as required by regulation.

Description:

The CAA requires that organizations using EPA funding to collect environmental data develop a 

QAPP for each type of ambient pollutant being measured (40 CFR 58 Appendix A, Section 2.1). 

The QAPP describes all technical and quality aspects of a project and documents the results of 

the QA planning process for environmental data collection operations, thereby providing a 

project-specific “blueprint” for obtaining the type and quality of environmental data needed for a 

specific decision or use. While the monitoring staff at SLOCAPCD are very knowledgeable and 

operate the ambient air monitoring network diligently, the lack of updated QAPP documentation 

reduces the ability to implement ambient air monitoring requirements effectively and 

appropriately. In addition, QAPPs should be revised when standards or instruments change.

At the time of the TSA, SLOCAPCD had developed agency-specific QAPPs and submitted

amendments to existing CARB QAPP documentation. SLOCAPCD QAPPs were informally 

approved by CARB on 7/20/2015 and were pending final signature approval. 

References:

In accordance with 40 CFR Parts 31 and 35, grant recipients are required to document their 

quality systems. 

40 CFR 58 Appendix A, Section 2.1.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. 

Further guidance on developing QAPPs can be found in the guidance documents “EPA 

Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, March 2001, and 

“Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002.

Suggested Actions to Address Finding:

Update QAPPs for all criteria pollutants as necessary, with the goal of having plans for all 

criteria pollutants in place and submitted to EPA as soon as possible. Adopt CARB’s QAPP 

documentation, develop one specific to SLOCAPCD, or adopt CARB’s with an addendum 

specific to SLOCAPCD. Any deviation from the CARB QAPP documentation must be formally 

approved by CARB as the lead agency in the PQAO.
Auditor: Dena Vallano

Final Report - August 2016

181

Finding # SLO3 (Quality Assurance)

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - Quality Assurance 

Finding:

SLOCAPCD did not have updated and complete SOPs to reflect recent changes in operation.

Description:

SOPs detail the work procedures that are to be conducted or followed within an organization. 

SOPs document the way activities are to be performed to ensure consistent conformance to 

technical and quality system requirements and to support data quality. SOPs are intended to be 

specific to the organization or facility whose activities are described and assist that organization 

to maintain their quality control and quality assurance processes and ensure compliance with 

governmental regulations. Well-written SOPs can also serve as training materials and as 

references for staff, particularly if they are updated regularly (the recommendation is every three 

years). SOPs should be distributed in a manner that ensures that only the most recent versions are 

used and that historical SOP revisions are retained (these are sometimes called “controlledcopies”). SOPs should also be developed to enable individuals to transition into new positions. 

Up-to-date SOPs help train operators new to accepted field operation and data management 

procedures. Deviations and changes from SOPs should be dated, documented, and kept in a 

bound or electronic document routinely accessed by and accessible to all staff.

Overall, SLOCAPCD maintained a nearly complete and readily accessible library of SOPs, but 

complete and updated SOPs were needed in the following areas: NO2 Trace level for TAPI 200u, 

SO2 Trace level for TAPI 100u, Zero Air Generator for TAPI 701 and data review for AirVision. 

Electronic SOPS were available at every monitoring station and desktop and additional 

hardcopies were available at the office, and at monitoring stations when possible. 

References:

40 CFR Part 58, Appendix A, Section 2 (SOPs)

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambient Air Quality 

Monitoring Program, EPA-454/B-13-003, May 2013. (Sections 5 and 17)

Suggested Actions to Address Finding:
Auditor: Dena Vallano

Final Report - August 2016

182

Update and obtain CARB’s approval for SOPs on a regular schedule or as needed to reflect 

current practices, in accordance with 40 CFR 58, Appendix A.

Provide operators with work aids until SOPs are finalized.

Develop a tracking system to indicate that staff have read and understood the SOPs and have had 

an opportunity to discuss them with their supervisor. 
Auditor: Dena Vallano

Final Report - August 2016

183

Finding # SLO4 (Field Operations)

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - Field Operations 

Finding:

SLOCAPCD did not have comprehensive training documentation for field technicians and 

contractors who perform routine monitoring system operations.

Description:

EPA noted that SLOCAPCD operators maintained a very high level of oversight and efficiency, 

had a thorough understanding of their field operations, and expressed a sense of ownership of the 

data. SLOCAPCD has been selected by CARB on several occasions to assist in teaching 

monitoring classes and trainings over the past few years. Field technicians received good handson training with frequent information sharing and issues communication via meetings. However,

training activities and/or demonstration of proficiency were not formally documented in all 

cases. 

An excellent internal document for training purposes has been developed, but a formalized 

process and comprehensive documentation for completed training has not been implemented. 

Formalized training objectives and a tracking system will help to develop proficiency in new 

staff, to give experienced staff the opportunity to keep skills current, to learn about emerging 

technologies and capabilities, and to ensure that all staff are familiar with the current QAPPs and 

SOPs.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring 

Program, EPA–454/B-13-003, May 2013. Section 4.2:

Appropriate training should be available to employees supporting the Ambient Air Quality 

Monitoring Program, commensurate with their duties. Such training may consist of classroom 

lectures, workshops, web-based courses, teleconferences, vendor provided, and on-the-job 

training.

Suggested Actions to Address Finding:

Maintain comprehensive training documentation to ensure that all staff and managers are 

familiar with the updated QAPPs and SOPs related to producing data in the field, proper data 

verification, validation, management and tracking, quality assurance, corrective action, and all 

data systems. Include a list of required and recommended training (including training on updated 

SOPs and QAPPs). Track staff progress to ensure they have access to training and to ensure 

completion.
Auditor: Dena Vallano

Final Report - August 2016

184

Finding # SLO5 (Data Management)

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo - Data Management 

Finding:

The data review and validation process was not complete and/or independent. 

Description:

Ensuring that data validation procedures are independent and have been followed consistently 

and thoroughly is critical to the generation of defensible environmental data and necessary 

for an adequate quality system. 

SLOCAPCD’s original data review and validation process did not include three independent 

levels of review and required site operators to perform the data collection, data review and 

validation roles for their own sites. Additionally, CARB provided QA oversight of the 

SLOCAPCD air monitoring programs and submitted QA data from these sites to AQS.

SLOCAPCD was in the process of implementing a new multi-tier data verification and 

validation review process. At the time of the TSA, half of the monitoring sites were being

reviewed under this new system, however, there was insufficient independent QA and data 

review at the sites for which the revised process had not been implemented. Complete 

implementation of the revised data validation process across all monitoring sites was 

expected to be finalized within a few months of completing the field audit. 

Under the new process, SLOCAPCD is implementing a 22-step validation process for 

operators (Level 1), peer review of data by other air monitoring staff (Level II), and data 

validation package review by an air monitoring manager (Level III). Routine review 

monitoring data is performed in a daily basis, and every site visit. The most thorough Level 1 

data QC review is done as soon as possible after the end of the month.

The Level II review is conducted by air monitoring staff other than the site operator after the 

Level 1 is completed. EPA reviewed an example data validation package submitted for Level 

III review and found that the new multi-level process is, for the most part, thorough and 

defensible. This new process is reflected in an updated SOP and SLOCAPCD addendum to 

CARB QAPP 2.0.2.6 (currently pending CARB approval). 

Although SLOCAPCD had a formalized system for tracking and documenting instrument and 

site maintenance, it lacked a management review process. While the instrument and site 

maintenance was recorded in the electronic data management logbooks and a detailed 

electronic summary spreadsheet was also used internally for tracking purposes, these 

documents did not undergo a formal review process by management and were not included as 

part of the data validation process. EPA also noted that some maintenance was behind 
Auditor: Dena Vallano

Final Report - August 2016

185

schedule in certain instances as marked on the internal electronic summary spreadsheet, but 

this schedule had not undergone any review. 

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality 

Monitoring Program, EPA–454/B-13-003, May 2013. 

Link to Data Verification and Validation resources: http://www.epa.gov/quality1/vandv.html

Suggested Actions to Address Finding:

Formalize the implementation of a data validation procedure that incorporates multiple levels 

of independent review on frequent (i.e. monthly) basis across all monitoring sites. 

Integrate independent QA oversight throughout the data validation process. 

Include the electronic summary spreadsheet that contains all relevant instrument and site 

maintenance information (i.e. repairs, calibrations, etc.) in the data validation package for 

Level III data validation review and communication to managers. 

Ensure all staff involved in validation are familiar with acceptance criteria, applicable flags to 

apply when criteria fail, and have access to the data used in generating the validation report. 

Revise and update data validation SOP and QAPPs to reflect the new multi-level data 

validation process. 
Auditor: Dena Vallano

Final Report - August 2016

186

Recommendation # SLO – Recommendation1

Agency: California Air Resources Board – San Luis Obispo County APCD

Date of Audit: August 26-27, 2015

Program Area: San Luis Obispo – Network Management

Finding:

The NO2 monitor at Morro Bay may be discontinued.

Description:

SLOCAPCD should consider the discontinuation of the NO2 monitor at Morro Bay if network 

priorities shift in the future or resources limit SLOCAPCD’s capacity to effectively operate 

their current network. As noted in the SLOCAPCD 2015 Ambient Air Monitoring Network 

Assessment, only one NO2 monitor is required for the County. The NO2 monitor at Atascadero 

measures the highest NO2 levels in the County, and concentrations rarely reach 50% of the 24-

hour NAAQS. Additionally, the Morro Bay NO2 monitor was originally established to monitor 

emissions from the Morro Bay power plant, located less than a mile upwind. The plant 

permanently closed in February 2014.

References:

Suggested Actions to Address Finding:

Coordinate with CARB to initiate early communication regarding any desired system 

modifications or potential changes to either the CARB- or SLO-operated sites within the 

monitoring network located within SLOCAPCD’s boundaries. Include any modifications to 

SLAM site(s) in the next Annual Monitoring Network Plan.
Final Report - August 2016

187

APPENDIX A: SUMMARY OF FINDINGS

G – Positive Finding 1: A number of important projects were underway to improve the 

PQAO.

G – Positive Finding 2: CARB has been providing increased support to the CARB PQAO 

districts.

G1: Not all districts have been fully attending trainings or identifying and requesting a sufficient 

level of technical support.

G2: Communication within the PQAO was not consistent. 

NM – Positive Finding 1: The CARB PQAO has been working with EPA to create the 

CAMNAT database.

FO- Positive Finding 1: Staff were professional and helpful, and in most instances very 

knowledgeable regarding field operations. 

FO- Positive Finding 2: The AMSS has developed an on-the-job training program for field 

technicians. 

FO- Positive Finding 3: A training database for field staff is under development. 

FO- Positive Finding 4: CARB has been working with several manufacturers as they 

develop new instruments. 

FO- Positive Finding 5: All regulatory PM2.5 BAM instruments within the PQAO were being 

converted to VSCCs.

FO1: Some field forms contained errors that were not properly identified or corrected in the 

initial form validation or subsequent peer reviews.

FO2: Staff were not required to demonstrate proficiency when assigned new tasks. 

FO3: QC check gas is not delivered externally to the instruments at the San Luis Obispo, 

Mojave Poole, and other O3-only sites, as the 400E/T400 IZS O3 monitor delivers quality control 

(QC) check gas to the monitor internally.

FO4 [Finding FO13 from 2011 TSA]: MLD staff were not formally documenting the quality of 

zero air being used in the program.

FO5: The field SOPs did not reflect many changes made to the program over the past several 

years.
Final Report - August 2016

188

FO6: Data loss has occurred at some sites due to CARBLogger malfunctions and lack of site 

operator awareness regarding how data can be recovered.

FO7: Filter data loss has resulted in several of the CARB-operated sites missing the 75% data 

completeness requirement for certain quarters.

FO8: In one instance, a field technician was not aware of the correct schedule for taking makeup samples on filter-based instruments.

FO9: Recertification of sensor probes was not based on the recommended instrument 

manufacturer’s date of recalibration. 

FO10: The San Luis Obispo monitoring station did not have an adequate guard rail to assure 

safety during routine operations.

FO - Recommendation1: The freezer used to store filters at Bakersfield-California was not kept 

secure and did not have a temperature gauge.

DM – Positive Finding 1: CARB installed a new data management system.

DM – Positive Finding 2: CARB was creating a new user guide/SOP for DMS.

DM – Positive Finding 3: CARB was creating an SOP for CARBLogger.

DM – Positive Finding 4: CARB was upgrading DMS so that codes used in DMS match 

those used in AQS.

DM – Positive Finding 5: CARB was creating guidance on minimum value reporting for PM 

instruments.

DM – Positive Finding 6: CARB was working with the PQAO districts to ensure that all 

regulatory PM10 instruments are reporting data in standard temperature and pressure (STP). 

DM – Positive Finding 7: CARB was creating a data visualization tool to aid data review. 

DM1: QA qualifier codes were not entered in instances where critical QC criteria were missed.

DM2: Calibrations were not always coded as such in AQS.

DM3: Field technicians were unable to apply codes in CARBLogger and DMS to missing data, 

resulting in missing non-coded data in AQS.

DM4: Negative values were inconsistently invalidated for PM2.5 monitors.

QA – Positive Finding 1: Since the 2011 TSA, the Quality Management Section has been 

created and, in coordination with other areas of CARB, has implemented significant 

changes within the PQAO.
Final Report - August 2016

189

QA – Positive Finding 2: R&R documents have been created between CARB and each of 

the twenty-one districts that conduct regulatory monitoring.

QA – Positive Finding 3: CARB has created a QA webpage that presents PQAO QA 

information in one, easy to navigate webpage.

QA – Positive Finding 4: CARB held three successful PQAO training modules, and is in 

the process of creating a condensed version of the training to be offered on a repeated 

basis.

QA – Positive Finding 5: CARB was creating guidance as to how districts should review 

data once they are entered into AQS.

QA1: Improvements to the AQDA and CAN processes are necessary to make them more 

efficient and effective, and ensure timely resolution of monitoring issues.

QA2: An AQDA that indicated a need for AQS to be changed was not appropriately closed out, 

and inaccurate AQS codes were entered overriding previously entered, more accurate AQS 

codes.

QA - Recommendation1: Annual performance audits were not being used to identify significant 

practices that needed correction.

QA - Recommendation2: Data validation and reporting responsibilities were unclear in one 

instance when a district utilized a non-CARB laboratory for analysis. This may extend to other 

such situations as well.

SL – Positive Finding 1: CARB was working to restructure roles and responsibilities 

within QMB to improve independence, review, and succession planning within the 

Standards Laboratory.

SL1: The Standards Laboratory QA system did not have written policies and procedures, a QA 

manual or a designated QA Officer.

SL2: Corrective action processes for the Standards Laboratory were not consistent with ISO or 

NIST requirements.

SL3: Calibrations/Certifications conducted by the Standards Laboratory were not reviewed by 

peers/management/QA in sufficient detail to prevent errors.

SL4: Forms to document standard certification were not completely validated or revalidated on a 

routine basis.

SL5: Calibration laboratory staff had not attended routine trainings or conferences that are 

specific to their field of work.
Final Report - August 2016

190

SL6: The main Standards Laboratory database and several data systems had not been updated 

recently and did not meet expectations of performance and/or documentation and compromise 

data defensibility.

SL7: Zero air was not being treated as a standard and the quality of zero air was not formally 

documented by the Standards Laboratory.

SL8: The materials used for plumbing and mixing for SO2 certifications were stainless steel. 

These are not appropriate materials for SO2 certifications.

SL – Recommendation1: Contracting issues affected the ability of the Standards Laboratory to 

certify standards and delayed required primary standard certifications.

PM – Positive Finding 1: CARB was creating a laboratory support section.

PM – Positive Finding 2: CARB’s PM laboratories were found to be well maintained and 

well run.

PM – Positive Finding 3: CARB was creating a system to ensure consistent documentation 

when a filter does not run. 

PM1: Statistical information on temperature and humidity compliance with method 

requirements was not consistently calculated or tracked in accordance with EPA’s specified 

methodology.

PM2: Preservation and custody requirements were not being met and/or tracked for samples that 

had been hand delivered to the MLD laboratory.

PM3: Laboratory blanks for PM2.5 were not processed in batches consistent with regulatory 

requirements.

PM4: PM2.5 MTL filter lots and the associated lot controls could not be properly evaluated and 

were not evaluated at a sufficient frequency.

PM5: The PM10 laboratory SOPs had not been updated in over five years.

PM6: The procedure for processing filters for PM10 speciation led to data being invalidated 

unnecessarily in one observed instance. 

PM7: Not all databases and electronic forms were tracked and validated, compromising data 

defensibility.

PM8: MLD weigh room practices did not protect the integrity of samples.

Pb1: CARB’s Pb laboratory procedures did not meet EPA requirements.
Final Report - August 2016

191

AV/MD - Positive Finding 1: AV/MD AQMD’s monitoring staff were dedicated and 

maintain sites spread out across a significant geographic area.

AV/MD1 (General): The data review and validation process was not complete and/or 

independent. Numerous errors were found with data that had been loaded into AQS.

AV/MD2 (General): The number of staff and managers and the amount of their time allocated to 

the air monitoring program were inadequate for the current network and associated activities.

AV/MD3 (General): Documentation, while generally present, was sometimes incomplete, not 

legally defensible, poorly organized, and not reviewed by a peer or a manager.

AV/MD4 (General): The AV/MD AQMD staff, while part of the PQAO, were generally not 

following CARB SOPs and QA planning documents as stated in their R&R document.

AV/MD5 (General): The AV/MD AQMD training program was not robust enough to ensure 

that new staff are properly trained, current staff are cross trained, and current staff’s knowledge 

is refreshed and up-to-date.

AV/MD6 (General): The AV/MD AQMD did not rely on the CARB PQAO for independent QA 

and support.

AV/MD7 (Network Management): The transition from filter-based PM10 to continuous PM10

was not properly defined.

AV/MD8 (Field Operations): The span/precision standard at the Lancaster site expired on 

8/6/2014.

AV/MD9 (Field Operations): AV/MD AQMD staff were not tracking residence time.

AV/MD10 (Field Operations): AV/MD AQMD staff were not evaluating manifold material 

requirements.

AV/MD11 (Field Operations): Zero air sources were not considered standards and maintained as 

recommended by the manufacturer.

AV/MD12 (Field Operations): PM2.5 filters from the Lancaster site were not being transported at 

a controlled temperature back to the office.

AV/MD13 (Field Operations): The PM2.5 instrument files were not being downloaded from the 

samplers.

AV/MD14 (Field Operations): The plug was missing from the rain cap at the Phelan Station and 

the Teflon O3 inlet tube was resting on the plastic funnel.
Final Report - August 2016

192

AV/MD15 (Data Management): Based on a limited review, EPA found a significant number of 

data errors in AQS.

AV/MD16 (Data Management): AV/MD AQMD’s system did not reliably back up all data.

AV/MD17 (Data Management): Some of the metadata in AQS were missing or incorrect.

AV/MD18 (Data Management): The data record was incomplete for several pollutants as 

negative values are routinely invalidated.

AV/MD19 (PM Laboratory): Temperature and humidity issues in the PM weigh room 

compromised the quality of data collected in 2014 and possibly earlier. Temperature and 

humidity were not tracked, so it was not easy to identify out-of-control situations.

AV/MD20 (PM Laboratory): The quality control activities utilized for PM2.5 filter weighing 

were insufficient and inconsistent with EPA regulation, guidance, and general laboratory 

principles.

AV/MD21 (PM Laboratory): Holding times were not being met for all PM filters.

 

AV/MD22 (PM Laboratory): The data system used for PM weighing was not documented, 

understood by current staff, flexible, or fully validated.

AV/MD23 (PM Laboratory): Storage and custody of PM filters were not consistent with general 

sample handling practice, EPA guidance and regulations.

AV/MD24 (PM Laboratory): Several aspects of general weigh laboratory procedures were not 

consistent with EPA guidance.

NCU – Positive Finding 1: NCUAQMD was implementing a number of good practices in

support of site and instrument maintenance, and data management.

NCU1 (General): The current purpose/objectives of NCUAQMD’s NAAQS monitoring network 

were unclear.

NCU2 (General/Quality Assurance): QA document gaps were identified.

NCU3 (General): Training specifications/requirements for air monitoring personnel were not 

well defined and completion was not tracked.

NCU4 (General): Major monitoring purchases have been problematic for NCUAQMD and 

CARB.

NCU5 (Field Operations): Field documentation gaps and defensibility issues were identified.
Final Report - August 2016

193

NCU6 (Field Operations): Monthly maintenance check sheets, as prescribed by CARB’s SOPs, 

have not been used.

NCU7 (Field Operations): Sampling lines/tubes were not regularly replaced.

NCU8 (Field Operations): Some rooftop equipment was not clean.

NCU9 (Field Operations): The refrigerator containing PM filters that have been collected from 

the field was not locked.

NCU10 (Data Management): Data collected with FEM/FRM instruments were not uploaded to 

AQS as required by CFR.

NCU11 (Data Management): NCUAQMD had not verified that their data processing software 

meets performance guidelines consistent with the QA Handbook, Volume II, Section 14.

NCU12 (Quality Assurance): The level of data quality performance requirements for gaseous 

pollutants were less stringent than those in EPA regulation and guidance.

NCU13 (Quality Assurance): NCUAQMD did not have a procedure for checking/verifying zero 

air.

NCU14 (Quality Assurance): NCUAQMD had not established procedures to initiate corrective 

actions for resolution of internal matters. 

NCU15 (Quality Assurance): A formal agreement describing the roles of NCUAQMD and the 

PM gravimetric contract lab (BAAQMD) in the generation of filter PM data was not in place.

SLO - Positive Finding 1: In general, the SLOCAPCD monitoring program was extremely 

robust and the district staff and managers involved in the program were committed to 

producing defensible data of known quality.

SLO - Positive Finding 2: SLOCAPCD field technicians were knowledgeable and produce 

excellent documentation.

SLO1 (General): SLOCAPCD had not fully adopted a centralized corrective action process to 

resolve internal issues.

SLO2 (Quality Assurance): QAPPs had not been completed or updated for O3, SO2, NO2, CO, 

PM2.5, and PM10 monitoring as required by regulation.

SLO3 (Quality Assurance): SLOCAPCD did not have updated and complete SOPs to reflect 

recent changes in operation.

SLO4 (Field Operations): SLOCAPCD did not have comprehensive training documentation for 

field technicians and contractors who perform routine monitoring system operations.
Final Report - August 2016

194

SLO5 (Data Management): The data review and validation process was not complete and/or 

independent. 

SLO – Recommendation1 (Network Management): The NO2 monitor at Morro Bay may be 

discontinued.
Final Report - August 2016

195

APPENDIX B: CARB ORGANIZATIONAL CHARTS

Organization of CARB’s Divisions and Offices
Final Report 

- August 2016

196
Final Report 

- August 2016

197
Final Report 

- August 2016

198
Final Report 

- August 2016

199
Final Report - August 2016

200

APPENDIX C: MOJAVE DESERT AIR QUALITY MANAGEMENT DISTRICT ORGANIZATIONAL CHARTS

& MONITORING STATIONS MAP
Final Report - August 2016

201

Air Monitoring Department and Information Systems Department.
Final Report - August 2016

202

Mojave Desert Air Basin Monitoring Stations Map. Shows monitors within Easter Kern, Antelope Valley, and Mojave Desert air 

districts.
Final Report - August 2016

203

APPENDIX D: NORTH COAST UNIFIED AIR QUALITY MANAGEMENT DISTRICT ORGANIZATIONAL CHART
Final Report - August 2016

204

APPENDIX E: SAN LUIS OBISPO COUNTY AIR POLLUTION CONTROL DISTRICT ORGANIZATIONAL CHART
Final Report - August 2016

205

APPENDIX F: MATRIX OF 2011 TSA FINDINGS

Finding 

Number Description

Corrective Action 

Plan Status Close Out Status

G1 [Previous Finding M1] CARB needs to complete the process of putting a formal PQAO into place. Approved

Conditionally 

approved

G2 The QMB does not have the structure and staff to manage QA oversight of the PQAO districts. Approved Approved

G3

[Previous Finding M6] While progress has been made on updating the CARB QA Manual with a 

QMP and QAPPs or equivalent documents, the process is behind schedule. Approved

Additional 

documentation 

needed

G4

Local districts within the CARB PQAO do not always have updated quality system documentation 

for all activities. Approved Approved

G5

[Previous Findings QM1 and M3] QA Authority and interactions between QMB and the other 

branches should be expanded and formalized. The corrective action system should be developed 

to include actions taken, in addition to reports issued by the QA auditors and the calibration 

laboratory. Approved Not submitted

G6 Coordination between CARB and districts and EPA should be improved. Approved Approved

NM1

Not all agencies within the CARB PQAO have approved network plans since this became a 

requirement in 2006. The current approach to network plans does not provide for a determination 

of network adequacy on a statewide basis. Approved Approved

NM2 The network assessment does not meet all CFR requirements. Approved Approved

NM3

There are PM10 monitors listed in local conditions (LC; parameter code 85101), but not Standard 

Temperature and Pressure (STP; parameter code 81102 in AQS), as required for FRM/FEM 

instruments. Approved Not submitted

FO1

[Related Previous Findings GB3, SJV3, & NS2] Documentation at the CARB field sites is inadequate 

and not reviewed by management. Approved Approved

FO2 Management oversight of site operators needs strengthening. Approved Approved

FO3 CARB field operators have not been trained on new SOPs. Approved Approved

FO4 Residence time calculations were not available at any CARB site visited. Approved

Additional 

documentation 

needed

FO5 Delay in sending PM2.5 samples has resulted in loss of data. Approved Approved

FO6 PM make-up samples are not being taken in accordance with EPA guidance. Approved Approved
Final Report - August 2016

206

FO7

PM10 QC checks are not consistently recorded. There is no document in which field operators are 

directed to record this information. Approved Approved

FO8 CARB field staff do not check data after sending information to CARB offices. Approved Approved

FO9 The Yuba City site has several significant siting issues that need to be resolved. Approved Approved

FO10

Records indicate that calibrations of gaseous pollutant instruments are not consistently done 

according to a schedule. Approved Approved

FO11

[Previous Finding AQSB7] The number of NO2 titration points taken during calibration does not 

meet regulatory requirements. Approved

Submitted; 

undergoing review

FO12 Multi-point calibrations of PM2.5 instruments are not done routinely. Approved

Submitted; 

undergoing review

FO13

[Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air being used in 

the program.

Revision 

requested – to be 

addressed 

through 2015 

TSA finding FO4

Not submitted – to 

be addressed 

through 2015 TSA 

finding FO4

FO14 Span and precision gases used in the field are not being calibrated routinely. Approved Approved

FO15

Instruments removed from the field are not always efficiently tracked and returned to the repair 

laboratory facility for diagnosis, repair, and reuse. Loss of data has occurred due to unavailability 

of spare instruments. Approved Approved

FO16 Positive Finding - CARB is working to improve communication with field staff. NA

FO17

Positive Finding - CARB maintains a well-equipped stockroom of spare parts, maintains a large 

equipment purchase order history, and develops thorough equipment testing procedures that are 

regularly updated. NA

DM1

The data validation and review/verification performed by AQSB, NLB, and AQAS, are not formally 

published in a control-copied SOP. Approved Approved

DM2

[Previous Finding M7] Data submitted by local districts within the CARB PQAO are not validated 

using consistent procedures. (See Findings SJV9, IMP10, and MEN11) Approved Approved

DM3

[Previous Finding DM5] AQAS does not ensure that local district data are validated prior to upload 

to AQS. Approved Approved

DM4 A few instances of erroneous continuous data were identified in AQS for CARB sites. Approved Not submitted

DM5 Erroneous continuous data were identified in AQS for non-CARB sites within the CARB PQAO. Approved Approved

DM6

[Previous Finding DM6] There are numerous deficiencies in the data certification process for the 

CARB PQAO, including: Not all NAAQS-compliant data within the CARB PQAO are routinely 

certified; Data certified by CARB for local districts are not typically reviewed or validated; Data are 

routinely certified by agencies within the State of California, but responsibility has not been 

formally delegated to any local agencies within the State of California. Approved Approved
Final Report - August 2016

207

DM7

Data, including those used for design value sites, have been changed after they are certified and 

subsequently not recertified. Approved Approved

DM8 Some local districts within the CARB PQAO are listed as PQAOs in AQS. Approved Approved

DM9

There were several instances of CARB altering data collected by local districts without 

communicating with the district. Approved Approved

QA1

The QA Audit group has made an effort to improve its documentation process; however, several 

inconsistencies were noted. Approved Approved

QA2

The audit trailer evaluated was using one expired gas cylinder along with others that had not 

been certified annually as required for the EPA National Performance Audit Program (NPAP). Approved Approved

QA3

The QA Section is not tracking monitors to ensure that 25% are being audited per calendar 

quarter. Approved Approved

QA4 The connection to the inlet on the audit trailer could pull in outdoor air. Approved Approved

QA5 Auditors do not review all applicable siting information in AQS prior to an audit. Approved Approved

QA6

[Previous Findings M4 & OPA2] Quality assurance for special projects is not developed in a process 

consistent with EPA quality system requirements. Approved Approved

QA7

Mass flow elements (MFEs) are used to establish calibration points outside of their calibrated 

range. Approved Approved

PM1

Communication of post-weigh information and transmission of documentation to local districts 

should be improved. Approved Approved

PM2

The PM laboratory does not have a formal corrective action process for addressing issues with PM 

filter collection. Approved Approved

PM3 Documentation of activities in the PM10 and PM2.5 laboratories should be improved. Approved Approved

PM4 PM10 trip blanks are not being used to assess potential bias from filter transport and handling. Approved Approved

TL1 The canister cleaning SOP does not reflect the current cleaning procedure. Approved Approved

TL2

An SOP is not documented for the batch certification of cleaned canisters. The canister cleaning 

SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method. Approved Approved

TL3

The batch certification of cleaned canisters described by staff for methods MLD 058 and MLD 066 

differs from existing VOC guidance. Approved Approved

TL4

[Previous Finding 19]. Pre-cleaning concentrations are not recorded in a logbook to allow for the 

selection of the most highly contaminated canister for batch certification. Approved Approved

TL5

Canisters are not routinely leak tested as prescribed in guidance. Instead, canisters are vacuum 

leak tested only when gross leaks are suspected. Approved Approved

TL6

[Previous Finding 21]. A retention time policy for re-cleaning and blanking canisters once they 

have been certified clean has not been established. Approved Approved

TL7 The CARB SOP states that old canisters are reconditioned, but this is inconsistently practiced. Approved Approved
Final Report - August 2016

208

TL8

CARB has not established a holding time for cartridges once samples have been collected for 

extraction or analysis. Approved Approved

TL9

The laboratory does not assign expiration dates to new sampling cartridges and allows cartridges 

to be used beyond the 90 days prescribed by the method. Approved Approved

TL10 CARB's procedure for analyzing DNPH lot blanks differs from the SOP. Approved Approved

TL11 No criterion is provided in the CARB SOP for passing DNPH lot cartridge blanks. Approved Approved

TL12

Gloves are not worn as a contamination protection measure when handling cartridges. A 

nitrogen-purged glove bag is not used for extractions. Approved Approved

TL13

[Previous Finding OL3] Staff stated that field blanks are not being analyzed at a frequency of 10% 

of field samples, as specified in Method TO-11, nor is there an SOP describing the procedure for 

the submission of field blanks. Approved Approved

TL14 CARB does not analyze trip blanks when needed. Approved Approved

TL15

Method TO-11 states that samples should be re-analyzed when results are 10% above the 

criterion, but the analyst was not aware of this criterion. Approved Approved

TL16

Working standards are tracked and used for six months, while the CARB SOP states that standards 

should be retained for four months under refrigeration. Approved Approved

TL17

Site name and sampling dates are recorded on a piece of tape loosely stuck to sample cartridges; 

the tape occasionally falls off, making it difficult to identify samples. Approved Approved

TL18

Positive Finding - Hexavalent chromium data undergo peer review, supervisory review, review by 

the Branch Chief, and a final review before going to AQS. NA

TL19 [Previous Finding OL5] There is no secondary review of logbooks. Approved Approved

TL20

CARB does not analyze audit samples or through-the-probe audit samples as suggested in Sec. 9.7 

of the CARB SOP. Approved Approved

TL21

Appendix V in the CARB SOP lists the standards that were used in 2003 and has not been updated 

to reflect the standards currently being used. Approved Approved

IMP1 The ICAPCD ambient air monitoring program is not operating under an approved QAPP. Approved Not submitted

IMP2 ICAPCD has not established an appropriate quality system for ambient air monitoring. Approved Not submitted

IMP3

Assessment of PM10 or PM2.5 sampling frequency throughout the ICAPCD network has not been 

performed as required. Approved Approved

IMP4 Neighborhood scale may be inappropriate for PM10 at the Westmorland site. Approved Approved

IMP5

One-point flow rate verifications for PM10 and PM2.5 are not performed by ICAPCD as required and 

are not well documented. Approved Approved

IMP6 Residence time for gaseous monitors operated by ICAPCD is not established. Approved Approved
Final Report - August 2016

209

IMP7

ICAPCD is internally post weighing high-volume PM10 filters without proper PM lab or quality 

control measures. Approved Approved

IMP8 Documentation of ICAPCD air monitoring activities is not complete. Approved Not submitted

IMP9 There are potential siting issues at the Calexico Ethel site. Approved

Additional 

documentation 

needed

IMP10 ICAPCD is not adequately reviewing and editing data. Approved

Additional 

documentation 

needed

MEN1 MeCAPCD staff was not familiar with the CARB QMP or instrument SOPs. Approved Approved

MEN2

MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2006 but is erroneously 

listed as its own PQAO in AQS Approved Approved

MEN3

Positive Finding - MeCAQMD stations were well-maintained. Staff and the manager were 

professional and helpful, and very knowledgeable about the county and potential pollution 

sources. The station operator was proactive about troubleshooting instrument issues. NA

MEN4

One-point QC checks (flow verifications) for PM10 and PM2.5 are not consistently performed by 

MeCAPCD site operators. Approved

Additional 

documentation 

needed

MEN5

MeCAPCD logbook entries are not consistently made and are not always in the most defensible 

form. Handwritten notes are occasionally illegible due to water (rain) marks. Approved

Additional 

documentation 

needed

MEN6 Residence time calculations were not available at the Ukiah Gobbi site. Approved Approved

MEN7 Trees at the Ukiah Gobbi and Library sites should be evaluated against siting requirements. Approved Approved

MEN8

The internal shelter thermostat is not operating correctly at the Ukiah Gobbi site and the issue has 

not been addressed to provide defensible data. Approved Approved

MEN9 MeCAPCD has no system for tracking and controlling station and instrument logbooks. Approved

Additional 

documentation 

needed

MEN10 MeCAQMD should have formalized training requirements for all air monitoring staff. Approved Approved

MEN11

MeCAQMD does not provide CARB AQAS with datasets that have been fully quality assured and 

ready for upload to AQS. Approved Approved

MEN12 MeCAQMD does not use a formal corrective action system. Approved Approved

SJV1 Positive Finding - In general, the SJVAPCD monitoring program is robust and the agency staff and 

managers involved in the program are committed to producing defensible data of known quality. NA

SJV2 SJVAPCD does not have updated quality system documentation for all activities. Approved Approved
Final Report - August 2016

210

SJV3 SJVAPCD has experienced significant data losses at required monitoring sites, including sites 

critical for demonstrating compliance with the NAAQS. Approved Approved

SJV4 SJVAPCD has initiated network modifications for several required sites without seeking EPA 

approval required by 40 CFR 58.14. Approved Approved

SJV5 The residence time of flow between the inlet and each instrument was not posted at every 

SJVAPCD site. Approved Approved

SJV6 Some SJVAPCD site logbooks lacked specific information about the date or type of maintenance 

performed on an instrument. Approved Approved

SJV7 There is no documentation of management review of station logbooks and other site activities for 

SJVAPCD operated sites. Approved Approved

SJV8

SJVAPCD site operators do not have a quick visual way to identify changes in instrument 

performance or QC checks that would indicate instrument issues, nor do they have the ability to 

remotely check on data or site operations. Approved Approved

SJV9 SJVAPCD experiences data loss due to instrument malfunction. Approved Approved

SJV10 It is unclear whether SJVAPCD is using appropriate criteria to invalidate or flag PM10 data. Approved Not submitted

SJV11 SJVAPCD experiences significant resource inefficiencies for staff and management as the current 

data management system relies solely on manual inputs. Approved Not submitted

SJV12 SVJAPCD does not have a formal corrective action process in place. Approved Approved