S)~41~

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

REGION IX

~ PRO~ 75 Hawthorne Street

San Francisco, CA 94105.3901



## AUG 312016

Mr. Richard Corey

Executive Officer
California Air Resources Board Transportation and Toxic Division

Post Office Box 2815 Sacramento, California 95812

Dear Mr. Corey:

I am writing to transmit the findings from the U.S. Environmental Protection Agency’s (EPA’s)

2015 Technical Systems Audit (TSA) of the California Air Resources Board’s (CARB’s)

ambient air monitoring program. This audit was performed in accordance with 40 CFR Part 58 Appendix A, Section 2.5, which requires EPA to conduct TSAs of primary quality assurance organizations (PQAO5) every three years.

After EPA’s previous TSA in 2011, CARB made substantial improvements to strengthen data quality within CARB and the CARB PQAO. The addition of new staff, restructuring to enhance independent oversight, and other changes within CARB have improved the quality and defensibility of air quality data collected by the entire CARB PQAO. EPA commends you and

your staff for these efforts.

EPA reviewed CARB’s ambient air monitoring program, including network management, field operations, laboratory operations, quality assurance, and data management procedures. In

addition to visiting laboratories, offices, and air monitoring stations run by CARB staff, EPA

also reviewed the air monitoring programs at four local districts within CARB’s PQAO: Antelope Valley Air Quality Management District (AVAQMD), Mojave Desert Air Quality Management District (MDAQMD), North Coast Unified Air Quality Management District (NCUAQMD), and San Luis Obispo County Air Pollution Control District (SLOCAPCD).

Generally, we found that the CARB PQAO’s air monitoring program is robust and meets EPA requirements. As with all TSAs, we also identified areas that can be strengthened and have accordingly listed these findings and suggested actions to address them in the report. Your

agency has been very responsive to the audit findings and has already begun working on corrective actions. The formal corrective action response is due to EPA within 60 days of receipt
of the final TSA report (enclosed).

I want to thank you and your staff for your full cooperation and participation during the audit.

The results of this audit significantly contribute to EPA’s overall understanding of ambient monitoring in the State of California.

Pnrned on Reycled PaperPlease contact me if you have questions, or your staff may contact Anita Lee at (415) 972-3958

or Gwen Yoshimura at (415) 947-4134.

Sincerely,

~ Elizab~J.

Actin: • irector, Air Division

Enclosure

cc: Brad Poiriez, AVAQMD and MDAQMD

Brian Wilson, NCUAQMD Larry Allen, SLOCAPCD

cc (via email): Michael Benjamin, CARB

Mike Miguel, CARB

Patrick Rainey, CARB

Michael Werst, CARB

Kenneth Stroud, CARB Ranjit Bhullar, CARB

Ravi Ramalingam, CARB Sylvia Vanderspek, CARB
Chris Collins, MDAQMD Wendy Caruso, NCUAQMD

Karl Tupper, SLOCAPCD

Jaime Contreras, SLOCAPCDSystem Audit of the Ambient Monitoring Program:

California Air Resources Board April

- August, 201 5

Conducted by:

Air Quality Analysis Office Quality Assurance Office

Air Division Environmental Management

Services Division

US EPA Region 9

75 Hawthorne Street San Francisco, California 94105

Final Report: August 2016Final Report

– August 2016



## Contents

GLOSSARY OF ACRONYMS

................................ ................................ ........................... i

EXECUTIVE SUMMARY

................................ ................................ ................................ 1

TSA ACTIVITIES

................................ ................................ ................................ .............. 5

O VERVIEW OF THE STATE AIR MONITORING PROGRAM

................................ . 10

OVERVIEW OF THE LOCAL DISTRICT AIR MONITORING PROGRAMS:

ANTELOPE VALLEY & MOJAVE DESERT AQMD

................................ .............. 29

NORTH COAST UNIFIED A QMD

................................ ................................ ............ 34

SAN LUIS OBISPO COUTY APCD

................................ ................................ ........... 37

FINDINGS

................................ ................................ ................................ ........................ 41

APPENDIX A: SUMMARY OF FINDINGS

................................ ................................ 187

APPENDIX B: CARB ORGANIZATIONAL CHARTS

................................ .............. 195

APPENDIX C: MOJAVE DESERT A QMD ORGANIZATIONAL CHARTS &

MONITORING STATIONS MAP

................................ ................................ ................. 200

APPENDIX D: NORTH COAST UNIFIED AQMD ORGANIZATIONAL CHART

. 203

APPENDIX E: SAN LUIS OBISPO COUNTY APCD ORGANIZATIONAL CHART

................................ ................................ ................................ ................................ ......... 204

APPENDIX F: MATRIX OF 2011 TSA FINDINGS

................................ .................... 205Final Report

- August 2016

i GLOSSARY OF ACRONYMS

ACAPCD

............................... Amador County Air Pollution Control District

ADAM

................................ ... Aerometric Data and Management

AMNS

................................ .... Air Monitoring Northern Section

AMSS

................................ ..... Air Monitoring Southern Section

APCD

................................ ..... Air Pollution Control District AQAS

................................ ..... Air Quality Analysis Section AQDA

................................ .... Air Quality Data Ac tion AQDAS

................................ .. Air Quality Data Acquisition System AQDB

................................ .... Air Quality Data Branch AQMD

................................ ... Air Quality Management District AQMIS

................................ ... Air Quality and Meteorological Information System AQPSD

................................ .. Air Quality Planning and Science Division AQS....................................

... . Air Quality System AQSB..................................... Air Quality Surveillance Branch AVAQMD.............................. Antelope Valley Air Quality Management District AV/MD AQMD

..................... Antelope Valley and Mojave Desert Air Quality Management Distric ts BAAQMD.............................. Bay Area Air Quality Management District

BAM...................................... Beta Attenuation Mass Monitor (continuous PM 2.5 ) BCAQMD..............................Butte County Air Quality Management District

CAA

....................................... Clean Air Act

CaCAPCD.............................. Calaveras County Air Pollution Control District

CAMNAT

.............................. California Monitoring Network Assessment Tool

CAN

................................ ....... Corrective Action Notification

CARB........................

............. California Air Resources Board

CBSA..................................... Core

- based Statistical Area

CFR........................................ Code of Federal Regulations

CO.......................................... Carbon Monoxide

CoC.....

................................... Chain of Custody

CoCAPCD.............................

. Colusa County Air Pollution Control District CPAQAB

............................... Consumer Products and Air Quality Assessment Branch

CSA

................................ ........ Combined Statistical Area

DMS

................................ ....... Data Management System DQO..

..................................... Data Quality Objective EDCAQMD........

................... El Dorado County Air Quality Management District

EKAPCD...............................

. Eastern Kern Air Pollution Control District

EPA.......................................

. Envir onmental Protection Agency

FEM....................................... Federal Equivalent Method FRAQMD.............................. Feather River Air Quality Management District

FRM....................................... Federal Reference Method

FTP

................................ ......... File Transfer ProtocolFinal Report

- August 2016

ii GBUAPCD............................ Great Basin Unified Air Pollution Control District

GCAPCD............................... Glenn County Air Quality Management District

ICAPCD................................. Imperial County APCD

ILS

................................ .......... Ino rganic Laboratory Section

ISO

................................ ......... International Organization for Standardization

IZS.........................................

. Internal Zero/Span

LC.......................................... Local Conditions

LCAPCD

................................ Lassen County Air Pollution Control District

L CAQMD

.............................. Lake County Air Quality Management District

LIMS...................................... Laboratory Information Management System

LSS

................................ ......... Laboratory Support Section MaCAQMD

........................... Mariposa County Air Quality Management District

MBUAPCD............................ Monterey Bay Unified Air Pollution Control District

MCAPCD.............................

.. Modoc County Air Pollution Control District MDAQMD............................. Mojave Desert Air Quality Management District MeCAQMD........................... Mendoc ino County AQMD

MFE....................................... Mass Flow Element

MLD....................................... Monitoring and Laboratory Division

MSA....................................... Metropolitan Statistical Area NAAQS.......................

........... National Ambient Air Quality Standard

NCore..................................... National Core multi

- pollutant monitoring stations NCUAQMD........................... North Coast Unified Air Quality Management District

NELAC....................

.............. National Environmental Laboratory Accreditation Conference

NIST....................................... National Institute for Standards and Technology

NLB.......................................

. Northern Laboratory Branch

NO

................................ .......... Nitric Oxide

NO 2

................................ ........ Nitrogen Dioxide

NO
x ................................ ........ Nitrogen Oxides

NPAP..................................... National Performance Audit Program

NPEP

................................ ...... National Performance Evaluation Program NSAQMD.............................. Northern Sierra Air Quality Management District NSCAQMD.....

...................... . Northern Sonoma County Air Pollution Control District

O 3

........................................... Ozone

OIS

................................ ......... Office of Information Services

OLS

................................ ........ Organics Laboratory Section

OMB...................................... Office of Manageme nt and Budget

OSS........................................ Operations and Data Support Section

PAMS..................................... Photochemical Assessment Monitoring Stations

Pb........................................... Lead

PCAPCD..................

.............. Placer County Air Pollution Control District

PM.......................................... Particulate matter

PM 2.5

...................................... Particulate matter 2.5 microns or less in aerodynamic diameter

PM 10

..................... .................. Particulate matter 10 microns or less in aerodynamic diameterFinal Report

- August 2016

iii PMOD

................................ .... Planning, Monitoring and Outreach Division

POC........................................ Parameter Occurrence Code PQAO..................................... Primary Quality Assurance Organization

PSD

................................ ........ Prevention of Significant Deterioration QA.......................................... Quality Assurance QAPP..................................... Quality Assurance Project Plan QAS.......................................

. Quality A ssurance Section QC.......................................... Quality Control QCM

................................ ...... Quality Control Manual QMB...................................... Quality Management Branch QMP....................................... Quality Management Plan QMS

................................ ....... Quality Mana gement Section

R&R

................................ ....... Roles and Responsibilities

RH.......................................... Relative Humidity

SAS

................................ ........ Special Analysis Section

SASS

................................ ...... Spiral Aerosol Speciation Sampler

SBCAPCD............................. Santa Barbara County Air Pollution Control District SCAQMD.............................. South Coast Air Quality Management District SDCAPCD............................ San Diego County Air Pollution Control District ShCAQMD............................ Shasta County Air Quality Management District

SiCAPCD............................... Siskiyou County Air Pollution Control District

SJVAPCD.............................. San Joaquin Valley APCD

SLAMS.................................. State or Local Air Monitoring Station

SLOCAPCD.....................

...... San Luis Obispo County Air Pollution Control District SMAQMD.............................

. Sacramento Metro Air Quality Management District

SO 2

......................................... Sulfur Dioxide

SOP........................................ Standa rd Operating Procedure

SPM.......................................

. Special Purpose Monitor SQL

................................ ........ Structured Query Language

STP......................................... Standard Temperature and Pressure

TeCAPCD.............................

. Tehama County Air Po llution Control District

TSA........................................ Technical System Audit

TSP......................................... Total Suspended Particulate

TTP

................................ ........ Through The Probe TuCAPCD..............................Tuolumne County Air Pollution Co ntrol District

VCAPCD............................... Ventura County Air Pollution District

VOC....................................... Volatile Organic Compound

VSCC

................................ ..... Very Sharp Cut Cyclones

WINS

................................ ..... Well Impactor Ninety

- Six

Y

- SAQMD.............................Y olo

- Solano Air Quality Management District

ZPS

................................ ......... Zero/Precision/SpanFinal Report

- August 2016

1 EXECUTIVE SUMMARY

This document is a report of the findings made by the U.S. Environmental Protection Agency, Region 9 (EPA) while conducting a Technical Systems Audit (TSA) on the air mon itoring program of the California Air Resources Board (CARB). A TSA is an on

- site review and inspection of a state or local ambient air monitoring program to assess its compliance with established regulations governing the collection, analysis, validation, and reporting of ambient air quality data. This TSA meets the requirements for EPA audits of CARB’s monitoring organization as described in 40 CFR Part 58, Appendix A, Section 2.5.

EPA notes that CARB has made substantial changes since the previous TSA t o strengthen its quality system. These include:

 An entirely new Quality Management Section was established within the Quality Management Branch (QMB) that includes positions for Primary Quality Assurance Organization (PQAO)

liaisons. Each district within t he CARB PQAO is assigned to a specific liaison to improve coordination and dialogue between the districts and CARB. CARB has been allocating budget, time and personnel to allow the liaisons to visit their districts in person at least once a year.

 CARB hel d three PQAO trainings, offering them in northern and southern California, with participation from the CARB PQAO districts as well as the other California PQAOs.

 Roles and Responsibilities (R&R) documents have been created between CARB and each of the

twe nty

- one CARB PQAO districts that conduct regulatory monitoring. These documents

describe districts ’ and CARB ’s responsibilities in order for the PQAO to function successfully.

 A new data management system (DMS) has been installed, allowing for improved da ta flow from the field, easier data visualization and editing.

 A finalized data validation standard operating procedure (SOP) outlines the steps involved in data validation, including what the different validation levels are, who does them, and what they

entail.

 A Quality Assurance (QA) webpage was created that houses information about the CARB QMB, the PQAO structure and contacts, quality management documents

, such as a quality management document repository, QA activities with links to Corrective Action Notifications (CANs) and Air Quality Data Actions (AQDAs), and services and resources

, including information on the CARB standards laboratory.

 CARB has been working with the districts and EPA to create a California Monitoring Network Assessment Tool (CAMN AT), to assist in development of annual monitoring network plans and

network assessments.

CARB has instituted these changes , many of which are noted as positive findings in this report, while managing site relocations, particulate matter (PM) filter rejec tion challenges, changes in instrumentation (including increased use of continuous PM 10 and PM 2.5 instruments), as well as carrying on the daily air monitoring system operations. While some of these changes have been fully implemented and are operating smo othly, the majority are relatively new

. As a result, a significant portion of the 2015 TSA findings emphasize implementation and checks to ensure that new systems are operating as desired.

EPA also notes that CARB implemented certain changes between the time the findings contained in this report were written and the issuance of this final report. These changes include: restructuring the Standards Laboratory; sending regulatory Pb

- TSP filters to an external lab; creating a quality assurance/quality control ( QA/QC ) oversight position for the Northern Laboratory; and issuing aFinal Report

- August 2016

2 technical bulletin addressing minimum values reporting. Other actions, such as development of a user guide/SOP for each DMS and CARBLogger, and offering a CAN session during the next PQ AO training, are under development

. As the TSA report is created to document what was observed during the audit itself, findings that call for improvements related to these items are still reflected in this report.

We look forward to final resolutions of t hese findings, and commend CARB for the work already underway.

CARB, an organization under the umbrella of the California Environmental Protection Agency, is the

gover nmental agency delegated under s tate law with the authority and responsibility for colle cting ambient air quality data as directed by the Clean Air Act (CAA) of 1977 and CAA Amendments of 1990. CARB and local air pollution control districts (hereafter referred to as “local districts” or “districts”) operate ambient monitoring stations through out the State of California. CARB is designated as the PQAO for the entire state with the exception of the ambient air monitoring programs of the Bay Area Air Quality Management District (BAAQMD), the South Coast Air Quality Management District (SCAQMD), a nd the San Diego County Air Pollution Control District (SDCAQMD). Many of the smallest local districts do not have active air monitoring programs and rely solely on CARB for the operation of monitoring stations within their jurisdictions.

The TSA was cond ucted by EPA staff from April to September, 2015. The audit evaluates all air monitoring activities since the previous EPA TSA, which was conducted during the summer of 2011. The audit team interviewed management and staff on specific aspects of the ambien t air monitoring program including network design, field operations, laboratory operations, data handling, quality assurance and quality control procedures. The audit team also inspected some of the CARB

- operated monitoring sites. The site inspections cons isted of an interview with the site operator when possible, review of station and instrument logbooks, and evaluation of the station siting with respect to EPA requirements for probe siting (40 CFR 58, Appendix E). The laboratory inspection included a revi ew of the standards and calibrations labs, the PM program for mass determinations, and laboratory analysis for

Pb.

Since CARB oversees the quality of data collected by local districts within its PQAO, EPA also reviewed field operations, data management a nd quality assurance activities at local districts. For this TSA, it was not possible to evaluate all of the 21 local districts within the PQAO that collect ambient air quality data; the audit team reviewed operations at four local districts: Antelope Vall ey Air Quality Management District (AVAQMD), Mojave Desert Air Quality Management District (MDAQMD), North Coast Unified Air Quality Management District (NCUAQMD), and San Luis Obispo County Air Pollution Control District (SLOCAPCD). The local districts in cluded in the PQAO have their own organizational structures and these vary depending on the size of the local district program. The four agencies visited are geographically diverse, located in northern, central, and southern California.

The TSA is part o f the oversight system by which EPA ensures that data collected by state, local, and tribal agencies meet certain minimum data quality objectives. Other assessments, such as network reviews and performance evaluations, are also used to collect information on the quality of ambient air monitoring data. These assessments enable agencies to identify and correct those program elements that may be adversely affecting the quality of ambient air data. The results of the TSA are summarized here and fully described in this report, along with recommended actions to address the findings. The specific actions to be taken by CARB will be determined through negotiations between EPA and CARB and will be documented in a corrective action plan prepared by CARB.Final Report

- August 2016

3 EPA would l ike to thank all the staff and management of CARB and the local districts for their support and cooperation during the audit.

A. Program Strengths:

 CARB has put into place multiple systems and infrastructure to promote better communication

between CARB an d the districts within the PQAO. The most notable of these is the staffing of a Quality Management Section, which has been critical in implementation of the PQAO.

 CARB’s increased interactions with the districts, impressive quality assurance website, Quali ty Management Plan (QMP), PQAO trainings, and Roles and Responsibilities documents are resulting in more consistency across the CARB PQAO and improved understanding of requirements and processes.

 CARB has extensive experience and expertise in ambient air m onitoring.

 Local districts within the PQAO are committed to addressing air quality concerns in their areas and see monitoring as the means by which to assess air quality. Some local districts have created a strong culture of continually working to improve their current practices and tools, and take the initiative to identify issues and improvements early.

 CARB and the local districts that participated in this TSA are dedicated to collecting credible and defensible air quality data.

B. Major Program Finding s:

 Districts within the PQAO need additional training and technical support. [Findings G1,

AV / MD5, NCU 3 ]

 Template forms used in the field and in the laboratories are not being properly validated or

reviewed

. During EPA’s brief inspection of forms as part o f the TSA, EPA found errors in forms used in the field and by the standards laboratory [Findings FO 1 , SL4, PM7]

 CARB and district QAPP/ SOPs are out of date or not being followed. [Findings FO 5 , PM5,

AV / MD4, NCU2 , SLO2, SLO3 ]

 The quality of zero air is not being documented by CARB or the PQAO districts

. [Findings FO 4

, SL7, AVMD1 1 , NCU1 3 ]

 Air Quality System ( AQS ) coding is not used consistently . [Findings DM1, DM2, DM3]

 The process of i mplementation and follow

- up for resolving AQDAs and CANs and internal

corr ective actions is incomplete . [Findings QA1, QA2, AV/MD6, NCU1 4 , SLO1]

 The CARB Standards Laboratory QA System has a number of deficien cies . [Findings SL1, SL3]

 The Standards Lab oratory corrective action process is not consistent with International Organiz ation for Standardization (ISO) or National Institute of Standards and Technology (NIST) requirements. [Finding SL2]

 Calibration laboratory staff have not had sufficient training. [Finding SL5]

 The CARB PM Laboratory is not tracking required temperature a nd humidity statistical

information in accordance with EPA’s specified methodology . [Finding PM1]

 One district lab oratory did not track temperature and humidity. [Finding AV/MD1 9 ]

 Several districts do not have complete and/or independent data review and va lidation procedures. [Findings AV/MD1, NCU1 5

, SLO 5 ]

 The CARB Pb Laboratory was not meeting EPA requirements. [Finding Pb 1]

 The number of staff and managers and the amount of their time allocated for air monitoring were inadequate for the current ne twork an d activities at one district. [Finding AV/MD1]

 One district was using less stringent d ata quality performance levels for gaseous pollutants than those in EPA regulation and guidance . [Finding NCU1 2 ]Final Report

- August 2016

4 The individual findings are reported in the topic sect ions of this document and are summarized in Appendix A. The major findings are those that are over

- arching, impact regulatory decisions, or data quality. It has been EPA Region 9’s experience that all monitoring programs have areas that could be improved. Unless explicitly stated, EPA’s findings are not cause for data invalidation. Overall, the CARB PQAO has a robust air monitoring program that produces reliable, defensible, valid data for regulatory use.Final Report

- August 2016

5 TSA ACTIVITIES

In the spring and summer of 20 15, EPA Region 9 conducted a TSA of the ambient monitoring program operated by and overseen by CARB. This report reflects the status of the program at the time of the

audit. EPA staff interviewed management and staff in three branches of CARB Monitoring an d Laboratory Division (MLD) and one branch of the Air Quality Planning and Science Division (AQPSD). The TSA covered the areas of Air Monitoring Network Management, Field Operations, Laboratory Operations, Data and Data Management, and Quality Assurance. I n addition, the EPA staff reviewed these same areas as implemented by four local districts: Antelope Valley & Mojave Desert Air Quality Management District (AQMD), North Coast Unified AQMD, and San Luis Obispo County Air Pollution Control District (APCD). CARB staff shadowed EPA during portions of this TSA.

CARB managers and staff were very accommodating to the EPA audit team, making themselves and their staff available for many interviews, procedural reviews and monitoring site visits. Branch Chiefs

inter viewed were:

Ken Stroud

– Chief, Air Quality Surveillance Branch (AQSB), MLD Michael Miguel

– Chief, QMB, MLD

Michael Werst

– Chief, Northern Laboratory Branch (NLB), MLD Ravi Ramalingam

– Chief, Consumer Products and Air Quality Assessment Branch (CPAQAB ),

AQPSD Many other individual section managers and staff were interviewed in Sacramento and in the field. We appreciate the fact that CARB gave the EPA audit team access to all key personnel involved in the collection and quality assurance of ambient air quality data.

The EPA regional staff members conducting the TSA were Gwen Yoshimura, Dena Vallano, and Elfego Felix of EPA Region 9’s Air Quality Analysis Office, and Mathew Plate and Joseph Eidelberg of the EPA Region 9 Quality Assurance Office. In addit ion to the EPA Audit Team, Meredith Kurpius and Eugenia McNaughton, managers of EPA Region 9's Air Quality Analysis Office and Quality Assurance Office, respectively, attended the opening and closing meeting representing EPA management.

The TSA began wi th a general meeting with CARB managers and staff on February 24, 2015 at the

MLD office in Sacramento, CA, with field activities and interviews commencing in April and continuing through August, 2015. The TSA covered the following program areas:

 General / Quality Management.

- Program organization.

- Facilities.

- Independent QA and quality control (QC).

- Planning documents (including QMP, Quality Assurance Project Plans (QAPPs), & SOPs).

- General documentation policies.

- Training.

- Corrective action.

- Quality improv ement.

- External performance audits.

 Network Management / Field Operations.

- Network design.

- Changes to the network since the last audit.Final Report

- August 2016

6 - Proposed changes to the network.

- Field support.

 Laboratory Operations: Standards and Calibrations, PM, and Pb

- Routine op erations.

- Quality control.

- Laboratory preventive maintenance.

- Laboratory record keeping.

- Laboratory data acquisition and handling.

- Specific pollutants: PM 10 , PM 2.5 , and Pb.

 Data and Data Management.

- Data handling.

- Software documentation.

- Data validation an d correction.

- Data processing.

- Internal reporting.

- External reporting As part of the TSA, EPA tracked supporting documentation for data points/sets from calendar years

2013 and 2014.

This report is divided the following sections:

 Executive Summary

– desc ribes the purpose of the TSA and summarizes the major findings.

 TSA Activities

– outlines the timing of this TSA and the programs that were covered.

 Overview of Air Monitoring Program

– describes the Air Monitoring Programs of CARB and the

local districts that were part of this TSA.

 Findings

– collection of findings and suggested actions to address findings that include details associated with findings.

 Appendix A

– list of findings.

 Appendix B

– CARB organizational charts.

 Appendix C, D, and E

– District o rganizational charts.

 Appendix F

– matrix of findings from the 2011 TSA.

The Findings section includes positive findings and best practice recommendations along with those that require corrective action. The findings are grouped by program area. Suggested actions to address findings are provided for each one that requires corrective action to give some indication of EPA ’s expectations as to how they may be addressed. If the air agency has other approaches or alternatives to address the concerns identified, EPA will consider them, provided the corrective action adequately addresses the finding. Ap pendix A is a summarized list of findings. Within each program area, findings,

are ordered according to priority

. Higher priority findings are those that: (1) are o ver

- arching issues, (2) have a greater impact on regulatory decisions, and/or (3) have a greater effect on data quality. EPA has also included best practice recommendations for some program areas. Regardless of priority, all findings, excluding positive fi ndings and recommendations

, should be addressed in a corrective action

report.

Appendix F contains a matrix summarizing the findings from the previous TSA and their status. The improvements noted in this TSA directly reflect changes made as a part of the corrective action process. EPA commends CARB on the work done to address these numerous, wide

- reaching findings. We will continue to work with CARB to close out the few remaining open findings from the 2011 TSA, while also moving forward to address the ne w findings from this report.Final Report

- August 2016

7 Network Management

EPA interviewed Gayle Sweigert and Craig Anderson (manager and air pollution specialist within the Air Quality Analysis Section, respectively) , as well as Ravi Ramalingam and Reginald Smith (manager, Operati on and Data Support Section) , and reviewed the CARB Annual Monitoring Network Plan and

Five

- year Network Assessment as part of this TSA. CARB submitted a 2015 Five

- Year Network

Assessment and Network Plan as required by 40 CFR 58.10. Both documents address a portion of the monitoring network of California; they present information from the smaller agencies that do not produce their own reports as well as CARB sites. As part of the Annual Network Plan process

, EPA

reviewed all Annual Network Plans associated with the CARB PQAO and provided detailed comments as part of that process.

Field Operations

The CARB site technicians interviewed were Tony Simoni, Glen Jennings, Carl Camp, John Latham, Phil Wagner, Adolfo Garcia, Steven Theantanoo , David Cardiel, Deb bie Henson , Ralph Robles, and Kathy Edwards. In general, the site technicians demonstrated a thorough knowledge of the monitoring equipment for which they were responsible.

EPA visited the following nine CARB

- operated monitoring stations:

 Yuba City in Sut ter County (Air Quality System (AQS) ID 06

- 101

- 0003),

 Jackson

- Clinton Road in Amador County (AQS ID 06

- 005

- 0002),

 San Andreas

- Gold Strike in Calaveras County (AQS ID 06

- 009

- 0001),

 Sonora in Tuolumne County (AQS ID 06

- 109

- 0005),

 South Higuera in San Luis Ob ispo County (AQS ID 06

- 079

- 2006),

 Paso Robles in San Luis Obispo County (AQS ID 06

- 079

- 0005),

 Bakersfield

- California in Kern County (AQS ID 06

- 029

- 0014),

 Oildale in Kern County (AQS ID 06

- 029

- 0232), and

 Mojave

- Poole in Los Angeles County (AQS ID 06

- 037

- 903 3).

Evaluation at these sites included inspection of the inlet manifolds, examination of station and instrument log books, and an evaluation as to whether appropriate QC checks and QA audits were being performed.

Data Management

This section covers dat a management for criteria pollutants (O 3 , CO, NO 2 , SO 2 , Pb, PM 2.5 , and PM 10 ). The following managers/staff who have data management responsibilities were interviewed:

 Kenneth Stroud, Reginald Smith

- continuous data for CARB

- operated sites. (Reginald Smit h ’s

staff upload the data generated by Ken neth Stroud ’s branch.)

 Gayle Sweigert

- CARB AQS data entry for district

- operated sites, data certification. (Dwight

Oda , a staff member within Gayle Sweigert’s group, uploads the data.)

 Michael Werst, Brenda Salda na, Samantha Scola

– PM and Pb filter data for CARB

- and district

operated sites using the CARB laboratory. (Samantha Scola uploads the filter data generated by

the lab oratory .)Final Report

- August 2016

8 Quality Assurance/Quality Control

EPA interviewed the QMB Chief, Quality Ma nagement Section Manager, Quality Assurance Section (QAS) Manager, AQSB Chief, Air Monitoring North Section Manager, Air Monitoring South Section Manager, Operations and Data Support Section Manager, and staff within each section. Managers and staff inter viewed included, but was not limited, to:

 Mike Miguel, QMB Chief

 Patrick Rainey, Quality Management Section Manager

 Ranjit Bhullar, QAS Manager

 Kenneth Stroud, AQSB Chief

 Joe Guerrero, Air Monitoring North Section Manager

 Fernando Amador, Air Monitoring So uth Section Manager

 Reginald Smith, Operation and Data Support Section Manager

CARB ’s quality management system me t the basic EPA requirements. CARB ha d a QMP that was approved in July 20 13. CARB’s core program conformed to or exceeded the method quality objectives systematically developed by EPA for criteria pollutants. EPA and CARB perform national performance and technical evaluations of the monitoring network.

Laboratory Operations

Standards and Calibrations

EPA interviewed staff (Ranjit Bhullar, Je rry Freeman, and Robert Russell) and reviewed procedures for the CARB Standards Laboratory. EPA focused on review of the flow calibration program and data tracking.

Particulate Matter Laboratory

EPA visited two gravimetric PM laboratories and interviewe d the following staff as part of the audit:

Michael Werst

– Chief, Northern Laboratory Branch

Brenda Saldana

– Inorganic Laboratory Section Manager Ranjit Ahuja

– PM 10 Analyst Nial Maloney

– PM 2.5 Analyst

Alicia Adams

– PM 2.5 Analyst Jeff Wright

– Laborat ory Support Section Manager

Samantha Scola

– Air Pollution Specialist

Dan Tackett

– Staff Air Pollution Specialist While the laboratory facilities are primarily used to process and weigh PM 10 and PM 2.5 filters, back

- up
PM 10 and PM 2.5 balances are maintaine d at both facilities so that filters may be weighed in either laboratory if issues arise at the primary laboratory location. Both of the PM laboratories were well

maintained, neat, and well

- organized. Generally, the PM 10 and PM 2.5 laboratory measurements w ere performed with a high level of technical expertise.

Pb Laboratory

On May 4 and 5, 2015, EPA interviewed Betsy Ronsse, the designated analyst who performed sample

preparat ion and analysis of lead

- TSP. Ms. Ronsse was responsible for performing acid dig estion of TSPFinal Report

- August 2016

9 filters using a heated ultrasonic water bath with subsequent analysis of digestates using a graphite

furnace atomic absorption instrument. Ms. Ronsse was responsible for ensuring compliance with quality control requirements and preparing repo rts.

EPA reviewed the practices for conformance to the Laboratory SOP and al l EPA approved TSP

- Pb

methods. EPA reviewed each step of the method to determine if it was substantively the same practice described by an already approved EPA method. The CARB me thod was not considered substantively the same as any one approved EPA method and some steps in the CARB method were not analogous to any step in approved EPA methods.

As part of its 2015 Annual Monitoring Network Plan, submitted July 2015, CARB requeste d EPA approval to close the one Pb

- TSP monitor in its network. EPA approved this closure on November 13,

2015. As CARB no longer analyzes Pb

- TSP for regulatory purposes, this laboratory operation is no longer active.

Review of Agencies within the CARB PQA O

Since CARB oversees the QA of data collected by local districts 1 within the PQAO, EPA also reviewed field operations, data management and quality assurance activities at local districts. As it was not possible to evaluate all of the 21 local districts w ithin the PQAO that collect ambient air quality data, the audit team reviewed operations at four local districts: AVAQMD, MDAQMD, NCUAQMD, and

SLOCAPCD.

Each of the local districts included in the CARB PQAO has its unique organizational structure. All fou r districts were small in terms of staff but the geographic ar ea and number of monitors varied for each district: NCUAQMD operate d only a few sites over a large geographic area; SLOCAPCD operate d more sites over a smaller geographic area; and AVAQMD and MD AQMD, which share d staff and facilities,

operate d sites and ran their own PM weigh lab. The four agencies are geographically diverse, located in northern, central, and southern California.

1 According to the California State and Local Air Monitoring Network Plan ( 2015), prepared by CARB, 21 local Districts operate air monitoring stations in the CARB PQAO.Final Report

- August 2016

10 OVERVIEW OF THE STATE AIR MONITORING PROGRAM

State and Local Moni toring Agencies within the State of California

CARB, an organization under the umbrella of the California Environmental Protection Agency, is the

gover nmental agency delegated under s tate law with the authority and responsibility for collecting

ambient ai r quality data as directed by the CAA of 1977 and CAA Amendments of 1990. Specifically, CAA Section 110(a)(2)(B)(i) directs the State to “ provide for establishment and operation of appropriate devices, methods, systems, and procedures necessary to ...(i) moni tor, compile, and analyze data on ambient air quality...”

At the time of the audit, there were 35 local air pollution control districts in the state of California (Table 1). Three of these local air districts, BAAQMD, SCAQMD, and SDCAPCD

, are PQAOs. Twenty

- one of the remaining air districts and CARB, comprising the CARB PQAO, collect ambient air monitoring data for National Ambient Air Quality Standard (NAAQS) compliance

.

A PQAO is a monitoring organization or a coordinated aggregation of such organizatio ns that is responsible for a set of stations that monitors the same pollutant and for which data quality assessments can logically be pooled because they have similar quality systems in place. Specifically, 40 CFR Part 5 8 Appendix A Section 3 requires that each ambient air monitoring PQAO conform to certain quality management practices. These include:

 Having a documented quality system that meets EPA requirements for QMPs and QAPPs.

 Having a quality management function that is independent of air monitoring operations.

 Developing or adopting data quality objectives ( DQOs ) , or equivalent systematic planning procedures, for all monitoring programs.

 Participating in National Performance Evaluation Programs, which consist of performance audits used to independent ly determine program adequacy, national monitoring network performance, and national consistency.

 Undergoing TSAs by EPA at a frequency of every three years or less as needed.

 Using certified reference materials to standardize monitoring equipment.

EPA vie ws these quality management system components as indispensable to maintain a credible monitoring program. Insufficient quality management and control has been cited as rationale to support legal challenges to NAAQS designation decisions.

CARB oversees th e quality assurance of data collected by local districts within the PQAO. Although

both CARB and local districts operate ambient monitoring stations throughout the state, responsibility for ambient air monitoring ultimately rests with CARB. Further, it is the responsibility of CARB to provide QA oversight to ensure that data quality within the PQAO meet s code of federal regulations ( CFR ) requirements and conform s to quality standards approved in the QAPP.Final Report

- August 2016

11 Table 1. List of Air Pollution Control Agencies in California

Air Pollution Control District Ambient Air Monitoring by

District for NAAQS Compliance* PQAO

Amador County (A C )APCD N CARB Antelope Valley (A V )AQMD Y CARB Bay Area (BA)AQMD Y BAAQMD Butte County (BC)AQMD N CARB Calaveras County (CaC)APCD N CARB

CARB Y CARB Colusa County (CoC)APCD N CARB El Dorado County (EDC)AQMD N CARB Feather River (FR)AQMD N CARB Glenn County (GC)APCD N CARB Great Basin Unified (GBU)APCD Y CARB Imperial County (IC)APCD Y CARB Eastern Kern (EK)APCD Y CARB

Lake Cou nty (LC)AQMD Y CARB Lassen County (LC)APCD N CARB Mariposa County (MaC)AQMD N CARB Mendocino County (MeC)AQMD Y CARB Modoc County (MC)APCD N CARB Mojave Desert (MD)AQMD Y CARB Monterey Bay Unified (MBU)APCD Y CARB North Coast Unified (NCU)AQMD Y CAR B Northern Sierra (NS)AQMD Y CARB Northern Sonoma County (NSC)APCD Y CARB

Placer County (PC)APCD Y CARB Sacramento Metro (SM)AQMD Y CARB San Diego County (SDC)APCD Y SDCAPCD San Joaquin Valley (SJV)APCD Y CARB San Luis Obispo County (SLOC)APCD Y CAR B

Santa Barbara County (SBC)APCD Y CARB Shasta County (ShC)AQMD Y CARB Siskiyou County (SiC)APCD Y CARB South Coast (SC)AQMD Y SCAQMD Tehama County (TeC)APCD Y CARB Tuolumne County (Tu C )APCD N CARB Ventura County (VC)APCD Y CARB

Yolo

- Solano (Y

- S)AQ MD Y CARB

*In some cases CARB and local districts share ambient air monitoring responsibilities for areas and sites within a local

district ’s jurisdiction.

The ambient air monitoring program in the State of California encompasses many air quality assessm ent activities, including collecting and analyzing data for federal criteria pollutants and many other air pollutants of concern; collecting data from special studies as directed by the CARB Board; assessingFinal Report

- August 2016

12 monitoring methods that are used by the s tate an d local districts in compliance with federal and state regulations; conducting annual performance audits of all monitoring equipment within the PQAO; implementing a calibration and certification of measurement standards program; and conducting training

in the operation of ambient air monitoring instruments.

Organization of the CARB Ambient Air Monitoring Program

At the time of the audit, r esponsibility for overseeing the ambient air monitoring program for CARB

reside d within the following branches:

Bran ch Division Air Quality Surveillance Branch (AQSB) Monitoring and Laboratory Division (MLD) Northern Laboratory Branch (NLB) Monitoring and Laboratory Division (MLD) Quality Management Branch (QMB) Monitoring and Laboratory Division (MLD)

Consumer Prod ucts & Air Quality

Assessment Branch ( CPAQAB ) Air Quality Planning and Science Division (AQPSD)

QA responsibility for CARB is covered primarily by the QMB. Michael Miguel is the QMB Chief. The QMB oversees the development of quality management document s with contributions from other branches, conducts on

- site audits, reviews precision and accuracy data, and initiates corrective action requests ( i.e. , AQDAs), among other duties. The AQSB, NLB, and CPAQAB contribute to QA activities primarily through data validation activities. QC responsibility for field activities is handled primarily by the AQSB, which includes field operations, calibrations, and repair, precision and accuracy data review/submittal, instrument acceptance testing, determination of CARB f ederal/ s tate ambient monitoring methods, development of SOPs, interpretation of CFRs pertaining to monitoring criteria/methods, and a variety of other monitoring functions. QC responsibilities for laboratory activities is handled primarily by NLB, which in cludes laboratory operations, calibrations, laboratory QC, data review/submittal, development of SOPs, interpretation of CFRs pertaining to laboratory criteria/methods, and a variety of other laboratory functions.

The management of QA/QC for local distri cts within the CARB PQAO is consistent with the EPA requirements for the five common factors of a PQAO, as applicable, and defined in the R&R document developed b etween CARB and each district. However, details and responsibilities are specific to each

loca l district based on their current structure and resources.

The CARB PQAO has a number of mechanisms to achieve common QA oversight

. CARB conducts

audits of all sites within California thus ensuring common QA for instruments. CARB conducts or participates in TSAs for all districts within the PQAO. CARB maintains an on

- line repository of the approved quality management documents (QMP, QAPP, SOPs) adopted and us ed by districts throughout the PQAO for the operation of their ambient air monitoring network. C AR B validates all continuous and laboratory data generated by C ARB sites and laboratory operations (collected from

CARB and non

- CARB sites), and upload s and certifi es data generated by districts for which CARB has AQS data authority. Local districts for whic h CARB has AQS entry and certification responsibilities are expected to provide a letter to CARB stating that data ha ve been validated according to approved procedures, and concurrence with certification of the data.

In most cases the local district is responsible for daily QC aspects of the air monitoring program ( e.g. , zero/precision/span checks, calibration, and regular on

- site review and maintenance), validation andFinal Report

- August 2016

13 certification of data according to CARB and/or EPA approved procedures (except as not ed above), and the review and updating of quality management documents to ensure they reflect current practices and requirements.

Network Management

At the time of the TSA, r esponsibi lity for network management lay within the CPAQAB. Ravi Ramalingam is the CPAQAB Chief. Network management responsibilities include network assessment and network plan preparation, analyses and decision

- making for CARB system modifications, and review of requests by local districts within the PQAO for system modifications. Network management

for non

- CARB sites has traditionally been managed by the local district

.

The s tate network consists of monitoring stations operated by CARB and the local districts. The network covers 15 air basins. The four PQAOs in the state operate monitoring networks that provide data from

all the air basins. Three local district PQAOs operate their own monitoring networks that provide data for the following air basins: San Francisco Bay, San Diego County, South Coast, and Salton Sea 2 . The CARB PQAO operates multiple monitoring networks that cover the following air basins: Great Basin, Lake County, Lake Tahoe, Mojave Desert, Mountain Counties, North Central Coast, North Coast, Northeast Plateau, Sacramento Valley, Salton Sea, San Joaquin Valley, and South Central Coast. In some instances, several local districts operate the monitoring networks with in a n air basin. Furthermore, the boundaries of metropolitan and micropolitan statistical areas (MSAs), and core

- based statistical areas (CBSAs) 3 and combi ned statistical areas (CSAs), which are established by the US Census Bureau, may also overlap air basins and local monitoring districts. EPA uses the population statistics of

MSAs /CBSAs to determine the number of the minimum state/local air monitoring stat ion ( SLAMS ) monitoring requirements for some criteria pollutants. Other minimum SLAMS monitoring requirements are based on source emissions, or a combination of emissions/sources and population.

Table 2 summarizes the number of criteria pollutant monitori ng sites operated in the CARB PQAO at

the time of the TSA

.

2 South Coast AQMD collects ambient air monitoring data in Coachella Valley, which is part of the Salton Sea Air Basin.

3 The definition of a Core

- Based Statistic al Area can be found at http://www. census.gov: Metropolitan and micropolitan statistical areas (metro and micro areas) are geographic entities defined by the Office of Management and Budget (OMB) for use by Federal statistical agencies in collecting, tabul ating, and publishing Federal statistics. The term "Core Based Statistical Area" (CBSA) is a collective term for both metro and micro areas. A metro area contains a core urban area of 50,000 or more population, and a micro area contains an urban core of at least 10,000 (but less than 50,000) population. Each metro or micro area consists of one or more counties and includes the counties containing the core urban area, as well as any adjacent counties that have a high degree of social and economic integration (as measured by commuting to work) with the urban core.Final Report

- August 2016

14 Table 2: Summary of Criteria Pollutant Monitors i n the CARB PQAO 4

Operating Agency O 3 CO NO 2 SO 2 PM 2.5 5 PM 10 Pb 6

CARB 34 7 7 14 8 2 21 22 1

AV APCD 1 1 1 0 1 1 0

BCAPCD 0 0 0 0 1 0 0

GBUAPCD 2 0 0 0 2 12 0

ICAPCD 2 1 1 0 2 4 0

EKAPCD 0 0 0 0 1 1 0
LCAQMD 1 0 0 0 1 3 9 0
MeCAQMD 1 0 0 0 2 1 0
MDAQMD 6 2 3 2 1 5 0

MBUAPCD 6 1 1 0 6 2 0
NCUAQMD 2 2 2 2 2 1 0
NSAQMD 1 0 0 0 5 0 0

NSCAPCD 1 0 0 0 0 3 0

PCAPCD 4 10 0 0 0 4 2 0
SMAQMD 6 4 5 0 5 5 1

SJVAPCD 13 5 10 0 16 12 0

SLOCAPCD 6 0 3 1 3 4 0

SBCAPCD 10 11 4 12 9 13 7 14 2 5 15 0
ShCAQMD 4 0 0 0 2 3 0

SiCAPCD 1 0 0 0 1 1 0

TeCAPCD 1 0 0 0 1 1 0

VCAPCD 5 0 2 0 5 2 0

Y

- SAQMD 2 0 0 0 2 3 0

TOTALS 109 27 51 14 86 91 2

Source: CARB/AQPSD/AQDB

4 Primary source for monitoring sites was the QA Monitoring Site database. Sites reported as closed on the QA database were not

counted in the totals

. Several gaseous sites listed in CAMNAT but not i n the QA database were also included.

5 Includes all federal reference method (FRM) and federal equivalent method (FEM) instruments and also non

- FEM beta attenuation method (BAM) instruments.

6 Pb is measured by total suspended particulate (TSP) at Fresno NCore, and Pb

- PM10 at Sacramento Del Paso Manor NCore.

7 Includes Arvin

– Di Giorgio (060295002) and Shafter (060296001).

8 Includes Shafter (06029601)

9 The PM 10 monitors in Lake County report in local conditions (LC)

. 10 Includes newly established Lincoln site (AQS ID 06

- 061

- 2001).

11 Includes 6 long

- term prevention of significant deterioration (PSD) monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1021, 06

- 083

- 1013, 06

- 083

- 1025, 06

- 083

- 1018, 06

- 083

- 1014, 06

- 083

- 4003).

12 Includes 3 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1008, and 06

- 083

- 4003).

13 Includes 8 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1021, 06

- 083

- 1018, 06

- 083

- 1013, 06

- 083

- 2004, 06

- 083

- 1014, 06

- 083

- 0011, 06

- 083

- 1008, and 06

- 083

- 4003).

14 Includes 4 long

- term PSD monitors that are overseen by Santa Barbara County APCD but ope rated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1013, 06

- 083

- 4003, and 06

- 083

- 1020).

15 Includes 2 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025 and 06

- 083

- 4003).Final Report

- August 2016

15 In 2006, EPA added a requirement for Annual Monitoring Network Plans and Five

- year Network Assessments (40 CFR 58.10). At the time when the first network plan was due (July 2007), numerous local agencies within the CARB PQAO expressed an interest in sub mitting their own plan. On this basis, an agreement was reached between EPA, CARB, and local districts wherein agencies within the CARB PQAO that wanted to submit their own plan could follow the process and submit a network plan directly to EPA. CARB submi tted a network plan to cover those agencies that did not choose to submit a plan (Table 3). The F ive

- year N etwork A ssessment that was due in 2010 and 2015 followed a similar process. The network plan per 40 CFR 58.14 (a)

must include recommendations from t he network assessment.

Table 3. List of Agencies in the ARB PQAO Drafting Annual Network Plans in California.

Air districts included in ARB plan Air districts in ARB PQAO drafting their own plan

Amador County APCD Great Basin Unified APCD *

Antelope Valley AQMD Monterey Bay Unified APCD

Butte County AQMD North Coast Unified AQMD

Calaveras County APCD Sacramento Metropolitan AQMD

Colusa County APCD San Joaquin Valley APCD

Eastern Kern APCD San Luis Obispo County APCD

El Dorado County AQMD Santa Barbara County APCD

Feather River AQMD

Glenn County APCD

Imperial County APCD

Lake County AQMD

Lassen County APCD

Mariposa County APCD

Mendocino County AQMD

Modoc County AP CD

Mojave Desert AQMD

Northern Sierra AQMD

Northern Sonoma County APCD

Placer County APCD

Shasta County AQMD

Siskiyou County APCD

Tehama County APCD

Tuolumne County APCD

Ventura County APCD

Yolo

- Sol ano AQMD*

Source: 2015 Annual Network Plan for 25 Air Districts in California

. *Yolo

- Solano is included in the Bay Area network assessment and Great Basin is included in the CARB network assessment. Otherwise,

all other districts in the CARB Annual Moni toring Network Plan are included in CARB’s 2015 Five

- year Network Assessment.Final Report

- August 2016

16 Requirements for developing an adequate network are not based on agency jurisdiction, but rather on CBSAs or PQAOs. A collaborative approach between CARB, EPA, and local distric ts is needed to evaluate whether the requirements for an adequate network continue to be met. To assist in the evaluation of network adequacy, EPA funded development of the CAMNAT database that provides monitoring information for the entire s tate . This dat abase was launched in the first quarter of 2015 and detailed site reports were incorporated in the 2015 C ARB Annual Monitoring Network Plan.

Requests for changes to the network may occur outside the network plan process by the

submission of a letter to EPA. In recent years, CARB and EPA have discussed proposed site changes via conference calls. Once a decision is made, CARB or the District sends a formal request to EPA for approval. When a local district wishes to modify its network, e.g. , shutting

down or relocating an existing site or establishing a new one, it consult s with C ARB. C ARB works with the district and often arranges a conference call with EPA. Based on the discussion

with C ARB and EPA, the local district will draft a letter requesting (1) si te termination

, or (2)

site relocation.

Field Operations

Network operations at CARB were primarily performed by the MLD AQSB. Ken Stroud is the Chief of AQSB. AQSB duties include the operation of CARB monitoring sites, monitoring support for CARB speci al studies, and general air monitoring support, such as repair and calibration facilities. AQSB also assists local districts with instrument trouble

- shooting and repair as resources allow. This section of the TSA report addresses AQSB's general operations, the calibration program, and field operations of the AQSB at CARB

- operated criteria pollutant monitoring sites.

Providing training and performing instrument certifications are the responsibilities of the Operations and Data Support Section (OSS) within AQSB. The OSS also provides independent review and approval of ARB field SOPs. Other responsibilities include instrument repair and technical support. While support ( e.g. , training, field procedures, and other technical support) is

available to the all loc al districts in California, the AQSB does not actively manage the local districts’ field monitoring quality systems.

Laboratory Operations

Analytical services at CARB were provided by MLD’s NLB. Michael Werst is Chief of NLB.

NLB is divided into four s ections: Inorganic Laboratory Section (ILS), Laboratory Support Section (LSS), Organics Laboratory Section (OLS), and Special Analysis Section (SAS). LSS provides laboratory support to the other three laboratories that includes the Laboratory

Information M anagement System (LIMS) Administrator, Laboratory QA / QC Officer, Laboratory Health and Safety Officer, hazardous waste management, and sample control. ILS and OLS analyze samples to support federal and state ambient air quality monitoring and toxic air

con taminant programs, including pesticide monitoring. The ambient air quality programs consist of criteria pollutants such as PM, including PM 2.5 , PM 10 , PM coarse , and Pb

- TSP

. ILS and OLS also provide analytical services for EPA’s PM 2.5 Speciation Trends Netwo rk (ions, metals,Final Report

- August 2016

17 elemental/organic carbon, and wood smoke tracers) and California’s PM 10 Ions P rograms. The toxic air contaminant programs include volatile organic compounds, carbonyl compounds, metals, hexavalent chromium, and various pesticides by reque st from California’s Department of Pesticide Regulation. SAS provides analytical services for California’s consumer products and composition wood product programs.

Data Management

Air quality data measured by continuous analyzers in the CARB ambient air quality network are captured by a data acquisition system. CARB’s data acquisition system is composed of two major components : 1) a PC based system utilizing a custom based Linux software package

referred to as CARBLogger ; and 2) DMS which is an SQL databa se developed by Sonoma Technology Inc.

CARBLogger continuously queries station analyzers via the instruments comm unication ports, and processes one

- minute concentration and meta

- data files (or hourly data for instruments that only provide hourly records). CARBLogger transmits air quality data to DMS every hour via a file transfer protocol ( FTP ) server operated by the C ARB Office of Information Services (OIS). DMS then ingests one

- minute air quality data, aggregates hourly averages, performs automated QC ch ecks and transmits hourly ambient data back to the FTP server, where data are available

for real

- time data applications such as the CARB Air Quality and Meteorological Information System ( AQMIS ) and the national AIRNow System, established by EPA, National Oceanic and Atmospheric Administration, National Park Service, tribal, state, and local agencies to provide the public easy access to national air qualit y information

.

DMS is the primary management tool for data collection, validation, and reporting of da ta
obtained at CARB

- operated stations. All raw data, QC records and chain of custody (CoC) documentation are retained in DMS for at least three years. Once data are reviewed by the data review process, they are uploaded to the EPA ’s AQS. Once submitted to AQS, the data are downloaded to the CARB database Air Data Management System (ADAM). ADAM is the official state database for ambient air quality data. Hard copy files (QC forms, calibration and audit reports, control charts, data reporting memos etc.) are stored locally at CARB headquarters and provide a supplemental record for the data validation process.

At the time of the TSA, a ir monitoring responsibilities were divided among five organizational

units in two different Divisions of CARB. The State of Ca lifornia has 25 separate air pollution control agencies through which ambient monitoring data enters the AQS database. Responsibility for managing the state’s ambient monitoring data required by the Clean Air Act (CAA) was divided among the following group s (descriptions of each group’s role follows the list):

1. Continuous data from CARB

- operated field monitoring stations

– MLD/AQSB, Ken Stroud, AQSB Chief.

2. CARB laboratory analytical data for both CARB and non

- CARB Sites

– MLD/NLB, Michael Werst, NLB Chief.

3. Q A performance audit program data and Standards Laboratory

– MLD/QAS, Ranjit Bhullar, QMB/QAS Manager.Final Report

- August 2016

18 4. Local district site data: operated by local district but AQS

- uploaded by CARB

– AQPSD

, Gayle Sweigert.

5. Local district site data: operated and AQS

- uploaded by local district

– various local

districts

. 6. Local di strict site d ata: Operated by local district with laboratory analyses by a different laboratory that is not CARB

– data upload is the responsibility of the District that performs the analyses. This is defined for each District in the Roles and Responsib i lities documents and QA Bulletins.

1. Continuous Data from CARB

- operated Field Monitoring Stations

Continuous data collected at CARB

- operated field monitoring stations includes data from all

continuou s air quality analyzers (O 3 , NO 2 , CO, SO 2 , CO 2 , CH 4 , N 2 O, and non

- filter

- based PM) and meteorology data. The CARB

- operated ambient monitoring stations are managed by Ken Stroud, AQSB Chief. There are two regional Supervisors, Joe Guerrero (Air Monitoring N orth

Section [ AMNS ] ), and Fernando Amador (Air Monitoring South Section [ AMSS ] ). Air quality data measured by the continuous analyzers at the field stations operated and maintained by CARB are stored in data loggers and station computers. Each station is p olled through broadband

internet connections and the data are transmitted directly to the central computer system in Sacramento. The computer system consists of a server located within the State of California

Federated Data Center and a backup server locat ed in a separate leased facility. The data are

collected in the Data Management System (DMS) , which was developed by Sonoma Technology , Inc. The DMS is the primary management tool for data collection, validation, and reporting at CARB

- operated stations. Da ta are retained in DMS for multiple years

, however, data submittals to AQS occur on a monthly basis. Once submitted to AQS, the data are replicated to
the ADAM database

, the official state database for ambient air quality data.

An updated SOP for data va lidation was finalized in March 2016 . The first review (first

- level validation) of the data is performed by the station operators. Station operators review minute data and take corrective actions as necessary. This review include s : max/min ; outliers ; repe ating

values ; automatically flagged values ; diurnal and seasonal trends ; Q C data and instrument

maintenance sheets ; review/edit/flag/comment data ; and document notes on the monthly maintenance data sheet/station logbook/DMS editor notes. Station operators also review QC data daily if possible, perform comparisons of “ b uddy site s ” and collocated data, review data for irregularities or issues, and inform the second level reviewers if follow up actions, such as calibrations, are needed. The station operator co mpiles a monthly data submittal package for each site consisting of copies of each relevant calibration control chart, instrument “monthly quality maintenance check sheets,” that month’s station logbook pages, the data capture report

, the station operator’ s name
, and what data have been reviewed in DMS. The package is reviewed by the station operator for completeness, and then submitted to the assigned second

- level data

reviewer.

The second

- level review is done by an Air Pollution Specialist and/or an Air Resources Engineer. The review consists of reviewing the min/max and exceedances, data flags, monthly

maintenance check sheets, QC charts, and perform parameter comparisons and “ buddy site ” comparisons. If QC issues may affect the data, the second

- level re viewer edits in DMS and anyFinal Report

- August 2016

19 corrective action is documented in the station logbook and monthly maintenance check sheet.

The second

- level reviewers are also responsible for reviewing the monthly calibration control chart webpage to confirm that the data edi ts were incorporated into DMS. Second

- level data reviewers must initial and date each page in the monthly data submittal package to signify review, and sign and lock hourly data in DMS. Significant issues should be noted on the cover page of the data packa ge submitted to the third

- level review.

The third

- level review is done by the appropriate Section Manager (Northern or Southern Section) who ensures that data are complete, the instruments are meeting operational criteria, and

the stations have been maint ained properly . The Section Managers also consider any significant issues that have been identified by previous

- level review

, perform “ buddy site ” comparisons and compare collocated data.

At this point, a final data validation summary report is sent to th e AQSB Chief with an initialed cover memo. The report includes all control charts, percent data capture, monthly data summaries, station logbook copies, and the cover memo

, which identifies any significant issues for each site. The AQSB Chief reviews the s ummary report and memo and perform s comparisons of “ b uddy site s ” and collocated data. He then initials the cover memos, gives the cover memo to the designated staff in OSS for submittal to EPA’s AQS database, and provides copies of the cover memo to statio n operators and second level

- data reviewers.

In summary, data validation for continuous pollutants goes through the following steps:

1. Review by station operator.

2. Review by Air Pollution Specialist and/or Air Resources Engineer.

3. Review by Section Manager.

4. Review by AQSB Chief.

5. Upload to AQS by OSS designated staff.

Overall, CARB submits all required data to the EPA AQS database, including concentrations for all criteria pollutants, and supporting precision and accuracy information. CARB certifies these

dat a annually as required by regulation (40 CFR 58.15).

2. CARB Laboratory Analytical Methods for both CARB and Non

- CARB Sites

Overview of Laboratory Data Management

Data management describes the basic flow of analytical data from generation, review (verif ication and validation), and reporting. Laboratory staff and management have responsibility for review of the data management system. The laboratory utilizes a LIMS database to carry out data management activities.

LIMS

LIMS facilitates the recording, verification and validation, transmittal, reduction, analysis, management, storage, retrieval, and reporting of analytical data generated by the laboratory. LIMS is maintained by the LIMS administrator. The administrator creates and/or modifiesFinal Report

- August 2016

20 approved l aboratory staff access to LIMS; creates and modifies methods, data templates and transfers, and data reports; and has access to modify locked data in LIMS. All sample and analysis information must be entered into LIMS or recorded in bound or electronic not ebooks. Changes to any data in LIMS must be made by authorized individuals and approved by

management.

All users must be authorized by management and receive a password to access LIMS. Different privileges are given to authorized users depending on ne ed. LIMS users are the laboratory staff who have approved access to the programs and data; they may log in, transfer, modify data, and/or run reports in order to validate data generated. Access may include read

- only, data entry and preliminary data modific ation, data reporting, and client distribution.

LIMS can be accessed to generate laboratory analytical work lists, summaries, and data reports, where applicable. Work lists are used by the analyst to schedule and prioritize sample analyses (e.g. sample ho ld times, inventory, etc.). Summary reports give laboratory staff an overall picture of samples logged into LIMS, and enable them to review raw and quality control data that have

been transferred into LIMS.

Initial Data Assessment

Samples are analyzed an d the instrument QC results are reviewed by the analyst to decide if sample analysis is valid prior to transfer into LIMS. W hen QC criteria are not met

, c orrective action should be taken as needed, such as re

- analysis, dilution, re

- integration, etc. Any sa mple result that has been invalidated must be reported as "invalid” or “not analyzed,” and the reason documented. All results reported as "not detected" must be associated with a reference value,

such as level of quantitation, estimated quantitation limit, or minimum reporting value.

Laboratory staff will contact site operators, or other appropriate staff, directly when issues arise that require clarification to validate a sample at log

- in, when a sample is invalidated, or when a

make

- up sample is recommen ded. This notification is performed as soon as possible, and the issue is documented on the CoC or sample report form, according to established laboratory procedures. If invalidated samples occur repeatedly and are deemed by management to be

indicative of a systemic issue, management will utilize the CAN process to initiate a formal corrective action process in order to inform all responsible and impacted parties; document the issue and resolution; and prevent potential future data loss. If a CAN is determ i ned to be unnecessary, management will document how the issue was resolved and which parties were

notified of the issue.

Data Transfer to LIMS

Data from the analytical system is uploaded to LIMS manually or electronically by the analyst.

Assistance from the LIMS administrator may be needed in certain circumstances. Instrument to LIMS transfers should be verified by the analyst. In management

- approved special situations, where LIMS transfer and storage is unnecessary, the data must be electronically store d in an appropriate file. All raw data should be archived appropriately.Final Report

- August 2016

21 Data Analysis Records

All raw data, calculations, observations, validation information, and results generated by the analyst must be placed in an appropriate computer file, bound or electronic laboratory notebook or other approved format. For bound notebooks, all entries must be initialed and dated by the analyst. All analysis hardcopies must be stored in an appropriate filing system until archiv ed

. Any raw analytical data stored on a computer hard drive should be routinely backed up. A backup copy of all instrument software, including NLB developed parameters, should be made after the initial development. All backup files are labeled.

An instrument maintenance logbook must be assign ed to each instrument. All calls for service, repair records, reconfigurations, or changes to the instrument operating parameters must be recorded, dated, and signed by the analyst or instrument service representative. The logbook must be kept with the ins trument and be available for inspection at any time.

Analytical Data Reports

Analytical data reports are generated by the analyst and submitted for review/approval after initial data assessment and transfer to LIMS in order to verify and validate the dat a. At a minimum, the following must be included in the data package: cover page with signature and date blocks, title of method, program, or project name, timeframe of analyses, and pertinent documentation related to the analytical run; analytical run sequ ence; data with comments and flags; quality control data; control charts; and calibrations.

LIMS Verification and Validation

LIMS has been programmed to automatically verify and validate data entered into the database. Any data outside QC criteria are hi ghlighted for analyst, peer, and management review and comment. QC parameters programmed into LIMS come from federal regulations, NLB’s Quality Control Manual (QCM), and SOPs. All programming has been tested and verified by the LIMS

administrator.

Data Re view and Approval

The data review and approval process consists of a series of checks to ensure the analytical data generated by the laboratory and transferred to LIMS meet all the method specific QC criteria. The multistep process includes, at a minimum, analyst and peer review followed by management review and approval prior to submittal to clients. All levels of review and approval are initialed and dated on the cover page of the data package.

Analyst Review

The following items are verified by the an alyst who performed the extractions and analyses, as appropriate for the method, as required by the method SOP:

 Extraction solvents and volumes.

 Instrument conditions.Final Report

- August 2016

22  Analytical run conducted per SOP

.

 Expiration dates of standards.

 Retention times.

 Inte grations.

 Peak identifications.

 Analytical sequences.

 Environmental conditions.

 QC (such as duplicates, standards, blanks, controls).

 Data reduction and calculations.

 Co C data login to LIMS

.

 Raw data concentrations transferred to LIMS.

 Holding times.

 Calibrations.

 Parameters of SOP and QCM are met.

 Anomalies and corrective actions are documented and management notified, as

necessary.

Peer Review

The following items are verified by a second analyst:

 Data package completeness.

 Spot

- check calculati ons.

 Check for documentation of unusual events.

 Corrective action review.

 Check for outliers.

 Analytical run sequence.

 Dilutions performed as necessary.

 QC (such as duplicates, standards, blanks, controls).

 Expiration dates of standards.

 Reasons for in valid samples.

 Flags and comments.

 Parameters.

 SOP and QCM are met.

If necessary, a data package is returned to the analyst for edits or clarification. After corrections are made, the data package is returned to the peer reviewer for confirmation. Onc e peer review is complete, the peer reviewer signs and/or initials, and dates the analytical data package.

Management Review and Approval

The following is reviewed by management prior to data release:

 Data package completeness.

 Spot

- check calculations

.

 Check for documentation of unusual events.Final Report

- August 2016

23  Corrective action review.

 Check for outliers.

 Analytical run sequence.

 QC (such as duplicates, standards, blanks, controls).

 Expiration dates of standards.

 Reasons for invalid samples.

 Flags and comments.

 Check for analyst and peer review.

 Parameters of SOP

 QCM are met.

If necessary, a data package is returned to the analyst for edits or clarification. After corrections are made, the data package will be returned to management for confirmation. Once rev iew is complete, management(s) signs and/or initials and dates the analytical data package.

Data Release and Reporting

Data generated by the laboratory goes through the defined data review and approval process prior to release and reporting to clients. During the review and approval process, the data files in LIMS are locked to ensure that data cannot be changed without proper management authorization. Data in LIMS can still be viewed (read only) by management and laboratory staff. If changes to finaliz ed data are made, the full review and approval process shall be conducted to amend the data report. The approved data reports are provided to management, who sends the hardcopy and/or electronic data report to the client or the client's representative.

Amendment to Data

Finalized and approved data may be amended in LIMS, AQS, and client reports per management approval. After the request is approved, laboratory staff and management follow the data review and approval process. If changes to finalized da ta are made, the client is notified and sent a revised report. Data may be amended for reasons such as CANs, AQDAs, requests by clients (i.e. requests to exclude codes), etc.

Data Archive

All final hardcopy reports with the analyst review, peer review, a nd management approval signatures ar e filed in a secure manner. Access to hardcopy and LIMS files is limited to authorized individuals only. Laboratory retention of hardcopy and electronic LIMS data files shall be a minimum of five years. Final archiving a nd/or destruction of all data reports are approved by management.

3. CARB QA Data

QA data are generated from state

- wide performance evaluation audits conducted by CARB

. These audits are performed independently from the ambient air monitoring operations. QAS is responsible for conducting performance audits of criteria and non

- criteria pollutant analyzers,Final Report

- August 2016

24 particulate samplers, meteorological equipment, and laboratory analyses utilized for generating ambient level measurements. QAS also reports quality asse ssment and quality control results.

Performance audits within the entire CARB PQAO are conducted annually for gaseous pollutant monitoring, mass analysis laboratories and meteorological instruments, and semi

- annually for particulate matter samplers. The results of the audits are maintained online on the CARB website and are uploaded by CARB to AQS. EPA conducts an annual comparison with the CARB audit vehicles to ensure comparability with EPA National Performance Audit Program (NPAP); these data are uploa ded to AQS by CARB.

In addition to collecting and managing data from audits, QAS reviews quality assurance data for the entire state. At least once a quarter, QAS staff retrieves the “QA Data Quality Indicator Report” (AQS report code AMP256) f rom AQS for all California sites and all pollutants. QAS staff review the reports for inconsistencies and completeness and works with each agency to address any issues. QAS also verifies and validates the “Certification Evaluation and Concurrence Report” (AQS report code AMP 600 ) during the annual data certification process.

4. CARB Special Purpose Monitoring

Special Purpose Monitoring is conducted on an as

- needed basis by the Special Purpose Monitoring Section, in AQSB. Depending on the nature of the project, data m ay be or may not be uploaded to AQS.

5. Collected by Local District / Uploaded to AQS by CARB

The Air Quality Analysis Section (AQAS) in AQPSD is responsible for uploading continuous air quality data (O 3 , NO 2 , CO, SO 2 , and non

- filter based PM), PM data that are weighed by the local district, and meteorology data from those local districts without direct access to AQS. These districts include: ICAPCD, LCAQMD, MeCAQMD, NCUAQMD, NSCAPCD, PCAPCD, SMAQMD, SiCAPCD, TeCAPCD, and Y

- SAQMD (Table 4). The AQAS is l ocated at the Cal EPA Headquarters building in Sacramento, CA and is managed by Gayle Sweigert. AQAS uploads continuous PM data from local districts, as well as PM filters data that are weighed by LCAQMD, NCAQMD, and SMAPCD. PM filters that are collected b y local districts and then weighed by CARB are managed by the CARB MLD NLB.

Data are received electronically by email from the ten local districts listed in the previous paragraph. Local districts typically send their data monthly or quarterly in the for m of an electronic file. When data arrive at AQAS, they are logged in and uploaded to AQS, which runs them through AQS review protocols. Staff sends an email to the district staff confirming the upload, which has attached the AQS raw data report and the ra w data inventory report. If the AQS review protocols identify a potential issue, AQAS staff contacts the district to resolve it. The districts are expected to submit only fully validated and reviewed data that are accompanied

with a data validation letter consistent with the protocols identified in the roles and responsibilities documents. If a district sends data without a data validation letter, CARB contacts the district to let them know the data will not be uploaded until a data validation letter isFinal Report

- August 2016

25 rec eived. CARB assumes that the data to be uploaded to AQS for local districts have been fully validated, and carry the appropriate flags.

Data uploaded by AQAS for the ten districts listed above are certified by CARB annually, as required by regulation (40 CFR 58.15).

Table 4. Summary of data upload and certification responsibilities for the State of

California. Operation of Monitoring

Site Upload of

Continuous Data Certification Upload of PM Filter

based Laboratory

Data Certification

A VAQMD MDAQMD MDAQM D MDAQMD MDAQMD

C ARB C ARB C ARB C ARB/*North Carolina C ARB

EKAPCD N/A N/A C ARB/ SDCAPCD (PM 2.5 ) C ARB

GBUAPCD GBUAPCD GBUAPCD GBUAPCD GBUAPCD

ICAPCD C ARB C ARB C ARB

- PM 10 /

SDCAPCD

- PM 2.5 C ARB

– PM 10 /

ICAPCD

– PM 2.5
LCAQMD C ARB C ARB C ARB + C ARB +

MeCAQM D C ARB C ARB N/A N/A

MDAQMD MDAQMD MDAQMD MDAQMD MDAQMD

MBUAPCD MBUAPCD MBUAPCD BAAQMD

- PM 2.5 MBUAPCD

NCUAQMD C ARB C ARB BAAQMD

- PM 2.5 NCUAQMD

- PM 2.5

NSAQMD NSAQMD NSAQMD C ARB C ARB

NSCAQMD C ARB C ARB N/A N/A

PCAPCD C ARB C ARB Pending Termination Pen ding

Termination

SMAQMD C ARB C ARB C ARB

- PM 2.5 C ARB

- PM 10 + C ARB

- PM 2.5 ,

C ARB

- PM 10 +

SJVAPCD SJVAPCD SJVAPCD C ARB C ARB

SLOCAPCD SLOCAPCD SLOCAPCD N/A N/A

SBCAPCD SBCAPCD SBCAPCD N/A N/A

ShCAQMD ShCAQMD ShCAQMD C ARB C ARB

SiCAPCD C ARB C ARB C ARB C ARB

TeCAPCD C ARB C ARB C ARB C ARB

VCAPCD VCAPCD VCAPCD N/A N/A

Y

- SAQMD C ARB C ARB C ARB C ARB

Note : Operating agencies are responsible for data management from generation through validation, except as noted above. Agencies not performing upload and certificati on of their own data must provide a letter to the agency performing data upload and certification verifying that data ha ve been validated according to a C ARB and/or EPA approved procedure.Final Report

- August 2016

26 * Spiral Aerosol Speciation Sampler (SASS) filters sent to EPA in North Carolina for the Bakersfield California Ave nue site only

.
+ These data are weighed by local air district but uploaded by C ARB.

6. Collected and Uploaded to AQS by Local District

The following districts have access and authority to upload th eir data to AQS: G BUAPCD, MDAQMD (includes AV APCD), NSAQMD, MBUAPCD, ShCAQMD, SJVAPCD, SLOCAPCD, SBCAPCD, and VCAPCD (see Table 4). The reporting of data into AQS by these local districts was agreed upon and commemorated in district

- specific Memorandums of Understanding signed by the districts, CARB, and EPA starting in 2002. All levels of data management are handled by the local districts and these responsibilities are formalized in the

R&R documents established between the local districts and CARB. Distri cts are required to adopt and follow CARB procedures for validation and certification of data, or submit their own procedures to CARB for review and approval. CARB and district procedures are posted on the

on

- line document repository and verified by CARB d uring the TSA process. Additionally, CARB provides training and guidance on the data validation and certification processes via in

person (PQAO training), webinar, and on

- line resources. CARB expects the data that it does not upload to be certified per reg ulation (40 CFR 58.15) by either the local district collecting it or by the agency uploading it

.

7. Collected by Local District / Laboratory Analyses Performed by a Laboratory Other

Than CARB

All continuous data collection is managed by each local distri ct; CARB uploads data for ten of the districts (see Table 4). Data generation and management for PM filter

- based data are not only

district

- specific, but can even be site

- specific. Some districts weigh their filters and upload and certify their PM data. So me collect the filters and send them to CARB for weighing, validation, upload, and certification. Other districts collect the filters and send them to a local district that has a weigh laboratory; responsibility for data validation, upload, and certificati on in these cases

is defined in the R&R document. Finally, some districts may send filters from some sites to CARB and filters from other sites to a different local district that may be closer to the site. In the

case where all filters are sent to a differ ent local district, responsibility for data validation, upload, and certification of the filters is defined in the R&R document

.

Quality Assurance/ Quality Control

An organization’s quality management system includes QA

, QC , and quality improvement

activ ities. EPA requires that ambient air monitoring agencies have a quality management system that conforms to 40 CFR Part 58 Appendix A and the EPA quality policy (EPA Order CIO 2106.0). Additionally, EPA grant regulations specifically require each grantee to provide for QA activities (40 CFR Part 31.45). Specifically, 40 CFR Part 58 Appendix A Section 3 requires that each ambient air monitoring PQAO conforms to certain quality management practices. These

include:

 A documented quality system that meets EPA req uirements for QMPs and QAPPs.

 A quality management function that is independent of air monitoring operations.Final Report

- August 2016

27  Stated data quality objectives or equivalent systematic planning procedures for all monitoring programs.

 Participation in National Performance Eva luation Programs, which consist of performance audits to independently determine program adequacy, national monitoring network performance, and national consistency.

 EPA

- led TSA every three years or less.

 Use of certified reference materials to standardize monitoring equipment.

EPA views these quality management system components as being integral to maintaining a credible monitoring program. Insufficient quality management and control has been cited in support of legal challenges to NAAQS designation deci sions.

QA and QC are the two components of a quality management system for a monitoring program that serve to document the assertion that the data collected represent the true air quality of the area. They are the means by which an organization manages i ts quality aspects in a systematic, organized manner to provide a framework for planning, implementing, and assessing work performed by an organization. A properly developed QA/QC program encompasses a variety of technical and administrative elements, incl uding policies and objectives, organizational authority, responsibilities, accountability, and procedures and practices. QA is a management or oversight function setting policy and overseeing an administrative system of management controls that cover plann ing, implementation, the review of data collection activities, and the use of data in decision making. QC is a technical function that includes all the scientific precautions, such as calibrations and duplications needed to acquire data of known and adequa te quality.

The CARB QMB is composed of three sections: QAS, the Quality Management Section (QMS), and the Office of Emergency Response. EPA did not include the Office of Emergency Response

with this TSA. P rimary responsibilities of QAS include:

 Conducti ng performance audits of MLD and district monitoring instruments.

 Conducting TSAs of California air districts.

 Updating SOPs specific to QAS activities.

 Validating MLD’s field generated monitoring data (accuracy assessments).

 Provid ing verification/ce rtification services within the Standards Laboratory.

QMS performs quality management oversight activities for C ARB internally as well as for the

entire PQAO. P rimary responsibilities of QMS include:

 Assisting with system audits of California air district s

 Preparing data quality summary reports for the CARB PQAO and other PQAOs in

California.

 Providing review and approval of quality management documents for C ARB and districts within the PQAO.

 Coordinating PQAO training on the fundamentals of ambient air monitoring.

 Conducting liaison activities to enhance communication and QA/QC activities across the PQAO.

QC

- related functions are performed by AQSB. AQSB performs several quality management

functions , including :Final Report

- August 2016

28  Developing and administering the traini ng program for instrument operators.

 Performing instrument acceptance testing.

 Validating MLD field generated monitoring data.

 Developing, preparing and reviewing SOPs for the air monitoring program.

 Determining monitoring methods used in the ambient air m onitoring network.

QA

- related functions in NLB include:

 Developing laboratory and ambient air collection test procedures.

 Conducting analyses of ambient air samples and consumer products.

 Performing self

- assessments and producing a n annual quality contro l summary report for

the Division Chief.Final Report

- August 2016

29 OVERVIEW OF THE MOJAVE DESERT AIR QUALITY MANAGEMENT DISTRICT

AIR MONITORING PROGRAM

General Program

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of t he Mojave Desert Air Quality Ma nagement District ( MDAQMD ). All information contained within this

report section reflects the status of the MDAQMD monitoring program at the time of audit (April 17, 2015). MDAQMD is currently part of CARB PQAO ; this audit included an agency

- specific

asses sment of network design, field operations, data handling, quality assurance and quality control procedures. As Mojave staff operate the Antelope Valley Air Quality Management District (AVAQMD) regulatory ambient air monitoring network, findings noted in th is report may apply to both agencies.

Agency Description

MDAQMD is the local agency responsible for controlling emissions from stationary sources of air pollution within the desert portion of San Bernardino County and the Palo Verde Va lley area

of Rivers ide County. Covering over 20,000 square miles of primarily desert terrain

, MDAQMD implements ai r quality programs required by state and f ederal mandates , enforces rules and regulations based on air pollution laws, and educates business and residents about their role in protecting air quality. The Governing Board consists of appointed representatives from each incorporated city within its jurisdiction, the County Supervisors whose districts are contained in whole or in part within its jurisdiction, and one p ublic member app ointed by the Governing

Board. The Governing Board appoints an Air Pollution Control Officer who serves as the Executive Officer of the district and controls day to day operations

. S/he is also the enforcement officer for the applicable law s, rules, and regulations. MDA Q MD funds its activities through a variety of fees, including but not limited to permit fees charged to local business and industry subject to its rules and regulations. MDAQMD is organized into a variety of divisions each wit h its own specific duties and responsibilities. The divisions primarily subject to this TSA are Ambient Air Monitoring and Information Systems Departments ( Appendix C )

.

EPA interviewed Christopher Collins, monitoring program manager; Aleksey Astakhov, st ation operator and repair technician; and Orlando Salinas, station operator and part iculate laboratory

technician. EPA visited four monitoring sites located in Lancaster, Victorville, Lucerne Valley,

and Phelan. CARB staff shadowed EPA during this TSA.

MD AQMD staff were very accommodating and worked with EPA to explain procedures and

obtai n the documentation requested.

Network Management

At the time of the TSA, t he amb ient air monitoring network operated and maintained by

MD AQMD staff consist ed of six a mbient air monitoring sites within MDAQMD and one siteFinal Report

- August 2016

30 within AVAQMD that utilize d a variety of air pollutant instruments. T able 5 provides information on the seven sites operated by MDAQMD.

Table 5 . Sites, Pollutants Monitored

, and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Barstow 06

- 071

- 0001 CO

NO/NO 2 /NOx

PM 10

– Continuous

O 3 MDAQMD

H esperia 06

- 071

- 4001 PM 10

– Continuous

O 3 MDAQMD

Lucerne Valley 06

- 071

- 0013 PM 10

– Continuous MDAQMD

Phelan 06

- 071

- 0012 O 3 MDAQMD

Trona 06

- 071

- 1234 H 2 S

NO/NO 2 /NOx

PM 10

– Continuous

O 3 SO 2 MDAQMD

Victorville 06

- 071

- 0306 CO

NO/NO 2 /NOx

PM 10

– Continuous

PM 2.5

– Filter (1in6)

SO 2 O 3 MDAQMD

Lancaster 1 06

- 037

- 9033 CO

NO/NO 2 /NOx

PM 10

– Continuous

PM 2.5

– Filter (1in6)

O 3 MD AQMD 1

1 The Lancaster site is ope rated by MDAQMD, but is located within AVAQMD.

A full description of the network is provided in the CARB Annual Monitoring Network Plan

. At the time of the audit, CARB ran an additional site in MDAQMD (Blythe), and the National Park

Service maintained a site in MDAQMD as we ll (Joshua Tree). See Appendix C for a map depicting the monitoring network within the Mojave Desert Air Basin.

MD AQMD submits an annual review of the a mbient a ir m onitoring s tations located in its jurisdiction t o C ARB

. The stations ho use monitoring instruments that measure ambient levels of gaseous and particulate (solid and liquid aerosol) a ir pollutants. Most of the stations also collect meteorological data. The a nnual m onitoring n etwork p lan is submitted to AQPSD, which compiles inf

ormation and sends an annual network plan on behalf of multiple air agencies within the PQAO to EPA. When MDAQMD seeks to make changes to its network outside the a nnual

m onitoring network p lan process

, it does so in agreement with CARB in orde r to resolve potential issues. After changes are approved

, CARB submits its request to EPA per 40 CFR

58.14

. The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for PM 10 ,Final Report

- August 2016

31 PM 2.5 and O 3 were being met for MDAQMD ; additional details can be found in th e CARB annual monitoring network plan and the EPA response letter

.

Field Operations

T he Lead Air Quality Instrument Technician ( hereafter referred to as “Lead Tech” ) and the Air Quality Instrument Technician ( hereafter referred to as “Instrument Tech” ) handle d all the day

to

- day operations for their assigned sites

, including instrument installation

, repair and

maintenance

, sample collection

, first

- level data review

, some quality control activities

, and general field site maintenance.

Technicians were r esponsible for three to four sites and visit ed each site at least two times per

mo nth. The monitoring stations were set up to perform automated nightly internal zero/spans (IZS). PM 10 and PM 2.5 flow

- rate verifications were performed by both the Lead Tech a nd the

Instrument Tech for their assigned sites. Corrective ac tions and repair/maintenance were dealt

with on a case

- by

- case basis.

In order to establish independence or peer cross verification

, the Lead Tech and I nstrument T ech

reviewed each other’s wor k.

Laboratory Operations

At the time of the audit, MDAQMD operate d a PM laboratory in which PM 2.5 filters were analyzed for comparison to the NAAQS . The results of the analys e s were uploaded to AQS. The procedures for the lab were delineated in the SOP created by C ARB. The file can be located

here: http://arb.ca.gov/airwebmanual/aqsbdocs1/v2apxaj.pdf

Data Management

Data management generally involves data collection, the data validati on process, and a data

management system. According to MDAQMD , their data management system followed the process described below at the time of the audit.

Data management follow ed two separate tracks

, one for lab data and one for continuous (i.e.,

non

- la b) data.

The filter cycle include d the following:

1. Laboratory conditioning

. 2. Schedules, assignments and sample tracking

. 3. Laboratory data and QC imported into ACCESS database

. 4. ACCESS s tore d , manage d data, and perform ed calculations

. 5. Data were prepared to b e loaded into AQS

. 6. AQS report generated to support data certification

.

The continuous gas and PM data cycle included the following: 1) Concentrations were measured by regulatory ambient air analyzers

.Final Report

- August 2016

32 2) Each a nalyzer transmit ted its data to a data logger, wher e the data were date

- and time

stamped, collated, organized and recorded. 3) The data were collected from the data loggers via computer software into the server, and

stored in a central database.

 AirVision software pull ed data from data loggers on an hourly b asis.

 AirVision store d raw data on servers
, and one

- hour readings are available for QA/QC. 4) The data were reviewed using peer

- review process.

 Level I, II and III checks were performed on a saved copy of raw data.

 Level I checks were made by assigned technic ian

.

 Level II checks were made by non

- assigned technician or supervisor

.

 Level III checks were made by supervisor and non

- assigned site technician. 5) Data went through a QA / QC process.

 Performance evaluations were based on audits performed by MDAQMD staff an d

external auditors.

 Internal checks and audits performed by Lead Tech and Instrument Tech were reviewed by manager

.

 External audits were performed by C ARB. 6) Data were archived and made available to the public via the MDAQMD website.

 Hourly files were store d.

 QA/QC data files were available to the public upon request.

 Certified and historical data were available through AQS.

QA/QC

40 CFR Part 58 Appendix A Section 2 requires that each ambient air monitoring PQAO conforms to certain quality management pract ices. These include: 1) Having a documented quality system that meets EPA requirements for QMPs and QAPPs.

 As part of a PQAO, MD AQMD follows the C ARB QMPs and QAPPs. 2) Having a quality management function that is independent of air monitoring operations.

 As par t of a PQAO, C ARB performs independent QA checks on the MDAQMD

system. 3) Developing or adopting DQOs, or equivalent systematic planning procedures, for all monitoring programs.

 MDAQMD follows the C ARB DQOs for its monitoring program. 4) Participating in Nationa l Performance Evaluation Programs (NPEP), which consist of performance audits used to independently determine program adequacy, national monitoring network performance, and national consistency.

 As part of a national network MD AQMD participates in NPAP for gaseous instruments and the N PEP for particulate samplers. The NPAP and NPEP audits are performed by EPA Region IX contractors

.

 EPA performed the last NPAP audit on 11/21/2014 on the Trona site (06

- 071

- 1234).

The O 3 , SO 2 , NO/NOx instruments were audited. 5) Participating in TSAs by EPA at a frequency of every three years or less.Final Report

- August 2016

33  As part of a national network C ARB is subject to TSAs performed by EPA ; EPA may audit MD AQMD as part of the TSA

. 6) Using certified reference materials to standardize monitoring equipme nt.

 Equipment is calibrated to Standards certified by C ARB.

 All standards are certified every year.

 All equipment is calibrated at least once a year using the certified standards.

How QA/QC wa s structured for MD AQMD

MD AQMD conduct ed most QC activities a nd relied on CARB for support for QA audits . QC activities conducted by MDAQMD include d :

 O ne

- point QC checks for gaseous monitors that occur nightly (nightly calibrations)

.

 M onthly r equired monthly flow verifications for PM 10 and PM 2.5

.

CARB conduct ed the following :

 QA activities: gaseous an nual performance evaluations

.

 S emi

- annual flow rate audits for PM 10 and PM 2.5

.

 Meteorological audits

.

 G aseous and flow standard verifications and certifications.

EPA contractors visited one site on 11/21/2014 t o perform a NPAP audit on the following

instruments:

 O 3

 SO 2

 NO/NOx

EPA contractors also perform ed NPEP audits on the PM monitors.Final Report

- August 2016

34 O VERVIEW OF NORTH COAST UNIFIED AIR QUAL I T Y MANAGEMENT

DISTRICT AIR MONITORING PROGRAM

General Program

As part of the CA RB TSA, EPA reviewed the ambient air monitoring activities of North Coast Unified Air Quality Management District (NCUAQMD). All information contained within this

report section reflects the status of the NCUAQMD monitoring program at the time of audit ( Ma y 26

- 28, 2015 )

.

NCUAQMD is currently part of CARB PQAO ; this audit included an agency

- specific assessment of network design, field operations, data handling, quality assurance and quality control procedures.

The NCUAQMD is located in the far northwester n portion of California. It is comprised of three counties: Del Norte, Humboldt, and Trinity, which together cover 7,753 square miles. It is bordered on the west by the Pacific Ocean and extends from the Oregon b order south approximately 140 miles to the M endocino County line in California. Eureka, the county seat of Humboldt County, is 284 miles north of San Francisco, 466 miles south of Portland, Oregon and on the coast of the Pacific Ocean. The area is made up of varied terrain, from coastal wetlands to rugged mountains.

Supervisors from each of the three counties serve on the NCUAQMD board along with a city representative from one of the cities within NCUAQMD’s jurisdiction. NCUAQMD primarily funds its activities through a combination of state and fede ral grants, permit fees charged to businesses/industries, and vehicle registration fee s

.

NCUAQMD is responsible for ensuring federal and s tate ambient air quality standards are achieved and maintained within the county

. At the time of the TSA, NCUAQMD wa s organized into four main areas, each of which plays a role in protecting the public from the adverse impacts of air pollution. The NCUAQMD Division that wa s the subject of this TSA wa s the Air Monitoring Division. The Air Monitoring Division included two staff members, both of whom

were Air Pollution Specialist s . The Air Monitoring Division wa s managed by the District Air

Pollution Control Officer.

EPA staff interviewed NCUAQMD management and staff and visited two monitoring sites:

 Eureka

- Jacobs in Humboldt County (AQS ID 06

- 023

- 1004), and

 Eureka

- Humboldt Hill in Humboldt County (AQS ID 06

- 023

- 1005).

NCAQMD management and staff were accommodating, making themselves available for several interviews, procedural reviews and monitoring site visits. Man agement and staff interviewed

were:

 Brian Wilson, Air Pollution Control Officer

 Wendy Caruso, Air Pollution Specialist

 Randy Dickau, Air Pollution SpecialistFinal Report

- August 2016

35 Overall, the monitoring staff, despite its small size, demonstrated good knowledge and showed

de dication to improving air monitoring operations. In the attached findings, EPA highlights several areas for suggested improvement.

Network Management

At the time of the TSA, t he ambient air monito ring network in NCUAQMD consisted of four

ambient air mon itoring sites that utilize d a variety of air pollutant instruments. A full description of the network is provided in NCUAQMD ’s Annual Monitoring Network Plan. Table 6 lists sites, pollutants monitored, an d operating agency

.

Table 6. Site, Pollutants Monit ored and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Eureka

- Jacobs 06

- 023

- 1004 O 3 , NO 2 , SO 2 , CO, PM 10 , PM 2.5 NCUAQMD

Eureka

- Humboldt

Hill 06

- 02301005 O 3 , NO 2 , SO 2 , CO, PM 2.5 NCUAQMD

Crescent City 06

- 015

- 0006 PM 2.5 NCUAQMD

Weaverville 06

- 105

- 0002 Non

- FEM PM 2.5 NCUAQMD

NCUAQMD submits its a nnual m onitoring n etwork p lan directly to EPA. If a local district within the CARB PQAO seeks to make changes to its network outside the annual monitoring network plan process, the district should fi rst work with CARB to resolve potential issues and then submit its request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for SLAMS were being met by NCUAQMD ; additional details can be found in NCUAQMD’s annual monitoring network plan and EPA’s response letter

.

Field Operations

Field technicians handle all day

- to

- day operations including : instrument installation, repair and maintenance; sample collection; first

- level data review; som e quality control activities; and general field site maintenance. At the time of the audit, NCUAQMD ha d on e primary field

technician who wa s responsible for visiting all four stations. The two gaseous stations in Eureka

were visited multiple times each wee k. The two particulate

- only sites were visited roughly once per month. The monitoring stati ons operated by the district were set up to perform automated zero/one point precision checks every thir d day. Automated span checks were run once per week.

PM 10 and PM 2.5 flow verifications were performed monthly by NCUAQMD field technicians

.

Corrective actions and repair/maintenance were dealt with on a case

- by

- case basis. Generally, equipment repairs were performed by NCUAQMD personnel. On rare occasions, instrum ents may be sent to the vendor or CARB for repair.Final Report

- August 2016

36 Laboratory Operations

NCUAQMD does not operate a PM laboratory ; instead it was sending PM 2.5 filters to BAAQMD for gravimetric analysis. NCUAQMD u sed the CARB laboratory for the following analyses: O 3

p rimary standard verification and calibrator mass flow controller certifications.

Data Management

Data management generally involves data collection, the data validation process, and a data management system. Data management at NCUAQMD followed two separa te tracks

, one for continuous (i.e., non

- lab) data and one for lab data.

Continuous data were entered into the DR DAS © management system. Data were collected at the monitoring site using a central computer, and polled from the main office air monitoring server hourly. The raw data were backed up nightly and never overwritten. The data were edited and validated following the procedures in NCUAQMD’s d ata m anagement and v alidatio n SOP.

First level data review wa s done by the primary field te chnician Randy Di ckau, who took note of any maintenance or observable events from the f ield. Second level data review wa s conducted from the office by Air Pollution Specialist Wendy Caruso , who graphed the data, noted problems, determined causes, and validate d the data as warranted. Electronic records were kept to document the data validation steps. Once validated by NCUAQMD, a n AQS file wa s generated and sent to CARB for AQS uploading. CARB would then run an automated final check on the continuous data prior to AQS uploadi ng , and contact NCUAQMD to address any potential problems that arose during the automated check.

BAAQMD assist ed NCUAQMD with PM 2.5 filter data analysis at the time of the audit. BAAQMD shipped conditioned filters with CoC forms to NCUAQMD approximately e very two to three weeks. NCUAQMD deploy ed filters to field instruments fo r data collection and then shipped filters back to BAAQMD for gravimetric analysis. Any field notes for data validation

consideration were documented on the CoC forms by NCUAQMD and B AAQMD. Once analysis was completed by BAAQMD, raw weights (in spreadsheet format) and filter conditions

were emailed back to NCUAQMD where the information was converted into a format

ap propriate for AQS. An AQS file wa s generated and sent to CARB for AQS u ploading.

QA/QC

NCUAQMD conducted some QA/QC activities and relied on CARB to support others. QA/QC activities conducted by NCUAQMD include d one

- point QC checks for gaseous monitors, and required flow verifications for PM 10 and PM 2.5 . CARB conduct ed the following QA/QC activities: gaseous annual performance evaluations, semi

- annual flow rate audits for PM 10 and

PM 2.5, meteorological equipment audits, multi

- point instrument calibrations, and photometer

certification. EPA contractors visit the sites on occ asion, conducting comparison monitoring using EPA field equipment

.Final Report

- August 2016

37 OVERVIEW OF SAN LUIS OBISPO COUTY AIR POLLUTION CONTROL DISTRICT

AIR MONITORING PROGRAM

General Program

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of Sa n Luis Obispo County Air Pollution Control District (SLOCAPCD). All information contained within

this report section reflects the status of the SLOCAPCD monitoring program at the time of audit (August 26

- 27, 2015).

SLOCAPCD is currently part of CARB PQAO ; this audit included an agency

- specific assessment of network design, field operations, data handling, quality assurance and quality control procedures

.

At the time of the audit, the Board of Directors of the SLOCAPCD had twelve members comprising all f

ive County Board of Supervisors and a representative from each of the seven incorporated cities located within the county. SLOCAPCD is responsible for ensuring f ederal

and s tate ambient air quality standards are achieved and maintained within the county. S tate law assigns to local districts the primary responsibility for control of air pollution from stationary sources. This is typically accomplished through the adoption and implementation of rules and regulations. SLOCAPCD is also responsible for the inspe ction of stationary sources, monitoring of ambient air quality, development and updating of attainment plans, and maintenance of the emission inventory. SLOCAPCD funds its activities through a combination of permit and authority to construct fees, vehicle registration fees, property taxes, and a small portion of State

and Federal aid.

When the TSA was conducted, SLOCAPCD was organized into three main areas, each of which plays a role in protecting the public from the adverse health impacts of air polluti on: 1) Program Administration and Management, 2) Engineering & Compliance and 3) Planning, Monitoring &

Outreach. The SLOCAPCD Division that wa s the subject of this TSA is the Planning, Monitoring and Outreach Division (PMOD).

At the time of the TSA, PMOD wa s comprised of one Division Manager (Aeron Arlin Genet), five Planning Air Quality Specialists and two Monitoring Air Quality Specialists (Jaime Contreras and Karl Tupper). The operations of the SLOCAPCD eight air monitoring stations

were divided equall y amongst the Monitoring Air Quality Specialists

. T his includes field operations, data management and QA/QC

.

EPA staff interviewed SLOCAPCD management and staff and visited four monitoring sites located in San Luis Obispo County:

 CDF (AQS ID 06

- 079

- 2007 )

,

 Mesa2 (AQS ID 06

- 079

- 2004 ),

 Nipo mo Regional Park (AQS ID 06

- 079

- 4002 ), and

 Red Hills (AQS ID 06

- 079

- 8005 ).Final Report

- August 2016

38 SLOCAPCD managers and staff were very accommodating, making themselves available for many interviews, procedural reviews and monitoring site visi ts. Management and staff

interviewed were :

 Aeron Arlin Genet

– Planning, Monitoring and Outreach Division Manager

 Karl Tupper

– Monitoring Air Quality Specialist

 Jaime Contreras

– Monitoring Air Quality Specialist

Overall, the monitoring staff was very de dicated and knowledgeable

. Their internal tracking forms, logbooks, and training forms were exemplary.

Network Management

At the time of the TSA, t he ambient air monitoring network in SLOCAPCD consist ed of ten ambient air monitoring sites that utilize d a variety of air pollutant instruments. Eight sites were operated by SLOCAPCD and two by CARB. A full description of the network is provided in

SLOCAPCD ’s a nnual monitoring n etwork p lan

. T able 7 lists sites, pollutants monitored, and operatin g agency

.

Tab le 7. Sites, Pollutants Monitored and Operating Agency

Site Name AQS ID Pollutants Operating Agency

Atascadero 06

- 079

- 8001 (old)

06

- 079

- 8002 (new) O 3 , NO 2 , PM 10 ,

PM 2.5 SLOCAPCD

Carrizo Plain 06

- 079

- 8006 O 3 SLOCAPCD

CDF 06

- 079

- 2007 PM 10 , PM 2.5 SLOCAPCD

Grover Beach 06

- 079

- 2001 NONE, met only SLOCAPCD Morro Bay 06

- 079

- 3001 O 3 , NO 2 SLOCAPCD Nipomo Regional Park 06

- 079

- 4002 O 3 , NO 2 , PM 10 SLOCAPCD

Mesa2 06

- 079

- 2004 SO 2 , PM 10 , PM 2.5 SLOCAPCD

Red Hills 06

- 079

- 8005 O 3 SLOCAPCD

Oso Flaco 06

- 079

- 9001 PM 10 S LOCAPCD San Luis Obispo 06

- 079

- 0005 O 3 , PM 10 , PM 2.5 CARB

Paso Robles 06

- 079

- 2006 O 3 , PM 10 CARB

SLOCAPCD submits its a nnual m onitoring n etwork p lan directly to EPA. If a local district within the CARB PQAO, such as SLOCAPCD, seeks to make changes to its network outside the

a nnual m onitoring n etwork p lan process, the district first work s with CARB to resolve potential

issues and then submit s the request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for PM 10 , PM 2.5 and O 3 were being met for San Luis Obispo County ; additional details can be found in SLOCAPCD’s annual monitoring network plan and EPA’s response letter

.

Field Operations

Field technicians handle all day

- to

- day operation s
, including instrument installation, repair and

maintenance

, data collection

, first

- level data review

, all quality control activities

, and generalFinal Report

- August 2016

39 field site maintenance. Field technicians were responsible for eight SLAMS sites and visit ed

each site accor ding to a biweekly or monthly schedule and whenever operational issues arise. The monitoring stations operated by SLOCAPCD were set up to perform automated nightly zero/precision/spans (ZPS). PM 10 and PM 2.5 flow verifications were performed by field

techni cians in a biweekly basis.

Corrective actions and repair/maintenance were dealt with on a case

- by

- case basis . Generally, equipment repairs were performed by SLOCAPCD ; occasionally repairs were performed by the

instrument manufacturer. Detailed documentat ion of repairs wa s logged in the station’s electronic logbooks and separately entered in the Errata electronic logbook

.

Responsibilities for the operation of the eight monitoring stations were assigned to two Air Quality Specialists. Each specialist wa s responsible for the complete QA/QC suite of data production and collection ; data verification of each parameter wa s a complementary process that

involve d both specialist and supervising manager. QA steps include data validation and reporting to AQS.

Labor atory Operations

SLOCAPCD does not operate a PM laboratory

. At the time of the audit, SLOCAPCD operate d

continuous BAM 1020 PM 10 and PM 2.5 samplers, thus lab oratory filter weight analysis was not required. SLOCAPCD used EPA laboratories for the certificat ion the O 3 photometer primary

standard verification. Scott

- M a rrin gas manufacturer certifies EPA protocol gas cylinder blends.

Mesa

- Labs is the manufacturer and certifier for NIST

- traceable flow transfer standards. Vaisala

is the manufacturer and certifier for NIST

- traceable relative humidity transfer standard. Met One Inc. is the manufacturer and wind tunnel verifier/certifier for sonic anemometers; SLOCAPCD no longer uses mechanical cup

- and

- va ne systems. Until recently, ICL Laboratories was the

certifier for the NIST

- traceable glass mercury thermometer transfer standard. Starting in 2015 and pursuant to the Roles and Responsibilities agreement with CARB

, SLOCAPCD has begun utilizing the C ARB Standards Laboratory for certification of transfer standards; to date , the O 3 transfer standard and ambient temperature sensor ha ve been C ARB certified. For major instrument repairs, the SLOCAPCD contracts the services of the instrument’s manufacturers.

Data Management

Data management generally involves data collectio n, the data validation process, and a data

management system.

SLOCAPCD utilize d the AirVision data management system from Agilaire Corp . This is a program that resided in a SLOC APCD server and perform ed routine data polling from each station’s data logg e rs. The data polling protocol wa s automated; each hour (or as nee ded) AirVision remotely collected data via telephone or cell phone modem connection. Each dat a packet from each site consisted of all the averaging methods including one

- minute, five

- minute, hourly averages and/or calibration verification responses. The polling program wa s a series of batch protocols that verifie d the complete transmission of data from the data logger to the server.

AirVision and the data loggers automatically applied initial QA flags to the data. This initialFinal Report

- August 2016

40 automated flagging protocol allow ed for an immediate p reliminary data validation and wa s the base for a consistent evaluation of subsequent data validation for Levels II and III. The complete

d ata v alidation protocol wa s described in S ection 5 of the SLOCAPCD SOP.

QA/QC

SLOCAPCD conducted some QA/QC activities and relied on CARB to support others. QA/QC activities conducted by the SLOC APCD include d daily calibration verification at zero, precision and span

, and semi

- annu al multi

- point QC checks for gaseous monitors. The station ga s diluters (calibrators) underwent quarterly flow verifications with a NIST

- traceable flow transfer standard.

The calibrators fitted with an O 3 ph otometer were verified semi

- annually with a Level III O 3 transfer standard following the EPA six

- by

- six Transfer Standards for Calibration of Air Monitoring Analyzers for Ozone (EPA 454/B10

- 001) . Continuous PM samplers underwent bi

weekly flow verifications for PM 10 and PM 2.5 using a NIST

- traceable certi fied transfer standard

that include d ambient temperature and barometric pressure , quarterly full calibration verification and annual complete service and 72 hours zero test . For wind sensors, SLOC APCD audited alignment and submitted all sonic anemometers t o the manufacturer for annual service and wind

tunnel calibration and certification.

CARB conduct ed the following QA/QC activities: gaseous annual performance evaluations and

semi

- annual flow rate audits for continuous PM 10 and PM 2.5, and ambient temperat ure sensors ; siting criteria evaluation ; residence time verification and inspect s QC documentation verification

for O 3 ; and gaseous standard verifications and certifications

. EPA contractors visit ed the sites annually (minimum) making flow calibration chec ks on the continuous PM monitors, as well as conducting PM coarse comparison using EPA FRM samplers.Final Report

- August 2016

41 FINDINGS

Program Area Finding Numbers EPA Contact

General G

- Positive 1

- 2 ; G1

- G2 Audit Team,

Mathew Plate

Network Management NM

- Positive1 Gwen Yoshimur a

Field Operations FO

- Positive 1

- 5 ; FO1

- FO1 0;

FO

- Recommendation1 Mathew Plate,

Gwen Yoshimura

, Dena Vallano

Data Management DM

- Positive 1

- 7 ; DM1

- DM 4 Gwen Yoshimura,

Dena Vallano

QA Management QA

- Positive 1

- 5 ; QA1

- QA 2 ; QA

- Rec ommendation 1

- 2 Dena Vallano

, Mathew Plate,

Gwen Yoshimura

Standards Laboratory SL

- Positive1; SL1

- SL 8;

SL

- Recommendation1 Mathew Plate

PM Laboratory PM

- Positive 1

- 3 ; PM1

- PM8 Mathew Plate

Pb Laboratory Pb1 Joe Eidelberg

A V AQMD & MDAQMD AV / MD

- Positive 1 ; AV / MD1

- AV / MD24 Mathew Plat e

NCUAQMD NCU

- Positive 1 ; NCU1

- NCU1 5 Gwen Yoshimura

SLOCAQCD SLO

- Positive1

- 2 ; SLO1

- SLO 5 ; SLO

Recommendation1 Dena VallanoAuditor: All

Final Report

- August 2016

42 Finding # G

– Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: General

Fin ding:

A number of important projects were underway to improve the PQAO

.

Description:

There were a number of activities underway that will further improve the CARB PQAO. Those included as individual positive findings in this report were specifically d iscussed during the TSA

. EPA commends CARB for addressing these important issues and working to establish procedures, processes, and infrastructure to support improvements to strengthen the CARB PQAO quality system.

EPA notes that some of the findings in t he report may be able to be addressed through full implementation of one or more of the activities noted in the positive findings

. While it is still appropriate for the findings to be included in the report, the positive findings indicate that CARB

is awar e of areas for improvement, and is already taking steps to address many of these issues.

References:

Suggested Actions to Address Finding : NAAuditor: All

Final Report

- August 2016

43 Finding # G

– Positive Finding 2 Agency: California Air Resources Board

Date of Audit: Summer 2015

Pr ogram Area: General

Finding:

CARB has been providing increased support to the CARB PQAO districts.

Description:

The CARB PQAO faces an ongoing challenge in ensuring consistency across its membership

. Since the 2011 TSA, CARB has made substantial c hanges to improve consistency.

Districts noted i mproved support from and communication with CARB. Highlights include :

 Assistance from the PQAO liaisons on a number of issues, including facilitating resolution of data reporting questions, network changes, and instrument issues;

 CARB providing s pare equipment for district use ;

 PQAO trainings.

CARB has developed R&R documents with each district, offered data certification trainings, created an online document repository for QA documents, solicited feedback o n documents such as the Annual Data Quality report and draft QA bulletins, continues to play an integral role on California Air Pollution Control Officers Association (CAPCOA) monthly monitoring calls, and assigns each PQAO liaison the task of visiting her /his districts on a regular basis

. These are just some of the activities CARB has undertaken to improve communication and consistency across the PQAO.

References:

Suggested Actions to Address Finding : NAAuditor: Mathew Plate

Final Report

- August 2016

44 Finding # G1 Agency: California Air Resou rces Board

Date of Audit: Summer 2015 Program Area: General

Finding:

Not all d istricts have been fully attend ing trainings or identifying and requesting a sufficient

level of technical support

.

Description:

The CARB PQAO staff have work ed dilige ntly and have improve d CARB PQAO

- specific training significantly . This has benefitted the districts within the CARB PQAO and created more consistency in operations across the PQAO. However, especially in cases where the districts

have very few staff and ar e more remotely located, districts may need CARB’s assistance to identify what additional trainings would benefit their staff. Districts should prioritize training s o that staff and managers receive the training required for proficient execut ion of their j ob responsibilities and meet their PQAO responsibilities.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Create a rec ommended training plan for : a) site operators, b) laboratory staff, c) QA/QC staff and

management, and d) data validators. Share this training plan with the districts.

F acilitate access to additional trainings by providing training resources, such as tra inings offered online, training vendors, and other training opportunities as they arise.

As needed, work with individual districts to create a district

- specific training plan.Auditor: All

Final Report

- August 2016

4 5 Finding # G2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: General

Finding:

Communication within the PQAO was not consistent

.

Description:

More effective c ommunication between CARB and some PQAO districts c ould prevent monitoring network and data quality challenges

. EPA found several instances where AQPSD ha d

not been notified of changes within a d istrict

. For example , when Mojave changed from filter

PM 10 instruments to continuous PM 10 instruments, poor communication resulted in challenges handling AQS data. (See finding AV/MD7.) Dist ricts were not consistently consulting CARB with QA or AQS questions.

Although it is generally the responsibility of the districts to inform CARB of network adjustments (i.e., per the R&R) and to raise data

- related questions to CARB

, in its oversight rol e, CARB should have mechanisms in place to keep up

- to

- date on district network changes and to provide guidance to the districts.

References:

40 CFR Part 58 Appendix A , S ection 3:

3.1 Primary Quality Assurance Organization. A primary quality assuranc e organization is defined as a monitoring organization or a coordinated aggregation of such organizations that is responsible for a set of stations that monitors the same pollutant and for which data quality assessments can logically be pooled. Each criter ia pollutant sampler/monitor at a monitoring station in the SLAMS network must be associated with one, and only one, primary quality assurance organization.

3.1.1 Each primary quality assurance organization shall be defined such that measurement

uncertai nty among all stations in the organization can be expected to be reasonably homogeneous, as a result of common factors. Common factors that should be considered by monitoring organizations in defining primary quality assurance organizations include: (a) Op eration by a common team of field operators according to a common set of procedures; (b) Use of a common QAPP or standard operating procedures; (c) Common calibration facilities and standards; (d) Oversight by a common quality assurance organization; and ( e) Support by a common management, laboratory or headquarters.Auditor: All

Final Report

- August 2016

46 Suggested Actions to Address Finding :

Institute a system for the districts to inform CARB of relevant changes or questions , either directly or at general meetings

. For example, the monthly CAPCOA call could be used to query

local districts on any network adjustments or general questions.

Track what districts do not participate on regular calls and work with EPA to encourage regular

communication.Auditor: Gwen Yoshimura

Final Report

- August 2016

47 Finding # NM

– Positive Finding 1 Age ncy: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Network Management

Finding:

The CARB PQAO has been working with EPA to create the CAMNAT database.

Description:

The CARB PQAO has a monitoring network that is operated both by the districts and CARB

. Different operating agencies overlap across various geographic regions, and different parts of the PQAO create their own ANPs and network assessments, making it challenging to assess the network as a whole. Development of CA MNAT will improve determination of network adequacy and state

- level planning.

References:

Suggested Actions to Address Finding : NAAuditor: Dena Vallano

Final Report

- August 2016

48 Finding # FO

- Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

Staff were professional and helpful, and in most instances very knowledgeable regarding field operations.

Description:

Specific examples of good practices observed during this audit include:

 CARB monitoring staff was a ctively engaged in improving the program, including monitoring systems and other data collection processes.

 Field technicians caught errors and made improvements with existing forms and

documentation.

 Field technicians developed excellent record

- keeping, internal documentation and forms.

 The field technician in San Luis Obispo noted and addressed instrument issues promptly and

maintained excellent record

- keeping and documentation at the monitoring sites.

 Where data issues were identified during data track ing

, staff were particularly helpful in identifying the root causes of these data anomalies.

References:

Suggested Actions to Address Finding : NAAuditor: Dena Vallano

Final Report

- August 2016

49 Finding # FO

– Positive Finding 2 Agency: California Air Resources Board

Date of Audit: Summer 20 15 Program Area: Field Operations

Finding:

The AMSS has developed an on

- the

- job training program for field technicians.

Description:

Phil Wagner and Fernando Amador created an on

- the

- job training program for their most recent

hire

. This program could be expanded and adopted MLD

- wide as a program for all CARB field

technicians.

References:

Suggested Actions to Address Finding :
NAAuditor: Mathew Pla te

Final Report

- August 2016

50 Finding # FO

– Positive Finding 3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Prog ram Area: Field Operations

Finding:

A training database for field staff is under development

.

Description:

A 2011 TSA f

inding highlighted the need for keeping a training database . CARB was working to develop that training database for field techn icians.

References:

Suggested Actions to Address Finding :
NAAuditor: Mathew Pla te

Final Report

- August 2016

51 Finding # FO

– Positive Finding 4 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

CARB has been working with several manufacturers as they develop new instruments.

Description:

CARB has been proactively working with instrument manufacturers: with MetOne to develop a modified, regulatorily

- compliant SASS, with TSI on particle counters, and with MetOne on a

black car bon sampler. Dedicating time to working with manufacturers on the front end increases

the likelihood that the products, which typically take years to develop, will meet CARB’s particular needs when put on the market.

References:

Suggested Actions to Address Finding :
NAAuditor: Mathew Pla te

Final Report

- August 2016

52 Finding # FO

– Positive Finding 5 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

A ll regulatory PM 2.5 BAM instruments within the PQAO were being converted to VSCCs

.

Description:

A ll regulatory PM 2.5 instruments operating with well impactor ninety

- six (WINS) impactors were being replaced within the entire PQAO. Replacing all WINS with very sharp cut cyclones (VSCC) will promote standardization and minimize potent ial bias in the system.

References:

Suggested Actions to Address Finding :
NAAuditor: Mathew Plate

Final Report

- August 2016

53 Finding # FO 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

Some field forms contained errors that were not properly identified or corrected in the initial form validation or subsequent peer reviews.

Description:

An example of what was found is highlighted in the BAM calibration form.

2014 version of form:

• The criterion on the fo rm was misinterpreted as 0.02 LPM for each multipoint level. A criterion of 2% is consistent with regulation and guidance.

• There was an incorrect response to failing criteria. The calibrator either used data labeled as “failed” or rechecked until system pa ssed.

• Flow is only evaluated against the Target and not the Display flow from the instrument.Auditor: Mathew Plate

Final Report

- August 2016

54 2015 Form Revision

 Recognizing that 0.02 was not achievable

, the form was changed to a different , but incorrect, criterion of 0.2 LPM.

It w as also noted on these forms that the term “fail,” as used for an as

- is calibration, only indicates that recalibration is needed. However, criteria for potential impacts to pre

- calibration

data should also be established.

References:

40 CFR Part 50 App endix L: 9.1.2 An authoritative flow rate standard shall be used for calibrating or verifying the sampler's flow rate measurement device with an accuracy of ±2 percent.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitor ing Program , EPA

– 454/B

- 13

- 003, May 2013, Appendix DAuditor: Mathew Plate

Final Report

- August 2016

55 Suggested Actions to Address Finding :

Review all field forms including calculations and criteria used and make corrections as needed

.

Institute version control and additional peer review evaluation t o ensure that forms are not

altered.Auditor: Gwen Yoshimura

Final Report

- August 2016

56 Finding # FO 2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

Staff were not required to demonstrate proficiency when assigned new tasks.

Descript ion:

EPA observed several cases where newly assigned instrument operations or calibration tasks were improperly executed, resulting in data loss. Data loss affects data quality. For example,

see issues stated in FO1.

As noted in FO

– Positive Finding 2, Phil Wagner and Fernando Amador implemented an on

the

- job training program with their most recent hire

, which could be expanded and adopted

MLD

- wide.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Expand and formalize on

- the

- job training programs and demonstrations of proficiency.Auditor: Mathew Plate

Final Report

- August 2016

57 Finding # FO 3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Prog ram Area: Field Operations

Finding:

QC check gas is not delivered externally to the instruments at the San Luis Obispo, Mojave Poole, and other O 3

- only sites , as the 400E/T400 IZS O 3 monitor delivers quality control (QC) check gas to the monitor inter nally

.

Description:

CARB was performing O 3 precision checks at the San Luis Obispo, Mojave Poole, and other sites using an API 400E/T400 internal zero/span ( IZS ) monitor option which is wholly internal

to, although independent of

, the monitor’s analyti cal bench

. Because the plumbing is internal, the precision check CARB was performing at these sites did not include any check of the external lines, which could be experiencing scavenging or other issues affecting O 3 detection

. In addition, the O 3 generato r was not calibrated with a standard that was independent from the

instrument

.

This practice is potentially inconsistent with the EPA regulatory requirements for precision

and is not how CARB performs precision checks at multi

- pollutant sites.

Reference s: 40 CFR Part 58 Appendix A Section 3.2.2.4 states “...and as much of the ambient air inlet system as is practicable”.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Transfe r Standards for Calibration of Air Monitoring Analyzers for Ozone, EPA

- 454/B

- 13

- 004, October 2013.

EPA Traceability Protocol for Assay and Certification of Gaseous Calibration Standards

, May 2012. Suggested Actions to Address Finding :

If CARB continues to use IZS systems to perform precision checks, CARB should

demonstrate equivalency of this methodology

, functionally and statistically

, according to

EPA’s Traceability Protocol for Assay and Certification of Gaseous Calibration Standards

such that the me thod is shown to meet EPA’s requirements, and is shown to provide QA equivalent to that at non

- O 3

- only CARB sites.

Perform quarterly calibrations of O 3 generators, as recommended in EPA guidance.Auditor: Mathew Plate

Final Report

- August 2016

58 F inding # FO 4 [Finding FO13 from 2011 TSA ] Agency: Ca lifornia Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

MLD staff were not formally documenting the quality of zero air being used in the program.

Description:

Zero air scrubbers are used in place of certi fied zero air for instrument calibrations. This is a common, acceptable practice. Because zero air is used to generate the zero point and the calibration mixes, it must be treated as a standard. Therefore, zero air scrubber maintenance

and verification mus t be documented.

During the 2015 TSA, MDL North appeared to have different practices than MDL South with respect to documenting the quality of zero air.

Since this was observed during the 2015 TSA, it is included as a finding in this report.

Because this finding was also a finding in the 2011 TSA (Finding FO13), documentation submitted to address this 2015 TSA finding will also address the 2011 TSA finding.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality

Mon itoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Document the quality of zero air on a periodic basis for all of MLD , at least annually or after any maintenance is performed.Auditor: Mathew Plate

Final Report

- August 2016

59 Finding # FO 5 Agency: California Air R esources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

The field SOPs did not reflect many changes made to the program over the past several years.

Description:

Based on interviews with the field technicians and calibr ators, it was clear that the lack of SOP and/or updated SOPs was affecting operations and potentially data quality. In some cases

, the

field technicians had been trained to do tasks ; however

, there were no SOPs or only inadequate

SOPs available to referenc e for some infrequent ly performed operations. EPA noted this was the case specifically for CARBLogger , DMS, and BAM operations.

CARB management did note that there are plans for developing SOPs/ u ser g uides for CARBLogger and DMS.

References:

QA Handbo ok for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Establish a schedule for updating SOPs.

Provide field technicians with work aids until SOPs ar e finalized.Auditor: Dena Vallano

Final Report

- August 2016

60 Finding # FO6 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

Data loss has occurred at some sites due to CARBLogger malfunctions and lack of site operator awareness regardi ng how data can be recovered

.

Description:

CARBLogger malfunctions have led to data loss at the San Luis Obispo and Paso Robles sites. During the TSA, it was observed that there were missing data points due to CARBLogger downtime from malfunctions or p rogram updates on 15 separate days during the 2013

- 2014

data record . This cursory review identified several missing hourly values per affected day at
these two sites

. The missing hourly values were not associated with qualifier codes in AQS (See Finding DM 3). Given the limited extent of the TSA and the large number of sites within

the CARB network, EPA was unable to determine the extent of this issue beyond the

aforementioned sites. While data completeness for the audited sites met requirements, the

instrum ent downtime and associated data loss could lead to vulnerabilities in overall data integrity and defensibility.

Based on the site operator interview, troubleshooting as a result of a malfunction often require d that the operator manually reboot the CARBl ogger. The operator was not aware of a backup system to recover lost data when the CARBLogger went down, which resulted in unrecoverable missing data points

. The operator also indicated during the interview that t hese malfunctions may be a result of runnin g the CARBLogger instruments beyond the expected life cycle and past the time when support from the manufacturer is available.

References:

State of California Air Resources Board, Air Monitoring QA, Vol. 5, Corrective Action Notification SOP ( http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf )

Good Automated Laboratory Practices, Principles and Guidance to Regulations For Ensuring Data Integrity In Automated Laboratory Operations , EPA, Section 2

- 1, August 1995.

Suggested Actions to Address Finding:

Train operators on the relevant SOPs that outline the re commended protocols and ways to

recover data during these events. Implement a formal tracking process for addressing suspect

or mi ssing data, instrument problems, or other problems observed in the field. This process

should include formal documentation so that the event can be reconstructed and reviewed toAuditor: Dena Vallano

Final Report

- August 2016

61 prevent recurrence. If this is the type of issue that could be addressed throu gh a CAN, ensure that station operators are aware that they can issue CANs for these issues.

Establish a nd effectively implement a backup system ready for field deployment in case of

instrument breakdown

. Compile a list of equipment replacement needs in t he event that funds become available (from EPA or other funding sources). Budget for equipment replacement according to the life expectancy of the dataloggers.Auditor: Gwen Yoshimura

Final Report

- August 2016

62 Finding # FO 7 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Ar ea: Field Operations

Finding:

Filter data loss has resulted in several of the CARB

- operated sites missing the 75% data completeness requirement for certain quarters.

Description:

EPA reviewed data from 2013 and 2014 for some of the CARB

- operated s ites running filter

based monitors. The following sites had quarters with less than 75% data completeness:

Yuba City PM 2.5 POC1

– 69% data completeness, 2014 Q3.

- R&P 2025 sequential sampler. Issues with sampler not running/shuttle error resulting in signi ficant data loss.

Yuba City PM 10 POC1

– 73% data completeness, 2013 Q3. ( Note: this monitor has an AQS close date of June 28, 2014.)

- GMW HiVol sampler. Issues with time being out, scheduled but not collected, flow

rate out.

Oildale PM 10 POC2

– 73% data completeness, 2013 Q2. (switching to PM 10 BAM)

- GMW HiVol sampler. Issues with timer malfunctions, filter contamination (dirt), Dickson recorder malfunctions, burned brushes.

Bakersfield

- California Ave PM 10 POC1

– 53% data completeness, 2013 Q1.

- GMW HiVol sampler. Issues with filter leaks, samples scheduled but not collected, timer malfunction/sample time out of limits, filter tear.

PM 2.5 and PM 10 data completeness requirements call for 75% data completeness per quarter. Collecting less than 75% data per quarter could result in several years of invalid design values.

EPA recognizes that CARB is taking steps to replace filter

- based instruments with continuous instruments in many locations within their network. For the remaining filter instruments, meeting data completeness continues to be a challenge.

References:

40 CFR part 50 Appendix K 40 CFR part 50 Appendix N

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air QualityAuditor: Gwen Yoshimura

Final Report

- August 2016

63 Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested A ctions to Address Finding :

C ontinue to retire unreliable filter

- based instruments. For those filter

- based instrument s remaining in the network, review recent year’s data and determine if there are additional
actions that, if taken, could minimize data lo ss.Auditor: Gwen Yoshimura

Final Report

- August 2016

64 Finding # FO 8 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

In one instance, a field technician was not aware of the correct schedule for taking make

- up samples on filter

- based in struments.

Description:

Implementation of SOPs and CARB technical bulletins is necessary to ensure data quality, data defensibility, and compliance with regulatory requirements.

EPA noticed that PM 10 filter samples were collected on an irregular basis at a site, i.e.

, not on the scheduled sample days and not on days that would count as make

- up samples. When asked, the field technician did not know the correct protocol for collecting make

- up samples.

The field technician ’s manager pointed out that there was a printed copy of the technical

bulletin on hand at the site that explains the protocol for collecting make

- up samples. This was

a PM 10 site that had recorded exceedances, and was an important site for making air planning

decisions.

All site operator s interviewed during the TSA were generally aware of the technical bulletins and SOP updates. They said they receive emails when new procedures or requirements are released, and significant changes are discussed at staff meetings.

References:

QA Handbo ok for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 4, 5)

Suggested Actions to Address Finding :

Create a method for verifying that site operators understand the information cont ained in technical bulletins and SOP updates, and determine whether additional training is appropriate. Take steps to support high data capture at critical sites.Auditor: Dena Vallano

Final Report

- August 2016

65 Finding # FO 9 Agency: California Air Resources Board

Date of Audit: Summer 2015 Progr am Area: Field Operations

Finding:

R ecertification of sensor probes wa s not based on the recommended instrument

manufacturer’s date of recalibration.

Description:

The pressure and temperature sensor probes at the Paso Robles and San Luis Obisp o stations missed annual

recertification. CARB maintain ed that the

certification’s effect ive date is upon installation rather

than the recommended instrument manufacturer’s date

of recalibration. For certification and recertification of standards, particul arly i n regard to flow

- rate standards,

it is EPA ’s position that the certification date is the date the instrument is officially certified and not the date placed into service. This includes both new purchases

as well as re

- certifications.

References:

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambient Air Quality Monitoring Program , EPA

- 454/B

- 13

- 003, May 2013

QA Eye, Office of Air Quality Planning and Standards , June 2014

Suggested Actions to Address Finding :

Use instrum ent manufacturer’s date entered on the certificate of calibration to plan for the ne xt
annual certification date

.Auditor: Dena Vallano

Final Report

- August 2016

66 Finding # FO1 0 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

The San L uis Obispo monitoring station did not have an adequate guard rail to assure safety during routine operations.

Description:

The San Luis Obispo monitors are located at the top of a roof ~10m from the ground with no guardrail to assure safety for techni cians servicing roof monitors, meteorological sensors and probes. This is also unsafe for any auditors who must access the roof.

Since the time of the audit, CARB has been working to address safety concerns at this site.

References:

40 CFR 58, Appendi x E

Suggested Actions to Address Finding :

Make corrective action a priority for stations that need guardrails for health and safety of monitoring and auditing staff.

Evaluate the safety of monitoring locations being used across the monitoring network and

install or re

- enforce needed infrastructure.Auditor: Gwen Yo shimura

Final Report

- August 2016

67 Finding # FO

– Recommendation1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Field Operations

Finding:

The freezer used to store filters at Bakersfield

- Califo rnia was not kept secure and did not have

a temperature gauge.

Description:

PM 2.5 filters must be kept below a specific temperature. The site operator had been periodically check ing the temperature of the freezer with a temperature gauge, and the filte rs had temperature strips. However, there was no permanent temperature gauge for the site operator to reference.

Samples should be kept in a secured location to minimize opportunity for contamination or alteration of the sample.

References:

40 CFR 50 Appendix L, Section 10.13

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 8)

Quality Assurance Guidance Document 2.12. Monitoring PM 2.5 in Ambient Air Using Designate d Reference or Class I Equivalent Methods. U.S. EPA, National Exposure Research Laboratory. Research Triangle Park, NC, November 1988 or later edition. (Section 8)

Suggested Actions to Address Finding :

Secure the freezer with a lock and purchase and in stall a temperature gauge.Auditor: Gwen Yoshimura

Final Report

- August 2016

68 Finding # DM

– Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

CARB installed a new data management system.

Description:

Since the 201 1 TSA, CARB completed installation of a new data management system. The new DMS allows for improved data flow from the field, easier data visualization and editing.

References:

Suggested Actions to Address Finding :
NAAuditor: Gwen Yoshimura

Final Report

- August 2016

69 Finding # DM

– Positive Fi nding 2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

CARB was creating a new user guide/SOP for DMS.

Description:

CARB was in the process of creating a new user guide/SOP for DMS, spec ifically for users

within the PQAO . When CARB started using DMS, Sonoma Technology, Inc. provided a user guide, and four training modules. CARB was working to consolidate these materials into a more

user

- friendly guide. This is extremely important given th e importance of DMS in the data

management process.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

70 Finding # DM

– Positive Finding 3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Find ing:

CARB was creating an SOP for CARBLogger.

Description:

In order for data to be successfully collected and reported in the field, it is essential that CARBLogger function properly and be used correctly. To improve field technician’s knowledge

of t his system, CARB was creating an SOP.

References:

Suggested Actions to Address Finding :
NAAuditor: Gwen Yoshimura

Final Report

- August 2016

71 Finding # DM

– Positive Finding 4 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

CARB wa s upgrading DMS so that codes used in DMS match those used in AQS.

Description:

The latest version of DMS did not match DMS codes to AQS codes. CARB was working to upgrade DMS to allow DMS codes to map to AQS codes

. This would improve the transfer of

c odes from DMS to AQS.

References:

Suggested Actions to Address Finding :
NAAuditor: Gwen Yoshimura

Final Report

- August 2016

72 Finding # DM

– Positive Finding 5 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

CARB was creating guida nce on minimum value reporting for PM instruments

.

Description:

In the past, districts within the PQAO had different policies concerning how to report minimum values for PM instruments. In order to address questions and reduce inconsistencies across th e PQAO, CARB was creating a QA bulletin that addresses how to report minimum values from

PM instruments.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

73 Finding # DM

– Posi tive Finding 6 Agency: California Air Resources Board

Date of Audi t: Summer 2015 Program Area: Data Management

Finding:

CARB was working with the districts to ensure that all regulatory PM 10 instruments are reporting data in standard temperature and pressure (STP)

.

Description:

During the 2011 TSA , EPA found t hat some regulatory PM 10 instruments were not reporting in

STP . CARB has addressed this for all CARB

- operated PM 10 sites. CARB began coordinating

with all districts to ensure that their regulatory PM 10 monitors report in STP to be comparable to
the federal NAAQS.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

74 Finding # DM

– Positive Finding 7 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

CARB was creating a data visualizati on tool to aid data review.

Description:

CARB was creating a data visualization tool that will make it easy for reviewers/users to identify potential issues, such as gaps in the data record. The existing data review formats did not
facilitate identifi cation of certain errors, such as missing hours of data. To aid data review,

CARB was creating a tool that will allow for a quick visualization of the data.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

75 Finding # DM1 Agency: California Ai r Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

QA qualifier codes were not entered in instances wh ere critical QC criteria were missed.

Description:

In the two examples given below, the QC checks, typically c onducted nightly, were missed for periods extending more than two weeks. One

- point QC checks and zero/span checks are both listed in Appendix H of the EPA QA Handbook as critical criteria, and are required once every

two weeks.

CARB stated that during th e data validation process, these data were retained based on the nightly calibrations bordering the missing time periods which showed that the O 3 instrument was within acceptable limits.Auditor: Gwen Yoshimura

Final Report

- August 2016

76 1. Mojave

- Poole (AQS ID 06

- 037

- 9033) ozone data, December 19, 2013

– January 13, 2014. No nightly checks.

a) December 2013:Auditor: Gwen Yoshimura

Final Report

- August 2016

77 b) January 2014:

2. Bakersfield

- California Avenue (AQS ID 06

- 029

- 0014) ozone, July 23, 2014

– August 16, 2014. No valid nightly checks. a) July 2014:Auditor: Gwen Yoshimura

Final Report

- August 2016

78 b) August 2014:

References:

40 CFR 58 Appendix A, Section 3.2.1

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Institute measu res to ensure that when critical criteria are missed but data are still determined to be valid, a QA Qualifier code (e.g. “1

– Deviation from a CFR/ Critical Criteria Requirement”) is entered along with the affected data points into AQS.Auditor: Gwen Yoshimura

Final Report

- August 2016

79 Finding # DM2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

Calibrations were not always coded as such in AQS.

Description:

40 CFR 58.16 requires reporting of “all ambient air quality data and assoc iated quality assurance data” to AQS. AQS is the official repository for ambient air data. During the TSA, EPA asked CARB staff about instances where AQS data pulls showed times of extended null codes indicating instrument down times, followed by data poin ts. No calibration codes were entered into AQS in between the instrument down time/malfunction null codes and the valid data points. Because calibration codes were not entered into AQS, EPA questioned why the data were considered valid for upload to AQS.

While there may be instances when a calibration is not necessary, station logs associated with

the situations EPA looked into during the TSA showed that calibrations had been performed but were not reflected in AQS

.

References:

40 CFR 58.16

QA Handbo ok for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 17)

AQS coding manual. Available at http://www.epa.gov/aqs

Suggested Actions to Address Finding :

Implement procedure s to ensure that the appropriate calibration null codes are entered when a calibration is performed, and that use of the appropriate codes is periodically assessed.Auditor: Gwen Yoshimura

Final Report

- August 2016

80 Finding # DM3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

Field technicians were unable to apply codes in CARBLogger and DMS to missing data, resulting in missing non

- coded data in AQS.

Description:

In reviewing AQS, EPA found multiple in stances of missing data without qualifier codes. The station operators indicated that they were unable to apply qualifier codes using the CARBLogger / DMS systems when no data were collected by an instrument. These codes can be applied at the time of AQS u pload ; h owever, this process is not followed consistently. In fact, the person applying codes is physically and operationally removed from staff who are aware of the operational issues. The codes applied cannot be reviewed by the field office staff

and man ager, as they are only found in AQS, not DMS.

References:

40 CFR Part 58.16 Data submittal and archiving requirements.

(a) The state, or where appropriate, local agency, shall report to the Administrator, via AQS all ambient air quality data and assoc iated quality assurance data for SO 2 ; CO; O 3 ; NO 2 ; NO; NOy; NO X

; Pb

- TSP mass concentration; Pb

- PM 10 mass concentration; PM 10 mass concentration; PM 2.5 mass concentration; for filter

- based PM 2.5 FRM/FEM the field blank mass, sampler

- generated average daily t emperature, and sampler

- generated average daily pressure; chemically speciated PM 2.5 mass concentration data; PM 10

- 2.5 mass concentration; meteorological data from NCore and PAMS sites; average daily temperature and average daily pressure for Pb sites if n ot already reported from sampler generated records; and metadata records and information specified by the AQS Data Coding Manual ( http://www.epa.gov/ttn/airs/airsaqs/manuals/manuals.htm ). The state, or where appropriate, local agency, may report site speci fic meteorological measurements generated by onsite equipment (meteorological instruments, or sampler generated) or measurements from the nearest airport reporting ambient pressure and temperature. Such air quality data and information must be submitted di rectly to the AQS via electronic transmission on the specified quarterly schedule described in paragraph (b) of this section.

(b) The specific quarterly reporting periods are January 1

- March 31, April 1

- June 30, July 1

- September 30, and October 1

- December 31. The data and information reported for each reporting period must contain all data and information gathered during the reporting period

, and be received in the AQS within 90 days after the end of the quarterly reporting period. For example, the data fo r the reporting period January 1

- March 31 are due on or before June 30 of that year.Auditor: Gwen Yoshimura

Final Report

- August 2016

81 (c) Air quality data submitted for each reporting period must be edited, validated, and entered into the AQS (within the time limits specified in paragraph (b) of this sec tion) pursuant to appropriate AQS procedures. The procedures for editing and validating data are described in the AQS Data Coding Manual and in each monitoring agency's quality assurance project plan.

Suggested Actions to Address Finding :

Update DMS to accept codes for data that were not collected.Auditor: Dena Vallano

Final Report

- August 2016

82 Finding # DM 4 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Data Management

Finding:

N egative values were inconsistently invalidated for PM 2.5 monitors.

Descri ption:

There was some inconsistency among site operators regarding the appropriate limit to flag and/or invalidate negative data for PM 2.5 monitors. It was CARB’s policy to report continuous particulate matter values to AQS to the a bsolute m inimum s ample v alue or the negative of the

f ederal m ethod d etection l imit as posted on the AQS parameters table, whichever is greater.

Field technicians were following CARB policy to invalidate anything <

- 2 μg/m3 , however this process did not appear to be implemented consistently among operators.

CARB has been working to make sure that minimum values are handled consistently across the PQAO. A QA technical bulletin on AQS Minimum Value Reporting Limits is being developed to provide clarification for the PQAO regardin g the minimum values that should be reported to AQS for data from continuous particulate matter monitors.

References:

40 CFR Part 58.16

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003 , May 2013.

Suggested Actions to Address Finding :

Develop and implement a consistent process throughout the PQAO to determine when negative values can be reported and/or a standardized process to evaluate and correct instruments when excessive negative values are reported.Auditor: Gwen Yoshimura

Final Report

- August 2016

83 Finding # QA

– Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

Since the 2011 TSA, the Quality Management Section has been created and, in coo rdination with

other areas of CARB, has implemented significant changes within the PQAO.

Description:

QMS, an entirely new section, was established within the QMB . This section has been working with other sections of CARB to improve quality and consist ency across the PQAO. Many of the improvements noted within this report are associated with work by QMS.

Unique to QMS are the PQAO liaison positions . Each district within the PQAO is assigned to a specific liaison . CARB has been allocating budget, time a nd personnel to allow the liaisons to

travel to their districts at least once a year. Creation of these positions has provided key improvements to coordination and dialogue between the districts and CARB. While there is room for growth, significant improve ment has been made since the 2011 TSA.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

84 Finding # QA

– Positive Finding 2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

R&R documents have been created between CARB and each of the twenty

- one districts that conduct regulatory monitoring.

Description:

CARB has worked with each of the twenty

- one districts that conduct regulatory monitoring to

create individualized R&R documen ts. These documents support a successful, functional PQAO structure. CARB plans to revisit these documents annually to ensure they continue to reflect current practices and provide adequate clarity.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

85 Finding # QA

– Positive Finding 3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

CARB has created a QA webpage that presents PQAO QA information in one, easy to navigate webpage.

Description:

A QA webpage was created that houses information about the CARB QMB, the PQAO structure and contacts, quality management documents including a quality management document repository, QA activities including links concerning CANs and AQDAs

, a nd services and

resources such as information on the standards laboratory. This webpage assembles a wealth of information in one location, so that districts, site operators, and others may easily find

information they need

.

References:

Suggested Acti ons to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

86 Finding # QA

– Positive Finding 4 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

CARB held three successful PQAO training modules, and is in the process of creating a condensed version of the training to be offered on a repeated basis.

Description:

Since the 2011 TSA, CARB created and held three training sessions for the PQAO

, held in

northern and southern California . Sessions were developed by a diverse group of PQAO members, and attended by PQAO districts as well as the other California PQAOs. CARB received very positive feedback, as well as requests for additional, ongoing trainings.

CARB is in the process of c onverting the three PQAO training modules into one three

- day training . They anticipate holding this training every 18 months. This training provides a forum

for districts and CARB to meet all together, to receive consistent training , and discuss current challenges and successes. EPA congratulates CARB on the quality and success of the previous trainings, and supports their ongoing work on this vital element of PQAO implementation.

References:

Suggested Actions to Address Finding : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

87 Finding # QA

– Positive Finding 5 Agency: California A ir Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

CARB was creating guidance as to how districts should review data once they are entered into AQS.

Description:

CARB has created a data validation SOP and shar ed it with the PQAO. The R&R documents for each district also require quarterly and annual data reviews. To support data review and consistency across the PQAO, CARB was creating a QA bulletin on post

- AQS data review. This bulletin will provide guidance as to what districts should check for when conducting the quarterly and annual reviews.

References:

Suggested Actions to Address Finding : NAAuditor: Dena Vallano

Final Report

- August 2016

88 Finding # QA1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality As surance

Finding:

Improvements to the AQDA and CAN processes are necessary to make them more efficient and effective, and ensure timely resolution of monitoring issues

.

Description:

The CAN process documents issues that may affect data quality, com pleteness, storage, or reporting. The objective of the CAN process is to improve data quality and to ensure compliance with s tate , federal, and local requirements. An AQDA is a request for an investigation of the validity of ambient air quality data for a certain period of time.

Issues EPA observed during the TSA:

 CANs and AQDAs were not addressed according to the timelines in the SOP and PQAO R&R documents

.

 In some instances, t he data quality issues could have been easily resolved without initiating a f

ormal CAN process

.

 T he full implications of these correct ive actions were not understood. For example, see

AV/MD8.

 Corrective actions resulting from an issued CAN were not communicated to the

individual who initiated the CAN. This was the case with a CAN initiated by a CARB station operator.

During EPA’s limited review of CANs and AQDAs, the following issues were observed:

 A CAN was initiated in June 26, 2014 with an expected completion date of July 10, 2014. However, the CAN was not communicated to the l ocal district until August 19, 2014. The CAN did not include an appropriate ID number

.

 A CAN was issued August 4, 2014. As of the date of the audit (summer 2015), no corrective action had been communicated to the initiator, either officially or

unofficial ly.

 AQDA 8186 was issued on December 24, 2013 for the Mojave

- Poole site. In January 2015, the QA group determined that 8186 had not been closed. In closing the AQDA, they determined that certified data in AQS required modification.

 CAN 51 was issued Novem ber 20, 2013 for expired gas cylinders at Mojave Desert AQMD’s Trona

- Athol/Telescope site. The CAN form shows that new cylinders were not ordered until May 15, 2014, and the expired cylinders were not replaced until July
1, 2014. The letter closing out the CAN stated, “The District should track cylinder expiration dates to ensure all instrument calibrations and verifications are completed using current gas standards.” On February 25, 2015 , a CAN was issued for an expired standard at Mojave Desert AQMD’s Lan caster site. No action had been taken toAuditor: Dena Vallano

Final Report

- August 2016

89 address this CAN when EPA visited Mojave Desert AQMD during the TSA on April 15, 2015

.

According to the SOP and PQAO R&R documents, CANs are to be completed within 45 days

from the date of issuance.

During the aud it, EPA found instances whe re CANs and AQDAs were not addressed in a timely fashion, and the full implications of the associated corrective actions were not always

understoo d or communicated

EPA recognizes that CARB is providing an annual summary of CANs. However, this process could be more transparent (e.g. sharing those CANs ahead of the summary , even if not resolved, that could affect multiple agencies ).

EPA acknowledges that CARB has only recently instituted the CAN process widely, and is

still worki ng through implementation issues.

References:

CARB, Air Monitoring Quality Assurance Volume 5, Corrective Action Notification SOP, Appendix AN, October 2014 ( http://www.arb.ca.gov/aaqm/q a/pqao/can/can_sop.pdf )

PQAO Roles and Responsibilities for the CARB and San Luis Obispo County Air Pollution Control District, June 2014 ( http://arb.ca.gov/aaqm/q a/pqao/repository/slo_rolesandresponsibilities.pdf )

CARB Quality Management Plan for Ambient Air Monitoring, Volume 1, Section 9, pp 37

- 38, July 2013 ( http://arb.ca.gov/a aqm/qa/pqao/repository/qmp_final.pdf#page=42 )

Suggested Actions to Address Finding :

Meet the time limits established in the SOP and R&R document for c ommunicating CANs and AQDAs to the local districts

.

Formally track and communicate initiation and c losure of CANs and AQDAs through the CARB QMS/district liaisons to the affected parties and CAN/AQDA initiators.

Enhance transparency and timeliness of the system used to inform all districts wh en significant issues and resolutions have arisen in CANs/AQD As.Auditor: Mathew Plate

Final Report

- August 2016

90 Finding # QA2 Agency: CARB

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

An AQDA that indicated a need for AQS to be changed was not appropriately closed out, and inaccurate AQS codes were entered overriding previous ly entered, more accurate AQS codes.

Description:

In August 2013, AQDAs were issued for the continuous PM 10 and PM 2.5 instruments at the Mojave Poole site (AQS ID 06

- 037

- 9033)

. The station operations staff investigated the problem

and invalidated the a ppropriate data a nd applied the AS flag in DMS. These data were uploaded to AQS and certi fied in 2014 with the AS code. In January 2015

, the QAS evaluated outstanding

PM 2.5 AQDAs and found that the PM 2.5 AQDA from August 2013 for Mojave Poole had not

been closed out and requested that the BJ code be applied to the data in AQS ( QA S was not
aware that an AS code h ad already been loaded to AQS). AQS was updated to BJ for PM 2.5 , with no change to the information in DMS

. The field technician was not consulted be fore this change

was made.

CARB’s June 30, 2015 AQDA SOP does not adequately address how data changes are to be addressed in AQS. Section 4.0 AQDA Data Follow Up In AQS states, “ When data must be invalidated, flagged or corrected, QMB staff will generate an AMP 350 report 45 days after the end of the quarter of the data range affected by the AQDA. Where CARB is not the direct submitter to AQS, the AQS submitter for that local district is responsible for the impacted data. The AQDA data resolutions will be compared to the AMP350 report for accuracy. If a discrepancy is noted or no action was taken by the monitoring organization, QMB will follow up to ensure AQS is properly updated. ”

This does not adequately describe who is responsible for comparing the AQDA data resolution to the AMP350, nor is it clear who is responsible for editing AQS in instances when CARB is the

submitter.

AQS codes are typically entered by the field technician , who knows what is happening at the site. Codes recommended by the field te chnician often differ from those applied by those who

are further removed from the daily operation.

References:

California Air Quality Management District, AQDA SOP. June 30, 2015.Auditor: Mathew Plate

Final Report

- August 2016

91 Suggested Actions to Address Finding :

Revise the AQDA SOP to de scribe:

 W ho is responsible for determining whether a change to AQS is appropriate following an AQDA ;

 W hat that change should be ;

 W ho makes the change;

 H ow data changes will be reflected in DMS.

Ensure that the field technician s are aware when data are bei ng changed, and that those in CARB responsible for data certification/recertification are aware when data are changed.Auditor: Mathew Plate

Final Report

- August 2016

92 Recommendation # QA

- Recommendation1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assu rance

Finding:

A nnual performance audits were not being used to identify significant practices that need ed

correction.

Description:

While recognizing that the CARB performance auditor’s role is not to perform a technical audit
of district staff, t hey do visit the districts more frequently than TSA staff. In fact, CARB has upgraded the performance audit program in recent years to be more comprehensive. EPA supports these changes, while noting that CARB’s auditors did not comment on issues such as

th e significant MDAQMD laboratory / PM sampling issues , and districts that are not following

SOPs

.

References:

Suggested Actions to Address Finding :

Review the CARB performance auditor’s responsibilities, and determine if there is a list of

critical items they could additionally check during their audits.Auditor: Gwen Yoshimura

Final Report

- August 2016

93 Recommendation # QA

– Recommendation2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: Quality Assurance

Finding:

Data validation and reporting responsibi lities were unclear in one instance when a district

utilize d a non

- CARB lab oratory for analysis. This may extend to other such situations as well.

Description:

During the TSA, EPA found a situation where a site operated by a district sent samples for

a nalysis to a laboratory other than CARB ’s

. T he responsibilities for data reporting, review, and

validation were unclear. See finding NCU16.

While the CARB R&R documents address data management to some extent, and CARB’s April

2016 Post

- AQS Data Confirmat ion QA Bulletin states that roles, responsibilities, and communication between the District and laboratory should be clearly outlined and documented in such cases, it was unclear whether the other districts in the same situation had been following these gu idelines

.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Verify that roles, responsibilities, and communication betw een districts and laboratories are

outlined and documented in the few cases where district

- run sites utilize a non

- CARB lab for analysis.Auditor: Gwen Yoshimura

Final Report

- August 2016

94 Finding # SL

– Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Ar ea: Standards Laboratory

Finding:

CARB was working to restructure roles and responsibilities within QMB to improve independence, review, and succession planning within the Standards Laboratory.

Description:

CARB was re

- evaluating roles and respons ibilities within QMB to determine if a different structure could better support independence, review, and succession planning within the Standards Laboratory. The new structure should separate staff who verify and certify the equipment used during CARB aud its from the staff using the equipment , and identify someone to oversee QA activities for the Standards Laboratory.

EPA notes that such changes could address several findings noted in this report.

References:

Suggested Actions to Address Finding : NAAuditor: Mathew P late

Final Report

- August 2016

95 Finding # SL1 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

The Standards Laboratory QA s ystem did not have written policies and procedures, a QA manual or a designated QA O ffi cer.

Description:

The role of the Standards Laboratory in the CARB PQAO is unique. The Standards Laboratory provides an essential service: to ensure consistency of standards used throughout the PQAO. However, as a metrology laboratory, the Standards L aboratory must function independently of the monitoring and oversight functions at CARB.

The primary role of the Standards Laboratory is to provide independent and unbiased certification of standards for the CARB PQAO and other clients. Systems should be in place to

ensure that independent QA systems and practices are implemented.

A documented QA system that addresses the issues of independence, integrity, conflicts of interest, an independent QA Officer, customer feedback, and quality improvement specifi cally for the Standards Laboratory should be developed.

References:

ISO/IEC 17025: 2005; Sections 4.1 & 4.2

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and General Requirements ; NIST; February 2006

Suggested Ac tions to Address Finding :

Develop QA system policies and procedures for the Standards Laboratory.

Identify an independent QA Officer to handle Standards Laboratory issues, reviews, and

corrective actions.

Develop a quality manual for the Standards Labo ratory.Auditor: Mathew P late

Final Report

- August 2016

96 Finding # SL2 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

Corrective action processes for the Standards Laboratory were not consistent with ISO or NIST requirements.

Description:

Standards certification laboratories provide an essential service by providing the baseline against which all measurements taken by their clients monitoring programs are compared. They must be both independent and responsive to their client s’ needs.

When nonconforming calibration work is identified internally, the issue should be immediately corrected and the laboratory QA O fficer should determine if a corrective action is required.

To be consistent with ISO 17025 and NIST 150, all correct ive actions identified internally or by customers must be corrected, evaluated as to the root cause, and monitored independently for effectiveness. Additional internal audits of calibration procedures should be initiated immediately.

If an internal ( i.e.

, CARB PQAO) client’s standards grossly fail as

- is calibration, the information should be reported promptly to the client and the PQAO quality management group for potential action. However, the Standards Laboratory must retain independence from

evaluation and tracking of any resulting corrective actions.

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and General Requirements ; NIST; February 2006

Suggested Actions to Address Finding :

Ev aluate the corrective action processes used for the Standards Laboratory and ensure that these are consistent with ISO and NIST requirements for complete, prompt, and independent

response.Auditor: Mathew P late

Final Report

- August 2016

97 Finding # SL3 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

Calibrations/Certifications conducted by the Standards Laboratory were not reviewed by peers/management/QA in sufficient detail to prevent errors.

Description:

The purpose of the S tandard Laboratory peer review system is for technicians to review the

work of their colleagues ; however, t here were no specific criteria for these reviews.

A ll calibrations should be independently verified and validated prior to being delivered to

clien ts. This should include verification of all data that were manually entered into data systems, evaluation of all calibration results, spot checks of supporting documents and calibration traceability, and spot checks of calculations to verify that errors we re not introduced into spreadsheets.

The auditor reviewed documentation for a few calibrations and noted several peer review discrepancies. Two examples are presented here :

1) Peer Review not Documented and/or PerformedAuditor: Mathew P late

Final Report

- August 2016

98 2) Change in Slope as the Resu lt of a Potential Typo Missed (two photos)Auditor: Mathew P late

Final Report

- August 2016

99 References:

QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring Program , EPA

- 454/B

- 13

- 003, May 2013

Suggested Actions to Address Finding :

Implement a more comple te second level/peer review.

Implement additional levels of review that include periodic technical evaluations. This could be done by a QA Officer and/or a manager.Auditor: Mathew P late

Final Report

- August 2016

100 Finding # SL4 Agency: California Air Resources Board

Date of Audit: June 4, 201 5 Program Area: Standards Laboratory

Finding:

Forms to document standard certification were not completely validated or revalidated on a

routine basis.

Description:

The electronic forms used to document standard certifications should:

 Be initiall y validated to ensure they are correct for a range of inputs (validations must be documented).

 Be re

- validated and assigned a new form version if any changes were made

 Be periodically spot

- checked for errors.

 Include documentation for all aspects of each c ertification.

 Be evaluated for linearity using established criteria.

 Clearly identify which “cells” contain manually entered data requiring verification.

If forms have not been fully validated, errors may be present in existing calibrations.

The auditor focused on the low volume and high volume flow calibration forms. The following errors were noted, which may be representative of errors throughout the program.Auditor: Mathew P late

Final Report

- August 2016

101 High Volume Form Example

Low Volume Form Example

References:

ISO/I EC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and General Requirements ; NIST; February 2006Auditor: Mathew P late

Final Report

- August 2016

102 5.4.7.2 When computers or automated equipment are used for the acquisition, processing, recording, reporting, storag e or retrieval of test or calibration data, the laboratory shall

ensure that: a) computer software developed by the user is documented in sufficient detail and is suitably validated as being adequate for use;

Suggested Actions to Address Finding :

All f orms should be validated and a locked copy of the validated forms should be saved.

Forms should be routinely evaluated.

Any form errors should be investigated and corrective action should be taken where appropriate.Auditor: Mathew P late

Final Report

- August 2016

103 Finding # SL5 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

Calibration laboratory staff had not attend ed routine trainings or conferences that are specific

to their field of work.

Description:

Standards Laborat ory staff conduct operations that are unique within CARB. With the exception of the O 3 standard reference photometer program, the calibrations completed are performed without significant external support or collaboration with other standards

laboratories.

Standards Laboratory operations, critical to PQAO data collection, are performed by a small staff that works separately from the rest of the organization. Training and interactions with

other calibration laboratories are critical for staff to be able to i mplement the program effectively.

There are available trainings and conferences, including an annual measurement conference in California. There also may be opportunities to work more collaboratively with the EPA protocol testing laboratories and State of California Metrology Laboratory within the California Department of Food and Agriculture (CDFA).

References:

ISO/IEC 17025

NIST Handbook 150, National Voluntary Laboratory Accreditation Program, Procedures and General Requirements ; NIST; February 200 6

QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring Program , EPA

- 454/B

- 13

- 003, May 2013

Suggested Actions to Address Finding :

Develop a training and collaboration plan for Standards Laboratory staff.Auditor: Mathew P late

Final Report

- August 2016

104 Fin ding # SL6 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

The main Standards L aboratory database and several data systems had not been updated recently and did not meet expectations o f performance and/or documentation and compromise data defensibility

.

Description:

Based on a cursory review of the database, errors were found. It was also not possible to track calibrator information during the audit visit due to the lack of a functi onal search or query

function in the main tracking database . A database that does not allow this type of information to be pulled, i.e. does not provide adequate documentation, compromises data defensibility. The single database reviewed during the TSA had missing information. For example, see highlighted cells below:Auditor: Mathew P late

Final Report

- August 2016

105 In order to be defensible, d atabases and data systems should:

 Be validated and conform to standards of programing.

 Transfer data digitally whenever possible to maintain data integrity.

 No t be prone to malfunctions, including locking

- up and/or crashing.

 Document all aspects of the calibration process available.

 Be secure with limited and tracked access.

 Track all changes made to data.

T he systems in use should be improved to comply with th ese requirements.

When using data systems developed by outside parties (including EPA/NIST), CARB should work with these parties to develop data system improvements that ensure data integrity.

References:

ISO/IEC 17025

NIST Handbook 150, National Vo luntary Laboratory Accreditation Program, Procedures and General Requirements ; NIST; February 2006

5.4.7.2 When computers or automated equipment are used for the acquisition, processing, recording, reporting, storage or retrieval of test or calibration da ta, the laboratory shall

ensure that: a) computer software developed by the user is documented in sufficient detail and is suitably validated as being adequate for use; b) procedures are established and implemented for protecting the data; such procedures shall include, but not be limited to, integrity and confidentiality of data entry or collection, data storage, data transmission and data processing; c) computers and automated equipment are maintained to ensure proper functioning and are provided with the environmental and operating conditions necessary to maintain the integrity of test and calibration data.

Suggested Actions to Address Finding :

Develop an inventory and assessment of each database and data system used by the Standards Laboratory.

Work with EPA to develop a schedule to update the Standards Laboratory data systems and

databases.Auditor: Mathew P late

Final Report

- August 2016

106 Finding # SL7 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding:

Zero air was not being treated as a standard and the quality of zero air wa s not formally documented by the Standards Laboratory.

Description:

Zero air scrubbers were being used in place of certified zero air for standard certifications. Because zero air is used to generate the zero point and the calibration mixes, it must be treated as a standard. As such, zero air scrubber maintenance and verification must be documented.

References:

EPA Traceability Protocol for Assay and Certification of Gaseous Calibration Standards

, September 1997, (EPA

- 600/R

- 97/121), Section 2.1.9, Zero Gas

Suggested Actions to Address Finding :

Institute procedures for certifying and documenting the quality of zero air.Auditor: Mathew P late

Final Report

- August 2016

107 Finding # SL 8 Agency: California Air Resources Board

Date of Audit: J une 4, 2015 Program Area: Standards Laboratory

Finding:

The materials used for plumbing and mixing for SO 2 certifications were stainless steel. These

are not appropriate materials for SO 2 certifications.

Description:

Sulfur dioxide exhibits a sl ow response when being delivered and mixed in stainless steel materials. This could potentially result in a small systematic bias and/or imprecision when certifying standards.

As stated in 40 CFR Part 58 Appendix E, Section 9(a): (a) For the reactive gase s, SO 2 , NO 2 , and O 3 , special probe material must be used for point analyzers. Studies 20

- 24 have been conducted to determine the suitability of materials such as polypropylene, polyethylene, polyvinyl chloride, Tygon ® , aluminum, brass, stainless steel, copp er, Pyrex ® glass and Teflon ® for use as intake sampling lines. Of the above materials, only Pyrex ® glass and Teflon ® have been found to be acceptable for use as intake sampling lines for all the reactive gaseous pollutants.
Furthermore, the EPA 25 has speci fied borosilicate glass or FEP Teflon ® as the only acceptable probe materials for delivering test atmospheres in the determination of reference or equivalent methods. Therefore, borosilicate glass, FEP Teflon ® or their equivalent must be the only material in the sampling train (from inlet probe to the back of the analyzer) that can be in contact with the ambient air sample for existing and new

SLAMs.

While the above CFR reference is specifically for probe material used in instruments measuring ambient air, the same principle applies for materials being used in certification

systems.

Stainless steel materials used in the certification system can be either coated with an inert material or replaced with Teflon to improve system performance.

References:

L ee Byers & John W. Davis (1970) Sulfur Dioxide Adsorption and Desorption on Various

Filter Media , Journal of the Air Pollution Control Association, 20:4, 236

- 238

Avraham Teitz (EPA Region 2) and Thien Bui (EPA Region 7), March 25, 2010, Standard Operating Procedures for the Verification of NO/NOx, SO2, and CO Concentrations in EPAAuditor: Mathew P late

Final Report

- August 2016

108 Protocol Gas Mixtures , Section 11.0 Apparatus and Materials

40 CFR Part 58 Appendix E, Section 9(a)

Suggested Actions to Address Finding :

Eliminate the use of uncoated stai nless steel surfaces in the SO 2 standard certification system.Auditor: Mathew P late

Final Report

- August 2016

109 Recommendation # SL

– Recommendation1 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: Standards Laboratory

Finding :

Contracting issues affected th e ability of the Standards Laboratory to certify standards and delayed required primary standard certifications.

Description:

CARB’s contracting requirements have limited its ability to contract directly with their low flow certification standard suppl ier and have delayed recalibration of the associated devices.

This and similar contracting issues have the potential to limit the Standards Laboratory’s ability to provide quality and continuous services in the future. Because the Standards Laborator y provides a critical role in the PQAO, CARB should work to avoid delays or service denials due to contracting.

As explained in subsequent discussions with CARB, the specific situation that prompted this finding was addressed around the time of the TSA. D ue to changes in CARB’s contracting language (language over which MLD does not have control) , the vendor who certifies the flow devices using proprietary software did not sign the new contract. While this did prevent timely certification of those particula r flow devices, CARB recognized the issue and started discussing the possibility of borrowing devices from other PQAOs. Before implementing this back

- up plan, CARB identified a secondary device that was certified, and used this device in lieu of the expire d devices. No data were compromised as a result.

References:

Suggested Actions to Address Finding :

CARB should maintain a level of attention and ability to quickly respond to contracting issues as they arise, and work to prevent them when ever pos sible

.Auditor: Gwen Yoshimura

Final Report

- August 2016

110 Finding # PM

– Positive Finding 1 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: PM Laboratory

Finding:

CARB was creating a laboratory support section.

Description:

CARB was in the process of creat ing a laboratory support section. This section will consolidate laboratory QA, sample handling, and database management and improve independence and

review.

References:

Suggested Actions to Address Finding :
NAAuditor: Mathew Plate

Final Report

- August 2016

111 Finding # PM

– Positive Finding 2 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: PM Laboratory

Finding:

CARB’s PM laboratories were found to be well maintained and well run.

Description:

Both of the PM laboratories were well maintained, neat, and well organized. Generally, the PM 10 and PM 2.5 laboratory measurements were being performed with a high level of technical expertise.

References:

Suggested Actions to Address Finding :
NAAuditor: Gwen Yoshimura

Final Report

- August 2016

112 Finding # PM

– Positive Finding 3 Agency: California Air Resources Board

Date of Audit: Summer 2015 Program Area: PM Laboratory

Finding:

CARB was creating a system to ensure consistent documentation when a filter does not run.

Description:

In some instances, CARB sends a filter out to a station but t he filter never collects any useable data (e.g. the instrument’s filter mechanism jams and the filter does not run). The docum entation process CARB had in place did not specify how this situation was to be documented. CARB was working to establish a system to create consistent documentation when a filter does not run.

References:

Suggested Actions to Address Finding : NAAuditor: Mathew Plate

Final Report

- August 2016

113 Finding # PM1 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

S tatistical information on temperature and humidity compliance with method requirements was

not consistently calculated or tracked in accordance with EPA’s specified methodology

.

Description:

EPA regulation requires s tatistical calculations of temperatu re and relative humidity variability and must be formally documented.

The MLD PM laboratory was using a system to track and graph temperature and humidity with calibrated traceable standards for both weigh rooms. The laboratory analysts evaluate d

temperat ure and humidity prior to each weighing session. However, compliance with the temperature and humidity control ranges were not formally calculated for all weighing sessions.

Calculations should be completed in accordance with the following sections of t he Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Method s , January 2016 ( a vailable at https://www3.epa. gov/ttn/amtic/files/ambient/pm25/qa/m212.pdf ):

 Section 9.4 Logging Systems.

 Section 10.4.1.3 Determining Compliance with Filter Conditioning Specifications

References:

40 CFR Part 50 Appendix J 7.4 Filter Conditioning Environment.

7.4.1 Temperature range: 15 to 30 C.

7.4.2 Temperature control: ±3 C.

7.4.3 Humidity range: 20% to 45% RH.

7.4.4 Humidity control: ±5% RH.

40 CFR Part 50 Appendix L 8.2 Filter conditioning. All sample filters used shall be conditioned immediately before both the pre

- an d post

- sampling weighings as specified below. See reference 2 in section 13.0 of this appendix for additional guidance.Auditor: Mathew Plate

Final Report

- August 2016

114 8.2.1 Mean temperature. 20

- 23° C.

8.2.2 Temperature control. ±2° C over 24 hours.

8.2.3 Mean humidity. Generally, 30

- 40 percent rel ative humidity; however, where it can be shown that the mean ambient relative humidity during sampling is less than 30 percent, conditioning is permissible at a mean relative humidity within ±5 relative humidity percent of the mean ambient relative humidit y during sampling, but not less than 20 percent.

8.2.4 Humidity control. ±5 relative humidity percent over 24 hours.

8.2.5 Conditioning time. Not less than 24 hours.

8.3.3 Filters must be conditioned at the same conditions (humidity within ±5 relative

h umidity percent) before both the pre

- and post

- sampling weighings.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013, Appendix D

Quality Assurance Guidance Document 2.12, Monitor ing PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Method s , January 2016

Suggested Actions to Address Finding :

Update the weigh room tracking system to calculate compliance with the EPA regulatory requirements for temperature and humidity.

Evaluate the temperature and humidity data for all conditioning and weighing sessions in 2013

- 2015 to verify compliance with regulatory requirements.

If data were reported from weighing periods where temperature and/or humidity were out of

con trol (including the 5% RH difference between weigh sessions for PM 2.5 ), consult with EPA as to whether data should be invalidated or flagged

, and what flag should be applied.Auditor: Mathew Plate

Final Report

- August 2016

115 Finding # PM2 Agency: California Air Resources Board

Date of Audit: June 4 , 2015 Program Area: PM Laboratory

Finding:

Preservation and custody requirements were not being met and/or tracked for samples that had

been hand delivered to the MLD laboratory.

Description:

District and MLD field technicians were hand deliver ing samples from sites geographically

close to Sacramento to the MLD PM sample handling room. These samples were not being signed over to a MLD laboratory sample custodian and were not being verified for preservation requirements when they arrive d . There w ere indications that some of the field

operators may not have been chilling samples from sample removal and during transport to

MLD.

Based on these procedures, it was unclear if holding times and custody are properly

maintained. A llowing uncontrolled acce ss to the MLD sample handling room has custody implications for other filters stored there.

At the time of our visit, MLD had recently formed a Laboratory Support Section which includes sample control. There were indications that procedures were in the pr ocess of being put in place to address these issues.

References:

40 CFR Part 50 Appendix L

8.3.6 The post

- sampling conditioning and weighing shall be completed within 240 hours (10 days) after the end of the sample period, unless the filter sample i s maintained at temperatures below the average ambient temperature during sampling (or 4 °C or below for average sampling temperatures less than 4 °C) during the time between retrieval from the sampler and the start of the conditioning , in which case the p eriod shall not exceed 30 days. Reference 2 in section 13.0 of this appendix has additional guidance on transport of cooled filters.

2003 NELAC Standard, EPA/600/R

- 04/003

Section 5.4.12

Suggested Actions to Address Finding :Auditor: Mathew Plate

Final Report

- August 2016

116 Implement a more formal cus tody process for hand delivered samples, including the verification of receipt temperature.

Implement procedures to better control access to sample handling areas.

Evaluate practices used for preservation of hand delivered PM 2.5 samples by field operator s :

 Recommend best practices to staff and agencies hand delivering samples

.

 Evaluate past data to determine if holding time requirements were violated

.Auditor: Mathew Plate

Final Report

- August 2016

117 Finding # PM3 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: P M Laboratory

Finding:

Laboratory blanks for PM 2.5 were not processed in batches consistent with regulatory requirements.

Description:

The regulatory language in Appendix L indic ates that blanks filters shall be weighed with each

set of field filte rs. This language also implies that these blanks should be reweighed with their corresponding field sample filters.

The MLD laboratory was not initiating new laboratory blanks with each set of filters as required by regulation. Instead , they were using mu ltiple weighing for laboratory blanks to represent multiple sets of filters. This practice does provide sufficient laboratory blank control

but does not link “new” blanks with new sets of filters.

Because the MLD sends filters to many different clients that are received and post

- weighed at

different times

, EPA understands that it may be impractical to post

- weigh each filter with the corresponding laboratory blank. However, CARB may be able to develop a system to simulate this practice. For example, post

- weigh each laboratory blank with a field batch approximately 30 days after pre weigh or another time period that corresponds with the average pre

- and post

weigh separation time for MLD.

References:

40 CFR Appendix L 8.3.7.2 New laboratory blank filter s shall be weighed along with the pre

- sampling (tare) weighing of each set of PM 2.5 filters. These laboratory blank filters should remain in the laboratory in protective containers during the field sampling and should be reweighed as a quality control chec k.

Suggested Actions to Address Finding :

Change procedures for implementing laboratory blanks to be more consistent with regulation.Auditor: Mathew Plate

Final Report

- August 2016

118 Finding # PM4 Agency: U.S. EPA

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

PM 2.5 MTL filter lots and the associated lot controls could not be properly evaluated and were not evaluated at a sufficient frequency.

Description:

The MTL filters provided by EPA do not have lot numbers or manufactured dates associated

with each box of filter s. Given the number of filters that CARB orders and uses each year, it is possible that these represent multiple manufacturing lots/days. CARB was considering each annual EPA shipment as a filter lot.

Additionally, the MTL filters received by MLD have had a very high rejection rate, some

boxes as high as 50 % . EPA should be taking action to improve the vendor’s product quality.

Should CARB continue to use MTL in to the future, or other filter suppliers that do not identify the filter lots, f

ilter lots and associated lot controls should be evaluated in accordance with EPA reg ulations and guidance documents.

References:

40 CFR Part 50 Appendix L

6.0 Filter for PM 2.5 Sample Collection.

6.9 Filter weight stability

. Filter weight loss shall be less than 20 μ g, as measured in each of the following two tests specified in sections 6.9.1 and 6.9.2 of this appendix. The following conditions apply to both of these tests: Filter weight loss shall be the average difference between the initial and the final filter wei ghts of a random sample of test filters selected from each lot prior to sale. The number of filters tested shall be not less than 0.1 percent of the filters of each manufacturing lot, or 10 filters, whichever is greater. The filters shall be weighed under laboratory conditions and shall have had no air sample passed through them, i.e., filter blanks. Each test procedure must include initial conditioning and weighing, the test, and final conditioning and weighing. Conditioning and weighing shall be in accord ance with sections 8.0 through 8.2 of this appendix and general guidance provided in reference 2 of section 13.0 of this appendix.

8.0 Filter Weighing.

8.3.7.1 New field blank filters shall be weighed along with the pre

- sampling (tare) weighing of each lo t of PM 2.5 filters. These blank filters shall be transported to theAuditor: Mathew Plate

Final Report

- August 2016

119 sampling site, installed in the sampler, retrieved from the sampler without sampling, and reweighed as a quality control check.

8.3.7.2 New laboratory blank filters shall be weighed along with the pre

- sampling (tare) weighing of each set of PM 2.5 filters. These laboratory blank filters should remain in the laboratory in protective containers during the field sampling and should be reweighed as a quality control check.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013, Appendix D

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, Janua ry 2016

Suggested Actions to Address Finding :

EPA should continue to share CARB’s experience with MTL filters with the EPA office in charge of filter contracts in order to resolve MTL filter lot and filter quality issues.

MLD should consider a more fr equent lot blank analysis procedure until information on lots is

available from the manufacturer.Auditor: Mathew Plate

Final Report

- August 2016

120 Finding # PM5 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

The PM 10 laboratory SOPs ha d not been updated in over five years.

Description:

The PM 10 SOP had not been formally updated in over five years. The laboratory analyst did have a version of this SOP with some “interim” changes included. While this is a good practice between revisio ns, the PM 10 “interim” changes are confusing because some were superseded without ever being formally implemented.

Also note, that w hile the PM 2.5 SOP had been recently updated, some of the practices in the

SOP did not reflect what the laboratory was curr ently doing.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013

Suggested Actions to Address Finding :

Update PM 10 laboratory SOP.

Implement a process for keepin g SOPs up

- to

- date.Auditor: Mathew Plate

Final Report

- August 2016

121 Finding # PM6 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

The procedure for processing filters for PM 10 speciation led to data being invalidated unnecessarily in o ne observed instance

.

Description:

In order to meet holding times for filter speciation analysis, PM 10 filters were processed (cut) shortly after weighing. This process was routinely completed prior to PM 10 data review. As a consequence, if the data r eview called for re

- analysis of the filter, CARB was unable to reanalyze the filter for PM 10 mass and the data point was lost. EPA found that multiple PM data points from a critical site (all as part of the same observed instance) were lost as a result

of this practice.

In general, the most critical use for PM 10 filter data from routine SLAMS is comparison to the NAAQS. The quality of data used to compare to the NAAQS should not be compromis ed in order to perform an optional secondary speciation analysis u ntil after filter weights are

validated

.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013, Appendix D.

Suggested Actions to Address Finding :

MLD should ensur e that PM filter weights are valid prior to releasing filters for speciation. If speciation must be performed and the sample review schedule cannot accommodate timely analysis of NAAQS data, CARB should install dedicated speciation samplers at monitoring

sites.Auditor: Mathew Plate

Final Report

- August 2016

122 Finding # PM7 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

Not all databases and electronic forms were tracked and validated , compromising data defensibility

.

Description:

The CARB laboratory has done a good job of developing a flexible database with onsite support and has taken steps to improve system functionality. However, the laboratory was using some electronic forms that were not being tracked and validated. For exampl e, excel files are used to track and calculate the differences between the primary and working weights. This procedure was not described in the relevant SOP. Another example, described in the

PM 10 weigh lab SOP, is the use of excel files for calibrations o f the PM 10 filter extractions.

Data defensibility is at risk when adequate documentation is not available. CARB must continue to develop th eir system to track additional quality control and statistical parameters and to continue to support electronic tra nsfer of data into the system.

A lso, a formal, documented validation of the laboratory database should be conducted to identify and address areas that compromise data defensibility.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

2003 NELAC Standard, EPA/600/R

- 04/003

Section 5.5.4.7.2

Suggested Actions to Address Finding :

E valuate data systems for consisten cy with NELAC Section 5.5.4.7.2, including the

de velopment of a formal, documented validation process.Auditor: Mathew Plate

Final Report

- August 2016

123 Finding # PM8 Agency: California Air Resources Board

Date of Audit: June 4, 2015 Program Area: PM Laboratory

Finding:

MLD weigh room practices did not protect the integrity of samples.

D escription:

Maintaining sample integrity is necessary to support data quality and defensibility. EPA found several practices that could compromise the integrity of the PM samples, including:

 Staff other than weigh room technicians had access to weigh roo ms, including janitorial staff.

 Materials that could generate particulates were stored in the weigh rooms.

 PM 2.5 samples were handled in a manner that could subject samples to particulate

contamination . For example, samples were handled on counter tops i n large batches.

 As noted in PM3, laboratory blanks were not batched with filters, which makes tracking laboratory contamination to specific filters less certain.

References:

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Usi ng Designated Reference of Class I Equivalent Methods

, January 2016

2003 NELAC Standard, EPA/600/R

- 04/003

Suggested Actions to Address Finding :

Review and revise weigh room access, storage, and sample handling procedures.Auditor: Joe Eidelberg

Final Report

- August 2016

124 Finding # Pb1 Agency: C alifornia Air Resources Board

Date of Audit: June 4, 2015 Program Area: Pb Laboratory

Finding:

CARB’s Pb laboratory procedures did not meet EPA requirements.

Description:

EPA found that CARB used a method that was no t approved as comparable to EPA methods.

References:

40 CRF 58, Appendix C, Section 2.8, Modification of Methods by Users

40 CFR 53 Ambient Air Monitoring Reference and Equivalent Methods

Suggested Actions to Address Finding :

EPA understands that CARB has ceased analyzing Pb

- TSP filters for NA AQS comparison. If this is confirmed, no further action is necessary. Should CARB wish to restart analyzing Pb

TSP filters for NAAQS comparison, they will need to conform to all EPA requirements and regulations.Auditor: Mathew Plate

Final Report

- August 2016

125 Finding # AV / MD

- Positive Finding 1 Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

AV/MD AQMD’s monitoring staff were dedicated and maintain sites spread out across a significant geographic area.

Description:

Overall , the two technicians are very dedicated and work hard to maintain a fair number of monitoring sites, some of which are a signification distance from their office.

References:

Suggested Actions to Address Finding :

NAAuditor: Mathew Plate

Final Report

- August 2016

126 Finding # AV / MD 1 ( General ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

The data review and validation process was not complete and/or independent. Numerous errors

were found with data that had been loaded into AQS.

Description:

The A V /MD AQMD (i.e., AVAQMD and MDAQMD) had two technicians dev oted to air monitoring full time and a manager who devote d less than half of his time to air monitoring. At this staffing level, technician s should review and validate one another’s work and all data should undergo a manager review prior to upload to AQS. The data in AQS should also be evaluated for accuracy on a regular basis.

At the time of the audit, a limited review of the continuous A V /MD AQMD data by one staff person and the manager was being performed. This review focuse d on information available in

the AirVision d ata system. There was no third level of review for these data. Supporting information (that is not available in AirVision) was not routinely reviewed. The review of data in the AirVision system focuse d on data where flags have been applied by the data system and/or

notes were inserted by the site operator. The accuracy of data uploaded to AQS was also not

verified.

For filter

- based data, the weigh laboratory technician review ed his own data and upload ed them

in to AQS. There was no second le vel independent review and validation. Elements of these data

could occasionally have been checked by the manager.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013

.

Link to Data Verification and Validation resources: http://www.epa.gov/quality1/vandv.html

Suggested Actions to Address Finding :

Devote the resources necessary to have a complete, consistent, r obust, and independent data review and validation process.

• Formalize the implementation of a data validation procedure that incorporates multiple

levels of independent review on a frequent (i.e. monthly) basis

.

• Assign data validation to staff /management independent of those generating results.Auditor: Mathew Plate

Final Report

- August 2016

127 • Integrate independent QA oversight throughout the data validation process, when possible

.

• Conduct independent data audits to help identify systematic deficiencies with data validation as well as specific data issu es. Develop a procedure to evaluate AQS periodically for erroneous data. Develop data tools ( e.g. , flags, figures, tables) to conduct

effective and efficient data audits.

• Ensure all staff involved in validation are familiar with acceptance criteria, appli cable flags to apply when criteria fail, and have access to the data used in generating the validation report.

• Revise and update data validation SOP and QAPPs to reflect the new multi

- level data validation process , and to include a process to determine wh en negative values can be reported and/or a process to evaluate and correct instruments as soon as possible when excessive negative values are reported. (See AV/MD18)

• Review and correct site/pollutant metadata in AQS. (See AV/MD17)

• Replace the current labo ratory data system. (See AV/MD22)

Evaluate the continuous data that were recently produced (2014/2015) and apply additional

review as needed.

Perform second

- level review and validation for all the filter

- based PM 10 and PM 2.5 data produced

in 2014 and 201 5, and correct data errors in AQS. (See AV/MD15 , AV/MD19 )

• Evaluate laboratory quality control for PM 2.5 and procedures for consistency with EPA regulation, guidance, and standard laboratory practices (such as holding times, and temperature and humidity reg ulatory requirements), and determine the impact of

deficiencies on the 2014

- 2015 data. (See AV/MD20, AV/MD21)

• Where temperature or humidity do not meet regulatory requirements, data must be invalidated; all other data should be flagged at a minimum. (See A V/MD19)

• Evaluate the impacts of transport temperature on the PM 2.5 data from the Lancaster Station and apply flags or invalidate data as appropriate. (See AV/MD12)Auditor: Mathew Plate

Final Report

- August 2016

128 Finding # AV / MD 2 ( General ) Agency: California Air Resources Board

– Antelope Valley an d Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

The number of staff and managers and the amount of their time allocated to the air monitoring

program were inadequate for t he current network and associated activities.

Description:

MDAQMD is responsible for operation and maintenance of seven sites across a sizeable geographic region. It also runs a weigh laboratory and is responsible for data review and

validation for all of its sites. Given the findings EPA notes in this report, current staffing levels and managerial time did not appear to be adequate to effectively support the existing monitoring

program.

References:

Suggested Actions to Address Finding :

Evaluate the monitoring network and associated workload and resources. Determine if workload can be reduced, if additional staff are needed, and/or additional management time is necessary to facilitate a high quality monitoring program.Auditor: Mathew Plate

Final Report

- August 2016

129 Finding # AV / MD3 ( Gener al ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

Documentation, while generally present, was sometimes incom plete, not legally defensible, poorly organized, and not reviewed by a peer or a manager.

Description:

For continuous data, information was generally recorded using field forms and in the AirVision system. The use of field and instrument logbooks was l imited.

For filter

- based data, information was generally recorded in the laboratory database and on sample tracking forms. The use of laboratory and instrument logbooks was limited.

 It could not be verified that peer and/or manager review were conducted f or the forms

evaluated.

 The system for organizing and storing field records did not appear to be consistent and there were opportunities for records to be lost or misplaced.

 Many of the records were written in pencil and some of the corrections were not pr operly marked. These practices compromise the defensibility of the data.

 The records reviewed did not sufficiently document aspects of the program so as to fully track the data generation process. Some activities were not being recorded and

information suc h as instrument serial numbers was not always recorded.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Sections 5 & 6)

2003 NELAC Standard , EPA/600/R

- 04/003, June 2003 (Quality Systems Section), that suggests documentation required for all aspects of ambient air monitoring operations, provides the following guidance: 5.5.5.5 The laboratory shall maintain records of each major item of equipment and its software significant to the environment tests performed. The records shall include at least the following information: a) The identity of the item of equipment and its software. b) The manufacturer’s name, type identification, and serial number or other unique identific ation. c) Checks that equipment complies with the specification (see 5.5.5.2). d) The current location.Auditor: Mathew Plate

Final Report

- August 2016

130 e) The manufacturer’s instructions, if available, or reference to their location. f) Dates, results and copies of reports and certificates of all calibrations, adju stments, acceptance criteria, and the due date of next calibration. g) The maintenance plan, where appropriate, and maintenance carried out to date;

documentation on all routine and non

- routine maintenance activities and reference material verifications. h) Any damage, malfunction, modification or repair to the equipment. i) Condition when received (e.g., new, used, reconditioned), if available.

Suggested Actions to Address Finding :

Develop a system to generate, store, and review records of field and laboratory operations.

A V /MD AQMD may want to consider adopting an electronic record keeping system for these

records

.Auditor: Mathew Plate

Final Report

- August 2016

131 Finding # AV / MD4 ( General ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

The A V /MD AQMD staff, while part of the PQAO, were generally not following CARB SOPs and QA planning documents as stated in their R&R document.

Description:

The A V /MD AQMD ha s agreed that it will follow the CARB PQAO SOPs and QAPPs where applicable and will generate its own if needed. A V /MD AQMD has evaluated its procedures and

was working to identify and create SOPs for activities not covered by CARB. At the time of the

TSA t hey had not implemented CARB SOPs and QA procedures for other aspects of their program. Note that the CARB PQAO needs to develop QAPPs to replace their PM 2.5 QAPP from 1998 and QA Manual from 2007.

For field operations, instead of following specific SOPs, A V/ MD AQMD relie d heavily on its senior field technician to ensure that procedures were adequate and consistent. For the most part

this had been successful. However, several inconsistencies with standard practices were noted. Additionally, not having spe cific procedures in SOPs compromises the defensibility of the data. Relying on one staff person to maintain institutional knowledge could also be problematic in the

future.

For laboratory operations, there was a transition several years ago to a new techn ician. Because of limited training and the lack of SOPs and a QAPP, these operations were compromised.

References:

40 CFR Parts 31 and 35 (grant recipients are required to document their quality systems)

Roles and Responsibilities document, CARB an d A V /MD AQMDs. January 24, 2014. http://arb.ca.gov/aaqm/qa/pqao/repository/rr_docs.htm

EPA Requirements for QA Project Plans (QA/R

- 5), EPA/240/B

- 01/003, March 2001.

40 CFR 58 Appendix A, Section 2.1, Quality Management Plans and Quality Assurance Project

Plans

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.Auditor: Mathew Plate

Final Report

- August 2016

132 Suggested Actions to Address Finding :

Adopt or develop SOPs and a QAPP for the PM laboratory as soon as is practical. These should:

• Evaluate laboratory quality control for PM 2.5 and institute procedures that are consistent with EPA regulation, guidance, and standard laboratory practices. (See AV/MD20)

• Include a system to consistently and completely evaluate temperature and humidity. All temperature and humidity data should be evaluated by a qualified technician during second level data review. (See AV/MD19)

• Develop a system to ensure that filters are u sed and weighed within established holding times. (See AV/MD21)

• Improve the PM filter sample custody and handling procedures. (See AV/MD23)

• Implement appropriate improvements to the weigh room procedures and conditions. (See AV/MD24)

• Include replacement of the current laboratory data system. (See AV/MD22)

Adopt or develop field SOPs. If CARB SOPs are adopted, the primary technician should help to develop a list of AV /MD AQMD

- specific procedural differences. These should include procedures to:

• Inspect sampl ing inlets during each routine site visit. (See AV/MD14)

• Transfer the electronic files for each run from the sampler and use as needed in the review and validation process ; describe how electronic files should be saved

. (See AV/MD13)

• Calculate and review r esidence time. (See AV/MD9)

• Eliminate non

- Teflon material from inlets/manifolds of gaseous monitoring sites. (See AV/MD10)

• Develop a periodic process for verifying the quality of zero air. (See AV/MD11)

• Develop a maintenance schedule for zero air scrubbers . (See AV/MD11)

• Document maintenance and verification of zero air. (See AV/MD11)

• Develop a cylinder replacement / re

- certification schedule to prevent the use of cylinders beyond their expiration data. (See AV/MD8)

• Transport PM 2.5 filters at a controlled t emperature. (See AV/MD12)

Once SOPs/QAPPS are adopted, ensure that staff are aware of the documents, and have been

trained on their use.Auditor: Mathew Plate

Final Report

- August 2016

133 Finding # AV / MD5 ( General ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Dat e of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

The A V /MD AQMD training program was not robust enough to ensure that new staff are properly trained, current staff are cross trained, and current sta ff’s knowledge is refreshed and

up

- to

- date.

Description:

A V /MD AQMD staff attended the CARB PQAO training and have begun to work with staff in other California districts to expand their understanding of the program. However, it is unclear

how new A V /MD AQMD staff will be trained in the future.

Additionally, there was inadequate cross

- training, possibly due to inadequate staff levels. If one or more staff person were to leave A V /MD AQMD, procedures may not be consistent and another loss in data integrity may occur. This concern is compound ed by a lack of documented procedures/SOPs.

Currently, additional training on PM 2.5 procedures, station operation, and EPA’s AQS is urgently

needed.

References:

EPA Requirements for QA Project Plans (QA/R

- 5), EPA/240/B

- 01/003, March 2001.

QA Hand book for Air Pollution Measurement Systems, Volume II , EPA

- 454/b

- 08

- 002, May 2013. (Section 4.0)

Suggested Actions to Address Finding :

Develop a training program that includes cross

- training, succession planning, and tracking.Auditor: Mathew Plate

Final Report

- August 2016

134 Finding # AV / MD6 ( Ge neral ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– General

Finding:

The A V /MD AQMD did not rely on the CARB PQAO for indepen dent QA and support.

Description:

C ARB has developed a n R&R document with the A V /MD AQMD. A V /MD AQMD has begun to implement the roles and responsibilities outlined in the document. As noted in several findings, A V /MD AQMD’s implementation of the PQAO d ocuments and processes was incomplete.

It was also noted that A V /MD AQMD was reluctant to ask for support from or initiate corrective actions with the CARB PQAO’s independent QA programs. This was evident in discussion with

A V /MD AQMD staff and from sever al significant QA issues that had not been elevated to the PQAO. A V /MD AQMD’s lack of an independent QA element necessitates reliance on CARB’s quality system to provide independent interpretation and evaluation. This requires that A V /MD AQMD elevate signi ficant issues to the CARB QA Manager for consultation. This can be done by self

- issuing a CAN or by formally contacting the CARB QA Manager or his representatives.

References:

40 CFR Part 58, Appendix A, Section 3.1: Primary Quality Assurance Organiza tion. A primary quality assurance organization is defined as a monitoring organization or a coordinated aggregation of such organizations that is responsible for a set of stations that monitors the same pollutant and for which data quality

assessments can logically be pooled. Each criteria pollutant sampler/monitor at a monitoring station in the SLAMS network must be associated with one, and only one, primary quality assurance organization.

40 CFR Part 58, Appendix A, Section 3.1.1: Each primary quality as surance organization shall be defined such that measurement uncertainty among all stations in the organization can be expected to be reasonably homogeneous, as a result of common factors. Common factors that should be considered by monitoring organizations in defining primary quality assurance organizations include: (a) Operation by a common team of field operators according to a common set of procedures; (b) Use of a common QAPP or standard operating procedures; (c) Common calibration facilities and standa rds;Auditor: Mathew Plate

Final Report

- August 2016

135 (d) Oversight by a common quality assurance organization; and (e) Support by a common management, laboratory or headquarters.

Suggested Actions to Address Finding :

Institute mechanisms by which AV/MD AQMD communicates regularly with CARB

. Among

ot her topics, communications should address:

• Formal notifications to EPA and CARB regarding air monitoring system modifications.

• AQS data corrections, so that metadata are correct and samplers are properly closed. (See AV/MD7)

Develop and implement an inter nal corrective action process and respond to CANs and AQDAs in a timely fashion. (See AV/MD8)Auditor: Mathew Plate

Final Report

- August 2016

136 Finding # AV / MD 7 ( Network Management ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Prog ram Area: Antelope Valley and Mojave Desert AQMDs

– Network Management

Finding:

The transition from filter

- based PM 10 to continuous PM 10 was not properly defined.

Description:

A V /MD AQMD did not notify EPA and CARB of the transition dates from fil ter

- based to

continuous PM 10 for each site. Additionally, information in AQS did not clearly define the

transition. Filter

- based samplers were not closed out, data were not combined, and at sites where both types of instruments were operating simultaneousl y, it is unclear which data set is intended to be primary.

References:

40 CFR Part 58

Suggested Actions to Address Finding :

See AV/MD7 portion of AV/MD6.Auditor: Mathew Plate

Final Report

- August 2016

137 Finding # AV / MD 8 ( Field Operations ) Agency: California Air Resources Board

– Antelope Va lley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Finding:

The span/precision standard at the Lancaster site expired on 8/6/2014.

Description:

The span/precision cylinder at the Lancaster site was expired. This was noted two months earlier by CARB on 2/26/15 (CAN # 159).

Because the standard was expired there is the possibility that the concentrations being measured are not accurate. This has significant implicati ons for the defensibility of the data.

References:

40 CFR Part 58 Appendix A, Section 2.6

Suggested Actions to Address Finding :

Replace the expired cylinder.

Re

- certify the expired cylinder to demonstrate that the data collected after 8/6/2014 a re accurate.

See AV/MD8 portion of AV/MD4 and AV/MD6.Auditor: Mathew Plate

Final Report

- August 2016

138 Finding # AV / MD 9 ( Field Operations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Dese rt AQMDs

– Field Operations

Finding:

A V /MD AQMD staff were not tracking residence time.

Description:

A V /MD AQMD relie d on residence time calculations conducted by CARB auditors. These residence times only represent station conditions during audit s
, which occur once per year. In addition, EPA noted that

CARB did not include the volume of

“water/debris” bottle in residence time

calculations at the Lancaster Station.

Without the bottle, the Lancaster

residence time is 16

- 17 seconds when all

three in strument are operating. If one

instrument is taken off line

, the residence

time for the site is over the maximum specified residence time of 20 seconds.

References:

40 CFR Part 58, Appendix E Section 9

Suggested Actions to Address Finding :

See AV /MD9 portion of AV/MD4.

If sites with multiple instruments have residence times over 10 seconds, add an additional pump to the manifold to ensure that residence time can be met when instruments are temporarily out of

service.Auditor: Mathew Plate

Final Report

- August 2016

139 Finding # AV/MD10 ( Field O perations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Finding:

A V /MD AQMD staff were not evaluating manifo ld material requirements.

Description:

EPA noted that many of the seals used on the glass manifold were not Teflon as required by regulation.

References:

40 CFR Part 58, Appendix E Section 9

Suggested Actions to Address Finding :

See AV/MD10 po rtion of AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

140 Finding # AV / MD 1 1 ( Field Operations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Findi ng:

Zero air sources were not considered standards and maintained as recommended by the

manufacturer.

Description:

Zero air scrubbers were used in place of certified zero air for instrument calibrations. This is a common practice and is acceptable. B ecause zero air is used to generate the zero point and the calibration mixes, it must be treated as a standard. As such, zero air scrubber maintenance and

verification must be documented.

The zero air source observed at the Lancaster site had not been ser viced in several years. T here was no documentation as to the quality of the zero air or exactly when it had last been serviced. The instrument manual indicates that the NOx scrubber should be changed out every six months.

The data evaluation also indicated that negative values were commonly seen during some periods of time. Negative values could be a sign of poor zero air quality during calibration and/or zero adjustments.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Ai r Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

40 CFR Part 50, Appendix F

1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the

NO/NO X /NO 2 analyzer or which might react with either NO, O 3 , or NO 2 in the gas phase titration.

Suggested Actions to Address Finding :

See AV/MD11 portion of AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

141 Finding # AV / MD 12 ( Field Operations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Finding:

PM 2.5 filters from the Lancaster site were not being transported at a controlled temperature back

to the office.

Description:

EPA regulation requires that samples be transported a nd stored at temperatures less than 25°C

for a 240

- hour filter holding time and less than 4°C or the average ambient sampling temperature (whichever is higher) for a 30

- day holding time. PM 2.5 samples from the Lancaster site were not transported at a contr olled temperature.

If samples were transported at 25°C or less, a 10

- day holding time can be applied to these samples.

References:

40 CFR Part 50, Appendix L, Section 8.3.6 & 10.13

8.3.6 The post

- sampling conditioning and weighing shall be completed within 240 hours (10 days) after the end of the sample period, unless the filter sample is maintained at temperatures below the average ambient temperature during sampling (or 4 °C or below for average sampling temperatures less than 4 °C) during the time between retrieval from the sampler and the start of the conditioning, in which case the period shall not exceed 30 days. Reference 2 in section 13.0 of this appendix has additional guidance on transport of cooled filters.

10.13 . . . During the period be tween filter retrieval from the sampler and the start of the conditioning, the filter shall be maintained as cool as practical and continuously protected from exposure to temperatures over 25 °C to protect the integrity of the sample and minimize loss of v olatile components during transport and storage . . .

Suggested Actions to Address Finding :

See AV/MD12 portions of AV/MD1 and AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

142 Finding # AV / MD 13 ( Field Operations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Finding:

The PM 2.5 instrument files were not being downloaded from the samplers.

Description:

Each PM 2.5 instrument records operational parameters during each sampling run. This information is used to demonstrate that instrument runs are valid and can be used to diagnose instrument or sample problems. A V /MD AQMD was only recording summary information from

the PM 2.5 instruments on the samp ling sheets.

References:

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, January 2016

Suggested Actions to Address Finding :

See AV/MD13 portion of AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

143 Find ing # AV / MD 14 ( Field Operations ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Field Operations

Finding:

The plug was missing from the rain cap at the Phelan Station and the Teflon O 3 inlet tube was resting on the plastic funnel.

Description:

The gaseous sampler inlet should be protected from rain.

Because the inlet line is resting on plastic, plastic is essentially in the O 3 sampler flow path.

References:

40 CFR Part 58, Appendix E Section 9 Suggested Actions to Address Finding :

Repair and re

- orient the rain cap at the Phelan Station.

See AV/MD14 portion of AV/MD4

.Auditor: Mathew Plate

Final Report

- August 2016

144 Finding # AV / MD 15 ( Data Management ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Data Management

Finding:

Based on a limited review, EPA found a significant number of dat a errors in AQS.

Description:

The following types of errors were found in AQS:

 Values were reported that were actually the result of instrument maintenance issues

.

 Values were included that were in error by an order of magnitude

.

 Improper flags

.

 Data w ere mistakenly invalidated instead of being flagged as exceptional events

.

 Data were missing without flags being applied, particularly for filter

- based data

.

 Repeating data from an improper AQS upload.

Most of these issues were found during a limited rev iew of data in AQS for several monitoring

stations. As A V /MD AQMD does not routinely evaluate AQS for potential errors, these would not necessarily have been found.

References:

40 CFR Part 58.16

Suggested Actions to Address Finding :

See AV/MD15 por tion of AV/MD1.Auditor: Mathew Plate

Final Report

- August 2016

145 Finding # AV / MD 1 6 ( Data Management ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Data Management

Finding:

AV/MD AQMD’s system did not reliably or consistently back up all data.

Description:

In order to preserve data integrity, all data systems should have a reliable backup.

A V /MD AQMD was back ing up most of the data to a central server, which has onsite backups. However, EPA noted that data from the laboratory data system had not been backed up to the server recently (this was an oversight that was corrected immediately). Other records and forms on satellite computers and laptops may not have been routin ely backed up to the A V /MD AQMD

server.

While the server was routinely backed up, there was no offsite backup of this system. Because

A V /MD AQMD is centralized into one building, an incident at the building could cause a temporary or permanent loss of pri mary and backup records.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

Review all records stored on remote systems and develop an inventory.

Develop a schedule for backups to the server and a process to verify that backups are occurring.

Consider having an offsite backup of the AV /MD AQMD server.Auditor: Mathew Plate

Final Report

- August 2016

146 Finding # AV / MD 1 7 ( Data Management ) Agency: California Air Resour ces Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Data Management

Finding:

Some of the metadata in AQS were missing or incorrect.

Description:

EPA found tha t site scale, a parameter that should be included in AQS, was not included for several pollutants at several sites.

Probe height was often incorrect in AQS.

References:

40 CFR Part 58.16

Suggested Actions to Address Finding :

See AV/MD17 portion of AV/MD1.Auditor: Mathew Plate

Final Report

- August 2016

147 Finding # AV / MD 1 8 ( Data Management ) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– Data Management

Finding:

The d ata record was incomplete for several pollutants as negative values are routinely

invalidated.

Description:

A V /MD AQMD policy was that all negative values reported by continuous instruments should be invalidated. On some occasions, instruments had cons istently reported negative values over a period of time and/or during certain hours of the day. As a result, the data record was not complete.

References:

40 CFR Part 58.16

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

See AV/MD18 portion of AV/MD1.Auditor: Mathew Plate

Final Report

- August 2016

148 Finding # AV / MD 1 9 (PM Laboratory) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audi t: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– PM Laboratory

Finding:

Temperature and humidity issues in the PM weigh room compromised the quality of data

collected in 2014 and possibly earlier . Temperature and humidity we re not tracked

, so it was not easy to identify out

- of

- control situations.

Description:

A V /MD AQMD indicated that the previous weigh room temperature and humidity control system was malfunctioning, causing multiple spike and dips in temperature and humi dity each hour and significant drift each day. In late 2014 a new control system was installed and these problems appeared to be resolved.

Although A V /MD AQMD identified this problem, no data were flagged or invalidated and no system was instituted to det ermine what levels of drift and fluctuation were acceptable.

Based on EPA’s evaluation of some of the temperature and humidity data, most or all of the

affected PM 2.5 data and some of the PM 10 data from 2014 may need to be invalidated because regulatory r equirements were not met.

According to PM 2.5 monitoring regulations, temperature must be between 20

- 23 o C and controlled

within 2 o C every 24 hours. Humidity must be between 30 and 40 percent RH, control within 5%

RH over 24 hours and not more than 5% RH d ifferent between pre

- and post

- weigh. Temperature variability, humidity variability, and % RH difference between pre

- and post

- weigh are not evaluated by A V /MD AQMD.

References:

40 CFR Part 50 Appendix L, Section 8.0

Quality Assurance Guidance Docume nt 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, January 2016

Suggested Actions to Address Finding :

See AV/MD19 portion of AV/MD1 and AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

149 Finding # AV / MD 20 (PM Laboratory) Agency: California Ai r Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– PM Laboratory

Finding:

The quality control activities utilized for PM 2.5 filter weighing were insuffic ient and inconsistent with EPA regulation, guidance, and general laboratory principles.

Description:

New laboratory blanks were not initiated with each new batch of filters as required by 40 CFR Part 50 Appendix L, and there w as an in sufficient number of new blank filters to assign one for each post

- weigh batch as recommended by EPA Method 2.12. However, A V /MD AQMD was selecting blank filters periodically and reweighing them routinely.

Samples were not routinely reweighed as laboratory duplicates as is recommended by Method 2.12 (one per weigh session) and standard laboratory practices (5

- 10% duplicates). A V /MD AQMD did record duplicate readings from 50% of field samples.

Lot blanks were not evaluated as required by Method 2.12. Without the evaluation of lot blanks,

it was possible that the equilibration time of 24 hours for pre

- weigh was insufficient to yield a

consistent filter mass.

The balance was not evaluated each day with masses that bracket the filter mass. A V /MD AQMD only use d one mass, 200m g, to verify the balance. Additionally the mass selected was significantly less than the filter masses

, which are close to 400 mg.

References: 40 CFR Part 50 Appendix L

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using

Des ignated Reference of Class I Equivalent Methods

, January 2016

2003 NELAC Standard , EPA/600/R

- 04/003, June 2003 D.5.2 Analytical Variability/Reproducibility Matrix Spike Duplicates (MSDs) or Laboratory Duplicates

– Shall be analyzed at a minimum of

1 in 2 0 samples per sample batch... Suggested Actions to Address Finding :

See AV/MD20 portions of AV/MD1 and AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

150 Finding # AV / MD 21 (PM Laboratory) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17 , 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– PM Laboratory

Finding:

H olding times were not being met for all PM filters.

Description:

Based on data that EPA tracked, filters T1643235 to T1643249 did not appear to meet post

samp ling holding time (there may be a typo on the weigh date as an equilibration time of one

minute rather than 24 hours is entered on the post

- run data entry log). Filters T1643242 to

T1643249 did not meet the pre

- sampling holding time.

M any filters from the Lancaster site may not have met the 10

- day holding time for filters not transported at a reduced temperature.

References:

40 CFR Part 50 Appendix L

8.3.5 The pre

- sampling (tare) weighing shall be within 30 days of the sampling period. 8.3.6 The post

- sampling conditioning and weighing shall be completed within 240 hours (10 days) after the end of the sample period, unless the filter sample is maintained at temperatures below the average ambient temperature during sampling (or 4 °C or below for average sampling temperatures less than 4 °C) during the time between retrieval from the sampler and the start of the conditioning, in which case the period shall not exceed 30 days. Reference 2 in section 13.0 of this appendix has additional guidance on

transport of cooled filters.

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, January 2016

Suggested Actions to Address Finding :

See AV/MD21 portions of AV/MD1 and AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

151 Finding # AV / MD 2 2 (PM Laboratory) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– PM Laboratory

Finding:

The data system used fo r PM weighing was not documented, understood by current staff, flexible, or fully validated.

Description:

A V /MD AQMD was us ing a laboratory data system for PM weighing that was program m ed by former staff. This data system was not flexible enough to acc ommodate needed changes to the weighing process. In order to save time and reduce the number of errors in the weighing process, the data system should be able to receive data directly from the balances, flag data outside of quality control limits, allow fo r flexibility in data sheet outputs and report generation, and recognize balance checks.

There was no documentation as to how the data system was programed and validated, and its programing was not easily understood. Generally all electronic systems shoul d be documented

and validated.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

2003 NELAC Standard, EPA/600/R

- 04/003, Section 5.5.4.7.2

Suggested Actions to Address Finding :

See AV/MD22 portions of AV/MD1 and AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

152 Finding # AV / MD 2 3 (PM Laboratory) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojav e Desert AQMDs

– PM Laboratory

Finding:

Storage and custody of PM filters were not consistent with general sample handling practice, EPA guidance and regulations.

Description:

The following were noted by EPA:

 The sample storage freezer was not sec ure and is used for storage of ice and food

.

 Temperature in the sample storage freezer was not monitored regularly with a calibrated

thermometer

.

 Field sampling/custody forms were not always kept with the filters prior to receipt by the laboratory technici an

.

 Access to t he weigh room was not limited.

 The dates and times that filters were prepared for transfer to the field and received from the field are not adequately tracked and custody was not always formally transferred

between technicians

.

 Filters were routinely stored three to four years

- not five years as specified by EPA regulation

- prior to disposal

.

References:

2003 NELAC Standard , EPA/600/R

- 04/003: Section 5.4.12.2, Technical Records; Section 5.5.8, Handling of Samples

5.5.8.4 The laborato ry shall have procedures and appropriate facilities for avoiding deterioration, contamination, loss or damage to the sample during storage, handling, preparation and testing. Handling instructions provided with the sample shall be followed. When samples have to be stored or conditioned under specified environmental
conditions, these conditions shall be maintained, monitored and recorded. . . . a) Samples shall be stored according to the conditions specified by preservation protocols: . . . 2) Samples sha ll be stored away from all standards, reagents, food and other potentially contaminating sources. Samples shall be stored in suchAuditor: Mathew Plate

Final Report

- August 2016

153

a manner to prevent cross contamination.

40 CFR Part 58.16 (f) Data Submittal and Archiving Requirements

QA Handbook for A ir Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Quality Assurance Guidance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, January 2016

Suggested Actions to Address Finding :

See AV/MD23 portion of AV/MD4.Auditor: Mathew Plate

Final Report

- August 2016

154 Finding # AV / MD 24 (PM Laboratory) Agency: California Air Resources Board

– Antelope Valley and Mojave Desert

AQMDs

Date of Audit: April 17, 2015 Program Area: Antelope Valley and Mojave Desert AQMDs

– PM Laboratory

Finding:

Several aspects of general weigh laboratory procedures were not consistent with EPA guidance.

Description:

The following deficiencies were noted:

 The primary and working weights were not used as re commended in Method 2.12.

 Filters received from EPA were not tracked.

 Cassette cleaning was not sufficient.

 Some materials observed in the weigh room could lead to particulate or static issues.

 The filter inspection process is not sufficiently rigorous

.

 Th e balances were not isolated from vibrations.

 Old c assettes in use may need to be replaced.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Quality Assurance Gu idance Document 2.12, Monitoring PM 2.5 in Ambient Air Using Designated Reference of Class I Equivalent Methods

, January 2016

Suggested Actions to Address Finding :

See AV/MD 24 portion of AV/MD 4.Auditor: Gwen Yoshimura

Final Report

- August 2016

155 Finding # NCU

– Positive Finding 1 Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– General

Finding:

NCUAQMD was implementing a number of good practices in support of site and instrument maintenance, and data managemen t.

Description:

NCUAQMD was implementing a number of good practices. Among others, EPA observed:

 The inside of monitoring shelters appeared clean and generally well kept.

 An inventory list of spare monitoring equipment was kept at monitoring sites.

 Cop ies of SOPs were found at the monitoring sites.

 Continuous instrument data was backed up nightly to NCU servers to help prevent data

loss.

 Original raw ambient data records are kept even if invalidated.

References:

Suggested Actions to Address Findi ng : NAAuditor: Gwen Yoshimura

Final Report

- August 2016

156 Finding # NCU1 (General) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– General

Finding:

The current purpose/objectives of NCUAQMD ’s NAAQS monitoring network were unclear.

Description:

D uring the audit, NCUAQMD staff provided some reasons behind the initiation of monitoring within NCUAQMD (permits, previous APCO directive, etc). Given the turnover in management and age of the permit, it was unclear why some monitors were continuing to be run and what were the current purpose/monitoring objectives for each one

.

Staff did not know the monitor type of the non

- FEM BAM PM 2.5 monitor that has been operating at Weaverville since January 2014. NCUAQMD was u nder impression that after 24 months, all SPM NAAQS parameters must be converted to SLAMS.

NCUAQMD ’s 2014 annual monitoring network plan labeled most monitors as special purpose

, while the 2015 TSA questionnaire forms labeled most monitors SLAMS.

A monit or’s purpose and monitor type affect what regulations apply. Resources are also required to maintain a monitoring network. It is therefore important for both planning and monitoring staff to know what is in their monitoring network, and assess annually whe ther the monitoring being conducted is appropriate.

References:

40 CFR 58, Appendix D

Suggested Actions to Address Finding :

W ork with CARB to define the objectives of the monitoring network and types of monitors

within the network. A lign the networ k to fulfill those objectives, taking resources into account.

Include updated information, including any SLAM site start

- up or shut

- down requests, to EPA

for approval as part of the next Annual Monitoring Network Plan

.Auditor: Gwen Yoshimura

Final Report

- August 2016

157 Finding # NCU2 (General/Quali ty Assurance) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– General/ Quality Assurance

Finding:

QA document gaps were identified.

Description:

SOPs that NCUAQ MD was using during the time of the audit had not been reviewed/approved by CARB , and/or QAPPs were not being followed

.

 NCUAQMD was using its own data management SOP for reviewing and validating NAAQS

- comparable data. This SOP had not been reviewed or app roved by CARB.

 TECO gas and PM 2.5 GRIMM 180 SOPs show as “in

- progress ” on C ARB ’s QA

doc ument online repository. All SOPs should be completed and approved by CARB.

 Some BAAQMD SOPs were present at sites, but there was no indication that NCUAQMD was using t hese according to CARB’s list of NCUAQMD SOPs.

 A records management plan for air monitoring activities was not being followed

.

References:

40 CFR Part 58, Appendix A, S ection 2 (SOPs)

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambi ent Air Quality Monitoring Program, EPA

- 454/B

- 13

- 003, May 2013. (Sections 5 and 17)

Suggested Actions to Address Finding :

Update QAPP/ SOPs to represent current practices and to be consistent with EPA regulation and guidance on a regular schedule or as needed in accordance with 40 CFR 58, Appendix A. Submit QAPP/SOPs to CARB for approval, giving priority to NAAQS

- related documents. This includes but is not limited to the following:

• Adjust action levels to be in conformance with EPA guidance and in consul tation with

CARB

. (See NCU12)

• F ollow EPA guidance

, and NCUAQMD quality document requirements as outlined in the R&R document, regarding appropriate documentation. (See NCU5)

• Use the monthly maintenance sheets as described by CARB’s Quality Assurance Manual

. The check sheets may be found online at http://www.arb.ca.gov/aaqm/qa/pqao/repository/qm_docs.htm

. (See NCU6)

• Replace sampling lines on a schedule that is in conformance with the replacement schedule followed by the CARB PQAO. (See NCU7)Auditor: Gwen Yoshimura

Final Report

- August 2016

158 • Conduct and document regular cleaning of rooftop equipment. (See NCU8)

• Lock the PM filter refrigerator and limit access to personnel handling filters. (See NCU9)

• Data collected with FEM/FRM should be submitted into AQS promptly as required. (See NCU10)

• Adopt a procedure for veri fication of zero air. (See NCU13 )

• Adopt a formalized and consistent corrective action process that can be used to resolve internal matters; involve C ARB when appropriate. (Se e NCU14 )

• Develop a document or process for documenting what QC checks and data validation processes are conducted for filter data by NCUAQMD, CARB, and NCUAQMD’s contract lab. (See NCU16)

• Ensure that NCUAQMD data management software is in conformance with section 14 of the QA Handbook. (See NCU11)

Provide operators with work aids until SOPs are finalized.

D evelop a tracking system to indicate that operators have read and understand the QAPP/ SOP s and have had an opportunity to discuss them with their supe rvisor.

Adopt and implement a records management plan.Auditor: Gwen Yoshimura

Final Report

- August 2016

159 Finding # NCU 3 (General) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– General

Finding:

Training s pecifications/requirements for air monitoring personnel were not well defined and completion was not tracked

.

Description:

NCUAQMD did not provide a training plan for air monitoring personnel. While the roles and responsibilities document between NCUAQ MD and CARB discusses training, it is not an actual training plan. That is, it does not specify for each job position what courses must be taken, what training must be accomplished, and what proficiencies must be demonstrated by what date or with what freq uency.

During the TSA, t he primary field technician noted not being familiar with 40 CFR 58 Appendix A requirements, EPA’s QA Manual , nor zero/precision/span QC checks conducted on gas instruments. The primary field technician did not attend CARB’s PQAO t raining

Modules I & II and was not familiar with CARB’s technical bulletins. NCUAQMD did plan to send the primary site technician to the CARB PQAO Module III training.

Trainings sh ould include review and sign

- off of approved QA documents (e.g. QMP, QAPPs

, SOP s ) with which staff are expected to be familiar, attendance at all PQAO module trainings or review of videos that are posted to the web after the trainings, instrument manufacturer trainings, other CARB or EPA regularly offered trainings, and on

- the

- j ob training.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 4)

Suggested Actions to Address Finding :

S pecify the training requirements and keep a t raining file for staff. Align t raining requirements with CARB PQAO requirements

. This includes but is not limited to reviewing requirements for filling out CoC forms with staff, provide training if necessary and check ing on implementation. (See NCU15)Auditor: Gwen Yoshimura

Final Report

- August 2016

160 Finding # NCU 4 (General) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– General

Finding:

M ajor monitoring purchases have been problematic for NCUAQMD and CARB

.

Description:

With the revision to the monitoring regulations in 2006, NCUAQMD formally became part of the CARB PQAO. Since that time, CARB has had a QA oversight role which necessitates communication and collaboration between districts within the CARB PQ AO and CARB.

Instruments collecting data for comparison to the NAAQS are subject to 40 CFR 58 Appendices A, C, and E. As one aspect of fulfilling its QA oversight role, CARB conducts audits of all sites within California to assist in meeting Appendix A re quirements. In order to conduct audits, CARB must have the appropriate equipment and therefore should be consulted with prior to new instrument purchases. CARB also provides other services such as acceptance testing, calibrations, support for QA collocatio n, and troubleshooting.

NCUAQMD did not consult with CARB on purchases of major equipment , which has led to

concerns a bout meeting PQAO requirements, challenges for CARB to provide support (such as audits and troubleshooting when instrument

- specific issue s arise) , concerns with instrum ent comparability to the NAAQS, and overall difficulty meeting the commitments laid out in the

R&R document.

References:

Suggested Actions to Address Finding :

C onsult and come to agree ment with CARB on major monitori ng equipment purchases so that

CARB can provide support (e.g. acceptance testing, calibrations, QA collocation, etc.), as

outlined in the R&R document.Auditor: Gwen Yoshimura

Final Report

- August 2016

161 Finding # NCU 5 ( Field Operations ) Agency: California Air Resources Board

– North Coast Unifie d AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Field Operations

Finding:

Field documentation gaps and defensibility issues were identified.

Description:

• A station log was kept via a Notepad text file in a computer insi de the monitoring shelter. NCUAQMD staff indicated that notes from instruments within the monitoring shelter were recorded on this log.

o The Notepad text file was live every time it is opened and entries previously made could be altered without leaving a tr ace in the system.

o NCUAQMD staff indicated that logbook entry copies were manually downloaded to maintain some integrity and historical record, however these

back

- ups may have been occurring anywhere from three days to a month after original entry.

• NCUAQMD staff were unable to trace last manifold cleaning conducted at Jacobs. Staff recollection was that the cleaning had been conducted around the last time CARB

auditors visited the site.

Entries made in electronic logbooks should not be easily alterable . It should be clear who made entries on what date, such that any alterations to entries can be seen in a data audit. Logbooks are otherwise not defensible. What is recorded in the hardcopy logbook versus the electronic station logbook should be standardized a nd outlined in an SOP.

References: QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Technical Note

- Use of Electronic Logbooks for Ambient Air Monitoring , EPA, April 20,

201 6. Available at: https://www3.epa.gov/ttnamti1/files/policy/Electronic__Logbook_Final_%204_20_16.pdf

Suggested Actions to Address Finding :

See NCU5 port ion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

162 Finding # NCU 6 ( Field Operations ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Field Operations

Finding:

Monthly maintenance check sheets , as prescribed by CARB’s SOPs, have not been used

.

Description:

According to the February 2015 NCUAQMD /CARB R&R document, NCUAQMD has agreed to follow CARB’s quality documents. Any deviations are to be specified in an addendum and

submitted to CARB for re view and approval. At the time of the audit, no deviations had been approved. CARB’s Quality Assurance Manual, Volume II Standard Operating Procedures for Air Quality Monitoring, section 2.0.1 General Operating Procedures, requires the use of “Monthly Qual ity Control Maintenance Check sheet[s]” to record specific QA/QC checks. Monthly quality control maintenance check sheets are included in CARB’s SOPs for each

instrument. These check sheets were not found at the sites.

Following the onsite audit, NCUAQMD emailed examples of documents used to record weekly checks. One example for gaseous instruments (Humboldt Hill, Sep. 10, 2014

– Nov 4, 2014)

follows:Auditor: Gwen Yoshimura

Final Report

- August 2016

163 The excel spreadsheet used to record PM 2.5 FRM checks at Humboldt Hill:Auditor: Gwen Yoshimura

Final Report

- August 2016

164 These check sheets do not co nsistently contain the information required to be recorded in

CARB’ s SOP monthly maintenance check sheets . If NCUAQMD wishes to continue to use their

own check sheets instead of the monthly maintenance check sheets in CARB’s SOPs, CARB must approve their u sage as part of NCUAQMD’s SOPs or an approved SOP addendum.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

See NCU6 p ortion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

165 Finding # NCU 7 (Field Operations) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Field Operations

Finding:

Sampling lines/tubes were not re gularly replaced.

Description:

NCUAQMD staff noted that sampling lines are generally cleaned but not replaced. Sampling lines should be replaced once per year as they can deteriorate in ambient conditions (e.g. UV exposure) and more frequent ly as indic ated by pollutant load or contamination

.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

40 CFR part 58, Appendix E

Suggested Actions to Address Finding :

S ee NCU7 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

166 Finding # NCU 8 ( Field Operations ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Field Operations

Finding:

Some rooftop equipment was not clean.

Description:

Regular cleaning should be conducted and documented to avoid the build

- up of debris that may

confound ambient data.

a. Gas inlet at Jacobs.Auditor: Gwen Yoshimura

Final Report

- August 2016

167

b. FRM PM 2.5 sampler temperature probe at Jacobs.

References:

QA Handbo ok for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 7, 11)

Suggested Actions to Address Finding :

See NCU8 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

168 Finding # NCU9 ( Field Operations ) Agency: Cal ifornia Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Field Operations

Finding:

The refrigerator containing PM filters that have been collected from the field was not locked.

Desc ription:

The NCUAQMD building is generally restricted to staff entry. However, the refrigerator containing the PM samples has been kept in a common space kitchen accessible to all NCUAQMD staff (including non

- monitoring staff).

The PM filter refrigerato r contained a visible lock. NCU AQMD staff indicated that the key for

this was not available.

Samples should be kept in a secured location to minimize opportunity for contamination or alteration of the sample.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Section 8)

“... Samples must be properly handled to ensure that there is no contamination and that the sample analyzed is actually the sample taken under the co nditions reported. For this reason, whenever samples are not under the direct control of the sample custodian, they should be kept in a secured location. This may be a locked vehicle, locked refrigerator, or locked laboratory with limited access. It is hig hly recommended that all samples be secured until discarded. These security measures should be documented by a written record signed by the handlers of the sample on the COC form or in a laboratory notebook, indicating the storage location and conditions ...”

Suggested Actions to Address Finding :

See NCU9 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

169 Finding # NCU 10 ( Data Management ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Data Manage ment

Finding:

Data collected with FEM/FRM instruments were not uploaded to AQS as required by CFR

.

Description:

As of the date of the TSA, GRIMM data from the Humboldt Hill, Weaverville, and Crescent City stations ha d not been entered into AQS, as discussed in EPA’s letter to NCUAQMD dated December 10, 2014.

At the time of the audit, NCUAQMD noted that Weaverville GRIMM data approved by EPA for NAAQS exclusion were pending submittal into AQS. NCUAQMD noted that other GRIMM

data are being held back pending EPA’s review of an anticipated FEM waiver request for th ese data.

EPA notes that on December 10, 2014, EPA sent NCUAQMD a detailed letter describing how data from the GRIMM, at various monitoring sites, should be considered. While the letter stat ed that EPA approved Weaverville for NAAQS exclusion, this approval was based on 40 CFR 58.20(d) (SPMs operating under 24 months may receive approval to use the NAAQS exclusion designation in AQS). The letter consistently states that data collected, at Wea verville or elsewhere, must be entered into AQS, as required by 40 CFR 58.16

.

CFR and grant requirements apply to these data, and require reporting to AQS.

References:

40 CFR 58.16

December 10, 2014 letter from Meredith Kurpius, U.S. EPA, to Richard L. Martin, NCUAQMD concerning GRIMM data.

Suggested Actions to Address Finding :

See NCU10 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

170 Finding # NCU1 1 ( Data Management ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Prog ram Area: North Coast Unified

– Data Management

Finding:

NCUAQMD had not verified that their data processing software meets performance guidelines consistent with the QA Handbook, Volume II, Section 14.

Description:

During the audit, NCUAQMD staff indicated that they did not know whether their data management software was consistent with the QA Handbook, Volume II, Section 14.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/ B

- 13

- 003, May 2013. (Section 14)

Suggested Actions to Address Finding :

See NCU11 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

171 Finding # NCU12 ( Quality Assurance ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Are a: North Coast Unified

– Quality Assurance

Finding:

The level of d ata quality performance requirement s for gaseous pollutants were less stringent than those in EPA regulation and guidance.

Description:

40 CFR part 58 Appendix A, Section 2.3 sets f orth data quality performance requirements.

 O 3 action level is 7%; NCUAQMD is using 10%.

 CO and SO 2 action level is 10%; NCUAQMD is using 15%.

References:

40 CFR part 58, Appendix A, Section 2.3

Suggested Actions to Address Finding :

See NCU12 port ion of NCU2.

I n consultation with CARB and EPA, determine how to address historical data collected using the higher thresholds should be reported to AQS.

Work collaboratively with C ARB to identify more effective means of communication such that NCUAQMD i s fully aware of requirements.Auditor: Gwen Yoshimura

Final Report

- August 2016

172 Finding # NCU13 ( Quality Assurance ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Quality Assurance

Finding:

NCUAQMD did not have a procedure for checking/verifying zero air.

Description:

Zero air scrubbers are used in place of certified zero air for instrument calibrations. Because zero air is used to generate the zero point and the calibration mixes, it must be treated as a standard. As such, zero air scrubber maintenance and verification must be documented.

NCUAQMD indicated that it would like to participate in CARB’s efforts to implement a zero air verification procedure.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

40 CFR part 50, Appendix F 1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the

NO/NO
x /NO 2 analyzer or which might react w ith either NO, O 3 , or NO 2 in the gas phase

titration.

Suggested Actions to Address Finding :

See NCU13 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

173 Finding # NCU14 ( Quality Assurance ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Audit: May 26

- 28, 2015 Program Area: North Coast Unified

– Quality Assurance

Finding:

NCUAQMD had not established procedures to initiate corrective actions for resolution of

internal matters.

Description:

The NCUAQMD and CARB R&R document states that NCUAQMD will “Utilize [C]ARB’s CAN process to report instrument malfunctions, operational problems, data quality issues...” It is unclear whether NCUAQMD intends to utilize CARB’s CAN process to resolve internal

matters as well.

During the audit, the staff relayed several ongoing instrument issues. Such systemic problems are the type that both NCUAQMD and CARB indicated wo uld be appropriate to raise in a corrective action

.

While CARB

- issued CANs for the NCUAQMD network have been used and closed, NCUAQMD staff indic ated that the CARB CAN SOP had not yet been reviewed and NCUAQMD had not initiated any CANs

.

References:

EPA Requirements for Quality Assurance Project Plans, EPA/240/B

- 01/003, March 2001, §3.4.1

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Sections 1.2.4, 10, & 15)

State of California Air Resources Board, Air Monitoring QA, Vol. 5, SOP Corrective Action Notification ( http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf )

Suggested Actions to Address Finding :

See NCU14 portion of NCU2.Auditor: Gwen Yoshimura

Final Report

- August 2016

174 Finding # NCU15 ( Quality Assurance ) Agency: California Air Resources Board

– North Coast Unified AQMD

Date of Aud it: May 26

- 28, 2015 Program Area: North Coast Unified

– Quality Assurance

Finding:

A formal agreement describing the roles of NCUAQMD and the PM gravimetric contract lab ( BA AQMD) in the generation of filter PM data was not in place.

Description:

NCUAQMD staff indicated that a portion of the 103 funding is directed to the BA AQMD to conduct the gravimetric analysis of PM filters. Reviewing the documents shared with EPA during the audit, there did not appear to be any information about the QC checks performed by

the lab or what documentation will be produced by whom

.

Filters are sampled and collected by NCUAQMD , sent to the contract lab for analysis, and the data are uploaded by CARB. EPA was unable to find documents with details about the roles

eac h agency plays in filter PM data handling and validation.

 NCUAQMD shared a one

- page document received from BAAQMD noting the QA/QC measures that the contract lab follows and affirming that the data can be used for NAAQS comparison. EPA would expect this ty pe of information, as well as the filter dates/range and the contact at the contract lab responsible for NCUAQMD gravimetric analysis, to be included in each batch of results.

 The CARB PQAO liaison note d that similar district

- contract lab agreements may be available for NCUAQMD to use as a guide.

 NCUAQMD should understand what data validation, if any, is conducted by CARB.

EPA notes that the 103 grant document does not contain the level of detail necessary to address this finding. For example, the agreemen t should address information such as how PM filter CoC forms are filled out. During the audit it was noted that:

 None of the sample C o C forms documented the filter temperature at the various filter handling steps. A column to record filter temperature was included on the C o C forms, however, these were left blank.

 The “Valid Time” field was blank in all forms.

 Two examples of voided C o C forms were provided that did not identify who wrote the lab comments nor were they dated. These forms were for sample dates 11/12/13 and

11/18/13.

 Inconsistent date formats are recorded on the C o C forms. NCUAQMD uses one format while the lab uses another. The template does not specify a date reporting format.

 Mass units are not specified on the C o C template form.Auditor: Gwen Yoshimura

Final Report

- August 2016

175 References :

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Suggested Actions to Address Finding :

See NCU15 portion of NCU2.Auditor: Dena Vall ano

Final Report

- August 2016

176 Finding # SLO

- Positive Finding 1 Agency: Ca lifornia Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- General

Finding:

In general, the SLOCAPCD monitoring progr am was extremely robust and the district staff and managers involv ed in the program were committed to producing defensible data of known quality.

Description:

Specific examples of good practices observed during this audit include:

 Management and staff at all levels were proactively engaged in improving the

program, including monitoring systems and other data collection processes.

 Staff and managers who participated were helpful during the audit.

 Field technicians were skilled and knowledgeable.

 Staff and management have created excellent internal tools and tracking f orms for data validation, field operations, and training.

 A new system for independent data review was being implemented and the process is

well documented.

References:

Suggested Actions to Address Finding :

NAAuditor: Dena Vall ano

Final Report

- August 2016

177 Finding # SLO

- Positive Finding 2 Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- General

Finding:

SLOCAPCD field technicians were knowledgeable and produce excellent documentation.

Descript ion:

SLOCAPCD field technicians were dedicated and knowledgeable. Their internal tracking forms, logbooks, and training forms were exemplary.

References:

Suggested Actions to Address Finding :

NAAuditor: Dena Vall ano

Final Report

- August 2016

178 Finding # SLO1 (General) Agency: California A ir Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- General

Finding:

SLOCAPCD had not fully adopted a centralized corrective action process to resolve internal

issue s.

Description:

EPA quality management standards (EPA QA/R

- 2, Quality Improvement Section) require that management and staff “ensure that conditions adverse to quality” are prevented, identified promptly, fully defined, corrected, prevented from recurring, and documented as corrective actions which are tracked to closure. A formalized corrective action process allows for clear tracking of problems, faster identification of systemic issues, and ensures that all those affected by the issue are notified and that the correctiv e action is followed through to the appropriate conclusion.

At the time of the TSA, SLOCAPCD had formally adopted the CARB Corrective Action Notification (CAN) process, and had been involved in CARB

- initiated CANs. SLOCAPCD had not yet initiated correct ive actions for internal issues or consistently communicated such issues to managers. Internally, s ite troubleshooting was noted in logbooks , and in an “Errata” log, which documented periods of data invalidation along with the rationale. Systemic issues

w ere noted in their SOPs. However, c orrective action was sometimes occurring informally via email, phone call, or in

- person communication without centralized documentation.

While instrument/data problems and their solutions were often documented, document ation varied across log books, errata logs, and data annotations. Management was typically not

informed of corrective actions. A centralized documentation and communication process should be in place for addressing suspect or missing data, instrument probl ems, or other problems observed in the field. Discussions should be documented so that the event can be reconstructed and reviewed to prevent recurrence. Relevant email records should be preserved and attached to the corrective action. Management should be sent copies of the corrective actions in a timely fashion, based on the severity of the problem, indicating whether the problem is ongoing or resolved. When complete, a final close

- out of the corrective action should be signed by the responsible party and documented in a logbook.

References:

EPA Requirements for Quality Assurance Project Plans, EPA/240/B

- 01/003, March 2001, §3.4.1Auditor: Dena Vall ano

Final Report

- August 2016

179 QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. (Sections 1.2.4, 10, & 15)

State of California Air Resources Board, Air Monitoring QA, Vol. 5, SOP Corrective Action Notification ( http://www.arb.ca.gov/aaqm/qa/pqao/can/can_sop.pdf )

Suggested Actions to Address Finding :

A dopt a centralized corrective action process that can be used to consistently resolve internal

matters and includes formal communication to managers ; involve CARB when appropriate (i.e. systemic issues or signifi cant data loss)

.Auditor: Dena Vall ano

Final Report

- August 2016

180 Finding # SLO2 ( Quality Assurance ) Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- Quality Assurance

Finding:

QAPPs had not been completed or updated for O 3 , SO 2 , NO 2 , CO, PM 2.5 , and PM 10 monitoring as required by regulation.

Description:

The CAA requires that organizations using EPA funding to collect environmental data develop a QAPP for each type of ambient pollutant being measured (40 CFR 58 Appendix A, Section 2.1). The QAPP describes all technical and quality aspects of a project and documents the results of the QA planning process for environmental data collection operations, thereby providing a project

- specific “blueprint” for obta ining the type and quality of environmental data needed for a specific decision or use. While the monitoring staff at SLOCAPCD are very knowledgeable and operate the ambient air monitoring network diligently, the lack of updated QAPP documentation

reduces the ability to implement ambient air monitoring requirements effectively and appropriately. In addition, QAPPs should be revised when standards or instruments change.

At the time of the TSA, SLOCAPCD had developed agency

- specific QAPPs and submitted

amend ments to existing CARB QAPP documentation. SLOCAPCD QAPPs were informally approved by CARB on 7/20/2015 and were pending final signature approval

. References: In accordance with 40 CFR Parts 31 and 35, grant recipients are required to document their

qua lity systems.

40 CFR 58 Appendix A, Section 2.1.

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Further guidance on developing QAPPs can be found in the guidance document s “EPA Requirements for Quality Assurance Project Plans,” EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002. Suggested Actions to Address Finding :

Update QAPPs for all criteria pollutants a s necessary, with the goal of having plans for all criteria pollutants in place and submitted to EPA as soon as possible. A dopt CARB’s QAPP documentation, develop one specific to SLOCAPCD , or adopt CARB’s with an addendum specific to SLOCAPCD . Any deviati on from the CARB QAPP documentation must be formally approved by CARB as the lead agency in the PQAO.Auditor: Dena Vall ano

Final Report

- August 2016

181 Finding # SLO 3 ( Quality Assurance ) Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Are a: San Luis Obispo

- Quality Assurance

Finding:

SLOCAPCD did not have updated and complete SOPs to reflect recent changes in operation.

Description:

SOPs detail the work procedures that are to be conducted or followed within an organization.

SOPs document the way activities are to be performed to ensure consistent conformance to technical and quality system requirements and to support data quality. SOPs are intended to be specific to the organization or facility whose activities are described and assist that organization to maintain their quality control and quality assurance processes and ensure compliance with governmental regulations. Well

- written SOPs can also serve as training materials and as references for staff, particularly if they are upd ated regularly (the recommendation is every three years). SOPs should be distributed in a manner that ensures that only the most recent versions are used and that historical SOP revisions are retained (these are sometimes called “controlled

copies”). SOPs should also be developed to enable individuals to transition into new positions. Up

- to

- date SOPs help train operators new to accepted field operation and data management procedures. Deviations and changes from SOPs should be dated, documented, and kept in a bound or electronic document routinely accessed by and accessible to all staff.

Overall, SLOCAPCD maintain ed a nearly complete and readily accessible library of SOPs, but complete and updated SOPs were needed in the following areas: NO 2 Trace level fo r TAPI 200u,

SO 2 Trace level for TAPI 100u, Zero Air Generator for TAPI 701 and data review for Air V ision.

Electronic SOPS were available at every monitoring station and desktop and additional hardcopies were available at the office, and at monitoring st ations when possible.

References:

40 CFR Part 58, Appendix A, Section 2 (SOPs)

QA Handbook for Air Pollution Measurement Systems, Volume II: Ambient Air Quality Monitoring Program, EPA

- 454/B

- 13

- 003, May 2013. (Sections 5 and 17)

Suggested Actio ns to Address Finding :Auditor: Dena Vall ano

Final Report

- August 2016

182 U pdate and obtain CARB’s approval for SOPs on a regular schedule or as needed to reflect current practices, in accordance with 40 CFR 58, Appendix A.

Provide operators with work aids until SOPs are finalized.

D evelop a tracking s ystem to indicate that staff have read and underst oo d the SOP s and have had an opportunity to discuss them with their supervisor.Auditor: Dena Vall ano

Final Report

- August 2016

183 Finding # SLO 4 ( Field Operations ) Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- Field Operations

Finding:

SLOCAPCD did not have comprehensive training documentation for field technicians and contractors who perform routine monitoring system operations.

Description:

EPA not ed that SLOCAPCD operators maintained a very high level of oversight and efficiency, had a thorough understanding of their field operations, and expressed a sense of ownership of the

data. SLOCAPCD has been selected by CARB on several occasions to assist i n teaching monitoring classes and tr ainings over the past few years. Field technicians received good hands

on training with frequent information sharing and issues communication via meetings. However

, training activities and/or demonstration of proficiency w ere not formally documented in all

cases

.

An excellent internal document for training purposes has been developed, but a formalized process and comprehensive documentation for completed training has not been implemented. Formalized training objectives and a tracking system will help to develop proficiency in new staff, to give experienced staff the opportunity to keep skills current, to learn about emerging technologies and capabilities, and to ensure that all staff are familiar with the current QAPPs a nd

SOPs.

References:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013. Section 4.2: Appropriate training should be available to employees supporting the Ambient Air Quality

Mo nitoring Program, commensurate with their duties. Such training may consist of classroom lectures, workshops, web

- based courses, teleconferences, vendor provided, and on

- the

- job training.

Suggested Actions to Address Finding :

Maintain comprehensive tra ining documentation to ensure that all staff and managers are familiar with the updated QAPPs and SOPs related to producing data in the field, proper data verification, validation, management and tracking, quality assurance, corrective action, and all

data systems. I nclude a list of required and recommended training (including training on updated SOPs and QAPPs). T rack staff progress to ensure they have access to training and to ensure completion.Auditor: Dena Vall ano

Final Report

- August 2016

184 Finding # SLO 5 ( Data Management ) Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

- Data Management

Finding:

The data review and validation process was not complete and/or independent.

Description:

Ensuring that da ta validation procedures are independent and have been followed consistently and thoroughly is critical to the generation of defensible environmental data and necessary for an adequate quality system.

SLOCAPCD’s original data review and validation proces s did not include three independent

levels of review and required site operators to perform the data collection, data review and validation roles for their own sites. Additionally, CARB provide d QA oversight of the SLOCAPCD air monitoring programs and subm itted QA data from these sites to AQS.

SLOCAPCD was in the process of implementing a new multi

- tier data verification and validation review process. At the time of the TSA, half of the monitoring sites were being reviewed under this new system , however, t here was insufficient independent QA and data

review at the sites for which the revised process had not been implemented. Complete implementation of the revised data validation process across all monitoring sites was expected to be finalized within a few m onths of completing the field audit.

Under the new process, SLOCAPCD is implementing a 22

- step validation proces s for operators (Level 1), peer review of data by other air monitoring staff (Level II), and data validation package review by an air monitori ng manager (Level III). Routine review monitoring data is performed in a daily basis, and every site visit. The most thorough Level 1 data QC review is done as soon as possible after the end of the month. The Level II review is conducted by air monitoring staff other than the site operator after the Level 1 is completed. EPA reviewed an example data validation package submitted for Level

III review and found that the new multi

- level process is, for the most part, thorough and defensible. This new process is reflected in an updated SOP and SLOCAPCD addendum to CARB QAPP 2.0.2.6 (currently pending CARB approval).

Although SLOCAPCD had a formalized system for tracking and documenting instrument and site maintenance, it lacked a management review process

. Whil e the instrument and site

maintenance was recorded in the electronic data management logbooks and a detailed electronic summary spreadsheet was also used internally for tracking purposes, these

documents d id not undergo a formal review process by managemen t and were not included as part of the data validation process . EPA also noted that some maintenance was behindAuditor: Dena Vall ano

Final Report

- August 2016

185 schedule in certain instances as marked on the internal electronic summary spreadsheet, but this schedule had not undergone any review.

Refe rences:

QA Handbook for Air Pollution Measurement Systems, Vol II: Ambient Air Quality Monitoring Program , EPA

– 454/B

- 13

- 003, May 2013.

Link to Data Verification and Validation resources: http://www. epa.gov/quality1/vandv.html

Suggested Actions to Address Finding : Formalize the implementation of a data validation procedure that incorporates multiple levels of independent review on frequent (i.e. monthly) basis across all monitoring sites

.

Integr ate independent QA oversight through out the data validation process

.

Include the electronic summary spreadsheet that contain s all relevant instrument and site

maintenance information ( i.e. repairs, calibrations, etc.) in the data validation package for

L evel III data validation review and communication to managers

.

Ensure all staff involved in validation are familiar with acceptance criteria, applicable flags to apply when criteria fail, and have access to the data used in generating the validation repo rt.

Revise and update data validation SOP and QAPPs to reflect the new multi

- level data validation process.Auditor: Dena Vall ano

Final Report

- August 2016

186 Recommendation # SLO

– Recommendation1 Agency: California Air Resources Board

– San Luis Obispo County APCD

Date of Audit: August 26

- 27, 2015 Program Area: San Luis Obispo

– Network Management

Finding:

The NO 2 monitor at Morro Bay may be discontinued.

Description:

SLOCAPCD should consider the discontinuation of the NO 2 monitor at Morro Bay if network priorities shift in the futur e or resources limit SLOCAPCD’s capacity to effectively operate

their current network. As noted in the SLOCAPCD 2015 Ambient Air M onitoring Network Assessment, only one NO 2 monitor is required for the County. The NO 2 monitor at Atascadero measures the high est NO 2 levels in the County, and concentrations rarely reach 50% of the 24

- h ou r NAAQS. Additionally, the Morro Bay NO 2 monitor was originally established to monitor emissions from the Morro Bay power plant, located less than a mile upwind. The plant

perma nently cl osed in February 2014.

References:

Suggested Actions to Address Finding :

C oordinate with CARB to initiate early communication regarding any desired system

modificati ons or potential changes to either the C ARB

- or SLO

- operated site s within the monitoring network located within SLOCAPCD’s boundaries. Include any modifications to SLAM site(s) in the next Annual Monitoring Network Plan.Final Report

- August 2016

187 APPENDIX A: SUMMARY OF FINDINGS

G

– Positive Finding 1: A number of important projects were underway to improve the
PQAO.

G

– Positive Finding 2: CARB has been providing increased support to the CARB PQAO

districts.

G1: Not all districts have been fully attending trainings or identifying and requesting a sufficient level of technical support.

G2: Comm unication within the PQAO was not consistent.

NM

– Positive Finding 1: The CARB PQAO has been working with EPA to create the

CAMNAT database.

FO

- Positive Finding 1: Staff were professional and helpful, and in most instances very knowledgeable regardi ng field operations.

FO

- Positive Finding 2: The AMSS has developed an on

- the

- job training program for field

technicians.

FO

- Positive Finding 3: A training database for field staff is under development.

FO

- Positive Finding 4: CARB has been worki ng with several manufacturers as they develop new instruments.

FO

- Positive Finding 5: All regulatory PM 2.5 BAM instruments within the PQAO were being

converted to VSCCs.

FO1: Some field forms contained errors that were not properly identified or corr ected in the initial form validation or subsequent peer reviews.

FO2: Staff were not required to demonstrate proficiency when assigned new tasks.

FO3: QC check gas is not delivered externally to the instruments at the San Luis Obispo, Mojave Poole, an d other O 3

- only sites , as the 400E/T400 IZS O 3 monitor delivers quality control (QC) check gas to the monitor internally

.

FO4 [Finding FO13 from 2011 TSA]: MLD staff were not formally documenting the quality of zero air being used in the program.

FO5: The field SOPs did not reflect many changes made to the program over the past several

years.Final Report

- August 2016

188

FO6: Data loss has occurred at some sites due to CARBLogger malfunctions and lack of site operator awareness regarding how data can be recovered.

FO7: Filter d ata loss has resulted in several of the CARB

- operated sites missing the 75% data completeness requirement for certain quarters.

FO8: In one instance, a field technician was not aware of the correct schedule for taking make

up samples on filter

- based inst ruments.

FO9: Recertification of sensor probes was not based on the recommended instrument

manufacturer’s date of recalibration.

FO10: The San Luis Obispo monitoring station did not have an adequate guard rail to assure safety during routine operatio ns.

FO

- Recommendation1: The freezer used to store filters at Bakersfield

- California was not kept secure and did not have a temperature gauge.

DM

– Positive Finding 1: CARB installed a new data management system.

DM

– Positive Finding 2: CARB was c reating a new user guide/SOP for DMS.

DM

– Positive Finding 3: CARB was creating an SOP for CARBLogger.

DM

– Positive Finding 4: CARB was upgrading DMS so that codes used in DMS match those used in AQS.

DM

– Positive Finding 5: CARB was creating guid ance on minimum value reporting for PM

instruments

.

DM

– Positive Finding 6: CARB was working with the PQAO districts to ensure that all regulatory PM

10 instruments are reporting data in standard temperature and pressure (STP).

DM

– Positive Finding 7: CARB was creating a data visualization tool to aid data review.

DM1: QA qualifier codes were not entered in instances where critical QC criteria were missed.

DM2: Calibrations were not always coded as such in AQS.

DM3: Field technicians were unable to apply codes in CARBLogger and DMS to missing data, resulting in missing non

- coded data in AQS.

DM4: Negative values were inconsistently invalidated for PM 2.5 monitors.

QA

– Positive Finding 1: Since the 2011 TSA, the Quality Management Section has been created and, in coordination with other areas of CARB, has implemented significant changes within the PQAO.Final Report

- August 2016

189

QA

– Positive Finding 2: R&R documents have been created between CARB and each of the twenty

- one districts that conduct regulatory monitoring

.

QA

– Positive Finding 3: CARB has created a QA webpage that presents PQAO QA information in one, easy to navigate webpage.

QA

– Positive Finding 4: CARB held three successful PQAO training modules, and is in the process of creating a condensed versio n of the training to be offered on a repeated

basis.

QA

– Positive Finding 5: CARB was creating guidance as to how districts should review data once they are entered into AQS.

QA1: Improvements to the AQDA and CAN processes are necessary to make them m ore efficient and effective, and ensure timely resolution of monitoring issues.

QA2: An AQDA that indicated a need for AQS to be changed was not appropriately closed out, and inaccurate AQS codes were entered overriding previously entered, more accurate AQS

codes.

QA

- Recommendation1: Annual performance audits were not being used to identify significant practices that needed correction.

QA

- Recommendation2: Data validation and reporting responsibilities were unclear in one

instance when a district u tilized a non

- CARB laboratory for analysis. This may extend to other

such situations as well.

SL

– Positive Finding 1: CARB was working to restructure roles and responsibilities within QMB to improve independence, review, and succession planning within t he Standards Laboratory.

SL1: The Standards Laboratory QA system did not have written policies and procedures, a QA manual or a designated QA Officer.

SL2: Corrective action processes for the Standards Laboratory were not consistent with ISO or

NIST re quirements.

SL3: Calibrations/Certifications conducted by the Standards Laboratory were not reviewed by peers/management/QA in sufficient detail to prevent errors.

SL4: Forms to document standard certification were not completely validated or revalidat ed on a

routine basis.

SL5: Calibration laboratory staff had not attended routine trainings or conferences that are specific to their field of work.Final Report

- August 2016

190

SL6: The main Standards Laboratory database and several data systems had not been updated recently and did not meet expectations of performance and/or documentation and compromise data defensibility.

SL7: Zero air was not being treated as a standard and the quality of zero air was not formally documented by the Standards Laboratory.

SL8: The materials u sed for plumbing and mixing for SO 2 certifications were stainless steel. These are not appropriate materials for SO 2 certifications.

SL

– Recommendation1: Contracting issues affected the ability of the Standards Laboratory to certify standards and delaye d required primary standard certifications.

PM

– Positive Finding 1: CARB was creating a laboratory support section.

PM

– Positive Finding 2: CARB’s PM laboratories were found to be well maintained and

well run.

PM

– Positive Finding 3: CARB was crea ting a system to ensure consistent documentation when a filter does not run.

PM1: Statistical information on temperature and humidity compliance with method requirements was not consistently calculated or tracked in accordance with EPA’s specified

metho dology.

PM2: Preservation and custody requirements were not being met and/or tracked for samples that had been hand delivered to the MLD laboratory.

PM3: Laboratory blanks for PM 2.5 were not processed in batches consistent with regulatory requirements.

PM4: PM 2.5 MTL filter lots and the associated lot controls could not be properly evaluated and were not evaluated at a sufficient frequency.

PM5: The PM 10 laboratory SOPs had not been updated in over five years.

PM6: The procedure for processing fil ters for PM 10 speciation led to data being invalidated unnecessarily in one observed instance.

PM7: Not all databases and electronic forms were tracked and validated, compromising data defensibility.

PM8: MLD weigh room practices did not protect the i ntegrity of samples.

Pb1: CARB’s Pb laboratory procedures did not meet EPA requirements.Final Report

- August 2016

19 1

AV/MD

- Positive Finding 1: AV/MD AQMD’s monitoring staff were dedicated and maintain sites spread out across a significant geographic area.

AV/MD1 (General): Th e data review and validation process was not complete and/or independent. Numerous errors were found with data that had been loaded into AQS.

AV/MD2 (General): The number of staff and managers and the amount of their time allocated to the air monitoring program were inadequate for the current network and associated activities.

AV/MD3 (General): Documentation, while generally present, was sometimes incomplete, not legally defensible, poorly organized, and not reviewed by a peer or a manager.

AV/MD4 (Gen eral): The AV/MD AQMD staff, while part of the PQAO, were generally not following CARB SOPs and QA planning documents as stated in their R&R document.

AV/MD5 (General): The AV/MD AQMD training program was not robust enough to ensure that new staff are p roperly trained, current staff are cross trained, and current staff’s knowledge is refreshed and up

- to

- date.

AV/MD6 (General): The AV/MD AQMD did not rely on the CARB PQAO for independent QA and support.

AV/MD7 (Network Management): The transition from filter

- based PM 10 to continuous PM 10 was not properly defined.

AV/MD8 (Field Operations): The span/precision standard at the Lancaster site expired on

8/6/2014.

AV/MD9 (Field Operations): AV/MD AQMD staff were not tracking residence time.

AV/MD10 (Fi eld Operations): AV/MD AQMD staff were not evaluating manifold material requirements.

AV/MD11 (Field Operations): Zero air sources were not considered standards and maintained as recommended by the manufacturer.

AV/MD12 (Field Operations): PM 2.5 filt ers from the Lancaster site were not being transported at a controlled temperature back to the office.

AV/MD13 (Field Operations): The PM 2.5 instrument files were not being downloaded from the samplers.

AV/MD14 (Field Operations): The plug was missing from the rain cap at the Phelan Station and

the Teflon O 3 inlet tube was resting on the plastic funnel.Final Report

- August 2016

192

AV/MD15 (Data Management): Based on a limited review, EPA found a significant number of data errors in AQS.

AV/MD16 (Data Management): AV/MD AQMD’s system did not reliably back up all data.

AV/MD17 (Data Management): Some of the metadata in AQS were missing or incorrect.

AV/MD18 (Data Management): The data record was incomplete for several pollutants as negative values are routinely invalidated.

AV/MD19 (PM Laboratory): Temperature and humidity issues in the PM weigh room compromised the quality of data collected in 2014 and possibly earlier. Temperature and humidity were not tracked, so it was not easy to identify out

- of

- control situations.

AV/ MD20 (PM Laboratory): The quality control activities utilized for PM 2.5 filter weighing were insufficient and inconsistent with EPA regulation, guidance, and general laboratory principles.

AV/MD21 (PM Laboratory): Holding times were not being met for al l PM filters.

AV/MD22 (PM Laboratory): The data system used for PM weighing was not documented, understood by current staff, flexible, or fully validated.

AV/MD23 (PM Laboratory): Storage and custody of PM filters were not consistent with general

sam ple handling practice, EPA guidance and regulations.

AV/MD24 (PM Laboratory): Several aspects of general weigh laboratory procedures were not consistent with EPA guidance.

NCU

– Positive Finding 1: NCUAQMD was implementing a number of good practices in support of site and instrument maintenance, and data management.

NCU1 (General): The current purpose/objectives of NCUAQMD’s NAAQS monitoring network

were unclear.

NCU2 (General/Quality Assurance): QA document gaps were identified.

NCU3 (General): T raining specifications/requirements for air monitoring personnel were not well defined and completion was not tracked.

NCU4 (General): Major monitoring purchases have been problematic for NCUAQMD and

CARB.

NCU5 (Field Operations): Field documentation g aps and defensibility issues were identified.Final Report

- August 2016

193

NCU6 (Field Operations): Monthly maintenance check sheets, as prescribed by CARB’s SOPs,

have not been used.

NCU7 (Field Operations): Sampling lines/tubes were not regularly replaced.

NCU8 (Field Operation s): Some rooftop equipment was not clean.

NCU9 (Field Operations): The refrigerator containing PM filters that have been collected from

the field was not locked.

NCU10 (Data Management): Data collected with FEM/FRM instruments were not uploaded to AQS as required by CFR.

NCU11 (Data Management): NCUAQMD had not verified that their data processing software meets performance guidelines consistent with the QA Handbook, Volume II, Section 14.

NCU12 (Quality Assurance): The level of data quality perform ance requirements for gaseous pollutants were less stringent than those in EPA regulation and guidance.

NCU13 (Quality Assurance): NCUAQMD did not have a procedure for checking/verifying zero

air.

NCU14 (Quality Assurance): NCUAQMD had not established procedures to initiate corrective

actions for resolution of internal matters.

NCU15 (Quality Assurance): A formal agreement describing the roles of NCUAQMD and the PM gravimetric contract lab (BAAQMD) in the generation of filter PM data was not in place

.

SLO

- Positive Finding 1: In general, the SLOCAPCD monitoring program was extremely robust and the district staff and managers involved in the program were committed to producing defensible data of known quality.

SLO

- Positive Finding 2: SLOCAPCD fi eld technicians were knowledgeable and produce

excellent documentation.

SLO1 (General): SLOCAPCD had not fully adopted a centralized corrective action process to

resolve internal issues.

SLO2 (Quality Assurance): QAPPs had not been completed or updated for O 3 , SO 2 , NO 2 , CO,

PM 2.5 , and PM 10 monitoring as required by regulation.

SLO3 (Quality Assurance): SLOCAPCD did not have updated and complete SOPs to reflect recent changes in operation.

SLO4 (Field Operations): SLOCAPCD did not have comprehensive t raining documentation for field technicians and contractors who perform routine monitoring system operations.Final Report

- August 2016

194

SLO5 (Data Management): The data review and validation process was not complete and/or independent.

SLO

– Recommendation1 (Network Management) : The NO 2 monitor at Morro Bay may be

discontinued.Final Report

- August 2016

195

APPENDIX B: CARB ORGANIZATIONAL CHARTS

Organization of CARB’s Divisions and OfficesFinal Report

- August 2016

196Final Report

- August 2016

197Final Report

- August 2016

198Final Report

- August 2016

199Final Report

- August 2016

200

APPENDIX C : MOJAVE DESERT AIR QUALITY MANAGEMENT DISTRICT ORGANIZATIONAL CHARTS

& MONITORING STATIONS MAPFinal Report

- August 2016

201

Air Monitoring Department and Information Systems Department

.Final Report

- August 2016

202

M ojave Desert Air Basin M onitoring Station s Map. Shows monitors within Easter Kern, Antelope Valley, and Mojave Desert air

districts.Final Report

- August 2016

203

APPENDIX D: NORTH COAST UNIFIED AIR QUALITY MANAGE MENT DISTRICT ORGANIZATIONAL CHARTFinal Report

- August 2016

204

APPENDIX E: SAN LUIS OBISPO COUNTY AIR POLLUTION CONTROL DISTRICT ORGANIZATIONAL CHARTFinal Report

- August 2016

205 APPENDIX F : MATRIX OF 2011 TSA FINDINGS

Finding

Number Description Corrective Action

Plan Status Close Out Status

G1 [Previous Finding M1] CARB needs to complete the process of putting a formal PQAO into place. Approved Conditionally
approved

G2 The QMB does not have the structure and staff to manage QA oversight of the PQAO districts. Approved Approved

G3 [Previous Finding M6] While progress has been made on updating the CARB QA Manual with a

QMP and QAPPs or equivalent documents, the process is behind schedule. Approved Additional

documentation

needed

G4 Local districts within the CARB PQAO do not always have updated quality system documentation

for all activities. Approved Approved

G5 [Previous Findings QM1 and M3] QA Authority and interactions between QMB and the other

branches should be expanded and formalized. The corrective action system should be developed

to include ac tions taken, in addition to reports issued by the QA auditors and the calibration
laboratory. Approved Not submitted

G6 Coordination between CARB and districts and EPA should be improved. Approved Approved

NM1 Not all agencies within the CARB PQAO have a pproved network plans since this became a

requirement in 2006. The current approach to network plans does not provide for a determination
of network adequacy on a statewide basis. Approved Approved

NM2 The network assessment does not meet all CFR requirem ents. Approved Approved

NM3 There are PM 10 monitors listed in local conditions (LC; parameter code 85101), but not Standard

Temperature and Pressure (STP; parameter code 81102 in AQS), as required for FRM/FEM

instruments. Approved Not submitted

FO1 [R elated Previous Findings GB3, SJV3, & NS2] Documentation at the CARB field sites is inadequate

and not reviewed by management. Approved Approved

FO2 Management oversight of site operators needs strengthening. Approved Approved

FO3 CARB field operators ha ve not been trained on new SOPs. Approved Approved

FO4 Residence time calculations were not available at any CARB site visited. Approved Additional

documentation

needed

FO5 Delay in sending PM 2.5 samples has resulted in loss of data. Approved Approved

F O6 PM make

- up samples are not being taken in accordance with EPA guidance. Approved ApprovedFinal Report

- August 2016

206

FO7 PM 10 QC checks are not consistently recorded. There is no document in which field operators are

directed to record this information. Approved Approved

FO8 CA RB field staff do not check data after sending information to CARB offices. Approved Approved

FO9 The Yuba City site has several significant siting issues that need to be resolved. Approved Approved

FO10 Records indicate that calibrations of gaseous poll utant instruments are not consistently done

according to a schedule. Approved Approved

FO11 [Previous Finding AQSB7] The number of NO 2 titration points taken during calibration does not

meet regulatory requirements. Approved Submitted;
undergoing review

FO12 Multi

- point calibrations of PM 2.5 instruments are not done routinely. Approved Submitted;
undergoing review

FO13 [Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air being used in

the program. Revision requested

– to be

a ddres sed
through 2015 TSA finding FO4 Not submitted

– to

be addres sed
through 2015 TSA finding FO4

FO14 Span and precision gases used in the field are not being calibrated routinely. Approved Approved

FO15 Instruments removed from the field are not alway s efficiently tracked and returned to the repair

laboratory facility for diagnosis, repair, and reuse. Loss of data has occurred due to unavailability
of spare instruments. Approved Approved

FO16 Positive Finding

- CARB is working to improve communication with field staff. NA

FO17 Positive Finding

- CARB maintains a well

- equipped stockroom of spare parts, maintains a large

equipment purchase order history, and develops thorough equipment testing procedures that are
regularly updated. NA

DM1 The data validation and review/verification performed by AQSB, NLB, and AQAS, are not formally

published in a control

- copied SOP. Approved Approved

DM2 [Previous Finding M7] Data submitted by local districts within the CARB PQAO are not validated

using consistent procedures. (See Findings SJV9, IMP10, and MEN11) Approved Approved

DM3 [Previous Finding DM5] AQAS does not ensure that local district data are validated prior to upload

to AQS. Approved Approved

DM4 A few instances of erroneous continuous data were ide ntified in AQS for CARB sites. Approved Not submitted

DM5 Erroneous continuous data were identified in AQS for non

- CARB sites within the CARB PQAO. Approved Approved

DM6 [Previous Finding DM6] There are numerous deficiencies in the data certification pro cess for the

CARB PQAO, including: Not all NAAQS

- compliant data within the CARB PQAO are routinely
certified; Data certified by CARB for local districts are not typically reviewed or validated; Data are routinely certified by agencies within the State of C alifornia, but responsibility has not been
formally delegated to any local agencies within the State of California. Approved ApprovedFinal Report

- August 2016

207

DM7 Data, including those used for design value sites, have been changed after they are certified and

subsequently not re certified. Approved Approved

DM8 Some local districts within the CARB PQAO are listed as PQAOs in AQS. Approved Approved

DM9 There were several instances of CARB altering data collected by local districts without

communicating with the district. Approved Approved

QA1 The QA Audit group has made an effort to improve its documentation process; however, several

inconsistencies were noted. Approved Approved

QA2 The audit trailer evaluated was using one expired gas cylinder along with others that had not

bee n certified annually as required for the EPA National Performance Audit Program (NPAP). Approved Approved

QA3 The QA Section is not tracking monitors to ensure that 25% are being audited per calendar

quarter. Approved Approved

QA4 The connection to the i nlet on the audit trailer could pull in outdoor air. Approved Approved

QA5 Auditors do not review all applicable siting information in AQS prior to an audit. Approved Approved

QA6 [Previous Findings M4 & OPA2] Quality assurance for special projects is no t developed in a process

consistent with EPA quality system requirements. Approved Approved

QA7 Mass flow elements (MFEs) are used to establish calibration points outside of their calibrated

range. Approved Approved

PM1 Communication of post

- weigh inform ation and transmission of documentation to local districts

should be improved. Approved Approved

PM2 The PM laboratory does not have a formal corrective action process for addressing issues with PM

filter collection. Approved Approved

PM3 Documentation o f activities in the PM 10 and PM 2.5 laboratories should be improved. Approved Approved

PM4 PM 10 trip blanks are not being used to assess potential bias from filter transport and handling. Approved Approved

TL1 The canister cleaning SOP does not reflect th e current cleaning procedure. Approved Approved

TL2 An SOP is not documented for the batch certification of cleaned canisters. The canister cleaning

SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method. Approved Approved

TL3 The batch certification of cleaned canisters described by staff for methods MLD 058 and MLD 066

differs from existing VOC guidance. Approved Approved

TL4 [Previous Finding 19]. Pre

- cleaning concentrations are not recorded in a logbook to allow for the

selection of the most highly contaminated canister for batch certification. Approved Approved

TL5 Canisters are not routinely leak tested as prescribed in guidance. Instead, canisters are vacuum

leak tested only when gross leaks are suspected. A pproved Approved

TL6 [Previous Finding 21]. A retention time policy for re

- cleaning and blanking canisters once they

have been certified clean has not been established. Approved Approved

TL7 The CARB SOP states that old canisters are reconditioned, b ut this is inconsistently practiced. Approved ApprovedFinal Report

- August 2016

208

TL8 CARB has not established a holding time for cartridges once samples have been collected for

extraction or analysis. Approved Approved

TL9 The laboratory does not assign expiration dates to new s ampling cartridges and allows cartridges

to be used beyond the 90 days prescribed by the method. Approved Approved

TL10 CARB's procedure for analyzing DNPH lot blanks differs from the SOP. Approved Approved

TL11 No criterion is provided in the CARB S OP for passing DNPH lot cartridge blanks. Approved Approved

TL12 Gloves are not worn as a contamination protection measure when handling cartridges. A

nitrogen

- purged glove bag is not used for extractions. Approved Approved

TL13 [Previous Finding OL 3] Staff stated that field blanks are not being analyzed at a frequency of 10%

of field samples, as specified in Method TO

- 11, nor is there an SOP describing the procedure for
the submission of field blanks. Approved Approved

TL14 CARB does not analyze trip blanks when needed. Approved Approved

TL15 Method TO

- 11 states that samples should be re

- analyzed when results are 10% above the

criterion, but the analyst was not aware of this criterion. Approved Approved

TL16 Working standards are tracked and us ed for six months, while the CARB SOP states that standards

should be retained for four months under refrigeration. Approved Approved

TL17 Site name and sampling dates are recorded on a piece of tape loosely stuck to sample cartridges;

the tape occasional ly falls off, making it difficult to identify samples. Approved Approved

TL18 Positive Finding

- Hexavalent chromium data undergo peer review, supervisory review, review by

the Branch Chief, and a final review before going to AQS. NA

TL19 [Previous Finding OL5] There is no secondary review of logbooks. Approved Approved

TL20 CARB does not analyze audit samples or through

- the

- probe audit samples as suggested in Sec. 9.7

of the CARB SOP. Approved Approved

TL21 Appendix V in the CARB SOP lists the s tandards that were used in 2003 and has not been updated

to reflect the standards currently being used. Approved Approved

IMP1 The ICAPCD ambient air monitoring program is not operating under an approved QAPP. Approved Not submitted

IMP2 ICAPCD has not established an appropriate quality system for ambient air monitoring. Approved Not submitted

IMP3 Assessment of PM 10 or PM 2.5 sampling frequency throughout the ICAPCD network has not been

performed as required. Approved Approved

IMP4 Neighborhood scale may be inappropriate for PM 10 at the Westmorland site. Approved Approved

IMP5 One

- point flow rate verifications for PM 10 and PM 2.5 are not performed by ICAPCD as required and

are not well documented. Approved Approved

IMP6 Residence time for gaseous moni tors operated by ICAPCD is not established. Approved ApprovedFinal Report

- August 2016

209

IMP7 ICAPCD is internally post weighing high

- volume PM 10 filters without proper PM lab or quality

control measures. Approved Approved

IMP8 Documentation of ICAPCD air monitoring activities is not complete. Approved Not submitted

IMP9 There are potential siting issues at the Calexico Ethel site. Approved Additional

documentation

needed

IMP10 ICAPCD is not adequately reviewing and editing data. Approved Additional

documentation

needed

MEN1 MeC

APCD staff was not familiar with the CARB QMP or instrument SOPs. Approved Approved

MEN2 MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2006 but is erroneously

listed as its own PQAO in AQS Approved Approved

MEN3 Positive Finding

- Me CAQMD stations were well

- maintained. Staff and the manager were

professional and helpful, and very knowledgeable about the county and potential pollution
sources. The station operator was proactive about troubleshooting instrument issues. NA

MEN4 One

- point QC checks (flow verifications) for PM 10 and PM 2.5 are not consistently performed by

MeCAPCD site operators. Approved Additional

documentation

needed

MEN5 MeCAPCD logbook entries are not consistently made and are not always in the most defensible

fo rm. Handwritten notes are occasionally illegible due to water (rain) marks. Approved Additional

documentation

needed

MEN6 Residence time calculations were not available at the Ukiah Gobbi site. Approved Approved

MEN7 Trees at the Ukiah Gobbi and Library sites should be evaluated against siting requirements. Approved Approved

MEN8 The internal shelter thermostat is not operating correctly at the Ukiah Gobbi site and the issue has

not been addressed to provide defensible data. Approved Approved

MEN9 MeCA PCD has no system for tracking and controlling station and instrument logbooks. Approved Additional

documentation

needed

MEN10 MeCAQMD should have formalized training requirements for all air monitoring staff. Approved Approved

MEN11 MeCAQMD does not pro vide CARB AQAS with datasets that have been fully quality assured and

ready for upload to AQS. Approved Approved

MEN12 MeCAQMD does not use a formal corrective action system. Approved Approved

SJV1 Positive Finding

- In general, the SJVAPCD monitoring pr ogram is robust and the agency staff and

managers involved in the program are committed to producing defensible data of known quality. NA

SJV2 SJVAPCD does not have updated quality system documentation for all activities. Approved ApprovedFinal Report

- August 2016

210

SJV3 SJVAPCD has experienced significant data losses at required monitoring sites, including sites

critical for demonstrating compliance with the NAAQS. Approved Approved

SJV4 SJVAPCD has initiated network modifications for several required sites without seeking EPA

approval required by 40 CFR 58.14. Approved Approved

SJV5 The residence time of flow between the inlet and each instrument was not posted at every

SJVAPCD site. Approved Approved

SJV6 Some SJVAPCD site logbooks lacked specific information about the date or type of maintenance

performed on an instrument. Approved Approved

SJV7 There is no documentation of management review of station logbooks and other site activities for

SJVAPCD operated sites. Approved Approved

SJV8 SJVAPCD site operators do not have a quick visual way to identify changes in instrument

performance or QC checks that would indicate instrument issues, nor do they have the ability to
remotely check on data or site operations. Approved Approved

SJV9 SJVAPCD experiences data loss due to inst rument malfunction. Approved Approved

SJV10 It is unclear whether SJVAPCD is using appropriate criteria to invalidate or flag PM 10 data. Approved Not submitted

SJV11 SJVAPCD experiences significant resource inefficiencies for staff and management as the current

data management system relies solely on manual inputs. Approved Not submitted

SJV12 SVJAPCD does not have a formal corrective action process in place. Approved Approved