System Audit of the Ambient Monitoring Program:

California Air Resources Board

June

- September, 2011

Conducted by:

Air Quality Analysis Office Quality Assurance Office

Air Division Management and Technical

Services Division

US EPA Re gion 9

75 Hawthorne Street San Francisco, California 94105

Final Report: October 2012i PAGE

GLOSSARY OF ACRONYMS

........................................................................ . ii

EXECUTIVE SUMMARY

............................... .................................................1

TSA ACTIVITIES

............................................................... ...............................5

OVERVIEW OF STATE AIR MONITORING PROGRAM

...................... 10

OVERVIEW OF LOCAL DISTR ICT AIR MONITORING PROGRAMS

IMPERIAL COUNTY APCD ..............................................................2 6 MENDOCINO COUNTY AQMD

......................... ..............................29 SAN JOAQUIN VALLEY APCD

................................... .....................32

FINDINGS..........................

.......................................... .............................35

APPENDIX A, Summary of Findings

............................ ............................... 14 5

APPENDIX B, Organizational Chart...............

... ........ ............................152

APPENDIX C, CARB Data Validation Documents

........ ............................ 15 5ii GLOSSARY OF ACRONYMS

ADAM................................... Aerometric Data and Management

AmAPCD..................

............. Amador County Air Pollution Control District

AMNS.................................... Air Monitoring Northern Section

AMSS..................................... Air Monitoring Southern Section AnAQMD.............................. Antelope Vall ey Air Quality Management District

APCD..................................... Air Pollution Control District AQAS..................................... Air Quality Analysis Section AQDA.................................... Air Quality Data Action AQDAS.......

........................... Air Quality Data Acquisition System AQDB.................................... Air Quality Data Branch AQMD................................... Air Quality Management District AQS....................................... Air Quality System AQSB..................................... Air Quality Surveillance Branch

ATP........................................ Acceptance Test Procedure

BAM...................................... Beta Attenuation Mass Monitor (continuous PM 2.5 ) BAAQMD

........ ...................... Bay Area Air Quality Management District BCAQMD..............................Butte County Air Quality Management District

CAA....................................... Clean Air Act

CaCAPCD.............................. Calaveras County Air Pollution Control District

CARB..................................... California Air Resources Board

CBSA..................................... Core

- based Statistical Area

CO.......................................... Carbon Monoxide

CoC.................

....................... Chain of Custody

CoCAPCD............................. Colusa County Air Pollution Control District

CFR........................................ Code of Federal Regulations DQO....................................... Data Quality Objec tive

EC.......................................... Elemental Carbon EDCAQMD...........................El Dorado County Air Quality Management District

EKAPCD............................... Eastern Kern Air Pollution Control District

EPA.....................

.................. Environmental Protection Agency

FEM....................................... Federal Equivalent Method FRAQMD.............................. Feather River Air Quality Management District

FRM....................................... Federal Re ference Method

GBUAPCD............................ Great Basin Unified Air Pollution Control District

GCAPCD............................... Glenn County Air Quality Management Districtiv GPS........................................ Global Positioning System

I CAPCD................................. Imperial County APCD

IZS......................................... Internal Zero/Span LakeCAQMD......................... Lake County Air Quality Management District LassenCAPCD.......................Lassen County Air P ollution Control District

LC.......................................... Local Conditions

LIMS...................................... Laboratory Information Management System MaCAQMD...........................Mariposa County Air Quality Management District

MB UAPCD............................ Monterey Bay Unified Air Pollution Control District

M e CA QMD........................... Mendocino County AQMD MDAQMD............................. Mojave Desert Air Quality Management District

MFE.............................

.......... Mass Flow Element

MLD....................................... Monitoring and Laboratory Division MoCAPCD.............................Modoc County Air Pollution Control District

MSA....................................... Metropolitan Statistical A rea NAAQS.................................. National Ambient Air Quality Standard

NCore..................................... National Core multi

- pollutant monitoring stations NCUAQMD........................... North Coast Unified Air Quality Management Di strict

NELAC.................................. National Environmental Laboratory Accreditation

Conference

NIST....................................... National Institute for Standards and Technology

NLB....................................... Northern Labora tory Branch

NPAP..................................... National Performance Audit Program NSAQMD.............................. Northern Sierra Air Quality Management District NSCAQMD........................... Northern Sonoma County Air Pollution Control Di strict

O 3

........................................... Ozone

OC.......................................... Organic Carbon

OMB...................................... Office of Management and Budget

OPAS..................................... Operations, Planning, and Assessments Section

OSS........................................ Operations Support Section

PAMS..................................... Photochemical Assessment Monitoring Stations

Pb........................................... Lead

PCAPCD........

........................ Placer County Air Pollution Control District

PEP......................................... Performance Evaluation Program

PM.......................................... Particulate matter

PM 2.5

...................................... Pa rticulate matter 2.5 microns or less in aerodynamic

diameter

PM 10

....................................... Particulate matter 10 microns or less in aerodynamic

diameter

POC........................................ Parameter Occurrence Code PQAO...............

...................... Primary Quality Assurance Organization

PTSD...................................... Planning and Technical Support Division QA.......................................... Quality Assurance QAPP..................................... Qualit y Assurance Project Planv QAS....................................... Quality Assurance Section QC.......................................... Quality Control QMB...................................... Quality Management Branch QMP..............................

......... Quality Management Plan

RH.......................................... Relative Humidity

SBCAPCD............................. Santa Barbara County Air Pollution Control District SCAQMD.............................. South Coast Air Quality Managemen t District

SDC APCD............................ San Diego County Air Pollution Control District ShCAQMD............................ Shasta County Air Quality Management District

SiCAPCD............................... Siskiyou County Air Pollution Control Di strict

SJVAPCD.............................. San Joaquin Valley APCD

SLAMS.................................. State or Local Air Monitoring Station

SLOCAPCD........................... San Luis Obispo County Air Pollution Control District SMAQMD.............

................ Sacramento Metro Air Quality Management District

SOP........................................ Standard Operating Procedure

SPM....................................... Special Purpose Monitor

SO 2

......................................... Sulfu r Dioxide

STP......................................... Standard Temperature and Pressure

TAD....................................... Technical Assistance Document

TeCAPCD............................. Tehama County Air Pollution Control District

TSA.........

............................... Technical System Audit

TSP......................................... Total Suspended Particulate

TuAPCD................................ Tuolumne County Air Pollution Control District

VCAPCD............................... Vent ura County Air Pollution District

VOC....................................... Volatile Organic Compound

Y

- SAQMD.............................Yolo

- Solano Air Quality Management District1 EXECUTIVE SUMMARY

This document is a report of the findings made by EP A while conducting a Technical Systems Audit (TSA) on the air monitoring program of the California Air Resources Board (CARB). A TSA is an on

- site review and inspection of a state or local ambient air monitoring program to assess its compliance with establ ished regulations governing the collection, analysis, validation, and reporting of ambient air quality data. This TSA meets the requirements for EPA audits of CARB’s monitoring organization as described in 40 CFR Part 58, Appendix A, Section

2.5.

Key Find ings from TSA: Finding Potential Impact Recommended Corrective

Action Need to formalize PQAO (Primary Quality Assurance Organization) structure of CARB PQAO Lack of coordination and oversight resulting in compromised data quality CARB and local districts within CARB PQAO need to identify a mechanism to def ine and formally implement the partnership Lack of approved/adopted quality system documents • Data defensibility questioned

• Inconsistent operation of ambient air monitoring

network within CARB and

within the CARB PQAO • CARB is finalizing its updated quality system

documents

• Local districts can adopt CARB’s or prepare their

own

• CARB needs effective

mechanism to share updates/changes/additions to quality system

documents such as SOPs Network management, whi ch includes network plans, network assessments, site closures, and data certification, have been inconsistently managed across air agencies in California • Ina bility to approve network plans

• Lack of understanding of network requiremen ts

• Regulatory decisions hindered by loss of required

sites and lack of data

certification • More coordination and oversight by CARB on network plans and data

certification

• Improve analyses and

coordination for the 5

- year Network Assessment

• Better coordinatio n with

EPA on site closures2 Data validation: lack of coordination and training has resulted in inadequate and

inconsistent data validation Erroneous data in the Federal regulatory air data system • Training on data validation

• Define roles/responsibilities

for data validation

• Data audits by CARB Inconsistent field operations • Loss of data

• Erroneous data

• D efensibility of data compromised • Comprehensive training

• Audits by EPA and CARB

• Establish a field operator

network

Coordination between CARB

and local dist ricts needs to be improved All the issues above • PQAO listserv

• CAPCOA Air Monitoring Managers Committee

• Standard conference calls/meetings for CARB PQAO

CARB, an organization under the umbrella of the California Environmental Protection Agency, is the g overnmental agency delegated under State law with the authority and responsibility for collecting ambient air quality data as directed by the Clean Air Act (CAA) of 1977 and CAA Amendments of 1990. CARB and local air pollution control districts (hereafter referred to as “local districts”) operate ambient monitoring stations throughout the State. CARB is designated as the Primary Quality Assurance Organization (PQAO) for the entire State with the exception of the ambient air monitoring programs of the Bay Ar ea Air Quality Management District (BAAQMD), the South Coast Air Quality Management District (SCAQMD), and the San Diego County Air Pollution Control District (SDAQMD). Many of the smallest local Districts do not have active air monitoring programs and rel y solely on CARB for the operation of monitoring stations within their jurisdictions.

The TSA was conducted by EPA Region 9 staff from June to September, 2011. The audit evaluates all air monitoring activities since the previous EPA TSA, which was conduc ted during the summer of 2007. The audit team interviewed management and staff on specific aspects of the ambient air monitoring program including network design, field operations, laboratory operations, data handling, quality assurance and quality control procedures. The audit team also inspected some of the CARB

- operated monitoring sites. The site inspections consisted of an interview with the site operator when possible, review of station and instrument logbooks, and evaluation of the station siting with respect to EPA requirements for probe siting (40 CFR 58, Appendix E). The laboratory inspection included a review of the particulate matter program for mass determinations, laboratory analysis for volatile organic compounds (VOC), Organic

Carbon/Elemental Carbon (OC/EC), hexavalent chromium, and carbonyl sample analysis.

Since CARB oversees the quality of data collected by local districts within the CARB PQAO, EPA also reviewed field operations, data management and quality assurance activities at

local districts. For this TSA, it was not possible to evaluate all of the 2 1 local districts within the CARB PQAO that collect ambient air quality data; the audit team revi ewed operations at three

local d istricts : San Joaquin Valley Air Pollution Control Distric t (SJVAPCD), Imperial County3 Air Pollution Control District (ICAPCD), and Mendocino County Air Quality Management District (M e CAQMD). The local districts included in the CARB PQAO have their own organizational structures and these vary depending on the siz e of the local district program.

SJ V APCD was chosen for review because it is the largest local district in the CARB PQAO and has the most significant air quality issues. ICAPCD was chosen as an example of a medium size organization and also because of the unique air quality problems that exist in that air basin. Finally, M e CAQMD was chosen to be representative of the small er districts.

The TSA is part of the oversight system by which EPA ensure s that data collected by state, local, and tribal agencies me et certain minimum data quality objectives. Other assessments

, such as network reviews and performance evaluations

, are also used to collect information on the quality of ambient air monitoring data. These assessments enable agencies to identify and correc t those program elements which may be adversely affecting the quality of ambient air data. The results of the TSA are summarized here and fully described in this report, along with recommended actions to address the findings. The specific actions to be tak en by CARB will be determined through negotiations between EPA and CARB and will be documented in a corrective action plan prepared by CARB.

EPA would like to thank all the staff and management of CARB and the local districts for their support and coope ration during the audit.

A. Program Strengths:

• CARB has extensive experience and expertise in ambient air monitoring

.

• CARB operates a robust audit program

, which benefits the entire State of California

.

• Local districts within the CARB PQAO are committed t o addressing air quality concerns in their areas and see monitoring as the means by which to assess air quality

.

• CARB and the local districts that participated in this TSA are dedicated to collecting credible and defensible air quality data.

• CARB has devel oped good infrastructure for conducting ambient air monitoring.

B. Program Major Findings:

• CARB needs to complete the process of putting a formal PQAO into place . [Finding G1]

• The QA Management Branch does not have the structure and sufficient staff to ma nage QA oversight of the PQAO districts. [Finding G2]

• While progress has been made on updating the CARB QA Manual with a QMP and QAPPs or equivalent documents, the process is behind schedule. [Finding G3]

• Local districts within the CARB PQAO do not have up dated quality system documentation for all activities. [Finding G4]

• Coordination between CARB

, the local districts and EPA needs to be improved. [Finding G6]

• Not all agencies within the CARB PQAO have an approved network plan. The current approach to netwo rk plans d oes not provide for a determination of network adequacy on a statewide basis . [Finding NM1]

• Field sites are operated inconsistently at both CARB and non

- CARB sites throughout the PQAO. [Findings FO1

- 8, IMP5

- 8, MEN4

- 10, SJV5

- 8)4 • Data within the CA RB PQAO are not validated using consistent procedures. (Findings DM2, SJV9, IMP10, and MEN11)

• There are numerous deficiencies in the data certification process for the CARB PQAO, including:

o Not all NAAQS

- compliant data within the CARB PQAO are routinely certified.

o Data certified by CARB for local districts are not typically reviewed or validated.

o Data are routinely certified by local agencies

, but responsibility has not been formally delegated to any local agencies within the State of California. [Findi ng DM6]

• Data uploaded for local districts by CARB’s Air Quality Analysis Section are not consistently validated. Erroneous data have been entered into AQS. [Findings DM5, IMP10, MEN11]

The individual findings are reported in the topic sections of this doc ument and are also summarized in Appendix A.5 TSA ACTIVITIES

In the summer of 2011, EPA Region 9 conducted a Technical System Audit (TSA) of the ambient monitoring program operated by and overseen by CARB. EPA staff interviewed management and staff in three branches of CARB Monitoring and Laboratory Division (MLD) and one branch of the Planning and Technical Support Division (PTSD). The TSA covered the areas of Air Monitoring Network Management, Field Operations, Laboratory Operations, Data

and Data Ma nagement, and Quality Assurance. In addition, the EPA staff reviewed these same areas as implemented by three local districts: SJVAPCD, ICAPCD, and M e CAQMD.

CARB managers and staff were very accommodating to the EPA audit team, making

themselves and thei r staff available for many interviews, procedural reviews and monitoring site

visits. Branch Chiefs interviewed were:

Ken Stroud

– Chief, Air Quality Surveillance Branch (AQSB), MLD Michael Miguel

– Chief, Quality Management Branch (QMB), MLD Cindy Castron ovo

– Chief, Northern Laboratory Branch, MLD Karen Magliano

– Chief, Air Quality Data Branch (AQDB), PTSD Many other individual section managers and staff were interviewed in Sacramento and in the field. We appreciate the fact that CARB gave the EPA audit team access to all key personnel involved in the collection and quality assurance of ambient air quality data.

The EPA regional staff members conducting the TSA were Elfego Felix, Michael Flagg, Katherine Hoag, Meredith Kurpius, and Gwen Yoshimura of the EPA Region 9’s Air Quality Analysis Office, and Mathew Plate and Steve Remaley of the EPA Region 9 Quality Assurance Office. In addition to the EPA Audit Team, Matthew Lakin and Eugenia McNaughton, Managers of EPA Region 9's Air Quality Analysis Office an d Quality Assurance Office, respectively, attended the opening and closing meeting representing EPA management.

The TSA began with a general meeting with CARB managers and staff on June 7, 2011 at the Monitoring and Laboratory Division office in Sacrame nto, CA and continued during the months of June, July, August, and September, 2011. The TSA covered the following program

areas:

• General / Quality Management

.

- Program organization

.

- Facilities

.

- Independent quality assurance and quality control

.

- Planning doc uments (including QMP, QAPPs, & SOPs)

.

- General documentation policies

.

- Training

.

- Corrective action

.

- Quality improvement

.

- External performance audits

.

• Network Management / Field Operations

.

- Network design

.

- Changes to the network since the last audit

.

- Propos ed changes to the network

.6 - Field support

.

• Laboratory Operations: toxics and particulate matter

.

- Routine operations

.

- Quality control

.

- Laboratory preventive maintenance

.

- Laboratory record keeping

.

- Laboratory data acquisition and handling

.

- Specific pollutants : PM 10 , PM 2.5 , and toxics

.

• Data and Data Management

.

- Data handling

.

- Software documentation

.

- Data validation and correction

.

- Data processing

.

- Internal reporting

.

- External reporting

. As part of the TSA, EPA tracked supporting documentation for data points/s ets from calendar year 2010.

This report is divided the following sections:

• Executive Summary

– describes the purpose of the TSA and summarizes the major findings.

• TSA Activities

– outlines the timing of this TSA and the programs that were covered.

• Overv iew of Air Monitoring Program

– describes the District’s Air Monitoring Program.

• Findings

– collection of findings and recommendations that includes details associated with findings.

• Appendix A

– list of findings.

• Appendix B

– CARB organizational charts.

• A ppendix C

– CARB data validation procedures.

The findings and recommendations in this report are grouped by program area. Recommended actions to address findings are provided to give some indication of EPA’s expectations. If CARB or local districts have o ther approaches or alternatives to address the concerns identified, EPA will consider them, provided the corrective action adequately addresses the finding.

Network Management EPA interviewed Karen Magliano, Gayle Sweigert and Pheng Lee, and reviewed

CAR B’s Annual Network Plan and Annual Network Assessment as part of this TSA. The most recent Annual Network Plan was submitted in July, 2011. CARB submitted a five

- year Annual Network Assessment as required by 40 CFR 58.10. Both documents address a portion o f the monitoring network of California ; they present information from the smaller agencies that do not produce their own reports as well as CARB sites. While both documents include the information that is required per CFR, and were approved in 2010 and 201 1 as meeting all of the requirements for annual network plans, the current structure does not provide for a determination of network adequacy or robustness of state

- level network planning. To review network adequacy, EPA reviewed all the network plans avai lable for the CARB PQAO, in addition to site lists that EPA7 has compiled. EPA reviewed the SLAMS monitoring network for the CARB PQAO and determined that the network is adequate for all areas within the CARB PQAO.

Field Operations

The CARB site technicia ns interviewed were Bob Land and Rick Rigsby. Both demonstrated a thorough knowledge of the monitoring equipment for which they were responsible.

EPA visited five of CARB’s monitoring stations (Colusa, Sutter Buttes, Tuscan Buttes, Willows, and Yuba City ). A more thorough evaluation was performed at Colusa, Willows, and Yuba City. The evaluation at these sites included inspection of the inlet manifolds, examination of station and instrument log books, and an evaluation as to whether appropriate QC checks and QA audits were being performed. All visible inlet manifolds appeared to be clean. Station logbooks and instrument logbooks were not consistently used, and entries often did not follow a formal protocol. Some QC checks were not being consistently record ed, and problems were not systematically documented. Generally we found that the station operators were very knowledgeable, but recordkeeping, corrective action, training, and oversight could be improved.

On June 27, 2011 EPA conducted a review of CARB’s instrument testing, certification, and repair procedures. During the review, EPA had the opportunity to interview CARB’s Operation Support Section Manager, Reggie Smith, as well as visit the instrument laboratory and stockroom for spare parts, both of whic h are located at the CARB MLD main laboratory . In general, considering the extent of CARB’s network, EPA found that the agency maintains an excellent instrument testing, certification, and repairs program. Some improvements could be

made to track malfuncti oning equipment so that they may be repaired and reinstalled in order to

reduce offline time

.

Data Management This section covers data management for criteria pollutants (O 3 , CO, NO 2 , SO 2 , PM 2.5 , and PM 10 ); non

- criteria pollutant data are addressed in th e laboratory section. The following managers/staff who have data management responsibilities were interviewed:

Norma Montez

- Air Pollution Specialist, data validator for continuous data for CARB

sites Gayle Sweigert

- AQS data entry for non

- CARB sites

, d ata certification Pheng Lee and Dwight Oda

- AQS data entry for non

- CARB sites

, data certification

Michael Werst

- PM filter data

Quality Assurance/Quality Control EPA interviewed the MLD Quality Management Branch Chief, MLD Air Quality

Surveillance Bra nch Chief, Quality Assurance Section (QAS) Manager, Operations Planning and Assessment Section Manager, Air Monitoring North Section Manager, and staff in the Quality Assurance Section. EPA evaluated a QAS performance audit and site evaluation at the Yuba City monitoring site. Members of the audit team interviewed staff of and reviewed procedures for the CARB Standards Laboratory.8 CARB’s quality management system meets the basic EPA requirements. CARB has a QA Manual that has been approved and is currently being updated. CARB’s core program conforms to or exceeds the method quality objectives systematically developed by EPA for criteria pollutants. EPA and CARB perform national performance and technical evaluations of the monitoring network.

Managers and s taff interviewed included: Mike Miguel, QMB Chief Ken Stroud, AQSB Chief Merrin Wright, QAS Manager Joe Guerrero, Air Monitoring North Section (AMNS) Manager Jeff Wright, Operations Planning and Assessments Section (OPAS) Manager Chris Deidrick, QAS Staff Hien Tran, QAS Staff Patrick Rainey, QAS Staff Harnek Nijjar, QAS Staff

Particulate Matter Laboratory EPA visited two gravimetric particulate matter laboratories and interviewed the following staff as part of the audit:

Michael Werst

- Inorganic Laboratory Section Manager, Ranjit Ahuja

- Air Pollution Specialist, lead PM 10 laboratory technician,

Brenda Saldana

- Air Pollution Specialist, lead PM 2.5 laboratory technician, and

Michelle Fristoe

- Air Pollution Specialist, backup PM 2.5 laboratory technician.

While the laboratory facilities are primarily used to process and weigh PM 10 and PM 2.5 filters, back

- up PM 10 and PM 2.5 balances are maintained at both facilities so that filters may be weighed in either laboratory if issues arise at the prim ary laboratory location. Both of the particulate matter laboratories were well

- maintained, neat, and well

- organized. Generally, the

PM 10 and PM 2.5 laboratory measurements are performed with very good level of technical expertise.

Toxics Laboratory Operati ons The analysts/chemists were found to be knowledgeable, skilled, and dedicated. CARB is in compliance with respect to performing analyses according to EPA methods in most areas of the laboratory. Analysts in the following programs were interviewed:

Qual ity Assurance All the laboratory staff and managers were asked about quality assurance in the laboratory. The laboratory does not have a dedicated QA officer, but most method quality assurance activities are being routinely performed throughout the indiv idual departments. Some corrective actions from the previous TSA were observed , and there were some repeat findings. Staff appeared very receptive to QA findings and suggestions.

Canister Cleaning

- MLD 0209 Sample Custodian Judy Hodgkins (Air Polluti on Specialist) was interviewed. Canister cleaning is generally being performed adequately in accordance with accepted protocols. Findings in this department mostly focused on outdated SOPs that do not reflect current procedure. Procedures to improve qualit y assurance were discussed.

Carbonyls Department

- MLD 022 John Medina is an analyst with many years of analytical experience but is new to carbonyls analysis. Some deviations from method and CARB protocols were identified.

Hexavalent Chromium

- MLD 039 Howard Bakes is responsible for the hexavalent chromium analysis. These analyses are generally being performed according to established protocols. The peer review process described was exemplary.

Aromatic and Halogenated Compounds

- MLD 066 & Oxygenated Hydrocarbons and Nitriles (MLS 058) Steve Madden and John Bricarello were interviewed. These methods were developed by CARB and are generally performed according to protocol. Some opportunities to improve

documentation were identified.

Review o f Agencies within the CARB PQAO

Since CARB oversees the quality assurance of data collected by local districts 1 within the CARB PQAO, EPA also reviewed field operations, data management and quality assurance

activities at local districts . As it was not po ssible to evaluate all of the 2 0 local districts within the CARB PQAO that collect ambient air quality data, the audit team reviewed operations at three local districts: SJVAPCD, ICAPCD, and M e CAQMD.

Each of the local districts included in the CARB PQAO has its unique organizational structure. SJAPCD was chosen for review because it is the largest local district in the CARB PQAO and has the most significant air quality issues. ICAPCD was chosen as an example of a medium size organization and also because of the unique air quality problems that exist in that air basin. Finally, M e CAQMD was chosen to be representative of the small districts.

1 According to the California State and Local Air Monitoring Network Plan (2011), prepared by the CARB

Pla nning and Technical Support Division, Air Quality Data Branch, 21 local Districts operate air monitoring stations in the CARB PQAO.10 OVERVIEW OF THE STATE AIR MONITORING PROGRAM

State and Local Monitoring Agencies within the State of California

CAR B, an organization under the umbrella of the California Environmental Protection Agency, is the governmental agency delegated under State law with the authority and responsibility for collecting ambient air quality data as directed by the CAA of 1977 and C AA Amendments of 1990. Specifically, CAA Section 110(a)(2)(B)(i) directs the State to “ provide for establishment and operation of appropriate devices, methods, systems, and procedures necessary

to ...(i) monitor, compile, and analyze data on ambient air quali ty...”

There are 35 local air pollution control districts in the state of California (Table 1). Three of these local air districts, BAAQMD, SCAQMD, and SDCAPCD are PQAO s. Twenty

- one of the remaining air districts and CARB, comprising the CARB PQAO

, collect ambient air monitoring data

.

A PQAO is a monitoring organization or a coordinated aggregation of such organizations that is responsible for a set of stations that monitors the same pollutant and for which data quality assessments can logically be pooled because they have similar quality systems in place. Specifically, 40 CFR Part 50 Appendix A Section 3 requires that each ambient air monitoring PQAO conform to certain quality management practices. These include:

• Having a documented quality system that me ets EPA requirements for QMPs and QAPPs

.

• Having a quality management function that is independent of air monitoring operations

.

• Developing or adopting DQOs, or equivalent systematic planning procedures, for all monitoring programs

.

• Participating in Nationa l Performance Evaluation Programs, which consist of performance audits used to independently determine program adequacy, national monitoring network performance, and national consistency

.

• Undergoing Technical Systems Audits by EPA at a frequency of every t hree years or less

as needed

.

• Using certified reference materials to standardize monitoring equipment

. EPA views these quality management system components as indispensible to maintain a credible monitoring program. Insufficient quality management and cont rol has been cited as rationale to support legal challenges to NAAQS designation decisions.

CARB oversees the quality assurance of data collected by local districts within the CARB PQAO. Although both CARB and local air pollution control districts operat e ambient monitoring stations throughout the state, responsibility for ambient air monitoring ultimately rests with CARB. Further, it is the responsibility of CARB to provide QA oversight to ensure that data quality within the CARB PQAO meets CFR requireme nts and conforms to quality standards approved in the QAPP.11 Table 1. List of Air Pollution Control Agencies in California

Air Pollution Control District Ambient Air Monitoring by Agency for NAAQS Compliance * PQAO

Amador County (Am)APCD N CARB Antelope V alley (An)AQMD Y CARB Bay Area (BA)AQMD Y BAAQMD Butte County (BC)AQMD N CARB Calaveras County (CaC)APCD N CARB

CARB Y CARB Colusa County (CoC)APCD N CARB El Dorado County (EDC)AQMD N CARB Feather River (FR)AQMD N CARB Glenn County (GC)APCD N CARB Great Basin Unified (GBU)APCD Y CARB Imperial County (IC)APCD Y CARB Eastern Kern (EK)APCD Y CARB Lake County (LakeC)AQMD Y CARB Lassen County (LassenC)APCD N CARB Mariposa County (MaC)AQMD N CARB Mendocino County (MeC)AQMD Y CARB Modoc County (MoC )APCD N CARB Mojave Desert (MD)AQMD Y CARB Monterey Bay Unified (MBU)APCD Y CARB North Coast Unified (NCU)AQMD Y CARB Northern Sierra (NS)AQMD Y CARB Northern Sonoma County (NSC)APCD Y CARB

Placer County (PC)APCD Y CARB Sacramento Metro (SM)AQMD Y C ARB San Diego County (SDC)APCD Y SDCAPCD San Joaquin Valley (SJV)APCD Y CARB San Luis Obispo County (SLOC)APCD Y CARB

Santa Barbara County (SBC)APCD Y CARB Shasta County (ShC)AQMD Y CARB Siskiyou County (SiC)APCD Y CARB South Coast (SC)AQMD Y SCAQMD Tehama County (TeC)APCD Y CARB Tuolumne County (Tu)APCD N CARB Ventura County (VC)APCD Y CARB

Yolo

- Solano (Y

- S)AQMD Y CARB
*In some cases CARB and local agencies share ambient air monitoring responsibilities for areas and sites within a local agency’ s jurisdiction.12 The ambient air monitoring program in the State of California encompasses many air quality assessment activities, including collecting and analyzing data for federal criteria pollutants and many other air pollutants of concern; collecting data from special studies as directed by the Board; assessing monitoring methods that are used by the State and local districts in compliance with federal and s tate regulations; conducting annual performance audits of all monitoring equipment within the PQ AO; implementing a calibration and certification of measurement standards program; and conducting training in the operation of ambient air monitoring instruments.

Organization of CARB’s Ambient Air Monitoring Program

Responsibility for overseeing the am bient air monitoring program for CARB resides within the following branches:

Branch Division Air Quality Surveillance Branch (AQSB) Monitoring and Laboratory Division (MLD) Northern Laboratory Branch (NLB) Monitoring and Laboratory Division (MLD) Quali ty Management Branch (QMB) Monitoring and Laboratory Division (MLD) Air Quality Data Branch (AQDB) Planning and Technical Support (PTSD)

QA responsibility for CARB is covered primarily by the Quality Management Branch (QMB). Michael Miguel is the QMB B ranch Chief. The QMB oversees the development of quality management documents with contributions from other branches, conducts on

- site audits, reviews precision and accuracy data, and initiates corrective action requests ( i.e. , AQDAs), among other duties. The Air Quality Surveillance Branch (AQSB) and Air Quality Data Branch (AQDB) contribute to QA activities primarily through data validation activities. QC responsibility is handled primarily by the AQSB, which includes field operations, calibrations,

and r epair, precision and accuracy data review/submittal, instrument acceptance testing, determination of CARB Federal/State ambient monitoring methods, development of SOPs, interpretation of CFRs pertaining to monitoring criteria/methods, and a variety of othe r monitoring functions.

The management of QA/QC for local districts within the CARB PQAO is specific to each local district. The CARB PQAO does have common QA oversight for instrument audits, since CARB conducts audits of all sites within California, but other aspects of QA/QC are specific to the relationship that CARB has with each local district. In most cases the local district is responsible for QC aspects of the air monitoring program ( e.g. , zero/precision/span checks, calibration, and regular on

- site review and maintenance). CARB repairs/replaces instruments for some districts, while others do this locally. Except for instrument audits, the approach to QA is variable. Many local districts have their own quality management documents; others reference

t he CARB quality management documents. CARB validates the data that it generates ( i.e. , data from their own sites and any filters they weigh), but expects that data generated by local districts

has been validated by the local district, even in the cases whe re CARB enters the data into AQS. For data certification, CARB certifies its data ( i.e. , continuous data collected by CARB that are limited to CARB sites, and filters weighed by CARB, which includes CARB and non

- CARB13 sites) and also data for air districts for which it enters data into AQS. CARB expects that all

other data is certified by the local district.

Network Management Responsibility for network management lies in the Air Quality Data Branch (AQDB). Karen Magliano is the AQDB Branch Chief. Network management responsibilities include network assessment and network plan preparation, analyses and decision

- making for CARB system modifications, and review of requests by local agencies within the CARB PQAO for system modifications. Network management for non

- CARB sites has traditionally been managed by the local agency.

The state network consists of monitoring stations operated by CARB and the local

districts. The network covers 15 air basins. The four PQAOs in the State of California operate monitoring networks that provide data from all the air basins. The three local district PQAOs operate their own monitoring networks that provide data for the following air basins: San Francisco Bay, San Diego County, South Coast, and Salton Sea 2

. The CARB PQAO operat es multiple monitoring n etworks that cover the following air basins: Great Basin, Lake County, Lake Tahoe, Mojave Desert, Mountain Counties, North Central Coast, North Coast, Northeast Plateau, Sacramento Valley, Salton Sea, San Joaquin Valley, and South C entral Coast. In some instances, several local districts operate the monitoring networks in a given air basin. Furthermore, the boundaries of metropolitan and micropolitan statistical areas (MSAs), which are established by the US Census Bureau , may also ov erlap air basins and local monitoring districts. EPA uses the population statistics of MSAs to determine the minimum SLAMS monitoring requirements for criteria pollutants.

Table 2 summarizes the number of criteria pollutant monitoring sites operated in the CARB PQAO.

T able 2: S ummary of C riteria P ollutant M onitors in the CARB PQAO Operating Agency Ozone CO NO 2 SO 2 PM 2.5 3 PM 10 TSP

Lead

CARB 3 4 4 5 12 5 2 38 24 2

A nAPCD 1 1 1 0 1 1 0

BCAPCD 0 0 0 0 1 0 0

GBUAPCD 0 0 0 0 1 17 0

ICAPCD 3 1 1 0 2 6 0

E K APCD 0 0 0 0 1 1 0 LakeCAQMD 1 0 0 0 1 0 0

MeC AQMD 1 0 0 0 2 1 0

M DAQMD 5 2 3 2 1 5 0

M BU APCD 5 1 1 0 6 2 0

2 South Coast AQMD collects ambient air monitoring data in Coachella Valley which is part of the Salton Sea Air

Basin.

3 Inc ludes all FRM/FEM instruments and also non

- FEM BAM instruments.

4 Includes Arvin

– Di Giorgio (060295002) and Shafter (060296001).

5 Includes Shafter (06029601)14 NCUAQMD 2 2 2 2 3 5 0 NSAQMD 1 0 1 0 9 0 0

NSCAPCD 1 0 0 0 0 3 0

PCAPCD 2 0 0 0 3 3 0 SMAQMD 7 6 4 5 1 5 5 1

SJVAPCD 1 3 4 9 0 15 7 0

SLOC APCD 5 0 3 1 3 4 0

S B CAPCD 10 7 5 8 11 9 6 10 2 5 11 0 ShCAQMD 2 0 0 0 4 5 0

SiCAPCD 1 0 0 0 1 1 0

TeCAPCD 1 0 0 0 1 1 0

VCAPCD 5 0 2 0 9 3 0

Y

- SAQMD 2 0 0 0 1 3 0

TOTALS 102 2 5 51 14 110 102 3

Source: CARB/PTSD/AQDB

EPA reviewed the monitoring networ k within the CARB PQAO and found that the number of sites meets minimum monitoring requirements per 40 CFR 58, Appendix D.

In 2006

, EPA added a requirement for Annual Monitoring Network Plans and Five

- year Network Assessments (40 CFR 58.10). At the time when the first network plan was due (July, 2007), numerous local agencies within the CARB PQAO expressed an interest in submitting their own plan. On this basis, an agreement was reached between EPA, CARB and local districts

where in agencies within the CAR B PQAO that wanted to submit their own plan could follow the process and submit a network plan directly to EPA. CARB submitted a network plan to cover those agencies that did not choose to submit a plan (Table 3). The five

- year network assessment

that was due in 2010 followed a similar process. The network plan per 40 CFR 58.14 (a) must

include recommendations from the network assessment.

6 Includes newly established Lincoln site (AQS ID 06

- 061

- 2001).

7 Includes 6 long

- term PSD moni tors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1021, 06

- 083

- 1013, 06

- 083

- 1025, 06

- 083

- 1018, 06

- 083

- 1014, 06

- 083

- 4003).

8 Includes 3 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1008, and 06

- 083

- 4003).

9 Includes 8 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1021, 06

- 083

- 1018, 06

- 083

- 1013, 06

- 08 3

- 2004, 06

- 083

- 1014, 06

- 083

- 0011,

06

- 083

- 1008, and 06

- 083

- 4003).

10 Includes 4 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025, 06

- 083

- 1013, 06

- 083

- 4003, and 06

- 083

- 1020).

11 Includes 2 long

- term PSD monitors that are overseen by Santa Barbara County APCD but operated by contractors (AQS IDs: 06

- 083

- 1025 and 06

- 083

- 4003).15 Table 3. List of Agencies Drafting Annual Network Plans in California.

Source: CARB 2012 Annual Monitoring Network P lan for Small Districts.

R equirements for developing an adequate network are not based on agency jurisdiction,

but rather on Core

- Based Statistical Areas 12 (CBSAs) or PQAOs

. A collaborative approach between CARB, EPA, and local districts is needed to evalu ate whether the requirements for an adequate network continue to be met.

Requests for changes to the network may occur outside the network plan process by the

submission of a letter to EPA. In recent years, CARB and EPA have discussed proposed site change s via conference calls. Once a decision is made, CARB sends a formal request to EPA for

12 The definition of a Core

- Based Statistical Area can be found at http://www. census.gov: Metropolitan and micropolitan statistical areas (metro and micro areas) are geographic entities defined by the Office of Management and Budget (OMB) for use by Federal statistical agencies in collecting, tabulating, and publishing Federal statistics. The term "Core Based Statistical Area" (CBSA) is a collective term for both metro and micro areas. A metro area contains a core urban area of 50,000 or more population, and a micro area contains an urban core of at least 10,000 (but less than 50,000) population. Each metro or micro area consists of one or more counties and includes the counties containing the core urban area, as well as any adjacent counties that have a high degree of social and economic integration (as measured by commuting to work) with the urban core.16 approval. When a local district wishes to modify its network, e.g. , shutting down or relocating an existing site or establishing a new one , it will often consult with C ARB informally before submitting a request to EPA, but the process for local districts has been generally less consistent.

Field Operations Network operations at CARB are primarily performed by the MLD AQSB . Ken Stroud is the Branch Chief of AQSB. AQSB d uties include the operation of CARB monitoring sites, monitoring support for CARB special studies, and general air monitoring support, such as repair and calibration facilities. AQSB also assists local districts with instrument trouble

- shooting and repair as resources allow. This section of the TSA report addresses AQSB's general operations, the calibration program, and field operations of the AQSB at CARB

- operated criteria pollutant monitoring sites.

Providing training and performing instrument certific ations are the responsibilities of the Operations Support Section (OSS) within AQSB. The OSS also provides independent review and approval of field SOPs. Other responsibilities include instrument repair and technical support. While support ( e.g. , training, field procedures, and other technical support) is available to the all local districts in California, the AQSB does not actively manage the local districts’ field monitoring quality systems nor does it have the resources to do so.

Laboratory Operations Analytical laboratories provide support for measurement methods that are either too complex or sensitive to perform in the field environment. In order to provide these services, laboratories have highly trained staff in charge of the complex instrumentati on

. If analyses are to be used for regulatory purposes, they must meet the following criteria:

• Equipment must be frequently and properly calibrated and maintained.

• Personnel must be qualified to make the analysis.

• Analytical procedures must be in accor dance with accepted practice.

• Complete and accurate records must be kept.

The CARB MLD Northern Laboratory Branch (NLB) 13 is divided into three sections: the Inorganic Laboratory Section, Organic Laboratory Section, and the Special Analysis Section.

Cind y Castronovo is the Branch Chief of NLB. The laboratory facility is adequate for NLB’s needs. The laboratory provides analytical support for the criteria pollutants PM 10 and PM 2.5

. Additionally, the laboratory supports the EPA PM Speciation Trends Networ k (STN), the California Air Toxics Monitoring Network, and Special Study Monitoring. CARB’s NLB laboratory facility is located in Sacramento, California. Analyses are performed in

- house by laboratory staff.

Particulate Matter Laboratory (Gravimetric Lab oratory) The gravimetric laboratory operations are managed by Michael Werst, Inorganic Laboratory Section Manager . Analyses are performed in

- house by laboratory staff. The

13 CARB also has a Southern Laborat ory Branch but this laboratory does not handle ambient air monitoring sample and analyses but rather source testing and other non

- ambient samples.17 particulate matter laboratory is responsible for the handling of PM filters, which i ncludes preparation, weighing, tracking, and storing PM 2.5 and PM 10 filters.

Toxics Laboratory In addition to PM responsibilities, the laboratory is also responsible for air toxics monitoring analysis. This TSA focused on the following compounds:

• Carbon yls (Method MLD022 MEK, acetaldehyde)

.

• Hexavalent chromium

.

• Aromatic and halogenated compounds

.

• Oxygenated hydrocarbons and nitriles

.

There are a number of additional activities that the laboratory undertakes to support the collection and analysis of air pollutants. These include canister cleaning and preparation, data validation, and sample storage.

Data Management Data management generally involves data collection and validation supported by a data management system. A primary goal of the EPA’s Quality System is “to ensure that environmental programs and decisions are supported by data of the type and quality needed for their intended use...” ( EPA Quality Manual for Environmental Programs, EPA Order 5360A1
[ EPA, 2000a ]). Achievement of this goal involves planning, implementation and assessment of the data collection process. Data verification and validation are key steps in the assessment of environmental measurements. EPA defines data verification as the process of evaluating completeness, correctness and compliance of a data set against the method requirements. Data validation extends the verification process to determine the analytical quality of a data set. As a part of this TSA, EPA evaluated CARB’s process of data handling, verification, validation,

s torage and upload to AQS of ambient monitoring measurements.

On

- going data collected from ambient air monitoring stations can either be generated by an analyzer in situ (continuous data) or by subsequent laboratory analyses of a sample (lab oratory data). Choosing an appropriate data management process depends on whether the data are generated continuous ly on site or in the laboratory. Within the CARB PQAO, the different agencies collect and generate air quality data. In addition to the on

- going collection of air quality data, periodic QC checks generate data that must also be managed. Table 4 lists which types of data are uploaded by which agency:18 Table 4. Summary of data upload and certification responsibilities for the State of

California.
Agency Operati ng Site Agency Uploading

Continuous Data Certified By? Agency Uploading PM

Filter

- based Laboratory Data Certified By?

An APCD M DAQMD Not certified* MDAQMD Not certified

CARB CARB (MLD

AQSB CARB (AQAS) CARB (NLB) /

VC APCD

- PM 2.5 (2 CARB sites)/SDCAPCD (1 CARB site) CARB (AQAS)/

not certified/not

certified

GBU APCD GBUAPCD GBUAPCD GBUAPCD GBUAPCD

IC A PCD CARB (AQAS) CARB (AQAS) CARB (NLB)

- PM 10 / SD CAPCD

- PM 2.5 CARB (AQAS)/PM2.5

- not certified LakeCAQMD CARB (AQAS) CARB(AQAS) CARB (AQAS) CARB (AQAS)

Me C AQMD CARB (AQAS) CARB (AQAS) N/A N/A

MDAPCD MDAPCD Not certified* M D APCD Not certified NCUAQMD CARB (AQAS) CARB (AQAS) CARB (AQAS)

- PM 10 + / (Bay Area)

- PM 2.5 CARB(AQAS)/

PM 2.5 - not certified

NSAQMD NS A QMD NSAQMD CARB (NLB) CARB (AQAS)

MBUAPCD MBUAPCD MBUA PCD BAAQMD Not certified

NS CountyA PCD CARB (AQAS) CARB (AQAS) CARB (AQAS)

- PM 10 CARB (AQAS)

PCAPCD CARB (AQAS) CARB(AQAS) N/A N/A

SMAPCD CARB (AQAS) CARB (AQAS) CARB (NLB)

– PM 2.5 and PM 10 coarse / CARB (AQAS)

– PM 10 + CARB (AQAS)

PM 2.5 , PM 10 coarse and PM 10 +

S J VAPCD SJVAPCD SJV APCD V CAPCD Not certified

SLO C APCD SLOC APCD SLOC APCD N/A N/A

SBC APCD SBC APCD SBC APCD SBC APCD x SBC APCD

S h CAQMD ShCAQMD ShCAQMD CARB (NLB) CARB (AQAS)

SiCAPCD CARB (AQAS) CARB (AQAS) CARB (NLB) CARB (AQAS)

TeCAPCD CARB (AQ AS) CARB (AQAS) CARB (NLB) CARB (AQAS)

VCAPCD VCAPCD VCAPCD VCAPCD VCAPCD

Y

- SAQMD CARB (AQAS) CARB (AQAS) CARB (NLB) CARB (AQAS)

* O 3 data for 2009

- 2011 was certified by the local district in 2011 only.

+ These data are weighed by local district but up loaded by CARB (AQAS)

x Applies only to filter

- based PM 10 measurements. Continuous sampling will begin in 2012.19 California has five organizational units in two different Divisions of CARB, and 26 separate Air Pollution Control Districts through which ambi ent monitoring data enters EPA’s AQS database. Responsibility for managing the state’s CAA

- required ambient monitoring data is divided between the following groups (descriptions of each group’s role follows the list) : 1. Continuous d ata from CARB

- operated f ie ld m onitoring stations

– MLD/AQSB, Ken Stroud, AQSB Chief

. 2. CARB l aboratory a nalytical d ata for both CARB and non

- CARB Sites

– MLD/NLB

, Cindy Castronovo, NLB Chief

. 3. Quality as surance performance a udit program d ata

– MLD/QAS , Merrin Wright, QAS Manager

. 4. Sp ecial p urpose m onitoring p rojects and Standards Laboratory

– QMB/OPAS , Jeff Wright, OPAS Manager; and MLD/ Special Purpose Monitoring Section (SPMS) , Mac

Mc Douga l l, SPM S Manager

. 5. Local d istrict site data: operated by local d istrict but AQS

- uploaded by CARB

– PTSD/AQAS , Gayle Sweigert, AQ A S Manager

. 6. Local d istrict s ite d ata: o perated and AQS

- uploaded by l ocal d istrict

– various local a ir

p ollution c ontrol agencies

. 7. Local District Site Data: Operated by Local District with Laboratory Analyses by a

Different L aboratory that is not CARB

– responsibility is variable and unclear.

1. Continuous Data from CARB

- operated Field Monitoring Stations

Continuous data collected at CARB

- operated field monitoring stations includes data from all continuous air quality analy zers (O 3 , NO 2 , CO, SO 2 , and non

- filter based PM) and meteorology data. The CARB

- operated ambient monitoring stations are managed by Ken Stroud, AQSB Chief. There are two regional Supervisors, Joe Guerrero (AMNS), and Fernando Amador (Air Monitoring Souther n Section [AMSS]). Air quality data measured by the continuous analyzers at the field stations operated and maintained by CARB are stored in data loggers and station computers. Each station is polled hourly by modem and the data are transmitted directly

to CARB’s central computer system in Sacramento. The computer system consists of a server located within a leased facility and a backup server located in a separate leased facility. The data are collected in the AQDAS, which was developed by EMC, Inc. The AQ DAS (now AQDAS

- II) is CARB’s primary management tool for data collection, validation, and reporting of data

obtained at CARB

- operated stations. Data are retained in AQDAS

- II for 180 days within which time they are uploaded to AQS. Once submitted to AQS, th e data are downloaded to the CARB database Air Data Management System (ADAM). ADAM is CARB’s official state database for ambient air quality data. Chart recorders and data loggers located at each station provide a supplemental record for the data validatio n process; the printouts are stored for four years as primary data records.

At the time of the audit, the updated SOP for data validation had not been finalized but the Air Pollution Specialist, who conducts the second

- level review, provided some overv iew sheets that describe the data review process (see Appendix C). The first review (first

- level validation) of the data is performed by the CARB station operators. Each field operator has password

- protected access to data from his/her own field sites. If data require correction, the station operator makes a notation on the data logger or chart recorder at the station and edits the20 data set. The need for data correction may originate from the QAS in the form of an Air Quality Data Action (AQDA). In these ca ses, the field operator reviews the data and determines and justifies the appropriate action.

The second

- level review is done by an Air Pollution Specialist and/or an Air Resources Engineer; this includes a review of data flags, completeness, QC charts, a udit results, monthly max/mins, and maintenance check sheets. The overview sheets for the second

- level review (Appendix C) indicate that the second

- level review should include data comparisons, such as tracking of pollutants, NO 2 +NO≤NO

x , PM 2.5 <PM 10 , etc. No formal process of data comparison ( e.g. , charts, figures, calculations) was observed . Instead, the second

- level reviewer scans printouts of data and spot checks strip charts. Any data corrections identified in the first

- level

r eview are reviewed and confirmed by the Air Pollution Specialist. In general, the Air Pollution Specialist looks at daily quality control checks, required QA checks ( e.g. , audits), monthly maintenance checks, and outliers. We did not observe any reviews of concentration patterns ( e.g. , seasonal or diurnal) or levels, nor any review of instrument drift. The data stream then proceeds to the next level of review (third

- level validation) by the appropriate Section Manager (Northern or Southern Section) who revi ews data for completeness and considers any significant issues that have been identified by previous

- level review.

At this point, a final data validation summary report

, the monthly data report

, is produced in the form of a memo to the AQSB Branch Chi ef identifying any significant issues for each site and reporting on completeness for all parameters. Upon approval by the AQSB Branch Chief, the data are stored in the state archive system and submitted to the EPA AQS database. AQS is submitted by the dat a validator, Norma Montez, through a password

- protected system on her computer. In summary, data validation for continuous pollutants goes through the following

steps : 1. Review by station operator

. 2. Review by Air Pollution Specialist and/or Air Resources Engi neer

. 3. Review by Section Manager

. 4. Review by AQSB Branch Chief

. 5. Upload to AQS by Norma Montez

.

Overall, CARB submits all required data to the EPA AQS database, including concentrations for all criteria pollutants, and supporting precision and accuracy info rmation. CARB certifies these data annually as required by regulation (40 CFR 58.15).

2. CARB Laboratory Analytical Methods for both CARB and Non

- CARB Sites

Overview: Data flow in the laboratory begins with the chemist, who runs the analytical method and generates the data (gravimetric or chemical analysis). Once collected, all laboratory data are stored in CARB’s Laboratory Information Management System (LIMS). The original LIMS was a product purchased from Perkin

- Elmer, but the system has had many mo difications to customize it for use by CARB over the years. The LIMS database, housed in the Monitoring and Laboratory Division, is backed up to tape once per week. It is accessible to all chemists and managers. The system makes use of limited access and p assword

- protection for security. The21 raw data in the system are stored for five years. LIMS assigns QC flags as defined by CARB SOPs. All data are subjected to peer

- review for level

- two data validation, which is followed by reviewing and ‘locking’ of the d ata by laboratory managers. Data peer

- review groups are organized around the analytical methods: PM 10 , PM 2.5 , PM 2.5 speciation, and TSP

- lead. The QC criteria as written in the laboratory and analytical methods are used for data validation.

Gravimetric (P M) Laboratory:

PM 10 and PM 2.5 filters arrive at the NLB with a Chain of Custody (CoC) form. The gravimetric laboratory handles filters from both CARB and local districts. Samples are linked with a barcode, which is read with a barcode reader. Mass data th at are linked to its barcode are entered automatically from the balance. The chemist enters field information from the CoC form manually, which does not include the mass data. Flags can be identified by anyone on the CoC. The PM data management process inc ludes many useful features including:

• Automatic checks on parameters such as flow and pressure and outlier values are highlighted

.

• Hold times for filters are tracked and priority of use indicated based on hold times

.

• A scheduling report identifying missi ng samples

.

The field operators review instrument operation and note any need to flag data on the CoC. The first level of data validation is done by the station operator. For sample validation, the chemist who weighs the filters does the second level of d ata validation. The chemist verifies sample receipts, information on CoC, correct logging of data into LIMS, and QC data. A monthly data package is generated and provided to a peer chemist ( i.e. , someone not involved in the data generation) for the third

- l evel data validation. After the third

- level data validation is done, the monthly data package, along with any notes on the data, is provided to the Inorganic Laboratory Section Manager for fourth

- level review. The NLB Chief approves the data. The

cover she et on the monthly data package includes a summary of the results of each level of review. Once data are approved by management, they are locked and can only be changed with management approval. Data are uploaded to AQS after approval by NLB Chief.

All PM data weighed and entered by the CARB laboratory is certified by CARB annually as required by regulation (40 CFR 58.15).

Toxics Laboratory: The toxics laboratory follows the same general protocols for data entry and validation as

the rest of the laborator y.

3. C ARB Quality Assurance Data

Quality assurance performance data include state

- wide annual performance audits conducted by CARB and a comparison between CARB and EPA’s audit systems findings . The Quality Assurance Section in the MLD conducts perfo rmance evaluation audits and technical system audits at ambient air monitoring stations throughout the state. Performance audits of each local air pollution control district are conducted annually for gaseous criteria pollutant monitoring and particulate m atter monitoring flow

. The results of the audits are maintained online on the CARB website and are uploaded by CARB to AQS in most cases. In some22 instances, CARB has not received update rights to some local district’s screening files in AQS ; in these cases the local district or EPA uploads the data. EPA conducts an annual comparison with the CARB audit vehicles to ensure comparability with EPA’s National Performance Audit Program (NPAP) and Performance Evaluation Program (PEP) ; these data are uploaded to AQ S by CARB. In addition to collecting and managing data from audits, the QAS reviews quality assurance data for the entire state. At least once a quarter, QAS staff retrieve AMP 255 reports from AQS for all California sites and all pollutants. QAS staff re view the report for inconsistencies and work with each agency to address any issues. QAS also verifies and validates the AMP 255 reports for the data certification process.

4. CARB Special Purpose Monitoring

Special Purpose Monitoring is conducted on an as

- needed basis by the following two sections within MLD: the Operations Planning and Assessment Section, in the Quality Management Branch, and the Special Purpose Monitoring Section, in AQSB. They are responsible for emerging air monitoring issues. In mo st cases, the data are uploaded to AQS. EPA did not interview staff in this section about data management practices.

5. Collected by Local District / uploaded to AQS by CARB

The Air Quality Analysis Section (AQAS) in the Planning and Technical Support

D ivision is responsible for uploading continuous air quality data (O 3 , NO 2 , CO, SO 2 , and non

filter based PM), PM data weighed by the local district, and meteorology data from those local districts without direct access to AQS. These districts include: ICAP CD, L ak CAQMD,

M e CAQMD, NCU A QMD, NSCA P CD, PC APC D , S MA QM D, Si C APCD, T e C APCD, and Y

S AQM D ( Table 4). T he AQAS is located at the Cal EPA Headquarters building in Sacramento, CA and is managed by Gayle Sweigert. AQAS uploads co ntin uous PM data from local distri cts, as well as PM filters data that are weighed by L a ke CAQMD, NCAQMD , NSCAPCD and

SMAPCD. PM filters that are collected by local districts and then weighed by CARB are managed by the CARB MLD Northern Laboratory Branch

.

Data are received electronically by email or as hard copy through the mail from the ten

local districts listed in the previous paragraph . Local districts typically send their data monthly in the form of an electronic file. When data arrive at AQAS, they are logged in and uploaded to AQS, which run s them through AQS review protocols. Staff sends an email to the district staff confirming the upload, which has attached the AQS raw data report and the raw data inventory

report. I f the AQS review protocols identify a potential issue, AQAS sta ff contacts the district to

resolve it . According to the CARB SOP, AQAS staff may not alter data without consent from the district. AQAS staff do not validate the data. CARB assumes that the data to be uploaded to AQAS for local districts have been fully v alidated, and carry the appropriate flags.

Data uploaded by AQAS for the ten districts listed above are certified by CARB annually, as required by regulation (40 CFR 58.15).23 6. Collected and Uploaded to AQS by Local District

The following districts h ave access and authority to upload their data to AQS: GBUAPCD, MDAQMD (includes A n APCD), NSAQMD, MBUAPCD, ShCAQMD, SJVAPCD, SLOCAPCD, SBCAPCD, and VCAPCD (see Table 4). The reporting of data into AQS by these local districts was agreed upon and commemorate d in district

- specific Memorandums of Understanding signed by the districts, CARB, and EPA starting in 2002. All levels of data management are handled by the local districts; CARB is neither involved nor familiar with data management protocols of districts that submit their own data. CARB expects the data that it does not upload to be certified per regulation (40 CFR 58.15) by either the local district collecting the data or by the agency uploading the data.

7. Collected by Local District / Laboratory A nal yses Performed by a Laboratory O ther T han

CARB

All continuous data collection is managed by each local district; CARB uploads data for ten of the districts (see Table 4). Data generation and management for PM filter

- based data are not only district

- speci fic

, but can even be site

- specific. Some districts weigh their filters and upload and certify their PM data. Other districts collect the filters and send them to CARB for weighing, validation, upload, and certification. Other districts collect the filters and send them to another local district that has a weigh laboratory; responsibility for data validation, upload, and certification in these cases is not always clearly defined. Finally, some districts may send filters

from some sites to CARB and filters fr om other sites to a different local district that may be more convenient to the site. In the case where all filters are sent to a different local district, responsibility for data validation, upload, and certification of the filters is not always clearly

d efined. For example, until 2011, filters from a CARB site in the San Joaquin Valley were being sent to and weighed by VCAPCD.

Quality Assurance/ Quality Control

An organization’s quality management system includes quality assurance, quality control,

and quality improvement activities. EPA requires that ambient air monitoring agencies have a quality management system that conforms to 40 CFR Part 58 Appendix A and the EPA quality policy (EPA Order CIO 2106 .0). Additionally, EPA grant regulations specifical ly require each grantee to provide for QA activities (40 CFR Part 31.45). Specifically, 40 CFR Part 58 Appendix A Section 2 requires that each ambient air monitoring PQAO conforms to certain quality management practices. These include:

• A documented quality system that meets EPA requirements for QMPs and QAPPs

.

• A quality management function that is independent of air monitoring operations

.

• S tated data quality objectives or equivalent systematic planning procedures for all monitoring programs

.

• P articipation i n National Performance Evaluation Programs, which consist of performance audits to independently determine program adequacy, national monitoring network performance, and national consistency

.

• EPA

- led Technical Systems A udits every three years or less

.

• U se of certified reference materials to s tandardize monitoring equipment

.24 EPA views these quality management system components as being integral to maintaining a credible monitoring program. Insufficient quality management and control has been cited in

suppor t of legal challenges to NAAQS designation decisions.

Quality assurance and quality control are the two components of a quality management system for a monitoring program that serve to document the assertion that the data collected represent the true ai r quality of the area. They are the means by which an organization manages its quality aspects in a systematic, organized manner to provide a framework for planning, implementing, and assessing work performed by an organization. A properly developed QA/QC program encompasses a variety of technical and administrative elements, including policies and objectives, organizational authority, responsibilities, accountability, and procedures and practices. Quality assurance is a management or oversight function set ting policy and overseeing an administrative system of management controls that cover planning, implementation, the review of data collection activities, and the use of data in decision making. Quality control is a

technical function that includes all the scientific precautions, such as calibrations and duplications needed to acquire data of known and adequate quality.

The CARB Quality Management Branch (QMB) is composed of two sections: the Quality Assurance Section (QAS) and the Operations Planning and Assessment Section (OPAS), which includes the Standards Laboratory. The QAS’s primary responsibilities include:

• Conducting performance audits of MLD and district monitoring instruments

.

• Assisting with system audits of California air districts

.

• Updating standard operating procedures (SOPs) specific to the QAS’s activities

.

• Validating MLD’s field generated monitoring data (accuracy assessments)

.

• Preparing annual reports on the status of QA activities occurring in MLD

.

• Preparing data quality summary repo rts for Reporting Organizations and districts in

California. The OPA section is responsible for providing recommendations to MLD laboratories to ensure the defensibility of the laboratory data. The Standards Laboratory resides in OPA and performs

standar ds certifications for all MLD gaseous, O 3 , flow, and meteorological transfer standards. Most districts within the CARB PQAO choose to employ these services as well.

QC

- related functions are performed by the Air Quality Surveillance Branch (AQSB). The

A QSB performs several quality management functions. These include:

• Developing and administering the training program for instrument operators

.

• Performing instrument acceptance testing

.

• Validating MLD field generated monitoring data

.

• Developing, preparin g and reviewing SOPs for CARB’s air monitoring program

.

• Determining monitoring methods used in CARB’s ambient air monitoring network

.

QA

- related functions in the Northern Laboratory Branch include:

• Developing laboratory and ambient air collection test pr ocedures

.

• Conducting analyses of ambient air samples and consumer products

.25 • Performing self

- assessments quarterly and producing a quality control summary report

for the Division Chief

.

QA

- related functions are divided among CARB’s air monitoring operatio ns. As a result, it can be difficult for the QMB to coordinate QA activities. Moreover, the scope and organization of the various QA activities are not fully understood by the QMB. With the exception of AQDA forms that are issued primarily out of the QAS and the Standards Laboratory in OPA

, and the technical bulletins from the AQSB, corrective action is limited. The process would benefit from expansion in scope and documentation. CARB has all the necessary components for an effective and robust QA system. Each division involved in the collection and reporting of ambient air data understands and performs the relevant QA functions. Expanding the oversight authority of the QMB, and developing and implementing an expanded corrective action process would enhanc e CARB’s QA system.

The QMB staff are not aware of the extent to which QA activities are performed at the local districts. The districts in the CARB PQAO are expected to follow the MLD Quality Assurance Program Plan (QAPP).26 OVERVIEW OF IMPERIAL COUNTY AIR POLLUTION CONTROL DISTRICT AIR

MONITORING PROGRAM

As part of the CARB TSA, EPA reviewed the ambient air monitoring activities of ICAPCD. ICAPCD is currently part of CARB PQAO, but this audit included an agency

- specific assessment of network design, f ield operations, data handling, quality assurance and quality control procedures. EPA staff interviewed ICAPCD management and staff and visited all the monitoring sites located in Imperial County: Calexico Ethel, Niland, Brawley, Westmorland, and

El Centro

.

ICAPCD managers and staff were very accommodating, making themselves available for many interviews, procedural reviews and monitoring site visits. Management and staff

interviewed were:

Brad Poiriez

– Air Pollution Control Officer Reyes Romero

– Ass istant Air Pollution Control Officer

Jesus Rameriz

– Air Pollution Control Division Manager

Monica Soucier

– Air Pollution Control Division Manager

Michael Green

– Air Pollution Control Technician Jon Barroga

– Air Pollution Control Technician

Emmanua l Sanchez

– Air Pollution Control Environmental Coordinator

Overall, the monitoring staff is very dedicated, knowledgeable, and operates the monitoring network to the best of their ability. As described in the attached findings, the major deficiency in the monitoring program is the lack of a quality system for ambient air monitoring. A quality system is the means by which an organization manages the quality of the monitoring information it produces in a systematic, organized manner; it provides a framew ork for planning implementing, assessing and reporting work performed by an organization and for carrying out required quality assurance and quality control activities.

Some of the findings in this TSA pertain to CARB’s role as a PQAO and its relationshi p and oversight responsibilities to local districts. I ncreased communication and coordination

between ICAPCD and CARB is needed to effectively maintain the ambient air monitoring network in Imperial County.

Network Management

There are five monitoring si tes in Imperial County. Four sites are operated by ICAPCD (Table 2).

ICAPCD submits its Annual Monitoring Network Plan directly to EPA. If a local district within the CARB PQAO, such as ICAPCD

, seek s to make changes to its network outside the

Annual Monit oring Network Plan process, the suggested process is for the district to first work with CARB to resolve potential issues and then submit its request to EPA per 40 CFR 58.14 with a copy to CARB

.27 The minimum monitoring requirements as outlined in 40 CFR 58 , Appendix D for PM 10 , PM 2.5 and O 3 are being met for Imperial County

.

Field Operations

Some quality control checks and maintenance are performed in accordance with EPA regulations. Field technicians are responsible for day

- to

- day operations, as well as minor instrument repair and preliminary data validation. The monitoring stations operated by the district are set up to perform automated nightly internal zero/spans (IZS). One

- point precision

checks for O 3 are performed manually, but PM 10 and PM 2.5 flow verifications are not performed by ICAPCD operators.

ICAPCD does not have its own SOPs; staff stated that they use CARB SOPs. Hardcopies of the SOPs are kept at the sites and online via the CARB website. Operators keep track of

unusual events or anomalie s for continuous instruments in the station log, though records at the sites are generally not sufficiently detailed or organized. Any special events or anomalies for

PM 10 and PM 2.5 filters are recorded on the CoC sheet and sent to CARB and SDCAPCD with t he filter, respectively (see Laboratory Operations). Although standard logbooks are not in place,

alternative document ation methods are utilized such as notes on the station calen dar and monthly

maintenance sheets

.

Corrective actions and repair/maintenanc e are dealt with on a case

- by

- case basis. Generally, minor equipment repairs are performed by ICAPCD, while major repairs are performed by the CARB Southern California office in El Monte.

Laboratory Operations

ICAPCD does not operate a PM laboratory ; i t send s PM 10 filters to CARB and PM 2.5 filters to SDCAPCD. ICAPCD uses CARB laboratories for the following analyses: O 3 primary standard verification, protocol gas certification, flow, and meteorological transfer standard calibration, instrument calibration, and major instrument repair.

Data Management

Currently, neither ICAPCD nor CARB are assessing whether the required data quality objectives and measurement quality objectives have been achieved.

ICAPCD station operators perform a preliminary assessm ent of the gaseous and

continuous PM 10 raw data, though this process is generally not documented or performed following an SOP or other procedures outlined in a relevant QAPP. After initial editing is performed, raw text files are sent to CARB for submissi on to AQS. Neither ICAPCD nor CARB perform further ( i.e. Level II) validation of the data.

Filter based PM 10 and PM 2.5 data are processed, validated, and submitted to AQS by CARB Northern Laboratory Branch and SDCAPCD, respectively, and follow procedures outlined in agency

- specific QAPPs and SOPs.28 QA/QC

ICAPCD conducts some QA/QC activities and relies on CARB to support others. QA/QC activities conducted by ICAPCD include one

- point QC checks for gaseous monitors. ICAPCD relies on a CARB site operator to perform the required flow verifications for PM 10 and PM 2.5

. ICAPCD should be conducting these activities, but does not have the equipment required to perform the required checks. CARB also conducts the following QA/QC activities: gaseous annual performanc e evaluations and semi

- annual flow rate audits for PM 10 and PM 2.5, and flow, meteorological ozone, and gaseous standard verifications and certifications

. EPA contractors visit the sites annually, making calibration checks on the PM monitors, as well as con duct ing comparison monitoring using EPA field equipment.

Although ICAPCD staff stated that they are using CARB Quality Management documents, this audit indicated that ICAPCD is not currently following approved CARB QAPPs, SOPs, or approved equivalents. IC APCD does not have a QA manager, a formal corrective action process, or an independent audit program.29 OVERVIEW OF MENDOCINO COUNTY AIR POLLUTION CONTROL DISTRICT

AIR MONITORING PROGRAM

As part of the CARB TSA, EPA reviewed the ambient air monitoring act ivities of

M e CAQMD. M e CAQMD is currently part of the CARB PQAO. M e CAQMD funds the bulk of its monitoring program with district funds. The district does not receive any direct support funding for its monitoring program from CARB. M e CAPCD’s sources of fundin g for air monitoring activities are EPA’s CAA 103 grant support for PM 2.5 ($5000 per site), permit fees, and vehicle fees. Since 2000, there has been a substantial drop in permit revenue, which has resulted in a 40% reduction in staffing, including the eli mination of one full time air quality monitoring technician.

This audit included an agency

- specific assessment of network design, field operations, data handling, quality assurance and quality control procedures. In July 2011, EPA staff

interviewed M e CAQ MD management and staff and visited all four of the monitoring sites located in Mendocino County: the Ukiah AQMD (06

- 045

- 0008), Ukiah Library (06

- 045

- 0006), Willits (06

- 045

- 2002), and Fort Bragg (06

- 045

- 0002).

The MCAQMD manager and staff were very accom modating, making themselves and their staff available for interviews, procedural reviews and monitoring site visits. Management and

staff interviewed were:

Christopher D. Brown

– Air Pollution Control Officer Robert Scaglione

– Senior Air Quality Specia list

Overall, the staff and manager were dedicated and professional , and very knowledgeable about the county and potential pollution sources. As described in the attached findings, the major deficiency in the monitoring program is the lack of a structure d and formalized framework that is inherent to a functioning quality system required for ambient air monitoring. Some of the findings in this TSA pertain to CARB’s role as a PQAO and its relationship and oversight responsibilities to local districts. Incre ased communication and coordination between MeCAQMD and CARB is needed to effectively maintain the ambient air monitoring network in Mendocino County.

Network Management

There are four monitoring sites in Mendocino County , a s identified in Table 5 . All four sites are operated by M e CAQMD.30 Table 5. Ambient Air Monitoring Network in Mendocino County, California

AQS ID Station O 3 PM 10 continuous PM 2.5 continuous

06

- 045

- 0002 Fort Bragg X (POC 2)

06

- 045

- 0006 Ukiah Library X (POC 3)

06

- 045

- 0008 Uk iah AQMD (Gobbi) X (POC 3)

06

- 045

- 2002 Willits X (POC 1)

The M e CAQMD network is included in CARB’s Annual Monitoring Network Plan. If a local district within the CARB PQAO, such as MeCAQMD, seek s to make changes to its

network outside the Annual Monitoring Network Plan process, the suggested process is for the district to first work with CARB to resolve potential issues and then submit its request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements, as outlined in 40 CFR 58, Appendix D for PM 10 , PM 2.5 and O 3 are being met in Mendocino County

.

Field Operations

MCAQMD operates a network of O 3 and PM monitoring instruments. The manager and staff responsible for field operations are Christopher D. Brown (Air Pollution Control Officer) and Robert Scaglione (Senior Air Quality Specialist).

The field technician exhibited a thorough knowledge of equipment operations. Some quality control checks and maintenance are performed in accordance with EPA regulations. The

field te chnician is responsible for day

- to

- day operations as well as instrument repair and maintenance at the assigned stations. CARB performs calibrations of all instruments twice a year and performs audits twice a year.

The monitoring stations operated by th e district are set up to perform automated QC checks. Zero/span checks for O 3 are programmed to occur weekly at 3 a.m., and one

- point precision checks are performed manually about once a week. Leak checks of PM instruments are performed once per month. M CAQMD does not consistently complete and document monthly flow checks on PM instruments.

M e CAQMD uses CARB SOPs, however, hard copies of the SOPs were not available and the staff did not know where electronic versions were kept. The site operator has th e instrument manuals but not the SOPs. Several types of documents are used to track performance and maintenance at the four sites, including station logs, a monthly maintenance and service log for the PM sites, a weekly QA/QC checklist and separate mainte nance log for the O 3 site. Logs and

checklists are archived into binders at the Ukiah M e CAQMD office. If anything unusual is noted, this information is passed along to CARB data validators with the email that conveys the

data.31 Corrective actions and repa ir/maintenance are dealt with on a case

- by

- case basis. Generally, minor equipment repairs are performed by M e CAQMD, while major repairs are performed by CARB.

Laboratory Operations

M e CAQMD relies on CARB for laboratory, calibration, and audit support.

Data Management

Currently, neither M e CAQMD nor CARB are assessing whether the required data quality objectives and measurement quality objectives have been achieved.

M e CAQMD station operators perform a preliminary assessment of the gaseous and

continu ous PM 10 and PM 2.5 raw data, though this process is generally not documented or performed with a predefined set of SOPs or other procedures outlined in a relevant QAPP. Raw text files are sent to CARB for submission to AQS, along with any relevant notes. After AQAS staff has uploaded the data to AQS, a confirmation email is sent to district staff, which has the AQS raw data report and raw data inventory report attached . MCAQMD does not have access to AQS and therefore does not subsequently check what is u ploaded into AQS.

QA/QC

M e CAQMD conducts some QA/QC activities, while relying on CARB to support other s

. QA/QC activities conducted by M e CAQMD include one

- point QC checks for its O 3 monitor.

M e CAQMD does not always perform monthly flow verifications fo r PM 10 and PM 2.5 . CARB conducts the following QA/QC activities: semi

- annual calibrations, semi

- annual audits for O 3 , PM 10 and PM 2.5 , and O 3 , flow, and gaseous certifications and calibrations lab services. EPA

contractors also conduct annual site visits and calibration checks on the PM monitors, as well as comparison monitoring with EPA field equipment.32 OVERVIEW OF SAN JOAQUIN VALLEY AIR POLLUTION CONTROL

DISTRICTAIR MONITORING PROGRAM

As part of the CARB TSA, EPA also reviewed the ambient air monitoring a ctivities of SJVAPCD. SJVAPCD is currently part of the CARB PQAO but this audit included an agency

specific assessment of network design, field operations, data handling, quality assurance and quality control procedures. EPA staff interviewed SJVAPCD mana gement and staff and performed site evaluations at Fresno

- Drummond, Clovis, Tranquility, and Hanford, and made

site visits to Huron and Corcoran.

The SJVAPCD managers and staff were very accommodating, making themselves and their staff available for many interviews, procedural reviews and monitoring site visits. Management and staff interviewed were:

Michael Carrera

– Central Region Compliance Manager

Nathan Trevino

– Supervising Air Quality Instrument Technician Ashley Ross

– Air Quality Instrument Tech nician Olan Bailey

– Air Quality Instrument Technician Errol Villegas

– Air Quality Planning Manager

Steve Shaw

– Supervising Air Quality Specialist, Air Quality Analysis Group Peter Biscay

– Air Quality Specialist, Air Quality Analysis Group

Jennifer Rid gway

– Air Quality Specialist, Air Quality Analysis Group

Overall, the monitoring and data analysis staff are very dedicated and knowledgeable, operating the monitoring network and producing data of known quality to the best of their ability. As describe d in the findings, the major deficiency in the monitoring program is the lack of a structured and formalized framework that is inherent to a functioning quality system required for ambient air monitoring. Some of the findings in this TSA pertain to CARB’s role as the PQAO and its relationship and oversight responsibilities to local districts. Increased communication and coordination between SJVAPCD and CARB is needed to effectively maintain the ambient air monitoring network in the San Joaquin Valley.

Netw ork Management

At the time of this TSA, there were 31 monitoring sites in the San Joaquin Valley

. Twenty

- one sites are operated by SJVAPD, 8 sites are operated by CARB and 2 sites are operated jointly by CARB and SJVAPCD (Table 2). In addition, there are two sites operated by the National Park Service and one site operated by the Tachi Yokut Tribe on the Santa Rosa

Rancheria.

SJVAPCD submits its own Annual Monitoring Network Plan directly to EPA that addresses all the sites within its jurisdiction. In g eneral, CARB and SJVAPCD have worked informally to address some network modifications; finding SJV4 addresses the deficiencies in this process. If a local district within the CARB PQAO, such as SJV APCD, seek s to make changes to its network outside the Ann ual Monitoring Network Plan process, the suggested33 process is for the district to first work with CARB to resolve potential issues and then submit its request to EPA per 40 CFR 58.14 with a copy to CARB.

The minimum monitoring requirements as outlined in 40 CFR 58, Appendix D for O 3, NO 2 , CO, PM 2.5 , PM 10 , PAMS and NCore are being met in the San Joaquin Valley . However, there are outstanding network modifications that have not been approved for the following sites: Corcoran, Bakersfield

- Golden State Highwa y, and Arvin

- Bear Mountain Road (CARB site).

Field Operations

SJVAPCD operates a network of O 3, NO 2 , CO, PM 2.5 , PM 10 , and PAMS monitoring instruments. The following manager(s) and staff are responsible for Field Operations:

Michael Carrera

– Central Regi on Compliance Manager

Nathan Trevino

– Supervising Air Quality Instrument Technician

SJVAPCD field technicians interviewed exhibited a thorough knowledge of equipment operations and an interest in producing high quality data that meet all the regulatory

r equirements. Most quality control checks and maintenance are performed in accordance with EPA regulations. Field technicians are responsible for day

- to

- day operations as well as instrument preventive maintenance and minor repairs at their assigned station s. If the repairs are major and cannot be completed by the staff at the headquarters office, the instruments are sent to the manufacturer. The senior air quality instrument technician performs calibrations of the monitors at the required frequency. The mo nitoring group schedules routine maintenance and calibrations together to reduce instrument downtime between modifications to an instrument and the required follow

- up calibration. SJVAPCD relies primarily on CARB’s Standards Laboratory

for O 3 , flow, and g aseous certifications and calibrations and instrument manufacturers for major instrument repair.

The monitoring stations operated by the district are set up to perform daily automated QC checks on gaseous instruments using certified standards, which excee ds EPA requirements. Flow verifications are performed quarterly for manual PM 10 samplers, semiannually for manual

PM 2.5 samplers, and biweekly for continuous PM 2.5 and PM 10 samplers.

All stations maintain log books to document site visits, preventive maintenance, resolution of operational problems, and corrective actions taken. Logbooks were generally detailed, but what is recorded could be more standardized. Operators archive station logbooks at the central monitoring office. Other station records include QC checklists and maintenance sheets that are also archived at the monitoring station and at the central office. All necessary calibration information is available to the field operators.

Corrective actions and repair/maintenance are dealt with o n a case

- by

- case basis in a timely fashion; however, there is no formalized corrective action process to document decisions or solutions and help communicate them to all field personnel.

The SJVAPCD does not operate a laboratory ; i t relies on VCAPCD for P M weighing laboratory support and on a contract laboratory for PAMS analyses.34 Data Management

SJVAPCD manages all of the ambient monitoring data generated and uploaded to AQS by the district. Data quality objectives and measurement quality objectives have been defined for SJVAPCD’s program. Station operators perform data collection and sample handling according to specific SOPs for most pollutants and the first level data validation for their

stations. S enior staff and supervisor observe activities o n an ongoing basis to ensure full implementation of current and recently changed procedures.

The supervising air quality instrument technician verifies and validates data through level two and the supervising air quality specialist and staff in the Air Qu ality Analysis group perform level three validation. Based on a Memorandum of Understanding with the district, CARB and EPA, SJVAPCD submits gaseous and continuous PM data to AQS. Prior to submitting the data to AQS, Air Quality analysis staff complete a two

- page checklist to document the review

elements. Filter based PM 10 and PM 2.5 data are processed, validated, and submitted to AQS by VCAPCD that follows procedures outlined in its agency specific QAPPs and SOPs.

QA/QC SJVAPCD conducts some QA/QC act ivities while relying on CARB to support others. QA/QC activities conducted by SJVAPCD include: zero, precision and span checks, routine maintenance and calibrations. CARB conducts the following QA/QC activities: annual performance audits and NPAP audits for gaseous instruments and one of the semiannual flow audits of PM instruments. EPA’s contractor performs the remaining semiannual flow audit and

the PM 2.5 - PEP audits. Starting in January 2012, CARB began to conduct both of the semiannual

flow audits. T he SJVAPCD follows its own SOPs, but does not have SOPs for some activities,

nor up

- to

- date QMP or QAPPs. The SJVAPCD does not have an independent QA manager responsible for overseeing the agency’s Quality System. However, the district does provide

some i ndependence in the activities that would typically fall to a QA manager by having operations and calibrations performed by different people. In addition, the third level data review and validation are separated from field operations and data collection by two levels of management. CARB’s audits, which include siting evaluations, and TSAs to be performed by CARB also provide independent oversight of the district’s operations.35 FINDINGS

Program Area Finding

Numbers EPA Contact

General G1

- G6 Audit Team

, Meredith Kurpius (lead) Network Management NM1

- NM3 Meredith Kurpius Field Operations FO1

- FO17 Gwen Yoshimura Data Management DM1

- DM9 Meredith Kurpius QA Management QA1

- QA7 Mathew Plate PM Laboratory PM1

- PM4 Michael Flagg Toxics Laboratory TL1

- TL21 St eve Remaley

ICAPCD IMP1

- IMP10 Michael Flagg MeCAQMD MEN1

- MEN12 Gwen Yoshimura

SJVAPCD SJV1

- SJV12 Kate Hoag36 Finding # G1 Agency: CARB

Date of Audit: Summer 2011 Program Area: General

Finding: [Previous Finding M1] CARB needs to complete the proc ess of putting a formal PQAO into place. Description: CARB has taken steps to strengthen the CARB PQAO by:

• Appointing a PQAO contact

.

• Improving the field audit and technical audit program of PQAO districts

.

• Beginning to provide QA training

.

• Reviewing PQ AO districts’ quality control data prior to routine data certifications

.

• Beginning to review PQAO districts’ SOPs

.

• Starting a process to put in place agreements with PQAO districts

.

• Evaluating and controlling the standards used by the PQAO through the stan dards laboratory and during technical audits

.

The CARB PQAO is able to produce data of known quality that can withstand legal and technical challenges to state and Federal regulatory decisions.

In order to complete the process of integrating CARB’s PQAO districts into a formal PQAO, the organization should be defined in greater detail. It should be noted that a PQAO can only be created and maintained if the organization conforms to the five criteria defined by EPA regulation (see 40 CFR 58, Appendix A, 3 .1 )

.

CARB has begun to define the organization of the PQAO by identifying contacts and performing outreach to the PQAO districts. In order to fully define the PQAO CARB must:

• Formally identify which districts, monitoring sites, and pollutants are include d

.

• Complete the process of having formal agreements in place between the districts and

CARB

.

• Develop and implement an organized and comprehensive training program to support the CARB PQAO

.

• Complete the CARB Quality Management Plan that defines PQAO organiz ation, roles,

and activities

.

In order to strengthen the PQAO so that it produces data of known and consistent quality, CARB should continue working to meet the five criteria. Below is a summary of the work to which CARB has committed to achieve this goa l.

(1) Although all field operators are not CARB staff, CARB can continue to take steps to ensure that all PQAO field operators have the benefit of access to background information and support37 by:

• Implementing routine training programs that are available to all personnel in the PQAO

.

• Increasing the level of technical support that is available to PQAO districts

.

• Enhancing communication between CARB and the PQAO districts

.

(2) The CARB PQAO has a “universal” QAPP and SOPs. In order to ensure that the pro cedures described are consistently followed throughout the PQAO, CARB should:

• Continue to update these documents and inform and train PQAO staff on changes

.

• Continue to review and approve SOPs from PQAO Districts and make these SOPs

available to the entire PQAO

.

• Continue to evaluate adherence of PQAO districts to the QAPP and SOPs

.

(3) The CARB PQAO has a standards laboratory that is available to all districts, but some PQAO standards are certified by outside sources. To ensure comparable standards throug hout the PQAO, CARB should:

• Continue to inventory all the standards and their traceability used by the PQAO

.

• Continue to evaluate the performance of standards sent to the standards laboratory and issue corrective actions as necessary

.

• Determine the need t o consolidate some of the standards/standard certifications used by the PQAO in order to promote consistency and save resources

.

(4) The CARB PQAO has a common QA/QC evaluation group. However, most of the PQAO districts do not have QA support staff excep t those available from CARB. In order to meet the criteria for a common QA, CARB needs to:

• Create a line of QA communication between PQAO staff and QA staff that is separate from the audit process

.

• Continue to work on corrective action processes that PQAO district staff can use to elevate QA issues to CARB’s QA program

.

• Ensure that consistent data validation procedures are in place

.

(5) The CARB PQAO does not have support of common management, headquarters, or laboratory facilities, with the exception of some analytical laboratory analyses performed by the MLD laboratory for some districts. CARB should promote common management practices by:

• Creating standards for oversight of monitoring stations and operations

.

• Providing training to monitoring managers

. References: 40 CFR Part 58, Appendix A , Section 3.1 : Primary Quality Assurance Organization. A primary quality assurance organization is defined as a monitoring organization or a coordinated aggregation of such organizations that is responsible

for a set of stations that monitors the same pollutant and for which data quality assessments can logically be pooled. Each criteria pollutant sampler/monitor at a monitoring station in the SLAMS network must be associated with one, and only one, primary quality as surance organization.

40 CFR Part 58, Appendix A, Section 3.1.1 :38 Each primary quality assurance organization shall be defined such that measurement uncertainty among all stations in the organization can be expected to be reasonably homogeneous, as a resul t of common factors. Common factors that should be considered by monitoring organizations in defining primary quality assurance organizations include: (a) Operation by a common team of field operators according to a common set of procedures; (b) Use of a c ommon QAPP or standard operating procedures; (c) Common calibration facilities and standards; (d) Oversight by a common quality assurance organization; and (e) Support by a common management, laboratory or headquarters.

QA Handbook Volume II

, Section 4.2 : Appropriate training should be available to employees supporting the Ambient Air Quality Monitoring Program, commensurate with their duties. Such training may consist of classroom lectures, workshops, web

- based courses, teleconferences, vendor provided, a nd on

- the

- job training. Recommendation to Address Finding: CARB should formalize the PQAO and develop regular communications with the local districts to ensure that the five PQAO criteria are being met.39 Finding # G2 Agency: CARB

Date of Audit: Summe r 2011 Program Area: General

Finding: The QMB does not have the structure and staff to manage QA oversight of the PQAO districts. Description: CARB should provide QA oversight of local district air monitoring programs. The designation of the QMB Ch ief as the primary QA contact for the PQAO districts would clearly indicate that the authority lies with CARB. Formal agreements between the districts and CARB are needed to support this authority, as noted in Finding G1. In order to meet these needs, the QMB will need to develop a corresponding organizational structure and staff expertise.

References: 40 CFR Part 58, Appendix A : 2.2 Independence of Quality Assurance. The monitoring organization must provide for a quality assurance management function

- that aspect of the overall management system of the organization that determines and implements the quality policy defined in a monitoring organization's QMP. Quality management includes strategic planning, allocation of resources and other systematic plan ning activities ( e.g. , planning, implementation, assessing and reporting) pertaining to the quality system. The quality assurance management function must have sufficient technical expertise and management authority to conduct independent oversight and

ass ure the implementation of the organization's quality system relative to the ambient air quality monitoring program and should be organizationally independent of environmental data generation activities. Recommendation to Address Finding: The QMB should d evelop the expertise, organization, and tools to oversee the CARB PQAO effectively. Corrective action to address this finding could include, but is not limited to, hiring a PQAO coordinator, assigning CARB staff responsibilities of overseeing activities wi thin the CARB PQAO, conducting TSAs of the local districts within the CARB PQAO, and developing network planning and data validation tools for use by CARB and local districts within the CARB PQAO.40 Finding # G3 Agency: CARB

Date of Audit: Summer 2011 Program Area: General

Finding: [Previous Finding M6] While progress has been made on updating the CARB QA Manual with a QMP and QAPPs or equivalent documents, the process is behind schedule. Description: The CARB QA Manual was regularly updated until 2007. Based on EPA’s TSA finding in 2007, CARB agreed to update or replace the QA Manual with a document that conformed to the requirements of the EPA QA system. In order for CARB’s system to be up

- to

- date, complete, and useful, current QA planning docu ments are needed . In addition, QAPPs/SOPs should be revised when standards or instruments change.

References: In accordance with 40 CFR Parts 31 and 35, grant recipients are required to document their quality systems. Specific ambient air monitoring req uirements are found in 40 CFR, Part 58, Appendix A, Section 2.1, “EPA Requirements for Quality Management Plans (QA/R

- 2)”,

EPA/240/B

- 01/002, March 2001 and “EPA Requirements for QA Project Plans (QA/R

- 5)”,

EPA/240/B

- 01/003, March 2001.

Recommendation to A ddress Finding: CARB should complete updating QA planning documents as soon as possible . SOPs should be updated and submitted with updated QAPPs, following a clear timeline. The QMP and QAPPs should be reviewed and resubmitted to EPA for approval every 5 years.41 Finding # G4 Agency: CARB

Date of Audit: Summer 2011 Program Area: General

Finding: Local districts within the CARB PQAO do not always have updated quality system

documentation for all activities. Description: Quality system documents in clude Quality Management Plans (QMPs), Quality Assurance Project Plans (QAPPs), and Standard Operating Procedures (SOPs). Local districts within the CARB PQAO can either adopt CARB’s quality system documents or prepare their own. Not all

local districts wi thin the CARB PQAO have their own approved quality system documents or use CARB’s (see Findings MEN1, IMP1, and SJV2)

References: 40 CFR 58 App. A 2.0, Quality System Requirements QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002

Further guidance on developing QAPPs can be found in the guidance documents “EPA Requir ements for Quality Assurance Project Plans,” EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002 Recommendation to Address Finding: Each local district within the CARB PQAO should have its own quality system documents approved by CARB or formally adopt the CARB quality system documents. CARB should oversee adoption and approval of quality system documents.42 Finding # G5 Agency: CARB

Date of Audit: Summer 2011 Program Area: General

Findi ng: [Previous Findings QM1 and M3] QA Authority and interactions between QMB and the other branches should be expanded and formalized. The corrective action system should be developed to include actions taken, in addition to reports issued by the QA audit ors and the calibration laboratory. Description: Based on feedback and observations made during the audit, CARB MLD is relying on the QMB to provide independent QA leadership. In order for the QMB to fulfill this role, the other MLD

branches should ackn owledge the QMB’s QA authority and staff people should be able to raise QA issues to the QMB. The QMB should be able to exercise QA authority and oversight in a judicious and cooperative manner.

The QMB should be involved in:

• Planning air monitoring activ ities and programs

.

• Overseeing the implementation of monitoring

.

• Evaluating monitoring data and programs

.

In addition to QA/QC support, the specific tasks that must be conducted by the QA independently are:

• Implementation of the QMP

.

• Review and approval of QAPPs and other monitoring plans

.

• Review and approval of QA components of SOPs

.

• Approval of formal corrective actions

.

• QA system training

.

• Documentation of required training

.

• Performing periodic internal audits (performance, technical, and data)

.

• Revie w of data quality summaries and/or control charts (including AMP255 reports)

.

• Evaluation of data validation process/reports

.

• Evaluation of final data used to make regulatory decisions

.

Several specific issues were noted that should be addressed and may be indicative of the broader issue of the QMB’s role in providing independent QA.

• The QMB Chief was not fully exercising the full extent of his authority and oversight over the AQSB

.

• The AQSB was hesitant to characterize the QMB’s role in special projects as oversight

.

• The QMB does not have approval authority for SOPs produced by the other MLD

branches

.

• Updates to the new QAPP sections requested by QMB from the other branches have not43 been completed

.

• New monitoring projects were initiated without QMB involvem ent in the planning

process

.

• During field audits, the auditors perform instrumental tasks that are the responsibility of the station operators.

The CARB QMB has expanded the corrective action (AQDA) process to include calibration laboratory and siting. However, the CARB PQAO has not established a corrective action process that is comprehensive and can be initiated by CARB or district staff. When a significant quality problem or area for improvement is identified, there should be a formal process to ensu re that the problem is addressed throughout the PQAO. The process should be “blind” to the initiator; it should allow for bottom

- up, non

- punitive initiation of formal corrective actions.

S everal issues identified by staff should have been elevated as for mal corrective actions requiring systematic changes (see specific findings).

References: 40 CFR Part 58, Appendix A

See Finding 2, References

EPA QA/R

- 2

3.11 QUALITY IMPROVEMENT Purpose

– To document how the organization will improve the organizatio n’s quality system.

Specifications

– Identify who (organizationally) is responsible for identifying, planning, implementing, and evaluating the effectiveness of quality improvement activities and describe the process to ensure continuous quality improveme nt, including the roles and responsibilities of management and staff, for ensuring that conditions adverse to quality are prevented and identified promptly including a determination of the nature and extent of the problem.

Recommendation to Address Findi ng: The QMB role/authority, including its independence, should be formalized and detailed in the CARB QMP. Additionally, it is recommended that the QMB approach QA tasks in a manner that balances independence and cooperation.44 Finding # G6 Agency: CA RB

Date of Audit: Summer 2011 Program Area: General

Finding:

Coordination between CARB and districts and EPA should be improved. Description: Several findings identified during this TSA relate to insufficient coordination and

communication between CARB and the local districts within the CARB PQAO, including:

• Previously unreported, but valid PM 2.5 samples for ICAPCD were found at SDCAMD that impacted a regulatory decision

.

• Issues with CARB data validation were identified for ICAPCD and MeCAQMD

.

• Distr ict staff sometimes lack ed knowledge of QMP, QAPPs, SOPs

.

CARB and the local agencies must take ownership of the data quality and work together to develop processes to avoid the recurrence of problems.

References:

Recommendation to Address Finding:

CARB should d evelop a process to routinely share information with districts ( e.g. , a PQAO listserve). CARB and the local districts should create a mechanism for resolving issues between the agencies in a well

- documented and transparent manner and a rticul ate clear expectations of the roles and responsibility of all the agencies in the PQAO through an MOU, as suggested by EPA Region 9’s PQAO strategy, to provide a framework for developing such processes.45 Finding # NM1 Agency: CARB

Date of Audit: Summer 2011 Program Area: Network Management

Finding: Not all agencies within the CARB PQAO have approved network plans since this became a requirement in 2006. The current approach to network plan s does not provide for a determination of network adequacy o n a statewide basis. Description: There are 35 local air pollution control districts in the state of California ( see Table 1) in addition to CARB. Three of these local air districts, BAAQMD, SCAQMD, and SDCAPCD are their own PQAO and submit annual monit oring network plans. The remaining 32 districts are within the CARB PQAO. Twenty

- one air districts plus CARB collect ambient air monitoring data under the CARB PQAO. In 2012, nine of the districts within the CARB PQAO prepared and

submitted their own ann ual monitoring network plan. CARB prepared and submitted an annual monitoring network plan for the remaining local districts in California and for its network. All districts in California except for MDAQMD/AVAPCD were covered in a network plan in 2011.

In the past, not all local districts within the CARB PQAO that had assumed responsibility for submitting an annual monitoring network plan have fulfilled the obligation ( e.g. , NSAQMD, MDAQMD, and AVAPCD ) . As a result, regulatory monitors in these areas have b een operated for some period without an approved annual monitoring network plan. However, data quality did not appear to be compromised for these periods.

Although the network plans for California have been approved by Region 9, the current system

of mult iple network plans produces information that cannot be easily combined. Since monitoring network requirements often span multiple districts, plans that contain inconsistent information do not provide for a determination of network adequacy on a statewide b asis, which is required as part of the annual monitoring network plan process.

References: 40 CFR 58.10(a) addresses network plan requirements. Recommendation to Address Finding:

CARB should compile an overlay of information from the annual monitorin g network plans that are submitted by local districts in the State of California. To accomplish this, it is suggested that:

• All local districts that submit annual monitoring network plans for the State of California provide a copy to CARB no later than the time of the annual monitoring network plan

submittal.

• CARB compile information from the plans addressing key requirements that apply across multiple districts ( e.g. , minimum monitoring requirements and co

- location requirements).

o T he summary should be crea ted between the deadline for Annual Network Plan submittal (July 1) and 30 days prior to annual monitoring network plan approval (Nov. 1)

.

o A fter receiving network plans, CARB should work with districts to reduce missing/deficient information.46 o I f the summa ry contains information that is different from that submitted in annual monitoring network plans, CARB should provide an opportunity for public

comment.

• EPA review the submitted plans and the CARB summary and document the basis for approval/disapproval dec isions.

• If a district with responsibility for submitting a plan does not fulfill its obligation, CARB provide the required information for that year.47 Finding # NM2 Agency: CARB

Date of Audit: Summer 2011 Program Area: Network Management

Finding:

T he network assessment does not meet all CFR requirements. Description: Several districts submit separate network assessments for the State of California. Since requirements for the ambient air monitoring network extend beyond the boundaries of local

dist ricts, the assessment must be done at a multi

- jurisdictional level. At the time the report was drafted, CARB’s network assessment for small agencies found that the minimum monitoring requirements were met, the monitoring objectives d efined in appendix D we re met, all operating sites were critical for the implementation of State and federal air quality standards, and none were proposed to be discontinued. The CARB network assessment for small agencies did not address whether new sites were needed, whether e xisting sites were no longer needed and could be terminated, or whether new technologies were appropriate for incorporation into the ambient air monitoring network, as required by CFR.

References: 40 CFR 58.10(d) addresses network assessments and states: “The State, or where applicable local, agency shall perform and submit to the EPA Regional Administrator an assessment of the air quality surveillance system every 5 years to determine, at a minimum, if the network meets the monitoring objectives defined in appendix D to this part, whether new sites are needed, whether existing sites are no longer needed and can be terminated, and whether new technologies are appropriate for incorporation into the ambient air monitoring

network.”

Recommendation to Address Finding:

CARB should conduct its own state

- wide assessment. Alternatively, CARB could develop a process to compile and synthesize information from network plans from local districts into a comprehensive network plan that addresses the CFR requirements.48 Finding # NM3 Agency: CARB

Date of Audit: Summer 2011 Program Area: Network Management

Finding:

There are PM 10 monitors listed in local conditions (LC; parameter code 85101), but not Standard Temperature and Pressure (STP; parameter code 81102 in AQS), as required for FRM/FEM

instruments. Description:

All PM 10 measurements collected with FRM/FEM instruments are required to be entered into AQS as STP (parameter code 81102). It is acceptable to report data under both LC and STP parameter codes. T he following CARB monitors were identified as entered under only the LC

code:

• South Lake Tahoe (060170011), POC 2

.

• Mojave

– Poole (060290011), POC 3

.

• Bakersfield

– California (060290014), POC 5

.

• Paso Robles (060792004), POC 2

.

• San Luis Obispo (060794002), POC 3

.

• Santa Barbara (060830011), POC 1

.

• Santa Maria (060831008), POC 2

. The following non

- CARB sites that are within the CARB PQAO were identified as being entered under only the LC code:

• Brawley (060250007), POC 3, Imperial County APCD

.

• Niland (060254004 ), POC 3, Imperial County APCD

.

• Corcoran (060310004), POC 7, San Joaquin Valley APCD

.

• Madera (060392010), POC 3, San Joaquin Valley APCD

.

• Lakeport (060333001), POC 2, Lake County AQMD

.

• Anderson Springs (060333010), POC 1, Lake County AQMD

.

• Glenbrook (06033 3011), POC 1, Lake County AQMD

.

• Nipomo (060794002), POC 2, San Luis Obispo County APCD

. References: 40 CFR 50, Appendix J, Section 2.2 (Note that there are no CFR regulations that specify operation and data treatment of PM 10 FEM instruments. In the absen ce of regulations specific to

PM 10 FEM instruments, they must adhere to requirements for FRM instruments, unless stated otherwise.) Recommendation to Address Finding: CARB should change or add the parameter code for the CARB sites listed above to STP ( i. e ., parameter code 81102). CARB should work with non

- CARB districts within its PQAO to have

the non

- CARB sites listed above changed to STP.49 Finding # FO1 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field

Finding: [Related Previous Fin dings GB3, SJV3, & NS2] Documentation at the CARB field sites is inadequate and not reviewed by management. Description: The level and consistency of documentation at the AQSB managed field stations was inadequate to reconstruct the monitoring that was c onducted and to resolve definitively the data quality

issues identified.

AQSB site operators use a variety of different documents to record information pertaining to site operations. These include station logbooks, station maintenance sheets, paper strip charts, and data report sheets. Field staff also do not consistently document when they notice something out of the ordinary about the site that could impact readings (construction, weather).

Several specific issues regarding documentation were noted :

• T here is no clear direction as to where information regarding instrument issues that could impact the validity of data is recorded and how such information is transmitted to the data

validators.

• It is not clear that information recorded by the site operator s on monthly data reports is

retained as an official record.

• There is not recent evidence

that there has been management review of the documentation produced by the station operators.

• Initials do not routinely accompany entries.

• Use of pencil and erasing of

records was observed.

• Use of white

- out on Chain of Custody forms was observed.

• Entries in logbooks are incomplete, without sufficient information as to who was present at the site, serial numbers of problematic instruments, descriptions of actions taken , and how much data could be impacted.

• There are no field maintenance logbooks for instruments. Logs are kept at the repair shop.

MLD should develop a consistent approach to site documentation and review. This may involve50 a short

- term solution to improve documentation consistency and completeness and a long term solution to convert all site documentation to electronic records that can be more efficiently produced, reviewed, and incorporated into the data validation process.

References: QA Handbook Volum e II, Section 5

2003 NELAC Standard (Quality Systems Section), which suggests documentation required for all aspects of ambient air monitoring operations, provides the following guidance:

5.5.5.5 The laboratory shall maintain records of each major item of equipment and its software significant to the environment tests performed. The records shall include at least the following: a) The identity of the item of equipment and its software. b) The manufacturer’s name, type identification, and serial number or othe r unique

identification. c) Checks that equipment complies with the specification (see 5.5.5.2). d) The current location. e) The manufacturer’s instructions, if available, or reference to their location. f) Dates, results and copies of reports and certificates of all calibrations, adjustments, acceptance criteria, and the due date of next calibration. g) The maintenance plan, where appropriate, and maintenance carried out to date ; documentation on all routine and non

- routine maintenance activities and reference

material v erifications. h) Any damage, malfunction, modification or repair to the equipment. i) If available, condition when received (e.g., new, used, reconditioned). Recommendation to Address Finding: Field documentation should be improved and a process developed and implemented to provide defensible electronic documentation. See Finding SJV7 Description section for recommendations as to the type of standard information that could be included in logbooks.51 Finding # FO2 Agency: CARB

Date of Audit: Summer 2011 Prog ram Area: Field Operations

Finding: Management oversight of site operators needs strengthening. Description: As stated in Finding FO1, the site operators are not consistently following EPA guidance for regulatory ambient air quality data collection. Due in part to the geographic extent of the network, management oversight of the site operations is especially challenging. Nonetheless, procedures for management controls are needed to ensure that site operations produce robust data for regulatory decisio ns. References:

Recommendation to Address Finding: EPA recommends that managers develop checklists and conduct regular site visits. It may be helpful for the managers to participate in routine training for field operators so the knowledge

base is simi lar. EPA also recommends using the logbooks and other records ( e.g. , maintenance logs and calibration sheets) as oversight aids. To this end, electronic records may be useful.52 Finding # FO3 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: CARB field operators have not been trained on new SOPs. Description: CARB field operators were generally proficient with the procedures they use to conduct their monitoring activities; however, field operators did not always unde rstand why it was important to follow specific protocols and were found to be lax in following requirements in some instances. Further, it was noted that training and/or demonstration of proficiency was not adequately documented.

References:

Recommenda tion to Address Finding: CARB should develop a formal system to ensure and document that all staff are familiar with the quality management system and are trained and proficient at the monitoring tasks that they are performing. Such a system could includ e trainings for field staff when SOPs are developed or revised; periodic refresher courses; monthly site operator meetings; regular manager visits to sites; standard logbooks that get checked and signed off on regularly.53 Finding # FO4 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding:

Residence time calcul ations were not available at any CARB sites visited. Description: Residence time is defined as the amount of time it takes for a sample of air to travel from th e opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 states that for the reactive gases (O 3 , NO 2 , and SO 2 ) residence times must be less than 20 seconds. Additionally, it is recommended that the residence time within t he manifold and sample lines to

the instruments is less than 10 seconds. The station technicians should calculate the residence time, document it in the station logbook, and periodically verify the data.

There were no clear records of residence time of the sampling lines at each site. The site operators did not know how recently the residence time had been recalculated. At a minimum, the residence time should be calculated for each instrument after any change is made to the sampling train, such as the removal or addition of other instruments, and posted at each site.

The station technicians should calculate the residence time, document it in the station logbook or other form available at the site, and periodically verify the measurement.

Reference s: 40 CFR Part 58, Appendix E Section 9 Recommendation to Address Finding:

CARB should c alculate residence times for all gaseous monitors , m odify sites with residence

times in excess of 20 seconds

, with a goal of 10 seconds

, and evaluate any impact on c ompliance

data due to excessive residence times. CARB should also h ave residence time calculated and posted or accessible on

- site.54 Finding # FO5 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: Delay in sending PM 2 .5 samples has resulted in loss of data. Description:

PM 2.5 samples are subject to a maximum of a 30

- day hold from the time the samples are taken to when they are conditioned and weighed. If samples are not maintained at temperatures below the

average am bient temperature during sampling, the hold time is limited to 10 days. Samples at one site, Yuba City, were held too long at the station post

- collection . In some cases the delay has

resulted in the need for immediate conditioning/weighing in laboratory ( e.g. , 12/4/10) and in

other cases has resulted in invalidation ( e.g. , 9/22/10 and 6/17/10 through 6/20/10).

References: 40 CFR 50, Appendix L, Section 8.3.6 Recommendation to Address Finding:

PM 2.5 samples should be sent to the laboratory within 15 day s of sample collection. Field operation managers should ensure that the protocol is followed.55 Finding # FO6 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding:

PM m ake

- up samples are not being taken in accordance with EPA guidance. Description: According to EPA’s April 1999 Guideline on Data Handling Conventions for the PM NAAQS,

PM 10 make up samples may count toward completeness when collected no more than 7 days after a scheduled sample or if they are collected bet ween the missed sample day and the next scheduled sampling date. For example, a missed sample for a 1

- in

- 6 day schedule could be made up before the next scheduled sample day or the day following the next scheduled sample day.

References: http://epa.gov/t tncaaa1/t1/memoranda/pmfinal.pdf , pg 32

- 33 Recommendation to Address Finding:

CARB should e nsure that the PM 10 QAPP/SOP describes this situation, and that the field operators are aware of this provision.56 Finding # FO7 Agency: CARB

Date of Audit: Sum mer 2011 Program Area: Field Operations

Finding:

PM 10 QC checks are not consistently recorded. There is no document in which field operators are

directed to record this information. Description:

PM 10 QC checks are being carried out by CARB field ope rators, but the checks are not consistently documented. The monthly check sheet does not have a monthly flow rate verification entry.

References:

Recommendation to Address Finding:

CARB should i nclude PM 10 QC checks on the monthly check sheet or in s ome other document

and e nsure that field operators consistently record the information.57 Finding # FO8 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding:

CARB f ield staff do not check data after sending information to the CARB offices. Description: Station operators were unable to account for some data in AQS. EPA found an instance where the station operator made an incorrect note, which resulted in a data point being entered into AQS that should have been invalidat ed.

Station operators make notes in the station log, on monthly check sheets, on strip charts, and on the monthly data report. They make notes on all flags contained in the monthly data report, edit the data, and then send everything to the data valida tor

, who reviews the information and calls with any questions.

The station operators do not review the data after the data validator makes changes, and do not look at the data entered into AQS. They often are not aware that there has been a problem, do not know why certain flags have been entered or why data were invalidated.

References:

Recommendation to Address Finding:

CARB should r eview the data validation SOP and determine if there should be an additional step where station operators revie w the data validator’s changes or the AQS data entry.58 Finding # FO9 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: The Yuba City site has several significant siting issues that need to be resolved. Description: The Yuba City site monitors for the following pollutants for comparison to the NAAQS: O 3 , NO 2 , PM 10 (high vol. filter

- based)

, PM 2.5 (filter

based)

The site also has a PM 2.5 BAM that

is use d for non

- NAAQS purposes.

The monitors are on the roof of a

sma ll commercial building in a generally residential neighborhood. The gaseous probe is on the northeastern portion of the roof. The particulate monitors are on the southern portion of the roof and the

BAM inlet is on the northwestern portion of the roof.

The gaseous probe is within 3 meters of trees and 4 meters from the roadway. This probe must be at least 10 meters from the roadway and the drip line of adjacent trees. This could be resolved by moving the probe to the south and trimming the adjacent tre es.

The particulate monitors are within 6 meters of a tree(s) to the east and 10 meters of a tree to the southwest. The instruments must be at least 10 meters from adjacent trees (a distance of 20 meters is preferable). This could be resolved by trimmin g trees.59 The meteorological tower is too short for the surrounding trees and buildings. It is recommended that this tower be elevated to 10 meters above the roof height, if possible. If the tower cannot be adjusted, the data should be used with caution

. References: 40 CFR Part 58 Appendix E Recommendation to Address Finding:

CARB should r esolve siting issues by moving probes/monitors and/or trimming trees.60 Finding # FO10 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: Records indicate that calibrations of gaseous pollutant instruments are not consistently done according to a schedule. Description: Staff and management indicated that calibrations are performed every six months. Calibrations were typically do ne within the six

- month timeframe, but there were instances when instruments

were not calibrated for 9

- 16 months.

References: Quality Assurance Handbook for Air Pollution Measurement Systems, Volume 2

• Section 12.3

• Appendix D, O 3 Validation Template

Reco mmendation to Address Finding: EPA recommends calibrating instruments every six months if zero/span checks are done biweekly and annually if zero/span checks are done daily.61 Finding # FO11 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: [Previous Finding AQSB7] The number of NO 2 titration points taken during calibration does not meet regulatory requirements. Description: EPA regulation requires that NO 2 calibrations be verified with a minimum of 3 points; 5 po ints are recommended. The AQSB calibration group only takes 2 NO 2 titration points.

References: 40 CFR Part 50, Appendix F describes the requirements for NO 2 calibration. Section 1.5.9.4 states: “Maintaining the same FNO, FO, and FD as in section 1.5. 9.1, adjust the

O 3 generator to obtain several other concentrations of NO 2 over the NO 2 range (at least five evenly spaced points across the remaining scale are suggested).” Recommendation to Address Finding:

CARB should a dd at least one more titration p oint to NO 2 calibrations.62 Finding # FO12 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding:

Multi

- point calibrations of PM 2.5 instruments are not done routinely. Description: The AQSB calibration group performs si ngle point calibrations of PM 2.5 instruments every six months. There is no provision for these instruments to be checked with a multi

- point calibration on a regular basis, as required by 40 CFR Part 50, Appendix L. It is recommended that multi

point chec ks be performed annually for sampler flow. Multi

- point checks of the PM 2.5 sampler temperature and pressure sensors should also be performed if physically possible.

References: 40 CFR Part 50, Appendix L

9.1.1 Multipoint calibration and single

- point ver ification of the sampler's flow rate measurement device must be performed periodically to establish and maintain traceability of subsequent flow

measurements to a flow rate standard.

Recommendation to Address Finding:

CARB should i mplement routine multip oint calibrations of PM 2.5 instruments.63 Finding # FO13 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

Finding: [Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air being used in the program. Description: Zero air scrubbers are used in place of certified zero air for instrument calibrations. This is a common practice and acceptable. Because zero air is used to generate the zero point and the calibration mixes, it must be treated as a standar d. As such, zero air scrubber maintenance and

verification must be documented.

References: QA Handbook Volume II

40 CFR Part 50, Appendix F

1.3.2 Zero air. Air, free of contaminants which will cause a detectable response on the

NO/NO X /NO 2 analyzer or whi ch might react with either NO, O 3 , or NO 2 in the gas phase

titration. Recommendation to Address Finding:

CARB should d ocument the quality of zero air when maintenance is performed on the zero air scrubbers and on a periodic basis.64 Finding # FO14 Agenc y: CARB

Date of Audit: Summer 2011 Program Area: Field

Finding: Span and precision gases used in the field are not being calibrated routinely. Description: In order to reduce the number of gaseous standards that are recertified, AQSB has not had t he field standards used for span and precision checks of CO, NO 2 , and SO 2 certified routinely. EPA regulations require that standards used to perform the required QC checks every two weeks must be certified. AQSB continued to use certified gases for rout ine instrument calibrations.

References: QA Handbook Volume II

40 CFR Part 50, Appendix F

1.3.1 NO concentration standard: Gas cylinder standard containing 50 to 100 ppm NO in

N 2 with less than 1 ppm NO 2 . This standard must be traceable to a National B ureau of Standards (NBS) NO in N 2 Standard Reference Material (SRM 1683 or SRM 1684), an

NBS NO 2 Standard Reference Material (SRM 1629), or an NBS/EPA

- approved commercially available Certified Reference Material (CRM).

40 CFR Part 58, Appendix A

3.2.1 One

- Point Quality Control Check for SO 2 , NO 2 , O 3 , and CO. . . . The standards from which check concentrations are obtained must meet the specifications of section 2.6 of this appendix.

2.6.1 Gaseous pollutant concentration standards (permeation devices or cyl inders of compressed gas) used to obtain test concentrations for CO, SO 2 , NO, and NO 2 must be traceable to either a National Institute of Standards and Technology (NIST) Traceable Reference Material (NTRM) or a NIST

- certified Gas Manufacturer's Internal St andard (GMIS), certified in accordance with one of the procedures given in reference 4 of this appendix. Vendors advertising certification with the procedures provided in reference 4 of this appendix and distributing gasses as “EPA Protocol Gas” must parti cipate in the EPA Protocol Gas Verification Program or not use “EPA” in any form of advertising.

Recommendation to Address Finding:

C ARB should c ertify all field gases used to perform QC checks.65 Finding # FO15 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

– Instrument Testing, Certification, and Repair

Finding:

Instruments removed from the field are not always efficiently tracked and returned to the repair laboratory facility for diagnosis, repair, and reuse. Loss of dat a has occurred due to unavailability of spare instruments. Description: The instrument tracking/information system has at least three different components: 1) an electronic database intended to keep basic tracking information for all of the agency’s supp lies and equipment; 2) a hard copy Parts and Supplies binder kept in the MLD Stockroom that at the time of the audit displayed a last revision date of August 2009; and 3) an instrument filing cabinet kept in the MLD instrument laboratory, with the intentio n that each instrument have its own individual folder with detailed information about acceptance tests, repairs, and other relevant history. It is likely that all of these sources together contain most of the useful and necessary information needed to acco mpany an instrument. It may become difficult to find and correlate information from the three different systems. A better approach would be to combine all three into one centralized system specific to monitoring equipment.

The operations support manager s tated that replaced instruments are sometimes left at sites and may go unnoticed until there is a shortfall in the laboratory. In such cases, the approach used to find these missing instruments can be rather tedious, involving calling multiple sites before finding the orphaned instrument. The instrument tracking system maintained by the agency is not efficient and has the potential to impact data completeness. In one instance, data completeness was impacted at the Sutter Buttes site during the summer of 201 1 when a malfunctioning O 3 instrument was not promptly replaced due to the lack of a spare. CARB should develop a system that tracks instruments so that they are diagnosed and repaired promptly to be available for reuse.

References: Quality Assurance Han dbook for Air Pollution Measurement Systems Volume II: Ambient Air Quality Monitoring Program (December 2008), Sections 11 & 13. Recommendation to Address Finding:

CARB should e stablish and implement a robust and centralized monitoring equipment tracking system that allows for prompt and accurate tracking containing all relevant information ( i.e. repairs, calibrations, etc.) of instruments. Given long distances between locations in the state and associated shipping costs and time, CARB may choose to explo re the establishment of a second instrument laboratory facility to serve the Southern California region.66 Finding # FO16 Agency: CARB

Date of Audit: 2011 Program Area: Field Operations

Positive Finding: CARB is working to improve communication wi th field staff. Description: CARB’s monitoring field operations manager is instituting quarterly meetings with all field staff in order to improve communications. EPA supports this as a way to improve consistency and

coordination between the field staff across California.

References:

Recommendation to Address Finding:

N/A67 Finding # FO17 Agency: CARB

Date of Audit: Summer 2011 Program Area: Field Operations

– Instrument Testing, Certification, and Repair

Positive Finding:

CARB maintains a we ll

- equipped stockroom of spare parts, maintains a large equipment purchase order history, and develops thorough equipment testing procedures that are regularly updated. Description: CARB maintains a stockroom within the MLD facilities that is well equipp ed with all types of replacement parts and spares. This usually allows for timely preventive care and operation of the monitoring network.

During the audit, CARB was able to provide most of the documentation for large equipment purchases including biding specifications based on EPA regulations, purchase receipts and condition reports, contract manufacturer service agreements, and procedures/conditions for the

release of funds to the manufacturer.

CARB develops extensive procedures to test all newly acquir ed instruments. Several of these Acceptance Test Procedures (ATP) were examined during the audit. CARB develops these forms specific to each instrument model. The ATP forms contain detailed procedures and specifications that shop technicians should check w hen instruments are first received. The ATP forms allow for review and approval by the Operations Support Manager, as well as the Air Quality Surveillance

Branch Chief.

References: Quality Assurance Handbook for Air Pollution Measurement Systems Volume I I: Ambient Air Quality Monitoring Program (December 2008), Sections 11. Recommendation to Address Finding:

N/A68 Finding # DM1 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding:

The data validation and review/verificat ion procedures performed by AQSB, NLB, and AQAS are not formally published in a control

- copied SOP. Description: SOPs detail the work procedures that are to be conducted or followed within an organization. SOPs document the way activities are to be perfo rmed to ensure consistent conformance to technical and quality system requirements and to support data quality. SOPs are intended to be specific to the organization or facility whose activities are described and assist that organization to maintain their q uality control and quality assurance processes and ensure compliance with governmental regulations. Well

- written SOPs can also serve as training materials and as references for staff, particularly if they are updated regularly (the recommendation is every three years). SOPs should be distributed in a manner that ensures that only the most recent versions are used and that historical SOP revisions are retained (these are sometimes called “controlled

copies”). SOPs should also be developed to enable individua ls to transition into new positions. Deviations and changes from SOPs should be dated, documented, and kept in a bound or electronic document routinely accessed by and accessible to all staff.

References: 40CFR Part 58 Appendix A Section 3.1.1 states tha t “Each primary quality assurance organization shall be defined such that measurement uncertainty among all stations in the organization can be expected to be reasonably homogeneous, as a result of common factors. Common factors...include use of a common QA PP or standard operating procedures”.

Guidance for Preparing Standard Operating Procedures", EPA/240/B

- 01/004, March 2001 Recommendation to Address Finding:

CARB should f inalize control

- copied SOPs for the data validation and review/verification procedu res in the AQSB.69 Finding # DM2 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding: [Previous Finding M7] Data submitted by local districts within the CARB PQAO are not validated using consistent procedures. (See Finding s SJV9, IMP10, and MEN11) Description: In order to maintain a consistent data set, a PQAO should have a standard for routine data validation. However, the CARB QA Manual does not require a specific validation scheme for each of the criteria pollutants. T his results in data validation that is inconsistent and has the appearance of being arbitrary

, which is of special concern when data are used for NAAQS

determination.

It is unclear to agencies within the CARB PQAO what the roles and responsibilities are f or data validation and submittal. For example, CARB/AQAS uploads continuous data for two of the

local districts we audited as part of this TSA . These districts expected that CARB would validate their data as part of this process. In fact, CARB /AQAS does n ot validate data for any agency. This misunderstanding has resulted in unvalidated and sometimes erroneous data being entered into AQS.

References:

Recommendation to Address Finding:

CARB and local districts should establish SOPs for data validation. They should establish formal documentation that outlines roles and responsibilities for data review and submittal. All CARB PQAO agencies should receive data validation training.70 Finding # DM3 Agency: CARB

Date of Audit: Summer 2011 Program Area: Da ta Management

Finding: [Previous Finding DM5] AQAS does not ensure that local district data are validated prior to upload to AQS. Description: CARB/AQAS uploads continuous data for ten local districts. CARB has an SOP for its staff who upload distric t data into AQS, but there is no formal documentation that guides roles and responsibilities for ensuring that appropriate data validation and submittal procedures are followed by the local districts. Several local districts are not validating data prior t o submittal to CARB/AQAS for upload. Those local districts that do validate their data are not following any consistent approach.

References: CARB/PTSD SOP for Ambient Air Quality Data Management, 2009. Recommendation to Address Finding:

CARB should en sure that all local districts having the responsibility for submitting data directly to AQS follow ing consistent procedures for reviewing and validating data before they are submitted to AQS. Formal documentation should be developed that define roles and

r esponsibilities for data review and submittal between CARB and each local district within the CARB PQAO.71 Finding # DM4 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding:

A few instances of erroneous continuous data wer e identified in AQS for CARB sites. Description: CARB’s AQSB validates continuous data for CARB sites, which involves reviewing >50,000 data points per month. Data review performed during the audit identified missing data that

should not have been invali dated and incorrect data that were not identified and corrected. The erroneous data were not identified by any level of review.

References:

Recommendation to Address Finding: CARB should organize data validation training and finalize SOPs to establish appropriate procedures for data validation (see previous findings on data validation). Additional tools and/or resources should be assigned to data validation. For example, developing data visualization tools to assist in reviewing large sets of data may make the review of CARB continuous data more efficient and effective. Data audits by an independent section of CARB ( e.g. , Quality Management Branch) would help identify systematic deficiencies with data validation as well as specific data issues. CARB sho uld develop data tools ( e.g. , flags, figures, tables) to conduct

effective and efficient data audits.72 Finding # DM5 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding: Erroneous continuous data were identified in AQS fo r non

- CARB sites within the CARB PQAO. Description: Each district within the CARB PQAO is expected to validate its own data; however, this is not done consistently (see Findings IMP10, MEN11, and SJV10). EPA identified incorrect data being collected by l ocal districts and submitted to AQS.

References:

Recommendation to Address Finding: CARB should organize data validation training and finalize SOPs to establish appropriate procedures for data validation (see Finding DM2). Each local district should e ither adopt CARB SOPs or develop their own. CARB and each local district should formally agree on consistent data validation procedures. Audits by an independent section of CARB ( e.g. , Quality Management Branch) would help identify systematic deficiencies with data validation as well as specific data issues. CARB should develop tools ( e.g. , flags, figures, tables) to conduct effective

and efficient data audits.73 Finding # DM6 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Find ing: [Previous Finding DM6] There are numerous deficiencies in the data certification process for the CARB PQAO, including:

• Not all NAAQS

- compliant data within the CARB PQAO are routinely certified.

• Data certified by CARB for local districts are not typi cally reviewed or validated.

• Data are routinely certified by agencies, but responsibility has not been formally delegated to any local agencies within the State of California. Description: Numerous agencies collect, analyze, and submit regulatory ambien t air monitoring data. Often the same agency does not perform all of these activities and so it is not clear which agency should certify data. Ultimate authority for certifying data rests with the State, but can be delegated to local agencies. Historically , the responsibility for certifying data has not had formal delegation. The audit revealed cases where r egulatory data submitted to AQS had not been certified by any agency (see Table 3). The lack of a formal structure for data certification within

the Sta te has resulted in incomplete and inappropriate data certification with the potential to jeopardize regulatory decisions. Additionally, CARB submits data for ten districts within the CARB PQAO and certifies these data without reviewing or verifying that th e district validated the data. As a result, some unvalidated, erroneous data have been certified by CARB and submitted to AQS.

References:

40 CFR 58.15 EPA guidance on data certification states:

“ 2. What types of monitoring organizations must certify the ir data? State and local government monitoring organizations must certify their data. A state official should certify all data submitted for affected monitors in that state, except where responsibility for compliance with 40 CFR Part 58 requirements has been delegated to a local monitoring agency. Note that even if multiple monitoring organizations are considered to be with a single Primary Quality Assurance Organization, the certification must come from the state level, or from each local agency which has delegated responsibilities for compliance with 40

CFR Part 58.” Recommendation to Address Finding:

CARB should establish a formal structure for data certification. This includes identification of all data to be certified and the parties responsible f or certification and formal delegation to those parties. In addition, CARB and local districts should establish formal roles and responsibilities so that no unvalidated data are certified and entered into AQS.74 Finding # DM7 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding: Data, including those for design value sites, have been changed after they are certified and subsequently not recertified. Description: The AQDB occasionally requests changes to data, based on cont inued higher level analyses, after

certification. The data are not recertified.

40 CFR Part 58.15 requires data be certified by May 1 of each year. Since the data are considered certified, they are official, and not subject to change after submittal of th e certification letter. Changing data after certification is a significant concern, as the expectation is that the data will not change and may be used for attainment and decision making purposes. Data verification should take place before upload to AQS, n ot after, when they may impact numerous decisions already made by several organizations. Any changes to data that occur subsequent to data certification must be recertified. Uncertified data cannot be used for regulatory decisions.

References:

40 CFR Par t 58.15 Recommendation to Address Finding: All data changes and certification should take place consistent with deadlines established in Part 58.15. If data need to be changed after they are certified, they should be recertified.75 Finding # DM8 Agency : CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding: Some local districts within the CARB PQAO are listed as their own PQAO in AQS. Description: The following agencies, which are within the CARB PQAO, are listed as their own PQA Os in AQS:

• GBUAPCD

• MeCAPCD

• SJVAPCD

• SBCAPCD

• SiCAPCD

• TCAPCD

In some cases some parameters/sites for the local districts are under the CARB PQAO and other parameters/sites are under their own PQAO.

References: 40 CFR 58, Appendix A, Section 3.1

Recommenda tion to Address Finding: CARB should identify the cases where data in AQS from agencies within the CARB PQAO are listed as a different PQAO and work with the districts to have data listed under the CARB PQAO. EPA can change the PQAO designation with appro val from the local district and CARB.76 Finding # DM9 Agency: CARB

Date of Audit: Summer 2011 Program Area: Data Management

Finding: There were several instances of CARB altering data collected by local districts without communicating with the dist rict. Description: CARB/AQAS enters continuous data for ten local districts. Local districts are expected to validate their data and submit them for direct upload to AQS. AQAS runs a routine AQS report that detects outliers. If any outliers are identifie d, AQAS staff must request that the district review the outliers, and revise the data outliers if necessary. It is AQAS policy, as specified in the Data SOP, not to revise local district data without the district’s consent.

AQAS uploads continuous data f or two of the districts that EPA visited during this TSA. Within the past year, for both districts there were instances where data had been altered without AQAS communicating with the local district.

References:

Recommendation to Address Finding:

CARB and local districts should establish formal documentation to establish roles and responsibilities for data review and submittal. CARB/AQAS should follow its policy to have each local agency revise its own data.77 Finding # QA1 Agency: CARB

Date of Audi t: Summer 2011 Program Area: QA Management

Finding: The QA Audit group has made an effort to im prove its documentation process; however, several

inconsistencies were noted. Description: During the certification process for CARB’s National Performan ce Audit Program, it was noted that the QMB performance audit group should make several improvements to its audit documentation process. These recommendations have been partially implemented through improvements to field documentation and logbooks.

Sever al discrepancies were noted:

• The audit trailer logbook entries are incomplete and written in pencil.

• There is no indication that the trailer logbook was recently reviewed by management.

• The equipment maintenance records were not current.

• Field sheets are f illed out in pencil and transferred to electronic documents. As these sheets may be maintained as official records for the data validator, they should be completed in indelible ink.

In order to ensure the data produced by the ambient air monitoring netwo rk can withstand legal challenge, documentation must be complete, definitive and sufficient to be used as evidence for CARB/EPA designation decisions.

References: QA Handbook, Volume II, Chapter 5

EPA’s APTI Course 444, Air Pollution Enforcement Student Manual, Chapter 8 Evidence and documentation are not necessarily the same thing. Evidence is used to establish the truth for an issue being contested in court or a formal hearing. Good documentation may become evidence or support evidence. Bad documentat ion will only raise more questions and often causes the truth to be lost. Recommendation to Address Finding: The performance audit group should continue improving its documentation procedures.78 Finding # QA2 Agency: CARB

Date of Audit: Summer 2011

P rogram Area: QA Management

Finding: The audit trailer evaluated was using one expired gas cylinder along with others that had not been certified annually as required for the EPA National Performance Audit Program (NPAP). Description: Of the gas cyl inders being used in the audit van, only the high concentration multi

- blend gas had been certified within the last year. The low concentration CO cylinder had not been certified in over three years and was presumably past its certification period.

The NPAP program states that gases should be certified annually. Because the CARB audit program performs NPAP audits, this criterion must be met as part of the CARB NPAP program

. References:

NPAP Field SOP 4/7/11

3.1.7.4 Annual Gas Cylinder Certification

N PAP mobile audit laboratory compressed gas standards should be certified annually using the EPA Calibration Gas Traceability Protocol. This could be done through CARB’s Standards Laboratory, any of the gas vendors, EPA Region 2, 7 or OAQPS support contract or. Recommendation to Address Finding:

CARB should c ertify gases used for audits at least annually.79 Finding # QA3 Agency: CARB

Date of Audit: Summer 2011 Program Area: QA Management

Finding: The QA Section is not tracking monitors to ensure th at 25% of monitors are being audited per calendar quarter. Description: EPA regulation requires that at least 25% of SLAMS monitors are audited each calendar quarter and that every monitor is evaluated at least once per year. The QAS has been meeting th is schedule by distributing site audits throughout the year. It is recommended that the evaluations be tracked by dividing the number of monitors audited each quarter by the total monitors in the network. While there is no requirement to schedule evaluat ions tracking each of the four gaseous criteria pollutants, it is recommended that this be a secondary goal of the program.

References:

40 CFR Part 58, Appendix A

“3.2.2 Annual performance evaluation for SO 2 , NO 2 , O 3 , or CO. Each calendar quarter (durin g which analyzers are operated), evaluate at least 25 percent of the SLAMS analyzers that monitor

for SO 2 , NO 2 , O 3 , or CO such that each analyzer is evaluated at least once per year. If there are fewer than four analyzers for a pollutant within a primary q uality assurance organization, it is suggested to randomly evaluate one or more analyzers so that at least one analyzer for that pollutant is evaluated each calendar quarter. The evaluation should be conducted by a trained, experienced technician other tha n the routine site operator.”

Recommendation to Address Finding: Evaluate the current systems with regard to their compliance with the requirement to audit 25% of monitors per quarter.80 Finding # QA4 Agency: CARB

Date of Audit: Summer 2011 Program A rea: QA Management

Finding: The connection to the inlet on the audit trailer could pull in outdoor air. Description: The probe connection being used by QAS might be creating a Venturi effect, bringing in outdoor air. When the diameter or

size of a tube or pipe is increased there is a resulting pressure drop that can overcome the inherent pressure differential and cause a Venturi effect that may overcome excess system

pressure.

By reconfiguring the design of the inlet attachment, the possibility of bringing in outdoor air can be significantly reduced. Note that excess flow from the vent should always

be verified.

References:

Recommendation to Address Finding:

CARB should r econfigure the inlet hardware to reduce the possibility of Ventur i pumping and verify excess flow before and after each audit, or verify that ambient air is not being pulled in.81 Finding # QA5 Agency: CARB

Date of Audit: Summer 2011 Program Area: QA Management

Finding: Auditors do not review all applicable sit ing information in AQS prior to an audit. Description: It was found that the GPS coordinates for the site where the audit program was reviewed were incorrect in AQS. The QAS was unaware of this discrepancy as staff had not evaluated the accuracy of the AQS siting information. Because AQS is the repository of official information on each monitoring site and the information is used by EPA to make regulatory decisions and in research studies, it should be periodically verified.

References:

40 CFR Part 58

§ 58.16 Data submittal and archiving requirements.

( a) The State, or where appropriate, local agency, shall report to the Administrator, via AQS all ambient air quality data and associated quality assurance data for SO 2 , CO, O 3 , NO 2 , NO, NO Y , NO X , Pb

- TS P mass concentration, Pb

- PM 10 mass concentration, PM 10 mass concentration, and PM 2.5 mass concentration; for filter

- based PM 2.5 FRM/FEM the field blank mass, sampler

- generated average daily temperature, and sampler

- generated average daily pressure; chemica lly speciated PM 2.5 mass concentration data; PM 10

– 2.5 mass concentration; chemically speciated PM 10

– 2.5 mass concentration data; meteorological data from NCore and PAMS sites; average daily temperature and average daily pressure for Pb sites if not already reported from sampler generated records; and metadata records and information specified by the AQS Data Coding Manual ( http://www.epa.gov/ttn/airs/airsaqs/manuals/manuals.htm ). Recommendation to Address Finding:

CARB a uditors should verify that site i nformation in AQS is correct.82 Finding # QA6 Agency: CARB

Date of Audit: Summer 2011 Program Area: QA Management

Finding: [Previous Findings M4 & OPA2] Quality assurance for special projects is not developed in a process consistent with EPA quali ty system requirements. Description: When EPA grant funds are used by CARB to collect environmental data, or when data are used to support an EPA regulatory decision, data collection and use must be covered by a quality system that meets EPA requirement s.

MLD does not have oversight authority for monitoring projects that are conducted entirely or initiated by other CARB Divisions or California Air Districts. The quality assurance planning and implementation for these projects is generally not transpare nt to MLD or EPA.

AQSB does implement quality assurance planning for special projects where MLD plays a significant role. These projects may or may not include planning and implementation review by the QMB.

References:

40 CFR Parts 31 and 35

Gran t recipients are required to document their quality systems.

CIO 2105.0 (formerly 5360.1 A2)

Approval Date: May 5, 2000

5. SCOPE AND FIELD OF APPLICATION

.

a. Scope . This Order defines the minimum requirements for quality systems supporting EPA

environm ental programs that encompass:

(1) the collection, evaluation, and use of environmental data by or for EPA, and

(2) the design, construction, and operation of environmental technology by EPA.

b. Applicability to Environmental Programs . This Order appl ies to (but is not limited to) the following environmental programs:

(1) the characterization of environmental or ecological systems and the health of human populations;83 (2) the direct measurement of environmental conditions or releases, including sampl e collection, analysis, evaluation, and reporting of environmental data;

(3) the use of environmental data collected for other purposes or from other sources (also termed “secondary data”), including literature, industry surveys, compilations from compute rized data bases and information systems, results from computerized or mathematical models of environmental processes and conditions; and

(4) the collection and use of environmental data pertaining to the occupational health and safety of personnel in EPA facilities ( e.g. , indoor air quality measurements) and in the field ( e.g. , chemical dosimetry, radiation dosimetry). Recommendation to Address Finding:

CARB should d evelop a process in the QMP to ensure that special projects collect monitoring

data sui table for their intended use and include independent quality assurance planning, implementation and assessment support. QMB should w ork with other CARB divisions to promote good quality assurance practices for all data collection activities.84 Finding # QA7 Agency: CARB

Date of Audit: Summer 2011 Program Area: QA Management

Finding: Mass flow elements (MFEs) are used to establish calibration points outside of their calibrated

range. Description: The factory calibration range for the MFE for the BGI Tetracal devices goes down to 1.8 standard liters per minute (slm). However, the lowest calibration point used in this calibration is 0.2 slm. While this is significantly below the calibrated range, the MFE’s linear range should

extend well below th is flow rate. The MFE should be calibrated below 0.2 slm so that stability of the standard is objectively measured across its linear range.

References:

NIST Handbook 150

- 2G, National Voluntary Laboratory Accreditation Program, Calibration Laboratories, Technical Guide for Mechanical Measurements, Section 2.6

2.6.5.3 A laboratory that certifies artifacts to tolerances should demonstrate a measurement uncertainty which does not exceed 50% of the tolerance. Exceptions to this ratio will be accepted for mea surement systems which are documented to be state

- of

- the

art.

NELAC Standard 2003 (Quality Systems Section) : 5.5.5.2.2.1 Initial Instrument Calibration The following items are essential elements of initial instrument calibration: f) The lowest calibration s tandard shall be the lowest concentration for which quantitative data are to be reported (see Appendix C). Any data reported below the lower limit of quantitation should be considered to have an increased quantitative uncertainty and shall be reported usin g defined qualifiers or flags or explained in the case narrative. g) The highest calibration standard shall be the highest concentration for which quantitative data are to be reported (see Appendix C). Any data reported above this highest standard

should be c onsidered to have an increased quantitative uncertainty and shall be reported using defined qualifiers or flags or explained in the case narrative. Recommendation to Address Finding: The flow calibration laboratory should adjust the MFEs calibration rang e to be below the lowest flow ranges expected.85 Finding # PM1 Agency: CARB

Date of Audit: Summer 2011 Program Area: PM Laboratory

Finding: Communication of post

- weigh information and transmission of documentation to local districts should be impr oved. Description: The PM Laboratory supports filter weighing operations for a number of districts throughout California. Some agencies have indicated that post

- weigh PM data have not always been transmitted in a timely fashion. Communication of PM data to local districts should be considered time critical, especially when there are exceedances of the standard.

References:

Recommendation to Address Finding:

CARB should f ormalize communication procedures in order to maintain timely transmission of

pos t

- weigh PM data that is consistent with the expectations and needs of the local districts.86 Finding # PM2 Agency: CARB

Date of Audit: Summer 2011 Program Area: PM Laboratory

Finding: The PM Laboratory does not have a formal corrective action proc ess for addressing issues with

PM filter collection. Description: Currently, the existing corrective action process for CARB is limited to the QMB performance audit program and Standards Laboratory calibration services. A similar process should be applie d to the PM Laboratory. The PM Laboratory supports filter weighing operations for a number of districts throughout California and often receives filters that have been damaged or deemed invalid due to other operational issues ( i.e. filters received after r equired weighing period). Currently, these issues are communicated informally via email or phone call conversations. Due to the recurring nature of these issues, which result in data loss, the PM Laboratory should develop a mechanism to minimize these loss es through a corrective action process.

References: EPA quality management standards (EPA QA/R

- 2, Quality Improvement Section) require that management and staff “ensure that conditions adverse to quality are” prevented, identified promptly, fully defined , corrected, prevented from recurring, and documented as corrective

actions which are tracked to closure.” Recommendation to Address Finding:

CARB should e stablish a formal corrective action process for the PM Laboratory.87 Finding # PM3 Agency: CARB

Date of Audit: Summer 2011 Program Area: PM Laboratory

Finding:

Documentation of activities in the PM 10 and PM 2.5 la boratories should be improved. Description: Although the majority of activities in the PM Laboratory are adequately tracked and

docum ented, there are some areas where improvements should be made. Specific examples

include:

• The PM 10 Laboratory does not maintain a general laboratory logbook.

• Expiration and replacement of electrostatic strips are not documented.

• Post

- weigh conditioning tim es are noted on post

- it note s and not formally documented.

• Honeywell charts are primary records that are accompanied by a digital Dickson logger, but RH/Temp erature are not transferred or tracked in LIMS.

• Post

- it note s are placed on archived Honeywell char ts when RH/Temp erature goes out of

spec ification and is not formally documented.

• Temperature of the refrigerator used for cold storage of filters is not documented.

• Removal of filters from cold storage is not documented.

References:

Recommendation to A ddress Finding:

CARB should i mplement or improve documentation of identified activities.88 Finding # PM4 Agency: CARB

Date of Audit: Summer 2011 Program Area: PM Laboratory

Finding:

PM 10 trip blanks are not being used to assess potential bias fro m filter transport and handling. Description: Trip blanks controls are useful in assessing potential contamination of filters from transport and laboratory handling.

References: Table 10.1 in Section 10.2: Internal vs. External Quality Control of the Q A Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002, December 2008 identifies trip blanks as a method for assessing bias due to contamination or operator error. Recommendation to Address Finding:

C ARB should c ollect PM 10 trip bl anks periodically.89 Finding # TL1 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding: The canister cleaning SOP does not reflect the current cleaning procedure

. Description: The number of cleaning cycles for a newly acquired cleaning system has been reduced from nine, as stated in the SOP, to five. Staff stated the SOP is being re

- written.

SOPs document an agency's official policies and procedures to which staff should adhere to

obtain consistent and reliable data. They are required as part of an agency's approved QAPP. SOPs are used in training staff in agency accepted analytical methodology and help demonstrate data defensibility.

References: 40 CFR Part 58, Appendix A

Recommendation to Addr ess Finding: CARB should update SOPs to reflect current practice to be compliant with 40 CFR Part 58, Appendix A. It is recommended that SOPs be reviewed annually at a minimum and updated as needed. It is further suggested that analysts initial the SOPs to indicate that they have read and understand the SOP and have had an opportunity to discuss them with their supervisor.90 Finding # TL2 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding:

An SOP i s not documented for the batch certification of cleaned canisters. The canister cleaning SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method. Description: CARB staff stated that current criteria are documented in the Qua lity Assurance Manual (QAM), but not in the SOP. It is unclear why there are different cleaning criteria for the two methods. Refer to previous finding.

References: 40 CFR 58, Appendix A Recommendation to Address Finding: Consistency across methods w ould facilitate the use of the QAM as a reference. Analysts’ initials on SOPs would document that they understand approved current procedures.91 Finding # TL3 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding: The batch certification of cleaned canisters described by staff for methods MLD 058 and MLD 066 differs from existing VOC guidance

. Description: PAMS guidance recommends that one cleaned canister out of eight be certified and Method TO

15, on which these methods are based, recommends certifying every canister. CARB currently tests one cleaned canister of twelve for residual contamination as part of the certification process.

Batch certification can identify excessively dirty canisters or a malfunction of the cleaning system during the cleaning cycle, but may be inadequate to certify that every canister in a batch is actually clean. Analyzing one out of twelve instead of eight canisters introduces even greater uncertainty. The uncertainty increases if the dirtiest canister in a batch is not selected as the

certification canister.

References:
PAMS TAD, Method TO

- 15. Recommendation to Address Finding: CARB should either certify more of the canisters in each cleaning batch in accordanc e with current VOC method guidance for these modified methods or conduct studies to demonstrate the batch certification process is effective for the specific contaminants in these methods by analyzing every canister in a batch for a period of time. Confid ence that the current practice is adequate would be increased if the number of canisters that failed certification is maintained in a log. Developing a trend chart of the level of contamination detected during canister certification may also serve to aler t staff that cleaning equipment maintenance is needed.92 Finding # TL4 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding: [Related to Previous Finding OL19]. Pre

- cleaning concentrations are no t recorded in a logbook to allow for the selection of the most highly contaminated canister for batch certification. Description: Canisters are randomly selected for certification. As a result of a finding from the previous TSA,

CARB has initiated a system of marking canisters that have been selected for testing as part of the batch certification to ensure that eventually all canisters are tested.

Certifying the canister with the most highly contaminated sample concentrations during batch

certificati on in accordance with guidance would provide a higher level of confidence that the entire batch of canisters has been effectively cleaned.

References:
PAMS TAD, Sec. 2.5.2.3 Recommendation to Address Finding: CARB should select the most highly conta minated canisters cleaned in a batch for analysis to certify that the batch is free of contamination.93 Finding # TL5 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding: Canisters are not routinely leak tested as prescribed in guidance. Instead, canisters are vacuum leak tested only when gross leaks are suspected. Description:
Canisters can become contaminated over time from leaks and micro leaks, which are not obvious from monitoring canister ga uge readings. Method TO

- 15, on which methods MLD 066 and MLD 058 are based, describes the process for leak testing canisters in Sec. 8.4.1.1 and establishes a

criterion of + 2 psig, beyond which the pressure should not vary. Canisters may also become

conta minated over time through micro leaks. The PAMS TAD, Section 2.5.3.6, states that “...obvious leaks may be checked by submerging canisters in water, but to check for micro leaks, the canister should be evacuated and its pressure observed for several days wit h a sensitive absolute pressure gauge connected."

References:

Method TO

- 15, Sec. 8.4.1.1; PAMS TAD, Sec. 2.5.3.6 Recommendation to Address Finding: CARB should establish a program of monitoring for leaks and micro leaks as prescribed in

Method TO

- 15 and the PAMS TAD.94 Finding # TL6 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Finding: [Previous Finding OL21]. A retention time policy for re

- cleaning and blanking canisters once they have been cer tified clean has not been established. Description: The Canister Custodian confirmed that she observes reappearance of contamination in cleaned

canisters over time. Canisters may become contaminated over time through small leaks or micro

leaks that ma y not be obvious from monitoring canister gauge readings (see Description, Finding 5). Additionally, PAMS TAD, Section 2.5.3.2 states that "... canisters may appear uncontaminated immediately after cleaning, but will out

- gas contaminants upon storage for

sev eral weeks. All canisters in use should be blanked checked frequently and particularly after extended periods of storage, to ensure that significant contamination does not appear." EPA observed probable out

- gassed contaminants from canister surfaces in a recent PAMS PE Study of Air District laboratories conducted by the Region 9 QA Office.

References:
PAMS TAD, Section 2.5.3.6 Recommendation to Address Finding:

CARB should establish a retention time for cleaned and certified canisters after which th ey must
be re

- cleaned and certified. A retention time of 30 days has been adopted by the EPA Region 9 Laboratory and other laboratories.95 Finding # TL7 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Canister Cleaning

Fi nding:
The CARB SOP states that old canisters are reconditioned, but this is inconsistent ly practice d

. Description: Staff stated that the reconditioning procedure was determined to be ineffective and has been

discontinued. Similar informati on was presented at the 2011 Air Conference in Dallas, Texas.

A procedure for reconditioning based on best available information does not currently exist.

References: 40 CFR Part 58, Appendix A Recommendation to Address Finding:

The CARB SOP rel ating to reconditioning of canisters should be revised or deleted, since the SOP does not reflect current practice or recommended procedure. CARB may wish to implement a procedure in its place for evaluating old canisters for replacement, as it has been ob served that coatings on older canisters are more subject to retaining and out

- gassing residual contaminants

over time.96 Finding # TL8 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding:

CARB has not est ablished a holding time for cartridges once samples have been collected for extraction or analysis. Description: Cartridges are kept for some weeks in the field before shipping them to the laboratory. Staff stated that cartridges are generally analyze d within the four weeks recommended by the cartridge vendor (Waters), but not within 14 days as specified in Method TO

- 11 or 30 days following extraction specified in the method.

Exceeding method prescribed holding times can result in data being qualifie d due to potential loss of sample or a risk of contamination from extraneous sources, even under refrigeration. Exceeding prescribed method holding times can result in data that are more vulnerable to challenge.

References:

Method TO

- 11

Recommenda tion to Address Finding: CARB should establish a policy for holding time based on the TO

- 11 method holding time (preferred) or, if necessary, conduct and document internal research demonstrating that a variance from the published holding time is justifi ed.97 Finding # TL9 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: The laboratory does not assign expiration dates to new sampling cartridges and allows cartridges to be used beyond the 90 days pr escribed by the method. Description:

Method TO

- 11 states in its discussion of the preparation of DNPH cartridges “that cartridges will maintain their integrity for up to 90 days stored in refrigerated, capped shipping tubes” (TO

- 11 note, sec. 9.5.2.1 6). Initial blank lot concentrations are provided with commercially purchased cartridges. Given the significant concern expressed throughout Method TO

- 11 over potential laboratory contamination, it is prudent to be alert to potential contamination during s torage. One of the air districts assigns a six month expiration date to cartridges. A commercial laboratory (AAC Laboratory, Ventura, CA) confirmed that it routinely monitors and observes that blank concentrations increase over time, although not past cri teria levels. The level of contamination will depend on how the cartridges are stored and if they b ecome exposed to contaminants. Therefore, unused cartridge lots are probably best recertified for QA documentation purposes after 90 days, as suggested by gu idance.

References:

Method TO

- 11, Note in sec. 9.5.2.16 Recommendation to Address Finding:

As recommended in Method TO

- 11 and confirmed by laboratory experience, it should be verified that cartridges meet blank certification requirements by analy zing a blank cartridge before using them past the 90

- day PAMS season.98 Finding # TL10 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: CARB's procedure for analyzing DNPH lot blanks differs from the SOP

. Description: The SOP states that 5% of new DNPH cartridges will be analyzed as lot blanks. Staff stated the practice has been changed to one cartridge per lot rather than one per box.

The SOP should be updated to reflect current practice. SOPs document an agency’s official policies and procedures that staff are to adhere to obtain consistent and reliable data and are required as part of an agency’s approved QAPP as required by 40 CFR 58 Appendix A.

References: 40 CFR Part 58, Appendix A

R ecommendation to Address Finding: CARB should update the SOP to reflect current practice.99 Finding # TL11 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: No criterion is provided in the CARB SOP for passing DNPH lot cartridge blanks. Description: Carbonyl Method TO

- 11 prescribes acceptance criteria for lot blanks of less than 0.15 μg/cartridge (formaldehyde) and less than 0.10 μg/cartridge (acetaldehyde). The SOP should be consistent with pr actice. Furthermore, the criterion used by CARB of 2X RL is not appropriate. The RL must be higher than the blank contamination.

References:

Method TO

- 11, sec. 5.8 Recommendation to Address Finding: CARB should establish appropriate acceptance cri teria for cartridge blanks consistent with the method prescribed criteria and the SOP should be updated to reflect current practice.100 Finding # TL12 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: Gloves are not worn as a contamination protection measure when handling cartridges. A nitrogen

- purged glove bag is not used for extractions. Description:

Method TO

- 11 cautions against the unintentional contamination of eluted samples due to aldehyde and ketone contamination in laboratory air, inks, adhesives, packaging, and vials with plastic caps. The use of gloves is prescribed when handling the cartridges. Extracting the cartridges in a nitrogen

- purged glove further reduces the risk of contaminatio n. Food and drink residue on hands can also present a contamination problem, in addition to safety issues related to working with acetonitrile without proper protection. The use of a glove box will vary with the laboratory air environment; working in a ver y clean hood may be sufficient.

References:

Method TO

- 11, Sec. 10.7, 11.2 Recommendation to Address Finding: CARB should evaluate its current procedures to ensure that current contamination control measures are adequate.101 Finding # TL13 Ag ency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: [Previous Finding OL3] Staff stated that field blanks are not being analyzed at a frequency of 10% of field samples, as specified in Method TO

- 11, nor is th ere an SOP describing the procedure for the submission of field blanks. Description: During the previous TSA, staff stated that CARB was correcting sample results based on an average of field blank results from a study performed 15 years prior. The s tudy was outdated and sample results should not be corrected. During the current TSA, staff stated that sample results are no longer being subtracted, but that field blanks are not being collected as prescribed in the

method. Field blanks increase the leve l of confidence that sample contamination detected is not

from extraneous sources.

References:

Method TO

- 11, Sec. 13.3.1 Table 5

- 3 Recommendation to Address Finding : CARB should use field blanks at the method prescribed frequency of 10% of field samples collected; a minimum of one field blank per sample collection batch is recommended. Data for field blanks and sample results should be reported separately.102 Finding # TL14 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laborat ory

– Carbonyls

Finding: CARB does not analyze trip blanks when needed

. Description:

Method TO

- 11 states that it is desirable to collect trip blanks at a frequency of 10% of field samples. In addition to field blanks and laboratory blanks, if f ield blank analysis show contamination, trip blanks should be collected and analyzed to distinguish between sources of

contamination.

References:

Method TO

- 11, Sec. 13.3.1 & Table 5

- 3 Recommendation to Address Finding : CARB should review its po licy for blanks and consider the value of collecting and analyzing trip blanks in addition to laboratory and field blanks if analytical results indicate blank

contamination.103 Finding # TL15 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

– Carbonyls

Finding:

Method TO

- 11 states that samples should be re

- analyzed when results are 10 % above the

criterion

, but the analyst was not aware of this criterion. Description: Up

- to

- date SOPs help train analysts new to accepted la boratory procedures. Having analysts initial SOPs annually to indicate that they have read the SOPs and have had an opportunity to discuss them with their supervisor is also valuable. Some laboratories administer a written test to qualify an analyst to per form a new method. Keeping charts of the duplicate results with control lines indicating the criterion can ensure that laboratory QC criteria are given adequate attention at the time of analysis.

References:

Method TO

- 11, Sec. 13.4.1 & Table 5

- 3, MLD 022 Sec. 9.6 Recommendation to Address Finding ; CARB should review the system of qualifying analysts to perform methods new to them and should implement procedures to ensure they are knowledgeable concerning all required QC.104 Finding # TL16 Agency : CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Carbonyls

Finding: Working standards are tracked and used for six months, while the CARB SOP states that standards should be retained for four months under refrigeration. Descript ion: Periodic reviews by the supervisor of logbooks and internal audits or reviews by a Quality Assurance Officer would help ensure that replacement schedules are kept. Using expired standards can result in inaccurate data and legal challenges.

Refer ences: CARB SOP; Method TO

- 11; GLP Recommendation to Address Finding : CARB should implement a policy to ensure that standards used for instrument calibration are replaced before they exceed holding times.105 Finding # TL17 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Hexavalent Chromium

Finding: Site name and sampling dates are recorded on a piece of tape loosely stuck to sample cartridges ; the tape occasionally fall s off, making it difficult to identify samp les. Description: A better system for labeling samples is needed to increase confidence that a data point is appropriately identified with a particular sample.

References:

Recommendation to Address Finding: CARB should devise a permanent sys tem for labeling cartridges.106 Finding # TL18 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- Hexavalent Chromium

Positive Finding:

Hexavalent chromium d ata undergo peer review, supervisory review, review by the Branch Chief, and a final review before going to A QS

. Description: Data go through a multi

- tiered review process. The data review procedure described represents a best practice.

References:

Recommendation to Address Finding:

N/A107 Finding # TL19 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- MLD058, Aromatic & Halogenated Compounds, VOCs (Modified TO

- 15)

Finding: [Previous Finding OL5] There is no secondary review of logbooks. Description: Secondary re view of logbooks by supervisory or QA staff can help ensure that proper protocol is being followed.

References:

Recommendation to Address Finding: Instrument run logbooks should routinely be reviewed and signed by supervisory or QA staff.108 Find ing # TL20 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- MLD058, Aromatic & Halogenated Compounds, VOCs (Modified TO

- 15)

Finding: CARB does not analyze audit samples or through

- the

- probe audit samples as suggested in Se c. 9.7 of the CARB SOP. Description : Audit samples are an important quality assurance tool to ensure the accuracy of analytical data. Through

- the

- probe audits help document that the sample and analysis system are within acceptable control limits. Sta ff stated that the SOP to analyze audit samples had been followed in the past, and have requested that this be reinstituted.

References:
CARB SOP, Section 9.7 Recommendation to Address Finding: CARB is encouraged to use audit samples and through t he probe audit samples when possible, as suggested in the CARB SOP, or CARB should revise its SOP to reflect actual practices.109 Finding # TL21 Agency: CARB

Date of Audit: Summer 2011 Program Area: Toxics Laboratory

- MLD 066 Oxygenated Hydrocarbons & Nitriles (Modified TO

- 15)

Finding: Appendix V in the CARB SOP lists the standards that were used in 2003 and has not been updated to reflect the standards currently being used. Description: Staff stated current standards are found in the QC report. Outdated information in an SOP can lead to misunderstanding in practice and would represent a vulnera bility if data are challenged. SOPs should be updated to reflect practice.

References: 40 CFR 58, Appendix A (SOPs)

Recommendation to Address Finding: SOPs should represent current practices and be updated on a regular schedule or as needed in accordance with 40 CFR 58, Appendix A.110 Finding # IMP1 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

– General

Findi ng: The ICAPCD ambient air monitoring program is not operating under an approved QAPP. Description: EPA requires that organizations develop a QAPP for each type of ambient pollutant being measured. The QAPP integrates all technical and quality aspects of a project, including planning, implementation, and assessment. The purpose of the QAPP is to document planning results for environmental data operations and to provide a project

- specific “blueprint” for obtaining the type and quality of environmental da ta needed for a specific decision or use. The QAPP documents the quality assurance and quality control that are applied to an environmental data operation to assure the results obtained are of the type and quality needed and expected.

References:

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans, QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002

Further guidance on developing QAPPs can be found in the guidance documents “EPA Requirements for Quality Assurance Project Plans,” EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002 Recommendation to Address Finding: ICAPCD is currently part of the CARB PQAO, which is responsible for mai ntaining consistency in the collection and assessment of ambient air quality data throughout the State of California so that the data may be combined to give meaningful information.

ICAPCD should develop agency specific QAPPs that are consistent with exi sting CARB QAPPs or formally adopt the applicable existing CARB QAPPs.111 Finding # IMP2 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- General

Finding: ICAPCD has not established an appropriate quality system for ambient air monitoring. Description: A quality system is the means by which an organization manages the quality of the monitoring information it produces in a systematic, organized manner. It provides a framework for planning implementing, assessing and reporti ng work performed by an organization and for carrying out required quality assurance and quality control activities. While the monitoring staff at ICAPCD is very knowledgeable and operates the ambient air monitoring network diligently, the lack of a

struct ured quality system is needed to effectively and appropriately implement ambient air monitoring requirements. Major components of a quality system include:

• Independence of Quality Assurance

.

• QMPs and QAPPs

.

• Data Quality Performance Requirements (Data Quali ty Objectives, DQOs)

.

• QA/QC Activities

. References: 40 CFR 58 App. A 2.0, Quality System Requirements QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002 Recommendation to Address Finding: ICAPCD should implement a quality sy stem consistent with EPA requirements and applicable guidance.112 Finding # IMP3 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Network Management

Finding:

Assessment of PM 10 or PM 2.5 sampling frequency throughout the ICA PCD network has not been performed as required. Description: The minimum required monitoring schedules for PM 10 in the area of expected maximum concentration should be based on the relative level of that monitoring site concentration with respect to the 24

- hour standard, as illustrated in Figure 1 of 40 CFR 58.12. The most recent year of data must be considered to estimate the air quality status at the site near the area of maximum concentration no less frequently than as part of each 5

- year network asses sment.

For PM2.5, required sites that meet the following criteria are required to sample at a 1

- in

- 3 day sampling frequency:

• Design value sites that are within + 10% of the NAAQS.

• Sites where one or more 24

- hour values have exceeded the NAAQS each year for a consecutive period of at least 3 years. In addition, required design value sites that are within 5% of the NAAQ must maintain an everyday sample schedule.

EPA may not be able to make attainment determinations from site where appropriate sampling

f requency is not achieved.

References: 40 CFR 58.12 (e) 40 CFR 58.12 (d)(ii) and (iii) Recommendation to Address Finding: CARB and ICAPCD should perform the required analysis to ensure PM 10 and PM 2.5 monitoring in Imperial County operate at the appropri ate sampling frequency. Necessary changes to sampling schedules should be made as expeditiously as possible.113 Finding # IMP4 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Network Management

Finding: Neighborhood scale may be inappropriate for PM 10 at the Westmorland site. Description: The area surrounding the Westmorland monitoring site is mostly residential surrounded by active agricultural fields, but is located on the Westmorland Wastewater Treatment property and may be influenced by local activity and not representative of a neighborhood spatial scale for PM 10

. The area directly adjacent to the monitoring site is mainly comprised of unpaved areas that are disturbed by vehicle traffic and heavy equipment. Due to s imilar surface conditions throughout the area, the PM 10 monitor is appropriately sited, but may be more appropriately characterized as a having a middle scale of representation.

Neighborhood scale defines concentrations within some extended area of the ci ty that has relatively uniform land use with dimensions in the 0.5 to 4.0 km range. The neighborhood and urban scales listed below have the potential to overlap in applications that concern secondarily formed or homogeneously distributed air pollutants, wh ile middle scale defines the concentrations typical of areas up to several city blocks in size with dimensions ranging from

about 100 m to 0.5 km.

References: 40 CFR App. D 1.2 (b) 40 CFR App. D 4.6 (b) Recommendation to Address Finding:

ICAPCD should evaluate the spatial scales associated with PM 10 monitoring at Westmorland and make changes to AQS and the next annual network plan, if appropriate.114 Finding # IMP5 Agency: CARB

– ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Field Operat ions

Finding:

One

- point flow rate verifications for PM 10 and PM 2.5 are not performed by ICAPCD as required

and are not well documented. Description: ICAPCD has not purchased flow rate transfer standards, and therefore the monitoring staff does

not p erform one

- point flow rate verifications as required. Currently, a nearby CARB site operator, responsible for the Calexico Ethel monitoring site, performs all flow rate verifications

on an “as needed” basis. Based on the available documentation at the moni toring sites, these checks have been missed in the past and have not been well documented. Many records were outdated or incomplete. Also, flow rate transfer standard certification records are not maintained by ICAPCD. Due to a lack of consistent documenta tion, it is unclear when flow rate verifications have been performed and whether the flow rate transfer standard used to perform the checks has

been certified relative to an authoritative standard as required.

A one

- point flow rate verification check mus t be performed at least once every month on each automated analyzer used to measure PM 10 and PM 2.5 . For ICAPCD, these should be performed monthly on the PM 10 BAM1020’s operating at Niland and Brawley and filter based PM 2.5 monitors at El Centro. The same i ssues are present for high

- volume PM 10 samplers, which are required to have one

- point flow rate verifications performed on a quarterly basis.

References: 40 CFR 58 App. A 2.6 40 CFR 58 App. A 3.2.3 40 CFR 58 App. A 3.3.2

Recommendation to Address Fin ding:

ICAPCD should obtain the appropriate flow rate transfer standards for automated PM 10 analyzers, filter based PM 2.5 samplers, and high

- volume PM 10 samplers and perform flow rate verifications as required by regulation. Also, these activities should b e consistently documented by ICAPCD.115 Finding # IMP6 Agency: CARB

– ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Field Operations

Finding: Residence time for gaseous monitors operated by ICAPCD is not established. Description:

Th e residence time is defined as the amount of time that it takes for a sample of air to travel from the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 states that for the reactive gases (O 3 , NO 2 , and SO 2 ) residence times must be less than 20 seconds. Additionally, it is recommended that the residence time within the manifold and sample lines to the instruments should be less than 10 seconds. The station technicians should calculate the residence time, document it in the station logbook, and periodically verify the data.

There was not a clear record of residence time of the sampling lines at each site. Also, the site operators did not know how recently the residence time had been recalculated. At a minimum,

the residence time should be calculated for each instrument after any change is made to the sampling train, such as the removal or addition of other instruments, and posted at each site.

The station technicians should calculate the residence time, document it in the station logbook or other form available at the site, and periodically verify the data.

References: 40 CFR 58 App. E 9 (c) Recommendation to Address Finding: In order limit the potential for significant losses of O 3 through the sampling line, residence times should be calculated, documented, and tracked. If residence times are higher than those required by regulation, ICAPCD should make the necessary changes to the sampling train to reduce the

residence time.116 Finding # IMP7 Agency: CARB

– ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Field Operations

Finding: ICAPCD is internally post weighing high

- volume PM 10 filters without proper PM lab or quality

control measures. Description: Traditionally, all high volume PM 10 f ilters are process ed and weighed by CARB in appropriately controlled environments and necessary quality control and quality assurance techniques. ICAPCD stated that often the post

- weigh information is not transmitted back to ICAPCD from CARB in a timely ma nner. As a result, ICAPCD has implemented a post

- weigh procedure for

PM 10 high

- volume filters in order to get a preliminary assessment of whether the samplers are measuring exceedances of the standard, so that the appropriate planning actions and preparati on can occur immediately after the sample has been collected.

These preliminary post

- weighing procedures are not performed in a controlled environment nor do they follow the required quality control procedures. Furthermore, the weighing and subsequent ha ndling of these filters prior to the official CARB post

- weigh may introduce bias in the sample.

References:

Method 2.12 Sec. 7 Recommendation to Address Finding: ICAPCD should discontinue the internal post

- weighing practices and work with CARB to

esta blish an appropriate procedure for the timely transmittal of CARB post

- weigh information to

ICAPCD.117 Finding # IMP8 Agency: CARB

– ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

– Field Operations

Finding:

Documentation of ICAPCD air mo nitoring activities is not complete. Description: Accurate and complete documentation is essential to the collection of air quality data used for regulatory purposes. Appropriate documentation includes, but is not limited to, standard operating procedure s for all aspects of an organization's program, data quality assessments, logbooks tracking actual day

- to

- day operations, and records of quality control, quality assurance, and maintenance checks. Oversight of personnel and activities involved in the colle ction, processing and submittal of data is facilitated by procedures that are standardized and responsible personnel record their compliance with these procedures.

Currently, ICAPCD does not have a formal or consistent process for documenting air quality monitoring activities. For example, many records are maintained on loose

- leaf paper or post

- it notes (instrument maintenance records, PM 10 make

- up sample dates, PM 10 motor repair, and notes on changes made to the data in the database). Many records or entr ies in logbooks are made in pencil, not initialed, and were limited in information or specificity.

In response to a data tracking request, documentation of flow rate verifications and calibrations

of PM 10 analyzers could not be located, and documentation supporting data invalidation was not

present.

References: QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002 Recommendation to Address Finding: ICAPCD should develop and implement procedures for maintaining adequate docume ntation of ambient air monitoring activities.118 Finding # IMP9 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Network Management

Finding: There are potential siting issues at the Calexico Ethel site. Description:

The Ca lexico Ethel monitoring site is located in the parking lot of a high school in a mostly residential area. The primary concern is the distance of the monitoring site to nearby trees. Trees can act as obstructions in cases where they are located between the air pollutant sources or source areas and the monitoring site, and where the trees are of a sufficient height and leaf canopy density to interfere with the normal airflow around the probe, inlet, or monitoring path. The scavenging effect of trees is greate r for O 3 than for other criteria pollutants and monitoring agencies must take steps to consider the impact of trees on O 3 monitoring sites

. To reduce the potential interference/obstruction, the probe or inlet must be at least 10 m or further from the drip

line of trees.

Other potential issues include monitor spacing on the roof and the distance of the collocated

PM 2.5 monitors to the trailer. Generally, the distance from the obstacle to the probe, inlet, or monitoring path must be at least twice the heigh t that the obstacle protrudes above the probe or

inlet.

References: 40 CFR 58 App. D 4 (a) 40 CFR 58 App. D 5 40 CFR 48 App. A 3.2.6.3 Recommendation to Address Finding: CARB staff have been working with EPA and ICAPCD to relocate the site. EPA recomme nds continuing this effort in a timely manner.119 Finding # IMP10 Agency: CARB

- ICAPCD

Date of Audit: Summer 2011 Program Area: Imperial

- Data Management

Finding: ICAPCD is not adequately reviewing and editing data. Description: (See Finding D M2)

The current database does not allow staff to adequately review and edit data. ICAPCD uses a WinCollect data management system developed by Ecotech for data acquisition, storage, and processing. Monitoring staff has indicated that the current system is hard to work with, and that data review and editing are cumbersome and time consuming tasks. For example, the data system can only automatically assign one flag (AY: “QC Control Points”) to the raw data. As a result, monitoring staff must manually edit ho urly text files to make any adjustments. This process introduces the potential for errors in the data and reduces monitoring staff’s ability to effectively review and edit data appropriately.

ICAPCD staff has contacted Ecotech about these issues, but hav e a difficult time implementing changes to the system, as Ecotech has been largely unresponsive.

ICAPCD would benefit from data validation training.

References: QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002

Recommendati on to Address Finding: ICAPCD should work with CARB to implement a more efficient and comprehensive data management system and to obtain data validation training.12 0 Finding # MEN1 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendoci no

– QA Management

Finding:

M e CAQMD staff was not familiar with the CARB QMP or instrument SOPs. Description: Staff appeared to be trained and proficient with the procedures that are use d to conduct his monitoring activities. However, it was noted t hat this training and/or demonstration of proficiency was not adequately documented.

MeCAQMD staff stated that the district operate s under CARB’s QMP and SOPs. The staff was not aware where electronic or hard copies of QMP or SOPs could be found

. Although the staff was not able to find these documents when EPA was on site, they were later located in MeCAQMD files and accessible online. MeCAQMD noted that CARB’s SOPs are not entirely relevant to the MeCAQMD sites since they refer to different data acquisiti on systems that are not used by MeCAQMD.

References:

Recommendation to Address Finding: MeCAQMD should review CARB QMP and relevant SOPs (see http://www.arb.ca.gov/aaqm/qa/pqao/pqao.htm). MeCAPCD should develop a formal system to

ensure and document t hat all staff are familiar with the quality management system and are trained and proficient at the monitoring tasks that they are performing. MeCAPCD should also ensure that field operators have access to information in relevant SOPs at the field station, either in hard copy or electronically. Finally, MeCAQMD should develop SOPs for activities that are not covered by CARB SOPs.121 Finding # MEN2 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

– Network Management

Finding: MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2006 but is erroneously listed as its own PQAO in AQS. Description: 40 CFR 58.1 defines a PQAO as “a monitoring organization or other organization that is responsible for a set of station s that monitor the same pollutant and for which data quality assessments can be pooled. Each criteria pollutant sampler/monitor at a monitoring station in the SLAMS and SPM networks must be associated with one, and only one, primary quality

assurance orga nization.” Many requirements specified in 40 CFR 58, such as those for collocation, QAPPs, QMPs, and audits, are determined on a PQAO basis.

MeCAQMD is part of CARB’s PQAO, not its own PQAO.

References:

40 CFR 58.1 Recommendation to Address Finding: MeCAQMD should be replaced by CARB as the official PQAO listing in AQS for all MeCAQMD data in order to reflect the current PQAO structure in California

. Although EPA does not encourage MeCAQMD to do so, if it wishes to become its own PQAO or join another PQAO, MeCAQMD should follow the procedure outlined in EPA Region 9’s PQAO Strategy to ensure that all regulatory requirements continue to be met.122 Finding # MEN3 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

– Field Operati ons

Positive Finding: MeCAQMD stations were well

- maintained. Staff and manager were professional and helpful, and very knowledgeable about the county and potential pollution sources. The station operator was proactive about troubleshooting instrument i ssues. Description:

References:

Recommendation to Address Finding:

N/A123 Finding # MEN4 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

- Field Operations

Finding:

One

- point QC checks (flow verifications) for PM 10 and PM 2.5 are not consistently performed by

the MeCAQMD site operator. Description: Mendocino site operators maintain “Maintenance and Service Log” sheets at each PM 10 and

PM 2.5 site. These include a line for monthly flow rate verifications. These a re not regularly notated as having occurred (see photo below). MeCAQMD indicated that resource constraints prevent this requirement from being met consistently.

References: 40 CFR part 58 Appendix A, section 3.2.3. “Flow Rate Verification for Parti culate Matter.” Recommendation to Address Finding: MeCAQMD should c onduct and log monthly flow rate verifications at each PM site every

month.124 Finding # MEN5 Agency: California Air Resources Board

– Mendocino County AQMD

Date of Audit: Summer 2011 Program Area: Mendocino

– Field Operations

Finding: MeCAQMD logbook entries are not consistently made and are not consistently in the most defensible form. Handwritten notes are occasionally illegible due to water (rain) marks. Description: Logbooks s hould be in the form of bound log books with numbered pages and all entries initialed and made in indelible ink. Corrections should be made by drawing a single line through the information, initial ing and dat ing . Information such as instrument down times should be

included.

References: Section 5 of the QA Handbook states that records supporting the operation of air monitoring measurement systems should be retained for at least three years or, if the records are part of any litigation, claim, negotiation , etc., the records should be kept until the issue has been resolved. Recommendation to Address Finding: MeCAQMD should create and follow a logbook entry convention to ensure thorough and

defensible record

- keeping. The records maintained by individuals should be periodically evaluated to ensure they are consistent, secure, regularly maintained, and (for electronic records)

backed

- up. MeCAQMD should take steps to prevent water damage to entries or loss of information due to misplaced single log sheets.125 Finding # MEN6 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

- Field Operations

Finding:

Residence time calculations were not available at the Ukiah Gobbi site. Description:

The residence time is defined as the amount of time that it takes for a sample of air to travel from the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 states that for the reactive gases (O 3 , NO 2 , and SO 2 ) residence times must be less than 20

seconds. Addi tionally, it is recommended that the residence time within the manifold and sample lines to the instruments should be less than 10 seconds. The station technician should calculate the residence time, document it in the station logbook, and periodically ver ify the data.

There was not a clear record of residence time of the sampling lines at the Gobbi site. The site operator did not know how recently the residence time had been recalculated. At a minimum, the

residence time should be calculated for the inst rument after any change is made to the sampling train, such as the removal or addition of other instruments, and posted at each site.

The station technician should calculate the residence time, document it in the station logbook or

other form available at the site, and periodically verify the data ( e.g. , annually).

References: 40 CFR Part 58, Appendix E Section 9 Recommendation to Address Finding: MeCAQMD should calculate residence times for all gaseous monitors. MeCAQMD should modify sites with res idence times in excess of 20 seconds with a goal of 10 seconds, and evaluate any impact on compliance data due to excessive residence times. Residence times should be posted or accessible on

- site.126 Finding # MEN7 Agency: CARB

- MeCAQMD

Date of Audit: S ummer 2011 Program Area: Mendocino

- Field Operations

Finding:

Trees at the Ukiah Gobbi and Ukiah Library sites should be evaluated against siting requirements. Description: 40 CFR Part 58, Appendix E, Section 5 states “trees can provide surfaces fo r SO 2 , O 3 , NO 2 adsorption or reactions, and surfaces for particle deposition . . . to reduce this possible interference/obstruction, the probe, inlet, or at least 90 percent of the monitoring path must be at least 10 meters from the drip line of trees.”

The tree drip line at Gobbi is coming close to the 10 m distance. The trees at the Library site appeared to meet siting requirements at the time of the TSA, but should be monitored over time.

References: 40 CFR Part 58, Appendix E, Section 5

Recommenda tion to Address Finding: MeCAPCD should verify that trees are meeting siting requirements, and check regularly.127 Finding # MEN8 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

- Field Operations

Finding:

The internal she lter thermostat is not operating correctly at the Ukiah Gobbi site and has not been

addressed to provide defensible data

. Description: The Ukiah Gobbi site operator determined that the internal shelter temperature is off by 4°, and is manually correcting the data. The issue and correction have not been formally documented.

References:

Recommendation to Address Finding: The issue should be addressed using a corrective action form. The resolution should be

defensible and well

- documented. Data that coul d have been affected by erroneous temperature readings should be checked and flagged or invalidated if appropriate.128 Finding # MEN9 Agency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

– Field Operations

Finding: MeCAQMD has n o system for tracking and controlling station and instrument logbooks. Description: Field procedures require that logbooks be kept. However, these logbooks were not tracked, identified, and archived in a manner to ensure that the critical documentation t hey contain will be

accessible and defensible.

References:

Recommendation to Address Finding: MeCAQMD should develop a standardized procedure for creating, labeling, and archiving logbooks.129 Finding # MEN10 Agency: CARB

- MeCAQMD

Date of Audit: Su mmer 2011 Program Area: Mendocino

– Field Operations

Finding: MeCAQMD should have formalized training requirements for all air monitoring staff. Description: MeCAQMD makes an effort to participate in trainings when opportunities arise. However, ther e is no formal program to ensure that staff are trained on procedures and demonstrate proficiency on tasks directly related to their job functions.

The QA Handbook, Section 4 discusses the need for a formalized training program. EPA recognizes that fund ing is limited and it is often difficult to send people to trainings. Developing a formalized training program can help agencies identify what trainings are needed, what the highest priority issues are, and what resources are available. If it is not possib le to fulfill the training need immediately, the training plan allows agencies to look for future funding or other opportunities.

EPA also encourages agencies to formalize and document on

- the

- job trainings. Trainings could be given by staff to provide c ommon understanding and competency and minimize future problems and questions. In

- house trainings could include information on the Envista program; training for site operators, data users, and data validators on AQS flags and why they are important; new QA PP/SOP training; data validation and analysis; instrument operation and maintenance training.

MeCAQMD should coordinate with CARB on trainings.

References: The QA Handbook, Section 4 Recommendation to Address Finding: MeCAQMD should develop a formal t raining program and tracking system to ensure that all staff and management are familiar with the relevant QAPPs and SOPs related to producing data in the field, data management and tracking, quality assurance, and all data systems.130 Finding # MEN11 Ag ency: CARB

- MeCAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

– Data Management

Finding: MeCAQMD does not provide CARB AQAS with datasets that have been fully quality assured and ready for upload to AQS. Description: MeCAQMD was unable t o account for some data in AQS. For example, instances were observed where AQS was missing a data point, or had a value when the County showed a span check with no associated value. MeCAQMD gives CARB an AQS

- formatted file with e

- mailed

notes. The local a gency leaves it to CARB’s discretion whether the e

- mailed notes should result in flagged data. There is no SOP for conducting data review and validation, and data are not checked after they are sent to CARB for entry into AQS.

References:

Recommendati on to Address Finding:

CARB should ensure that all local agencies under the PQAO have a data validation SOP , and MeCAQMD and CARB should work together to ensure that an SOP is in place for MeCAQMD

collected data . The SOP should include a procedure whereby the local agency checks any changes

, including flagging

, that CARB makes to the data, either before or after posting, as well as a procedure to discuss any issues the agency may have with the changes. The local agency should keep correspondence and data t hey send to CARB where it can be easily retrieved and reviewed. This SOP should also include procedures to ensure that data are provided to CARB in a form and condition that is ready for direct upload. See Finding DM2.131 Finding # MEN12 Agency: CARB

- Me CAQMD

Date of Audit: Summer 2011 Program Area: Mendocino

- QA Management

Finding:

MeC AQMD does not use a formal corrective action system. Description:

MeC AQMD staff does a considerable amount of troubleshooting. But information as to what the

initi al problem was, when the issue was first noted

, what steps were taken to resolve the issue,

and when it was resolved is not consistently recorded and is kept in different locations.

EPA quality management standards (EPA QA/R

- 2, Quality Improvement Secti on) require that management and staff “ensure that conditions adverse to quality are” prevented, identified promptly, fully defined, corrected, prevented from recurring, and documented as corrective

actions which are tracked to closure. There is a correct ive action process for the performance audit program. There should also be a formal, documented mechanism for elevating potentially significant corrective actions originating from the laboratory staff or field operators.

References:

Recommendation to A ddress Finding: MeCAQMD should institute a formal corrective action process for problems originating in the field or laboratory

.132 Finding # SJV1 Agency: CARB

– SJVAPCD

Date of Audit: Summer 2011 Program Area: SJV

- General

Positive Finding:

In general, the SJVAPCD monitoring program is robust and the agency staff and managers involved in the program are committed to the objective of producing defensible data of known quality. Description: Specific examples of good practices observed during t his audit include:

1. Management and staff at all levels are actively engaged in improving the program

, including monitoring systems and other data collection processes.

2. Staff and managers that participated were professional and helpful during the audit

. 3. Fie ld operators are skilled and knowledgeable.

4. S ite operators receive good hands

- on training with frequent information sharing and issues communication via meetings held every other Monday.

5. A new system is being developed for verifying zero air generators.

6. Th ere are three levels of independent data review and the process is well documented.

References:

Recommendation to Address Finding:

N/A133 Finding # SJV2 Agency: CARB

– SJVAPCD

Date of Audit: Summer 2011 Program Area: SJV

- QA Management

Finding : SJVAPCD does not have updated quality system documentation for all activities. Description: A quality system is the means by which an organization manages the quality of the monitoring information it produces in a systematic, organized manner. It prov ides a framework for planning implementing, assessing and reporting work performed by an organization and for carrying out required quality assurance and quality control activities. While the monitoring staff at SJVAPCD is very knowledgeable and operates t he ambient air monitoring network diligently, the lack of a structured quality system reduces its ability to implement ambient air monitoring requirements effectively and appropriately . Major components of a quality system include:

• Independence of Quality Assurance

.

• QMP, QAPP s and SOP s

.

• Data Quality Performance Requirements (Data Quality Objectives, DQOs)

.

• QA/QC activities

. References: 40 CFR 58 App. A 2.0, Quality System Requirements QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/ b

- 08

- 002

40 CFR 58 App. A 2.1, Quality Management Plans and Quality Assurance Project Plans QA Handbook for Air Pollution Measurement Systems, Volume II, EPA

- 454/b

- 08

- 002

Further guidance on developing QAPPs can be found in the guidance documents “EPA

Re quirements for Quality Assurance Project Plans,” EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002 Recommendation to Address Finding:

SJVAPCD management may adopt CARB’s Quality System docum entation, develop its own, or adopt CARB’s with changes to match its program. Any deviation from the CARB QMP or QAPPs documentation must be formally approved by CARB as the lead agency in the PQAO.134 Finding # SJV3 Agency: CARB

- SJV APCD

Date of Audit : Summer 2011 Program Area: SJV

- Network Management

Finding: SJVAPCD has experienced significant data losses at required monitoring sites, including sites critical for demonstrating compliance with the NAAQS. Description:

There have been several re cent examples of significant data loss due to downtime for temporary site closures for repairs and site relocations, including the Corcoran and Bakersfield

- Golden State Highway sites. T he upgrades were necessary for safety and long

- term longevity of a

sta tion, and the site relocations in question were largely driven by circumstances beyond SJVAPCD’s control. However, these modifications could be implemented in a manner that would minimize the amount of data loss, including better communication or the cons truction of temporary sites to cover data collection during site closure.

References:

Recommendation to Address Finding: SJVAPCD should develop a process to ensure that routine site maintenance or unexpected site relocations do not compromise data com pleteness.135 Finding # SJV4 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

- Network Management

Finding: SJVAPCD has initiated network modifications for several required sites without seeking EPA approval as required by 40 CFR 58.14. Description: Monitoring agencies are required per 40 CFR 58.14 to seek EPA’s approval for network modifications, including site closure or relocation. SJVAPCD has often informally communicated network changes but has not always followed the formal process as required by 40 CFR 58.14. The request submitted to EPA must address how the criteria in 40 CFR 58.14 are met. Early communication between agencies is particularly crucial for high concentration or design value sites in order to develop accepta ble plans for concurrent monitoring at the old and

new sites in order to meet future data needs.

References:

40 CFR 58.14 Recommendation to Address Finding: SJVAPCD should work with CARB and EPA to develop a plan for site closure or relocation that

mee ts agencies’ needs and federal requirements.136 Finding # SJV5 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

- Field Operations

Finding: The residence time of flow between the inlet and each instrument was not posted at each

SJV APCD site. Description: The residence time is defined as the amount of time that it takes for a sample of air to travel from the opening of the cane to the inlet of the instrument. 40 CFR Part 58, Appendix E Section 9 states that for the reactive gases ( O 3 , NO 2 , and SO 2 ) residence times must be less than 20 seconds. Additionally, it is recommended that the residence time within the manifold and sample lines to the instruments should be less than 10 seconds. The station technicians should

calculate the residence time, document it in the station logbook, and periodically verify the data.

There was not a clear record of residence time of the sampling lines at each site. Also, the site operators did not know how recently the residence time had been recal culated. At a minimum, the residence time should be calculated for each instrument after any change is made to the sampling train, such as the removal or addition of other instruments, and posted at each site.

The station technicians should calculate the residence time, document it in the station logbook or other form available at the site, and periodically verify the data.

References: 40 CFR Part 58, Appendix E Section 9 Recommendation to Address Finding:

SJVAPCD should calculate residence times f or all gaseous monitors. SJVAPCD should modify

sites with residence times in excess of 20 s econds , with a goal of 10 s econds , and evaluate any impact on compliance data due to excessive residence times. Residence times should be posted

or accessible on

- s ite. Each time a modification is made to the sampling train, the residence time should be recalculated and posted.137 Finding # SJV6 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

– Field Operations

Finding:

Some SJVAPCD site l ogbooks lacked specific information about the date or type of maintenance performed on an instrument. Description: In general, documentation should show data are of adequate quality

, as well as any related

unusual circumstances. Documentation of the act ivities occurring at monitoring stations should be consistent throughout the network and should, at a minimum, include all repairs, calibrations, audits, or other maintenance performed. Maintaining complete logbooks will help to develop a comprehensive his tory of the station. This will aid field technicians pinpoint and assess problems that may arise with the station and provide important information for data validation.

Overall documentation at sites was generally thorough; however, more specifics shoul d be included in logbooks at the site. For example, an entry noting that maintenance was performed on a certain date should identify the instrument and either what specific activities were performed or where that information can be found. Currently, SJVA PCD does not have a standard system in place to ensure consistency of documentation.

Standardizing logbook entries to include the following may be helpful. This might include:

• Date, time and initials of the person(s) who have arrived at the site

.

• Visi tors

.

• Brief description of the weather ( e.g. , clear, breezy, sunny, raining)

.

• Brief description of exterior of the site. Any changes that might affect the data should be

recorded

– for instance, if someone is parking a truck or tractor near the site, this may explain high NO X values

.

• Any unusual noises, vibrations, or anything out of the ordinary

.

• Records of any station maintenance or routine operations performed

.

• Description of the work accomplished at the site ( e.g. , calibrated instruments, repaired

anal yzer)

.

• Dates that instrumentation were repaired or changed and serial numbers of replacement

instruments

.

• Detailed information about the instruments that may be needed for repairs or troubleshooting

.

• Other pertinent information recorded in other logbooks

. References: QA Handbook Volume II, Section 11.2.3, Instrument and Site Logs138 Recommendation to Address Finding: SJVAPCD should have a more consistent policy as to what station operators enter into the station logbook, including routine maintenance, inst rument repair, audits and calibrations, and logbook closeout comments. Logbooks should be routinely reviewed to ensure that pertinent information is being recorded.139 Finding # SJV7 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

- Field Operations

Finding: There is no documentation of management review of station logbooks and other site activities for SJVAPCD operated sites. Description: The monitoring manager plays a very active oversight role

, including in

- person site visits and checks of log books and maintenance sheets. This practice is very useful and should be documented by initialing the site logbook or maintenance sheet, to indicate what was reviewed.

I f the vacant position of senior technician were filled, that person could assume some of the responsibilities currently performed by the manager, including this oversight role.

References:

Recommendation to Address Finding: The SJVAPCD monitoring manager or senior technician should initial logbook or maintenance

shee t and indicate what dates were reviewed.140 Finding # SJV8 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

- Field Operations

Finding: SJVAPCD site operators do not have a quick visual way to identify changes in instrument perfor mance or QC checks that would indicate instrument issues , nor do they have the ability to remotely check on data or site operations. Description: The SJV monitoring network covers a large geographic area, so identifying efficiencies for site operators in their routine site maintenance is critical for resource management. Operators spend

much time verifying data. Providing remote access capability to real

- time site data or instrument

meta

- data would help identify priority issues and make the operator’s tri ps to the sites more efficient. Visual tools to track instrument performance or QC checks would also reduce the

amount of time needed for level 1 data validation review.

References: The use of control charts to monitor quality control parameters is rec ommended in the QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring Program, December 2008. Recommendation to Address Finding: The district is currently developing a data management system, and should consider t he potential benefit of having a system capable of remote access and different types of data visualization. As SJVAPCD develops new tools for data access and review, the use of control charts to track long

term performance of the instruments graphically sh ould be considered

.141 Finding # SJV9 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

– Field Operations

Finding: SJVAPCD experiences data loss due to instrument malfunction. Description: Even though routine maintenance and cal ibrations are scheduled to minimize it , significant

downtime occurs, possibly the result of running instruments beyond the expected life cycle, and past the time when support from the manufacturer is available.

References:

Recommendation to Address Fi nding: SJVAPCD should budget for equipment replacement according to the life expectancy of the monitor and have backup instrumentation ready for field deployment in the case of a prolonged

instrument breakdown.

SJVAPCD should also compile a list of equi pment replacement needs and share it with EPA in the event that equipment replacement funds become available.142 Finding # SJV10 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

– Data Management

Finding:

It is unclear whether SJV APCD is using appropriate criteria to invalidate or flag PM 10 data. Description: SJVAPCD has a thorough, multi

- level data validation process, but it is unclear that appropriate criteria are being used to invalidate or flag data, specifically in the cas e of continuous PM data. Standard data review and validation procedures should be documented in detail , including the criteria used to flag and invalidate data.

References: Guidance for Preparing Standard Operating Procedures ( QA/G

- 6 ) EPA600/B

- 07/001/Ap ril 2007

. Recommendation to Address Finding: SJVAPCD should develop step

- by

- step instructions for data review and validation in SOPs or QAPPs, including specific criteria for appropriate flagging of data.143 Finding # SJV11 Agency: CARB

- SJV APCD

D ate of Audit: Summer 2011 Program Area: SJV

- Data Management

Finding: SJVAPCD experiences significant resource inefficiencies for staff and management as the current data management system relies solely on manual inputs. Description:

While SJVAPCD’ s current three

- level data review process is very thorough, performing this process entirely by hand is inefficient and very time

- consuming. Implementing a new data management system should decrease the amount of time needed for this task and free up much

needed resources. In developing a new system, all staff involved in the current review process should participate , as well as talking to other monitoring agencies that have recently developed these systems, to ensure that any system under consideration e ncompasses all necessary features.

References:

Recommendation to Address Finding: SJVAPCD should continue to work on upgrading the data management system

, identify ing the components that are necessary for accomplishing SJVAPCD’s monitoring program goa ls.144 Finding # SJV12 Agency: CARB

- SJV APCD

Date of Audit: Summer 2011 Program Area: SJV

- QA Management

Finding: SVJAPCD does not have a formal corrective action process in place. Description: EPA quality management standards (EPA QA/R

- 2, Quali ty Improvement Section) require that management and staff “ensure that conditions adverse to quality are” prevented, identified promptly, fully defined, corrected, prevented from recurring, and documented as corrective

actions that are tracked to closure.

While corrective action seems to be occurring in a timely fashion for most issues, a formal corrective action process would serve as documentation for the issues being resolved, capture the process and keep it consistent through staff or management turno ver, and share the results of the

corrective action with staff.

References: EPA QA/R

- 2, Quality Improvement Section Recommendation to Address Finding: SVJAPCD should institute a corrective action process to provide a formal, documented

mechanism for el evating potentially significant corrective actions originating from field or data review operations.145 APPENDIX A: SUMMARY OF FINDINGS

G1: [Previous Finding M1] CARB needs to complete the process of putting a formal PQAO into place.

G2: The QMB does n ot have the structure and staff to manage QA oversight of the PQAO

districts.

G3: [Previous Finding M6] While progress has been made on updating the CARB QA Manual with a QMP and QAPPs or equivalent documents, the process is behind schedule.

G4: Local di stricts within the CARB PQAO do not always have updated quality system

documentation for all activities.

G5: [Previous Findings QM1 and M3] QA Authority and interactions between QMB and the

ot her b ranches should be expanded and formalized. The corrective action system should be developed to include actions taken, in addition to reports issued by the QA auditors and the calibration laboratory.

G6: Coordination between CARB and districts and EPA should be improved.

NM1: Not all agencies within the CARB PQA O have approved network plans since this became a requirement in 2006. The current approach to network plan s does not provide for a determination of network adequacy on a statewide basis.

NM2: The network assessment does not meet all CFR requirements.

NM 3: There are PM 10 monitors listed in local conditions (LC; parameter code 85101), but not Standard Temperature and Pressure (STP; parameter code 81102 in AQS), as required for

FRM/FEM instruments.

FO1: [Related Previous Findings GB3, SJV3, & NS2] Docum entation at the CARB field sites is inadequate and not reviewed by management.

FO2: Management oversight of site operators needs strengthening.

FO3: CARB field operators have not been trained on new SOPs.

FO4: Residence time calculations were not availa ble at any CARB site visited.

FO5: Delay in sending PM 2.5 samples has resulted in loss of data.

FO6: PM make

- up samples are not being taken in accordance with EPA guidance.146 FO7: PM 10 QC checks are not consistently recorded. There is no document in which field operators are directed to record this information.

FO8: CARB field staff do not check data after sending information to CARB offices.

FO9: The Yuba City site has several significant siting issues that need to be resolved.

FO10: Records indicate t hat calibrations of gaseous pollutant instruments are not consistently done according to a schedule.

FO11: [Previous Finding AQSB7] The number of NO 2 titration points taken during calibration does not meet regulatory requirements.

FO12: Multi

- point calib rations of PM 2.5 instruments are not done routinely.

FO13: [Previous Finding AQSB8] AQSB is not formally documenting the quality of zero air being used in the program.

FO14: Span and precision gases used in the field are not being calibrated routinely.

FO15: Instruments removed from the field are not always efficiently tracked and returned to the repair laboratory facility for diagnosis, repair, and reuse. Loss of data has occurred due to unavailability of spare instruments.

FO16 (Positive): CARB is wor king to improve communication with field staff.

FO17 (Positive): CARB maintains a well equipped stockroom of spare parts, maintains a large equipment purchase order history, and develops thorough equipment testing procedures that are regularly updated.

D M1: The data validation and review/verification performed by AQSB, NLB, and AQAS, are not formally published in a control

- copied SOP.

DM2: [Previous Finding M7] Data submitted by local districts within the CARB PQAO are not validated using consistent proc edures. (See Findings SJV9, IMP10, and MEN11)

DM3: [Previous Finding DM5] AQAS does not ensure that local district data are validated prior to upload to AQS.

DM4: A few instances of erroneous continuous data were identified in AQS for CARB sites.

DM5: E rroneous continuous data were identified in AQS for non

- CARB sites within the CARB PQAO.

DM6: [Previous Finding DM6] There are numerous deficiencies in the data certification process for the CARB PQAO, including:147 • Not all NAAQS

- compliant data within the CA RB PQAO are routinely certified.

• Data certified by CARB for local districts are not typically reviewed or validated.

• Data are routinely certified by agencies within the State of California, but responsibility has not been formally delegated to any local agencies within the State of California.

DM7: D ata, including those used for design value sites, have been changed after they are

certified and subsequently not recertified.

DM8: Some local districts within the CARB PQAO are listed as PQAOs in AQS.

DM9: There were several instances of CARB altering data collected by local districts without communicating with the district.

QA1: The QA Audit group has made an effort to improve its documentation process; however,

several inconsistencies were noted.

QA2: T he audit trailer evaluated was using one expired gas cylinder along with others that had not been certified annually as required for the EPA National Performance Audit Program (NPAP).

QA3: The QA Section is not tracking monitors to ensure that 25% are be ing audited per calendar

quarter.

QA4: The connection to the inlet on the audit trailer could pull in outdoor air.

QA5: Auditors do not review all applicable siting information in AQS prior to an audit.

QA6: [Previous Findings M4 & OPA2] Quality assuran ce for special projects is not developed in a process consistent with EPA quality system requirements.

QA7: Mass flow elements (MFEs) are used to establish calibration points outside of their calibrated range.

PM1: Communication of post

- weigh informatio n and transmission of documentation to local districts should be improved.

PM2: The PM laboratory does not have a formal corrective action process for addressing issues

with PM filter collection.

PM3: Documentation of activities in the PM 10 and PM 2.5 lab oratories should be improved.

PM4: PM 10 trip blanks are not being used to assess potential bias from filter transport and handling.

TL1: The canister cleaning SOP does not reflect the current cleaning procedure

.148 TL2: An SOP is not documented for the b atch certification of cleaned canisters. The canister cleaning SOP lists cleaning criteria for the MLD 058 method, but not for the MLD 066 method.

TL3: The batch certification of cleaned canisters described by staff for methods MLD 058 and

MLD 066 diffe rs from existing VOC guidance.

TL4: [Previous Finding 19]. Pre

- cleaning concentrations are not recorded in a logbook to allow for the selection of the most highly contaminated canister for batch certification.

TL5: Canisters are not routinely leak t ested as prescribed in guidance. Instead, canisters are vacuum leak tested only when gross leaks are suspected.

TL6: [Previous Finding 21]. A retention time policy for re

- cleaning and blanking canisters once they have been certified clean has not been established.

TL7: The CARB SOP states that old canisters are reconditioned, but this is inconsistently practiced.

TL8: CARB has not established a holding time for cartridges once samples have been collected for extraction or analysis.

TL9: The labo ratory does not assign expiration dates to new sampling cartridges and allows cartridges to be used beyond the 90 days prescribed by the method.

TL10: CARB's procedure for analyzing DNPH lot blanks differs from the SOP.

TL11: No criterion is provided in the CARB SOP for passing DNPH lot cartridge blanks.

TL12: Gloves are not worn as a contamination protection measure when handling cartridges. A nitrogen

- purged glove bag is not used for extractions.

TL13: [Previous Finding OL3] Staff stated tha t field blanks are not being analyzed at a frequency of 10% of field samples, as specified in Method TO

- 11, nor is there an SOP describing the procedure for the submission of field blanks.

TL14: CARB does not analyze trip blanks when needed.

TL15: Me thod TO

- 11 states that samples should be re

- analyzed when results are 10% above the criterion, but the analyst was not aware of this criterion.

TL16: Working standards are tracked and used for six months, while the CARB SOP states that

standards should be retained for four months under refrigeration.

TL17: Site name and sampling dates are recorded on a piece of tape loosely stuck to sample cartridges; the tape occasionally falls off, making it difficult to identify samples.149 TL18 (Positive): Hexavalent ch romium d ata undergo peer review, supervisory review, review by the Branch Chief, and a final review before going to AQS.

TL19: [Previous Finding OL5] There is no secondary review of logbooks.

TL20: CARB does not analyze audit samples or through

- the

- pr obe audit samples as suggested in

Sec. 9.7 of the CARB SOP.

TL21: Appendix V in the CARB SOP lists the standards that were used in 2003 and has not been updated to reflect the standards currently being used.

IMP1: The ICAPCD ambient air monitoring pr ogram is not operating under an approved QAPP.

IMP2: ICAPCD has not established an appropriate quality system for ambient air monitoring.

IMP3: Assessment of PM 10 or PM 2.5 sampling frequency throughout the ICAPCD network has not been performed as require d.

IMP4: Neighborhood scale may be inappropriate for PM 10 at the Westmorland site.

IMP5: One

- point flow rate verifications for PM 10 and PM 2.5 are not performed by ICAPCD as required and are not well documented.

IMP6: Residence time for gaseous monitors operated by ICAPCD is not established.

IMP7: ICAPCD is internally post weighing high

- volume PM 10 filters without proper PM lab or quality control measures.

IMP8: Documentation of ICAPCD air monitoring activities is not complete.

IMP9: There are potentia l siting issues at the Calexico Ethel site.

IMP10: ICAPCD is not adequately reviewing and editing data.

MEN1: MeCAPCD staff w as not familiar with the CARB QMP or instrument SOPs.

MEN2: MeCAQMD has been part of the CARB PQAO since PQAOs were created in 2 006 but is erroneously listed as its own PQAO in AQS.

MEN3 (Positive): MeCAQMD stations were well

- maintained. Staff and the manager were professional and helpful, and very knowledgeable about the county and potential pollution sources. The station operat or was proactive about troubleshooting instrument issues.

MEN4: One

- point QC checks (flow verifications) for PM 10 and PM 2.5 are not consistently performed by MeCAPCD site operators.150 MEN5: MeCAPCD logbook entries are not consistently made and are not al ways in the most defensible form. Handwritten notes are occasionally illegible due to water (rain) marks.

MEN6: Residence time calculations were not available at the Ukiah Gobbi site.

MEN7: Trees at the Ukiah Gobbi and Library sites should be evaluated against siting requirements.

MEN8: The internal shelter thermostat is not operating correctly at the Ukiah Gobbi site and the

issue has not been addressed to provide defensible data

.

MEN9: MeCAPCD has no system for tracking and controlling station and in strument logbooks.

MEN10: MeCAQMD should have formalized training requirements for all air monitoring staff.

MEN11: MeCAQMD does not provide CARB AQAS with datasets that have been fully quality assured and ready for upload to AQS.

MEN12: MeCAQMD does no t use a formal corrective action system.

SJV1 (Positive): In general, the SJVAPCD monitoring program is robust and the agency staff and managers involved in the program are committed to producing defensible data of known quality.

SJV2: SJVAPCD does not h ave updated quality system documentation for all activities.

SJV3: SJVAPCD has experienced significant data losses at required monitoring sites, including sites critical for demonstrating compliance with the NAAQS.

SJV4: SJVAPCD has initiated network mod ifications for several required sites without seeking EPA approval required by 40 CFR 58.14.

SJV5: The residence time of flow between the inlet and each instrument was not posted at every

SJVAPCD site.

SJV6: Some SJVAPCD site logbooks lacked specific inf ormation about the date or type of maintenance performed on an instrument.

SJV7: There is no documentation of management review of station logbooks and other site activities for SJVAPCD operated sites.

SJV8: SJVAPCD site operators do not have a quick vis ual way to identify changes in instrument performance or QC checks that would indicate instrument issues , nor do they have the ability to remotely check on data or site operations.

SJV9: SJVAPCD experiences data loss due to instrument malfunction.151 SJV10: It is unclear whether SJVAPCD is using appropriate criteria to invalidate or flag PM 10 data.

SJV11: SJVAPCD experiences significant resource inefficiencies for staff and management as the current data management system relies solely on manual inputs.

S JV12: SVJAPCD does not have a formal corrective action process in place.152 APPENDIX B: CARB ORGANIZATIONAL CHARTS

Organization of CARB’s Divisions and Offices153 Organizational Chart for CARB’s Monitoring and Laboratory Division (MLD)154 Organizational Cha rt for CARB’s Planning and Technical Support Division155 APPENDIX C: CARB DATA VALIDATION DOCUMENTS PROVIDED BY DATA

VALIDATION STAFF 14

14 CARB’s QAPP is the formal document that outlines data validation procedures for CARB.156157158159160