## System Audit of the Ambient Monitoring Program:

Bay Area Air Quality Management District

May 26

– June 4, 2009

Conducted by:

Air Quality Analysis Office Quality Assurance Office

Air Division Management and Technica l

Services Division

US EPA Region 9

75 Hawthorne Street San Francisco, California 94105

Final Report: November 20 10ii INDEX

PAGE

GLOSSARY OF ACRONYMS

.......................................... .............................. iii

EXECUTIVE SUMMARY

................................................................................ 1

INTRODUCTION

............................................................... ...............................3

A. GENERAL PROGRAM

.......................... .......................................................... 5

B. NETWORK MANAGEMENT

........................................................................ .7

C. FIELD OPERATIONS

............................................................................... ...... . 9

D. LABORATORY OPERATIONS

..................................... ...............................14

E. DATA AND DATA MANAGEMENT

............................................................ 22

F. QUALITY ASSURANCE AND QUALITY CONTROL

................... .. ........ 2 6

APPENDIX A, Summary of Findings

............................................................ .3 2

APPENDIX B, Good Practice Recommendations

....................................... ..35iii GLOSSARY OF ACRONYMS

APCD......

......... ..................... . Air Pollution Control District AQS......................................

. Air Quality System BAAQMD

............................. . Bay Area Air Quality Management District

BAM.......

............................... Beta Attenuation Mas s Monitor (continuous PM 2.5 )

CARB

.................................... . California Air Resources Board

CARE....................................

. Community Air Risk Evaluation

CO........................................

. . Carbon Monoxide

CFR..................

.................... . . Code of Federal Regulations

DMS

...................................... . Data Management System DQO....................................... Data Quality Objective

EC........................................

.. Elemental Carbon

EPA........

............................... Environmental Protection Agency

FEM......................................

. Federal Equivalent Method

FRM......................................

. Federal Reference Method

GC

- MS

.................................. . Gas Chromatogr aphy

- Mass Spectrometry

MDL

...................................... . Method Detection Limit

MET/PE................................

. Meteorology and Performance Evaluation NAAQS................................

. . National Ambient Air Quality Standard

NAMS......

............................ . . National Air Monitoring Station

NIST

....................................... National Institute for Standards and Technology NOx/NOy.............................

.. Nitrogen Oxides

O 3

.......................................... . Ozone

OC........................................

.. Organic Carbon

ODAMN.

............................... Operations Data Action Management Monitoring

Notification

ORD......................................

. Office of Research and Development

PAMS

............ ......................... Photochemical Assessment Monitoring Stations

PE..........................................

. Performance Evaluation

PM........................................

.. Particulate matter

PM 2.5

..................................... . Particulate matter 2.5 microns or less in aerodynamic

diameter

PM 10

...................................... . Particulate matter 10 microns or less in aerodynamic

diameter PQAO....................................

. Primary Quality Assurance Organization QA....

.................................... .. Quality Assuranceiv

QAPP...................................

.. Quality Assurance Project Plan QC.......................................... Quality Control

SPM.....................................

.. Special Purpose Monito r SOP......................................

.. Standard Operating Procedure

SLAMS................................

.. State or Local Air Monitoring Station

SO 2

....................................... .. Sulfur Dioxide

SRP

........................................ Standard Reference Photometer

STN

........................................ Speciation Trends Network

TAD......................................

. Technical Assistance Document

TSA......................................

.. Technical System Audit

USEPA or EPA..

................ .. .. United States Environmental Protection Agency

VOC

....................................... Volatile Organic Compound1

EXECUTIVE SUMMARY

This document is a report on the findings made by the U.S. EPA Region 9 while conducting a Technica l Systems Audit (TSA) on the air monitoring program of the Bay Area Air Quality Management District (BAAQMD). A TSA is an on

- site review and inspection of a state or local ambient air monitoring program to assess its compliance with established regulations governing the collection, analysis, validation, and reporting of ambient air quality data. This TSA meets the requirements for EPA Regional 9 audits of the BAAQMD monitoring organization required by 40 CFR Part 58, Appendix A, Section 2.5.

The TSA was conducted by U.S. EPA Region 9 staff during the weeks of May 26

– June 4, 2009. The audit team interviewed management and staff on specific aspects of the air monitoring program including network design, field operations, laboratory operations, data

handl ing, quality assurance and quality control procedures. The audit team also inspected some of the monitoring sites operated by BAAQMD. The site inspections consisted of an interview with the site operator (when possible), reviewing station and instrument logbooks, and evaluating the station siting with respect to EPA requirements for probe siting 1 . The laboratory inspection included a review of the particulate matter program for mass determinations, laboratory analysis for volatile organic compounds (VOC) , OC/EC and carbonyl sample analysis.

The TSA is one of the ways that EPA provides oversight to ensure that data collected by state and local agencies meets certain minimum data quality objectives. Other assessments such as network reviews and performa nce evaluations are also used to collect information on the overall quality of ambient air monitoring data. These assessments also enable agencies to identify and correct those program elements which may be adversely affecting the quality of

ambient air d ata. The results of the TSA are summarized here and fully described in this report, along with recommended actions to address the findings. The specific actions to be taken by BAAQMD will be determined through negotiations between EPA and BAAQMD and will be documented in a corrective action plan prepared by BAAQMD.

EPA would like to thank all the staff and management of Bay Area Air Quality Management District for their support and cooperation during the audit.

A. Program Strengths: BAAQMD has a ver y robust ambient air monitoring program, as evidenced by the very high quality data with very little missing data. This can be attributed to a combination of skilled and

dedicated field and PE/QA staff, sound analytical work in the laboratory, highly comp etent validators, and committed managers. Some specific examples of the strength of BAAQMD’s ambient air monitoring program include:

 BAAQMD managers exhibit a solid awareness of air quality issues within the San Francisco Bay Area along with a strong unde rstanding of EPA network requirements and

recommendations . The result i s very effective network planning.

1 40 CFR, Part 58, Appendix E, “Probe and Monitoring Path Siting Criteria for Ambient Air Quality Monitoring”2

 BAAQMD has developed an exemplary network plan.

 Field station operators have primary responsibility for at most three sites and therefore keep the in struments and sites in good working condition

.

 The District has a ‘Station Operator SOP’ which provides operators with an overview of their many responsibilities in running a monitoring station.

 The troubleshooting SOP for station operators compiles techn ical insights from other operators and gives direction for troubleshooting problems previously encountered.

 The laboratory has established an effective system of tracking and documenting instrumental quality control results using control charting in the OC /EC analysis area.

B. Program Major Findings:

During the TSA, EPA identified areas where the BAAQMD monitoring program should be strengthened. The major findings are : 1. BAAQMD has had significant quality issues with the Thermo Scientific Partisol FRM,

Mo del 2000.

2. BAAQMD lacks a QA Officer position whose sole responsibility is to oversee quality management of the laboratory.

3. The BAAQMD QAPP’s approval is conditioned on establishing DQOs for data uses that are not defined by EPA.

4. Most quality management functions are being performed; however, QA authority is not clearly defined.

5. BAAQMD does not have a formal corrective action system outside of the performance audit program.

The individual findings , along with minor findings, are reported in the topic se ctions of this document and are also summarized in Appendix A. In addition to those areas which should be strengthened, EPA also identified areas which could be strengthened and those recommendations and suggested best practices are included in Appendix B for the information and use of the BAAQMD’s management.3

INTRODUCTION

The Bay Area Air Quality Management District (BAAQMD or the District) has been in existence for more than 40 years and has primary responsibility for ensuring the health and

welfare of citizens of the San Francisco Bay Area under the Clean Air Act of 1970 and its Amendments of 1990. BAAQMD operates the ambient monitoring network for seven full counties (Marin, Napa, Contra Costa, Alameda, Santa Clara, San Mateo, San Francisco) and tw o partial counties (southern Sonoma and western Solano).

From May 26 through June 4, 2009, EPA Region 9 conducted a Technical System Audit (TSA) of the ambient monitoring program operated by BAAQMD for all program areas except certain aspects of data m anagement. BAAQMD is in the process of developing and implementing a new data management system (DMS) and therefore issues specifically pertaining

to the DMS have been deferred until the next TSA . In its entirety, the TSA covers the following

program are as:

Network Management Network Design and Monitoring Station Siting Organization, Staffing and Training

Facilities Field Operations Routine Operations Field Quality Control

Preventive Maintenance Record Keeping Data Acq uisition and Handling Laboratory Operations Routine Operation Laboratory Quality Control

Preventive Maintenance Record Keeping Data Acquisition and Handling Specific Pollutants:

PM

VOCs Carbonyls

OC/EC

Data and Data Management Data Handling

Software Documentation

Data Validation and Correction Data Processing Internal Reporting External Reporting Quality Assurance and Quality Control Status of Quality Assurance Program4 Audi ts and Audits System Traceability National Performance Audit Program (NPAP)

Audit Procedure Review Documentation and Data Processing Review Corrective Action System Audit Result Acceptance Criteria

Participating managers and supervisors of t he Technical Services Division and positions

at the time of the TSA were:

Eric Stevenson, Air Monitoring Manager Jim Hesson, Laboratory Services Manager Dick Duker, Meteorology/Performance Evaluation Manager Mark Stoelting, QA Officer Jeff Robbins, Laborat ory QA Officer Bob Franicevich, Performance Evaluation Supervisor Stan Yamaichi, Air Monitoring Supervisor Glen Colwell, Air Monitoring Supervisor

The EPA regional staff members conducting the TSA were Meredith Kurpius, Catherine Brown, and Michael Flagg of the Region 9 Air Division’s Air Quality Analysis Office and Mathew Plate, Roseanne Sakamoto and Steve Remaley of the Region 9 Quality Assurance Office. Eugenia McNaughton, Manager of the Quality Assurance Office attended the exit meeting representing E PA Management.

This report is divided into ten sections. The first section is an executive summary that describes the purpose of the TSA and a summary of the major findings. The body of the report includes both major and minor findings . Minor findings are those that have a smaller impact on the integrity of the program but still require corrective action. The second section is an introduction that provides a brief description of the organization of BAAQMD’s air monitoring program, the EPA audit team, and the report organization. The third section discusses our general findings on the BAAQMD monitoring program as a whole. The remaining five sections address specific aspects of the air monitoring program: network management, field operations,

laborator y operations, data management and quality assurance/quality control. Appendix A is a summary listing of the findings contained in this report. Appendix B is a list of recommended good practices that are not required by Federal regulation but would streng then BAAQMD’s monitoring program, nonetheless. The findings and recommendations in this report are listed in priority order within each section. Recommendations to resolve findings are provided to give some indication of the Region’s expectations as to h ow findings can be addressed. If BAAQMD has other approaches or alternatives to address the concerns identified, the Region will consider them, provided the corrective action adequately addresses the finding.5

A. GENERAL PROGRAM

The Bay Area Air Quality Man agement Dist r ict has been in existence for more than 40 years and has primary responsibility for ensuring the health and welfare of citizens of the San Francisco Bay Area under the Clean Air Act of 1970 and its Amendments of 1990. The District

conducts am bient air quality monitoring in support of its role in developing and enforcing air pollution control regulations. BAAQMD is an independent agency with more than 300 employees, governed by a 21

- member Board of Directors. The Board is made up of publicly

elected officials apportioned according to the population of the represented county.

To ensure that the policies and regulations adopted by the Board are working to improve the quality of the air in the San Francisco Bay Area, BAAQMD continuously monitors air quali ty at 24 SLAMS locations and eight SPM locations throughout the nine

- county area

. In addition, CARB operates one non

- FEM BAM PM 2.5 monitor in Point Reyes and Monterey Bay Unified Air Pollution Control District operates one FRM PM 2.5 monitor in t he city of Hollister

which is in Northern San Benito County. This network also allows BAAQMD to notify the public whenever air quality reache s levels regarded as unhealthy

.

The BAAQMD sections that are the subject of this TSA are Air Monitoring, Laborat ory Services, and Meteorology and Performance Evaluation (MET/PE), which are all in the

Technical Services Division and are under the direction of Technical Services Director . At the

time this TSA was conducted Gary Kendall was the Director of Technical S ervices ( see attached organization chart 1 ). Source Test operations, which are also within the Technical Services Division, were not part of this audit.

The Air Monitoring Section, managed at the time of TSA by Eric Stevenson, oversees field station oper ations. Within this section there are two supervisors, who were Stan Yamaichi

and Glen Colwell at the time of the TSA , and 16 technical staff.

The Laboratory Services Division, managed by Jim Hesson, has five chemists and an additional vacancy that is n ot currently being filled. One of the chemists, Jeff Robbins, also

serves as the laboratory QA Officer.

Meteorology and Performance Evaluation (PE) are grouped into one section (MET/PE), managed at the time of the TSA by Dick Duker. The QA Officer res ponsibility for ambient air monitoring resides within this section. Mark Stoelting, who is also an air and meteorology specialist, was recently assigned to this position. P erformance E valuation (e.g., field audits) is

also within this section

, but has be en separated from other QA activities and is supervised by

Bob Fran i cevich. In addition to the manager, this section has one QA Officer/Air and Meteorology Monitoring Specialist, one PE Supervisor, four PE technical staff, and four meteorologists.

1 Since this TSA was conducted numerous management changes hav e occurred: Gary Kendall has retired and been replaced by Eric Stevenson; Eric Stevenson was replaced by Glen Colwell; and Dick Duker retired and has not yet been replaced.6

The or ganization of BAAQMD’s Technical Services Division divides QA responsibility among various sections and various personnel: Laboratory QA Officer, QA Officer within the MET /PE Section, and PE Supervisor within the M ET /PE Section. All QA functions are embe dded within the sections and rather than having a formal QA role to oversee all ambient air monitoring operations at the management level.

FINDING A1 (Major) :

BAAQMD has had significant quality issues with the Thermo Scientific Partisol FRM, Model

2000.

Discussion: Due to the discontinuation of support for the Andersen PM 2.5 samplers

, BAAQMD converted their monitoring network to Partisol monitors. For conversion of a larg e monitoring network

, BAAQMD found that the Partisol was the only FRM instrumen t available. This is due to a deficit of currently available models of filter

- based reference methods. BAAQMD subsequently has had problems with instrument performance and vendor support.

Recommendation: USEPA Region 9 needs to follow

- up wit h OAQPS and ORD on the issues of lack of an adequate number of viable FRMs and vendor support for FRMs.

FINDING A2:

BAAQMD keeps multiple hard copies and electronic copies of many documents, making it unclear which is the official copy.

Discussion:

M any of the BAAQMD documents are generated electronically and subsequently printed. This procedure generates both an electronic record and one or more hard

- copy records. Generally BAAQMD retains both records. However, at a later time the hard copy

record s may be annotated and the electronic records c an be modified. In some circumstances it was unclear to staff interviewed which are the preferred records to be

referred to and retained.

Recommendation: For records that have multiple copies BAA QMD should designate the official copy and handle it appropriately. S taff should understand which records are official documents and how they should be handled.7

B. NETWORK MANAGEMENT

The ambient air monitoring network in San Francisco Bay Area consis t ed of 32 monitoring sites at the time this TSA was conducted (25 State/Local Air Monitoring Stations ( SLAMS ) plus nine Special Purpose Monitoring ( SPM) sites) that utilize a variety of air pollutant measuring instruments. The following table summarizes t he SLAMS air monitoring sites in the San Francisco Bay Area.

SITE O 3 CO NO

x SO 2 PM 10 FRM

PM 2.5 BAM*

PM 2.5 Bethel Island X X X X X

Concord X X X X X X

Fremont X X X

Livermore X X X X X X Napa X X X X X

Oakland X X X Pittsburg X X X X X Redwood City X X X X X X

San Francisco X X X X X X X

San Jose

– Jackson X X X X X X

San Jose

– Tully X

San Pablo X X X X X

San Rafael X X X X

Santa Rosa X X X X X Vallejo X X X X X X X

Crockett X

Fairfield X Gilroy X X Hayward X

Los Gatos X

Martinez X

San Leandro X

San Martin X Sunnyvale X

Hollister + X

* Non

- FEM BAM

+ operated by Monterey Bay Unified Air Pollution Control District Source: BAAQMD’s “Annual Air Qu ality Monitoring Network Plan 2008”

BAAQMD also operates eight special purpose monitoring sites (SPM) and CARB operates one within the San Francisco Bay Area:

 Point Reyes (CARB)

– Non

- FEM BAM PM 2.5

 Oakland, West Grand

– PM 2.5 speciation

 Point Richmond

– H 2 S8

 Richmond

– SO 2 , H 2 S

 Rodeo

– H 2 S

 Benicia

– O 3 , CO, NO

x , SO 2 , PM 10 , Non

- FEM BAM PM 2.5

 Berkeley

– O 3 , CO, NO

x , SO 2 , CH 4 , PM 10 , Non

- FEM BAM PM 2.5 , VOCs, metals

 Cupertino

– Hexavalent chromium

 Hunter’s Point

– asbestos

In addition, t he District operates a network of air toxics stations, including a NATTS at

San Jose

– Jackson. This audit addressed air toxics activities that support the NATTS, including

both field and laboratory operations. The San Jose

– Jackson site is also the NCore site ( N ational

Core multipollutant monitoring station ). Other air monitoring activities include PM 2.5 speciation monitoring at San Jose

– Jackson, Vallejo, Livermore, and Oakland

– West Grand. BAAQMD is also establishing PAMS

- like monitoring at Livermore and two new sites in eastern Contra Costa County.

The air monitoring network within the San Francisco Bay Area meets or exceeds the requirements for the minimum number of monitoring sites designated as SLAMS for all of the criteria pollutants. BAAQMD managers exhibit broa d knowledge of air quality issues within the San Francisco Bay Area along with a strong understanding of EPA network requirements and

recommendations . The result is very effective network planning. In addition, t he monitoring sites are all properly locat ed in terms of monitoring objectives and spatial scales.

The District submits an annual monitoring network plan . The most recent submittal, in 2008, includes all of the required information about the monitoring stations. 40 CFR 58.20 requires States t o perform an annual review of their existing ambient monitoring networks to ensure that they continue to meet the monitoring objectives of 40 CFR 58, appendix D. The current BAAQMD document meets the requirements of describing the network and the intended monitoring objectives of each site and is, in fact, an exemplary document that has been provided to other agencies as a model.

According to 40 CFR 58.10 (d), “The State, or where applicable local, agency shall perform and submit to the EPA Regional Adm inistrator an assessment of the air quality surveillance system every 5 years to determine, at a minimum, if the network meets the monitoring objectives defined in appendix D to this part, whether new sites are needed, whether existing sites are no longer needed and can be terminated, and whether new technologies are appropriate for incorporation into the ambient air monitoring network.” The first network assessment is due July 1, 2010. BAAQMD is aware of this requirement and the deadline for

submission a nd is preparing to undertake this effort.

There are no findings that require corrective action related to network management.9

C. FIELD OPERATIONS

Field operation duties for monitoring are primarily managed by the Air Monitoring Section with support fo r instrument audits handled by the MET/PE S ection. Both sections are

with in the Technical Services Division but are separate sections with different managers. Air Monitoring S ection staff handle all day

- to

- day operations including: instrument installati on, repair and maintenance; sample collection; first

- level data review; some quality control activities; and general field site maintenance. Most field operators are responsible for 2

- 3 sites and visit each site multiple times per week. Generally we found that the station operators have a very high level of oversight and a comprehensive understanding of their field sites. As a result, BAAQMD has very robust ambient air quality data with very little missing data

.

The main function of the Performance Eval uation Group is to assess quality control in the Air Monitoring Section by conducting performance evaluations of gas analyzers and particulate samplers at the District’s air

- monitoring sites. The group also performs calibrations and verifications of pressu re and temperature standards for the Air Monitoring Section as a service for the division. Other than pressure and temperature standards, PE staff are not involved in any Air Monitoring Quality Control activities. These activities are more thoroughly add ressed in the Quality Assurance section.

As part of this TSA, managers and supervisors for the Air Monitoring and MET/PE

S ections were interviewed. We also spoke with station operators at all the stations visited and reviewed logbooks . Station logboo ks are archived on site whereas instrument logbooks remain with the instrument. All station operators demonstrated a thorough knowledge of the monitoring equipment for which they were respon sible and expressed a sense of ownership of the data. Current copi es of QAPPs and SOPs were available to all station operators electronically at most field stations via secure access to the District’s central computing network.

The field sites visited were: Bethel Island, Concord, Livermore, Napa, Oakland (Internatio nal Blvd.), San Francisco, San Jose (Jackson St.), Vallejo, Fairfield, Gilroy, and Berkeley (SPM site). Most sites were generally in pre

- existing commercial buildings rented by

the District with inlets or instruments located on the roof. Some sites with o ne or two monitors were outfitted simply with mounted containers to house instruments and supplies. Each station operator has a “home station” that serves as the operator’s office. The home stations contain basic office functions such as computing, netwo r king, and printing capabilities along with tools, supplies, storage capability, and other field necessities.

Our evaluation at these sites included inspection of the inlet manifolds, examination of station and instrument logbooks, and a specific check o f whether appropriate QC checks and QA audits were being performed for dates in 2008 selected for review prior to the audit. All sites

were well

- operated and met the siting criteria of 40 CFR 58, Appendix E. For example, instruments were sited appropriate ly away from each other and obstacles/trees, sites had clean inlet manifolds, calibration standards were current, and station logbooks and instrument logbooks were generally up

- to

- date and contained relevant information on operations and

maintenance activi ties.10

Overall we found the field station operations to be very strong and we identified many positive findings, including:

 Very competent and dedicated field station operators and managers.

 Field station operators have primary responsibility for at most three sites and therefore keep the instruments in good working order

.

 The District has a ‘Station Operator SOP’ which provides operators with an overview of their many responsibilities in running a monitoring station.

 Most long

- term monitoring stations ar e housed in permanent structures with sufficient amenities and space, which ensures smooth, efficient and convenient field operations.

 The troubleshooting SOP for station operators compiles technical insights from other operators and gives direction for t roubleshooting problems previously encountered.

FINDING C1 :

PM10 flow checks may not be occurring at the required frequency at all sites.

Discussion: As of December 2006 flow checks for PM10 became a regulatory requirement. The requirement is once a mo nth for low volume samplers (40 CRF Part 58, Appendix A, Section 3.2.3) and once a quarter for high volume samplers (40 CRF Part 58, Appendix A, Section 3.3.2). One of the station operators noted that PM10 flow checks are performed annually and t he PM10 i nstrument logbook confirmed that these checks were being performed no more frequently than annually.

Recommendation: BAAQMD should ensure that flow rate checks are conducted and documented at the appropriate frequency.

FINDING C 2 :

BAAQMD operates many o f their gaseous monitors using sample pumps that deviate from the

reference method.

Discussion: USEPA defines methods for establishing Federal Reference and Equivalent Methods (FRMs and FEMs) in 40 CFR Part 53. All the monitors being used by BAAQMD for

c ompliance with the NAAQS are either FRM or FEM monitors. It must be demonstrated

that a ny modification to FRM or FEM monitoring equipment does not impact the equivalency of the instrument and approved by USEPA.

BAAQMD has indicated that changing to the s ampling pumps has resulted in significant cost and energy savings. USEPA Region 9 is supportive of these changes and, based on monitoring network performance, has confidence that these deviations could be approved. USEPA Region 9 and BAAQMD have consult ed USEPA Office of Research and Development (ORD) to determine if using different sample pumps qualifies as a11

change to a FRM. USEPA ORD indicated that this change would qualify as a modification requiring USEPA approval.

Recommendation: Develop and subm it documentation that demonstrates that changing the instrument sample pumps does not significantly change instrument performance.

FINDING C 3 :

Several different SOPs appear to apply to instrument calibrations that are not consistent. Additionally, docum entation of calibration was hard to follow and incomplete.

Discussion: USEPA defines how gaseous instruments must be calibrated in 40 CFR Part 50.

BAAQMD has defined gaseous calibration procedures in both the instrument SOPs and in SOP 220. However, t hese procedures do not appear to be consistent. This particularly evident when comparing SOP 711, for NOx monitors, which is detailed and follows 40 CRF Part 50, Appendix F very closely, to SOP 220.

The only documentation of the calibration procedure app ears to be included on the sheet titled “TECO 146C Calibration System Control Settings.” However, this worksheet is hard to follow and does not include evaluations of linearity and converter efficiency for

NO 2 . Additionally, the low calibration level for ozone does not adequately address linearity at concentrations near or below the NAAQS of 75 ppb. It was also noted that the worksheet is annotated with additional information by site operators between

calibrations.

Recommendation: BAAQMD needs to reconc ile its gaseous calibration SOPs, ensure that the calibration procedures used conform or 40 CFR Part 50, and improve documentation of calibration.

FINDING C4 :

The level of information recorded in the station logbooks is not consistent and at some sites t he logbooks are not frequently used.

Discussion: Station logbooks are generally used to record all pertinent information regarding station operations. This could include : days the site operator was present at the site ; site visitors ; instrument or site m aintenance; instrument QC checks ; i nstrument repair or replacement;

instrument calibration ; weather conditions ; unexpected problems; and other information not included in another logbook.

The level of information noted to be included in station logbooks, in BAAQMD SOP 101, is minimal. Additionally, the level of information found in the field varied by12

operator. In some cases most logbook entries are not made by the routine site operator. The following specific examples were noted:

 One of the items requi red by SOP 101, “Dates that instrumentation were repaired or changed out and serial numbers of replacement instruments” was not included in the logbook.

 At the Napa site it was noted that the bi

- weekly instrument filter changes were not being documented in the instrument logbook but instead on a piece of masking tape on the back of the instrument rack. This is a good practice to help the site operator remember when the filter was changed last. However, a record of each filter change should also be include d in the instrument logbook.

 Reagent changes and maintenance to the zero air scrubber at the Fairfield site are not being documented.

 Some archived logbooks did not contain a closeout comment . For example, an appropriate closeout comment would include dat e, last entry, and new logbook

with control number XXXXX to be used

, followed by ha t ch marks. Whenever a new logbook is employed, the old logbook should be closed out with an entry

such as noted above. This is to ensure that no additional entries are mad e in the old logbook. Any entries made below the statement and within the ha t ch marks

are not considered official documentation.

 The calibration of the NOx instrument, at the Vallejo site, conducted on 12/29/08

was not found in the instrument logbook.

St andardizing logbook entries to include the following may be helpful. This could

include:

 Date, time and initials of the person(s) who have arrived at the site

 Visitors

 Brief description of the weather (e.g., clear, breezy, sunny, raining)

 Brief descript ion of exterior of the site. Any changes that might affect the data

should be recorded

– for instance, if someone is parking a truck or tractor near the site, this may explain high NO X values

 Any unusual noises, vibrations, or anything out of the ordinary

 Records of any station maintenance or routine operations performed

 Description of the work accomplished at the site (calibrated instruments, repaired analyzer)

 Dates that instrumentation were repaired or changed out and serial numbers of replacement instr uments

 Detailed information about the instruments that may be needed for repairs or troubleshooting

 Other pertinent information recorded in other logbooks Source: QA Handbook Volume II, Section 11.2.3, Instrument and Site Logs

Recommendation: BAAQMD shou ld have a more consistent policy on what station operators enter into the station logbook, including routine maintenance, instrument repair, audits and calibrations, and logbook closeout comments. Logbooks should be routinely reviewed to ensure that perti nent information is being recorded.13

FINDING C5 :

Toxics samplers are not routinely verified by checking with blanks and/or standards for

contamination or losses.

Discussion:

USEPA Method TO

- 14A, Sections 11.2.2 and 11.2.3 and other USEPA documents

descri be methods for sampling system certifications. In general, these certifications are completed to verify that the sampling system is capable of adequately recovering the analyte of interest from ambient air. For example, TO

- 14A Section 11.2.2 describes

hu mid air zero certification for VOC sampling. Under this section certification is defined as “evaluating the sampling system with humid zero air and humid calibration gases that pass through all active components of the s ampling system. The system is ‘cer tified’ if no significant additions or deletions (<0.2 ppbv of targeted compounds) have occurred when challenged with the test gas stream.”

BAAQMD does not have a program to certify the toxic sampling systems. Additionally, the toxic sampling systems a re not routinely checked using blank air to verify that they are not introducing contaminants into the sampling containers/media.

Recommendation: To produce definitive toxics data

, BAAQMD should use sampling systems that are routinely certified and screen ed for blank contamination.

FINDING C 6 :

I nstrument SOPs were not available at some sites with limited monitors (e.g., Fairfield and Gilroy).

Discussion: There is no remote access computer installed at some sites to a ccess SOPs or other

information. Nei ther were hard copies of SOPs maintained at the site or carried by the station operator.

Recommendation: Ensure that field operators have access to information in r elevant SOPs at the field

station , including remote field stations

.14

D. LABORATORY OPER ATIONS

Analytical laboratories provide support for measurement methods that are either too complex or sensitive to perform in the field environment. In order to provide these services, laboratories employ complex instrumentation and staff with highly spe cialized training.

For ambient air samples to provide useful information or evidence, laboratory analyses must meet the following four basic requirements:

1. Equipment must be frequently and properly calibrated and maintained

2. Personnel must be qualified t o make the analysis

3. Analytical procedures must be in accordance with accepted practice

4. Complete and accurate records must be kept

Laboratory operations are managed by Jim Hesson, Manager of the Laboratory Services Section, within the Technical Services Di vision. BAAQMD’s laboratory facility is located in the District’s headquarters office in San Francisco, California. Most analyses are performed in

house by BAAQMD laboratory staff. Those compounds for which BAAQMD does not have staff or laboratory resou rces to perform are sent to support laboratories such as ERG and DRI for analysis. In addition, BAAQMD provides laboratory support services for other districts, such as

PM 2.5 weighing for North Coast Unified AQMD and Monterey Bay Unified APCD. One of the primary responsibilities of laboratory is the handling of PM filters which includes preparation, weighing, tracking, and storing PM 2.5 and PM 10 filters. In addition to PM responsibilities, the laboratory is also responsible for sample analyses for air to xics (including NATTS), PAMS

- like monitoring, and special projects run by BAAQMD such as monitoring near the Port of Oakland as part of the BAAQMD CARE project. The compounds that BAAQMD routinely analyzes for

include:

- VOCs (currently by TO

- 14 but expand ing to include TO

- 15)

- Carbonyls, except acrolein (TO

- 11)

- Acrolein for NATTS (modified TO

- 15)

- OC/EC

BAAQMD collects but does not analyze samples for speciated PM 2.5 , PM 10 metals, PAHs, and Chrome(VI). There are a number of additional activities the labora tory undertakes to support the collection and analysis of air pollutants. These include canister cleaning and preparation, data validation, and sample storage.

The BAAQMD laboratory staff is one of the most important assets of the laboratory. The

anal ysts/chemists were knowledgeable, talented, and clearly dedicated. It was generally found that BAAQMD laboratory is in compliance with respect to performing analyses according to

EPA methods but lacks a dedicated laboratory Quality Assurance Officer to ov ersee the review and documentation of quality assurance results and implementation of corrective actions on a consistent basis. While the District has assigned a QA Officer for the laboratory , the laboratory ’s current staffing situation (i.e., short

- staff ed with an unfilled vacancy) prevents the appropriate level of effort required to fulfill this role. Internal quality assurance audits are not15

performed and many critical Standard Operating Procedures (SOPs) do not reflect current laboratory practice. Th e laboratory has established an effective system of tracking and documenting instrumental quality control (QC) results using control charting in the OC/EC analysis area and it would be of considerable benefit to expand implementation of a similar system th roughout all departments of the laboratory as originally planned. The laboratory would benefit from assigning dedicated logbooks for instrument maintenance, standards preparation, instrument injection run logs and MDL studies in each area in order to simp lify review. The laboratory has purchased a new Gas Chromatograph/Mass Spectrometer which it plans to dedicate for volatile organic carbon (VOC) analyses according to EPA method TO

- 15. Additional facility space and possibly a separate air handling system may be needed to accommodate the new system.

TOXICS LABORATORY FINDINGS

FINDING D

- T1 (Major) :

BAAQMD lacks a QA Officer dedicated to overseeing quality management of the laboratory

.

Discussion:

The laboratory has identified a chemist to serve as the QA Officer for the laboratory and the Technical Services Division has assigned a Quality Assurance Officer for ambient air monitoring; however, neither of these positions has been fully developed to address the quality management needs of the laboratory

. For example, SOPs are not up

- to

- date ( Finding D

- T2 ), and logbook and data reviews are not done on a consistent basis ( Findings D

- T6 and D

- T7 ). The laboratory currently has a vacant position and staffing

issues w ere cited as the primary reason that the la boratory QA role has not been fully developed.

Recommendation: Fully develop and allocate adequate time for a laboratory QA Officer role in the laboratory

.

FINDING D

- T2 :

Many of the SOPs do not reflect actual laboratory practices.

Discussion:

SOPs docu ment the formal laboratory procedures that are adhered to in generating sample results by all analysts and by doing so represent a critical component of data defensibility. There were numerous instances where SOPs do not reflect current laboratory practic es, including:

 SOPs describe in

- house preparation of the TO

- 14 standards but BAAQMD has moved to purchasing a gas cylinder standard from a vendor.

 Sample matrix spikes for carbonyls are being performed at a desirable frequency but the frequency is incons istent with the laboratory's written SOP.16  A secondary control standard is not currently being analyzed for TO

- 14 by the laboratory as stated in the BAAQMD SOP.

 BAAQMD SOP specifies that 1 out of every 8 canisters in each batch should undergo certification but in reality only 1 in 16

- 20 is certified.

 Laboratory staff indicated that the OC/EC analyses have changed but these changes are not reflected in the SOP.

Recommendation: It is recommended that the laboratory follow through with current plans to update all laboratory SOPs so they reflect actual laboratory practices.

FINDING D

- T3 :

The TO

- 14 standard certification and stability are uncertain

.

Discussion: Calibration directly affects sample quantitation and certification of standards is a critical QA ste p. Laboratory staff stated that calibration gas standards are purchased from one of two commercial vendors but the concentration certification is not purchased because the cost is prohibitive. In place of the vendor certification

, the laboratory relies o n analysis of a sample from the new gas cylinder by CARB and by Air Toxics Laboratory. CARB purchases a fully certified gas standard directly from NIST and so the CARB analysis is regarded as a definitive certification analysis. Current District practice differs from

vendor certi fication in some important ways, for example, cylinders are analyzed multiple times by the vendor but only once by CARB and the transport of the cylinders between BAAQMD to CARB could introduce problems if proper container and/or system surface conditioning is not utilized. This procedure may be adequate but the robustness of the procedure needs to be demonstrated.

In addition, there is no expiration date assigned to the gas standard. The stability of the

standard has not been formally studied and the laboratory is unaware of any outside studies that establish an expiration date. Therefore, periodic checks at suitable intervals are needed to ensure components of the standard have not cross reacted or degraded. It is

necessary to document the ongoing integrity of standards as this forms the basis for sample contaminant quantitation in order for data results to be considered defensible.

Recommendation: Review and justify the reliability of standard certification procedure. Inc luded in the review, EPA suggests that the laboratory determine: the impact of transporting cylinders, and if new cylinders can feasibly be directly analyzed by CARB; how many times the cylinder or sample should be analyzed by CARB; if the CARB values are to be used as the certified values or if the values provided by the vendor are to be used; and, acceptance limit for allowable percent difference between the CARB results and the vendor supplied results. S tudies should be performed to assess the stabilit y of the standards over time and to establish an appropriate expiration/recertification schedule.17

FINDING D

- T4 :

Canister certification for cleanliness is not performed according to the "Technical Assistance Document for Sampling and Analysis of Ozone Pr ecursors" and the BAAQMD SOP.

Discussion: Laboratory staff were unclear as to the number of canisters in each batch that are selected for certification of cleanliness. Laboratory staff estimated that one ou t of 20 or possibly

one out of 16 canisters in every batch are certified. This practice is outside the

recommendation in the “ Technical Assistance Document for Sampling an d Analysis of

Ozone Precursors” (Ozone Precurs or TAD) and the BAAQMD SOP specification that

one out of every eight canisters in ea ch batch should undergo certification. A greater level of assurance that contamination reported in samples did not originate from sources extraneous to the sample or from a previous sample in the canister would be provided by adhering to the procedures sp ecified in the BAAQMD SOP and TAD of certifying one out of eight canisters in every cleaned batch. This would strengthen the defensibility of

data.

Canisters are selected at random for certification following cleaning. Laboratory staff stated they do no t take into consideration which canisters had the highest concentrations of contaminants when selecting the canister in each batch for cleanliness certification. Staff further clarified the cleaning process has been very effective in their experience and therefore they believed a random selection of canisters for certification has been sufficient. The current random selection practice is outside the recommendation in the Ozone Precursor TAD, page 105 and may not be adequate with the pla nned transition to

the TO

- 15 GC

- MS method which has lower detection limits.

Recommendation: It is recommended that the District review its canister cleanliness certification process and ensure compliance with the BAAQMD SOP on all issues including the number of

canisters se lected for certification. In addition, BAAQMD should consider adopting

recommendations outlined in the Ozone Precursor TAD

, such as tracking the contamination history and selecting canisters that are the most highly contaminated for

certification.

FINDIN G D

- T5 :

The laboratory has not established a retention time for certified canisters after which they must

be recertified.

Discussion:

Canisters can be subject to contamination following certification. The District monitors the p ressure gauge after can ister shipment to the field follo wing cleaning and

certification. While this is a good quality assurance measure for ensuring significant

vacuum loss has not occurred

, it has been the experience of the EPA Region 9 Laboratory18

that canisters may become con taminated over time through small leaks or micro leaks that may not be obvious from monitoring canister gauge readings. Laboratory staff agreed that canisters have on occasion exhibited leaks. Although it was stated that canisters are generally used with in two or three weeks of certification

, a procedure is not in place to ensure they are not used beyond this time

, which could result in contamination. Establishing a retention time policy for canisters stored at the laboratory after they have been cleaned and certified would add additional assurance that canisters have not become contaminated over time through smaller leaks.

Recommendation: It is recommended that the laboratory establish a retention time policy for clean canisters

after which date they will be re

- cleaned and certified as an added quality assurance measure they have not become contaminated. A retention time of 30 days is used in

some other laboratories and would be reasonable.

FINDING D

- T6 :

Documentation and record

- keeping in the lab oratory does not fully capture laboratory activities, was found to have errors, and is not organized for easy tracking and retrieval of information

.

Discussion: Accurate record documentation serves an important QA function in the defensibility of

data and becomes critically important if data are ever questioned. Specific issues

identified include:

 Dedicated, separate logbooks are currently not maintained for instrument maintenance, instrument analyses and minimum detection limit studies. All

information is currently recorded in the same laboratory instrument logbook. Using separate logbooks dedicated for instrument analyses, instrument maintenance activities, standards preparation, and MDL studies would facilitate the tracking and retrieval of informatio n.

 Computer generated instrument batch sequence lists or raw data printouts constitute the current method of documenting analytical analyses. Supplementing these records with handwritten, bound instrument logbooks would allow a permanent record of notes concerning everything that happened during an analytical run

, including the analysis of highly contaminated samples with potential carryover would allow the entire instrument analysis run to be reconstructed when necessary.

 Errors were identified for d ates entered in the standard preparation logbook. For example, numerous entries spanning a year's duration appeared to fall under a single August, 2008 date caption

, which the analyst agreed was in

error.

 A system of tracking and documenting the receipt a nd monitoring of expiration dates of standards is lacking. New gas standard cylinders are not logged in upon arrival. Logbooks providing the time of receipt of new standards and certifications or ongoing verification analysis by outside19 laboratories shou ld document the integrity of standards used for sample quantitation.

 Laboratory staff stated that standards used for the purpose of establishing contaminant peak retention times are analyzed but not documented in the computer sequence lists of analyses tha t serve as the laboratory's instrument run logbook. There is no documentation of the date and time that peak

retention times were updated. Since the identification of sample contaminants is based on the retention time comparison of sample peaks to known standard contaminant peaks, the analysis of standards used to determine, update or re

verify retention times should be documented.

Recommendation: Improve documentation of some quality assurance and routine laboratory activities.

FINDING D

- T7 :

Seco ndary review of records was found t o be irregular. Approximately six month intervals between secondary reviews were observed in some instances.

Discussion: Regular review of instrument logbooks by a supervisor or QA Officer helps to ensure and

document t hat proper analysis protocol is being followed, e.g. calibrations, blanks analyses etc. Repeated failures or attempts to pass calibrations noted in logbooks can be

an indication that instrument maintenance or other corrective actions need to be performed.

Recommendation: It is recommended that a policy of regular secondary review and signature of logbooks or other records of analyses be adopted. A weekly or greater frequency would be desirable.

FINDING D

- T8 :

Laboratory space is constrained and may be i nadequate once the additional capabilities such as

GC

- MS that are on

- line

.

Discussion: Currently, the laboratory facilities are crowded and barely adequate for the current analyses. For example, liquid nitrogen D ewars have to be stored in the hallway, wh ich partially blocks egress, because there is no room in the laboratory . Moreover, there is no dedicated air handling system for the VOC analyses which may result in high background contamination, potentially interfering with the performance of the new hi gh

- sensitivity

GC

- MS.

Recommendation:

Assess and address the laboratory facilities needs.20

FINDING D

- T9 :

The laboratory is not tracking the retention time of sample cartridges to ensure they are used

within six months of the manufacturer's certification analysis as recommended by the

manufacturer.

Discussion: The laboratory is not tracking the retention time of sample cartridges to ensure they are used within six months of the manufacturer's certification analysis as recommended by the manufacturer. Rep resentative cartridges which have exceeded the six month expiration date are not being analyzed for contamination as specified in the BAAQMD SOP, page 8. Furthermore, the laboratory has changed its practice of shipping the exact number of cartridges neede d to the field for sampling events by sending a full box of cartridges which can result in some cartridges not being used and increasing the potential for cartridges to exceed the six month expiration time. Certification that cartridges are

free of contam ination prior to sampling is an important quality assurance measure to document that contamination reported in samples did not originate from an extraneous

source.

Recommendation: It is recommended that the laboratory formalize a procedure for tracking th e expiration date of cartridges to ensure the date has not been exceeded.

PARTICULATE MATTER LABORATORY FINDINGS

FINDING D

- PM1 :

BAAQMD does not perform periodic field blanks for PM 10

.

Discussion:

The use of b lank controls determine the potential for c ontamination of filter and help

evaluate the consistency of the balance readings over the filter handling period. Additionally, field (i.e., filter) blanks are required by regulation as a mechanism for evaluating filter media, see 40 CFR Part 50, Appendix J, Section 7.2.3, and EPA Compendium Method IO

- 3.1 notes in Section 5.4, “Provide one blank sample with every 10 actual samples.” BAAQMD performs laboratory blanks for PM 10 on a routine basis. By adding field blanks BAAQMD would provide for control of fi eld as well as laboratory handling.

Recommendation: It is recommended that BAAQMD collect periodic PM 10 field blanks.

FINDING D

- PM 2 :

The records of the Polonium strips were not retained and they could not be verified.21

Discussion: Polonium strips have a set shelf life and should be tracked as a laboratory standard.

Recommendation: Retain documentation of the polonium strips purchase, use, and expiration date.

FINDING D

- PM 3 :

The PM 2.5 filter associated with the data point tracked for January 4, 2008 a t Livermore was invalidated because it was outside the holding time.

Discussion: The data was flagged in the database as a laboratory error. However, it appears that the filter was not delivered to the laboratory until over three weeks after the sampling day.

This resulted in the laboratory to have less than 1 week to perform the filter weighing.

Recommendation: BAAQMD should ensure that filters are transported to the laboratory on a timely basis to avoid holding time violations.22

E. DATA MANAGEMENT

D ata management generally involves both a data validation process and a data management system. The District is currently developing a new data management system (DMS) which should be operational within the next few months. For this reason we have

deferre d review of the data management system an all related components (e.g., software documentation, data processing, etc.) until our next TSA in 2012. This data management section, therefore, focuses on data validation.

Data validation is a systematic proces s for reviewing a body of data against a pre

established set of quality control acceptance criteria. When data do not meet the acceptance

criteria

, they are flagged with a qualifier identifying the associated problem and possibly

invalidated. Data valida tion should occur at the earliest opportunity for optimizing cost effectiveness. This allows for corrective actions to take place early in data collection and analytical process es . One of the primary objectives of data validation is to determine if data are adequate to meet the intended purposes, such as Air Quality System (AQS) reporting.

The MET/PE, Laboratory Services, Air Monitoring

, and Planning program staff were interviewed regarding their involvement in data validation for criteria pollutants, PM 2.5 , PM 10 , toxics and meteorology results. Interviewees , with their respective areas of review responsibility in parenthesis , were :

Air Monitoring Nancy Balberan ( i nitial review of f ield o perator s ubmitted d ata for

g aseous and PM)

Laboratory Se rvices Mary Beacon (PM a nalysis and review)

MET/PE Mike Basso ( c ontinuous g aseous, t oxics) Ken Chrysler (PM 10 ) Dick Duker (PM 2.5 )

Planning Office Jim Cordova ( m eteorology) Jeff Matsuoka ( m eteorology and c omputer s upport)

Each office has specific vali dation responsibilities as outlined in the BAAQMD SOP 101, and 601 through 609.

Data points for BAAQMD's ambient air monitoring program are generated either in the field for continuous monitoring data (e.g., gaseous criteria pollutants and BAM PM 2.5 ) or in the laboratory for time

- integrated samples collected in the field (e.g., PM filters, VOC canisters, carbonyl cartridges, etc.). Continuous data are logged by computer with strip chart backup and23

are telemetered to the District office from each site on ce per day with the exception of ozone and

continuous PM 2.5 which are telemetered hourly for AQI reporting. At the field station, operators review and evaluate strip chart recordings and compare the information to the data loggers ; for example, they check the zero and span results to ensure the instruments have operated within established limits. The Senior Supervising Technician reviews the data daily. The precision and accuracy data is reviewed by Nancy Balberan with the final review and entry into AQS by the Monitoring and Meteorology Manager. Time

- integrated samples are collected by field station operators and either delivered to the District laboratory or sent to a contract laboratory for analyses.

Once data arrive at the District office, a workin g copy of all collected data is produced from the master copy. Validation is performed on the working copy while the master is maintained separately in the BAAQMD database in case the working copy becomes corrupted or

a need arises to review the master co py. No one has “write” access to the master.

The BAAQMD validation process excels in that they have assigned very skilled individuals to address specific areas of validation as identified. In summary, all gaseous and PM data is initially reviewed by t he Air Monitoring office; PM data is also reviewed by the Laboratory Services office. Subsequent review of both gaseous and PM data is performed by the MET/PE office and is then uploaded into AQS. Initial validation for toxics data is performed by

the La boratory Services Office and then forwarded to the MET/PE office for further validation. The MET/PE staff occasionally consult the Laboratory Services Office for clarification when performing validation. Meteorology data is initially reviewed by the MET/ PE office and then transferred to the Planning office for further validation to maintain objectivity in review since the MET/PE Office has responsibility for collecting the meteorology data. Any data points voided through any of the validations identified above is reviewed by the manager of the MET/PE office prior to upload into AQS reporting.

Taking these individual efforts in aggregate, the validation program is one of great strength, as each validator performs a focused review of their area of respons ibility. The challenge with this level of expertise is that it cannot easily be replicated by other staff in the

event that a validator is absent and the process may not be fully transparent to others.

FINDING E1 :

Finding: SOPs 601

- 609 for data mana gement are general and do not provide step

- by

- step
instruction of how data are actually validated by the data reviewers (validators) in Ambient Monitoring, Laboratory Services, MET/PE, and Planning Sections.

Discussion: Although SOPs for data management a re contained in the Quality Assurance Project Plan (QAPP), they are descriptions rather than true SOPs that inform one how to perform a particular function in a consistent, reproducible manner. More detailed SOPs or checklists have been developed by some, but not all, reviewers to help themselves perform validation and to ensure they do not miss any important steps.24

It is important to prepare SOPs that cover all the activities expected to be performed by

the validator : 1) to enable tracking of what act ually is being checked and the criteria used to determine acceptability of data; and 2) to provide guidance to new, back

- up or cross

trained personnel unfamiliar with validation or a particular step in validation. The SOPs

should cover validation activiti es taking place by Laboratory Services, Ambient Monitoring, and MET/PE staff.

Recommendation: Develop and update SOPs to explicitly identify the specific activities that should be performed following EPA’s Guidance for Preparing Standard Operating Procedu res ( QA/G

- 6 ) EPA600/B

- 07/001/April 2007.

The following are suggested to be covered in BAAQMD SOPs for all phases of

validation:

 identification of the location and specific databases accessed

 the checks performed

 criteria used to determine data acce ptability

 points of contact when questions arise

 documentation of the communications

 whether validated data is approved by another party or the individual performing

validation

 where validated data is saved

 how subsequent individuals are notified that review is completed by the p revious

validator

 identification of all voided data with explanation

 identification of the individuals receiving final evaluation

 how the organization knows validatio n is complete

The SOPs should be comple te enough such th at additional SOPs do not need to be developed separately by the individuals validating data.

Cross

- training for validators is also suggested for consistency in reviews and to act as

back

- ups, when necessary.

FINDING E2 :

Questions raised by data v alidators in the MET/PE to the Ambient Monitoring or Laboratory Services are not uniformly documented. Some validators are not documenting the discussions of data quality or the reason for changing data points.

Discussion: In the majority of cases, valid ators are going beyond the scope of the data management

SO Ps in a commendable manner by documenting their discussions with Ambient Monitoring or Laboratory Services staff whenever they need clarification for determining data quality and whether data will b e kept or voided. Communications were saved in25

various forms, e.g., on individual or network computer drive s , individual forms or meeting logbooks. These were accessible only by the individual documenting the

communication and n ot attached to the data th at had been validated. All communications and decisions made regarding data points of concern should identify the:

 the party raising the question*

 the date the question was raised

 the parties who were contacted to reach resolution*

 the party approving t he resolution*

 the date of resolution

* there should be a signature associated with each step identified with asterisk

Where resolution impacts field operations, they should be communicated to field station operators and their supervisors.

Recommend ation:

S tandardize the documentation process for validation communications and ensure they are taken to completion , such as is done for ODAMNs . Include all relevant personnel on communications and final decision on questioned data, including field station operators

, so that field staff are aware of particular or recurring conditions that impact data quality (e.g., partisols).

RECOMMENDATIONS FOR NEW DMS Although we did not specifically address the data management system during this phase of the TSA, we did identify some recommendations for the new DMS. These include:

 Incorporate a tool to allow field station operators and validators to verify the accuracy of the daily printouts. These could be control charts or some other tool.

 Provide validators with access to instrument performance data to allow a secondary check on instrument performance.

 Initiate use of electronic chain of custody for data validation with “write” capability lock out activated after each validator completes his/her review. Once lock out is activated, it is recommended that a program be written that initiates a chain of custody and sends notice to subsequent reviewers that the data is ready for review.

 Use one system for m et eorological data validation. For example, perform meteorol ogical data validation on FOXPRO. Export validated data to DMS and have a program or additional staff verify that the DMS contents mirror that of FOXPRO prior to complete

conversion to DMS.

 F lag data reported below MDL (e.g., ozone)

.

 Data should be easily retrievable both in the field and in the office

.

 Time of filter weighing should be documented and visible in the electronic database.

 A system should be put in place to alert the PM lab analyst if the temperature goes out of range for short time periods ( i.e., five minutes or less).26

F. QUALITY CONTROL (QC) AND QUALITY ASSURANCE (QA)

Quality management is the system put in place that oversees quality assurance, quality control, and quality improvement activities

. EPA requires that ambient air monitori ng agencies have a quality management system that conforms to 40 CF R Part 50 Appendix A and the EPA quality policy ( EPA Order CIO 2105.0). Additionally, USEPA grant regulations specifically require each grantee to provide for QA activities (40 CFR Part 31 .45).

Specifically, 40 CF R Part 50 Appendix A Section 2 requires that eac h ambient air monitoring PQAO conforms to certain quality management practices. These include:

 Having a documen ted quality system that meets EPA requirement s for QMPs and QAPPs.

 Ha ving a quality management function that is independent of air monitoring operations.

 Developing or a dopting DQOs, or equivalent systematic planning procedures, for all monitoring programs.

 Participating in National Performance Evaluation Programs, which co nsist of performance audits used to independent ly determine program adequacy, n ational monitoring network performance, and n ational consistency.

 Submitting to Technical Systems Au dits by EPA Regional Offices at a frequency of every three years or less.

 Usi ng certified reference materials to standardize monitoring equipment. EPA views the application of these quality management system components as integral to monitoring programs. Insufficient quality management and control has been cited as rationale to

su p port legal challenges to NAAQS designation decisions.

The BAAQMD quality management system meets the basic EPA requirements. BAAQMD has a QMP and QAPP that are approved by EPA Region 9. BAAQMD has implemented mechanisms to independently manage its qu ality system by assigning these duties

to staff in the MET/PE section that are independent from data collection activities (with the exception of meteorological instrument calibrations). BAAQMD conforms to the data quality objectives and/or the sys tematic planning developed by EPA for cri teria pollutants. EPA and

C ARB perform national performance and technical evaluations of the monitoring network.

Quality control and quality assurance are the two components of quality management within a monitoring pr ogram that support the assumption that the data they collect represent the true air quality of the area. They are the means by which an organization manages its quality aspects in a systematic, organized manner and provides a framework for planning, imple menting, and assessing work performed by an organization. A properly developed QA/QC program encompasses a variety of technical and administrative elements, including policies and objectives, organizational authority, responsibilities, accountability, and procedures and practices.

Quality assurance is a management or oversight function; it deals with setting policy and running an administrative system of management controls that cover planning, implementation, the review of data collection activities, and the use of data in decision making. Quality control is27

a technical function that includes all the scientific precautions, such as calibrations and duplications that are needed to acquire data of known and adequate quality.

The organization of BAAQMD’s Technical Services Division divides QA responsibility among various sections and various personnel: Laboratory QA Officer, QA Officer within the Meteorology/PE Section, and PE Supervisor within the M ET /PE Section. All QA functions are

embedded within the sections rather than having a formal QA role that oversees all ambient air monitoring operations at the management level. Most quality assurance functions, including the QA Officer for ambient air monitoring and the PE group, reside in the MET/PE S ection

, managed by Dick Duker. The QA Officer for ambient air monitoring, Mark Stoelting, also has responsibilities as an air and meteorology specialist and does not have authority to direct activities in the Laboratory Operations or Air Monitoring sections. T he BAAQMD MET/PE

Section has a PE Group with three performance evaluation auditors that perform audits of the particulate and gaseous monitoring networks. The PE Group has also explored performance audits of BAAQMD Laboratory operations. The Meteorologi cal and Performance Evaluation Section also performs audits of the meteorological monitoring stations periodically with an instrument specialist and an independent contractor. Audits of gaseous pollutants are generally conducted at a frequency that exceed s the minimum specified by regulation.

GENERAL QA/QC FINDINGS

FINDING F

- QA1 (Major) :

The BAAQMD QAPP’s approval is conditioned on establishing DQOs for data uses that are not defined by USEPA.

Discussion: The USEPA has developed DQOs, or equivalent q uality planning, for data uses that support NAAQS decisions and several other data use s (including NATTS a nd the STN ). It was identified in the review of the QAPP that BAAQMD collects data that supports

additional uses . In order to meet EPA’s requirement s for systematic planning, BAAQMD must determine the data quality required for each data use and ensure that the data collected is of sufficient quality to support the manner in which it is being used.

Recommendation: Work with data users inside BAAQMD t o determine the quality needs to support their data uses and/or decisions. Revise the QAPP to reflect these uses and modify data

collection activities as needed.

FINDING F

- QA2 (Major) :

Most quality management functions are being performed ; however

, QA authority is not clearly

defined.

Discussion:28

All ambient air monitoring agencies must demonstrate Independence of Quality Assurance Activities as outlined in 40 CFR Part 58, Appendix A, Section 2.2. The

MET/PE Section manages the QMP, QAPP, SOP s, and data review. However, there is the perception that quality management responsibility and authority for other aspects of the air monitoring program are not the responsibility of the MET/PE Section.

The MET/PE Office manager’s position is not iden tified as a QA Manager in the QAPP organization chart. Although it is implied that he is a QA Manager because he oversees the QA Officer’s position, his management authority is limited to the MET/PE Office. 40 CFR Part 58, Appendix A, Section 2.2 states “the quality assurance management function must have sufficient technical expertise and management authority to conduct independent oversight and assure the implementation of the organization’s quality system relative to the ambient air quality monitoring program and should be organizationally independent of environmental generation activities.” This section also states “the monitoring organization’s quality system must have adequate resources both in personnel and funding to plan, implement, assess and re port on the achievement of the requirements of this appendix and its approved QAPP.”

A QA Officer’s position is identified in the BAAQMD’s organization chart, however, QA functions are carried out by several individuals e.g., QA Officer, PE group, and

Lab oratory Services. While the organizational structure may remain the same, a QA manager needs to be designated to provide leadership to the previously mentioned positions and offices. The designated individual should have overall QA authority to

conduct i ndependent oversight and require corrective action. In order to enable the QA manager to function effectively, a QA manager should oversee rather than perform staff level duties (e.g., PM 2.5 validation).

In addition, the QA Manager should be given the a uthority to perform:

 Periodic audits of the quality management system (management system reviews), routine procedures, and data quality to identify areas of improvement and to ensure that monitoring programs continue to consistently follow sound and

docume nted procedures,

 A quality assurance training program and oversight of training for air monitorin g and laboratory staff training, and

 Audits of data quality, including data tracking, especially following deployment

of the new DMS.

In general, EPA QA/R

- 5

, EPA Requirement for Quality Assurance Project Plans

, discusses assessments that should be elements of a quality management program. These include, “surveillance, management systems reviews, readiness reviews, technical systems audits, performance evaluat ions, audits of data quality, and data quality

assessments.”

Recommendation:

The MET/PE Section should be identified as the lead group for ambient air monitoring QA and all monitoring activities should include independent oversight either directly by29

MET /PE Section or with their concurrence. Further, it is recommended that BAAQMD

i dentify a QA Management position in the QAPP and provide this position leadership authority and resources to conduct:

 independent reviews of the District’s overall Quality Syst em i.e., Laboratory Service, Ambient and Meteorological Monitoring (including Field Operations), and Data Validation;

 overall QA oversight for the District including corrective actions whether uncovered during internal audits or otherwise

, e.g., through s taff innovation that improves upon data defensibility.

FINDING F

- QA3 (Major) :

BAAQMD does not have a formal corrective action system for corrective actions outside of the performance audit program.

Discussion: EPA quality management standards (EPA Q A/R

- 2, Quality Improvement Section) require that management and staff “ensure that conditions adverse to quality are” prevented, identified promptly, fully defined, corrected, prevented from recurring, and

documented as corrective actions which are tracked to closure. There is a corrective action processes for the performance audit program. There should also be a formal, documented mechanism for elevating potentially significant corrective actions originating from the laboratory staff or field operators.

Recommendation: The BAAQMD air monitoring program should include a formal corrective action process for actions originating from the laboratory or field, in addition to the performance audit

process

.

FINDING F

- QA4 :

BAAQMD should have a documented system to ensure that staff are trained on appropriate procedures and are familiar with the general QA system as defined in the QMP and QAPP(s).

Discussion: BAAQMD staff appear to be trained and proficient with the procedures they use to

conduct their m onitoring activities. However, it was noted that this training and/or demonstration of proficiency was not adequately documented for either laboratory or

field staff

.

Copies of the QMP and QAPP are available on the BAAQMD network to all staff.
However, there are no trainings or other mechanisms to ensure that staff understand the QA system as outlined in these documents.

Recommendation:30

BAAQMD should develop a formal system to ensure and document that all staff are familiar with the quality management system and are trained and proficient at the monitoring tasks that they are performing.

FINDING F

- QA5 :

BAAQMD does not have a standard procedure to ensure that the performance contractors used to conduct or support monitoring are evaluated by BAAQMD Q A staff.

Discussion: In order to ensure that quality objectives are met, quality requirements for monitoring data must also apply to contractors involved in the data collection process. This is achieved by including quality needs in contracts and by ev aluating the quality of the contractor’s work product.

BAAQMD uses a minimal amount of contract support for laboratory analysis and quality assurance tasks (e.g ., meteorological audits). Normally BAAQMD does provide quality assurance oversight to these c ontractors. However, there is no procedure in place to assure that contract oversight is provided. Additionally, USEPA or other agencies may be already doing parallel oversight of these contractors and may be able to work jointly with BAAQMD (i.e., condu ct audits and share contractor evaluations).

Recommendation: BAAQMD should establish a procedure to ensure that the performance contractors used to conduct or support monitoring are evaluated by BAAQMD QA staff.

PERFORMANCE EVALUATION FINDINGS

FINDI NG F

- PE 1 : Audits are performed for NO 2 by using ozone as the reference concentration instead of NO. Additionally, NO provided is in excess of recommendations.

Discussion: No specific NO 2 titration procedure is specified in 40 CRF Part 58, Appendix A, S ection 3.2. However USEPA specifies that NO 2 should be calibrated using the NO original and NO residual concentrations in 40 CFR Part 50, Appendix F, Section 1.5.9. While this procedure is only required for calibrations, alternative audit procedures shou ld be validated. Additionally, sub

- section 3.2.2.2 states that “NO concentrations substantially higher than 0.08 ppm, as may occur when using some gas phase titration (GPT) techniques, may lead to evaluation errors in chemiluminescence analyzers due to

in evitable minor NO

– NO x channel imbalance.”

Recommendation: BAAQMD should evaluate the NO 2 audit procedure to ensure and document that the procedure being used is equivalent or better than the procedure recommended by EPA.31

FINDING F

- PE2 :

It was noted during data tracking that the manufacturer’s certification for Delta Cal 249 had the wrong ID.

Discussion: Traceability of standard certification is a key component of the air monitoring program.

A review of the certification documentation indicated th at the manufacturer had inadvertently provided the wrong instrument identification in 2008 for Delta Cal 249.

The PE Group confirmed that this had been noted when the instrument returned from certification. However, this error was not corrected on the ce rtification documentation and the manufacturer was not requested to provide a “clean” certification document.

Recommendation: The certification documentation should be annotated to explain and correct this error. In

the future errors in standard certifi cation should be corrected by the certifying entity.

FINDING F

- PE3 : The 2008 calibration for the DPI 70524767 pressure monitor was not in the instrument logbook

.

Discussion: During the data tracking process USEPA noted that the logbook for the DPI 7 0524767 pressure standard did not include the calibration performed in 2008. It was indicated that this information was available on the auditors’ computer.

Recommendation: Instrument calibration information should be entered into the logbook when the

c alibration is performed, or when calibration information is received to avoid inadvertent

omissions.32

APPENDIX A: SUMMARY OF FINDINGS

MAJOR FINDINGS Finding A1: BAAQMD has had significant quality issues with the Thermo Scientific Partisol FRM, Model 2000.

Finding D

- T1: BAAQMD lacks a QA Officer dedicated to overseeing quality management of the laboratory.

Finding F

- QA1: The BAAQMD QAPP’s approval is conditioned on establishing DQOs for data uses that are not defined by USEPA.

Finding F

- QA2: Mo st quality management functions are being performed; however, QA authority is not clearly defined.

Finding F

- QA3: BAAQMD does not have a formal corrective action system for corrective actions outside of the performance audit program.

MINOR FINDINGS

Fin ding A2: BAAQMD keeps multiple hard copies and electronic copies of many documents, making it unclear which is the official copy.

Finding C1: PM10 flow checks may not be occurring at the required frequency at all sites.

Finding C2: BAAQMD operates ma ny of their gaseous monitors using sample pumps that

deviate from the reference method.

Finding C3: Several different SOPs appear to apply to instrument calibrations that are not consistent. Additionally, documentation of calibration was hard to follow and incomplete.

Finding C4: The level of information recorded into the station logbooks is not consistent and at some sites the logbooks are not frequently used.

Finding C5: Toxics samplers are not routinely verified by checking with blanks and/or

stan dards for contamination or losses.

Finding C6: No instrument SOPs were available at some sites with limited monitors (e.g., Fairfield and Gilroy).

Finding D

- T2: Many of the SOPs do not reflect actual laboratory practices.

Finding D

- T3: The TO

- 14 stan dard certification and stability are uncertain.33

Finding D

- T4: Canister certification for cleanliness is not performed according to the "Technical Assistance Document for Sampling and Analysis of Ozone Precursors" and the BAAQMD SOP.

Finding D

- T5: The laboratory has not established a retention time for canisters after they have been certified beyond which they must be recertified.

Finding D

- T6: Documentation and record

- keeping in the laboratory does not fully capture laboratory activities, was foun d to have errors, and is not organized for easy tracking and

retrieval of information.

Finding D

- T7: Secondary review of records was found to be irregular. Approximately six month intervals between secondary reviews were observed in some instances.

Fin ding D

- T8: Laboratory space is constrained and may be inadequate once the additional capabilities such as GC

- MS that are on

- line.

Finding D

- T9: The laboratory is not tracking the retention time of sample cartridges to ensure they are used within six mon ths of the manufacturer's certification analysis as recommended by

the manufacturer.

Finding D

- PM1: BAAQMD does not perform periodic field blanks for PM 10

.

Finding D

- PM 2 : The records of the Polonium strips were not retained and they could not be

verifi ed.

Finding D

- PM 3 : The PM 2.5 filter associated with the data point tracked for January 4, 2008 at

Livermore was invalidated because it was outside the holding time.

Finding E1: Finding: SOPs 601

- 609 for data management are general and do not provide s tep

by

- step instruction of how data are actually validated by the data reviewers (validators) in Ambient Monitoring, Laboratory Services, MET/PE, and Planning Sections.

Finding E2: Questions raised by data validators in the MET/PE to the Ambient Monitori ng or Laboratory Services are not uniformly documented. Some validators are not documenting the

discussions made for determining data quality or for changing data points.

Finding F

- QA4: BAAQMD should have a documented system to ensure that staff are tra ined on appropriate procedures and are familiar with the general QA system as defined in the QMP and QAPP(s).

Finding F

- QA5: BAAQMD does not have a standard procedure to ensure that the performance contractors used to conduct or support monitoring are ev aluated by BAAQMD QA staff.

Finding F

- PE1: Audits are performed for NO 2 by using ozone as the reference concentration instead of NO. Additionally, NO provided is in excess of recommendations.34

Finding F

- PE2: It was noted during data tracking that the m anufacturer’s certification for Delta Cal 249 had the wrong ID.

Finding F

- PE3: The 2008 calibration for the DPI 70524767 pressure monitor was not in the instrument logbook.35

APPENDIX B: GOOD PRACTICE RECOMMENDATIONS

NETWORK RECOMMENDATION 1 :

The BAAQMD has not developed a schedule for implementation of the Lead monitoring

program.

Discussion: USEPA establish new monitoring requirements for Lead beginning in January

, 2010 and January

, 2011. Sites within BAAQMD’s jurisdiction will need to be operational by January, 2011. In order to meet this schedule

, agencies will need to establish programs including establishing approved QAPPs and FEM for laboratory analysis prior to the commencement of monitoring.

Recommendation: BAAQMD should develop a schedule that ensures that pre

- monitoring testing and the appropriate supporting documentation and approvals are in place prior to the commencement of monitoring.

FIELD RECOMMENDATION 2 :

The trees adjacent to the Napa site (behind the building to the ea st) are starting to encroach on the probe and will need to be trimmed soon.

Discussion: USEPA siting criteria require that trees be at least 10 meters from instrument inlets and at

least 20 meters when the trees act as an obstruction. The trees at the Napa site are currently just below the height of the probe at a distance of approximately five meters.

Recommendation: BAAQMD should continue to ensure that the trees at the Napa site are routinely trimmed.

FIELD RECOMMENDATION 3 :

The San Jo se Jackson Street site requires an electrical upgrade.

Discussion: The San Jose Jackson site is nearing the limit of the electrical capacity for that site. The current system may not accommodate the addition of all NCore instruments.

Recommendation: Upg rade the San Jose Jackson Street electrical system to accommodate additional NCore requirements.

FIELD RECOMMENDATION 4 :36

There is no system for tracking and controlling station and instrument logbooks.

Discussion: Field procedures require that logbooks be kept. However, these logbooks were not tracked, identified, and archived in a manner to ensure that the critical documentation they contain will be accessible and defensible.

Recommendation: Develop a standardized procedure for creating, labeling, and archiving logbooks.

FIELD RECOMMENDATION 5 :

The Ambient Monitoring SOP 101, Station Operator, Section IV, Data Retrieval and Review, Section 5.e instructions does not contain example calculations.

Discussion: The captioned SOP states that the percenta ge chart values should be converted to concentrations (PPM) using the most recent calibration curve. No information is provided as to how to perform this conversion

.

Recommendation: Provide one or two example calculations to ensure calculatio ns are properly performed.

FIELD RECOMMENDATION 6 :

Not all m eteorological stations have sufficient power to store and transmit collected data.

Discussion: Meteorological stations need sufficient power to be provided so that data capture can

m eet 75% com pleteness criteria by January

, 2011 at the NCore site. This would be a good practice at all the meteorological sites.

Recommendation:

Ensure sufficient power to Meteorological stations to ensure data capture of 75% completeness

.

FIELD RECOMMENDATION 8 :

Strip charts and daily printouts were not present at the Fairfield site.

Discussion: Charts and printouts for Fairfield are currently maintained at the Concord monitoring station. Charts should be maintained at the site, similar to instruments logs (i.e

., instrument logs follow the instrument), to compare current results from historical

responses.37

Recommendation: Maintain charts and daily printouts at each site.

LAB RECOMMENDATION 1:

Inter

- laboratory comparison sample studies are being co nducted with CARB

, but a formal schedule and frequency for reviewing and documenting the results and a follow up guideline has

not been established.

Discussion: The analysis of samples collected every twelve days as co ncurrent samples analyzed by

CA RB and BAAQMD provides useful quality assurance information concerning the accuracy of analyses. The review of results is informal and irregular due to time constraints. Laboratory staff stated that the results for samples collected from the same

manifold and analyzed by CARB and BAAQMD usually compare well but occasionally they do not. A guideline criteri on for apparent outliers has not been established that would trigger follow up and communication with the CARB laboratory and internal analyst staff and any corrective action and documentation. Divergent results should trigger review of the analytical system by both laboratories to ensure they are in control.

Method TO

- 11 specifies that results should fall within a ± 20% relative percent

difference. The r esults of investigations into outliers should be documented.

Recommendation: It is recommended that a policy of timely and formal review of inter

- laboratory sample comparison results be established and that conclusions from the review and any corrective actions be documented in a dedicated QA file along with documentation of any follow up communications with CARB or the BAAQMD analyst.

LAB RECOMMENDATION 2 :

The expiration date for laboratory standards provided by the manufacturer is used to d etermine when standards used to calibrate the HPLC instrument need to be replaced. An expiration date for standards is not currently based on the date the sealed standard vial wa s first opened. In addition, the standard vial in current use was not initia led by the analyst who opened it.

Discussion: A new time clock for expiration should be established whenever a sealed standard vial from the manufacturer is first opened and exposed to the air and it becomes a working standard. This expiration date calc ulated from the time the vial is first opened will be shorter than the expiration date provided by the manufacturer which only applies to sealed, ampulated materials and not opened vial materials that are exposed to air and more subject to breakdown and s olvent loss.

Recommendation:38

It is recommended that a policy for calculating expiration dates for standards from the date the standard vials are opened be established. The date the standard is opened and the (new) expiration should be documen ted in a dedicated standards logbook. Vials should be initialed by the person opening the standard along with the date and new expiration

date.

LAB RECOMMENDATION 3 :

BAAQMD does not have a formal system to notify the laboratory when PM filter sample

cus tody has been transferred from the Air Monitoring Section to the Laboratory Services

Section.

Discussion: Sample custody is a process that ensures the integrity of samples and sampling media. Under a formal custody procedure

, samples must be in the contr ol of a single individual until formally relinquished to the next individual. This process is documented and verifiable. Because ambient air samples are not normally intended to be used to support

enforcement

, custody can be somewhat less formal. Howeve r, the custody process should still ensure sample integrity and help to manage sample processing.

The Air Monitoring Section routinely delivers exposed samples to the PM Laboratory by placing them in a shared freezer in the laboratory area. However, ther e is no formal procedure for notifying the laboratory of when and what samples are available to begin the weighing process. The PM analyst indicated that normally there is a verbal notification of sample delivery.

Recommendation: The Air Monitoring Secti on and the Laboratory Services Section should develop a more formal mechanism to transfer samples ready for laboratory analysis.

DATA RECOMMENDATION 1 :

There is a significant time lag for meteorological data validation.

Discussion:

Most of the meteoro logical data validation responsibilities reside with the Planning Office’s validator, who is not in the Technical Services Division. The Planning Office’s meterological data validator’s primary responsibilities and priorities are with planning,

not valida tion, and so he can only validate meteorological data on a time

- permitting basis. This has resulted in a two year lag in meteorological data validation. According to BAAQMD’s Quality Management Plan, meteorological data is used by the District and public for forecasting, photochemical modeling, source modeling, and data analysis. It is

not clear how accurate forecasting, modeling, analysis can be performed using data of unknown quality (i.e., unvalidated data). While users of the data are warned of the risk, the delay in review of data by two years is a significant concern.39

Per 40 CFR Part 58.13(a) The network of NCore multipollutant sites must be physically established no later than January 1, 2011, and at that time, operating under all of the requir ements of this part, including the requirements of appendices A, C, D, E, and G to this part. Data submittal and archiving requirements for meteorological data is quarterly as established at §58.16. The ability to meet completeness criteria in 2011 will suffer if validation continues to be performed on a time permitting basis. Further, meteorological monitoring is a requirement of NCore and validated data will have to be submitted to AQS quarterly.

Recommendation: In order to prepare for NCo re reporting in January 2011, establish a process to accomplish timely and accurate meteorological data validation and reporting.

DATA RECOMMENDATION 2 :

Validators do not always have access to field notes (PM 2.5 and PM 10 ) and initial meteorological QC ch ecks for making data quality determinations.

Discussion: Readily available field notes makes for efficient data validation. Validators currently

contact relevant individuals who maintain field notes or initial QC check info. Where the

field station oper ators notes are insufficient to clarify a concern, there was no hesitation by validators to make contact with field station operator s or their Supervisors to address

concern.

Recommendation: Provide copies of field station operators chain of c ustodies and initial meteorological QC

checks to validators who have a need for this information in performing validation. If several parties need access, it may be cost effective for completed chain of custody (e.g., PM) and other relevant information to be scanned into the mainframe for access by interested parties.

DATA RECOMMENDATION 3 :

The evaluation of data quality trends by the MET/PE Section does not include evaluation of data quality over three year intervals.

Discussion: The primary decisions made with criteria pollutant data is comparison to the NAAQS. Three years of data is used to compare to the NAAQS. Therefore, evaluation of the DQOs and the quality assurance measures that support the DQOs should be made over

the same time interval.

The BAAQMD MET/PE Section reviews quality assurance data and trends. However,

staff do not specifically evaluate trends over three years.40

Recommendation: The MET/PE Section should specifically evaluate quality assurance data in intervals of

thre e years in addition to other intervals.

DATA RECOMMENDATION 4:

Null code flags are not always being identified in AQS.

Discussion: Quality Assurance Handbook for Air Pollution Measurement Systems

, Volume 2, Section

17.3 http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/QA

- Handbook

- Vol

- II.pdf ),

states : " NOTE: it is strongly suggested that flags, specifically the appropriate null data code flags, be used in place of any routine values that are invalidated. This provides some indication to data users and data quality assessors to the reasons why data that were expected to be collected were missing. "

Recommendation: Provide null codes for gaseous data that have been i nvalidated and removed when reported to AQS.

QA RECOMMENDATION 1:

BAAQMD performance audit corrective action process should include a section that documents

how re

- occurrence of problems will be prevented.

Discussion: The primary purpose of the q uality system corrective action process is to correct quality

deficiencies and to prevent future deficiencies. The BAAQMD uses an ODAMN report request to initiate and track corrective action resulting from a performance audit.

However these forms do not have a specific place for the discussion as to how a deficiency will be prevented in the future.

Recommendation: Modify the ODAMN form to include a specific discussion of how future deficiencies will be prevented.

QA RECOMMENDATION 2 :

ODAMNs do not record the assessment of the individuals performing review of corrective actions performed. Superceded ODAMNs were still in use, despite revision on February 10,

2009.

Discussion: ODAMNs were recently revised to include a summary assessment by the Air Monitoring Manager and MET/PE Manger of the corrective actions taken to address any findings41

identified (adequate, inadequate, etc.). Entries were not completed by these individuals for ODAMN dated 02/10/09 for Livermore #1023. Superceded ODAMNs dated

0 2/09/09 and older had signatures by several individuals

, including A ir Monitoring and MET/PE manager, but no assessment of the corrective actions taken were documented. ODAMN dated 02/26/09 was recorded on superceded format.

Recommendation: Record the assessments of individuals performing review of corrective actions performed. Include the revision date and the version on recently revised ODAMN. Archive superceded ODAMN as historical document.

QA RECOMMENDATION 3 :

The Director of Technic al Services resolutions to QA related concerns are not documented.

Discussion: Decisions made by the Director related to QA are undocumented. Whether the Director’s

decision s are the result of resolving differences in opinion between the data generating and reviewing offices o f his directive, they should be documented.

Recommendation: Develop a form similar to ODAMN that identifies date, concern, resolution, date of resolution or issuance of new directive by Director. Should changes be neces sary to any part of the data collection operations, information documented in these forms should be

communicated to relevant, affected staff.

PE RECOMMENDATION 1:

Due to differences in sampler design the PM 2.5 Partisol sampler’s temperature can only be audited at a single point.

Discussion: The procedure of PM 2.5 sampler audits, written for an Anderson sampler

, indicates that

multi

- point temperature sensor audit will be performed. USEPA agrees that this is a good practice.

Recommendation: BAAQMD should take steps to evaluated Partisol temperature sensor performance across the operating range (e.g., track the temperatures at which the audits were performed ).

PE RECOMMENDATION 2 :

During the field audit it was noted that, as expected, the de livery of audit gases produce d a slightly positive pressure on the station manifold.42

Discussion: USEPA has observed that, when delivering gases through the probe, or to the manifold of a monitoring station, excess positive pressure can bias audit results. Each BAAQMD station manifold is equipped with a pressure monitor (Magnehelic) and the auditor and station operator monitor manifold pressure during audits. However, there is no criterion for pressurizing the manifold.

Recommendation:

The PE Group should have a criterion to prevent over

- pressurizing the monitoring

system.