System Audit of the Ambient Monitoring Program: San Diego County Air Pollution Control District September 28

– September 30, 2010

Conducted by:

Air Quality Analysis Office Quality Assurance Office

Air Division Management and Tech nical

Services Division

US EPA Region 9

75 Hawthorne Street San Francisco, California 94105

Final Report: April 2012ii INDEX

PAGE

GLOSSARY OF ACRONYMS

........................................................................ iii

EXEC UTIVE SUMMARY ...............................................................................1

TSA ACTIVITIES .............................................................................................3

OVERVIEW OF AIR MONITORING PROGRAM

...... ........ ....................... 7

FINDINGS..........................

........................................ ............................... 14

APPENDIX A, Summary of Findings

............................................................. 5 2

APPENDIX B, Good Practice Recommendati ons

......................................... 55

APPENDIX C, Organizational Chart...............

......................................... 56iii GLOSSARY OF ACRONYMS

APCD..................................... Air Pollution Control District AQNS.......... ...........................Air Quality Network Section AQS....................................... Air Quality System

BAM...................................... Beta Attenuation Mass Monitor (continuous PM 2.5 )

CARB..................................... Califo rnia Air Resources Board

CARE..................................... Community Air Risk Evaluation

CO.......................................... Carbon Monoxide

CFR........................................ Code of Federal Regulations

DMS.......................

................ Data Management System DQO....................................... Data Quality Objective

EC.......................................... Elemental Carbon

EPA....................................... Environmental Protection Agency

ESC..........

............................. Environmental Systems Corporation

ETS....................................... Electronic Technicians Section

FEM....................................... Federal Equivalent Method

FRM....................................... Federa l Reference Method

GC

- MS................................... Gas Chromatography

- Mass Spectrometry

MDL....................................... Method Detection Limit

MET/PE................................. Meteorology and Performance Evaluation

MTSD..........

.......................... Monitoring and Technical Services Section NAAQS.................................. National Ambient Air Quality Standard

NAMS.................................... National Air Monitoring Station

NIST................................

....... National Institute for Standards and Technology NOx/NOy............................... Nitrogen Oxides

O 3

........................................... Ozone

OC.......................................... Organic Carbon

ORD..............................

......... Office of Research and Development

PAMS..................................... Photochemical Assessment Monitoring Stations

PE........................................... Performance Evaluation

PM.......................................... Pa rticulate matter

PM 2.5

...................................... Particulate matter 2.5 microns or less in aerodynamic

diameter

PM 10

....................................... Particulate matter 10 microns or less in aerodynamic

diameter PQAO......................

............... Primary Quality Assurance Organizationiv QA.......................................... Quality Assurance QAPP..................................... Quality Assurance Project Plan QC.......................................... Quality Control

SPM.

...................................... Special Purpose Monitor

SOP........................................ Standard Operating Procedure

SLAMS.................................. State or Local Air Monitoring Station

SO 2

....................................... .. Sulfur Dioxide SQL

........... ............................. Sample Quantitation Limit

SRP........................................ Standard Reference Photometer

STN........................................ Speciation Trends Network

TAD....................

................... Technical Assistance Document

TSA........................................ Technical System Audit

USEPA or EPA...................... United States Environmental Protection Agency

VOC....................................... Volatile Organi c Compound1 EXECUTIVE SUMMARY

This document is a report on the findings made by the U.S. EPA Region 9 (EPA) while conducting a Technical Systems Audit (TSA) on the air monitoring program of the San Diego County Air Pollution Control District (hereafter r eferred to as “the District”)

. A TSA is an on

site review and inspection of a state or local ambient air monitoring program to assess its compliance with established regulations governing the collection, analysis, validation, and reporting of ambient air q uality data. This TSA meets the requirements for EPA audits of the

District’s monitoring organization required by 40 CFR Part 58, Appendix A, Section 2.5.

The TSA was conducted by EPA staff from September 28 through September 30, 2010

. The District Divis ion that is the subject of this TSA is the Monitoring and Technical Services Division (MTSD). The audit team interviewed management and staff on specific aspects of the air monitoring program including network design, field operations, laboratory operation s, data handling, quality assurance and quality control procedures. The audit team also inspected some of the monitoring sites operated by the District

. The site inspections consisted of an interview with the site operator (when possible), review of statio n and instrument logbooks, and evaluati on of station siting with respect to EPA requirements for probe siting 1

. The laboratory inspection included a review of the particulate matter program for mass determinations, laboratory analysis for volatile organic compounds (VOC), OC/EC (organic carbon/elemental carbon) and carbonyl sample analysis.

The TSA is one of the ways that EPA provides oversight to ensure that data collected by state and local agencies meets certain minimum data quality objectives. Other assessments such as network reviews and performance evaluations are also used to collect information on the overall quality of ambient air monitoring data. These assessments also enable agencies to identify and correct those program elements which may be a dversely affecting the quality of

ambient air data. The results of the TSA are summarized here and fully described in this report, along with recommended actions to address the findings. The specific actions to be taken by the

District will be determined t hrough negotiations between EPA and the District and will be documented in a corrective action plan prepared by the District

.

EPA would like to thank all the staff and management of the District for their support and cooperation during the audit.

A. Pr ogram Strengths:

The District has a robust ambient air monitoring program that is generally well

- run

. Some specific examples of the strength of the District ’s ambient air monitoring program include:

• Sites are well set up with good access, and are clean, or ganized, and have spare parts on

site.

1 40 CFR, Part 58, Appendix E, “Probe and Monitoring Path Siting Criteria for Ambient Air Quality Monitoring”2 • Site operators perform quality control ( QC ) checks every morning, ensuring quick identification of instrument problems and minimizing instrument down time and data

loss.

• There is a good system in place to track preven tive maintenance

.

• There is good back up for field sites since field technicians rotate sites and build expertise

for numerous sites in the network

.

• There is e ffective, routine communication between sections within MTSD (e.g., morning coordination meetings)

.

• The current data management system for continuous data is very robust.

• In most cases

, there is good t raining for technicians.

• For the TO

- 15 method

, the laboratory has established a retention time of 30 days from canister cleaning and certification

, afte r which canisters are to be re

- cleaned prior to shipment and use in the field.

• OC/ EC measurements are very robust: Standard Operating Procedures ( SOPs ) are written and adhered to and the analyses are performed using state

- of

- the

- art instrumentation.

• Staff and management throughout the MTSD were found to be knowledgeable, diligent,

and dedicated.

B. Program Major Findings:

During the TSA, EPA identified areas where the District’s monitoring program should be strengthened. The major findings are:

1. There is not a central, independent, dedicated Quality Assurance (QA) Manager responsible for communicating and ensuring that quality assurance activities are carried out in field operations, information management and laboratory operations. (Finding 1)

2. The Distr ict’s Quality Assurance program documentation, including a Quality Management Plan (QMP), Quality Assurance Project Plans (QAPPs) and SOPs , was not complete and was not approved internally or by EPA at the time of the audit. (Finding 2)

3. The Dist r ict lacks a QA Officer position whose sole responsibility is to oversee quality management of the laboratory. (Finding 5 )

4. Data management issues for particulate matter ( PM ) resulted in incorrect invalidation of

PM data. (Finding 19 )

The individual findings are repo rted in the topic sections of this document and are also summarized in Appendix A. In addition to those areas which should be strengthened, EPA also

identified areas which could be strengthened and those recommendations and suggested best practices are inc luded in Appendix B for the information and use of the Dis trict ’s management.3 TSA ACTIVITIES

From September 28 through September 30, 2010, the U.S. EPA Region 9 Air Quality Analysis Office and the Region 9 Quality Assurance Office conducted a Technical System Audit (TSA) of the District’s ambient air monitoring program. This TSA reviews one aspect of the District’s program responsibility

, collecting and analyzing ambient air quality data

, which is the responsibility of the Monitoring and Technical Servi ces Division of the District. The TSA covered the following program areas:

• General / Quality Management

- Program o rganization

- Facilities

- Independent q uality assurance and q uality c ontrol

- Planning d ocuments (including QMP, QAPPs, & SOPs)

- General d ocumentatio n p olicies

- Training

- Corrective a ction

- Quality i mprovement

- External p erformance a udits

• Network Management / Field Operations

- Network d esign

- Changes to the n etwork since the last audit

- Proposed changes to the n etwork

- Field su pport

• Laboratory Operations : toxi cs and particulate matter

- Routine o perations

- Quality c ontrol

- Laboratory preventive m aintenance

- Laboratory record k eeping

- Laboratory d ata acquisition and h andling

- Specific p ollutants: PM 10 , PM 2.5 , t oxics, PAMS, and OC/EC

• Data and Data Management

- Data h andli ng

- Software d ocumentation

- Data validation and c orrection

- Data p rocessing

- Internal r eporting

- External r eporting

EPA also tracked supporting documentation for data points/sets from calendar years 2008 and

2009.

The EPA staff who conducted the TSA were Me redith Kurpius, Katherine Hoag, Michael Flagg, and Elfego Felix of the Regional Air Division’s Air Quality Analysis Office and Steve Remaley of the Regional Quality Assurance Office. Participating managers and supervisors of the Monitoring and Technical Se rvices Division and positions at the time of the TSA were:4 Mahmood Hossain, Director of Monitoring and Technical Services (MTSD) David Shina, Supervisor of the Air Quality Network Section (AQNDS) David Craig, Supervisor of the Electronic Technicians Sectio n (ETS)

This report is divided the following sections:

• E xecutive Summary

– describes the purpose of the TSA and a summary of the major findings ;

• TSA Activities

– outlines the timing of this TSA and the programs that were covered ;

• Overview of Air Monitor ing Program

– describes the District’s Air Monitoring Program ;

• Findings

– collection of positive, major, and minor findings that includes details associated with each finding ;

• Appendix A

– summarized list of findings with priority ranking ;

• Appendix B

– Rec ommendations that are not required by Federal regulation but would strengthen the District’s monitoring program ; and,

• Appendix C

– District organizational chart

The F indings section includes positive findings along with findings that require corrective ac tion

. T he findings are grouped by program area

. Recommendations t o resolve findings are provided for each finding that requires corrective action to give some indication of the EPA expectations as to how findings can be addressed. If the District has other approaches or alternatives to address the concerns identified, EPA will consider them, provided the corrective action adequately addresses the finding. Appendix A is a summarized list of findings and each finding, excluding positive findings, is grouped a ccording to priority, as either highest priority , moderate priority, or all other significant findings . Higher priority findings are those that: (1) are over

arching issues, (2) have a greater impact on regulatory decisions, and/or (3) have a greater effec t

on data quality. Regardless of priority, all findings, excluding positive findings, need to be addressed in a corrective action report. EPA also identified areas that could be strengthened but corrective action is not required: these are listed as Good P ractice Recommendation in Appendix

B.

Network Management EPA interviewed David Shina and reviewed the District’s Annual Network plan as part

of this TSA. The most recent network plan, submitted in 2010, includes all of the required

information about the m onitoring stations. The air monitoring network within San Diego County meets or exceeds the requirements for the minimum number of monitoring sites designated as SLAMS for all of the criteria pollutants.

Field Operations As part of this TSA, three Elect rical Instrument Technicians in the ETS were interviewed, in addition to Mr. Craig. The Technicians interviewed were Arthur Agas, Noel Seefeldt, and Garnet West. We also interviewed Mr. Shina and observed a quarterly gaseous

audit and a PM 2.5 flow check at Escondido with David Roque and Larry Owusu. All four technicians and the auditor demonstrated a thorough knowledge of the monitoring equipment. In lieu of formal QAPPs and SOPs, the technicians had their own copies of District operational manuals, includi ng information in three

- ring binders that provided all critical information on instrument performance and acceptance criteria.5 EPA visited six of the District’s monitoring stations during the course of the TSA (Alpine, Donovan, Otay Mesa, Escondido, Cam p Pendleton, and El Cajon). All sites met the siting criteria of 40 CFR 58, Appendix E. A more thorough evaluation was performed at all sites except Otay Mesa

. The evaluation included inspection of the inlet manifolds, examination of

station and instrument log books, and a check of whether appropriate QC checks and QA audits were being performed.

All five sites evaluated were well

- run. All sites had clean inlet manifolds. Station logbooks and instrument logbooks were up to date and contained relevant inf ormation on operations and maintenance activities. The stations were clean and neat.

Toxics Laboratory Operations The District’s laboratory staff is one of the most important assets of the laboratory. Some of the analysts/chemists are new to their positi ons, but were found to be properly credentialed, knowledgeable, skilled and dedicated. It was found that the District’s laboratory is in compliance with respect to performing analyses according to EPA methods in most areas of the laboratory,

but does not h ave the benefit of a dedicated laboratory Quality Assurance Officer to oversee the review and documentation of quality assurance results and implementation of corrective actions

on a consistent basis. This is believed to have contributed to the findings

. A nalysts working on the following programs were interviewed as part of this TSA. Quality Assurance : All the laboratory staff and managers were asked about quality assurance in the laboratory.

OC

- EC Carbon Analysis : The analyst working on OC

- EC has several years of experience with the analysis and recently returned from training at Desert Research Institute (DRI)

, where the carbon analyzer currently in use was designed and made

. The laboratory adheres to the SOP written by DRI and updated its SOP to conform to the recently updated DRI SOP. From the interview and observation, it was concluded that the analyst is skilled and knowledgeable and is working with state

- of

- the

- art instrumentation and current SOPs. The department is performing well and there are no f indings requiring corrective action.

PAMS : The analyst s for the PAMS program and the PAMS Canister Cleaning Department were interviewed and found to be knowledgeable and generally doing an effective job with the

resources available. Some deviations from guidance protocols have been made that were well thought out. However, EPA has some additional suggestions in some areas that are presented in the findings.

Method TO

- 15 Analysis : EPA interviewed staff conducting analytical work and cleaning canisters fo r t he TO

- 15

method. The TO

- 15 analyst is tenured, knowledgeable and skilled and generally adheres to guidance and good laboratory practice. The Department monitors the certification and ongoing stability of calibration standards

, a current focus of the reg ion. It is noted the laboratory is using a qu adratic curve fit for the multi

- point calibration in place of the linear multi point fit and use of average response factor specified in Method TO

- 15.6 Method TO

- 15 is a guidance document. EPA evaluated the quadr atic calibration process used by the laboratory and has determined it to be an acceptable deviation from the procedure described in method TO

- 15. An EPA statistician evaluated the calibration used by the laboratory and determined it to be acceptable.

Carbonyls, Method TO

- 11 Analysis : The Carbonyls Department is staffed by a new analyst who is knowledgeable and appears to have good organizational skills. The Department appears to be in a transitional period where procedures and documentation are being reviewed against method requirements. The method is generally being followed

.

Particulate Matter Laboratory EPA visited the particulate matter (gravimetric) laboratory and interviewed Berdine Heckman, the lead PM 2.5 /PM 10 weigher and lead PM 10 data review er. The particulate matter laboratory was well

- maintained, neat and well

- organized. T he PM 10 and PM 2.5 laboratory

measurements demonstrate technical expertise.

Data Management This section covers data management for criteria pollutants (O 3 , CO, NO 2 , SO 2 , PM 2.5 , and PM 10 ); non

- criteria pollutant data management is addressed in the laboratory section. The following staff were interviewed based on their roles in data management: Adam Canter (System Administrator) David Shina (Supervisor of AQNS)

Berdeen Heckman ( PM laboratory staff )

Bill Brick and Lora Kear

- Padilla also have responsibilities for data validation but neither were available during the TSA. Bill Brick provided a document entitled “Level II Data Validation Procedures for Hourly Air Quality and Meteorological Data” that describes the validation procedures.

Quality Assurance/Quality Control EPA interviewed the MTSD Director, AQNS Supervisor, and ETS Supervisor during the TSA and reviewed quality management documents. The District’s quality mana gement system meets the basic EPA requirements. The District has submitted a QMP and a QAPP that have been reviewed by EPA

. The District ’s performance conforms to the data quality objectives and/or the systematic planning developed by EPA for criteria poll utants. EPA and CARB perform national performance and technical evaluations of the monitoring network.7 OVERVIEW OF AIR MONITORING PROGRAM

General Program

The San Diego Air Pollution Control District (District) is the government agency that regulate s sources of air pollution within San Diego County. The San Diego County Board of Supervisors serves as the Air Pollution Control Board. The District funds its activities through a combination of State and Federal grants, permit fees charged to businesses and industries, and vehicle registration fees. The District is organized into five main areas, each of which plays a role in protecting the public from the adverse impacts of air pollution. The District Division that is the subject of this TSA is the Monit oring and Technical Services Division (MTSD)

.

The MTSD is managed by the Director, Mahmood Hossain, who reports directly to the Air Pollution Control Officer of the District, Bob Kard. The MTSD is organized into five sections, four of which are responsib le for producing and submitting ambient air quality data to

EPA: Division Chief, Air Quality Network, Electronic Technicians and Meteorology. The operations of the fifth section, Source Testing, were not evaluated during this TSA. The Air Quality Network S ection (AQNS), managed by David Shina, oversees the laboratory , most QA/QC activities, and network planning. The Electronic Technicians Section (ETS), managed by David Craig, oversees the operation of stations and instruments.

The organization of the Dis trict’s MTSD divides QA /QC responsibility between various sections and various personnel

. For example, calibrations and audits are performed by the Air Quality Network Section ; routine QC is done by field operators in the Electronic Technicians

Section ; la boratory personnel are responsible for validation of data generated by analytical

instruments (Air Quality Network Section) ; and data validation for continuous pollutants ( outside the laboratory ) is performed by the Air Quality Network and Meteorology Sect ions. All QA functions are e mbedded within the sections

. No one functions in a formal QA role to oversee all ambient air monitoring operations at the management level.

Network Management

The ambient air monitoring network in San Diego consists of ten am bient air monitoring sites that utilize a variety of air pollutant instruments and four meteorology sites. Numerous sites have been operating for 20 to 30 years. These long term sites allow the District to examine trends in air quality and assess the progr ess being made in reducing ambient concentrations of criteria pollutants. The following table summarizes the air monitoring sites in San Diego County and the pollutants monitored.

Table 1: Air Monitoring Sites Operated by the District

.8 Source: San Diego Air Pollution Control District 2009 Ambient Air Monitoring Network Plan, Submitted June 29, 2010.

The District network of Photochemical Assessment Monitoring Stations (PAMS) consists

of four sites: Camp Pendleton is a Type 1 (upwind/background/transport ) PAMS site, Kearny Mesa and El Cajon are Type 2 (maximum precursor emission/central business district) sites, and Alpine is a Type 3 (downwind/maximum ozone) site. The District's PAMS network, operating schedule and forecasting s cheme have been approved b y EPA. The District has an approved NCore site ( N ational Core multi

- pollutant monitoring station)

, El Cajon

, which began collecting NCore data beginning January, 2011. Other air monitoring activities include PM 2.5 speciation monitoring and air toxics as in dicated in Table 1.9 Field Operations

Field operation duties are split between two sections within the MTSD

, the AQNS and

the ETS. The day

- to

- day operations at the monitoring stations ar e performed by staff in the ETS

. The ETS has ten full time employees

. The AQNS staff performs the calibrations, standard certifications, and quality control and quality assurance activities at the monitoring sites, in addition to running the District laboratory.

Toxics Laboratory Operations

Analytical laboratories provi de support for measurement methods that are either too complex or sensitive to perform in the field environment. In order to provide these services, laboratories employ complex instrumentation for which staff need highly specialized training.

For ambient air samples to provide useful information or evidence, laboratory analyses must meet the following four basic requirements:

1. Equipment must be frequently and properly calibrated and maintained

. 2. Personnel must be qualified to perform the analys is

. 3. Analytical procedures must be in accordance with accepted practice

. 4. Complete and accurate records must be kept

.

Laboratory operations are managed by David Shina, Supervisor of the AQNS. The District’s laboratory facility is located in t he headquarters office in San Diego, California. Analyses are performed in

- house by laboratory staff with the exception of metal samples which

, at the time of the TSA, were being collected and stored, but not analyzed

, due to staff turnover. It was anticip ated that a recently hired chemist would process the backlog of metals samples. One of the primary responsibilities of laboratory is to process the PM filters

, which includes preparation, weighing, tracking, and storing PM 2.5 and PM 10 filters. In addition to PM responsibilities, the laboratory is also responsible for air toxics (including NATTS) analysis and PAMS monitoring. The compounds that the District routinely analy ze s include:

• PAMS hydrocarbons

• VOCs (Method TO

- 15 target compounds)

• Carbonyls, (Metho d TO

- 11 acetone, acetaldehyde, and formaldehyde)

• Carbon by OC/EC There are a number of additional activities the laboratory undertakes to support the collection and analysis of air pollutants. These include canister cleaning and preparation, data validat ion, and sample storage.

Particulate Matter Laboratory (Gravimetric Laboratory)

The District’s gravimetric laboratory is housed in a small building (large environ mental chamber) located in the h eadquarters’ courtyard. PM 2.5 and PM 10 have a lead staff f or weighing

with a back

- up and a lead staff for data review with a back

- up. Filters are weighed manually

, but data are automatically entered into the GLIMS database.

Data Management10 Data management includes data collection, the data validation process, and a data

management system. Data management at the District follows two separate tracks: one for continuous (i.e., non

- laboratory ) data and one for laboratory data

.

Continuous Data

Data Collection/Data Management System Air quality data produced by continuous analyzers in the field are stored in dataloggers and station computers

. D ata are transmitted hourly directly to the central computer via telemetry. Data flows directly into an SQL database at the District office. Once in the SQL database

, EDES/D TX (formerly ESC) can then be used to view the data. At this point in the data stream

, data are sent to AirNow, the district website, the district backup, and CARB.

Initial data review occurs at the end of each week and second and third levels of review

o ccur at the end of each month (see description below). Changes to data may occur after each of these reviews and are implemented using EDES/DTX: the original SQL database is overwritten when changes are implemented. The original data are saved in hard copy , on the backup website,

and as text files. The System Administrator is the only person who has access to edit the

database , with an administrative version of EDES/DTX that is password protected. Once a month’s data are finalized

, EDES/DTX is used to gener ate an AQS file for that month and uploaded through CDX. The District subm its all required data to the EPA AQS database

, including concentration data for all criteria pollutants, supporting precision and accuracy data

and data from the Photochemical Assess ment Monitoring Station (PAMS) network. Data submitted to AQS is certified annually by the District. The District’s central computer is backed up on tape and disk storage every night

. W eekly archives are stored offsite at a data storage facility. Loss of data in the event of significant computer problems is expected to be minimal given the redundant backup systems in place at the air monitoring stations and at the offsite data storage facility.

Data V alidation

The data are first reviewed by the station technicians. At the end of each week

, the station technicians send an email to the AQNS Supervisor and the System Administrator identifying any data issues. Station technicians suggest data treatment (e.g., invalidation) and a flag. The Supervisor and Syst em Administrator confer and document their decision on a printed copy of the email that is signed and stored with a hard copy of the hourly data. In some cases

, flags are generated automatically by the datalogger, e.g., the maintenance flag is generated wh en ever instruments are in maintenance mode. Flags that are generated by station dataloggers and any other information describing the data being processed are included in the data stream at this point

as well. The System Administrator has to change automati c flags that are inappropriately applied (e.g., maintenance flag applied to points during a calibration) man ually

. All flags identified at the weekly review are implemented by the System Administrator. At the end of each month the System Administrator gene rates a preliminary AQS file and provides this to the Senior Meteorologist for the next level of review (referred to as Level II data validation). This process is new , having been implemented within the past year. The Senior Meteorologist conducts the foll owing activities, as described in “Level II Data Validation Procedures for Hourly Air Quality and Meteorological Data”:11 • Computerized data screening for consistency and accuracy

• Manual inspection of the data

• Visual inspection of graphical representations o f summary statistics

• Detailed data investigation where problems are found or suspected

• Data resolution where problems are found

• Comparison of summary statistics with data from other sites within the monitoring

network. The System Administrator implements any additional changes and the data are final ized and

submitted to the AQS database.

Lab oratory Data (PM)

Data Collection/Data Management System: PM data are automatically logged in the GLIMS data system as the filters are weighed.

After data are valid ated, they are converted to AQS format using an Excel conversion program.

The l ead PM Specialist emails data to the System Administrator for storage until all monthly data have been reviewed and are ready for upload. When data are ready for submission, the System Administrator uploads all laboratory and non

- laboratory data.

Data Validation for PM 2.5 : The first level of review of PM data is performed by the field technician or assistant , who notes any issues on the chain

- of

- custody form. The person in char ge of weighing PM 2.5 filters

, reviews the chain

- of

- custody form for any unusual issues. These are note d in the GLIMS system

, and in a separate logbook for the lead PM 2.5 data review. At the next level

, the issues identified

are evaluated as to whether the data should be invalidated

. That decision is noted in the same logbook. In addition, the lead reviewer looks at all the PM 2.5 data, not only those that have been specifically noted

.

Data Validation for PM 10 : The first level of review of PM data is perfor med by the field technician or assistant , who notes any issues on the chain

- of

- custody form. The person in charge of weighing PM 2.5 filters

review s the chain

- of

- custody form for any unusual issues

. T hese are noted in the GLIMS system. At the next level, th e decision is made whether or not to invalidate data. As the data review team personnel are the same as the weighing team, each person tries to review data from filters that

s/he did not weigh; however, this is not always possible

. After the second level o f validation

, data are invalidated in the GLIMS system, as needed.

Quality Control and Quality Assurance

Quality management is the system put in place that oversees quality assurance, quality control, and quality improvement activities. EPA requires th at ambient air monitoring agencies have a quality management system that conforms to 40 CFR Part 50 Appendix A and the EPA quality policy (EPA Order CIO 2105.0). Additionally, EPA grant regulations require each grantee to provide for QA activities (40 CFR Part 31.45).12 Specifically, 40 CFR Part 50 Appendix A Section 2 requires that each ambient air monitoring PQAO conforms to certain quality management practices. These include:

• Having a documented quality system that meets EPA requirements for QMPs and QAP Ps.

• Having a quality management function that is independent of air monitoring operations.

• Developing or adopting DQOs, or equivalent systematic planning procedures, for all monitoring programs.

• Participating in National Performance Evaluation Programs, wh ich consist of performance audits used to independently determine program adequacy, national monitoring network performance, and national consistency.

• Submitting to Technical Systems Audits by EPA at a frequency of every three years or

less.

• Using certifie d reference materials to standardize monitoring equipment. EPA views the application of these quality management system components as integral to a credible monitoring program. Insufficient quality management and control has been cited as rationale to supp ort legal challenges to NAAQS designation decisions.

Quality control and quality assurance are the two components of quality management within a monitoring program that support the assumption that the data they collect represent the true air quality of the area. They are the means by which an organization manages its quality

activitie s in a systematic, organized manner and they provide a framework for planning, implementing, and assessing work performed by an organization. A properly developed QA/QC

prog ram encompasses a variety of technical and administrative elements, including policies and objectives, organizational authority, responsibilities, accountability, and procedures and practices. Quality assurance is a management or oversight function that in cludes setting policy and running an administrative system of management controls that cover planning, implementation, the review of data collection activities, and the use of data in decision making. Quality control is a technical function that includes a ll the scientific precautions, such as

calibrations and verifi cations

, needed to acquire data of known and a ppropriate quality.

MTSD divides QA responsibility among various sections and personnel. For example,

calibrations and audits are performed under the Air Quality Network Section

, while routine QC is done by field operators under the Electronic Technicians Section. All QA functions are embedded within the sections rather than having an assigned QA function that would oversee all

ambient air monitori ng operations at the management level. Most quality assurance functions reside in the Air Quality Network Section, managed by David Shina. In general, different people perform calibrations rather than performing audits of a specific instrument type.13 FIND INGS

Program Area Finding

Numbers EPA Contact

General 1

- 2 Meredith Kurpius Network Management 3 Meredith Kurpius Field Operations 4 Meredith Kurpius Laboratory

– QA 5

- 6 Steve Remaley Laboratory

– PAMS 7

- 12 Steve Remaley Laboratory

– TO

- 15 13

- 14 Stev e Remaley Laboratory

– TO

- 11 15

- 16 Steve Remaley Particulate Matter (Gravimetric) Laboratory 17 Michael Flagg Data Management 18

- 20 Meredith Kurpius Quality Assurance/Quality Control 21

- 22 Kate Hoag14 Finding # 1 Agency: San Diego County APCD

Date o f Audit: September 28

- 30, 2010 Program Area: General Finding:

There is no independent, dedicated QA Manager responsible for communicating and ensuring that quality assurance activities are carried out in field operations, information management, and

lab oratory operations. Most QA functions were being carried out but some specific oversights

were identified such as:

• There is no secondary, independent QA review for logbook review, data review or peer

review of benchtop laboratory data

.

• Biweekly precision QC checks are not always performed according to schedule

.

• MDLs for TO

- 15 and PAMS missed or not performed

.

• There is a lack of training files and Initial Demonstration of Analyst Proficiency

.

• There is no over

- arching corrective action process

.

• Internal audi ts (i.e., data tracking) are not being performed

.

• S ome PM 2.5 data points were incorrect ly invalidat ed

. Discussion: 40 CFR, Part 58, Appendix A, Section 2.2, requires that each ambient air monitoring organization ha ve an independent quality assurance func tion. Quality assurance functions are currently spread throughout the District’s air monitoring sections. While the organizational

struc ture may remain the same, a QA M anager should be designated to provide leadership to the

various sections. The designate d individual should have overall QA authority to conduct independent oversight and to require corrective action. In order to function effectively, the QA

M anager should oversee

, rather than perform

, staff level duties.

Th e QA M anager would be responsible for:

• Periodic audits of the quality management system (management system reviews) ; routine procedures ’ and data quality to identify areas of improvement and to ensure that monitoring programs continue to follow sound and documented procedures consistently

.

• Review of performance for certain QA/QC activities

, such as MDLs; span/precision checks;

annual certification of the O 3 transfer and flow standards; the certification of calibration gas cylinders for NOx, CO and SO 2 instrument calibration; and the condu ct of flow audits of PM instruments samplers.

• D evelopment of a formal corrective action process.

• R outine review and tracking of precision and accuracy data.

In general, EPA QA/R

- 5, EPA Requirement for Quality Assurance Project Plans , discusses

assessment s that should be elements of a quality management program. These include “surveillance, management systems reviews, readiness reviews, technical systems audits, performance evaluations, audits of data quality, and data quality assessments.”

References:

4 0 CFR, Part 58, Appendix A, Section 2.2 EPA QA/R

- 5, EPA Requirement for Quality Assurance Project Plans15 Recommendation to Address Finding: Establish a centralized, independent and dedicated QA M anager position. This position should be responsible for est ablishing, communicating, and overseeing (approving) the QA/QC activities conducted in the field, information management, and laboratory operations. Additionally, because laboratory oversight involves knowledge of complex chemical methods different from

th e technical expertise employed to run field stations, there should be separate laboratory and field operations quality assurance responsibilities assigned to support the QA M anager (see Finding 5 ).16 Finding # 2 Agency: San Diego County APCD

Date of A udit: September 28

- 30, 2010 Program Area: General Finding: The District’s Quality Assurance program documentation, including a Quality Management Plan (QMP), Quality Assurance Project Plans (QAPPs) and Standard Operating Procedures (SOPs), was not compl ete and was not approved internally or by EPA at the time of the audit.

Discussion: EPA requires that organizations develop a QAPP for each type of ambient pollutant being measured. The QAPP integrates all technical and quality aspects of a project, inc luding planning, implementation, and assessment. The purpose of the QAPP is to document planning activitie s for collecting environmental data operations and to provide a project

- specific “blueprint” for obtaining the type and quality of environmental data needed for a specific decision or use.

The QAPP documents the quality assurance and quality control that are applied to an environmental data operation to assure the results obtained are of the type and quality needed and expected. Further guidance on de veloping QAPPs can be found in the guidance documents “EPA Requirements for Quality Assurance Project Plans,” EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002.

The District has many SOPs th at are used internally, but these have not been submitted as part of a QAPP for EPA approval. The District is currently developing SOPs for gaseous pollutant audits and NCore monitoring activities.

References:

EPA/240/B

- 01/003, March 2001 “Guidance for Q uality Assurance Project Plans,” EPA/240/R

- 02/009, December 2002. Recommendation to Address Finding: The District should work with EPA to develop approved QA planning documents. T he District should develop and implement a schedule to update these documen ts on a routine schedule

.17 Finding # 3 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Network Management Finding: The scale of representativeness for some sites does not reflect contributing sources.

Discussion:

PM 10 at the Otay Mesa

– Border Crossing is heavily influenced by emissions from the busy border crossing that is adjacent to the monitor. Currently Otay Mesa PM 10 is listed as neighborhood scale

, which is intended to characterize “conditions throughout som e reasonably homogenous urban sub

- region with dimensions of a few kilometers” (40 CFR 58, Appendix D, Section 4.6(b)(3)). The scale designation does not reflect the very localized impacts of the truck crossing, which significantly impact the PM 10 levels at the site. NO 2 would be similarly affected by local emissions from the border crossing. While some O 3 may be scavenged by high levels of NO, EPA does not expect this to be the dominant influence and therefore O 3 should remain listed as neighborhood scale.

NO 2 at El Cajon is listed as an urban scale site with a monitoring objective of source impact. Urban scale is intended to represent areas of several kilometers up to 50 kilometers and to

characterize area

- wide concentrations (40 CFR 58, Appendix D). As su ch, urban scale is not an appropriate scale for a source impact measurement.

References: 40 CFR 58, Appendix D Recommendation to Address Finding:

Review PM 10 and NO 2 at the Otay Mesa

– Border Crossing site and change scale of representativeness as appr opriate. Review the monitoring objective and scale of representativeness of NO 2 at El Cajon and choose the appropriate combination.18 Finding # 4 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Field Operations

Positi ve Finding:

Overall

, the field station operations are performed very well. The positive findings includ e :

• C ompetent and dedicated field station operators and managers

.

• Comprehensive training support for staff

.

• Field station operators have primary responsi bility for no more than three sites

.

• I nstruments are kept in good working order

.

• Very well organized, clean and safe field stations

.

• Good backup system of rotating field operators

.

• K nowledgeable audit staff

.

• Preventive maintenance is tracked effectively

.

• Operators perform QC checks daily , ensuring quick identification of instrument problems and minimizing instrument down time and data loss.

Discussion:

N/A

References:

N/A Recommendation to Address Finding:

N/A19 Finding # 5 Agency: San Diego Coun ty APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– QA Finding: The laboratory does not have an independent QA Officer who can ensure effective implementation of a QA program. Instead, each analyst is responsible for reviewing the q uality

of the data s/he has collected

. As a result, some routine QA functions are not being performed and others are not being performed on schedule (see Finding 1).

Discussion:

T he laboratory would benefit significantly from centralizing the QA function and assigning an independent QA Officer

. The QA O fficer function is best executed completely independently from the data generation process and influence of pressures to meet deadlines. The observation

of lack of secondary peer review and missed MDL studi es indicate s that the analysts are pressed

for time and would benefit from a QA O fficer who would be able to bring resolution to quality

assurance issues.

References: 40 CFR, Part 58, Appendix A, Section 2.2 EPA QA/R

- 5, EPA Requirement for Quality Assura nce Project Plans Recommendation to Address Finding:

It is recommended that a full time QA Officer be assigned to oversee implementation of the laboratory’s QA Program. QA records should be centralized in order to facilitate independent monitoring of rou tine activities and the corrective action process to ensure non

- conformances

and out of control events are resolved

.20 Finding # 6 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– QA Finding:

The District ’s SOP for training does not reflect current practice regarding laboratory training

. Discussion: The District’s SOP states that new employees will receive training from instrument vendors, however the audit revealed that funding was not available for re cent hires to receive vendor training

. I nstead

, recent analyst hires receive on the job mentoring from an experienced analyst. Additionally, a formalized procedure for assessing and documenting analyst proficiency is not documented (i.e., Initial Demonstra tion of Proficiency).

SOPs serve to document official policy for critical functions and can play an important role in demonstrating data defensibility and evidentiary role. It is important therefore that SOPs to reflect actual practice. Alternate procedur es
, such as mentoring

, should be documented in the

SOP. The procedure for assessing new analyst proficiency could be improved upon by requiring analysts to demonstrate proficiency in running an audit sample and/ or passing a written test.

Results of Initial Demonstration of Proficiency and ongoing assessments and training should be documented in a dedicated training file.

References:

Recommendation to Address Finding: The training SOP should be revised to reflect actual practice. Additionally, the SOP s hould describe how Initial Demonstration of Proficiency of new analysts will be assessed and any planned ongoing assessments. Records should be documented in a dedicated training file.21 Finding # 7 Agency: San Diego County APCD

Date of Audit: Septembe r 28

- 30, 2010 Program Area: Laboratory

- PAMS Finding:

The PAMS GC

- FID is an old instrument and may not be performing reliably.

Discussion: Staff stated that some components of the GC

- FID, such as the concentrator

, are no longer supported by the equip ment vendor. The transfer lines were recently replaced to eliminate active

sites

, which caused unacceptable recoveries for some compounds reflected in the Regional PE sample study and laboratory standard recoveries. Although recoveries improved by 20% with the replacement lines, staff stated recoveries are still low by 20%. Additionally, the current equipment configuration does not have the apparatus that facilitates the dilution of standards used for multipoint calibration and method detection limit studie s (MDLs). Most importantly, responses from staff suggest the instrument detector response does not always give reproducible results on successively analysis for daily calibration and the efforts to identify and correct the problem have not been successful

. References:

Recommendation to Address Finding: The PAMS program is currently undergoing a reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determine if the PAMS program reinv ention will affect the District’s PAMS program. It is recommended that the laboratory research the purchase of replacement instrumentation. Replacing the instrument with

either a new GC

- FID or an upgrade to a GC equipped with a dual FID

- mass spectrometer
d etector configuration would represent a significant up grade for the detection process
, reducing uncertainty as to compound identification. The dual GC mass spectrometer

– FID configuration has the advantage of allowing (simple) calibration of the FID detec tor with propane for quantitation

. T he mass spectrometer detector would provide superior target and non target compound identification and reduce co

- elution and thus the effort needed to interpret very complex PAMS chromatograms.22 Finding # 8 Agency: S an Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– PAMS Finding:

Multi

- point calibrations are not being performed according to guidance and good laboratory practice. A single point calibration is being performed instea d.

Discussion: Laboratory staff stated that multi

- level calibrations are not being performed due to the difficulty

of diluting calibration gas standards without the benefit of dynamic dilution equipment. According to guidance (PAMS TAD, Sec. 2.3.4.2, Tab le 2

- 17), a multi

- point calibration consisting of a minimum of three points bracketing the expected sample concentration range is to be performed prior to analysis at the start of the season and when system maintenance is performed. No t performing a multi

- point calibration may produce less accurate data

. References:

Recommendation to Address Finding: The PAMS program is currently undergoing reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determine if the PAMS program reinvention will affect the District’s PAMS program. Calibration

, including

multi

- point calibration

, should be performed according to guidance to ensure the most accurate sample quantitation. Based on analyst respo nses, it is believed that the use of upgraded PAMS

instrumentation with dynamic dilution equipment should address this issue.23 Finding # 9 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

- PAMS Finding:

P AMS m ethod detection limit (MDL) studies are not being performed as required by guidance. According to staff, MDL studies have not been performed because the instrument lacks the dilution equipment accessory necessary to perform the studies at low concentr ation levels.

Discussion: The PAMS TAD (Table 2

- 17) specifies that MDL studies must be performed according to procedures specified in 40 CFR 136 Part B. MDL studies should be performed prior to the start

of the PAMS season. Acceptance criteria of 2 ppbC or better for sensitivity must be met in order to achieve method data quality objectives. MDL studies are needed so that accurate reporting limits may be documented and submitted whenever non

- detect data is reported.

References: PAMS TAD (Table 2

- 17)

40 CFR 136 Part B Recommendation to Address Finding: The PAMS program is currently undergoing a reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determine if the PAMS program reinve ntion will affect the District’s PAMS program. MDL studies should be performed according to guidance. The purchase of new testing equipment with the dilution accessory would facilitate the process and address this issue.24 Finding # 10 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

- PAMS Finding: Internal standards with calculated relative retention times and reference peaks are not being used as part of the target peak compound identification process as prescribed in the PAMS guidance

document. In addition, quality control measures supportive of proper peak identification and quantitation used to be, but are no longer being

, performed as recommended in guidance. Examples of these quality control measures include periodic GC/MS sample confirmation of a percentage of analysis, or periodically re

- analyzing samples on a nother, well

- characterized GC system to identify co

- eluting compounds. Staff stated that co

- elution is not evaluated

. Discussion:

The PAMS TA D provides guidance on peak identification (Sec. 2.3.7.2). Although gas chromatographic systems are generally stable, variations in conditions do occur that can shift

absolute retention times. This can result in improper or missed compound identification. The use

of internal standards with calculated relative retention times would address this concern and generally facilitate the target compound identification process. This seems especially important with the complex chromatography of PAMS analysis. Althoug h the PAMS GC FID method is an acceptable method that can provide accurate results, it is believed that upgrading the system configuration to a dual column

- dual detector FID and MSD or similar configuration would yield significant benefits. Quantitation could still be performed by calibrating the FID detector with propane

, while the mass spectrometer would be used for ( more precise ) compound

identification. Such a system configuration may represent a natural evolution for PAMS analysis

. It is already bein g used in some commercial laboratories. Incorporation of a MS detector into the system configuration would greatly facilitate the target and any non

- target contaminant peak identification process

, while significantly reducing the human error factor. T he pr ocess would

add a higher level of confidence to contaminant identification inherently afforded by mass spectral identification. The mass spectrometer detector would eliminate co

- elution issues and the need to manually edit and integrate co

- eluting peaks.

References:
PAMS TAD, Sec. 2.3.7.2 Recommendation to Address Finding: The PAMS program is currently undergoing a reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determine if th e PAMS program reinvention affects the District’s PAMS program. At a minimum, it is recommended that the laboratory use i nternal standards as prescribed in the PAMS method. GC/MS confirmation or cross duplicate analyses on a second GC system should also be performed on a percentage of samples according to guidance. GC/MS should be used to resolve issues with samples containing co

- eluting compounds in place of manual integration of unresolved gas chromatographic peaks. The laboratory is encouraged to investi gate and consider upgrading its system configuration to one that includes a mass selective detector for compound

identification

.25 Finding # 11 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

- PAMS

Findi ng: According to analyst staff, replicate samples are being analyzed, but precision and corrective

action criteria have not been established.

Discussion: Guidance prescribes replicate analysis at a frequency of 10% of samples (PAMS TAD, Table 2

- 17). Gu idance further requires sample analysis to be repeated whenever precision for target concentrations greater than 5 times the MDL exceed ±25% RPD. Precision results that fall outside prescribed limits are an indication of analytical problems that require at tention.

References:
PAMS TAD, Table 2

- 17 Recommendation to Address Finding: The PAMS program is currently undergoing a reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determi ne if the PAMS program reinvention will affect the District’s PAMS program. It is recommended that the laboratory determine MDLs and then establish precision and corrective action criteria.26 Finding # 1 2 Agency: San Diego County APCD

Date of Audit: Se ptember 28

- 30, 2010 Program Area: Laboratory

– PAMS Finding: Staff stated that during the PAMS season when the canister selected for blanking fails to meet criteria, the cans are used for collection and the data are flagged. However, guidance that stat es that the entire bank of canisters being cleaned should be rejected. Additionally, laboratory staff stated that although the “bad canister” that failed blanking is re

- cleaned, it is not necessarily re

tested per the TAD and the District’s SOP. The SOP st ates, “ If the batch of canisters requires re

cleaning, the same canister should be selected for a QC sample and analyzed again

. ”

Discussion: Cleaning canisters to meet predetermined criteria ensures they are free of surface contamination and that sample s collected are representative. Not following this process results in flagged data

which are less useful because there is uncertainty as to the true source of target compound

detects. Both the PAMS guidance document and the District’s SOP state that a blan ked canister that failed the certification process should be re

- tested following re

- cleaning to confirm the

canister meets certification criteria.

References: TAD Sec. 2.5.2.3, pg. 105 TAD, Sec 2.5.2.3, pg. 105; SOP pg. 4

Recommendation to Address Findi ng: The PAMS program is currently undergoing a reinvention at the national level. Prior to implementation of any corrective actions, the District should confer with EPA to determine if the PAMS program reinvention will affect the District’s PAMS program. It is recommended that the laboratory adhere to guidance and reject the bank of cleaned cans when a certification blank fails

to meet criteria. If this is not possible due to resource/time constraints then it is recommended

that all canisters in the batch are flagged for the failed compounds

.27 Finding # 13 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– TO

- 15

Positive Finding:

T he laboratory has established a retention time for TO

- 15 canisters of 30 days from canister cleaning and certification

, after which canisters are to be re

- cleaned prior to shipment and use in

the field.

Discussion:

N/A

References:

N/A Recommendation to Address Finding:

N/A28 Finding # 14 Agency: San Diego County APCD

Dat e of Audit: September 28

- 30, 2010 Program Area: Laboratory

– TO

- 15 Finding: The laboratory performed MDL studies in 2007 and 2009, but did not perform an MDL study in

2008.

Discussion: MDL studies should be performed annually according to procedures d escribed in 40 CFR Part 136 or any time significant maintenance is performed on

, or changes are made to

, the instrument (TO

- 15 TAD, Sec 11.2). MDL studies are necessary when non

- detect results for samples are reported and to ensure the method objective sen sitivity of 0.2 ppbv is met for target compounds.

References:

TO

- 15 TAD, Sec 11.2

40 CFR Part 136 Recommendation to Address Finding: MDL studies should be performed at a minimum annually or any time significant changes are

made to the instrument. The t arget method sensitivity for Method TO

- 15 is 0.2 ppbv.29 Finding # 15 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– TO

- 11

Positive Finding: Duplicate and co

- located samples are being analyzed as a QA m easure. Discussion:

N/A

References:

N/A Recommendation to Address Finding:

N/A30 Finding # 16 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Laboratory

– TO

- 11 Finding: Analyst staff stated that field blank r esults are not being submitted and that the field sample results are being submitted corrected for field blanks.

Discussion: Carbonyl sample data should not be field blank corrected. Field blank results and sample results should be reported separately. C onsistency at each site is of utmost importance because the purpose of NATTS is long term trending. If data for one year are corrected, all other data must
be corrected.

References:

Recommendation to Address Finding: The laboratory should report uncorrected carbonyl sample data. Results for field blanks should also be reported.31 Finding # 17 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: PM Laboratory

Positive Finding: Many of the previous findings from EPA’s 2 006 TSA have been addressed and the laboratory has made many improvements in the past years.

Discussion:

N/A

References:

N/A Recommendation to Address Finding:

N/A32 Finding # 18 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Data Management

Positive Finding: Overall, the current data management system for continuous data is very robust. Specific examples of this effective system include:

• Communication between the System Administrator and the AQNS Supervisor wa s effective and well

- documented.

• The System Administrator was able to access specific data points for continuous data quickly as well as the documentation for decisions made regarding specific data points.

• The recent improvements in data validation that include analyses and visual tools such as graphs and regressions provide for a very high quality level of review.

Discussion:

N/A

References:

N/A Recommendation to Address Finding:

N/A33 Finding # 19 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Data Management Finding: Data management issues for PM resulted in incorrect invalidation of PM data.

Discussion:

We found numerous instances of lost data due to improper data management in the PM laboratory

. In som e cases valid PM 2.5 data were invalidated in AQS with no record of the reason and in other cases data were simply missing with no valid reason or documentation. Some data were subsequently found and uploaded to AQS. Specific examples include:

• PM 2.5 data at Escondido on January 1, 2009 were invalidated in AQS due to a laboratory

error. The original file for the Escondido data point was found to have a valid entry of 64.9 μg/ m 3

, which was overwritten with the AR null data code in a later file with no associat ed

documentation. The invalidated Escondido data point was the highest value of the year.

• PM 2.5 data at El Cajon on March 2, 2009 were invalidated and entered into AQS as due to a

machine malfunction. The El Cajon data point was missing in the GLIMS datab ase, although

it was determined in the laboratory books to have been a valid sample. The District believes that this data point was overwritten with a data point from 1999 due to a data file issue that

the District encountered when the barcode reader was r eprogrammed

.

• Calexico data for December 17, 2007 was originally missing from AQS. Following an inquiry by EPA, these data were located and uploaded to AQS. The District indicated that data for this data were not originally in the data file that was sent t o the System

Administrator. While most PM data are stored and uploaded reliably, the incidents above indicate that the data management and validation of PM data could be made more robust.

References:

Recommendation to Address Finding:

There should be one more level of data validation, similar to that of the gaseous pollutants

, and data tracking performed by the QA Manager.34 Finding # 20 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: Data Management / PM Labora tory Finding:

Full s econdary review of PM 10 filter data is not being performed by an independent reviewer.

Discussion: Currently, PM 10 filters are weighed and the initial data review is performed by the laboratory technicians. Unlike the PM 2.5 indepen dent, secondary d ata review process , the secondary data review for PM 10 is often performed by the same staff who weighs the filters and performs the primary data review.

References:

Recommendation to Address Finding: Secondary review of PM 10 filter da ta should be performed by a staff member who is independent

of the weighing and data collection process.35 Finding # 21 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program Area: QA/QC Finding: There is no formal tracking of fa iled gaseous audits.

Discussion: EPA quality management standards (EPA QA/R

- 2, Quality Improvement Section) require that management and staff “ensure that conditions adverse to quality are” prevented, identified promptly, fully defined, corrected, preven ted from recurring, and documented as corrective

actions which are tracked to closure. There should be a formal, documented mechanism for elevating potentially significant corrective actions originating from the audit staff concerning

audit findings

. Refe rences: EPA QA/R

- 2, Quality Improvement Section Recommendation to Address Finding: Develop a formal corrective action process for the gaseous audit program.36 Finding # 22 Agency: San Diego County APCD

Date of Audit: September 28

- 30, 2010 Program A rea: QA/QC Finding: Some documentation could be improved to ensure complete records retention at the site and transmission of accurate site operations information to data validators.

Discussion:

EPA observed that the District has some excellent forms a nd procedures to document activities and that the written documentation by the field technicians was generally very thorough.
However, some information is stored in binders at the site and at the office, and the written comments did not always match what w as recorded on the QC sheets. For example, some of the calibration sheets were missing in the instrument logs. Also, there is not a clear record of PM calibrations and outcome at each site, as there is for O 3 calibrations

. EPA noted that audits performed by CARB should also be logged with the other calibration information. The lack of clear documentation for a recent CARB audit led to it being flagged as routine maintenance/repair.

General information should be recorded in logbooks that are bound, tracke d, and have page number s in order to ensure legal defensibility. Routine checks may be recorded on prepared forms (including particulate monitor flow checks) that include page numbers or another method of ensuring information is not missing.

References: Quality Assurance Handbook for Air Pollution Measurement Systems, Volume II, Section 5 Recommendation to Address Finding: Ensure that all field/audit/calibration staff are aware of documentation procedures. Add summary sections to the logbooks for CARB a udits and also PM 2.5 audits and calibrations

. Consistency and completeness of the logbook information may also be improved by implementing a more formal review by the ETS manager, David Craig.37 APPENDIX A: SUMMARY OF FINDINGS

Highest Priority Finding 1: There is no independent, dedicated QA Manager responsible for communicating and ensuring that quality assurance activities are carried out in field operations, information management, and laboratory operations. Most QA functions were being carried out but some specific oversights were identified such as:

• There is no secondary, independent QA review for logbook review, data review or peer

review of benchtop laboratory data

.

• Biweekly zero/span QC checks are not always performed according to schedule

.

• MDLs f or TO

- 15 and PAMS missed or not performed

.

• There is a lack of training files and Initial Demonstration of Analyst Proficiency

.

• There is no over

- arching corrective action process

.

• Internal audits (i.e., data tracking) are not being performed

.

• Some PM 2.5 da ta points were incorrectly invalidated.

Finding 2: The District’s Quality Assurance program documentation, including a Quality Management Plan (QMP), Quality Assurance Project Plans (QAPPs) and Standard Operating Procedures (SOPs), was not complete and wa s not approved internally or by EPA at the time of

the audit.

Finding 5 : The laboratory does not have an independent Quality Assurance (QA) Officer who

can ensure effective implementation of a QA program. Instead, each analyst is responsible for reviewing the quality of the data s/he has collected

. As a result, some routine QA functions are not being performed and others are not being performed on schedule (see Finding 1).

Finding 19 : Data management issues for PM resulted in incorrect invalidation of PM data.

Moderate Priority Finding 3: The scale of representativeness for some sites does not reflect contributing sources.

Finding 7 : The PAMS GC

- FID is an old instrument and may not be performing reliably.

Finding 20: Full s econdary review of PM 10 filte r data is not being performed by an independent

reviewer.

Finding 21: There is no formal tracking of failed gaseous audits.

Finding 22: Some documentation could be improved to ensure complete records retention at the

site and transmission of accurate sit e operations information to data validators.

All Other Significant Findings Finding 6 : The District’s SOP for lab training does not reflect current practice.

Finding 8 : PAMS m ulti

- point calibrations are not being performed according to guidance and good laboratory practice. A single point calibration is being performed instead.38 Finding 9: PAMS m ethod detection limit (MDL) studies are not being performed as required by guidance. According to staff, MDL studies have not been performed because the instrume nt lacks the dilution equipment accessory necessary to perform the studies at low concentration

levels.

Finding 10: Internal standards with calculated relative retention times and reference peaks are not being used as part of the target peak compound iden tification process as prescribed in the PAMS guidance document. In addition, quality control measures supportive of proper peak identification and quantitation used to be, but are no longer being

, performed as recommended in guidance. Examples of these qua lity control measures include periodic GC/MS sample confirmation of a percentage of analysis, or periodically re

- analyzing samples on a nother, well

characterized GC system to identify co

- eluting compounds. Staff stated that co

- elution is not

evaluated

.

Fi nding 11: According to analyst staff, replicate samples are being analyzed for PAMS , but precision and corrective action criteria have not been established. Additionally, staff stated they have discussed control charting precision results, but this has not been implemented to date.

Finding 12: Staff stated that during the PAMS season when the canister selected for blanking fails to meet criteria, the cans are used for collection and the data are flagged. However, guidance that states that the entire bank o f canisters being cleaned should be rejected. Additionally, laboratory staff stated that although the “bad canister” that failed blanking is re

cleaned, it is not necessarily re

- tested per the TAD and the District’s SOP. The SOP states, “ If

the batch of ca nisters requires re

- cleaning, the same canister should be selected for a QC sample and analyzed again

. ”

Finding 14: The laboratory performed TO

- 15 MDL studies in 2007 and 2009, but did not perform an MDL study in 2008.

Finding 16: Analyst staff stated t hat field blank results are not being submitted and that the field sample results are being submitted corrected for field blanks for TO

- 11

.

Positive Findings Finding 4: Overall

, the field station operations are performed very well. The positive findings

i nclud e :

• C ompetent and dedicated field station operators and managers

.

• Comprehensive training support for staff

.

• Field station operators have primary responsibility for no more than three sites

.

• I nstruments are kept in good working order

.

• Very well organize d, clean and safe field stations

.

• Good backup system of rotating field operators

.

• K nowledgeable audit staff

.

• Preventive maintenance is tracked effectively

.

• Operators perform QC checks daily , ensuring quick identification of instrument problems

and minimi zing instrument down time and data loss.39 Finding 13 (Positive) : T he laboratory has established a retention time for TO

- 15 canisters of 30 days from canister cleaning and certification

, after which canisters are to be re

- cleaned prior to shipment and use i n the field.

Finding 15 (Positive) : Duplicate and co

- located samples are being analyzed as a QA measure

for TO

- 15

.

Finding 17 (Positive) : Many of the previous findings from EPA’s 2006 TSA have been

addressed for the PM laboratory and the laboratory has made many improvements in the past

years.

Finding 18 (Positive): T he current data management system for continuous data is very robust.40 APPENDIX B: GOOD PRACTICE RECOMMENDATIONS

NETWORK DESIGN RECOMMENDATION 1: Frequent land use changes on large dirt l ots by the border and construction activities near both Otay Mesa

- Donovan and Alpine sites indicate the possibility of local PM sources interfering with the PM measurements. SDAPCD should track the activity of these local fugitive dust

sources and evaluate whether they may be affecting the representativeness of the siting scale intended for the PM monitors at the Otay Mesa

- Donovan and Alpine sites.

LABORATORY RECOMMENDATION 1 : Laboratory staff responses indicated that the VOC laboratory does not have a sep arate air handling system. It was unknown wheth er there was positive pressure. Keeping VOC areas free of solvents and maintaining a separate air handling system be important in contamination

management. This may not present a problem and is pointed out to the laboratory as a suggestion. The laboratory may wish to evaluate background levels of contamination in the VOC laboratory. A logical time to do this may be at the time of purchase and installation of a new

PAMS instrument because the PAMS and TO

- 15 ins trumentation are housed in the same area.

LABORATORY RECOMMENDATION 2 : Duplicate and co

- located samples are being analyzed as a QA measure, which is a positive finding. Mak ing c ontrol chart s of the results would be a further enhancement. A s with all qua lity control results, it is of paramount importance that QC results are given attention by analyst and quality assurance staff when corrective action is needed, and that outliers are taken into account when data are evaluated against their intended use.

LABORATORY RECOMMENDATION 3 : Based on the analyst interview, holding times for DNPH cartridges are occasionally longer than

the recommended two weeks. It is recommended that the laboratory strive to meet the

recommended two week hold time

.

LABORATORY RECO MMENDATION 4 : The laboratory does not assign expiration dates to new sampling cartridges and allows cartridges to be used beyond the 90 days prescribed by the method. W here it discusses the preparation of DPNH cartridges

, Method TO

- 11 states

, “that cartrid ges will maintain their integrity for up to 90 days stored in refrigerated, capped shipping tubes” (TO

- 11 note, sec. 9.5.2.16). Initial blank lot concentrations are provided with commercially purchased cartridges. The auditor views the shelf

life as stable , but given the significant concern expressed throughout Method TO

- 11 over potential laboratory contamination, it is prudent to be concerned over potential contamination during storage. The level of contamination will depend on how the cartridges are store d and if they are exposed to contaminants. Therefore, it is likely best practice to recertify for QA documentation purposes after 90 days

, as suggested by guidance. The District indicated that the manufacturer of the cartridges, Waters, allows for a hold t ime of up to six months. It is

recommended that

, in addition to storing unused cartridges under refrigeration, the laboratory verify that cartridges meet blank certification requirements by analyzing a blank before using the

lot past the 90

- day PAMS season

. Alternatively, the District can follow up with Waters to determine the basis for the six month hold time, given studies that show contamination after 9041 days.

PM LABORATORY RECOMMENDATION 1 : The laboratory does not have a formalized system for archiving PM 2.5 filters. PM 2.5 filters are currently stored in a refrigerator located in the toxics laboratory . Currently, all filters are stored randomly, without any system in place to track the archived filters. While the laboratory

technician was able to locate specific filters on request, a formalized process would provide structure and create any easy and traceable way of storing filters. Establish procedures to

formalize the PM 2.5 filter archiving system.

QUALITY ASSURANCE RECOMMENDATION 1: Conducting data t racking audits on a routine basis may help identify issues throughout the data collection, verification and reporting process, and therefore, improve data quality.42 APPENDIX C: ORGANIZATIONAL CHART