17 Technical System Audit Report

South Coast Air Quality Management District

April 13

– April 16, 2010

Conducted by

Air Quality Analysis Office Quality Assurance Office

Air Divi sion Management and Technical Services Division

US EPA Region 9

75 Hawthorne Street San Francisco, California 94105

March 2011SCAQMD TSA Report: March 2011

2 CONTENTS

PAGE

GLOSSARY OF ACRONYMS

......................................................................... 3

EXECUTIVE SUMMARY

........................................... ..................................... 4

INTRODUCTION

............................................................................................ . 6

A. GENERAL PROGRAM

............................................................................... 8

B. NE TWORK MANAGEMENT

..................................................................... 14

C. FIELD OPERATIONS

.................................................................................. 17

D. LABORATORY OPERATIONS

.................................... ............................. 20

E. DATA AND DATA MANAGEMENT

........................................................ 22

F. QUALITY MANAGEMENT

........................................................................ 23

APPENDIX A: Summary of Findings

......... .............................. ............ 25

APPENDIX B: Additional Recommendations

...... ..................... ............. 27

APPENDIX C: Final Report of NATTS Program Review by RTI

...... ......... 27SCAQMD TSA Report: March 2011

3 GLOSSARY OF ACRONYMS

AQS...................................... Air Qu ality System AQIS..................................... Air Quality Instrument Specialist

BAM..................................... Beta Attenuation Monitor

CA........................................ California

CARB................................... Califor nia Air Resources Board

CO........................................ Carbon Monoxide

CFR...................................... Code of Federal Regulations

EMS............................ Electronic Management System

EPA................................... United States Envir onmental Protection Agency

FEM...................................... Federal Equivalent Method

FID........................................ Flame Ionization Detector

FRM...................................... Federal Reference Method

GC......................

.................. Gas Chromatograph (Gas Chromatography)

ICP

- MS.......................... Inductively Coupled Plasma

- Mass Spectroscopy

IO................................. Inorganic Compendium

MATES.......................... Multiple Air Toxics Exposure Study

MDL..................................... Method De tection Limit

MS........................................ Mass Spectrometer

MSD...................................... Mass Selective Detector NAAQS................................ National Ambient Air Quality Standard

NATTS................................

. National Air Toxics Trends Stations

NCore.................................. National Core multipollutant monitoring stations

NIST............................ National Institute for Standards and Technology NOx/NOy............................. Nitrogen Oxides

O 3

........ .................................. Ozone

ORD............................. Office of Research and Development

PAMS................................... Photochemical Assessment Monitoring Stations

Pb........................................ Lead

PE..............................

.......... Performance Evaluation

POC...................................... Parameter Occurrence Code

PM........................................ Particulate matter

PM 2.5

..................................... Particulate matter 2.5 microns or less in aerodyn amic diameter

PM 10

...................................... Particulate matter 10 microns or less in aerodynamic diameter PQAO................................... Primary Quality Assurance Organization QA........................................ Quality Assura nce QC........................................ Quality Control QAPP................................... Quality Assurance Project Plan QMP............................ Quality Management Plan

RTI....................................... Research Triangle Institute (EPA Contr actor) SCAQMD............................ South Coast Air Quality Management District

SPM..................................... Special Purpose Monitor

SOP...................................... Standard Operating Procedure

SLAMS.............................

... State or Local Air Monitoring Station

SO 2

....................................... Sulfur DioxideSCAQMD TSA Report: March 2011

4

SRP............................. Standard Reference Photometer

STN............................. Speciation Trends Network

STP....................................... Standard Temperature and P ressure TAD.............................. Technical Assistance Document

TEOM................................... Tapered Element Oscillating Microbalance

TO................................ Toxic Organic Compendium

TSA...................................... Technical System Audit

TSP................

....................... Total Suspended Particles

XRF...................................... X

- ray FluorescenceSCAQMD TSA Report: March 2011

5

EXECUTIVE SUMMARY

This report documents the findings made by the US Environmental Protection Agency (EPA) Region 9 during its Technical System Audit (TSA) of the South Coast Air Quality Management District (SCAQMD) ambient air monitoring program. A TSA is an on

- site review and inspection of a state or local ambient air monitoring program to assess its compliance with established regulations gove rning the collection, analysis, validation, and reporting of ambient air quality measurement data. This TSA meets the requirements for EPA Region 9 audits of the SCAQMD monitoring organization required by 40 CFR Part 58, Appendix A, Section 2.5.

The TSA was conducted by EPA Region 9 staff from April 13

– April 16, 2010. The audit team interviewed management and staff on specific aspects of the air monitoring program including network design, field operations, laboratory operations, data handling, quality assurance and quality control procedures. The audit team also inspected a number of monitoring sites and visited several additional monitoring sites. The site inspections consisted of interviewing the site operator, reviewing station and instrument logb ooks, and evaluating the station siting with respect to EPA requirements for probe siting 1

.

The TSA is one of the ways EPA oversees data collection activities performed by state and local agencies to ensure they meet certain minimum data quality objecti ves. Other assessments, such as network reviews and performance evaluations, are also used to collect information on the overall quality of ambient air monitoring data. These assessments enable agencies to identify and correct program elements which may be adversely affecting the quality

of ambient air data.

The results of the TSA are summarized below and discussed in more detail in the specific sections of this report

. Recommendations for addressing each of the findings are provided. The specific a ctions taken by SCAQMD to address the findings should be approved by EPA and documented by SCAQMD in a corrective action plan.

EPA would like to thank all the staff and management of SCAQMD for their support and cooperation during the audit.

Program S trengths:

Overall, SCAQMD staff was very professional and helpful.

SCAQMD site operators were very competent (e.g., the Rubidoux operator is exceptionally knowledgeable and documents station operations thoroughly and consistently.

1 40 CFR, Part 58, Appendix E, “Probe and Monitoring Path Siting Criteria for Ambient Air Quality Monitoring”SCAQMD TSA Report: March 2011

6

Since the last audit i n 2007, SCAQMD has initiated a process to put a Quality Management program in place.

Instrument down time has decreased significantly since the last audit. This is due in part to decreased repair and calibration response times.

The senior field technici ans have implemented systems to document instrument down

time on forms and to review results from the electronic data loggers that have improved data quality. Logbooks are well maintained at the monitoring sites.

The Particulate Matter (PM) 2.5 laboratory is well organized and the electronic data handling system for the information gathered in the laboratory is well designed and comprehensive.

Program Findings:

While acknowledging the strengths of SCAQMD’s monitoring program, EPA identified several pro gram elements that could be improved. Specifically:

While significant improvement has been made to put the QA system in place, the lack of completed, finalized, and approved QAPPs and SOPs is an outstanding concern.

Documentation produced by the monit oring, quality, and laboratory systems generally is good. However, several areas do not produce records that are fully reproducible and

defensible.

Several areas were identified where staffing and/or equipment limitations affected the quality of data bei ng produced. Additionally, there is a need to plan for a shift in staff to accommodate future changes to the monitoring program.

The laboratory has improved its quality assurance oversight program. However, this program needs to be refined further to en sure consistency in implementation of laboratory quality systems and procedures.

Quarterly data entry to AQS and annual data certification were not completed on time.

In addition to these significant findings, EPA has identified other priority findings and made good practice recommendations. The individual findings are reported in the topic sections of this document and are summarized in Appendix A and good practice recommendations are included in Appendix B.SCAQMD TSA Report: March 2011

7

INTRODUCTION

The SCAQMD is the governmen t agency to which EPA delegated the authority and responsibility for collecting ambient air quality data pursuant to the Clean Air Act of 1977 and

the Clean Air Act Amendments of 1990.

From April 13 through April 16, 2010, the Region 9 Air Quality Anal ysis Office and the Region 9 Quality Assurance Office conducted a Technical System Audit (TSA) of SCAQMD’s ambient air monitoring program. The TSA covered the following program areas: General / Quality Management

o Program Organization

o Facilities

o Independen t Quality Assurance and Quality Control

o Planning Documents (including QMP, QAPPs, & SOPs)

o General Documentation Policies

o Training

o Corrective Action

o Quality Improvement

o External Performance Audits Network Management / Field Operations

o Network Design

o Changes to the Network since the last audit

o Proposed changes to the Network

o Field Support

 SOPs

 Instrument Acceptance

 Calibration

 Repair

 Site and Monitor Information Form Laboratory Operations

o Routine Operations

o Quality Control

o Laboratory Preventive Maintenance

o La boratory Record Keeping

o Laboratory Data Acquisition and Handling

o Specific Pollutants: PM 10 , PM 2.5 , Toxics, PAMS, and Lead Data and Data Management

o Data Handling

o Software Documentation

o Data Validation and Correction

o Data Processing

o Internal Reporting

o Exter nal Reporting

EPA also tracked supporting documentation for data points/sets from calendar years 2008

and 2009.SCAQMD TSA Report: March 2011

8

Participating SCAQMD managers, supervisors and staff included:

Chung Liu, Deputy Executive Officer, Science & Technology Rudy Eden, Manag er, Laboratory Solomon Teffera, Pr. Chemist, Laboratory Steve Barbosa, Pr. Chemist, Laboratory Dan Houghton, Sr. Chemist, Laboratory

Tai

- Ching Hu, Sr. Chemist, Laboratory Sandra Hom, Sr. Chemist, Laboratory Wimol Chanjamsri, Chemist, Laboratory

Ken Sanchez , Chemist, Laboratory Yang Song, Chemist, Laboratory Jason Low, Manager, Quality Assurance Raul Dominguez, Sr. Chemist, Quality Assurance Mike Hamdan, Sr. Instrument Specialist, Quality Assurance Philip Fine, Manager, Monitoring
Rene Bermudez, Pr. Instrume nt Specialist, Monitoring David Sawyer, Pr. Instrument Specialist, Monitoring Summer Wilson, Pr. Instrument Specialist, Monitoring Albert Dietrich, Sr. Instrument Specialist, Monitoring Paul Chavez, Sr. Instrument Specialist, Monitoring Claudius Phipps, Sr . Instrument Specialist, Monitoring Donald Jacobs, Sr. Instrument Specialist, Monitoring Mike Koch, Sr. Instrument Specialist, Monitoring Keith Brown, Sr. Instrument Specialist, Monitoring Brian Vlasich, Sr. Instrument Specialist, Monitoring Jorge Diez, In strument Specialist, Monitoring David Carillo, Instrument Specialist, Monitoring Terry Corcoran, Instrument Specialist, Monitoring Brian Speaks, Instrument Specialist, Monitoring Norman Broellos, Instrument Specialist, Monitoring Tuong Mac, Instrument Spec ialist, Monitoring Robert Wimmer, Instrument Specialist, Monitoring Cirila Enriquez, Instrument Specialist, Monitoring Twyla Miner, Instrument Specialist, Monitoring Jeff Barenwald, Instrument Specialist, Monitoring Richard Rodgers, Instrument Specialist, Monitoring Leonard Edwards, Data Translator, Monitoring Hanna Zhuang, Data Translator, Monitoring Kevin Durkee, Sr. Meteorologist, Planning

This report is divided into eleven sections. The first section is an executive summary that describes the purpose of the TSA and a summary of the most significant findings. The followingSCAQMD TSA Report: March 2011

9

section provides a brief description of the SCAQMD air monitoring program, identifies participating SCAQMD personnel and the EPA audit team, and describes the report’s organization. The third section discusses the general/significant findings. The next five sections address specific aspects of the air monitoring program: network management, field operations, laboratory operations, data management and quality assurance/quality control . Appendix A lists the findings contained in this report. Appendix B includes minor concerns and recommendations for SCAQMD to consider. Appendix C includes final comments from a concurrent review of the SCAQMD NATTS program by EPA’s RTI contractor. Re commendations for each finding are provided to give some indication of EPA Region 9’s expectations as to how findings can be addressed. If the SCAQMD has other approaches or alternatives to address the concerns identified, the Region will consider them, p rovided the corrective action adequately addresses the finding.SCAQMD TSA Report: March 2011

10

A. GENERAL PROGRAM

Federal and State of California laws require that clean air standards be met and maintained throughout the country and the State of California. An ambient monitoring pro gram is essential to verifying compliance with these standards. The SCAQMD is defined in the California Health and Safety Code, Division 26, Air Resources, Section 40412 as the “Sole and exclusive agent having responsibility for air pollution control with in the District.” The State defines the geographic extent of SCAQMD as “portions of Counties of Los Angeles, Orange,
Riverside, and San Bernardino included within the area of the South Coast Air Basin, as

described in Section 60104 of the Title 17 of the California Administrative Code.” This area is home to over 16 million people, nearly half the population of the State of California, making it the second most populated urban area in the United States. The area covered by the SCAQMD has historically expe rienced some of the highest air pollution in country.

A Governing Board that adopts policies and regulations that promote clean air within the District’s jurisdiction oversees SCAQMD. The Governing Board has thirteen members, of

which ten are local ele cted officials and three are state appointed.

SCAQMD currently has 34 active monitoring stations for criteria pollutants. The pollutants measured include:

Carbon Monoxide
Nitrogen Dioxide

Sulfur Dioxide

Lead

Ozone

Particulate Matter (PM 10 and PM 2.5 )

T he Executive Officer of SCAQMD is, Barry Wallerstein. Chung Liu, Deputy Executive Officer, heads the Science & Technology Advancement Division

. T he Monitoring & Analysis

sub

- Division is the primary organization in Science & Technology Advancement respons ible for air monitoring. The Monitoring and Analysis group has three functional areas: Laboratory Services & Source Test Engineering (Rudy Eden, manager), Atmospheric Measurements (Philip Fine, manager), and Quality Assurance (Jason Low, manager).

Atmosp heric Measurements (Monitoring), Laboratory Services & Source Test Engineering (Laboratory), and Quality Assurance (QA) each have a role in the collection and evaluation of ambient air data as defined by the SCAQMD Quality Management Plan (QMP).

The Monit oring group is responsible for most of the ambient air data collection, including sampling and data processing of continuous air monitors. The Laboratory is responsible for preparation of sampling media, analysis of analytical samples from the non

- continu ous monitors, and processing of data for these samples analyzed by the laboratory. Both groups are responsible for implementing routine quality assurance and quality control (QA/QC) procedures. The QA group is responsible for tracking and oversight of tr aining, corrective actions, and data handling.SCAQMD TSA Report: March 2011

11

The QA group also implements performance and technical audits and coordinates and participates in the preparation of QA planning documents.

Overall, the monitoring progr am at SCAQMD is well executed, having taken a number of corrective actions since the last EPA TSA in 2007. Quality management functions are formally integrated into the program and in most areas are well documented. EPA noted an improvement in the quality and quantity of data SCAQMD entered into the AQS system over the past several years. However, there are still some significant issues that need to be addressed. The most significant findings are discussed below. Other priority findings follow in subsequent sections, while good practice recommendations are included as a courtesy in Appendix B.

FINDING A1:

SCAQMD’s Quality Assurance program documentation, including most Quality Assurance Project Plans (QAPPs) and Standard Operating Procedures (SOPs), was not complete and was not approved by EPA at the time of the audit.

Discussion: EPA requires that organizations develop a QAPP for each type of ambient pollutant being measured. The QAPP integrates all technical and quality aspects of a project, including planning, implementation, and as sessment. The purpose of the QAPP is to document planning results for environmental data operations and to provide a project

- specific “blueprint” for obtaining the type and quality of environmental data needed for a specific

decision or use.

The QAPP d ocuments the quality assurance and quality control that are applied to an environmental data operation to assure the results obtained are of the type and quality needed and expected. Further guidance on developing QAPPs can be found in the guidance docume nts “EPA Requirements for Quality Assurance Project Plans,”

EPA/240/B

- 01/003, March 2001, and “Guidance for Quality Assurance Project Plans,”

EPA/240/R

- 02/009, December 2002.

Most major SOPs were completed, tracked and finalized with internal review and signatures. SCAQMD had draft versions of quality assurance documentation available to staff and the auditors for many of the air monitoring programs and procedures. While these updated documents have not been approved by EPA , the fact that staff and

manag ement worked together on SOP development helps to ensure consistent and complete implementation of the monitoring program. There were some SOPs that were not updated or completed , for example:

An SOP reflecting current laboratory practice was not availab le to the auditors. According to initial staff responses while on

- site, significant deviations to the TO

15 method had been made, and because there was no SOP available, it was

difficult to evaluate the effects of the deviations.SCAQMD TSA Report: March 2011

12

Recommenda tion: SCAQMD should work with EPA Region 9 to develop approved QA planning

documents.

FINDING A2:

In several operational areas, official records were not available or were stored in a manner that did not ensure their reproducibility.

Discussion:

EPA R egion 9 Laboratory Documentation Required for Data Evaluation , August 2001

notes that:

In order for data to be used for decision

- making purposes, it is essential that it be of known and documented quality. Verification and validation of data requires

tha t appropriate quality assurance and quality control (QA/QC) procedures be followed, and that adequate documentation be included for all data generated both in the laboratory and in the field.

Most air monitoring and support tasks included sufficient, comp lete documentation such that the work performed could be verified and validated. However, reviews of documentation in several areas highlighted the need for a consistent policy to produce data that support analytical results. Specifically in the QA Audit Laboratory and Monitoring Calibration Laboratory, EPA found that:

Not all records were readily accessible

Electronic records are not maintained in a manner that prevents

modification There is no system in place to back up some electronic records

Some f or ms are not consistent for different calibrations and some lack space to document pertinent information Required information is not always recorded Recording errors were found that should have been corr ected by the primary technician , by the peer reviewer, or during management review These deficiencies in organizationally separate areas of SCAQMD indicate the need to evaluate all monitoring program documentation procedures.

Recommendation: SCAQMD should review documentation practices in all oper ational areas, develop a QA policy for producing and archiving documentation, and implement improvements.

FINDING A3:

Limited staffing in several key areas is significantly impacting monitoring program performance and quality.SCAQMD TSA Report: March 2011

13

Discussion:

EPA obser ved deficiencies in data validation and reporting (Finding A5), QA audits (Finding F1), QA management (Findings A1 & A3), and laboratory operations (D5) that are directly related to the lack of sufficient resources. Additionally, it was observed that:

Th e PAMS Department appeared to be close to its capacity to handle the sample load as indicated by the fact that there was not enough time to analyze sample duplicates as a quality control measure during the PAMS season due to the quantity of PAMS samples ta ken in 2009. During the Laboratory TSA debrief meeting, SCAQMD management concurred that the PAMS department was nearing capacity. New monitoring requirements may necessitate a shift in how the resources within the monitoring, lab and QA groups are alloca ted. The QA Manager has a small staff, which limits his ability to shift staff to respond to current needs (e.g., ensure field audits are conducted at required frequency and are appropriately documented and ensuring validation has sufficient staff and

reso urces).

Capacity issues arising fr om staffing shortages can compromise quality; over scheduling staff time can increase opportunities for introducing error.

Recommendation: SCAQMD should evaluate current and long

- term staffing needs t o ensure that core tasks are being accomplished in a timely manner and with reliable and appropriate quality.

FINDING A4:

The QA s taff o versight of f ixed l aboratory needs to be strengthened.

Discussion: EPA noted the following issues related to the Q A oversight of the data quality from the laboratory: The laboratory must finalize Standard Operating Procedures (SOPs) , including EPA approval

. The final SOPs should incorporate a regular schedule for which the QA group performs activities to ensure ongoi ng data quality. These activities include, but are not limited to, performing internal QA audits, running PE samples and MDL studies, participating in inter

- laboratory comparisons, and conducting and documenting reviews of logbooks and control charts.

M ore frequent and comprehensive internal laboratory audits should be performed. Ideally, these audits should be performed annually or more frequently. In order to improve oversight, QA staff should become more familiar with on

going laboratory operations. In addition, SCAQMD should place special emphasis on facilitating the process by which quality assurance findings and

corrective actions are tracked. Adding method acceptance limits or control limits to calibration control process charts or charting d uplicate sample precision results can help staff review qualitySCAQMD TSA Report: March 2011

14 control results more easily. The stated lack of confidence by QA staff as to the reliability of TO

- 15 standards purchased by the laboratory needs to be addressed.

Validation of laboratory P M data should be performed by staff not performing sample collection and/or anal ysis. For more information see the QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring Program, December 2008 (Section 17.2 “Effectiv e data validation procedures usually are handled completely independently from the procedures of initial data collection.”)

Recommendation: The Laboratory and QA group need to work together on methods to promote consistency and a robust, ind ependent QA program for the fixed laboratory.

FINDING A5:

Quarterly data entry to AQS for filter

- based PM 2.5 and PM 10 and annual data certification were not completed on time.

Discussion: EPA requires that data must be received by AQS 90 days after the end of each quarterly reporting period (40 CFR, Part 58.16) and that this data be certified by May 1 of each year starting in 2010 (40 CFR, Part 58.15). SCAQMD significantly exceeded these requirements. This limited EPA’s ability to evaluate and as sess data collected by the SCAQMD network.

Recommendation: SCAQMD needs to meet the data submission and certification schedules required by

EPA. This could be aided by the addition of a staff person whose responsibility is to

ensure these deadlines are met.SCAQMD TSA Report: March 2011

15

B. NETWORK MANAGEMENT

The ambient air monitoring network in the South Coast AQMD currently consists of 34 State/Local Air Monitoring Stations (SLAMS) plus several source oriented lead monitors, special studies and Special Purpose Monitoring (SPM) sites that utilize a variety of air pollutant measuring instruments. The following table summarizes the SLAMS sites in the SCAQMD.SCAQMD TSA Report: March 2011

16

Table 1: SLAMS sites in the SCAQMD

Site O 3 CO NO
x SO 2 FRM

PM 10 BAM

PM 10 TEOM

PM 10 FRM

PM 2.5 BAM

PM 2.5 Pb

Anaheim X X X X X X X

Azusa X X X X X
Banning Airport X X X
Big Bear X

Burbank X X X X X X X X
Compton (Replaced Lynwood) X X X X X

Costa Mesa X X X X

Crestline X X X

Fontana X X X X X X

Glendora X X X X

Indio X X X X

, X

La Habra X X X

LA

- Main St X X X X X X X

, X X

Lake Elsinor X X X X
LAX Hastings X X X X X X
Long Beach

- North X X X X X X X X X

Mira Loma

- Van Buren (replacing Jurupa) X X X X X X

Mission Viejo X X X X

Norco X

Ontario Fire Station X X
Palm Springs X X X X X X

Pasadena X X X X

Perris X X

Pico Rivera #2 X X X X X

Pomona X X X

Redlands X X

Reseda X X X X

Riverside X X X X

Rubidoux X X X X X X X

, X X X

San Bernadino X X X X X X X

Santa Clarita X X X X
South Long Beach X X X
Upland X X X X X

West LA X X X Source: SCAQMD 2009 Air Monitoring Network Plan17

SCAQMD also operates the following s pecial purpose monitoring sites (SPM) and source

oriented lead sites: Port study sites at the Ports of Los Angeles and Long Beach Pb at Exide: Rehrig in Vernon, and ATSF in City of Commerce Pb at Quemetco (Closet World) in City of Industry Pb at Trojan Bat tery (Uddelholm) in Santa Fe Springs Pb at Van Nuys Airport

In addition, the District operates a network of air toxics stations, including a National Air Toxics Trends Stations (NATTS) at the Los Angeles (Main) and Rubidoux sites. This audit

addressed air toxics activities that support the NATTS, including both field and laboratory operations. These two sites are also designated as National Core (NCore) multipollutant monitoring sites.

The District operates a network of Photochemical Assessment Moni toring Stations (PAMS). The PAMS network consists of seven sites. LAX/Hastings is a Type 1 (upwind

, background , and transport) PAMS site; Azusa, Burbank, Los Angeles (Main), and Pico Rivera are Type 2 (maximum precursor emission/central business district) sites; Rubidoux and Santa Clarita are Type 3 (maximum ozone concentration) sites. EPA has approved the District’s PAMS network, operating schedule and forecasting scheme.

The SCAQMD network currently meets or exceeds the requirements for the minimum

num ber of monitoring sites designated as SLAMS for all of the criteria pollutants. SCAQMD managers exhibit broad knowledge of air quality issues within the District and a thorough understanding of current and upcoming EPA network requirements and recommendati ons. As a result, SCAQMD has very effective network planning. In addition, most monitoring sites are properly located with respect to monitoring objectives, spatial scales and other siting criteria.

40 CFR 58.20 requires States to perform an annual re view of their existing ambient monitoring networks and to provide information about the monitoring stations, including the information required by 40 CFR Part 58, Appendix A (quality assurance requirements) and Appendix D (network requirements and monitori ng site objectives). Based on changes to the monitoring regulations, which went into effect in December 2006, the annual network plan now requires evidence of a public inspection process. The District submits an annual monitoring network plan and the most recent submittal, in 2009, includes all of the required information about the monitoring stations. The current SCAQMD document meets the requirements to describe the network and the intended monitoring objectives of each site.

In addition to the network plan, 40 CFR 58.10 (d) states “The State, or where applicable local, agency shall perform and submit to the EPA Regional Administrator an assessment of the air quality surveillance system every 5 years to determine, at a minimum, if the network meets

the monitoring objectives defined in appendix D to this part, whether new sites are needed, whether existing sites are no longer needed and can be terminated, and whether new technologies are appropriate for incorporation into the ambient air monitoring networ k.”SCAQMD TSA Report: August 2010

18

Network Management

FINDING B1:

PM 10 data at the Indio monitoring site may be affected by middle scale sources

.

Discussion:

Currently, the PM 10 data from the Indio monitoring site is characterized as representing neighborhood scale monitoring. 4 0 CFR 58 Appendix D states that, for PM 10 neighborhood scale monitoring measurements should “represent conditions throughout some reasonably homogeneous urban sub

- region with dimensions of a few kilometers.” The CFR clarifies that “homogeneity refers to th e particulate matter concentrations, as well as land use and land surface characteristics.” Upon visiting this site, it was discovered that it is located in very close proximity to an unpaved vacant lot, which is actively used for vehicle parking. Due to its close proximity to a known source of PM 10 , it is reasonable to assume that the Indio monitoring site is representative of an area

smaller than neighborhood scale monitoring.

Recommendation: The spatial scale of the PM 10 monitoring at the I ndio site should be changed to middle scale or specific actions should be taken to mitigate fugitive dust from the vacant lot to minimize the impact of local sources on the monitoring data.

FINDING B2:

Exide

- Rehrig should be a collocated lead (Pb) monito ring site.

Discussion: As specified in 40 CFR part 58, Appendix A, section 3.3.4.3, the first collocated Pb site must be the site measuring the highest concentrations in the network. In the SCAQMD Pb network, the highest site is now (based upon 2009 d ata) Exide

- Rehrig. There are currently three monitors at Exide Rehrig, each operating on a staggered 1:3 schedule.

The three monitor s report to a single Parameter O ccurrence Codes (POC), and collectively have an every

- day sampling schedule. Distinct POC s for each monitor are strongly recommended. A combined site report may still be pulled . This report would sum the data from all monitors at the site, while the information as to which data are

associated with each monitor would still be retained. (Also see Finding C2)

Recommendation s : 1. In order to meet the collocation requirement, SCAQMD needs a collocated

monitor at Exide

- Rehrig.

2. SCAQMD should h ave distinct POCs for each monitor.SCAQMD TSA Report: August 2010

19

FINDING B3:

Placement of PM monitors at Anaheim does not meet co ll o cation requirements.

Discussion: At the site visit to Anaheim, EPA noted that there are two reference method samplers for

both PM 10 and PM 2.5 , one filter

- based and one continuous for each method at the site. The distance between each of these pairs ex ceed the four meter horizontal and one meter vertical requirement for collocated monitoring. For PM 2.5 there are network collocation requirements for filter

- based FRM and continuous FEM. If both of these PM 2.5 samplers are to remain at this site, they sh ould be moved to within one to four meters of each other and mounted at a similar height so they can be considered to be formally collocated. There is no collocation requirement for PM 10 filter

- based and continuous instruments,

however it is still recomme nded that, if practical, these samplers be moved to within two to four meters horizontally and one meter vertically of each other.

Recommendation:

Confirm whether or not the PM 2.5 samplers are intended to meet the collocation requirements. EPA suggests t hat all monitors measuring the same pollutant meet these distance parameters in order to produce comparable data per the collocation requirements.

FINDING B4:

SCAQMD does not routinely notify EPA about collecting data from SPMs and sometimes does

not ent er that data into AQS.

Discussion: The SCAQMD operates many temporary monitors as part of short

- term studies or to address public concerns. EPA fully supports SCAQMD’s responsiveness to address community concerns or other issues, however, any data thus collected that meets the QA requirements should be entered into AQS. This is consistent with 40 CFR Part 58.20(b)

that states:

Data collected at an SPM using a FRM, FEM or ARM meeting the requirements of Appendix A must be submitted to AQS according to the requirements of §58.30

.

It is also useful for EPA to be aware of all the data being collected by SCAQMD to remain informed about air quality in the basin

.

Recommendation: SCAQMD should:

1. I nclude a list of major projects that are ongoing or have been completed in the previous year in the Annual Network Plan. This list would include the start date of sampling, what pollutants are measured, the sampling methods being used (or FRM/FEM status), the data objective and expected duration

.SCAQMD TSA Report: August 2010

20 2. P ut the data from the site into AQS, beginning with the start date, if a study is extended and lasts over one year.

3. EPA may also request an annual updated list of all ongoing monitoring projects. This list would be the same as the list in the network plan, but would includ e all special monitoring projects.

FINDING B5:

The SCAQMD has a dense monitoring network that may include redundant or low

- value sites.

Discussion: The SCAQMD operates one of the most extensive networks in the US. While a dense network can provide impo rtant spatial detail, it is expensive to maintain and does not always provide necessary information. The SCAQMD recently submitted a 5

- year Network Assessment that identified sites that could be considered for closure.

Recommendation:

Seek to close sit es that have been identified as redundant or low

- value.SCAQMD TSA Report: August 2010

21

C. FIELD OPERATIONS

Field operations are performed by the Atmospheric Measurements Division, which is managed by Philip Fine. This division is divided into two groups: Ambient Monitoring and Spec ial Monitoring. Routine ambient monitoring is conducted by the Ambient Monitoring Group. The Special Monitoring Group undertakes special projects and seasonal monitoring as needed. The Ambient Monitoring group is subdivided into Operations, Support, and Data Management sections. Day

- to

- day monitoring station operations are performed by the Operations section, which is broken into sub

- sections. Each sub

- section has at least one Senior Air Quality Instrument Specialist (AQIS) and five AQISs. Repairs and calibrations are performed by the AQISs in the Support section.

As part of this TSA, Operations section AQISs were asked about their duties related to monitoring station operation. Additionally, AQISs in the support section were interviewed about

aspect s of the instrument repair and calibration processes. All AQISs interviewed demonstrated a thorough knowledge of the monitoring and calibration equipment for which they were responsible.

The Field Sites reviewed included Anaheim, Exide, Fontana, Indio, Lake Elsinore, Los Angeles North Main, Mission Viejo, Palm Springs, Pasadena, Perris, Pico Rivera, Pomona, Rubidoux, and Van Nuys. EPA evaluation at these sites included inspection of the inlet manifolds, examination of station and instrument log books, and a specific check of whether appropriate Quality Control (QC) checks and Quality Assurance (QA) audits were being performed for dates in 2008 and 2009 that had been selected for review prior to the audit. Each of the sites visited was well organized. In strument logbooks were up to date and contained relevant information on operations and maintenance activities. Most sites met the siting criteria of 40 CFR 58, Appendix E.

Field Operations

FINDING C 1 :

One

- point flow rate verifications were not being performed on continuous PM 10 instruments throughout the SCAQMD monitoring network until approximately August of 2008.

Discussion: 40 CFR 58, Appendix A 3.2.3 states that “a one

- point flow rate verification must be performed at least once every month on ea ch automated analyzer used to measure PM 10 ”. The BAM 1020 and the Thermo TEOM instruments are subject to this requirement. On visits to various monitoring sites, it was discovered that these checks were not being performed until mid 2008. The QA Handbook V olume II, Appendix D lists this requirement as a “critical criterion” in the Continuous PM 10 Standard Temperature and Pressure (STP) Conditions Validation Template and is a very important measure of the performance of the instrument. These checks are used to not only assess the bias of the monitoring data, but also serve as a critical indicator of potential instrument malfunctionSCAQMD TSA Report: August 2010

22 or other data quality issues. Failure to perform these checks could compromise the quality

of the PM 10 data being measured.

Recommendation: Though SCAQMD has already corrected this issue and has been performing these checks post August 2008, it is recommended that SCAQMD assure these checks are performed

in the future.

FINDING C 2 :

Collocated filter based PM 10 data a re not currently submitted to AQS under an independent

monitor POC.

Discussion: Data from multiple filter

- based PM 10 samplers are being reported to AQS under one monitor and POC. POCs indicate how may monitors are operating at a site for a given pollutan t. For collocated PM 10 monitoring, 40 CFR 58 Appendix A 3.3.1.3 states that “for each pair of collocated samplers, designate one sampler as the primary sampler whose samples will be used to report air quality for the site, and designate the other as the au dit sampler.” The CFR also directs state or local agencies to “report the measurements from both samplers at each collocated sampling site.” Typically, the primary sampler is designated as POC 1, while the collocated sampler is designated as POC 2. All oth er samplers should be designated as POC 3, 4, or 5 as needed. These practices ensure that the data in AQS accurately represents what is physically being monitored at a given

location.

Recommendation: SCAQMD should designate collocated and prim ary samplers appropriately and report data to AQS from each PM 10 sampler under a separate POC.

FINDING C3:

The inlet at the Pomona station does not meet the 40 CFR 58, Appendix E siting criteria for O 3 and NO 2 monitoring.

Discussion: The gaseous inlet probe at the Pomona monitoring station is approximately 6.5m from the roadway (Garey Avenue), but the 2008 data year network plan indicates that it is 50m away and that the vehicle counts are 10,000 vehicles/day. 40 CFR 58, Appendix E,

section 6.1 states that for a station approved before December 18, 2006, a probe for monitoring ozone and oxides of nitrogen should be at least 10 m from a road with traffic of 10,000 vehicles per day.

Recommendation: SCAQMD should confirm the correct traffic counts for Gar ey Ave. and determine the minimum distance between the road and the probe to meet the siting requirements ofSCAQMD TSA Report: August 2010

23 CFR 58, Appendix E, section 6.1, and correct the information in the next network plan. The District should consider moving the inlet at this site t o meet the siting requirements, move the site or discontinue monitoring O 3 and NO 2 at this location.SCAQMD TSA Report: August 2010

24

D. LABORATORY OPERATIONS

Analytical laboratories provide support for measurement methods that are either too complex or too sensitive to perform in the fi eld environment. In order to provide these services, laboratories employ advanced instrumentation and staff with highly specialized training.

For ambient air samples to provide useful information or evidence, laboratory analyses must meet the following basic requirements:

1. The laboratory must maintain a suitable facility for sample receipt, storage, analysis, and reduction and storage of data. 2. The laboratory must have sufficient and appropriate equipment that must be calibrated and

mainta ined frequently. 3. The laboratory must have an adequate number of qualified staff. 4. Analytical procedures must be in accordance with official guidance, EPA methods and accepted practice. 5. Complete and accurate records must be kept.

SCAQMD has a clean, modern, expansive laboratory facility located at the headquarters office in Diamond Bar, California. The laboratory supports many chemical analyses necessary to understand a large complex air basin and its diverse source mi x. One of the primary responsibilities of the laboratory is the handling of PM filters, which includes preparation, weighing, tracking and storing PM 2.5 , PM 10 and total suspended particle (TSP) filters. In addition to PM responsibilities, the laboratory is also responsible for sample analyses of Speciation Trends Network (STN), PAMS, NATTS and special projects run by SCAQMD, such

as the Multiple Air Toxics Exposure Study (MATES) series. The compounds SCAQMD routinely analyzes for include:

- VOCs for PAM S (TO

- 14)

- VOCs for NATTS (TO

- 15)

- Carbonyls for PAMS and NATTS (Toxic Organic Compendium [TO]

- 11A)

- PM 2.5 metals for STN by X

- ray Fluorescence (XRF) (Inorganic Compendium [IO]

- 3.3)

- PM 10 metals for NATTS by Inductively Coupled Plasma

- Mass Spectros copy (ICP

- MS) (IO

- 3.5)

- Pb

- TSP by XRF

- Acrolein for NATTS (modified TO

- 15)

- Hexavalent chrome for NATTS by ion chromatography (modified California Air Resources Board (CARB) method)

- Anions for STN

There are a number of additional activities the laboratory undertakes to support the collection and analysis of air pollutants. These include canister cleaning and preparation, data validation and sample storage.SCAQMD TSA Report: August 2010

25

Laboratory Operations

FINDING D1:

The cleaning process of canisters appeared to be at maximum capacity, which falls outside guidance and good laboratory practice.

Discussion: Laboratory staff stated that the length of time that canisters are subjected to vacuum and heating during the cleaning process varies depending on the deman d for canisters. When demand for canisters is high, the cleaning time is shortened to the point where canisters begin to fail the certification process for cleanliness. At that point, the process is manipulated until the shortest time necessary to pass c ertification is achieved. (Technical Assistance Document [TAD], Sec. 2.5.2.2).

The cleaning process described by staff can lead to undetected, residual carryover contamination in canisters. Cleaning canisters for 10

- 12 minutes is less than the 30

minu tes prescribed in the guidance, and balancing the cleaning time as finely as described against passing certification versus using a fixed cleaning time raises the potential for residual contamination. Carryover contamination from previous samples may go

u ndetected because only a percentage of the canisters cleaned in every batch is selected for certification analysis. This indicates a need for additional canisters, cleaning apparatus or staff time.

Recommendation: Evaluate the Laboratory’s capacity to meet the demand for clean canisters against the guidance and add whatever combination of new canisters, new cleaning apparatus, and

staff is needed to achieve it.

FINDING D 2 :

Deficiencies observed in an external TO

- 15 performance audit sample have not be en satisfactorily resolved.

Discussion: SCAQMD Quality Assurance staff attributed deficiencies observed in analysis of an

external TO

- 15 performance audit sample to problems with the accuracy of concentrations of target compounds provided by the vendor f or calibration gas standards. Staff further expressed a general lack of confidence in the reliability of the gas standards provided by the vendor. Use of calibration gas standards forms the basis for determining

the concentration levels of contaminants r eported in samples. Their accuracy must be

assured.

Several options are available for addressing the reliability of the gas calibration standards. They range from verifying gas cylinder concentrations are within a "tolerance range" when analyzed against a second source gas standard to purchasing customizedSCAQMD TSA Report: August 2010

26 TO

- 14 gas calibration standards containing most TO

- 15 compounds directly from the National Institute of Standards and Technology (NIST), where the target compound

concentrations have been established w ith accuracy and have been certified through a rigorous testing protocol. NIST will also analyze commercially purchased gas standards and provide a certification of the concentration of each target compound for a fee.

Also, the California Air Resources Board (CARB) is available to analyze one sample obtained from a cylinder or the cylinder itself against a certified NIST cylinder that CARB routinely uses for TO

- 15 analyses and provide the results to SCAQMD at no charge. CARB will not provide certified concentrations for the cylinder analysis since it is based on one run. This analysis is similar to a high quality second source analysis where results fall within a tolerance range, but accurate concentrations have not been established for each compound.

Because research by CARB and NIST has shown that stability is cylinder specific and that degradation occurs for some compounds, an established verification schedule is required. To perform these verifications, purchasing a certified cylinder directly fr om NIST and purchasing a NIST certification of a commercially purchased gas standard cylinder are preferred options.

Recommendations:

1. The laboratory should take prompt measures to assure the accuracy of target compound concent rations in the gas standard cylinders used for testing equipment

calibration.

2. The laboratory should outline the frequency for verifying continued stability with an independent standard or re

- certifying TO

- 15 gas standard concentrations in its QAPP and SO Ps.

FINDING D 3 :

The QA measures used to control PAMS analyses are generally less rigorous than specified in guidance and by good laboratory practices.

Discussion: Internal standards and calculated relative retention times are not being used as part of the target peak compound identification process as prescribed in the PAMS guidance. In addition, quality control measures supporting proper peak identification and quantitation, such as periodic sample confirmation of a percentage of analytical results by Gas Chromatography/Mass Spectrometry (GC/MS), are not being performed as per the guidance. Instead, they are performed sporadically. Additionally, while duplicate analyses are being performed, they were performed outside the 2009 PAMS season (TAD, Sec. 2 .3.7).

Although gas chromatographic systems are generally stable, variations in conditions can and do occur that can shift absolute retention times. This can result in improper or missed compound identification. The use of internal standards with calc ulated relativeSCAQMD TSA Report: August 2010

27 retention times would address this concern and generally facilitate the target compound identification process. This seems especially important considering the complex chromatography observed with PAMS analysis.

In discussing the chroma tography process, staff stated they are able to identify target compounds in calibration standards by studying the profile of peak intensities relative to each other in the chromatogram. This type of data reduction process increases the potential for erro r when identifying sample peaks where published or established profiles are not used for comparison.

When instruments are deployed in the field, PAMS guidance suggests that changes in ambient conditions can cause changes in gas flow rates, column temper ature, detector response, sample injection volumes and sample moisture, and that internal standards help to compensate for system variability. Sec. 7.3 of the laboratory's TO

- 15 SOP states that staff have found that the most common problem with that metho d is a retention time shift caused by oven temperature control, column flow and concentration operation.

Although the PAMS Gas Chromatograph Flame Ionization Detector (GC FID) method can provide accurate results, upgrading the system configuration to a dual column

- dual detector FID and Mass Selective Detector (MSD) or similar configuration would yield significant benefits. Quantitation could still be performed by calibrating the FID detector with propane while the mass spectrometer would be used for i dentification. Such a system configuration may involve a natural evolution for PAMS analysis and is already being used in some commercial laboratories.

Incorporation of an MSD into the system configuration would greatly facilitate target and

non

- target contaminant peak identification, and significantly reduce the potential for

human error. The process would provide a higher level of confidence for contaminant identification. The use of a mass spectrometer detector would address the quantitation proble ms with manual editing and integration of co

- eluting peaks as described by the laboratory and confirmed by the PAMS PE study sample.

Recommendation s : 1. At a minimum the laboratory should transition to the use of internal standards for

all instrumentation as prescribed in the PAMS method. GC/MS confirmation should also be performed on a percentage of samples according to guidance.

2. GC/MS should be used to resolve issues with samples containing co

- eluting compounds in place of the current practice of manua l integration of unresolved gas chromatographic peaks. The laboratory is strongly encouraged to investigate and consider upgrading its system configuration to one that includes an MSD for compound identification.

FINDING D 4 :

The primary TO

- 15 analyst s tated that a method detection limit (MDL) study has not been performed for many years due to time constraints.SCAQMD TSA Report: August 2010

28 Discussion: According to method TO

- 15, MDL studies should be performed according to CFR Part 136, part B. MDL studies should be performed at a minimum annually, and any time a significant change is made or maintenance is performed that might affect instrument sensitivity. MDL studies are needed any time data are reported with non detect results.

Recommendation: MDL studies should be performed on a regular schedule, at a minimum annually, and any time significant changes are made to the instrument.

FINDING D 5 :

The laboratory's process for certifying a batch of eight cleaned PAMS canisters as clean based on selecting three canisters for certi fication analysis and allowing one to fail before the batch is rejected is outside guidance and common laboratory practice.

Discussion: Technical guidance calls for one out of every batch of eight canisters cleaned on a cleaning apparatus at the same time to meet analysis certification criteria. Without appropriate documentation, it is uncertain if the laboratory's procedure yields an equivalent level of confidence in canister cleanliness. (TAD, Sec. 2.5.2.3)

According to guidance, 100% of new cani sters should undergo the blanking and certification process. Following initial certification, one out of eight canisters cleaned as a batch should be tested for contamination. If the canister tested meets certification criteria, the batch may be certifie d as clean. Since the laboratory's practice deviates from guidance, the rationale and demonstration of equivalency should be documented. Please contact Region 9 for assistance with the statistical evaluation, if needed.

Recommendation: The laboratory should provide the rationale and documentation to demonstrate equivalency to the process described in the PAMS TAD guidance.

FINDING D 6 :

There was no evidence of consistent secondary review of instrument run logbooks. This was also observed during the previous TSA.

Discussion: Periodic review of instrument logbooks ensures that proper analysis protocols are being followed for calibration, blanks, duplicates and other quality control procedures. Repeated failures or attempts to analyze calibration s noted in logbooks can be an indication that instrument maintenance or other corrective action needs to be performed.SCAQMD TSA Report: August 2010

29 Initials or signatures of the reviewing official should be included to document that these

reviews occurred.

Recommendation:

It is rec ommended that the lead chemist or supervisory staff regularly review instrument run logbook entries and document their review by signing or initialing and dating the logbook. The QA group should also periodically spot

- check these logbooks.

FINDING D 7 :

The method being used for analysis of Pb for NAAQS compliance is a modification of an existing Federal Equivalent Method (FEM).

Discussion: Approximately five years ago, SCAQMD upgraded the method being used for analysis of

Pb on TSP filters. The meth od is based on an existing FEM, and has been verified with performance evaluation samples. However, it was not clear whether the method modifications represent a significant change to the FEM. In order to clarify the issue of method equivalency, SCAQMD p lans to share results from filter strip audits with EPA and have filters analyzed by an independent lab following a different FEM. SCAQMD has indicated they are planning to use a different analyzer for Pb in the future and may apply for FEM status.

Reco mmendation: Perform an evaluation of the current Pb method to validate consistency with an FEM for data generated from this method. SCAQMD should either apply for FEM status for current method or change to one of the newly designated Pb FEM methods.

FIND ING D 8 :

The sampling systems for collection of PAMS and Toxics sample are not routinely certified using gaseous standards

.

Discussion:

The Technical Assistance Document for Sampling and Analysis of Ozone Precursors

, September 1998, Section 2.5.1.7 speci fies the procedure that should be followed to certify samplers. It is recommended that certification occur one to three times per year for each sampler. SCAQMD is currently blank

- certifying these samplers

.

Recommendation: SCAQMD should develop a proced ure, schedule, and criteria for certification of PAMS and Toxics sampling equipment with gaseous standards at a frequency of at least once a

year.SCAQMD TSA Report: August 2010

30

FINDING D 9 :

The monitoring field technician was not aware of storage and handling requirements of carbon yl sampling cartridges in the field.

Discussion: The field technician was not aware of any criteria for storage temperature or time for the carbonyl media prior to use. Additionally, no expiration data was found on the carbonyl media. All sampling med ia should include an expiration date and handling instructions to ensure that they are not used for sampling beyond their expiration date. Method TO

- 11A specifies a 90

- day certification period for sampling media prepared by analytical

laboratories. It is possible that the vendor may have tested the media and can justify recommending a storage period longer than 90 days.

Recommendation: SCAQMD should develop and inform the field technician of handing procedures for carbonyl media.

FINDING D1 0 :

The proce dure to test PM 2.5 lot blanks is inconsistent with EPA guidance.

Discussion:

In EPA Quality Assurance Guidance Document 2.12 , 1998, Section 7.7 it is specified that

three lot blanks be tested from each lot of PM 2.5 filters and periodically re

- weighed.

S CAQMD currently uses lot blanks from the original filter lot as “exposure blanks” instead of filters from the current filter lot. While these blanks yield similar information, they are not as representative as using blanks from the current lot.

Recommend ation: SCAQMD should use lot blanks that are from the current filter lot.SCAQMD TSA Report: August 2010

31

E. DATA MANAGEMENT

SCAQMD has defined procedures for handling data from the time of acquisition to the time that it is submitted to EPA. The procedures are well known to the prin cipal data providers and reviewers. Responsibility for managing ambient monitoring data is divided between the Atmospheric Measurements Branch operators, data validators, and Senior and Principal Air Quality Instrument Specialists (AQIS).

The Quality A ssurance Manager reports results of QA audits and evaluations to a Quality Assurance Alert Log, AQS Event Flag Summary, and Corrective Action Request Log. These reports are incorporated into the review processes of Atmospheric Measurements and the

Laborat ory.

EPA identified four separate data management and validation processes in place for different data streams reported to AQS by SCAQMD. Continuous monitoring data is handled by Atmospheric Measurements. The laboratory has three separate data streams : 1) filter

- based

PM 2.5 ; 2) filter

- based PM 10 , TSP, Pb, and non

- NAAQS metals; and 3) organic analyses.

Continuous Monitoring for Gaseous Pollutants and Particulates

Air quality data measured by the continuous analyzers at the field stations

are stored i n SCAQMD data loggers and station computers. Each station is polled

minute

- by

- minute and the data are transmitted directly to the District’s central computer. Electronic chart recorders (Eurotherm Chessels) located at each station

serve as a back

- up syst em and provide a supplemental record for the data validation. Data may also be transferred manually using various devices, including laptop computers and flash drives.

SCAQMD performs four levels of validation for continuous data. The station data logg ers and the FORTRAN

- based data system perform automated checks. Field Operation staff review data and recommend flags. Data validators review quality control parameters. Data validators then evaluate data in relation to concurrent, corresponding data se ts.

Laboratory Data Systems: Filter

- Based PM 2.5 Particulate Monitoring; Monitoring of Lead

and non

- NAAQS Metals; and Monitoring of Organic Compounds

PM 2.5 filters are prepared and pre

- weighed by the filter laboratory and

transferred to the field techni cians. The laboratory also prepares other sampling media (canisters, filters, and sorbents) to be collected by the field technicians, with the exception of PAMS auto GCs, which are operated in the field by the laboratory staff. The field group performs a review of the field handling, sampler performance, and other field collection information. When filters are processed at the laboratory, the filter laboratory staff performs a review of the field operational parameters, including electronic records produ ced by the samplers. TheSCAQMD TSA Report: August 2010

32 laboratory reviews the Quality Assurance Alert Log, AQS Event Flag Summary, and Corrective Action request log for items specific to the data being processed. Data is appropriately flagged or invalidated and a Senior Laboratory Ch emist performs the final levels of data review prior to uploading data to AQS.

FINDING E1:

For continuous monitoring data, SCAQMD uses a limited number of validations flags that may not reflect the most appropriate AQS flags.

Discussion:

Numerous fl ags are available in AQS to invalidate or qualify data. SCAQMD is not using most of these flags because the FORTRAN

- based data management system SCAQMD uses for continuous monitors is only programmed for up to six different flags. This results in many da ta points being coded using the “miscellaneous void” flag that is not very informative. EPA and others evaluate data submitted to AQS for quality purposes and/or when taking air quality actions. EPA relies on descriptive and appropriately coded informati on for making these decisions.

Recommendation: SCAQMD needs to ensure that appropriate data flags are submitted to AQS.

FINDING E 2 :

Data validators do not differentiate between upper limit flat

- lined data and instrument error data

issues.

Discussion:

T he BAM 1020 instrument has an upper limit on reporting hourly PM 10 concentrations. When the upper limit is reached for an hourly value, a default reading of the upper limit is displayed. Unfortunately, this is very similar to the default reading for an ins trument error. Currently, SCAQMD invalidates all data that appears to be flat

- lining for long periods of time with a miscellaneous void (AM) flag, under the assumption that there was a potential instrument malfunction. While this is an appropriate procedur e in most cases,

PM 10 hourly values that are extremely high may be confused for values associated with

an instrument malfunction or error.

Recommendation s : 1. New procedures should be developed for continuous PM 10 data validation to

minimize the potential for improper invalidation of extremely high PM 10 data.

2. When the upper limit of an instrument is reached, SCAQMD should use the proper AQS Quality Assurance Qualifier code “EH” (estimated value: exceeds upper range) in the place of the current “A M” flag.SCAQMD TSA Report: August 2010

3 3

FINDING E 3 :

No validation flag was assigned for the Perris ozone monitor from May 2

- 5, 2009.

Discussion: EPA noted that data were missing from the Perris ozone monitor from May 2

- 5, 2009, but no validation flag was assigned.

Recommendati on: SCAQMD should determine why data are missing, restore information from the chessel backup system (if appropriate), and apply appropriate flags to the data in AQS.

FINDING E 4 :

Electronic records are not always handled in a manner that maintains their reproducibility

.

Discussion: The original raw data file used by the laboratory for particulate matter is editable and used by the data validators. Similar practices were observed in the calibration laboratory. It is essential that record integrity be p reserved. The 2003 NELAC Standard , June 5, 2003 (EPA/600/R

- 04/003) notes that “records that are stored or generated by computers or personal computers shall have hard copy or write

- protected backup copies.”

Recommendation: SCAQMD should ensure that any e lectronic records are appropriately write

- protected and

backed

- up.

FINDING E 5 :

The data validators do not review the PM 2.5 filter custody forms.

Discussion:

The PM 2.5 custody forms are the original records for many steps in the filter

- based data

collec tion process. These forms contain information that may have been entered erroneously into the laboratory data system. Additionally, these forms may contain notes from the field technicians that provide useful information for the validation process.

Ther efore it is essential that these forms be reviewed as part of the data validation

process.

Recommendation:

The PM 2.5 data validation process should include a review of the filter custody forms.SCAQMD TSA Report: August 2010

34

F. QUALITY MANAGEMENT

The Code of Federal Regulations Vo lume 40, Part 58, Appendix A, establishes minimal requirements for quality assurance for PQAOs such as SCAQMD. These requirements include quality planning, independent quality management, and cooperation with performance and

technical audits. Insufficie nt quality management and control has been cited a as rationale for legal challenges to NAAQS redesignation decisions.

SCAQMD has a defined quality management system, which includes a quality management plan and oversight by an independent quality assura nce manager and audit program. Draft QAPPs are in place or in development for all aspects of the monitoring program although these need to be finalized, please see General Comment A1.

Soon after the last EPA TSA, SCAQMD established a QA group lead by a QA Manager, Jason Low, which was given organizational parity with the Ambient Monitoring and Laboratory Managers. There are two direct reports to the QA Manager, a QA auditor and a QA Chemist. The QA group also manages a support contract that performs ad ditional QA/QC audits of the monitoring network. Based on the current workload, as noted in Finding A3, the current QA group staffing is inadequate to maintain a rigorous, long

- term quality management program.

Despite the limited staffing levels, the QA group has laid the foundations of a strong QA

program.

EPA regulations require independent performance audits of gaseous pollutants and flow

audits of PM 2.5, and PM 10 and monitors. For gaseous pollutants, the current regulation requires

that 25% of ea ch pollutant monitoring network be audited per calendar quarter so that each instrument is audited once per year. Prior to 2007, flow audits of these monitors were also required. Additionally, quarterly independent flow audits of PM 2.5 samplers were requ ired. Beginning in calendar year 2007, semi

- annual flow audits of each PM 2.5 , PM 10 , and TSP sampler are required (40 CFR, Part 58, Appendix A, Sections 3.2.4 and 3.3.3.) This is in addition to the required monthly flow evaluations (40 CFR, Part 58, Appen dix A, Sections 3.2.3 and 3.3.2.).

The Quality Assurance group is responsible for conducting the required gaseous pollutant

audits and PM 10 and PM 2.5 flow audits. An independent contractor supports the flow audits of

filter

- based samplers and provides aud it results to site operators. A contractor also supports meteorological audits.

Quality Assurance and Quality Control

FINDING F1:

Flow audits of continuous particulate monitors are not conducted at the required frequency.SCAQMD TSA Report: August 2010

35 Discussion:

Semi

- annual fl ow rate audits are required in 40 CFR Part 58, Appendix A, Section 3.3.3 for all FRM and FEM monitors measuring particulates. Audits of filter

- based monitors that are conducted by a SCAQMD contractor have been completed at the required frequency. Audits conducted by the SCAQMD auditor of the continuous monitor have not met the required frequency. Prior to October 2006, audits were required annually for

PM 10 and TSP and quarterly for PM 2.5 . SCAQMD indicated that, for continuous monitoring, there are i nsufficient audit staff to meet this requirement. These audits constitute required, independent evaluations of SCAQMD monitors. The 2007 EPA Region 9 TSA Report also identified this activity as one that required implementation. .

Recommendation: SCAQMD should schedule and conduct the required audits of continuous particulate

monitors.

FINDING F 2 :

Logbooks in all operational areas were not formally tracked and controlled.

Discussion: Logbooks are used extensively by SCAQMD to record information that supports the validity and defensibility of data. Because of the size of the SCAQMD monitoring organization, a significant number of logbooks are used. For document tracking purposes, the existence and status of each logbook should be recorded and tracke d. This function should be performed by the QA program or in each operational unit. For more information on records management, refer to 2003 NELAC Standard , June 5, 2003 (EPA/600/R

- 04/003).

Recommendation: SCAQMD should create and implement a system fo r management of logbooks and

similar documentation.

FINDING F 3 :

The SCAQMD formal corrective action process should be improved.

Discussion: During the audit EPA noted: Some significant corrective actions were not handled through the formal

corrective a ction process. Staff indicated that some identified problems had not been resolved. The formal corrective action process did not include a discussion of the cause of the problem, nor did it document the steps taken to prevent similar deficiencies

from occu rring in the future.SCAQMD TSA Report: August 2010

36 The SCAQMD corrective action process should be the primary system to document how significant deficiencies have been addressed. This information should be accessible to all monitoring staff and managers. Additionally, staff should b e encouraged to use this process to resolve or document issues that they believe are significant or are not being resolved through normal management communications. It was also noted that the

corrective action form did not included a discussion of how sim ilar deficiencies would be prevented from recurring.

Recommendation: SCAQMD should evaluate the formal corrective action process to make it more transparent, accessible, and useful.

FINDING F 4 :

The control charts used by the QA group to track data quali ty do not include all the data being produced.

Discussion: The use of control charts to monitor quality control parameters is strongly recommended

in the QA Handbook for Air Pollution Measurement Systems, Volume II, Ambient Air Quality Monitoring Program , December 2008. The SCAQMD monitoring group uses control charts to monitor individual field sites. However, control charts are not commonly used by the QA group to monitor network performance. Given the size of the SCAQMD network, it would be helpful to use a visual/graphic approach to understand and respond to trends in the quality of monitoring data. SCAQMD’s QA support contractor produces control charts for quality control audits that they perform.

Recommendation: The QA group should develop metrics and control charts to support possible interpretations of the ambient air data sets.

FINDING F 5 :

QA audits of gaseous pollutants were not performed as required by regulation.

Discussion: 40 CFR Part 58, Appendix A, Sections 3.2.2 indicates that each c alendar quarter at least 25% of the monitoring sites in the network must be audited. For two or more quarters, fewer than 25% of monitoring sites are audited. Additionally, the time between audits of an individual monitoring site can be overly long (a 20

- month auditing gap was noted for the Pasadena monitoring station).

Recommendation: The QA group should develop and meet schedules for conducting performance audits that

are consistent with 40 CFR Part 58, Appendix A.SCAQMD TSA Report: August 2010

A

– 1

APPENDIX A: Summary of Findings

Significant Findings

FINDING A1: SCAQMD’s Quality Assurance program documentation, including most Quality Assurance Project Plans (QAPPs) and Standard Operating Procedures (SOPs), was not complete and was not approved by EPA at the time of the audit.

FINDING A2:

In several operational areas, official records were not available or were stored in a manner that did not ensure their reproducibility.

FINDING A3: Limited staffing in several key areas is significantly impacting monitoring program performance and quality

.

FINDING A4: The QA s taff o versight of f ixed l aboratory needs to be strengthened.

FINDING A5: Quarterly data entry to AQS for filter

- based PM 2.5 and PM 10 and annual data certification were not completed on time.

FINDING B1:

PM 10 data at the Indio monitoring site may be affected by middle scale sources

.

FINDING B2:

Exide

- Rehrig should be a collocated lead (Pb) monitoring site.

FINDING B3:

Placement of PM monitors at Anaheim does not meet coll o cation requirements.

FINDING B4: SCAQMD does not rout inely notify EPA about collecting data from SPMs and sometimes does not enter that data into AQS.

FINDING B5 The SCAQMD has a dense monitoring network that may include redundant or low

- value sites.

FINDING C 1 : One

- point flow rate verifications were not being performed on continuous PM 10 instruments throughout the SCAQMD monitoring network until approximately August of 2008.SCAQMD TSA Report: August 2010

A

– 2

FINDING C 2:

Collocated filter based PM 10 data are not currently submitted to AQS under an independent

monitor POC.

FINDING C3:

Th e inlet at the Pomona station does not meet the 40 CFR 58, Appendix E siting criteria for O 3 and NO 2 monitoring.

FINDING D1: The cleaning process of canisters appeared to be at maximum capacity, which falls outside guidance a nd good laboratory practice.

FINDING D 2 : Deficiencies observed in an external TO

- 15 performance audit sample have not been satisfactorily resolved.

FINDING D 3 : The QA measures used to control PAMS analyses are generally less rigorous than specified in guidance and by good laborato ry practices.

FINDING D 4 : The primary TO

- 15 analyst stated that a method detection limit (MDL) study has not been performed for many years due to time constraints.

FINDING D 5 : The laboratory's process for certifying a batch of eight cleaned PAMS canister s as clean based on selecting three canisters for certification analysis and allowing one to fail before the batch is rejected is outside guidance and common laboratory practice.

FINDING D 6 : There was no evidence of consistent secondary review of instr ument run logbooks. This was also observed during the previous TSA.

FINDING D 7 : The method being used for analysis of Pb for NAAQS compliance is a modification of an existing Federal Equivalent Method (FEM).

FINDING D 8: The sampling systems for co llection of PAMS and Toxics sample are not routinely certified using gaseous standards

.

FINDING D 9 : The monitoring field technician was not aware of storage and handling requirements of carbonyl sampling cartridges in the field.SCAQMD TSA Report: August 2010

A

– 3

FINDING D1 0 : The procedu re to test PM 2.5 lot blanks is inconsistent with EPA guidance.

FINDING E1: For continuous monitoring data, SCAQMD uses a limited number of validations flags that may not reflect the most appropriate AQS flags.

FINDING E 2 : Data validators do not differen tiate between upper limit flat

- lined data and instrument error data

issues.

FINDING E 3 : No validation flag was assigned for the Perris ozone monitor from May 2

- 5, 2009.

FINDING E 4 : Electronic records are not always handled in a manner that maintains thei r reproducibility

.

FINDING E 5 : The data validators do not review the PM 2.5 filter custody forms.

FINDING F1: Flow audits of continuous particulate monitors are not conducted at the required frequency.

FINDING F 2 : Logbooks in all operational areas were n ot formally tracked and controlled.

FINDING F 3 : The SCAQMD formal corrective action process should be improved.

FINDING F 4 : The control charts used by the QA group to track data quality do not include all the data being produced.

FINDING F 5 : QA audits o f gaseous pollutants were not performed as required by regulation.SCAQMD TSA Report: August 2010

B

– 1

APPENDIX B: Good Practices Recommendations

Recommendations for Field Operations

I. SCAQMD uses horizontal manifolds at some monitoring sites. EPA recommends (Appendix F of the QA handbook) that horiz ontal manifolds be oriented in a manner so as to prevent condensation from getting into the instrument lines. Several SCAQMD sites have horizontal manifolds oriented with the inlets on the bottom, which would not prevent debris or condensations from getti ng into sampling lines.

II. Based on a review of field PM 2.5 tracking forms received by the laboratory, it was observed that the field technicians do not always complete these forms. The information included in these forms documents the field collection pro cess and should be completed.

III. The toxics sampling site visited, Pico Rivera, is equipped with a manifold system similar to a standard air monitoring station. The technician confirmed that the proper operation of this manifold and “kicker pump” is evaluat ed during site visits. However, this QC check is not recorded on a site maintenance form or in the station logbook.

IV. In order to verify the integrity of ambient air samples, it is recommended that the final canister pressures be recorded from a certifie d pressure gauge both in the field and in the laboratory.

V. Canisters should be stored in the field at ambient temperature and away from direct sunlight. If they will be stored for several hours in a vehicle in the field, they should be placed in an insula ted container.

VI. Station temperatures need to be recorded consistently.

VII. Other siting recommendations:

a. Monitor the distance and height of the trees near the Rubidoux site.

b. Check to determine if the LA Main site height meets requirements for all

program s.

c. Evaluate whether the TSP monitor at the Pasadena site can be moved to an

unobstructed location.

d. Construction activities near the Pico Rivera site are probably impacting the PM monitoring, which could lead to problems with data completeness. Considerat ion should be given to moving this site so that it is not impacted by construction.

e. Mission Viejo PM monitors are sited next to a building on the downwind side. While the distance from the building to the monitors may meet siting requirements, predominan t wind on the day of the EPA visit was such that the monitors may have been experiencing an altered air mass from the inlets on the roof. EPA suggests moving the monitors out further from the building to sample a more representative air mass.SCAQMD TSA Report: August 2010

B

– 2

Recommendati ons for Laboratory Operations

I. The run channels of the carbonyl samplers should be evaluated periodically for blank contamination. As currently operated, the carbonyl samples have channels dedicated for blank samples. This approach does not provide for a ny blank control of the normal sampling channels.

II. PM filter disposal should be documented so that the laboratory can track the filters that are available in storage and those that have been disposed of.

III. Filter custody forms should originate from the lab oratory and include information on filter ID, conditioning, preweight, and expiration dates. The recorder should initial the

entry.

IV. The filter laboratory temperature and humidity sensor should be placed so that it cannot be influenced by the analyst's br eath.

V. The ventilation system for the filter weighing room brings hot air to an air

- conditioned interior space. This configuration wastes energy. Altering the configuration to vent outside would be more energy efficient and might reduce costs.

VI. The field log books for PAMS and Toxics should be checked by the validators.

VII. The laboratory should investigate and consider upgrading to a dual GC/FID /mass selective detector configuration. Such as configuration would reduce the effort involved in the current p rocess and significantly facilitate proper compound identification. It would add another measure of confidence in compound identification.

VIII. The Pb data report has a place for a senior chemist's signature, but was not signed. The District should complete d ocumentation of QA activities, including initials or signatures.

IX. Two student interns perform visual inspection of all the filters to be used for Pb analysis. This is a repetitive task that relies heavily on analyst concentration of effort. The

laborator y may wish to introduce a quality control check point, such as introducing filters with known defects, into the filter inspection stream.

X. Internal chain of custody forms are not used once filters are received from the field. Filters are logged in wh en they are received, but there is no record of when they are

transferred to the ICP

- MS for analysis. Demonstration of custody control after samples are in the laboratory can be an important factor if data are challenged. Internal custody

can be tracked using forms or sign

- out sheets The secondary source standard that is used to verify carbonyl calibration is from a different lot, but is purchased from the same vendor as the primary standard (TO

- 11, Sec. 11.4.4). The method requires that each

calibration curve be verified against a standard prepared from a material obtained from a second source. Although the standards used for this purpose are obtained from aSCAQMD TSA Report: August 2010

B

– 3 different lot, it would be a more appropriate to use material from a different vendor.

Standard s from the same vendor, but a different lot, probably do not qualify as second source standards. The importance of second standard source verification in adding confidence to the reliability of standards is highlighted by the laboratory's stated experien ce with problem gas standards in the TO

- 15 Department.

XI. The laboratory's method of bringing cartridge extracts to 5mL volume introduces a source of analytical error. Extracts are not brought to volume using volumetric glassware. Instead, extracts are bro ught to volume in vials on which the 5mL level has been marked and measured against a reference vial with a similar mark (TO

- 11, Sec. 11.2.3 and 11.2.4). It is recommended that volumetric glassware be used at all stages of analysis, and specifically where cartridges are extracted and brought to 5mL volume.

XII. Analyst and other staff were unaware of the target value for carbonyl MDLs, which increases the risk that they fall short of measurement goals. MDL results were reported in liquid (mL) units, and no t back calculated to air volume units, which made it difficult to assess if method MDLs were achieved. Presenting MDLs study results in air units instead of liquid units would make it easier to confirm that measurement quality objectives are being met. Re gardless of how MDL calculations are presented, target criteria for MDLs should be established for passing or failing, and staff should be familiar with the target values present in the SOP in order to ensure they are met.

XIII. The laboratory tracks daily cal ibration standard results using a process quality control chart. However, it was not easy to determine if results were within control limits. Including method acceptance limit control or quality control lines to the plots would be

useful.

XIV. Duplicate sa mples are being analyzed, but the analyst was not able to state the control limits for acceptable results or what result would trigger a corrective action. Replicate results are being documented in tabulated format. It is important that the analyst and

q uality assurance staff refer to QC results when corrective action is needed, and that outliers are taken into account when data are evaluated against their intended use. Plotting duplicate results or calculated percent difference of duplicate results woul d help staff evaluate quality control results and limits.

XV. Laboratory staff were not clear if the acetonitrile used for extracting cartirdges is tested for purity, nor what the upper limit of acceptable formaldehyde contamination is. According to the T O

- 11 method (Sect. 6.3; 9.2.1), it is important to know the amount of formaldehyde in the acetonitrile reagent since it will be converted to hydrazone, and therefore should be checked on a regular basis.SCAQMD TSA Report: August 2010

B

– 4

Recommendations for Quality Assurance

I. There ar e no assigned document custodians in the operational areas. Some documents are tracked by the staff that created them. A formal system should be developed whereby official documents are created, used and archived so that they are available to all staff at
all times.

II. The QA performance auditor does not audit short term projects. However, data from

some short

- term projects may be used to make critical decisions regarding human health and the environment, in which case the data needs to be of known qualit y. It is

recommended that when short

- term projects are in the planning stage, the project data quality objectives (DQOs) be evaluated to determine whether QA audits should be

included.

III. The QA Laboratory should develop a schedule for certifying primary an d transfer standards. Certified standards should be verified upon receipt or soon thereafter.

IV. The QA group should conduct regular audits of data quality. This should include selecting several reported values, verifying all the steps of the data collectio n process and recalculating the results.