## Technical System Audit Report

Clark County Department of Air Quality and Environmental Management Ambient Air Quality Monitoring Program

July 28 – July 30, 2009

C onducted by

Air Quality Analysis Office Quality Assurance Office

Air Division Management and Technical

Services Division

US EPA Region 9

75 Hawthorne Street San Francisco, California 94105

February 2010Clark Co. TSA Report: September 2009

CONTENTS

PAGE

GLOSSARY OF ACRONYMS

......................................................................... 3

EXECUTIVE SUMMARY

................................................................................ 4

INTRODUCTION

............................................................................................ . 6

A. GENERAL PROGRAM

............................................................................... 8

B. NETWORK MANAGEMENT

..................................................................... 14

C. FIELD OPERATIONS

.................................................................................. 17

D. LABORATORY OPERATIONS

................................................................. 20

E. DATA AND DATA MANAGEMENT

......................................................... 22

F. QUALITY MANAGEMENT

........................................................................ 23

APPENDIX A, Summary of Findings

................................................... 25

APPENDIX B, QMP and QAPP Comments

........................................... 27

2Clark Co. TSA Report: September 2009

GLOSSARY OF ACRONYMS

AQS...................................... Air Quality System

CO........................................ Carbon Monoxide CFR...................................... Code of Federal Regulations EMS............................ Electronic Management System LEADS......................... Leading Environmental Analysis and Display System NAAQS................................ National Ambien t Air Quality Standard NCore.................................. National Core multipollutant monitoring stations NIST............................ National Institute for Standards and Technology NOx/NOy............................. Nitrogen Oxides

NV............................... State of Nevada

O 3 .......................................... Ozone

ORD............................. Office of Res earch and Development

PM........................................ Particulate matter

PM 2.5 ..................................... Particulate matter 2.5 microns or less in aerodynamic

diameter

QA........................................ Quality Assurance PM 10 ...................................... Particulate matter 10 micr ons or less in aerodynamic

diameter PQAO................................... Primary Quality Assurance Organization

QC........................................ Quality Control QAPP................................... Quality Assurance Project Plan QMP............................ Quality Management Plan SPM..................................... Special Purpose Monitor SOP...................................... Standard Operating Procedure

SLAMS................................ State or Loca l Air Monitoring Station

SO 2 ....................................... Sulfur Dioxide

SRP............................. Standard Reference Photometer STN............................. Speciation Trends Network TSA...................................... Technical System Audit EPA...................................... United States Environmental Protection Agency

3Clark Co. TSA Report: September 2009

EXECUTIVE SUMMARY

This report documents the findings ma de by EPA Region 9 during its Technical System Audit (TSA) of the Clark County Depa rtment of Air Quality and Environmental Management (DAQEM) ambient air monitori ng program. A TSA is an on-site review and inspection of a state or local ambient air monitoring program to assess its compliance with established regulations governing the co llection, analysis, validation, and reporting of ambient air quality measurement data. This TSA meets the requirements for EPA Regional 9 audits of the DAQEM monitoring organization required by 40 CFR Part 58, Appendix A, Section 2.5.

The TSA was conducted by EPA Region 9 staff from July 28 – July 30, 2009. The audit team interviewed management and staff on specific aspects of the air monitoring program including network design, field operations, laboratory operations, data handling, quality assuran ce and quality control procedur es. The audit team also inspected several monitoring sites and visited several a dditional monitoring sites including the proposed NCore monitoring sta tion. The site inspections consisted of interviewing the site operato r, reviewing station and inst rument logbooks, and evaluating the station siting with respect to EPA requirements for probe siting 1

.

The TSA is one of the ways EPA oversees data collection acti vities performed by state and local agencies to ensure they meet certain minimum data quality objectives. Other assessments, such as network reviews and performance evaluations, are also used to collect information on the overall qualit y of ambient air monitoring data. These assessments enable agencies to identify and correct program elements which may be adversely affecting the quality of ambient air data.

The results of the TSA are summarized be low and discussed in more detail in the specific sections of this report Recommendati ons for addressing each of the findings are provided. The specific acti ons taken by DAQEM to addres s the findings should be approved by EPA and documented by DAQE M in a corrective action plan.

EPA would like to thank all the sta ff and management of DAQEM for their support and cooperation during the audit.

Program Strengths:

• Overall, DAQEM has well qualified, very knowledgeable, and experienced staff.

• DAQEM has recently upgraded the data collection system for continuous pollutants. This upgrade has improved its ability to process da ta efficiently and control data quality.

1 40 CFR, Part 58, Appendix E, “Probe and Monitoring Path Siting Criteria for Ambient Air Quality Monitoring”

4Clark Co. TSA Report: September 2009

• Each DAQEM monitoring station visited is well sited to collect representative

data.

• DAQEM data specialists were able to easil y track the data points requested by the

auditors.

• The PM 2.5 laboratory is well maintained a nd organized by the analysts.

• Many DAQEM staff members have implement ed measures to improve the quality of the overall monitoring program.

Program Findings:

While acknowledging the strengths of DAQEM’s monitoring program, EPA identified several program elements th at could be improved. Specifically, DAQEM

should:

• Strengthen the Quality Assurance program by giving it a more formal role in its oversight of the monitoring program, a nd by providing tec hnical capacity at higher levels of management to oversee the QA officer’s work.

• Ensure the Quality Management Plan and Quality Assurance Plans include sufficient information to meet EPA requirements.

• Improve procedures for maintaining hard copy records that are consistent, secure, and regularly updated. DAQEM should also standardize proce dures for creating records in the Leading Environmental Analysis and Display System (LEADS) data system.

• Improve the procedures for reporting data in Air Quality System (AQS) by adequately identifying compromised data. DAQEM should maintain records in the LEADS data system that document w hy data have been flagged in AQS and should verify that data submitted to AQS is consistent with the data that are published on its website.

• Improve the systems for tracking the lo cation and status of equipment, and improve the procedures for initiating and documenting equipment repairs.

In addition to these significant findings, EPA has identified additional findings and made recommendations. The individual findings are reported in th e topic sections of this document and are also summarized in Appendix A.

5Clark Co. TSA Report: September 2009

INTRODUCTION

The Clark County DAQEM is the govern ment agency with delegated authority and responsibility for collecting ambient air qua lity data pursuant to the Clean Air Act of

1977 and the Clean Air Act Amendments of 1990.

From July 28 through July 30, 2009, Michael Flagg of the Region 9 Air Quality Analysis Office and Matthew Plate of the Region 9 Quality Assurance Office conducted a Technical System Audit of DAQEM’s ambient air monitoring program. The TSA covered the following program areas:

• General / Quality Management

o Program Organization

o Facilities

o Independent Quality Assurance and Quality Control

o Planning Documents (including QMP, QAPPs, & SOPs)

o General Documentation Policies

o Training

o Corrective Action

o Quality Improvement

o External Performance Audits

• Network Management / Field Operations

o Network Design

o Changes to the Network since the last audit

o Proposed changes to the Network

o Field Support

 SOPs

 Instrument Acceptance

 Calibration

 Repair

 Site and Monitor Information Form

• Laboratory Operations

o Routine Operations

o Quality Control

o Laboratory Preventive Maintenance

o Laboratory Record Keeping

o Laboratory Data Acqui sition and Handling

o Specific Pollutants: PM 10 , PM 2.5 and Lead

• Data and Data Management

o Data Handling

o Software Documentation

o Data Validation and Correction

o Data Processing

o Internal Reporting

o External Reporting

6Clark Co. TSA Report: September 2009

Additionally, EPA tracked supporting docum entation for several data points/sets from calendar year 2008.

Participating DAQEM managers, su pervisors and staff included:

• Mike Sword, Engineering Manager

• Yousaf Hameed, Monitoring Supervisor

• Mickey Palmer, Senior M onitoring Tech. (QA Officer)

• Mickey Turner, Senior Monitoring Tech.

• Phillip Wiker, Senior Monitoring Tech.

• Pravin Perna, Monitoring Tech. II

• Dave Dickens, Monitoring Tech. II

• Mathew Nelson, Monitoring Tech. II

• Kris Simonian, Monitoring Tech. II

This report is divided into ten sections. The first section is an executive summary that describes the purpose of the TSA and a summary of the most significant findings. The following section is an introduction that provides a brief desc ription of the DAQEM air monitoring program, identifies partic ipating DAQEM personnel and the EPA audit team, and describes the report’s organi zation. The third section discusses the general/significant findings. The next five se ctions address specific aspects of the air monitoring program: network management, fi eld operations, laborat ory operations, data management and quality assurance/quality c ontrol. Appendix A is a summary listing of the findings contained in this report. Appendix B includes EPA concerns with the DAQEM Quality Management Plan, QA Project Plans, and associated Standard Operating Procedures. Reco mmendations for each finding ar e provided to give some indication of the EPA Region 9’s expectations as to how findings can be addressed. If the DAQEM has other approaches or alternatives to address the concerns identified, the Region will consider them, provided the co rrective action adequately addresses the finding.

7Clark Co. TSA Report: September 2009

A. GENERAL PROGRAM

The Governor of Nevada gave authority for Air Quality Matters in Clark County to the Clark County Board of Commissioners in 2001. This is reflected in the Nevada Revised Statutes 445B.500, which delegate sign ificant air pollution control authority to Nevada counties with populations greater than 400,000. The Clark County Board of Commissioners created an Air Quality Management Board to govern air quality programs and DAQEM to implement air quality progr ams, including ambient air quality monitoring. Prior to 2001, ambient air quality monitoring was implemented by the Clark County District Health Department.

DAQEM currently has 16 active monitoring stations for criteria pollutants. The pollutants measured include:

• Carbon Monoxide

• Nitrogen Dioxide

• Ozone

• Particulate Matter (PM 10 and PM 2.5 )

DAQEM is headed by a Director and two Assi stant Directors. At the time of the audit one of the Assistant Director positio ns was vacant. Ambient air monitoring is conducted by the Engineering Division, which is lead by an Engineering Manager, who reports to an Assistant Director. The Engineering staff consists of a Monitoring Supervisor, who oversees 11 technical staff. At the time of the audit one of these staff positions was vacant. An Office Specialist a nd a Senior Technician/QA Officer report directly to the E ngineering Manager.

FINDING A1:

DAQEM’s Quality Assurance program may lack sufficient management and should have a more formal and independent role in the oversight of the monitoring program.

Discussion: The EPA requirements for independence of quality assurance for Primary Quality Assurance Organizations (PQAOs) are in 40 CFR Part 58, Appendix A, Section 2.2. This section states that “the qual ity assurance management function must have sufficient technical expertise and management authority to conduct independent oversight and assure implem entation of the organization’s quality system.” The DAQEM program includes one senior technician who serves as the quality assurance auditor and quality assu rance officer. While this organizational structure provides a level of independen ce for the QA functions performed that is generally acceptable, it is less than th e two layers of management that EPA recommends. More significantly, the qua lity assurance function does not include sufficient formal technical oversight of the monitoring program and there is no mechanism for independent review of th e QA officer’s work. EPA also found that

8Clark Co. TSA Report: September 2009

the following functions, which should be performed by the independent QA officer, were either not being performed or were not being performed adequately:

• Periodic audits of the quality management system (management system reviews), routine procedures, and da ta quality to identify areas of improvement and to ensure that monitoring programs continue to follow sound and documented procedures.

• Formalized review of certain QA activit ies, such as data validation, annual

certification of the O 3 transfer and flow standard s, and the certification of calibration gas cylinders for NOx, CO, and SO 2 instrument calibration.

• Review and approval of corrective actions.

• Review and tracking of precision and accuracy data.

EPA QA/R-5, EPA Requirement for Quality Assurance Project Plans , lists the types of assessments that should be elem ents of a quality management program. These include, “surveillance, management systems reviews, readiness reviews, technical systems audits, performance eval uations, audits of data quality, and data quality assessments.”

Recommendations:

2. DAQEM should strengthen the Quality Assurance/Quality Management function by giving it more formal au thority to oversee the monitoring

program. 1. DAQEM should consider alternative or ganizational or reporting structures which will provide for greater technical capacity at higher levels of management to oversee the work of the QA officer.

3. DAQEM should also consider moving f unctions, such as data management and validation, into the QA unit because it is not directly involved in data

collection.

FINDING A2:

The Quality Management Plan (QMP) and Quality Assurance Project Plans (QAPPs) need additional information to meet EPA requirements.

Discussion: The QMP and QAPPs submitted to EPA by DAQEM in 2008 do not include many elements required in QA planning documents. Specific comments on these documents are provided in Appendix B.

All organizations conducting enviro nmental programs funded by EPA are required to establish and implement a quality management system. In accordance with 40 CFR Parts 31 and 35, grant recipi ents are required to document their quality systems in a Quality Management Plan through EPA Order 5360.1 A2, Policy and Program Requirements for the Mandatory Agency-wide Quality

9Clark Co. TSA Report: September 2009

System (EPA 2000). Additionally, specifi c ambient air monitoring requirements are found in 40 CFR, Part 58, Appendix A, Section 2.1. Guidance on developing QMPs can be found in the EPA guidance document “EPA Requirements for Quality Management Plans”, EPA/240/B-01/002, March 2001.

EPA also requires that organizations de velop a QAPP for each type of ambient pollutant that is being measured. The QA PP integrates all technical and quality aspects of a project, including planni ng, implementation, and assessment. The purpose of the QAPP is to document planning results for environmental data operations and to provide a project-specifi c “blueprint” for obtaining the type and quality of environmental data needed for a specific decision or use.

The QAPP documents the quality assurance a nd quality control th at are applied to an environmental data operation to assure the results obtained are of the type and quality needed and expected. Further guidance on developing QAPPs can be found in the guidance documents “EPA Requirements for Quality Assurance Project Plans,” EPA/240/B-01/003, Marc h 2001, and “Guidance for Quality Assurance Project Plans,” EPA/240/R-02/009, December 2002.

Recommendation: DAQEM should work with EPA Region 9 to develop approved QA planning

documents.

FINDING A3:

The consistency of records generated and maintained in the LEADS Electronic Management System (EMS) is inadequate. Inconsistencies in the generation and

retention of other record s may also exist.

Discussion: Many of the records DAQEM generates a nd maintains are kept in the LEADS EMS, which is an useful central reposito ry. However, EPA noted inconsistencies in the generation and completeness of records in the system. Significant examples are the monitoring station l ogs and data validator’s notes.

When comparing data in the LEADS syst em to data in AQS, it was not always clear why certain data were flagged in AQS and other data were not. This was mainly due to the level of detail in the electronic “logs” and comments. Because the LEADS EMS entry fields are “free form ”, it is especially important that the operators and validators develop and use a standardized minimal level of detail when entering information. The procedure for documenting peer, supervisor, validator, and QA Officer review of these records should also be standardized.

DAQEM has established systems for maintain ing other information in instrument logbooks and laboratory records. For ot her records maintain ed by individual

10Clark Co. TSA Report: September 2009

staff, the department has no clear reco rdkeeping procedures. The Department

should evaluate the records maintained by individuals to ensure they are consistent, secure, regularly maintained, a nd (for electronic r ecords) backed-up.

Also see Finding C6.

Recommendation: DAQEM should create a standardized method for creating and maintaining “station logs” and other entries within the LEADS system. Additionally, the Department should evaluate other elec tronic and hard-copy documentation to ensure that they are consistently prepared and maintained.

FINDING A4:

Procedures for submitting data to the AQS, fl agging data in AQS, and reporting data to the public need to be improved.

Discussion: In the past year and a half, DAQEM has sh ifted most data reporting to a new data system supported by IPS MeteoStar.

If data are compromised, the appropri ate AQS flags should be applied. DAQEM uses the “miscellaneous void (AM)” flag to describe a number of different issues with the data. Using flags that specifically describe how the data are compromised enables DAQEM and EPA staff and the public to understand better what the

issues associated with the data are. All AQS codes and flags can be found at: http://www.epa.gov/ttn/airs/air saqs/manuals/codedescs.htm

.

Data that DAQEM intends to submit for consideration under EPA’s exceptional events rule have not been flagged as such in AQS. Pursuant to 40 CFR 50.14(c)(2)(iii), “Flags placed on data bei ng due to exceptional events together with an initial description of the event shall be submitted to EPA not later than July 1 st of the calendar year following the year in which the flagged measurement occurred.” Because the exceptional events rule allows for the exclusion of data from consideration against the NAAQS, values affected by exceptional events will impact the planning process of both DAQEM and EPA Region 9. This timeline ensures the efficiency of excep tional event evaluation and should be met

in accordance with the CFR.

Finally, EPA noted discrepancies when comparing validated data reported by DAQEM on its website with data in AQS. An example of this is ozone data reported from the E. Craig Road site from February 1-4, 2008. The Department web-based system reported zeros during th is time period, while the data in AQS

were invalidated. An evaluation of sim ilar data during the audit found that this was an artifact of importing data into the sy stem. This raises the possibility that a

11Clark Co. TSA Report: September 2009

significant amount of data available to the public may not match what was reported to AQS.

One instance of data not being reported to AQS was also noted. This was for

ozone at the Joe Neal site from Novemb er 3-5, 2008. An evaluation of the data system revealed the data were not re ported to AQS by the LEADS EMS because

it had not been marked as “val idated” in the system.

Recommendations: 1. DAQEM should improve its use of flagging by using more descriptive flags and by avoiding overuse of the “m iscellaneous void (AM)” flag. 2. DAQEM should identify and resolve all discrepancies between the data in AQS and the data reported on its website. DAQEM should additionally implement procedures to eliminate such discrepancies in the future.

FINDING A5:

DAQEM lacks an adequate system for tracking equipment status, initiating equipment repairs, and documenting repair history.

Discussion: DAQEM does not have an established mechanism for tracking instrument location, condition, and repair history. Currently monitors that are not functioning properly are repla ced with working instruments as available from the main office. Because poorly functioning equipment may have bearing on data quality, this information should be tracked in a manner that is easily recoverable. Improved efforts to track such informa tion may prevent the deployment of poorly functioning instruments and allow DAQEM to reduce the number of instruments

held in reserve.

At the time of the audit, it was also no t clear who was responsible for initiating repairs to broken instruments. This should be resolved so that instrument repair is undertaken and verified by qualified individuals.

Overall, it is evident that establishing a formal communication system related to instrument operation and maintenance woul d improve data quality in all elements of the quality management system, including some which may not have been

identified in this audit.

Recommendations: 1. DAQEM should develop a system to tr ack the location and status of all

instruments. 2. DAQEM should establish standard procedures for initiating and documenting equipment repairs.

12Clark Co. TSA Report: September 2009

B. NETWORK MANAGEMENT

The Clark County network for state and local air monitoring stations (SLAMS)

consists of 16 stations. The following table summarizes the SLAMS air monitoring sites in Clark County by the respective pollutants m onitored and the year monitoring began at

the site.

SITE NAME (year established) POLLUTANT

CO SO 2 NO 2 O 3 PM 10 PM 2.5 PM 2.5 continuous Apex(1998) X X X Boulder City (1998) X X E. Craig Road (1992) X X X E. Sahara (1995) X Green Valley (1998) X X J.D. Smith (1998) X X X X X X Jean (1995) X X X X Joe Neal(2000) X X X Lone Mountain (1998) X X Mesquite(2001) X X Orr (2002) X X X Palo Verde (1998) X X Paul Meyer (1998) X X Sunrise Acres (1996) X X X X Walter Johnson (1995) X Winterwood (1979) X X Source: DAQEM 2008 Air Monitoring Network Plan

The Clark County network currently meet s or exceeds the requirements for the minimum number of monitoring sites designa ted as SLAMS for all of the criteria pollutants. All of the monitoring sites are pr operly located with respect to monitoring objectives, spatial scales a nd other siting criteria.

Pursuant to 40 CFR 58.20, states must perf orm an annual review of their ambient monitoring network. The annual network revi ew is used to determine how well the network is achieving its required air monito ring objectives, how well it is meeting data users needs, and how it should be modified (e.g., through termination of existing stations, relocation of stations, or establishment of new stations) to continue to meet its monitoring objectives and data needs. The main purpose of the review is to improve the network to ensure it provides adequate, representative, and useful air quality data. Based on changes to the monitoring regulations, which went into effect in December 2006, the annual network plan now requires evidence of a public inspection process. Clark County DAQEM submits a monitoring network plan ev ery year. The most recent submittal by DAQEM, in 2009, contains all of the required information about the monitoring stations,

13Clark Co. TSA Report: September 2009

including the information required by 40 CF R Part 58, Appendix A (quality assurance requirements) and Appendix D (network require ments and monitoring site objectives).

Network Management

Finding B1 :

The DAQEM system lacks the necessary flexibi lity to establish new monitoring stations in response to changing conditions.

Discussion: DAQEM has been working through the l ogistics of establishing a new NCore monitoring station for over a year. DAQEM ha s indicated that this is due to Clark County facility and permitting requirements, which are outs ide its control. In the future, DAQEM may be required to move and/or establish new monitoring sites to accommodate new monitoring requirements and/or other changes in the network. Excessive start-up times could mean required data are not collected.

Recommendation: DAQEM should develop a system to establish monitoring sites more expeditiously (e.g., develop standardi zed pre-approved monitoring station configurations).

Finding B2 :

DAQEM is working to resolve the status of its formal application for a PM 2.5 Approved Regional Method (ARM) for the continuous monitors.

Discussion: In 2006, EPA published criteria for approving continuous methods for PM 2.5 monitoring ARMs in 40 CFR Part 58, Appe ndix C. ARMs must be approved or disapproved by EPA within 120 days of submittal. EPA ORD is currently requesting more information from DAQE M to supplement its ARM application, which will reset the 120-day clock.

A more recent model of the Thermo Be ta Attenuation Monitoring being used by DAQEM has recently been approved as a Fe deral Equivalent Method (FEM). It would be advantageous from a collocation standpoint to use the FEM rather than

an ARM method as the latter requires 30% collocation with an FRM or FEM and 7.5% collocation ARM to ARM (see th e text below from 40 CFR Part 58, Appendix C).

14Clark Co. TSA Report: September 2009

2.4.4.5 Collocation with an ARM under this part for purposes of determining the coefficient of varia tion of the method shall be conducted at a minimum of 7.5 percent of the site s with a minimum of 1 per network. This is consistent with the requirements in Appendix A to this part for onehalf of the required collocation of FRM /FEM (15 percent) to be collocated

with the same method.

Recommendation: DAQEM and EPA Region 9 should follow up on the status of the ARM submittal, but also consider initiating a program to upgrade the contin uous instruments to

FEMs in order to save on l ong-term operational costs.

15Clark Co. TSA Report: September 2009

C. FIELD OPERATIONS

Field operations for ambient monitoring are performed by staff who report to the Monitoring Supervisor. Staff are assigned to monitoring sta tions and are responsible for all criteria pollutant operations at their resp ective stations. One Senior Technician is assigned the responsibility for calibration of field standards for gaseous pollutants and

flow.

As part of this TSA, Monitoring Techni cians were interviewed about their duties related to monitoring station ope ration. All statio n operators interviewed demonstrated a thorough knowledge of the monitoring equipmen t for which they were responsible.

The Field Sites reviewed included Apex, Jo e Neal, East Craig, and East Sahara. EPA evaluation at these sites included inspecti on of the inlet manifo lds, examination of station and instrument log books, and a speci fic check of whether appropriate Quality Control (QC) checks and Quality Assurance ( QA) audits were being performed for dates in 2007 selected for review prior to the audit. Each of the eight sites visited were well organized and had clean sampling lines. Instrument logbooks were up to date and

contained relevant informa tion on operations and maintena nce activities. Station logbooks are maintained electronically in the data acquisition system. All sites met the siting criteria of 40 CF R 58, Appendix E.

Field Operations

Finding C1:

EPA and DAQEM staff noted significant probl ems with the operations of the Tanabyte Engineering Calibrators. DAQEM staff also noted issues with the nitrogen dioxide

instruments.

Discussion:

In order to maintain an effective and efficient monitoring system, problems with equipment should be identified and corrected promptly. If problems cannot be prevented from occurring, formal proce dures should be established to minimize the impact of instrument deficiencies on day-to-day monitoring operations. These

could include:

• Changing the QC frequency of the given instrument.

• Adding additional QC checks.

• Having additional back-up instruments available.

Recommendation: DAQEM should issue a formal correctiv e action plan to address potential equipment problems. For equipment with persistent or recurrent problems, the plan should call for the developm ent of a replacement schedule.

16Clark Co. TSA Report: September 2009

Finding C2:

DAQEM typically fits each probe line end with an unacceptable stainless steel mesh filter cap.

Discussion: According to DAQEM staff, the stainless steel mesh filters are used to keep insects out of the probe lines. However, this material is inconsistent with the requirements of 40 CFR Part 58, Appendi x E, Section 9, which states that borosilicate glass or Teflon are the “onl y acceptable probe materials” for reactive gasses (ozone, sulfur dioxide, and nitrogen dioxide).

Recommendation: DAQEM must replace the stainl ess steel mesh caps with an equivalent Teflon mesh cap.

Finding C3:

DAQEM has identified upgrades that need to be made to many of the monitoring stations, some of which ha ve safety implications.

Discussion: DAQEM has developed a list of “Site Cond itions & Safety Upgrades.” Items on the list are specific to physical condition, aesthetics, and unsafe conditions; they do not include information about any need ed equipment and instrument upgrades. While these items do not directly relate to monitoring, they have the potential to impact the monitoring program if they are not addressed.

Recommendation: DAQEM should develop a target schedule for site upgrades.

Finding C4:

DAQEM lacks a consistent method for doc umenting activities and required duties performed at each site. There is variability in documentation based on the site location and the site operator.

Discussion: The level of detail in the logbooks va ries with the station operator. DAQEM currently utilizes two different met hods for documenting field and equipment operation. An “equipment logbook” is used to document calibrations, problems associated with the instrument, and re pairs. This log book is kept with the

17Clark Co. TSA Report: September 2009

instrument (e.g. the logbook follows the instru ment from site to site). The “station logbook” is maintained in the LEADS system and does not have a formal structure (see finding A3).

Recommendation: DAQEM should establish a standardized method for documenting field operations.

Finding C5:

DAQEM does not document the quality of zero air used in the program.

Discussion: Zero air scrubbers are used in pla ce of certified zero air for instrument calibrations. This is a common, acceptable pr actice. Because zero air is used to generate the zero point and it is mixed as part of the calibratio n, it must be treated as a standard. As such, zero air scrubbe rs should be verified after maintenance

and this verification must be documented.

Recommendation: DAQEM should document the performance verification of zero air scrubbers.

Finding C6:

DAQEM has inadequate procedures for scheduling and documenting preventive maintenance for the gaseous and PM instruments.

Discussion: Preventive maintenance is a key compone nt in the successful operation of air monitoring equipment. These procedures help to increase the lifetime of instruments and ensure they continue to operate efficiently and correctly. The preventive maintenance schedules are currently loosely kept within the “equipment logbook.” Some site operators keep them in different locations.

The current form of documentation is in complete and inconsistent. There is no place on the schedule to indicate who has performed the operation and whether or not the required maintenance is being checked by the QA officer or the monitoring manager.

Recommendation:

DAQEM should reevaluate the structure of the preventive maintenance schedule to address inconsistent and incomplete documentation. These documents should be kept in a common location to ensu re they are not misplaced or lost.

18Clark Co. TSA Report: September 2009

D. LABORATORY OPERATIONS

DAQEM has a PM 2.5 laboratory located in the basement of the main County Office Building Complex. The primary res ponsibility of the laboratory is the handling of PM filters, which includes prep aration, weighing, track ing, and storage.

The PM2.5 laboratory is well run and or ganized. It was noted that the laboratory relies on manual data collection methods for the recording of laboratory conditions and filter weighing sessions. This process is acceptable. However, it is recommended that DAQEM evaluate the co st/benefit of automa ting the laboratory processes. If filter weighing sessions were automated, less hand written documentation would be required and the anal yst and data validator would not need to evaluate the manually entered data for tr anscription errors. This would have the potential to substantially increase laboratory productivity and data quality.

Laboratory

Finding D1:

The times that filters are weighed and the time that filter conditioning begins are not

documented.

Discussion: Pursuant to 40 CFR Part 50, Appendix L, Section 8.2, PM 2.5 filters must be

conditioned for a minimum of 24 hours prior to weighing. In order to document that filters have been conditioned appropriately, sample weighing time and the conditioning start time must be documented.

Recommendation: DAQEM should record the times that f ilter batches begin conditioning and the times that individual filters are weighed.

Finding D2:

The temperature and humidity of the weigh -room occasionally fluctuate outside of the acceptable control limits.

Discussion: It was noted that the laboratory temperat ure and humidity occasionally violate the control limits established in 40 CFR Part 50, Appendix L, Section 8.2. The analyst indicated that th e average temperature and humidity over 24 hours is

evaluated when excursions are noted. Wh ile EPA recognizes that the temperature and humidity may fluctuate at times, such as when the air handling system cycles

19Clark Co. TSA Report: September 2009

or when individuals enter and exit the conditioning environment, the regulatory criteria for the laboratory environmen t are not based on a daily average.

Recommendation: DAQEM should identify and address the cau ses of unacceptable temperature and humidity fluctuations in the weigh-room . In the absence of other corrective measures, laboratory staff should wait until conditions in the room re-stabilize before proceeding.

Finding D3:

The dates that filters are r eceived from EPA, and the da tes that exposed filters are disposed of are not tracked.

Discussion: Sampling media and filters should be tracked through the laboratory.

Recommendation: DAQEM should develop a procedure to track the dates of new filter receipt and exposed filter disposal.

20Clark Co. TSA Report: September 2009

E. DATA MANAGEMENT

DAQEM has defined procedures in place for handling data from the time of acquisition to the time that it is submitted to EPA. The procedures are well known to the principal data providers and reviewers. The Air Monitoring Group is responsible for managing ambient monitoring data and reporting to the EPA AQS database.

Air quality data measured by the conti nuous analyzers at the field stations are stored in data loggers and on DAQEM servers. Each sta tion is polled every fifteen minutes and the data are transmitted directly to the central server in the IT facility. The first review of the data is perf ormed by the station operators.

Data from continuous analyzers are received and processed using an IPS MeteoStar data management system that began operations 2008. The system provides real-time, as well as historical, data on th e DAQEM internet portal. Several of the monitoring staff are involved in evaluatin g/validating the gaseous measurement and quality control data provided by the LEADS Electronic Management System (EMS). The LEADS EMS prepares the validated data for submittal to the AQS system.

DAQEM laboratory-generate d data includes 24-hour PM 2.5 data. Laboratorygenerated data are processed using spread-sheets and validated and submitted separately to AQS from the LEADS EMS.

DAQEM submits all required data to the AQS database, including concentrations for all criteria pollutants, supporting precisi on and accuracy information and data from

the network. Data submitted to AQS are annually certified by DAQEM.

Please refer to items 3 and 4 in Secti on A for a discussion of data management findings.

21Clark Co. TSA Report: September 2009

F. QUALITY MANAGEMENT

The Code of Federal Regulations Volume 40, Part 58, Appendix A, establishes minimal requirements for quality assurance for Primary Quality Assurance Organizations (PQAOs) such as DAQEM. These requirement s include quality planning, independent quality management, and cooperation with performance and technical audits by EPA. EPA views the application of quality manage ment systems as integral to monitoring programs. Insufficient quality management a nd control has been cite d a as rationale for legal challenges to NAAQS redesignation decisions.

DAQEM has a defined quality management system, which includes a quality management plan and oversight by an indepe ndent quality assuran ce officer/auditor. While some QA issues are formally raised with the Engineering Manager, many formal QA oversight and review functions are either not performed or are performed informally

with minimal or no documentation. EPA r ecognizes that the QA Officer’s role has recently been elevated and DAQEM is developi ng a more formal and comprehensive role for the QA Officer.

EPA regulations require independent perf ormance audits of gaseous pollutants

and flow audits of PM 2.5, and PM 10 and monitors. For gaseous pollutants, the current regulation requires that 25% of each pollutant monitoring network be audited per calendar quarter so that each instrument is audited once per year . Prior to 2007, flow audits of these monitors were also require d. Additionally, quarterly independent flow

audits of PM 2.5 samplers were required. Beginni ng in calendar year 2007, semi-annual

flow audits of each PM 2.5 , PM 10 , and TSP sampler are required (40 CFR, Part 58, Appendix A, Sections 3.2.4 and 3.3.3.) This is in addition to the re quired monthly flow evaluations (40 CFR, Part 58, A ppendix A, Sections 3.2.3 and 3.3.2.).

The Quality Assurance Officer is responsib le for conducting the required gaseous pollutant audits and PM 10 and PM 2.5 flow audits. Audits are performed at the minimum frequency or more for all pollutants. The aud itor provides audit result s to site operators,

but does not use audit data to perform statistical analyses or develop control charts. DAQEM does not have an audit program for meteorological equipment, but plans to

institute one in the near future. EPA recomm ends that a meteorologi cal audit program be instituted prior to bringi ng the NCore site online.

Quality Assurance and Quality Control

Finding F1:

The DAQEM QA Auditor does not formally communi cate failing audit results to the data managers/validators.

22Clark Co. TSA Report: September 2009

Discussion: The results of the required QA audits of gaseous and particulate monitors are not transmitted so that they can be respo nded to formally by the site Monitoring Technicians and the Monitoring Superv isor and evaluated during the data validation process. The audit results s hould be transmitted to the data validator and other staff formally, using the LE ADS EMS or another formal process developed by the QA Officer.

Recommendation: DAQEM should develop a formal process fo r the auditor to inform the data management group of failing audits, as th is can be incorporated into data

validation decisions.

23Clark Co. TSA Report: September 2009

APPENDIX A: Summary of Findings

Significant Findings

FINDING A1: DAQEM’s Quality Assurance program may lack sufficient management and should have a more formal and independent role in the oversight of the monitoring program.

FINDING A2: The Quality Management Plan (QMP) and Quality Assurance Project Plans (QAPPs) need additional information to meet EPA requirements.

FINDING A3: The consistency of records generated and maintained in the LEADS Electronic Management System (EMS) is inadequate. Inconsistencies in the generation and

retention of other reco rds may also exist.

FINDING A4: Procedures for submitting data to the EPA Air Quality System (AQS), flagging data within AQS, and reporting data to the public need to be improved.

FINDING A5: DAQEM lacks an adequate system for tracking equipment status, initiating equipment repairs, and documenting repair history.

Other Findings

Finding B1 : DAQEM lacks the necessary flexibility to esta blish new monitoring st ations in response to changing conditions.

Finding B2 : DAQEM is working to resolve the status of its formal application for a PM 2.5 Approved Regional Method (ARM) for the continuous monitors.

Finding C1: EPA and DAQEM staff noted significant probl ems with the operations of the Tanabyte Engineering Calibrators. DAQEM staff also noted issues with the nitrogen dioxide

instruments.

Finding C2: DAQEM typically fits each probe line end with an unacceptabl e stainless steel mesh filter

cap.

24Clark Co. TSA Report: September 2009

Finding C3: DAQEM has identified upgrades that need to be made to many of the monitoring stations, some of which ha ve safety implications.

Finding C4: DAQEM lacks a consistent method for doc umenting activities and required duties performed at each site. There is variability in documentation based on the site location and the site operator.

Finding C5: DAQEM does not document the quality of zero air used in the program.

Finding C6: DAQEM has inadequate procedures for scheduling and documenting preventive maintenance for the gaseous and PM instruments.

Finding D1: The times that filters are weighed and the time that filter conditioning begins are not

documented.

Finding D2: The laboratory does not have cr iteria for evaluating short-term excursions of weigh-room temperature and humidity outsi de of the control limits.

Finding D3:

The dates that filters are r eceived from EPA, and the da tes that exposed filters are disposed of are not tracked.

Finding F1: The DAQEM QA Auditor does not formally communi cate failing audit results to the data managers/validators.

25Clark Co. TSA Report: September 2009

APPENDIX B: QMP and QAPP Comments

All Documents

1) [Distribution List] These documents should include a distribution list.

Quality Management Plan

2) [Quality Policy] The QMP should include a Quality Policy for the organization that clearly and concisely states the importance of QA/ QC to the organization, the QA goals of the organization, and the comm itment to allocate and prioritize resources for QA/QC activities. 3) [Authority] The QMP should discuss how DAQEM derives its authority from the EPA, the State of Nevada, and Clark C ounty to provide air monitoring to support the NAAQS. 4) [Additional Sections] This document s hould include sections that discuss Planning and Assessment & Response. 5) [Section 2, User] This section refers to EPA’s “Quality Management Plan requirement.” EPA recommends that th e plan discuss the specific regulatory requirements. Note that EPA requires or ganizations it funds which use or collect environmental information to have a Qu ality Management System (40 CFR Part 30, 31, and 35). EPA also requires that data collected for NAAQS decision making must be performed under a Quality Management System that conforms to EPA requirements and guidance for Quality Management (EPA QA/R-2) (40 CFR Part 58, Appendix A). 6) [Section 3, User Requirements] This sect ion should discuss the requirements in 40 CFR Part 58, Appendix A, Section 2 to: develop QA planning documents; provide independent Quality Assurance; use data quality objectives or another systematic planning process; provide fo r and submit to independent system and performance audits; and follow all rele vant EPA requirements and guidance. 7) [Section 4.2, Organization] The QMP s hould provide the entire organizational chart for DAQEM in addition to the chart provided for the Engineering Division. 8) [Section 4.2, Organization; Section 4.3, Responsibilities] These sections should discuss the responsibilities of organizatio nal groups that are important to DAQEM air monitoring efforts such as EPA, the State of Nevada, and the DAQEM Engineering Division. Additionally, the re sponsibilities of the senior DAQEM managers for the Quality System should be discussed (i.e., who authorizes the QA/QC system, provides resources , and reviews QA summaries). 9) [Section 4.4, Training] This section should be expande d to describe personnel qualifications associated with key QA pos itions or responsibili ties. The present discussion does not explain why monitori ng technicians are in the Engineering

Division. A reference is made to a tr aining plan, but no detailed information about the training program is provided.. 10) [Sections 5, User Requirements Reconciliation (Quality Control); Section 6, User Reconciliation Assurance (Qua lity Assurance)] These sections should include a discussion of policy. Such issues as the minimum requirements for quality

26Clark Co. TSA Report: September 2009

control and quality assuran ce, and their authority should be discussed. The DQO process and systematic planning should be explained. Note that DQOs for criteria pollutant NAAQS monitoring are developed by EPA. Other monitoring objectives should be evaluated by DAQEM to determine if they require additional QA/QC measures or more st ringent criteria, as compar ed to relevant federal regulations. These sections should expa nd on the different documents and what their role is in defining the DAQEM air monitoring quality system. 11) [Section 7, Dispute Resolution] This section discusses the QA Manager’s responsibilities with respect to corrective action requests (CARs) without first describing the process associated with them. The discussion should be expanded to include a description of the overall corrective action system. A discussion of what triggers a CAR, what paperwork is generated and how is it tracked should be

included.

13) 12) [Section 8, Recording and Documentation] This section should define how custody of the documentation, in both hard copy and electronic form, will be

tracked and it should discuss how documents are made accessible to all staff. The section also should contain provis ions for retaining copies of some documents, such as SOPs, for historical pur poses in case there is a need to track how data were generated. Some of the terms in the accompanying table are not well defined (i.e., “keeper,” “access,” “med ium,” “retention,” a nd “disposition”). The section does not provide informa tion on documentation requirements for

routine activities. At a minimum, the QMP should indicate where these documentation requirements can be found, fo r example in SOPs or a QA Project

Plan. [Section 10.1.1, Purchasing and Procurem ent; Section 10.1.2, Subcontracting] These sections should discuss the quality a ssurance measures that will be taken to assure goods and services are of acceptable quality and that problems with such goods and services will be corrected. 14) [Section 10.2, Electronic Data Handling Syst ems] These sections should discuss the policies and/or procedures for validation and monitori ng of the quality of data from these systems.

Quality Assurance Project Plans (QAPPs)

15) [Additional Sections] The following sections should be specifically included in each QAPP:

a. Project descripti on and background

b. Sampling design (i.e., probe design and residence time)

c. Quality control checks

d. Instrument inspection and mainte nance, instrument calibration

e. Inspection of supplies

f. Assessment and response (including internal and EPA technical and performance assessments), reports to management, data review and validation, and reconciliation with user requirements 16) [Section 4.1, Organization] Th is section should include a discussion of external organizations that support this monitoring program.

27Clark Co. TSA Report: September 2009

28 17) [Section 5, User Requirements Reconcilia tion] This section should include a discussion of the EPA’s DQO for Ozone and PM2.5 monitoring. Additionally, any non-NAAQS specific data uses should be evaluated to determine if they have monitoring or data quality obj ectives that are more stringe nt than those prescribed by regulation. 18) [Table 5, Ozone Measurement Quality Objectives (MQOs) and Measurement Quality Indicators (MQIs) by Sampling S ite] The MQOs and MQIs for ozone should be updated to reflect the 7% DQO for ozone monitoring. 19) [QAPP, Appendix Equipment] This a ppendix discusses various standards and certifications of these standards, but doe s not include criteria for evaluation of

standards. 20) [SOPs, Standards] All supporting standa rds and calibration a nd verification of

these standards should be included the QAPP and/or the attached SOPs. 21) [SOPs, Criteria] Many of the required tasks discussed in the SOPs do not include acceptance criteria. 22) [SOPs, Ozone Standard] The procedure a nd criteria for certification of ozone primary and transfer standards is not included in the SOPs. 23) [SOPs, Zero Air] The equipment used fo r generation of zero air is not specified

in the SOPs.